Skip to content

Instantly share code, notes, and snippets.

View Lenamaxpsi's full-sized avatar

Lena Maxutenko Lenamaxpsi

  • psifas
View GitHub Profile
@Lenamaxpsi
Lenamaxpsi / pytest_fixture_mock_moto_lambda.py
Created March 25, 2024 15:41 — forked from mikegrima/pytest_fixture_mock_moto_lambda.py
pytest fixture to mock out the moto AWS Lambda capability such that it DOES NOT try to execute the code in a Docker container
import pytest
import moto
from mock import mock, patch
from moto import mock_lambda
@pytest.fixture(scope='function')
def aws_credentials():
"""Mocked AWS Credentials for moto."""
@Lenamaxpsi
Lenamaxpsi / s3_async_dowload_folder.py
Created August 7, 2023 17:23 — forked from ripiuk/s3_async_dowload_folder.py
A tool for downloading a list of files from s3 using asyncio
import os
import time
import asyncio
import aiobotocore
import aiofiles
AWS_ACCESS_KEY_ID = '' # aws access key
AWS_SECRET_ACCESS_KEY = '' # aws secret key
READ_TIMEOUT = 500
@Lenamaxpsi
Lenamaxpsi / large_file_server.py
Created July 28, 2023 13:18 — forked from kgantsov/large_file_server.py
Example of serving large files using aiohttp server without reading entire file into a memory
import asyncio
import os
from aiohttp import web
from aiohttp import streamer
@streamer
async def file_sender(writer, file_path=None):
"""
@Lenamaxpsi
Lenamaxpsi / download_progress.py
Created July 28, 2023 13:18 — forked from motebaya/download_progress.py
python snippet for download file with Asyncrhonous http and rich progress bar
#!/usr/bin/env python3
"""
credit: @gist.github.com/motebaya
File: bar.py
Size: 2116 Blocks: 8 IO Block: 4096 regular file
Device: 18374h/89408d Inode: 243436 Links: 1
Access: (0777/-rwxrwxrwx) Uid: ( 1000/ mochino) Gid: ( 1000/ mochino)
Access: 2023-06-21 01:55:32.428195500 +0700
Modify: 2023-06-21 01:55:32.353622000 +0700
Change: 2023-06-21 01:55:32.353622000 +0700
@Lenamaxpsi
Lenamaxpsi / async_download_files.py
Created July 28, 2023 13:18 — forked from darwing1210/async_download_files.py
Script to download files in a async way, using Python asyncio
import os
import asyncio
import aiohttp # pip install aiohttp
import aiofile # pip install aiofile
REPORTS_FOLDER = "reports"
FILES_PATH = os.path.join(REPORTS_FOLDER, "files")
def download_files_from_report(urls):
@Lenamaxpsi
Lenamaxpsi / s3_multipart_upload.py
Created July 28, 2023 13:08 — forked from holyjak/s3_multipart_upload.py
boto3 S3 Multipart Upload with the ability to resume the upload after a failure
#!/usr/bin/env python3
# See https://gist.github.com/teasherm/bb73f21ed2f3b46bc1c2ca48ec2c1cf5
import argparse
import os
import boto3
class S3MultipartUpload(object):
# AWS throws EntityTooSmall error for parts smaller than 5 MB
@Lenamaxpsi
Lenamaxpsi / s3_multipart_upload.py
Created July 28, 2023 13:08 — forked from teasherm/s3_multipart_upload.py
boto3 S3 Multipart Upload
import argparse
import os
import boto3
class S3MultipartUpload(object):
# AWS throws EntityTooSmall error for parts smaller than 5 MB
PART_MINIMUM = int(5e6)
@Lenamaxpsi
Lenamaxpsi / upload_to_s3.py
Created July 27, 2023 17:55 — forked from skonik/upload_to_s3.py
Asyncio S3 Multipart Upload
# Details https://skonik.me/uploading-large-file-to-s3-using-aiobotocore/
import asyncio
import math
import os
import aiobotocore
import aiofiles
AWS_S3_HOST = 'http://localhost:9000'
AWS_SECRET_ACCESS_KEY = 'SECRET_KEY'
@Lenamaxpsi
Lenamaxpsi / basespace_cli.py
Created July 26, 2023 19:54 — forked from tubuliferous/basespace_cli.py
Python interface for the BaseSpace CLI
# --- BaseSpace CLI Installation:
# Mac:
# brew tap basespace/basespace && brew install bs-cli
#
# Linux:
# Download the following and then `chmod u+x` to make the file executable:
# https://bintray.com/basespace/BaseSpaceCLI-EarlyAccess-BIN/latest/1.3.1#files/latest%2F1.3.1%2Famd64-linux
# * Make sure the executable is in a directory in the PATH variable, e.g. add `export PATH=$PATH:/directory/with/executable` to $HOME/.bashrc
# --- BaseSpace CLI Examples: