test local s3 lke env

local_s3_notes.txt

Running a local s3 like object storage to practice

- Docker Desktop
- MinIO
- python
- boto3

Install Docker desktop
https://docs.docker.com/desktop/#download-and-install

Run minio 
(https://hub.docker.com/r/minio/minio/)

docker run --rm -p 9000:9000 minio/minio server /data

Go to localhost:9000 and enter in the access key and access secret to interact with minio 


Run python and get host as network inside container
docker run --rm -it --network="host" python:3.8-slim-buster /bin/bash

	apt-get update
	apt-get install curl unzip

	++ Install awscliv2
		curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip"
		unzip awscliv2.zip
		./aws/install
	

	pip install boto3 ipython
	
	++ Now we can run ipython and use the boto3 lib to interact with the minio server
	(https://docs.min.io/docs/how-to-use-aws-sdk-for-python-with-minio-server.html)
	(https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/s3.html#s3)
	
		import boto3
		from botocore.client import Config
		client = boto3.client('s3', endpoint_url='http://localhost:9000', aws_access_key_id='minioadmin', aws_secret_access_key='minioadmin', config=Config(signature_version='s3v4'), region_name='us-east-1')

		client.create_bucket(ACL='private', Bucket='raw')

		! echo "hi this is a test" > raw.txt
		! ls | grep raw.txt

		client.upload_file('raw.txt', Bucket='raw', Key='raw.txt')

		client.get_object(Bucket='raw', Key='raw.txt')['Body'].read()