saurabhshri

Install and use pip in a local directory without root/sudo access.

By: @saurabhshri

Why?

Many users when are given server access, do not have root (or sudo) privileges and can not simply do sudo apt-get install python-pip . Here's an easy way you can install and use pip without root (or sudo) access in a local directory. Note : This works without easy_install too.

How?

simecek

nisrulz

import pandas as pd
import csv

with open("<filename>.csv", 'r') as f:
    with open("updated_file.csv", 'w') as f1:
        f.next()  # skip header line
        f.next()  # skip header empty line
        for line in f:
            f1.write(line.replace('\x00', ''))

codspire

Getting Started With Superset: Airbnb’s data exploration platform

Update Python and PIP versions on EC2 (Amazon AMI)

At the time of writing, Python v3.5 and PIP v9.0.1 were available on AWS EC2.

sudo yum update -y
sudo yum install python35 -y

uhho

def s3_to_pandas(client, bucket, key, header=None):

    # get key using boto3 client
    obj = client.get_object(Bucket=bucket, Key=key)
    gz = gzip.GzipFile(fileobj=obj['Body'])
    
    # load stream directly to DF
    return pd.read_csv(gz, header=header, dtype=str)

def s3_to_pandas_with_processing(client, bucket, key, header=None):

christopherhan

AWSTemplateFormatVersion: "2010-09-09"
Description: Put a SecureString parameter in SSM Parameter Store
Parameters:
  KmsKeyId:
    Type: String
    Description: The KMS Key you want to use to encrypt the string.
  SSMParameterKey:
    Type: String
    Description: The Parameter Store Key
  SSMParameterValue:

egpbos

philippkeller

Getting Started With Superset: Airbnb’s data exploration platform

These instructions are for Amazon Linux Version 2

Update Python and PIP versions on EC2 (Amazon AMI)

sudo yum update -y
sudo yum install python3 -y

mrshu

awscli==1.14.45
boto3==1.5.35
botocore==1.8.49
colorama==0.3.7
docutils==0.14
futures==3.2.0
jmespath==0.9.3
numpy==1.14.0
pandas==0.22.0
pip==9.0.1

mrshu

import pkg_resources
installed_packages = pkg_resources.working_set
for package in sorted(installed_packages, lambda x: x.key):
    print "{}=={}".format(package.key, package.version)