Skip to content

Instantly share code, notes, and snippets.

@bedekelly
Last active July 16, 2020 16:52
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save bedekelly/040cd21062d9a01264a8486c0b4b4ca5 to your computer and use it in GitHub Desktop.
Save bedekelly/040cd21062d9a01264a8486c0b4b4ca5 to your computer and use it in GitHub Desktop.
Import a CSV file into a DynamoDB table
import os
import ast
import sys
import csv
import boto3
AWS_ACCESS_KEY = "<YOUR ACCESS KEY HERE>"
AWS_SECRET_KEY = "<YOUR SECRET KEY HERE>"
AWS_REGION = "<YOUR AWS REGION HERE>"
TABLE_NAME = "<YOUR DYNAMODB TABLE NAME HERE>"
CSV_FILE = "<YOUR CSV FILEPATH HERE>"
# Establish a connection to DynamoDB via Boto 3.
dynamodb = boto3.resource(
"dynamodb",
aws_access_key_id=AWS_ACCESS_KEY,
aws_secret_access_key=AWS_SECRET_KEY,
region_name=AWS_REGION
)
# Lazy-load the DynamoDB table in question.
table = dynamodb.Table(TABLE_NAME)
coercion_funcs = {
"(N)": int,
"(S)": str,
"(M)": ast.literal_eval # lol fite me
}
# Load the CSV file into memory.
with open(CSV_FILE) as csv_file:
reader = csv.reader(csv_file, delimiter=",")
with table.batch_writer() as batch:
# The first row of a CSV file is its headings.
header = next(reader)
headings = []
# Tag each heading with an associated "coercion" function.
# This function restores the original type of the data.
for header_value in header:
title, type_ = header_value.split()
headings.append((title, coercion_funcs[type_]))
# For each row, coerce the value and add the item to Dynamo.
for row in reader:
item = {}
for val, (heading, func) in zip(row, headings):
item[heading] = func(val)
batch.put_item(Item=item)
@onenessboy
Copy link

i am getting title, type_ = header_value.split()
ValueError: not enough values to unpack (expected 2, got 1)

how should be my csv looks like ?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment