Skip to content

Instantly share code, notes, and snippets.

@codersquid
Created August 10, 2014 16:30
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save codersquid/bb6bfab7209b786b081e to your computer and use it in GitHub Desktop.
Save codersquid/bb6bfab7209b786b081e to your computer and use it in GitHub Desktop.
ia-wrapper UnicodeEncodeError example using http://video.ep14.c3voc.de/3.mp4
{
"3": {
"abstract": "Lessons learned when building a client for a fully distributed system and trying to minimize context-switching pains when using multiple languages.",
"collection": "opensource_movies",
"conference": "EuroPython 2014",
"date": "2014-07-24T14:00:00+0200",
"description": "Last year we decided to create official clients for the most popular languages, Python included.\r\n\r\nSome of the goals were:\r\n\r\n* support the complete API of elasticsearch including all parameters\r\n* provide a 1-to-1 mapping to the rest API to avoid having opinions and provide a familiar interface to our users consistent across languages and evironments\r\n* degrade gracefully when the es cluster is changing (nodes dropping out or being added)\r\n* flexibility - allow users to customize and extend the clients easily to suit their, potentially unique, environment\r\n\r\nIn this talk I would like to take you through the process of designing said client, the challenges we faced and the solutions we picked. Amongst other things I will touch on the difference between languages (and their respective communities), the architecture of the client itself, mapping out the API and making sure it stays up to date and integrating with existing tools.\r\n",
"extent": "00:30",
"is-part-of": "https://ep2014.europython.eu/en/",
"language": "en",
"location": "Berlin, Germany",
"mediatype": "movies",
"schedule_event_id": "3",
"schedule_event_type": "lecture",
"speakers": [
"Honza Kr\u00e1l"
],
"title": "Lessons learned from building Elasticsearch client",
"type": "conference",
"year": "2014"
}
}
#! /usr/bin/env python
# -*- coding: utf-8 -*-
import json
import os
from internetarchive import get_item
ACCESS_KEY="access"
SECRET_KEY="secret"
if __name__ == '__main__':
with open('example.json') as fh:
schedule = json.load(fh)
md = schedule['3']
item = get_item('europython_2014_event_3')
# this will throw a UnicodeEncodeError
item.upload(os.path.abspath('videos/3.mp4'), metadata=md, access_key=ACCESS_KEY, secret_key=SECRET_KEY)
./example.py ~/upload
Traceback (most recent call last):
File "./example.py", line 18, in <module>
item.upload(os.path.abspath('videos/3.mp4'), metadata=md, access_key=ACCESS_KEY, secret_key=SECRET_KEY)
File "/home/sheila/.virtualenvs/upload/src/internetarchive/internetarchive/item.py", line 579, in upload
resp = self.upload_file(body, key=key, **kwargs)
File "/home/sheila/.virtualenvs/upload/src/internetarchive/internetarchive/item.py", line 494, in upload_file
prepared_request = request.prepare()
File "/home/sheila/.virtualenvs/upload/src/internetarchive/internetarchive/iarequest.py", line 51, in prepare
queue_derive=self.queue_derive,
File "/home/sheila/.virtualenvs/upload/src/internetarchive/internetarchive/iarequest.py", line 69, in prepare
self.prepare_headers(headers, metadata, queue_derive)
File "/home/sheila/.virtualenvs/upload/src/internetarchive/internetarchive/iarequest.py", line 113, in prepare_headers
if any(c in str(value) for c in ['\n', '\r']):
File "/home/sheila/.virtualenvs/upload/src/internetarchive/internetarchive/iarequest.py", line 113, in <genexpr>
if any(c in str(value) for c in ['\n', '\r']):
UnicodeEncodeError: 'ascii' codec can't encode character u'\xe1' in position 8: ordinal not in range(128)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment