Skip to content

Instantly share code, notes, and snippets.

@flavioamieiro
Created July 24, 2015 12:37
Show Gist options
  • Save flavioamieiro/101d3da4c2a3550a224d to your computer and use it in GitHub Desktop.
Save flavioamieiro/101d3da4c2a3550a224d to your computer and use it in GitHub Desktop.
Send documents to pypln with celery
import glob
from celery import Celery
import pymongo
import pypln.api
import settings
app = Celery('tasks', backend="mongodb")
client = pymongo.MongoClient()
database = client["send_to_pypln"]
sent_documents = database["sent_documents"]
@app.task
def upload_to_pypln(filename):
corpus = pypln.api.Corpus.from_url(settings.PYPLN_CONFIG["corpus_url"],
settings.PYPLN_CREDENTIALS)
with open(filename, "rb") as fp:
pypln_document = corpus.add_document(fp)
sent_documents.insert({"filename": filename,
"url": pypln_document.url})
return pypln_document.url
for filename in glob.glob("Documentos/*.pdf"):
upload_to_pypln.delay(filename)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment