Skip to content

Instantly share code, notes, and snippets.

@manmohan24nov
Created November 3, 2020 18:44
Show Gist options
  • Save manmohan24nov/fac1e29b8d21bc9941e848c6953b0517 to your computer and use it in GitHub Desktop.
Save manmohan24nov/fac1e29b8d21bc9941e848c6953b0517 to your computer and use it in GitHub Desktop.
>>> from transformers import pipeline
>>> summarizer = pipeline('summarization', model='facebook/bart-large-cnn', tokenizer='facebook/bart-large-cnn')
>>> text = " ".join(tweet_data)
>>> TEXT_CLEANING_RE = "@\S+|https?:\S+|http?:\S|[^A-Za-z0-9]+"
>>> text = re.sub(TEXT_CLEANING_RE, ' ', str(text).lower()).strip()
>>> summarizer(text, min_length = round(0.1 * len(text.split(' '))), max_length = round(0.2 * len(text.split(' '))), do_sample=False)
[{'summary_text': "Don't miss the most comprehensive non stop uselections2020 coverage on india s only global news channel wionews.
A reminder as you seek comfort food in the days ahead that calories don t count if you don't use a plate handtomouth.
A new poll shows potus leading in one of the most important swing states pennsylvania."}]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment