Skip to content

Instantly share code, notes, and snippets.

@manmohan24nov
Created November 3, 2020 20:19
Show Gist options
  • Save manmohan24nov/b32bc2422b0b6633f0126aca77517110 to your computer and use it in GitHub Desktop.
Save manmohan24nov/b32bc2422b0b6633f0126aca77517110 to your computer and use it in GitHub Desktop.
>>> from summarizer import TransformerSummarizer
>>> import re
>>> xlnet_model = TransformerSummarizer(transformer_type="XLNet",transformer_model_key="xlnet-base-cased")
>>> text = " ".join(tweet_data)
>>> TEXT_CLEANING_RE = "@\S+|https?:\S+|http?:\S|[^A-Za-z0-9]+"
>>> text = re.sub(TEXT_CLEANING_RE, ' ', str(text).lower()).strip()
>>> summerize = ''.join(xlnet_model(text, min_length=60, max_length=120))
>>> summerize
"The fixwithohimai and chidiodinkalu look ahead to tomorrow's presidential election. The uselections2020 overnight show will feature guests on both sides of the at trump s defeat.
A new poll shows potus leading in one of the most important swing states pennsylvania."
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment