Skip to content

Instantly share code, notes, and snippets.

View mohdsanadzakirizvi's full-sized avatar

Mohd Sanad Zaki Rizvi mohdsanadzakirizvi

View GitHub Profile
from inltk.inltk import get_sentence_similarity
# similarity of encodings is calculated by using cmp function whose default is cosine similarity
get_sentence_similarity('मुझे भोजन पसंद है।', 'मैं ऐसे भोजन की सराहना करता हूं जिसका स्वाद अच्छा हो।', 'hi')
from inltk.inltk import setup
from inltk.inltk import predict_next_words
# download models for Gujarati
setup('bn')
# predict the next words of the sentence "The weather is nice today"
predict_next_words("আবহাওয়া চমৎকার", 10, "bn", 0.7)
The Landon Bears shut out the visiting Whitman Vikings, 34-0, on Friday.
Landon opened the game with a 90-yard kickoff return for a score by Jelani Machen. Landon added to their lead on John Geppert's five-yard touchdown run. The first quarter came to a close with Landon leading, 14-0.
In the second quarter, the Bears went even further ahead following Joey Epstein's four-yard touchdown run. The Bears scored again on Geppert's one-yard touchdown run.
Landon had the lead going into the second half, 27-0. The Bears extended their lead on Tommy Baldwin's nine-yard touchdown reception.
Neither team scored in the fourth quarter.
The developers behind BERT have added a specific set of rules to represent the input text for the model. Many of these are creative design choices that make the model even better.
For starters, every input embedding is a combination of 3 embeddings:
Position Embeddings: BERT learns and uses positional embeddings to express the position of words in a sentence. These are added to overcome the limitation of Transformer which, unlike an RNN, is not able to capture “sequence” or “order” information
Segment Embeddings: BERT can also take sentence pairs as inputs for tasks (Question-Answering). That’s why it learns a unique embedding for the first and the second sentences to help the model distinguish between them. In the above example, all the tokens marked as EA belong to sentence A (and similarly for EB)
Token Embeddings: These are the embeddings learned for the specific token from the WordPiece token vocabulary
MONEY, Miss. — Along the edge of Money Road, across from the railroad tracks, an old grocery store rots.
In August 1955, a 14-year-old black boy visiting from Chicago walked in to buy candy. After being accused of whistling at the white woman behind the counter, he was later kidnapped, tortured, lynched and dumped in the Tallahatchie River.
The murder of Emmett Till is remembered as one of the most hideous hate crimes of the 20th century, a brutal episode in American history that helped kindle the civil rights movement. And the place where it all began, Bryant’s Grocery & Meat Market, is still standing. Barely.
Today, the store is crumbling, roofless and covered in vines. On several occasions, preservationists, politicians and business leaders — even the State of Mississippi — have tried to save its remaining four walls. But no consensus has been reached.
Some residents in the area have looked on the store as a stain on the community that should be razed and forgotten. Others have said it should be restored
climate change is real and irreversible," says David Brown, the director of the Earth Institute and a professor at the University of London. "It is easy to imagine that the increasing sunbursts will continue as the planet warms, and all its climate change is reversible." But climate change, according to Brown, will be real if only we continue pumping carbon dioxide into the atmosphere for long enough. It may be possible to keep CO2 at a level around 450 parts per million, as we did in the case of our planet-warming ancestors. Yet these older man-made pollutants will continue to creep up on the planet. Brown acknowledges that life could use new water, or even air, resources to survive in conditions that took place thousands of years ago, in addition to sedimentary rock. (It is estimated that the planet may have seen a flood of an unprecedented magnitude of CO2 in the early 2000s.)
Adding to the urgency to avoid a spill of more CO2 — or more of it in its current form — can be the concern that the recent surfac
// Find similar words based on embedding
func embedCheck(word: String){
// Extract the language type
let lang = NLLanguageRecognizer.dominantLanguage(for: word)
// Get the OS embeddings for the given language
let embedding = NLEmbedding.wordEmbedding(for: lang!)
// Find the 5 words that are nearest to the input word based on the embedding
let res = embedding?.neighbors(for: word, maximumCount: 5)
// Print the words
// Sentiment Analysis
// Set up our input
let input = "I hate this apple pie."
// Feed it into the NaturalLanguage framework
let tagger = NLTagger(tagSchemes: [.sentimentScore])
tagger.string = input
// Ask for the results
let sentiment = tagger.tag(at: input.startIndex, unit: .paragraph, scheme: .sentimentScore).0
// Places, People, Organizations using NER
let text = "Apple is looking at buying U.K. startup for $1 billion."
// Initialize NLTagger with ".nameType" scheme for NER
let tagger = NLTagger(tagSchemes: [.nameType])
tagger.string = text
// Ignore Punctuation and Whitespace
let options: NLTagger.Options = [.omitPunctuation, .omitWhitespace, .joinNames]
// Tags to extract
let tags: [NLTag] = [.personalName, .placeName, .organizationName]
let text = "Hello world, I am a data scientist. I work with machine learning!"
// Initialize the tagger
let tagger = NLTagger(tagSchemes: [.lexicalClass])
// Ignore whitespace and punctuation marks
let options : NLTagger.Options = [.omitWhitespace, .omitPunctuation]
// Process the text for POS
tagger.string = text
// loop through all the tokens and print their POS tags