Anwarvic

{
    "$schema": "https://raw.githubusercontent.com/JanDeDobbeleer/oh-my-posh/main/themes/schema.json",
    "blocks": [
      {
        "alignment": "left",
        "newline": true,
        "segments": [
          {
            "background": "#fbfbfb",
            "foreground": "#0077c2",

Anwarvic

import os 
import edlib
import string
import pandas as pd
from sklearn import metrics
from itertools import chain
from nltk.tokenize import word_tokenize


# HELPFUL FUNCTIONS

Anwarvic

I created this GitHub Gist to show how to install Cuda on a newly-created VM instance on GCP. If you don't know which GPU model you should use, check out this stackoverflow question.

Before You Start

Simply put, Cuda is just an API for software to use GPU. So, before you install Cuda you have to make sure that it's compitable with your GPU model. In this link, you can find the following table which summarizes the minimum Cuda for each machine:

GPU type

Anwarvic

# install dependencies (src: https://kheafield.com/code/kenlm/dependencies/)
sudo apt-get install build-essential libboost-all-dev cmake zlib1g-dev libbz2-dev liblzma-dev

# clone official GitHub repo
git clone https://github.com/kpu/kenlm/

# build the repo using cmake
cd kenlm
mkdir -p build
cd build

Anwarvic

Anwarvic

from joblib import Parallel, delayed

def post_request(word):
	response = requests.post("http://172.18.7.39:8181/stemSentence", 
				data={'sentence': word})
	stem = ''
	if '"error":false' in response.text:
		response = eval(response.text.replace('false', 'False'))
		stem = response['stemmedSentence']
	return stem

Anwarvic

def getEncoding(rawdata, min_confidence=0.9):
    """
    Get the encoding of a stream of BYTES (rawdata)
    with a minimum confidence.
    In case that the confidence is low, return "cp1256"
    as I work with Arabic text!!
    """
    det = chardet.detect(rawdata[:1000])
    data_encoding, confidence = det["encoding"], det["confidence"]
    if confidence > min_confidence:

Anwarvic

"""
extract names like:
Mohamed Anwar
Mohamed Anwar Ghanem
Mohamed Anwar Saeed Ghanem
but not:
Mohamed
Mohamed A. Ghanem
"""
regex = r'(([A-Z]{1}[a-z]+)\s){2,}'

Anwarvic

import re
"""
Here, I will provide a test case of how to remove a regex with another regex in python.
Our test case will be to extract only the comma within numbers and remove it.
So, this sentence "How a 2,000 year old letter can revolutionize your life, it's really amazing."
should be "How a 2000 year old letter can revolutionize your life, it's really amazing."
As you can see, the comma within 2,000 was removed.

We should notice that this number is small, what about a number like 1,123,456,789,101.
In this regex, I've handled all the cases that came into my mind.

Anwarvic

ASCII PUNCTUATIONS:
'",.;:?!-/\

ASCII SPECIAL CHARACTERS:
#$%&*+_=@^|~

BRACKETS
()[]{}<>⟨⟩«»‹›

UNICODE PUNCTUATIONS: