Skip to content

Instantly share code, notes, and snippets.

View kristjan-eljand's full-sized avatar

Kristjan Eljand kristjan-eljand

  • Eesti Energia
View GitHub Profile
@kristjan-eljand
kristjan-eljand / plotly_barchart.py
Created May 25, 2022 10:38
Making barchart with plotly graph objects
import plotly
import plotly.graph_objects as go
x = pd.Series({0:2020, 1:2021, 2:2022})
y = pd.Series({0: 10, 1:20, 2:30})
fig = go.Figure()
fig.add_trace(go.Bar(x = x, y = y))
fig.update_traces(
marker_color='rgb(68, 84, 106)',
marker_line_width=0,
@kristjan-eljand
kristjan-eljand / plotly_js_barchart.html
Created May 25, 2022 09:11
Create plot with plotly.js and the data is provided by the Flask back-end
<!--
============================================================
Plotly.js barchart
============================================================
-->
<div class="row">
<div class="col-lg-4" style="outline:1px solid black;">
<strong>Day-ahead prices</strong>
<div id="chart" class="container col-12"></div>
</div>
@kristjan-eljand
kristjan-eljand / python_speed_profiler.py
Created April 19, 2022 11:38
Measure the time that the specific python code/function takes and see the results element by element
# Profiler
# cumulative time -- time in function + calls to other functions
# total time -- time only in function without calls to other funs
import cProfile
import pstats
import io
def print_c_profiler(pr, lines_to_print=25):
"""
Create the speed profile of the arbitrary code.
Example usage:
@kristjan-eljand
kristjan-eljand / summarize_est_to_eng_to_est.py
Last active April 19, 2021 06:34
Summarize Estonian text using pre-trained English models
# 1. Initiate pipeline for Text summarization
summarizer = pipeline("summarization", model="t5-base")
# 2. Input sentence in Estonian
sentence_est = r"""
E-Lab on Eesti Energia IT osakonda kuuluv uurimis- ja arendusüksus.
Üksuse eesmärk on kiirendada innovatsiooni ja aidata kaasa uute ideede
esimeste arendusetappide (kontseptsiooni tõestus ja prototüüpimine) läbimisele.
Tiimis on täna 12 liiget, kelle seas seitse tarkvarainseneri, kaks andmeteadurit,
tarkvaraarhitekt, tooteomanik ning tehnoloogiaskaut.
@kristjan-eljand
kristjan-eljand / ner_est_to_eng_to_est.py
Last active April 15, 2021 10:17
Named Entity Recognition using Estonian text as an input and pre-trained English models for NER
# 1. Initiate pipeline for Named Entity Recognition (ner)
ner = pipeline("ner")
# The output included encoded classes
# Here I give reasonable Estonian names to these classes
classes_est = {
"O": "Ei ole nimi",
"B-MIS": "Nime algus kohe pärast teist nimeüksust",
"I-MIS": "Muu üksus",
"B-PER": "Inimese nime algus kohe pärast teise inimese nime",
@kristjan-eljand
kristjan-eljand / text_generation_est_to_eng_to_est.py
Last active April 14, 2021 08:32
Text generation with hugging face pre-trained models on non-English language
from pprint import pprint #for user-friendly output printing
# 1. Create a pipeline for text-generation task
generator = pipeline('text-generation', model='distilgpt2')
# 2. Translate the sentence beginnings from Est to Eng
beginnings_origin = [
"Eesti toodab elektrienergiat peamiselt",
"Taastuvenergia on oluline, sest"
]
translated_beginnings = [translate(b, EST_TO_ENG)[0]['translation_text'] for b in beginnings_origin]
@kristjan-eljand
kristjan-eljand / est_to_eng_question_answering.py
Last active April 13, 2021 08:13
Extractive question answering by providing text in Estonina, using pre-trained English models and translating answer back to Estonian.
# 1. create a pipeline for question answering task
respondent = pipeline("question-answering")
# 2. Translate the text from Estonian to English
context_to_translate = r"""
E-Lab on Eesti Energia IT osakonda kuuluv uurimis- ja arendusüksus.
Üksuse eesmärk on kiirendada innovatsiooni ja aidata kaasa uute ideede
esimeste arendusetappide läbimisele. Kui mõnel äriüksusel on soov innovaatilise
IT-lahenduse loomiseks, aitame teostada kontseptsiooni tõestuse ja arendada
välja prototüübi. Lisaks testib ja demonstreerib E-Lab ka uusi digitehnoloogiaid,
@kristjan-eljand
kristjan-eljand / est_to_eng_sentiment_analysis_failure.py
Last active April 13, 2021 08:18
est_to_eng sentiment analysis failed translation example
# Translate the text from input language to english
input_to_translate = "Parim argument demokraatia vastu on viieminutiline vestlus keskmise valijaga"
translated_input = translate(input_to_translate)[0]['translation_text']
# Using the sentiment classifier is oneliner
result = classifier(translated_input)[0]
print(translated_input)
print(f"label: {result['label']}, with score: {round(result['score'], 4)}")
# Output:
@kristjan-eljand
kristjan-eljand / est_to_eng_sentiment_analysis.py
Last active April 13, 2021 08:18
Sentiment analysis from Estonian to English using Huggingface transformers
# 1. Create sentiment classifier with pipeline function
# and the name of the task
classifier = pipeline('sentiment-analysis')
# 2. Translate the text from input language to English
input_to_translate = "Tahtsime parimat, aga välja kukkus nagu alati"
translated_input = translate(input_to_translate)[0]['translation_text']
# 3. Using the sentiment classifier is oneliner
result = classifier(translated_input)[0]
@kristjan-eljand
kristjan-eljand / huggingface_translation.py
Last active April 12, 2021 11:43
Translate from Estonian to English by using Huggingface's Inference API
import json
import requests
# We use "Helsinki-NLP/opus-mt-et-en" model for translation
API_URL = "https://api-inference.huggingface.co/models/Helsinki-NLP/opus-mt-et-en"
# Register an account in Hugging Face to get your API_TOKEN
# you'll find it under settings
headers = {"Authorization": f"Bearer {API_TOKEN}"}
# Function to run the post request to the API