Skip to content

Instantly share code, notes, and snippets.

View manisnesan's full-sized avatar
🎯
Focusing

Manikandan Sivanesan` manisnesan

🎯
Focusing
View GitHub Profile
@manisnesan
manisnesan / chi-square.py
Created October 13, 2021 19:38
Chi Squared Test to compare the models
# Source: https://towardsdatascience.com/chi-squared-tests-to-compare-two-machine-learning-models-and-determine-whether-they-are-random-2a405fc55181
import numpy as np
import pandas as pd
from scipy.stats import chi2_contingency
data=[[58,7],[11,24]] #Model M1 table
#Chi square statistic,pvalue,DOF,expected table
stat, p, dof, expected = chi2_contingency(data)
print('Chi-square statistic=',stat)
print('Pvalue=',p)
alpha=0.05
https://fosspost.org/things-to-do-after-installing-fedora-34/
https://www.debugpoint.com/2021/04/10-things-to-do-fedora-34-after-install/
@manisnesan
manisnesan / nbdev-cheatsheet.md
Created May 23, 2021 14:54
nbdev cheatsheet
  • put #hide at the top of any cell you want to completely hide in the docs
  • put #hide_input at the top of a cell if you don't want code to be shown in the docs
  • cells containing #export or show_doc have their code hidden automatically
  • put #hide_output at the top of a cell if you don't want output to be shown in the docs
  • use #collapse_input or #collapse_output to include code or output in the docs under a collapsable element
@manisnesan
manisnesan / 2020-12-15-pseudolabelingdatacleaning.ipynb
Created April 10, 2021 16:10
2020-12-15-PseudoLabelingDataCleaning.ipynb
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@manisnesan
manisnesan / 05_pet_breeds.ipynb
Created March 21, 2021 14:18
05_pet_breeds.ipynb
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
poetry run pytest --no-cov
============================================================================================================ test session starts ============================================================================================================
platform linux -- Python 3.8.6, pytest-6.2.1, py-1.10.0, pluggy-0.13.1
rootdir: /home/msivanes/0Work/routing/language-detection
plugins: cov-2.10.1
collected 14 items / 1 error / 13 selected
================================================================================================================== ERRORS =============
@manisnesan
manisnesan / gist:324f3f55f9f781d83c5d6ea63489595b
Created January 18, 2021 15:09 — forked from xcodeit/gist:3726868
General principles for good URI design
General principles for good URI design:
Don't use query parameters to alter state
Don't use mixed-case paths if you can help it; lowercase is best
Don't use implementation-specific extensions in your URIs (.php, .py, .pl, etc.)
Don't fall into RPC with your URIs
Do limit your URI space as much as possible
Do keep path segments short
Do prefer either /resource or /resource/; create 301 redirects from the one you don't use
Do use query parameters for sub-selection of a resource; i.e. pagination, search queries
@manisnesan
manisnesan / gist.md
Last active December 21, 2020 22:09
nbdev make test failures
(fastchai)  ~/0fastai/nbdev   master  make test
nbdev_test_nbs --flags ''
testing /home/msivanes/0fastai/nbdev/nbs/00_export.ipynb
Error in /home/msivanes/0fastai/nbdev/nbs/00_export.ipynb:
preprocess() missing 1 required positional argument: 'resources'
testing /home/msivanes/0fastai/nbdev/nbs/02_showdoc.ipynb
Error in /home/msivanes/0fastai/nbdev/nbs/02_showdoc.ipynb:
preprocess() missing 1 required positional argument: 'resources'
testing /home/msivanes/0fastai/nbdev/nbs/03_export2html.ipynb

Task

Adding a new visualization in Apache Superset with the data workflow tool such as Airflow for periodic refresh. We will be creating a python script to fetch, tranform & load the data in sqlite.

Subtasks

  • Learn airflow
  • Setup airflow and create a DAG for periodic data collection
  • Transform the data into records
  • Insert into a database
  • Learn Superset and Create a superset dashboard.
{
"red hat enterprise linux":
{
"aliases": ["rhel"],
"parent": ["infrastructure"],
"synonyms": ["redhat linux", "linux"],
"related": ["operating system", "fedora", "centos"],
"type": "product"
},
"openshift container platform":