Skip to content

Instantly share code, notes, and snippets.

View vijayanandrp's full-sized avatar
👑

Vijay Anand Pandian vijayanandrp

👑
View GitHub Profile
@vijayanandrp
vijayanandrp / Wikipedia Scraper using Scrapy.py
Last active March 16, 2024 14:55
Wikipedia Scraper using Scrapy
# -*- coding: utf-8 -*-
import scrapy
import re
import unicodedata
from scrapy.crawler import CrawlerProcess
from urllib.parse import urlparse
class WikiSpider(scrapy.Spider):
name = 'wiki_company'
@vijayanandrp
vijayanandrp / Bigquery_util.py
Last active October 27, 2023 07:21
Big Query to Google Cloud storage
#!/usr/bin/env python
# Copyright 2016 Google Inc. All Rights Reserved.
import os
import sys
import time
os.environ['GOOGLE_APPLICATION_CREDENTIALS'] = 'BigQuery.json'
from google.cloud import bigquery
from google.cloud.bigquery.job import DestinationFormat, ExtractJobConfig, Compression
@vijayanandrp
vijayanandrp / system_design_interview_notes_with_python.md
Created June 17, 2023 14:16
System Design - All in One Interview - Reading notes with python examples

Complete System Design Series

With examples and intelligible explanations…

Pic credits : Github

Welcome back peeps. We are now starting System Design Series ( over weekends) where we will cover how to design large ( and great) systems, the techniques, tip/tricks that you can refer to in order to scale these systems. As a senior software engineer it’s expected that you know not just the breadth but also depth of the system design concepts.

@vijayanandrp
vijayanandrp / Step by Step method to parse XML files and store into sqlite db and vice versa.py
Created June 15, 2017 22:24
Step by Step method to parse XML files and store into sqlite db and vice versa
__author__ = 'admin'
import sqlite3
db = sqlite3.connect('books_store.db')
cursor = db.cursor()
try:
cursor.execute('''DROP TABLE books''')
print 'Deleted the old table'
@vijayanandrp
vijayanandrp / kconnect.py
Last active January 7, 2023 01:01 — forked from rueedlinger/kconnect.py
Kafka Connect Python Script - https | user auth | Status | restart | pause | resume
# credits source : https://gist.github.com/rueedlinger/76af36d04a0798a8e1f43ed16595bd97
import sys
import os
import json
import argparse
from base64 import b64encode
PYTHON_MAJOR_VERSION = sys.version_info.major
DEFAULT_HOST = 'localhost'
@vijayanandrp
vijayanandrp / ip_address_class_selector.py
Created June 15, 2017 22:08
How to do IP Address Class Identification using python program?
#!/usr/bin/python
import sys, getopt
def __help():
sys.stderr.write('\n Usage: \n ip_classifier.py -h \n ip_classifier.py --help \n ip_classifier.py -i 128.12.14.23 \n ip_classifier.py --ipv4=128.12.14.23 \n ')
sys.exit()
@vijayanandrp
vijayanandrp / Learn XML - parsing in python with simple examples.py
Created June 15, 2017 22:19
Learn XML - parsing in python with simple examples
# 1. How to read and XML file using Python ?
# For example purpose, kindly save the above XML content into a file named as 'data.xml'
# import a class named xml.etree.ElementTree
import xml.etree.ElementTree as element_tree
# Before reading we should parse the whole XML file content as tree
tree = element_tree.parse('data.xml')
# Now with tree we have to fetch the root
@vijayanandrp
vijayanandrp / How to learn Python in a 15 Minutes !! Believe its very cool language to have fun and learn .py
Created June 15, 2017 22:27
How to learn Python in a 15 Minutes !! Believe its very cool language to have fun and learn :)
#!/usr/bin/env python
# I have learned these codes from Learning Python O'reilly
# Python 2.7
"""
#! is often called hash bang or shebang
/user/bin/python - path of the python executable
#!/usr/bin/python - Unix Styled Executable Scripts (If u know exact path of the python binary)
@vijayanandrp
vijayanandrp / How to write Class in Python with examples.py
Created June 15, 2017 22:29
How to write Class in Python with examples
#!/usr/bin/env python
class Worker:
def __init__(self, name, pay):
self.name = name
self.pay = pay
def last_name(self):
return self.name.split()[-1]
@vijayanandrp
vijayanandrp / Creating a public key using openssl.sh
Created June 15, 2017 22:43
Creating a public key using openssl
root@Vijay:# openssl rsa -inform PEM -text -pubin -in publickey.pem -modulus
Public-Key: (704 bit)
Modulus:
00:e1:34:18:93:fe:6e:68:16:ce:c8:a9:70:a3:9c:
00:fa:54:7c:7d:a2:cd:ed:ab:0a:62:b9:1c:46:51:
a8:3f:96:38:0b:cf:ae:e2:6f:7e:86:61:07:90:63:
89:42:1b:1e:68:d0:a1:7a:ad:c9:87:0b:98:58:e9:
56:28:6e:39:99:e9:8c:ec:98:81:53:4a:c7:72:ae: