Skip to content

Instantly share code, notes, and snippets.

View ngopal's full-sized avatar

Nikhil Gopal ngopal

21 followers · 47 following
View GitHub Profile
@NiloyPurkait
NiloyPurkait / MNIST_Keras2DML.py
Last active October 20, 2021 05:26
An example of using Apache SparkML to train a convolutional neural network in parallel using the MNIST dataset, on IBM watson studio. Written for medium article: https://medium.com/@niloypurkait/how-to-train-your-neural-networks-in-parallel-with-keras-and-apache-spark-ea8a3f48cae6
################################### Keras2DML: Parallely training neural network with SystemML#######################################
import tensorflow as tf
import keras
from keras.models import Sequential
from keras.layers import Input, Dense, Conv1D, Conv2D, MaxPooling2D, Dropout,Flatten
from keras import backend as K
from keras.models import Model
import numpy as np
import matplotlib.pyplot as plt
@geffy
geffy / stacking_example.py
Created October 7, 2017 17:33
Stacking example
# -*- coding: utf-8 -*-
"""
Created on Mon Sep 23 23:16:44 2017
@author: Marios Michailidis
This is an example that performs stacking to improve mean squared error
This examples uses 2 bases learners (a linear regression and a random forest)
and linear regression (again) as a meta learner to achieve the best score.
The initial train data are split in 2 halves to commence the stacking.
@chriddyp
chriddyp / dash_simple_example_pandas_datareader.py
Last active March 26, 2024 01:05
# See official docs at https://dash.plotly.com
# pip install dash pandas
from dash import Dash, dcc, html, Input, Output
import plotly.express as px
import pandas as pd
df = pd.read_csv('https://raw.githubusercontent.com/plotly/datasets/master/gapminderDataFiveYear.csv')
@vickyqian
vickyqian / twitter crawler.txt
Last active July 23, 2023 16:52
A Python script to download all the tweets of a hashtag into a csv
import tweepy
import csv
import pandas as pd
####input your credentials here
consumer_key = ''
consumer_secret = ''
access_token = ''
access_token_secret = ''
auth = tweepy.OAuthHandler(consumer_key, consumer_secret)
@primaryobjects
primaryobjects / mapReduce.R
Last active December 8, 2020 22:03
Map Reduce example in R. Demonstrates applying a map and reduce function to an input array, resulting in a hash of key/value pairs. This example maps a method to find prime divisors of each input value and then reduces the result by summing the input value for each prime divisor. See demo at http://www.r-fiddle.org/#/fiddle?id=tyir23SG&version=1
is.prime <- function(num) {
if (num == 2) {
TRUE
} else if (any(num %% 2:(num-1) == 0)) {
FALSE
} else {
TRUE
}
}
@0asa
0asa / sklearn-pyspark.py
Created January 27, 2015 11:12
Run a Scikit-Learn algorithm on top of Spark with PySpark
from pyspark import SparkConf, SparkContext
from sklearn.datasets import make_classification
from sklearn.ensemble import ExtraTreesClassifier
import pandas as pd
import numpy as np
conf = (SparkConf()
.setMaster("local[*]")
.setAppName("My app")
.set("spark.executor.memory", "1g"))
@shanebutler
shanebutler / sql.export.randomForest.R
Last active September 14, 2020 14:04
Deploy your RandomForest models in SQL! This tool enables in-database scoring of Random Forest models built using R. To use it, you simply call the function with the Random Forest model, output filename, SQL input data table and the name of the unique key on that table. For example:sql.export.rf(rf.mdl, file="model_output.SQL", input.table="sour…
# sql.export.rf(): save a randomForest model as SQL
# v0.04
# Copyright (c) 2013-2014 Shane Butler <shane dot butler at gmail dot com>
#
# sql.export.rf is free software: you can redistribute it and/or modify it
# under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 2 of the License, or
# (at your option) any later version.
#
# sql.export.rf is distributed in the hope that it will be useful, but
@tsiege
tsiege / The Technical Interview Cheat Sheet.md
Last active April 20, 2024 16:52
This is my technical interview cheat sheet. Feel free to fork it or do whatever you want with it. PLEASE let me know if there are any errors or if anything crucial is missing. I will add more links soon.

ANNOUNCEMENT

I have moved this over to the Tech Interview Cheat Sheet Repo and has been expanded and even has code challenges you can run and practice against!






\

@erichurst
erichurst / US Zip Codes from 2013 Government Data
Created December 9, 2013 23:00
All US zip codes with their corresponding latitude and longitude coordinates. Comma delimited for your database goodness. Source: http://www.census.gov/geo/maps-data/data/gazetteer.html
This file has been truncated, but you can view the full file.
ZIP,LAT,LNG
00601,18.180555, -66.749961
00602,18.361945, -67.175597
00603,18.455183, -67.119887
00606,18.158345, -66.932911
00610,18.295366, -67.125135
00612,18.402253, -66.711397
00616,18.420412, -66.671979
00617,18.445147, -66.559696