Suriyadeepan Ramamoorthy suriyadeepan

## wiki_images.py
from bs4 import BeautifulSoup
import requests

url = 'https://en.wikipedia.org/wiki/Transhumanism'
# get contents from url
content = requests.get(url).content
# get soup
soup = BeautifulSoup(content,'lxml') # choose lxml parser
# find the tag : <img ... >
image_tags = soup.findAll('img')

## log_normal_shadowing_model.ipynb

      
              1 file
            
          
              0 forks
            
          
              0 comments
            
          
              0 stars
            
          
                suriyadeepan
                / log_normal_shadowing_model.ipynb
            
            
              Last active
              October 28, 2021 11:01
            
          
      Sorry, something went wrong. Reload?
      Sorry, we cannot display this file.
      Sorry, this file is invalid so it cannot be displayed.
      
          Viewer requires iframe.
      
    
## overlapping_area.ipynb

      
              1 file
            
          
              0 forks
            
          
              0 comments
            
          
              0 stars
            
          
                suriyadeepan
                / overlapping_area.ipynb
            
            
              Created
              April 8, 2021 09:19
            
              
                Overlapping Area
              
          
      Sorry, something went wrong. Reload?
      Sorry, we cannot display this file.
      Sorry, this file is invalid so it cannot be displayed.
      
          Viewer requires iframe.
      
    
## nocors.json

    [
        {
            "date": "2014-01-01",
            "value": 190000000
        },
        {
            "date": "2014-01-02",
            "value": 190379978
        },

## create_soil_type_feat.py
def create_feat_soil_type(df):
    soil_type_cols = [ col for col in df.columns if 'Soil_Type' in col ]
    soil_type = df[soil_type_cols].idxmax(axis=1)
    return soil_type.apply(lambda x : int(x[9:]))

## amazon_src.json
{
src : `
<!doctype html><html lang="en-in" class="a-no-js" data-19ax5a9jf="dingo"><!-- sp:feature:head-start -->
<head><script>var aPageStart = (new Date()).getTime();</script><meta charset="utf-8"/>

<script type='text/javascript'>var ue_t0=ue_t0||+new Date();</script>
<!-- sp:feature:cs-optimization -->
<meta http-equiv='x-dns-prefetch-control' content='on'>
<link rel="dns-prefetch" href="https://images-eu.ssl-images-amazon.com">
<link rel="dns-prefetch" href="https://m.media-amazon.com">

## hyp_gen_data_science_pipeline.md

      
              1 file
            
          
              0 forks
            
          
              0 comments
            
          
              0 stars
            
          
                suriyadeepan
                / hyp_gen_data_science_pipeline.md
            
            
              Last active
              November 16, 2020 10:06
            
              
                Blog: Hypothesis Generation
              
          
Stage
Description


1
Hypothesis Generation
Study the business problem. Build a conceptual model by developing a deeper understanding of the problem and domain. Generate Hypotheses.


2
Data Collection
Go out in the wild and collect data based on the generated hypotheses.


3
Study the variables
Identify potential predictors using data visualization


4
Data Preparation
Clean the data. Fill in missing data points. Scale, normalize and transform data as necessary.


5
Bivariate/Multivariate Analysis
Test the hypotheses you've generated earlier. Choose predictors based on correlation with target.


6
Data Transformation
Perform non-linear transformations (log) on variables to fish out non-l


## model_factors.md

      
              1 file
            
          
              0 forks
            
          
              0 comments
            
          
              0 stars
            
          
                suriyadeepan
                / model_factors.md
            
            
              Created
              November 16, 2020 09:39
            
              
                Blog: Hypothesis Generation (3)
              
          
Factor
Type
Hypothesis


Age
Personal
A person above the age of 35 is more likely to stay than a younger person


Occupation
Professional
A person with a steady income is more likely to stay than a person who is in between jobs


Married
Relationship
A married person is more likely to stay than an unmarried person


Children
Relationship
People with children are more likely to stay than people without


Type of Service
Company
Location-based services could lead to higher churn rate if your customer base is young and unmarried


Monthly Charges
Company
High monthly charges may lead to termination of subscription


Period of Subscription
Company
Longer subscription period indicates loyal customers


## model.md

      
              1 file
            
          
              0 forks
            
          
              0 comments
            
          
              0 stars
            
          
                suriyadeepan
                / model.md
            
            
              Created
              November 16, 2020 09:37
            
              
                Blog: Hypothesis Generation (2)
              
          
Factor
Type
Comments


Age
Personal


Occupation
Professional


Married
Relationship


Children
Relationship


Type of Service
Company


Monthly Charges
Company


Period of Subscription
Company


## identify_target.md

      
              1 file
            
          
              0 forks
            
          
              0 comments
            
          
              0 stars
            
          
                suriyadeepan
                / identify_target.md
            
            
              Created
              November 16, 2020 09:28
            
              
                Blog: Hypothesis Generation (1)
              
          
Question
Target Variable
Type


What sorts of people were more likely to survive?
Survival
Binary (Yes or No)


Predict (What is) the final price of each home.
Housing Price
Numerical (in USD)


Predict (What is) the probability that a driver will initiate an auto insurance claim in the next year.
Insurance Claim
Probability (Ranged [0-1] numerical)
	from bs4 import BeautifulSoup
	import requests

	url = 'https://en.wikipedia.org/wiki/Transhumanism'
	# get contents from url
	content = requests.get(url).content
	# get soup
	soup = BeautifulSoup(content,'lxml') # choose lxml parser
	# find the tag : <img ... >
	image_tags = soup.findAll('img')

	[
	{
	"date": "2014-01-01",
	"value": 190000000
	},
	{
	"date": "2014-01-02",
	"value": 190379978
	},
	def create_feat_soil_type(df):
	soil_type_cols = [ col for col in df.columns if 'Soil_Type' in col ]
	soil_type = df[soil_type_cols].idxmax(axis=1)
	return soil_type.apply(lambda x : int(x[9:]))
	{
	src : `
	<!doctype html><html lang="en-in" class="a-no-js" data-19ax5a9jf="dingo"><!-- sp:feature:head-start -->
	<head><script>var aPageStart = (new Date()).getTime();</script><meta charset="utf-8"/>

	<script type='text/javascript'>var ue_t0=ue_t0\|\|+new Date();</script>
	<!-- sp:feature:cs-optimization -->
	<meta http-equiv='x-dns-prefetch-control' content='on'>
	<link rel="dns-prefetch" href="https://images-eu.ssl-images-amazon.com">
	<link rel="dns-prefetch" href="https://m.media-amazon.com">
	Stage	Description
1	Hypothesis Generation	Study the business problem. Build a conceptual model by developing a deeper understanding of the problem and domain. Generate Hypotheses.
2	Data Collection	Go out in the wild and collect data based on the generated hypotheses.
3	Study the variables	Identify potential predictors using data visualization
4	Data Preparation	Clean the data. Fill in missing data points. Scale, normalize and transform data as necessary.
5	Bivariate/Multivariate Analysis	Test the hypotheses you've generated earlier. Choose predictors based on correlation with target.
6	Data Transformation	Perform non-linear transformations (log) on variables to fish out non-l
Factor	Type	Hypothesis
Age	Personal	A person above the age of 35 is more likely to stay than a younger person
Occupation	Professional	A person with a steady income is more likely to stay than a person who is in between jobs
Married	Relationship	A married person is more likely to stay than an unmarried person
Children	Relationship	People with children are more likely to stay than people without
Type of Service	Company	Location-based services could lead to higher churn rate if your customer base is young and unmarried
Monthly Charges	Company	High monthly charges may lead to termination of subscription
Period of Subscription	Company	Longer subscription period indicates loyal customers
Question	Target Variable	Type
What sorts of people were more likely to survive?	Survival	Binary (Yes or No)
Predict (What is) the final price of each home.	Housing Price	Numerical (in USD)
Predict (What is) the probability that a driver will initiate an auto insurance claim in the next year.	Insurance Claim	Probability (Ranged [0-1] numerical)