DharmendraRathor

--conf spark.driver.memory=8G --num-executors=2 --executor-memory=14G --executor-cores=2 

Explanation

DharmendraRathor

How to trim minutes and seconds from date filed in Pyspark datarame.
Different apporaches to do that 

Input : 2019-01-31 23:16:28
output : 2019-01-31 23:00:00
  
  Not effecient 
  
     df.withColumn('tpep_pickup_datetime', concat(df.tpep_pickup_datetime.substr(0, 13), lit(‘:00:00’)))
       

DharmendraRathor

Create new branch from git commit withouth checkingout new branch
git branch newBranchName <commit-id>

Create new branch from git commit with checkingout new branch
git checkout -b newBranchName <commit-id>

DharmendraRathor

Git Cherry Picking 
if you have to cherry pick commit from one branch to another in Git repository with Commit id.

git cherry-pick <commit-id>
eg git cherry-pick ...31ff5e.....

DharmendraRathor

git checkout -b  <branchName> <tagName>

DharmendraRathor

How to view git remote url or remote branch details.

git remote show <branchName>

eg git remote show origin

DharmendraRathor

Check certificate details
  keytool -list -v -keystore keystore.jks
  

DharmendraRathor

You have branch having latest changes “latestBranch” and if you want to override master with latestBranch changes. 
Use following commands 

1. Checkout latestBranch.
git checkout latestBranch

2. Merge latestBranch with master with strategy as ours. Current branch changes will be used in case of conflicts.
git merge -s ours master

3. Checkout master branch.

DharmendraRathor

A  Moving branches/tags from one git repository to another

1. Clone existing git repo (source repository)
git clone --mirror <source git url>

eg 
git clone --mirror git@github.com:source/source.git

2. 
got to <folder with name of repo>