Skip to content

Instantly share code, notes, and snippets.

Aslan

Block or report user

Report or block Aslan

Hide content and notifications from this user.

Learn more about blocking users

Contact Support about this user’s behavior.

Learn more about reporting abuse

Report abuse
View GitHub Profile
@Aslan
Aslan / web-servers.md
Created May 17, 2017 — forked from willurd/web-servers.md
Big list of http static server one-liners
View web-servers.md

Each of these commands will run an ad hoc http static server in your current (or specified) directory, available at http://localhost:8000. Use this power wisely.

Discussion on reddit.

Python 2.x

$ python -m SimpleHTTPServer 8000
@Aslan
Aslan / gitflow-breakdown.md
Last active Apr 18, 2017 — forked from JamesMGreene/gitflow-breakdown.md
A comparison of using `git flow` commands versus raw `git` commands.
View gitflow-breakdown.md

Initialize

gitflow git
git flow init git init
git commit --allow-empty -m "Initial commit"
git checkout -b develop master

Connect to the remote repository

View gist:a78e6855001416bec75afc679d021885
* service[opscode-erchef] action nothing (skipped due to action :nothing)
Recipe: private-chef::opscode-erchef
* runit_service[opscode-erchef] action enable
Recipe: <Dynamically Defined Resource>
* directory[/opt/opscode/sv/opscode-erchef] action create
- create new directory /opt/opscode/sv/opscode-erchef
- change mode from '' to '0755'
* template[/opt/opscode/sv/opscode-erchef/run] action create
- create new file /opt/opscode/sv/opscode-erchef/run
- update content in file /opt/opscode/sv/opscode-erchef/run from none to 8ef29e
View gist:d6ae8b011ad767b1ae45c401355c6179
❯❯❯ aws s3 ls s3://xxxxxxxxxxxxxx/production/MPX/2016-04-04/
2016-04-05 01:20:07 0 _SUCCESS
2016-04-05 01:16:41 93211 part-r-00000-68103d9b-8201-49c5-91f8-f832337648db.gz.parquet
2016-04-05 01:16:42 97383 part-r-00001-68103d9b-8201-49c5-91f8-f832337648db.gz.parquet
2016-04-05 01:16:43 92641 part-r-00002-68103d9b-8201-49c5-91f8-f832337648db.gz.parquet
2016-04-05 01:16:44 97808 part-r-00003-68103d9b-8201-49c5-91f8-f832337648db.gz.parquet
2016-04-05 01:16:45 96843 part-r-00004-68103d9b-8201-49c5-91f8-f832337648db.gz.parquet
❯❯❯ aws s3 ls s3://xxxxxxxxxxxxxxxx/production/MPX/2016-04-05/
2016-04-06 01:21:08 0 _SUCCESS
2016-04-06 01:17:45 93367 part-r-00000-ba75a2d7-6e02-45f6-a1fe-dca02e3f2620.gz.parquet
View gist:a053bb4fa1057b6017a0
org.apache.spark.SparkException: Job aborted.
at org.apache.spark.sql.execution.datasources.InsertIntoHadoopFsRelation$$anonfun$run$1.apply$mcV$sp(InsertIntoHadoopFsRelation.scala:156)
at org.apache.spark.sql.execution.datasources.InsertIntoHadoopFsRelation$$anonfun$run$1.apply(InsertIntoHadoopFsRelation.scala:108)
at org.apache.spark.sql.execution.datasources.InsertIntoHadoopFsRelation$$anonfun$run$1.apply(InsertIntoHadoopFsRelation.scala:108)
at org.apache.spark.sql.execution.SQLExecution$.withNewExecutionId(SQLExecution.scala:56)
at org.apache.spark.sql.execution.datasources.InsertIntoHadoopFsRelation.run(InsertIntoHadoopFsRelation.scala:108)
at org.apache.spark.sql.execution.ExecutedCommand.sideEffectResult$lzycompute(commands.scala:57)
at org.apache.spark.sql.execution.ExecutedCommand.sideEffectResult(commands.scala:57)
at org.apache.spark.sql.execution.ExecutedCommand.doExecute(commands.scala:69)
at org.apache.spark.sql.execution.SparkPlan$$anonfun$execute$5.apply(SparkPlan.scala:140)
@Aslan
Aslan / scala_sbt_jenkins.sh
Last active Nov 2, 2015
Install scala on jenkins server(ubuntu 12.04) based on https://gist.github.com/visenger/5496675
View scala_sbt_jenkins.sh
# upgrade from java 1.6 to 1.7
apt-get install openjdk-7-jre
update-alternatives --config java # chose java 1.7
# install scala 2.10.4 (same version as AWS EMR)
apt-get remove scala-library scala
wget http://www.scala-lang.org/files/archive/scala-2.10.4.deb
apt-get update
apt-get install scala
View gist:d4bae0841ae83799c396
[hadoop@ip-10-65-200-150 ~]$ /usr/bin/spark-submit --master yarn-cluster --class com.truex.prometheus.CLIJob /home/hadoop/Prometheus-assembly-0.0.1.jar -e 'select x.id, x.title, x.description, x.mediaavailableDate as available_date, x.mediaexpirationDate as expiration_date, mediacategories.medianame as media_name, x.mediakeywords as keywords, mediaratings.scheme as rating_scheme, mediaratings.rating, cast(mediaratings.subRatings as String) as sub_ratings, content.plfileduration as duration, x.plmediaprovider as provider, x.ngccontentAdType as ad_type, x.ngcepisodeNumber as episode, ngcnetwork as network, x.ngcseasonNumber as season_number, x.ngcuID as ngc_uid, x.ngcvideoType as video_type from etl lateral view explode(entries) entries as x lateral view explode(x.mediacategories) cat as mediacategories lateral view explode(x.mediaratings) r as mediaratings lateral view explode(x.mediacontent) mediacontent as content lateral view outer explode(x.ngcnetwork) net as ngcnetwork' -j http://feed.theplatform.com/f/n
View gist:875ef9ed0c0cca24dda7
15/10/26 19:19:38 INFO client.RMProxy: Connecting to ResourceManager at ip-10-65-200-150.ec2.internal/10.65.200.150:8032
Container: container_1444274555723_0062_02_000003 on ip-10-169-170-124.ec2.internal_8041
==========================================================================================
LogType:stderr
Log Upload Time:26-Oct-2015 19:18:20
LogLength:9332
Log Contents:
SLF4J: Class path contains multiple SLF4J bindings.
View gist:ebb24e747f630dc898a1
15/10/24 01:00:34 INFO client.RMProxy: Connecting to ResourceManager at ip-10-65-200-150.ec2.internal/10.65.200.150:8032
Container: container_1444274555723_0060_01_000003 on ip-10-169-170-124.ec2.internal_8041
==========================================================================================
LogType:stderr
Log Upload Time:24-Oct-2015 01:00:10
LogLength:4299
Log Contents:
SLF4J: Class path contains multiple SLF4J bindings.
View gist:dcb2fd1933c857519d86
executing pwd:
/home/hadoop
executing /usr/bin/spark-submit --class com.truex.prometheus.CLIJob Prometheus-assembly-0.0.1.jar -e 'select x.id, x.title, x.description, x.mediaavailableDate as available_date, x.mediaexpirationDate as expiration_date, mediacategories.medianame as media_name, x.mediakeywords as keywords, mediaratings.scheme as rating_scheme, mediaratings.rating, cast(mediaratings.subRatings as String) as sub_ratings, content.plfileduration as duration, x.plmediaprovider as provider, x.ngccontentAdType as ad_type, x.ngcepisodeNumber as episode, ngcnetwork as network, x.ngcseasonNumber as season_number, x.ngcuID as ngc_uid, x.ngcvideoType as video_type from etl lateral view explode(entries) entries as x lateral view explode(x.mediacategories) cat as mediacategories lateral view explode(x.mediaratings) r as mediaratings lateral view explode(x.mediacontent) mediacontent as content lateral view outer explode(x.ngcnetwork) net as ngcnetworkr' -j http://feed.theplatform.com/f/ngc/ngcngw-analytics /tmp/t
You can’t perform that action at this time.