Skip to content

Instantly share code, notes, and snippets.

Gianfranco Reppucci giefferre

Block or report user

Report or block giefferre

Hide content and notifications from this user.

Learn more about blocking users

Contact Support about this user’s behavior.

Learn more about reporting abuse

Report abuse
View GitHub Profile
@giefferre
giefferre / store_and_reuse_dataframe_schema.py
Last active Jul 3, 2018
Save the schema of a Spark DataFrame to be able to reuse it when reading json files.
View store_and_reuse_dataframe_schema.py
# read a part of the whole datalake just to extract the schema
part = spark.read.json("s3a://path/to/json/part")
# create a temporary rdd in order to store the schema as binary file
temp_rdd = sc.parallelize(part.schema)
temp_rdd.coalesce(1).saveAsPickleFile("s3a://path/to/destination_schema.pickle")
# from now on, the schema will be saved.
# it could be used to improve the speed of reading json files.
@giefferre
giefferre / keybase.md
Created Aug 23, 2016
keybase verification
View keybase.md

Keybase proof

I hereby claim:

  • I am giefferre on github.
  • I am giefferre (https://keybase.io/giefferre) on keybase.
  • I have a public key ASC5MZ4LhY4fbsPqLY6kt-PA8ifPTGW5zEAt5P-jkDmLCwo

To claim this, I am signing this object:

@giefferre
giefferre / remote-rsyslog.go
Created Mar 31, 2015
Using a remote rsyslog in Golang
View remote-rsyslog.go
package main
import (
"log"
"log/syslog"
)
func main() {
logwriter, e := syslog.Dial("tcp","123.123.123.123:12345", syslog.LOG_DEBUG, "your.software.identifier")
if e != nil {
You can’t perform that action at this time.