Skip to content

Instantly share code, notes, and snippets.

View zoltanctoth's full-sized avatar

Zoltan C. Toth zoltanctoth

108 followers · 36 following
View GitHub Profile
@zoltanctoth
zoltanctoth / newmonth.sh
Last active April 14, 2018 07:40
create new month in datapao admin
gfind 2018-02\ február -type d -print0 | sed 's/2018-02 február/2018-03 március/g' | xargs -0 -I {} mkdir -p {}
@zoltanctoth
zoltanctoth / ggplot2-solutions.r
Created May 9, 2017 04:43
Here you can find the solutions for the exercises we used at the class.
library(ggplot2)
# Take a look at our example dataset
View(diamonds)
# Make a chart from scratch
x = ggplot() +
layer(
data = diamonds, mapping = aes(x=carat,y=price),
stat='identity', position="identity", geom="point"
@zoltanctoth
zoltanctoth / ggplot2-examples.r
Last active January 17, 2021 18:10
ggplot2 examples and exercises
library(ggplot2)
# Take a look at our example dataset
View(diamonds)
# Make a chart from scratch
x = ggplot() +
layer(
data = diamonds, mapping = aes(x=carat,y=price),
stat='identity', position="identity", geom="point"
@zoltanctoth
zoltanctoth / spark-kafka.scala
Last active February 6, 2017 20:23
How to use the Direct Kafka Source in Scala with offset Specification
import org.apache.spark._
import org.apache.spark.sql.Column
import org.apache.spark.streaming._
import _root_.kafka.serializer.StringDecoder
import org.apache.spark.streaming._
import org.apache.spark.sql.SparkSession
import org.apache.spark.sql.SQLContext
import org.apache.kafka.clients.consumer.ConsumerRecord
import org.apache.kafka.common.TopicPartition
import org.apache.kafka.common.serialization.StringDeserializer
@zoltanctoth
zoltanctoth / spark-kafka.scala
Created February 6, 2017 20:09
How to use the Direct Kafka Source in Scala
object Anomymizer extends App {
val spark = SparkSession.builder
.master("local[3]")
.appName("Anonimizer")
.getOrCreate()
val salt = "SAALT"
def anonimizeStr(a:Any) = {
a match {
@zoltanctoth
zoltanctoth / spark-kafka.scala
Created February 6, 2017 20:09
How to use the Direct Kafka Source in Scala
object Anomymizer extends App {
val spark = SparkSession.builder
.master("local[3]")
.appName("Anonimizer")
.getOrCreate()
val salt = "SAALT"
def anonimizeStr(a:Any) = {
a match {
@zoltanctoth
zoltanctoth / spark-kafka.scala
Created February 6, 2017 20:09
How to use the Direct Kafka Source in Scala
object Anomymizer extends App {
val spark = SparkSession.builder
.master("local[3]")
.appName("Anonimizer")
.getOrCreate()
val salt = "SAALT"
def anonimizeStr(a:Any) = {
a match {
@zoltanctoth
zoltanctoth / h2o-sparkling-water-deep-learning.scala
Created September 13, 2016 20:09
This is a Spark <-> H2O / Sparkling water deep learning prototype.
import org.apache.spark.{SparkConf, SparkContext}
import org.apache.spark.h2o.{H2OContext, H2OFrame}
import org.apache.spark.sql.DataFrame
import hex.deeplearning.DeepLearning
import water.app.SparkContextSupport
import hex.deeplearning.DeepLearningParameters
import hex.deeplearning.DeepLearningParameters.Activation
import org.apache.spark.h2o.{DoubleHolder, H2OContext, H2OFrame}
@zoltanctoth
zoltanctoth / move-wordpress-to-different-domain.sh
Last active September 25, 2015 06:26
Moving wordpress to an other domain can be a hassle. Here is a script on how to do it in without the pain.
#!/bin/bash -xeu
# This script moves your wordrpress page under a different domain
# Zoltan C. Toth
export HISTCONTROL=ignorespace
ORIGIN_DOMAIN=teszt2.gyulahus.hu
TARGET_DOMAIN=teszt.gyulahus.hu
ORIGIN_DIR=/home/gyulahus/public_html/$ORIGIN_DOMAIN
TARGET_DIR=/home/gyulahus/public_html/$TARGET_DOMAIN
TARGET_DB=teszt2_gyh
@zoltanctoth
zoltanctoth / pyspark-udf.py
Last active July 15, 2023 13:23
Writing an UDF for withColumn in PySpark
from pyspark.sql.types import StringType
from pyspark.sql.functions import udf
maturity_udf = udf(lambda age: "adult" if age >=18 else "child", StringType())
df = spark.createDataFrame([{'name': 'Alice', 'age': 1}])
df.withColumn("maturity", maturity_udf(df.age))
df.show()