Skip to content

Instantly share code, notes, and snippets.

Long Cao longcao

Block or report user

Report or block longcao

Hide content and notifications from this user.

Learn more about blocking users

Contact Support about this user’s behavior.

Learn more about reporting abuse

Report abuse
View GitHub Profile
@longcao
longcao / SparkCopyPostgres.scala
Last active Sep 14, 2019
COPY Spark DataFrame rows to PostgreSQL (via JDBC)
View SparkCopyPostgres.scala
import java.io.InputStream
import org.apache.spark.sql.execution.datasources.jdbc.JdbcUtils
import org.apache.spark.sql.{ DataFrame, Row }
import org.postgresql.copy.CopyManager
import org.postgresql.core.BaseConnection
val jdbcUrl = s"jdbc:postgresql://..." // db credentials elided
val connectionProperties = {
View words.scala
/**
* Intersperses words with spaces and trims ends, accounts for basic smilies.
*/
def wordsToSentence(words: Vector[String]): String = {
val punctuation = Set(":", ";", ".", ",", "!", "?")
val smilies = Set(":)", ":(", ":<", ":}", ":|", ":o")
// @TODO really bad implementation probably, clean this up later
words.zipWithIndex.foldLeft("") { case (acc, (word, i)) =>
val isLastWordPairSmilie = if (words.isDefinedAt(i - 1)) {
@longcao
longcao / problems.md
Created Dec 20, 2015
sample scala problems to cover in cats docs?
View problems.md
  1. Handling stacked monads and the like e.g. List[Option[T]], `Future[Option[T]``
  2. Referentially transparent error handling
  3. Error accumulation/'parallel' validation
  4. Adding together Option[T]s (a la http://stackoverflow.com/a/16319667, not sure how to word this well)
  5. More explicitly typesafe collections (e.g. non-empty, guarantee that list.head never throws, etc)
@longcao
longcao / Server.scala
Created Dec 20, 2013
Server source code.
View Server.scala
import akka.actor._
object Server extends App {
val system = ActorSystem("serverSystem")
val serverActor = system.actorOf(Props(new ServerActor), name = "ServerActor")
}
class ServerActor extends Actor {
def receive = {
case message: String =>
@longcao
longcao / Client.scala
Last active Dec 31, 2015
Client source code.
View Client.scala
import akka.actor._
object Client extends App {
val system = ActorSystem("clientSystem")
val clientActor = system.actorOf(Props(new ClientActor), name = "ClientActor")
clientActor ! SayHello
}
class ClientActor extends Actor {
@longcao
longcao / client.conf
Last active Dec 31, 2015
client and server conf
View client.conf
# 'client' conf
akka {
actor {
provider = "akka.remote.RemoteActorRefProvider"
}
remote {
enabled-transports = ["akka.remote.netty.tcp"]
netty.tcp {
hostname = "127.0.0.1"
port = 2553
View dependencies.sbt
libraryDependencies ++= Seq(
"com.typesafe.akka" %% "akka-actor" % "2.2.3",
"com.typesafe.akka" %% "akka-remote" % "2.2.3"
)
You can’t perform that action at this time.