Skip to content

Instantly share code, notes, and snippets.

View al102964's full-sized avatar

Arturo Gonzalez al102964

  • Data Scientist ITAM
  • México city
View GitHub Profile
@al102964
al102964 / SparkCopyPostgres.scala
Created April 6, 2021 04:58 — forked from longcao/SparkCopyPostgres.scala
COPY Spark DataFrame rows to PostgreSQL (via JDBC)
import java.io.InputStream
import org.apache.spark.sql.execution.datasources.jdbc.JdbcUtils
import org.apache.spark.sql.{ DataFrame, Row }
import org.postgresql.copy.CopyManager
import org.postgresql.core.BaseConnection
val jdbcUrl = s"jdbc:postgresql://..." // db credentials elided
val connectionProperties = {