Skip to content

Instantly share code, notes, and snippets.

View al102964's full-sized avatar

Arturo Gonzalez al102964

  • Data Scientist ITAM
  • México city
View GitHub Profile
@longcao
longcao / SparkCopyPostgres.scala
Last active December 26, 2023 14:47
COPY Spark DataFrame rows to PostgreSQL (via JDBC)
import java.io.InputStream
import org.apache.spark.sql.execution.datasources.jdbc.JdbcUtils
import org.apache.spark.sql.{ DataFrame, Row }
import org.postgresql.copy.CopyManager
import org.postgresql.core.BaseConnection
val jdbcUrl = s"jdbc:postgresql://..." // db credentials elided
val connectionProperties = {