Skip to content

Instantly share code, notes, and snippets.

Embed
What would you like to do?
(defn extract-dataframe-schema
[rec]
(let [fields (reduce (fn [lst schema-line]
(let [k (first schema-line)
t (if (= (count schema-line) 3) (last schema-line) DataTypes/StringType) ]
(conj lst (DataTypes/createStructField (name k) t NULLABLE)))) [] rec)
arr (ArrayList. fields)]
(DataTypes/createStructType arr)))
(defn as-rows
[rec]
(let [values (object-array (reduce (fn [lst v] (conj lst v)) [] rec))]
(RowFactory/create values)))
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment