Skip to content

Instantly share code, notes, and snippets.

@DaveWM DaveWM/csv-to-kafka.clj

Last active Jul 10, 2020
Embed
What would you like to do?
#!/usr/bin/env bb
;; ^^ this tells our shell to use Babashka to run this script
;; read the file path of our CSV and the key field from the command line args
(def csv-file-path (first *command-line-args*))
(def key-field (second *command-line-args*))
;; read the CSV line-by-line into a data structure
(def csv-data
(with-open [reader (io/reader csv-file-path)]
(doall (csv/read-csv reader))))
(def headers (first csv-data))
(def body (rest csv-data))
;; For each line in the body, create a map with the headers as the keys
(def values
(->> body
(map (partial zipmap headers))
;; if you need to do any additional processing on each line, do it here
))
(def output-lines
(->> values
(map #(str (get % key-field) "::" (json/generate-string %)))))
(doseq [output output-lines]
(println output))
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.