Data conversion scripts

README.md

Data conversion scripts

This is a collection of scripts to convert data from one format to another.

CSV to JSON

What? Turns CSV into JSON Why? JSON is nicer to navigate and query using jq or fx How?

# Pass the file on the command-line
./csv-to-json.rb $FILE
# or use stdin
./csv-to-json.rb -

HTML table to CSV

What? Turns an HTML table into CSV Why? Sometimes, websites don't provide the right APIs and you just want to export data from a table. How?

# Pass the file on the command-line
./table-to-csv.rb $FILE
# or use stdin
./table-to-csv.rb -

csv-to-json.rb

#!/usr/bin/env ruby

require 'csv'
require 'json'

file = ARGV[0] == '-' ? STDIN.read : File.new(ARGV[0])
csv_table = CSV.new(file, headers: true)

values = []

csv_table.each do |row|
  values << row.to_hash
end

puts JSON.pretty_generate(values)

table-to-csv.rb

require 'nokogiri'
require 'csv'

file = ARGV[0] == '-' ? STDIN.read : File.new(ARGV[0])
doc = Nokogiri::HTML(file)

headers = doc.xpath('//table//th').map {|header| header.text }
rows = []
doc.xpath('//table//tr').each do |row|
  csv_row = CSV::Row.new([], [])
  row.xpath('td').each_with_index do |cell, index|
    csv_row << [headers[index], cell.text.strip]
  end
  next if csv_row.empty?

  rows << csv_row
end

print CSV::Table.new(rows).to_csv