Skip to content

Instantly share code, notes, and snippets.

@igrigorik
Last active June 8, 2023 01:37
Show Gist options
  • Star 1 You must be signed in to star a gist
  • Fork 3 You must be signed in to fork a gist
  • Save igrigorik/2017506 to your computer and use it in GitHub Desktop.
Save igrigorik/2017506 to your computer and use it in GitHub Desktop.
require 'open-uri'
require 'zlib'
require 'yajl'
gz = open('http://data.gharchive.org/2015-01-01-12.json.gz')
js = Zlib::GzipReader.new(gz).read
Yajl::Parser.parse(js) do |event|
print event
end
@brianmario
Copy link

fwiw you should be able to pass the GzipReader instance directly to yajl-ruby and it'll read 8kb chunks from it at a time

@igrigorik
Copy link
Author

@brianmario: Yajl::Parser.parse(Zlib::GzipReader.new(gz))? Speaking of which, I think I saw a comment indicating that in 2.0 the plan is to remove the Gzip StreamReader?

@brianmario
Copy link

After looking at the yajl-ruby source, looks like passing a Zlib::GzipReader won't actually work because I pass a 2nd argument (buffer to fill) to the read method and it doesn't support that signature :\

And yes I was planning on removing basically everything but Yajl::Parser and Yajl::Encoder from 2.0. I wanted to just drop in some more stuff into the examples/ to show how those things might be done without being officially supported by the gem anymore. Are you using any of that stuff currently?

@igrigorik
Copy link
Author

Nope, not on this end. I'm sure it'll break a few downstream clients, but 'simple core' strategy makes sense to me.

@surekha803
Copy link

Hello,
Can you please let me know the process how to download the GH Archive files. I am new to this.I want this data to analysis.I tried through postman but didn't work.I am not getting how to run this ruby script. Can anyone Please help me here

@GPAP15
Copy link

GPAP15 commented Jun 8, 2023

Can someone please let me know the process of how to download the GH Archive files? I am new to this. I want this data to analysis.I am not getting how to run this Ruby script.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment