Subject: ANNOUNCEMENT: Traject MARC->Solr indexer beta release
Jonathan Rochkind (Johns Hopkins), along with Bill Dueber (University of Michigan), is happy to announce a first beta release of "traject," a framework for indexing MARC data to Solr.
traject, in the vein of solrmarc, allows you to define your indexing rules using simple macro and translation files. However, traject runs under JRuby and is "ruby all the way down," so you can easily provide additional logic by simply requring ruby files.
traject is currently in a beta release, but is already being used in production to generated the HathiTrust Catalog (http://www.hathitrust.org/). traject was developed under a test-first mentality and has undergone both continuous integration and an extensive benchmarking/profiling period to keep it fast.
You can view the code[1] on github, and easily install it as a (jruby) gem using "gem install traject".
To get a feel for traject, there is a well-documented sample configuration[2] you can clone. It includes a guide to the most important moving parts[3], which will continue to be fleshed out.
Part of that guide is a document[4] describing the major features, but briefly they are:
- It's all just well-crafted and documented ruby code; easy to program, easy to read, easy to modify (the whole code base is only 6400 lines of code, more than a third of which is tests)
- Fast. Traject by default indexes using multiple threads, so you can use all your cores!
- Decoupled from specific readers/writers, so you can use ruby-marc or marc4j to read, and write to solr, a debug file, or anywhere else you'd like with a little extra code.
- Designed so it's easy to test your own code and distribute it as a gem
We're hoping to build up an ecosystem around traject and encourage people to ask questions and contribute code (either directly to the project or via gem releases).
[1] http://github.com/jrochkind/traject [2] http://github.com/billdueber/traject_sample [3] https://github.com/billdueber/traject_sample/tree/master/guide [4] https://github.com/billdueber/traject_sample/blob/master/guide/why_traject.md
I'll want to review the traject_sample stuff, I guess.
I am still dubious that all that is needed for people to get started, the traject_sample stuff. You really think the extensive README and additional docs in traject aren't enough, and the announcement needs to direct people to the sample stuff on top of the copious documentation in traject?