billdueber

require 'rubygems'
require 'threach'
require 'jruby_streaming_update_solr_server'

solrURL = 'your solr url'
sussQueueSize = 128 # number of docs to queue up
sussThreads = 1 # number of threads to use to send stuff to solr

threads = 3 # number of threads to use to process the data

billdueber

1006    DLI
1040    GyWOH
1059    SciDir
1062    NYU
1077    ItFiC
1150    CSt
1175    FrPJT
1176    DGPO
1182    NjP
1196    BLKDR

billdueber

# Then just create a simple XML file pointing to the 50k line files
# Don't forget to gzip the files first

#!/usr/local/bin/perl

my $numfiles = ARGV[0]; # number of files generated before
my $urlToSitemapDir = 'http://www.my.machine.edu/dir/for/sitemaps';

print q{<?xml version="1.0" encoding="UTF-8"?>
<sitemapindex xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">

billdueber

{
 "responseHeader":{
  "status":0,
  "QTime":0,
  "params":{
	"explainOther":"",
	"fl":"*,score",
	"indent":"on",
	"start":"0",

billdueber

require 'rubygems'
require 'marc'
require 'open-uri'

r = MARC::Reader.new(open('http://mirlyn.lib.umich.edu/Record/000039829.marc')).first
puts r

billdueber

require 'java'


def showBreakProblem &blk
  threads = 2
  consumers = []
  threads.times do |i|
    consumers << Thread.new(i) do |i|
      Thread.current[:num] = i
      begin

billdueber

1. Go to the app directory

   cd /Applications/Google\ Chrome.app/Contents/MacOS/

2. Rename the app to app.orig

   mv Google\ Chrome Google\ Chrome.orig

3. Create a shell script with the original name that uses the args you want

billdueber

require 'parslet'
require 'pp'

class AdvParser < Parslet::Parser
  rule(:space)  { match['\\s\\t'].repeat(1) } # at least one space/tab
  rule(:space?) { space.maybe } # zero or 1 things that match the 'space' rule
  
  rule(:startexpr) { str('(') >> space? } # '(' followed by optional space
  rule(:endexpr)   { space? >> str(')') }
  

billdueber

module A
  def a
  end
end

class C
end

n = 100_000

billdueber

I upgraded my ruby 1.8 to the latest patchlevel and all of a sudden ruby-marc was super-slow. I found the same thing on 1.9 and in JRuby, so I investigated.

There's a marc.bytes.to_a call inside the loop in Reader#decode. All the fix does is move it outside the loop so it only happens once.

You can see the patch in the "slowreadfix" branch at https://github.com/ruby-marc/ruby-marc/commit/beba83745ebe0848218496e967edd65d632fb01e

As you can see, the speedup is about a factor of five.

Test case is reading in a Marc21 file with about 18K records in it.