00_README.md

Nobody understands the GIL

Nobody understands the #ruby GIL https://www.jstorimer.com/blogs/workingwithcode/8085491-nobody-understands-the-gil

It's kind of normal that ruby developers think about concurrency like something given and done in a Thread block. It's not that easy and straightforward.

bat pushing_nil.rb

array = []

5.times.map do
  Thread.new do
    1000.times do
      array << nil
    end
  end
end.each(&:join)

puts array.size

(For the sake on running in isolation I did prepare a wrapper) bat wrapper.sh

for i in {1..5}
  do
    $RUBY_BIN data/pushing_nil.rb
  done

(about to execute the code to probe)

docker run -ti --rm -v $PWD:/data -e RUBY_BIN=ruby ruby:2.4 bash data/wrapper.sh
5000
5000
5000
5000
5000

Looks nice in cruby:

Now in jruby

docker run -ti --rm -v $PWD:/data -e RUBY_BIN=jruby jruby bash data/wrapper.sh
4998
4976
4994
5000
5000

Nope, the numbers looks evenly random

(Continuing 2/3)

Sometimes @jruby will warn you using exceptions (shows the same code for pushing_nil.rb and wrapper.sh)

docker run -ti --rm -v $PWD:/data -e RUBY_BIN=jruby jruby bash data/wrapper.sh
5000
5000
4998
4991
warning: thread "Ruby-0-Thread-2: data/pushing_nil.rb:1" terminated with exception (report_on_exception is true):
ConcurrencyError: Detected invalid array contents due to unsynchronized modifications with concurrent users
                   << at org/jruby/RubyArray.java:1281
  data/pushing_nil.rb at data/pushing_nil.rb:5
  data/pushing_nil.rb at data/pushing_nil.rb:4
warning: thread "Ruby-0-Thread-1: data/pushing_nil.rb:1" terminated with exception (report_on_exception is true):
ConcurrencyError: Detected invalid array contents due to unsynchronized modifications with concurrent users
                   << at org/jruby/RubyArray.java:1281
  data/pushing_nil.rb at data/pushing_nil.rb:5
  data/pushing_nil.rb at data/pushing_nil.rb:4
ConcurrencyError: Detected invalid array contents due to unsynchronized modifications with concurrent users
                   << at org/jruby/RubyArray.java:1281
  data/pushing_nil.rb at data/pushing_nil.rb:5
  data/pushing_nil.rb at data/pushing_nil.rb:4
warning: thread "Ruby-0-Thread-5: data/pushing_nil.rb:1" terminated with exception (report_on_exception is true):
ConcurrencyError: Detected invalid array contents due to unsynchronized modifications with concurrent users
                   << at org/jruby/RubyArray.java:1281
  data/pushing_nil.rb at data/pushing_nil.rb:5
  data/pushing_nil.rb at data/pushing_nil.rb:4
  [...more...]

Did you see that? jRuby detected the problem, but it's not guaranteed

So, what can we do about it? A lot of things, one easy "workaround" is using a mutex to be sure the not-thread safe data structure can handle it... In our example cruby and @jruby now have same results.

bat pushing_nil_synced.rb

array = []
mutex = Mutex.new

5.times.map do
  Thread.new do
    mutex.synchronize do
      1000.times do
        array << nil
      end
    end
  end
end.each(&:join)

puts array.size

bat wrapper_sync.sh

for i in {1..5}
  do
    $RUBY_BIN data/pushing_nil_synced.rb
  done

Let's try again in cruby

docker run -ti --rm -v $PWD:/data -e RUBY_BIN=ruby ruby:2.4 bash data/wrapper.sh
5000
5000
5000
5000
5000

Still looks good in cruby

Let's try in jruby

docker run -ti --rm -v $PWD:/data -e RUBY_BIN=jruby ruby:2.4 bash data/wrapper_sync.sh
5000
5000
5000
5000
5000

Now we are talking... the results are the same in cruby and jruby, but need to be synced.

01_pushing_nil.rb

array = []
5.times.map do
  Thread.new do
    1000.times do
      array << nil
    end
  end
end.each(&:join)

puts array.size

02_wrapper.sh

for i in {1..5}
  do
    $RUBY_BIN data/pushing_nil.rb
  done

03_pushing_nil_synced.rb

array = []
mutex = Mutex.new

5.times.map do
  Thread.new do
    mutex.synchronize do
      1000.times do
        array << nil
      end
    end
  end
end.each(&:join)

puts array.size

04_wrapper_sync.sh

for i in {1..5}
  do
    $RUBY_BIN data/pushing_nil_synced.rb
  done

05_docker_executions.sh

docker run -ti --rm -v $PWD:/data -e RUBY_BIN=ruby ruby:2.4 bash data/wrapper.sh
docker run -ti --rm -v $PWD:/data -e RUBY_BIN=jruby jruby bash data/wrapper.sh
docker run -ti --rm -v $PWD:/data -e RUBY_BIN=ruby ruby:2.4 bash data/wrapper_sync.sh
docker run -ti --rm -v $PWD:/data -e RUBY_BIN=jruby jruby bash data/wrapper_sync.sh