Ruby 3 introduced Ractors, which are a mechanism for concurrency. However, in order to ensure that the concurrency does not create conflicts, Ractors impose constraints on the code. In particular, they limit the usage of variables that might otherwise allow for data races among Ractors. For example, class variables and class instance variables cannot be used within Ractors.
There are many existing libraries that currently rely on code that is incompatible with Ractors. Consequently, these libraries have features that cannot be used within Ractors. This prevents users from taking full advantage of the concurrency provided by Ractors. The objective of this project was to modify Ruby’s standard libraries so that they comply to the restrictions of Ractors. Ideally, this would allow users writing code in Ractors to freely utilize most features in Ruby’s standard libraries.
For each library, it had to be determined as to whether or not the library contained Ractor-incompatible features. This was done by searching the code for elements that are known to not work in Ractors. Additionally, running the library’s regular tests in Ractors revealed areas that caused errors. If these techniques did not show any issue with the code, the library would be regarded as Ractor-safe.
If a problem with the library was detected, it would be addressed by modifying the code. The modifications needed to be done in a way that would allow for backward-compatibility, having minimal effect on existing code using the library. Once the changes were made, the altered part would be tested. This cycle of searching, fixing, and testing would be repeated until the library achieved compatibility.
After a few libraries were inspected, it became possible to identify general solutions for particular patterns in the code.
One of simplest problems to deal with was the presence of non-frozen constants that were not intended to be modified. These variables were very common, and the problem was solved simply by freezing them.
Class variables and class instance variables that would not change value could be replaced by a frozen constant. The ones that were supposed to be modified had to be dealt with differently. In some cases, it would be possible to avoid using the variable in Ractors by calling a method that re-computes the value instead. In other cases, it may be okay for the variable to be Ractor-local. For these situations, the usage of the variable would be replaced with a call to a method that returns the data from Ractor-local storage (and initializes it to the appropriate default value if it is not yet set). This solution could sometimes be also applied to constant arrays/hashes that need to remain mutable.
It would also be important to search the library for methods defined using define_method or define_singleton_method, as methods defined in this way would not automatically be usable from Ractors. In many cases, this could be fixed by taking the block passed to define_method/define_singleton_method and changing it into a proc. Then, Ractor.make_shareable would be called on the proc before passing it to define_method/define_singleton_method. This allows the method to be isolated, making it comply with the Ractor restrictions. For cases where the method relies on shared data, a more complicated strategy would need to be devised.
Early on in the project, it became apparent that ENV posed a problem for Ractor-compatibility. ENV’s methods were not Ractor safe, and so the libraries that relied on it could not become fully Ractor-compliant.
In order to fix this, much of the first half of the project was dedicated to making ENV Ractor-safe. To do this, Ruby’s C code had to be modified to ensure that accessing ENV from multiple Ractors would not cause data races and crash the program.
In the end, much progress was made toward making ENV Ractor-safe. However, the new code still allows some inconsistencies to occur in ENV when it is accessed by more than one Ractor at once. There is still ongoing discussion about how to improve the proposal to avoid these situations.
The pull request can be found here: ruby/ruby#4636
Additionally, two bugs were discovered in Ractor’s behavior. The first was that class variables could be accessed within non-main-Ractors if they had been accessed once previously in the main Ractor. The other bug was that Ractor-incompatible global variables could be accessed in Ractors through an alias. These bugs were not a priority for this project, but were reported to the project mentor.
One library that required a different procedure from the others was Singleton. Because this library allows a class to maintain exactly one instance, classes that include Singleton cannot be fully Ractor-compatible (as the instance can only be in one Ractor). Instead of modifying the existing code, a new module, RactorLocalSingleton, was proposed. This would allow classes to have exactly one instance per Ractor, which resolves the incompatibility issue. However, because there are general concerns about the Singleton design pattern, there is some discussion about whether RactorLocalSingleton should be made, since it promotes the continued use of Singleton.
Table 1 lists the libraries that appeared to be Ractor-safe without any modification:
| Library |
|---|
| abbrev |
| find |
| time |
| matrix |
| observer |
| tsort |
| shellwords |
| open3 |
| base64 |
| prettyprint |
Table 2 shows the libraries that were modified, and provides a link to the pull request for the modifications:
| Library | Pull Request | Pull Request Status (as of end of project) |
|---|---|---|
| uri | ruby/uri#29 | Submitted PR |
| tmpdir | ruby/tmpdir#9 | Submitted PR |
| timeout | ruby/timeout#4 | Submitted PR |
| benchmark | ruby/benchmark#11 | Submitted PR |
| weakref | ruby/weakref#2 | Submitted PR |
| ipaddr | ruby/ipaddr#30 | Submitted PR |
| delegate | ruby/delegate#4 | Submitted PR |
| pstore | ruby/pstore#2 | Submitted PR |
| ostruct | ruby/ostruct#29 | PR merged |
| forwardable | ruby/forwardable#21 | Submitted PR |
| singleton | ruby/singleton#4 | Submitted PR; Approach under discussion |
| prime | (None) | Waiting for Singleton |
| csv | ruby/csv#218 | Submitted PR |
| mutex_m | ruby/mutex_m#7 | Submitted PR |
| error_highlight | ruby/error_highlight#11 | Submitted PR |
| did_you_mean | ruby/did_you_mean#163 | Submitted PR |
Currently, the pull request for ostruct has been merged, while the others are still under review. Tests for all of the modifications are also currently being written.
- Changes to uri:
- In uri/ftp.rb:
- Froze TYPECODE
- In uri/ldap.rb:
- Froze SCOPE, SCOPE_ONE, SCOPE_SUB, SCOPE_BASE
- In uri/rfc2396_parser.rb:
- Froze ALPHA, ALNUM, HEX, ESCAPED, UNRESERVED, RESERVED, DOMLABEL, TOPLABEL, HOSTNAME
- In uri/rfc2396_parser.rb and uri/rfc3986_parser.rb:
- Addressed class variable @@to_s
- Removed @@to_s, which was set to Kernel.instance_method(:to_s)
- Replaced necessary usage directly with Kernel.instance_method(:to_s)
- Addressed class variable @@to_s
- In uri/ftp.rb:
- Changes to tmpdir:
- In tmpdir.rb:
- Addressed class variable @@systmpdir
- Created a method that returns @@systmpdir when in the main Ractor, and returns the default value for @@systmpdir otherwise
- Replaced usage of @@systmpdir with calls to this method
- Fixed unshareable constant RANDOM
- Made RANDOM an instance of Object instead of Random (because instances of Random cannot be shareable)
- As a side effect, RANDOM now uses Kernel#rand instead of Random#rand (which should not make a difference in this case)
- Made RANDOM shareable
- Made RANDOM an instance of Object instead of Random (because instances of Random cannot be shareable)
- Addressed class variable @@systmpdir
- In tmpdir.rb:
- Changes to timeout:
- In timeout.rb:
- Froze VERSION
- In timeout.rb:
- Changes to benchmark:
- In benchmark/version.rb:
- Froze VERSION
- In benchmark/version.rb:
- Changes to weakref:
- In weakref.rb:
- Addressed class variable @@__map
- Added a private method weakref_map:
- If Ractor is defined, returns Ractor.current[:__WeakRef_map__] (and initializes it if nil)
- Otherwise, returns @@__map
- Replaced usage of @@__map with calls to weakref_map
- Added a private method weakref_map:
- Addressed class variable @@__map
- In weakref.rb:
- Changes to ipaddr:
- In ipaddr.rb:
- Froze IN6FORMAT, Socket::AF_INET6
- In ipaddr.rb:
- Changes to delegate:
- In delegate.rb:
- Addressed private constant KERNEL_RESPOND_TO, which held an UnboundMethod (which cannot be frozen)
- Deleted the constant
- In the method it was used in, created a regular variable kernel_respond_to instead
- For all occurrences of define_method and define_singleton_method, made sure that the Proc was shareable before the method was defined
- Addressed private constant KERNEL_RESPOND_TO, which held an UnboundMethod (which cannot be frozen)
- In delegate.rb:
- Changes to pstore:
- In pstore.rb:
- Froze EMPTY_MARSHAL_DATA, EMPTY_MARSHAL_CHECKSUM
- Made Proc in define_method shareable (for defining on_windows?)
- In pstore.rb:
- Changes to ostruct:
- In ostruct.rb:
- Made Procs shareable for defining methods of OpenStruct members
- Allows the methods to still be usable if the members are moved into other Ractors
- Made Procs shareable for defining methods of OpenStruct members
- In ostruct.rb:
- Changes to forwardable:
- In forwardable.rb:
- Froze VERSION, FORWARDABLE_VERSION
- In forwardable.rb:
- Changes to singleton:
- In singleton.rb:
- Organized code in the Singleton module into more sub-modules
- Created a new module RactorLocalSingleton
- Includes and extends modules in Singleton
- Overrides some methods in order to let the instance be stored in Ractor-local storage
- In singleton.rb:
- Changes to prime:
- In prime.rb:
- Froze VERSION
- Moved from Singleton to RactorLocalSingleton
- In prime.rb:
- Changes to csv:
- In csv.rb:
- Addressed constant CONVERTERS
- Created DEFAULT_CONVERTERS, to store the starting content
- Created a method that returns a Ractor-local hash, which is set to the content of DEFAULT_CONVERTERS if it does not exist yet
- Set CONVERTERS to refer to the hash in the main-Ractor
- Addressed contants HEADER_CONVERTERS
- Applied the same process that was done for CONVERTERS
- Replaced @@instances with a Ractor-local hash
- Had "\n" be used instead of $INPUT_RECORD_SEPARATOR in Ractors
- Addressed constant CONVERTERS
- In csv/parser.rb:
- Replaced class variable @@string_scanner_scan_accept_string with a method
- Had "\n" be used instead of $INPUT_RECORD_SEPARATOR in Ractors
- In csv/writer.rb:
- Had "\n" be used instead of $INPUT_RECORD_SEPARATOR in Ractors
- In csv.rb:
- Changes to mutex_m:
- In mutex_m.rb:
- Froze VERSION
- In mutex_m.rb:
- Changes to error_highlight:
- In formatter.rb:
- Replaced @@formatter with a Ractor-local variable
- In version.rb:
- Froze VERSION
- In formatter.rb:
- Changes to did_you_mean:
- In did_you_mean.rb:
- Addressed constant hash SPELL_CHECKERS
- Created a new method self.spell_checkers_map
- The method returns a Ractor-local hash, and initializes it if not yet set
- Replaced usage of SPELL_CHECKERS with calls to the method
- Addressed class variable @@formatter
- Changed methods self.formatter and self.formatter= to return a Ractor-local variable instead
- Addressed constant hash SPELL_CHECKERS
- In did_you_mean/spell_checkers/method_name_checker.rb:
- Addressed constant hash NAMES_TO_EXCLUDE
- Created method that returns an initial value of this hash
- Created a method that returns a Ractor-local version of the hash, and uses the initialization method if necessary
- Replaced usage of NAMES_TO_EXCLUDE with calls to this method
- Made RB_RESERVED_WORDS shareable
- Addressed constant hash NAMES_TO_EXCLUDE
- In did_you_mean/spell_checkers/name_error_checkers/variable_name_checker.rb:
- Addressed constant hash NAMES_TO_EXCLUDE
- Created method that returns an initial value of this hash
- Created a method that returns a Ractor-local version of the hash, and uses the initialization method if necessary
- Replaced usage of NAMES_TO_EXCLUDE with calls to this method
- Made RB_RESERVED_WORDS shareable
- Addressed constant hash NAMES_TO_EXCLUDE
- In did_you_mean/spell_checkers/require_path_checker.rb:
- Froze ENV_SPECIFIC_EXT
- Addressed class instance variable @requireables
- Replaced with a Ractor-local variable initialized to INITIAL_LOAD_PATH if necessary
- In did_you_mean/version.rb:
- Froze VERSION
- In did_you_mean.rb:
Many of the pull requests have yet to be merged.
There are some standard libraries for which issues have been found, but they are not yet resolved. They are listed in Table 3:
| Library |
|---|
| tempfile |
| erb |
| ripper |
| securerandom |
| test-unit |
| digest |
| optparse |
| English* |
| set |
| rinda |
| getoptlong |
| pp |
| open-uri |
| fileutils |
| logger |
| rdoc |
| resolv** |
| reline** |
*The issue found in English was that it aliases Ractor-incompatible global variables, which causes a bug mentioned in the Issues Encountered section **These libraries are in progress as of the end of the project
Also, many of Ruby’s standard libraries have not yet been checked. A large portion of these are libraries that have code written in C. Handling these libraries will require additional techniques and strategies. Table 4 shows the remaining standard gems that have not yet been checked (based on the list of standard gems from https://stdgems.org/)
| Library |
|---|
| bigdecimal |
| bundler |
| cgi |
| date |
| dbm |
| debug |
| drb |
| etc |
| fcntl |
| fiddle |
| minitest |
| net-ftp |
| net-http |
| net-imap |
| net-pop |
| net-protocol |
| net-smtp |
| nkf |
| openssl |
| pathname |
| power_assert |
| psych |
| racc |
| rake |
| rbs |
| readline |
| readline-ext |
| resolv-replace |
| rexml |
| rss |
| rubygems |
| stringio |
| strscan |
| syslog |
| tracer |
| typeprof |
| un |
| win32ole |
| yaml |
| zlib |
Finally, there are many popular non-standard libraries as well, and it would be ideal to eventually make these libraries Ractor-compatible too.