My project of the Google Summer of Code 2018 is adding multi-factor (or so-called two-factor) authentication to gem, the default shipped package management tool of Ruby, and RubyGems.org, the site hosting gems.
| Item | Content |
|---|---|
| Project | Adding multi-factor authentication to RubyGems |
| Organization | Ruby |
| Project link | https://summerofcode.withgoogle.com/projects/#5815164901785600 |
| Project reports | https://ecnelises.github.io |
| Mentors | André Arko, Aditya Prakash |
| Student | Qiu Chaofan |
| Proposal | Link to PDF |
| Total changes | 49 Commits / 1637++ / 106-- |
| Link | Description | Status |
|---|---|---|
| RubyGems.org PR #1729 | Multi-factor authentication in UI part. | Merged |
| RubyGems.org PR #1756 | Add expiry (30 minutes) of QR-code. | Merged |
| RubyGems.org PR #1753 | Multi-factor authentication in API part. | Merged |
| RubyGems.org PR #1761 | Remove check against OTP reuse. | Merged |
| RubyGems PR #2369 | Add multi-factor authentication to RubyGems client. | Merged |
| Guides PR #218 | Add pages on multi-factor authentication to rubygems guides. | Merged |
All related PRs are listed here. Some of them have not been merged until first edition of this document. I'll update status as they got changed.
Currently, our gem hosting site, RubyGems.org uses an email and password based authentication method, with an API key provided for client requests. The main goal of this project is to add an extra authentication phase to them. Users are expected to use digit code on their authenticator apps (like Google Authenticator) in their mobile phone or other devices, to confirm critical actions. Also, user can set their multi-factor authentication status, like disabling it.
In general, my project consists of four parts, from addition to existing package hosting site to client tool and documents.
Although gem uses standard RESTful style API to communicate with server, we can login and edit profiles on the hosting site. The non-API related code is often called UI part. Main work in this stage includes:
- Prepare for new external dependencies for OTP (one-time password, namely the digit code) checking work.
- Write basic authentication methods for models.
- Add options for user to enable or disable their settings about this feature.
- Add extra authentication to login action.
- Write unit tests and integration tests.
I found some multi-factor authentication related libraries on GitHub at first, but some of them are no longer maintained and most of them are extension of devise (a famous authentication extension for Ruby on Rails). I went through their code and found core parts of them all relies on a library named rotp. As a result, rotp is added into the project. Also, designed implementation should contain a QR-code for user to scan, so another QR-code related library was also added.
After suitable library is chosen and tested, I wrote some basic authentication methods to User model, which is the base of further work on both UI and API. I recorded implementation details in my progress reports. See 5/25 report.
For what’s next, I designed related pages first. This issue contains related contents. And the final page is like below.
Before merged into code base, I wrote thorough unit tests for MFA settings, verification functionality of users, and integration tests for the login and MFA setting workflow.
MFA in UI part has been deployed. (as in Jul. 2018) But people who want to have it a try should set a special cookie entry, mfa_feature to true, to enable it. It's still in test phase, will be turned on by default in the future.
RubyGems API is used by gem client tool and other third-party providers to create, update or delete packages of users. In this stage, I wrote a wrapper for API entries to verify OTP from user.
In my implementation, OTP field will be sent in OTP field in HTTP header. An API action will check Authorization field for API key first to query for current API user. If current user has enabled mfa login and write option and the action (like Rubygems#create) needs extra authentication, the OTP will be verified. Otherwise, server just ignores the field.
OTP here is verified through rotp library, but with a drift in case of unexpected time inconsistency. OTP generated in last interval (30 seconds since UNIX timestamp epoch) or next interval is still valid in verification. In addition, every time when OTP is successfully verified, that timestamp will be recorded and the corresponding OTP cannot be used again.
7/22 report is a good detailed reference for my API implementation.
| Action | Path | Use |
|---|---|---|
rubygems#create |
POST /api/v1/rubygem |
Pushing new gem |
owners#create |
POST /api/v1/owner |
Add owner information to gem |
owners#destroy |
DELETE /api/v1/owner |
Remove owner information from gem |
Work on API part is also tested.
gem uses Git-like style of commands. Each command is actually a class at lib/rubygems/commands. Currently, I finished multi-factor authentication on three commands.
gem push, release or update gemsgem owner, show, add and remove owner information in a gemgem signin, login into rubygems.org (or other self-hosting site)
Code in RubyGems is part of Ruby distribution. Project structure is different from a standard Rails way. I started my work at GemcutterUtilities module and PushCommand class.
At first, I added a new API endpoint GET /api/v1/multifactor_auth, for client to detect if current user has enabled multi-factor authentication. But this will affect other test cases in the same command and break existing API request order. So later I changed it to two phase of requests.
- Send API requests as normal.
- If received
401 Unauthorizedand response text says MFA is enabled, ask for OTP and re-send the request.
This works well without adding times of requests. I can focus on adding tests related to my own work, without breaking others.
Users are expected to be able to fill their digit code through interactive prompts as below.
$ gem push hello-0.0.1.gem
Pushing gem to https://rubygems.org...
This command needs digit code for multifactor authentication.
Code: 111111
Successfully registered gem: hello (0.0.1)
But sometimes using command line options is necessary for scripts. I added --otp option for commands with MFA. In above case, typing gem push hello-0.0.1.gem --otp 111111 is equivalent. Actually, in my implementation, OTP is stored in options[:otp].
One thing to note out is, gem will ask you to login without terminating current process if you are doing a command but not logged in. Since we have a check against OTP reuse, here the program will delete OTP stored and re-ask it again after login. User should better wait until next interval to prevent verification failure. There's some discussion at PR to delete the check.
I had a mistake at beginning of writing tests for rubygems. I run individual tests using ruby test_file.rb without add -I option. So result is different from running rake. This confused me for a few days.
Test functionality of gem should require a running instance of rubygems.org in concept. But that would complicate things. So thanks to flexibility of Ruby language, we have a FakeFetcher to provide fake request results to API requests.
I wrote tests for each command, corresponding for two occasions: MFA verification success and failure. But FakeFetcher only supports matching a path to a static Rack style response. I did little change so that it can accept a callable object. Now I can set the response returns 401 first but returns 200 in second attempt.
In the first half of project process, I give an article as progress report to my mentors. They are archived at ecnelises.github.io.
Articles
| Link | Content |
|---|---|
| May 19th Report | About an issue about login sessions and basic preparation. |
| May 25th Report | Workflow & pages design, changes to user model. |
| Jun 3rd Report | Refactor for MFA related controller methods. |
| Jun 8th Report | Multi-factor authentication for login. |
| Jun 19th Report | Feature flags, drifts in OTP check, unit tests. |
| Jul 22nd Report | QR-code expiry, OTP check in API, work on client. |
| Summary of UI part work | Detailed work description for UI part. |
Also, I wrote a guide page for how to use multi-factor authentication in RubyGems and RubyGems.org. See this PR for detail.
Several parts of my project has not been merged into main branch yet. The first task is to discuss with community members, improve the code, make them into production. Some RFCs may be needed in the process.
Next should be about finishing more issues on the list.
Besides,
gem yankcommand also needs MFA support.- Support U2F-based authentication.
- Use a sign based method in stead of raw API key. This can also bring a session in which MFA is not needed in every action.
Great thanks to my mentors, @indirect and @sonalkr132. They gave me a lot of advice on design of the project and careful, patient review on my commits. The project would be much harder without their help!
Also, I should appreciate the kindness from GSoC-CN group. The members showed me their interesting projects and gave me important suggestions in the development process.
Hi @ecnelises
I think in this gist, you say that you implemented
api/v1/multifactor_authin Rubygems.org but then pivoted away. I see that the endpoint is still in routes, but that the controller is gone from the folder. Does this mean that endpoint doesn't do anything anymore, and can be removed? For context, I'm writing documentation for some missing API endpoints