Skip to content

Instantly share code, notes, and snippets.

@stouset
Last active May 27, 2022 16:46
Show Gist options
  • Save stouset/f80dd031a57bbfd7dd2ca2636dddc023 to your computer and use it in GitHub Desktop.
Save stouset/f80dd031a57bbfd7dd2ca2636dddc023 to your computer and use it in GitHub Desktop.
Best-practices based API key authentication
# frozen_string_literal: true
# == Schema Information
#
# Table name: accounts
#
# id :uuid not null, primary key
# created_at :datetime not null
# updated_at :datetime not null
#
class Account < ApplicationRecord
has_many :api_keys,
inverse_of: :account,
dependent: :delete_all
def self.authenticate_by_api_key!(key)
ApiKey.authenticate!(key).account || raise(ActiveRecord::NotFound)
end
end
# frozen_string_literal: true
# == Schema Information
#
# Table name: api_keys
#
# id :uuid not null, primary key
# account_id :uuid not null
# key_id :string not null
# authenticator :string not null
# last_used_at :datetime not null
# created_at :datetime not null
# updated_at :datetime not null
#
class ApiKey < ApplicationRecord
MASTER_KEY = Rails.application.secrets.api_key_hkdf_master!
ENTROPY = 32
CONTEXT_KEY = 'api_key'
CONTEXT_AUTHENTICATOR = 'api_key/authenticator'
belongs_to :account,
inverse_of: :api_keys
attr_readonly :account_id
attr_readonly :key_id
attr_readonly :authenticator
attribute :authenticator, :secret_token
validates :key_id,
presence: true,
uniqueness: true
validates :authenticator,
presence: true,
uniqueness: true,
format: { with: SecretToken.regexp(ENTROPY, CONTEXT_AUTHENTICATOR) }
validates :last_used_at,
presence: true
def self.generate!
key_id = SecureRandom.uuid
token = SecretToken.generate(ENTROPY, CONTEXT_KEY, username: key_id)
authenticator = MASTER_KEY.authenticator(token)
self.create!(
key_id: key_id,
authenticator: authenticator,
)
token
end
def self.authenticate!(key)
token = SecretToken[key]
api_key = self.find_by!(key_id: token.username)
MASTER_KEY.authenticate!(token, api_key.authenticator)
api_key.touch(:last_used_at)
api_key
rescue StandardError => _ex
raise ActiveRecord::RecordNotFound, 'provided api key is unknown'
end
def self.authenticate(key)
self.authenticate!(key)
rescue ActiveRecord::RecordNotFound
nil
end
def last_used_at
self[:last_used_at] ||= Time.current
end
end
# frozen_string_literal: true
require 'hkdf'
require 'openssl/digest'
#
# Secure operations on secret tokens of all kinds. Tokens are encoded
# with a textual representation compatible with RFC 8959, which allows
# for automatic detection of secrets committed to public storage.
#
class SecretToken
# The scheme for URI-encoded secret tokens, set by RFC 8959.
SCHEME = 'secret-token'
# The domain associated with these secrets. Changing this will
# invalidate derived subkeys as it's part of the data included in the
# HDKF invocation.
DOMAIN = # insert your authentication domain here
# The hashing algorithm to use when using HKDF to derive subkeys.
# Changing this will invalidate derived subkeys.
ALGORITHM = 'BLAKE2b512'
# The minimum entropy (in bytes) allowed for cryptographic tokens.
MINIMUM_ENTROPY = 128 / 8 # 128 bits
# The scheme to use for the URI-encoded token.
attr_reader :scheme
# The domain to use for the URI-encoded token.
attr_reader :domain
# The context under which this token is valid.
attr_reader :context
# Optionally, an associated username or identifier for the token.
attr_reader :username
#
# Returns a randomly-generated secure token with the given bits of
# `entropy` for a specific `context` and `username`.
#
def self.generate(entropy, context, username: nil)
self.from_bytes(
SecureRandom.bytes(entropy),
context,
username: username,
)
end
#
# Returns a regex to match tokens of the given entropy and either any
# supported `context` or a speficic, fixed `context`.
#
def self.regexp(entropy, context = nil, scheme: SCHEME, domain: DOMAIN)
%r{
\A
(?<scheme> #{Regexp.escape(scheme)} ) ://
(?<username> [a-zA-Z0-9\-_.@]* ) :
(?<token> [a-f0-9]{#{Regexp.escape((entropy * 2).to_s)}} ) @
(?<domain> #{Regexp.escape(domain)} ) /
(?<context> #{context ? Regexp.escape(context) : "[a-zA-Z0-9\\-._/]+"} )
\z
}x
end
#
# Instantiates a token from any supported non-ambiguous format.
#
def self.[](token)
return token if
token.is_a?(self)
uri = Addressable::URI.parse(token)
self.from_hex(
uri.password,
uri.path.delete_prefix('/'),
scheme: uri.scheme,
domain: uri.host,
username: uri.user,
)
end
def self.from_bytes(bytes, context, username: nil, scheme: SCHEME, domain: DOMAIN)
self.new(
bytes,
context,
scheme: scheme,
domain: domain,
username: username,
)
end
#
# Instantiates a token from hex-encoded cryptographic bytes, given a
# scheme, domain, context, and username.
#
def self.from_hex(hex, context, username: nil, scheme: SCHEME, domain: DOMAIN)
raise ArgumentError, 'token must be lowercase hex-encoded bytes' unless
hex.match?(%r{ \A (?: [a-f0-9]{2} )+ \z }x)
self.new(
[hex].pack('H*'),
context,
scheme: scheme,
domain: domain,
username: username,
)
end
class << self
protected :new
end
#
# Instantiates a token from parts.
#
def initialize(token, context, scheme:, domain:, username:)
raise ArgumentError, 'refusing to encode a token with fewer than 128 bits of entropy' if
token.bytesize < MINIMUM_ENTROPY
raise ArgumentError, 'context must be present' if
context.blank?
@scheme = scheme.freeze
@domain = domain.freeze
@context = context.freeze
@username = username.freeze
@token = token.freeze
self.freeze
# sanity-check that we actually match the format we've committed to
raise ArgumentError, 'be less creative with your context and username' unless
self.class.regexp(self.length, scheme: scheme, domain: domain).match(self.to_s)
end
#
# Generates an authenticator from a master key for the provided token.
# Calling `authenticate!` on a token and authentictator will return
# `true` if the token was the original input that generated the
# authenticator and will raise otherwise.
#
# ```ruby
# master = SecretToken.generate(64, 'api_key_root')
# api_key1 = SecretToken.generate(32, 'api_key')
# api_key2 = SecretToken.generate(32, 'api_key')
#
# authenticator = master.authenticator(api_key1)
#
# master.authenticate!(api_key1, authenticator) # => true
# master.authenticate!(api_key2, authenticator) # => ArgumentError
# ```
#
def authenticator(token, context = "#{token.context}/authenticator")
raise ArgumentError, 'token must be from the same scheme and domain' unless
self.related?(token)
# SECURITY: the full token is passed in as the HKDF salt so the
# authenticator is tightly bound to all of the token's metadata and
# not only the cryptographic bytes within
self.subkey token.length, context,
username: token.username,
salt: token.to_s
end
#
# Verifies that `token` was originally used to generate
# `authenticator` (given this master key). Returns true if so,
# otherwise raises an `ArgumentError`.
#
def authenticate!(token, authenticator)
# SECURITY: we use the computed authenticator as the LHS of the
# comparison to ensure that we use constant-time comparison
raise ArgumentError, 'provided token could not be authenticated' unless
self.authenticator(token, authenticator.context) == authenticator
true
end
#
# Derives a subkey from this key with the requested amount of
# `entropy`. Given the same inputs an identical subkey will be
# derived. A random `salt` may be provided to generate unique subkeys.
#
def subkey(entropy, context, username: self.username, salt: nil)
hdkf_salt = self.hkdf_salt
hkdf_info = self.hkdf_canonicalize(context, username, salt)
# SECURITY: use the full representation of self as the IKM to ensure
# the output subkey is tightly bound to all of the token's metadata
# and not only the cyrptographic bytes within
token = HKDF.new(self.to_s,
algorithm: ALGORITHM,
info: hdkf_info,
salt: hkdf_salt,
).read(entropy)
self.class.from_bytes(
token,
context,
scheme: self.scheme,
domain: self.domain,
username: username,
)
end
#
# Securely compares two tokens for equality.
#
def ==(other)
ActiveSupport::SecurityUtils.secure_compare(self.to_s, other.to_s)
end
#
# Returns true if two tokens are in the same scheme and domain.
#
def related?(other)
self.scheme == other.scheme && self.domain == other.domain
end
#
# Returns true if two tokens are in the same scheme, domain, and context.
#
def sibling?(other)
self.related?(other) && self.context == other.context
end
#
# Returns the bytes of entropy contained within the token.
#
def length
@token.length
end
#
# Renders the token with the cryptographic bytes removed.
#
def inspect
self.to_uri.tap { |uri| uri.password = '[REDACTED]' }.to_s.inspect
end
#
# Returns just the cryptographic bytes contained within the token.
#
def to_bytes
@token
end
#
# Returns just the cryptographic bytes contained within the token,
# encoded as hex.
#
def to_hex
@token.unpack1('H*')
end
#
# Renders the token in its canonical URI format as a string.
#
def to_s
self.to_uri.to_s
end
#
# Renders the raw cryptographic bytes contained within the token. This
# is so that tools like `OpenSSL` which may expect this to simply be
# raw bytes can work directly with tokens.
#
def to_str
self.to_bytes
end
#
# Renders the token in its canonical URI format. The host component of
# the URI is the token's domain, the path is its context, and the
# userinfo are the username and hex-encoded cryptographic bytes,
# respectively.
#
def to_uri
Addressable::URI.new(
scheme: self.scheme,
user: self.username,
password: self.to_hex,
host: self.domain,
path: self.context,
)
end
protected
#
# Produces an HDKF salt value. This is *not* the same as the `salt`
# parameter to the `subkey` function, which should be a unique random
# value each invocation. The HDKF `salt` parameter must be a fixed
# random value, so we use the `scheme` and `domain` for domain
# separation.
#
# See: https://soatok.blog/2021/11/17/understanding-hkdf/
#
def hkdf_salt
OpenSSL::Digest.digest(
ALGORITHM,
self.hkdf_canonicalize(self.scheme, self.domain)
)
end
#
# Canonicalizes multiple `part` strings into an unambiguously-encoded
# length-prefixed format to avoid canonicalization attacks. This
# guarantees that parts like "context" and "username" are encoded
# differently than "contex" and "tusername".
#
def hkdf_canonicalize(*parts)
format = %(Q>A*) * parts.length
parts
.map(&:to_s)
.map { |s| [s.length, s] }
.inject(&:+)
.pack(format)
end
end
# frozen_string_literal: true
def derive_secret(master_name, name, entropy, **opts)
app = Rails.application
# secret_key_base needs to be put into `self.secrets` before we
# can derive secrets from it
@_derive_secret_initialized ||=
(app.secrets.secret_key_base = ::SecretToken.from_hex(app.secret_key_base, 'secrets/secret_key_base'))
raise ArgumentError if
app.secrets.key?(name.to_sym)
master = app.secrets.send(:"#{master_name}!")
subkey = master.subkey(entropy, "secrets/#{name}", **opts)
app.secrets[name.to_sym] = subkey
end
Rails.application.reloader.to_prepare do
self.derive_secret(:secret_key_base, :api_key_hkdf_master, 32)
end
class CreateApiKeys < ActiveRecord::Migration[6.1]
def change
create_table :api_keys, id: :uuid do |t|
t.references :account,
type: :uuid,
null: false,
index: true,
foreign_key: true
t.string :key_id, null: false, index: { unique: true }
t.string :authenticator, null: false, index: { unique: true }
t.datetime :last_used_at, null: false
t.timestamps null: false, index: true
t.index %i[account_id last_used_at]
end
end
end
# frozen_string_literal: true
class ActiveRecord::Type::SecretToken < ActiveRecord::Type::Value
def type
:secret_token
end
def cast_value(value)
SecretToken[value]
end
def serialize(value)
cast(value)&.to_s
end
def changed_in_place?(raw_old_value, new_value)
raw_old_value != serialize(new_value)
end
end
ActiveRecord::Type.register(:secret_token, ActiveRecord::Type::SecretToken)
Copyright © 2021 Stephen Touset <stephen@touset.org>
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the “Software”), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
# frozen_string_literal: true
# == Schema Information
#
# Table name: accounts
#
# id :uuid not null, primary key
# created_at :datetime not null
# updated_at :datetime not null
#
require 'test_helper'
class AccountTest < ActiveSupport::TestCase
test '::authenticate_by_api_key! returns the user who owns the api key' do
token = accounts(:stouset).api_keys.generate!
account = Account.authenticate_by_api_key!(token)
assert_equal accounts(:stouset), account
end
test '::authenticate_by_api_key! allows multiple API keys per account' do
account = accounts(:stouset)
keys = [
account.api_keys.generate!,
account.api_keys.generate!,
account.api_keys.generate!,
]
keys.each do |key|
assert_equal account, Account.authenticate_by_api_key!(key)
end
end
test '::authenticate_by_api_key! raises if the key is incorrect' do
assert_raises(ActiveRecord::RecordNotFound) do
Account.authenticate_by_api_key!('fake')
end
end
end
# frozen_string_literal: true
# == Schema Information
#
# Table name: api_keys
#
# id :uuid not null, primary key
# account_id :uuid not null
# key_id :string not null
# authenticator :string not null
# last_used_at :datetime not null
# created_at :datetime not null
# updated_at :datetime not null
#
require 'test_helper'
class ApiKeyTest < ActiveSupport::TestCase
test '::generate creates new, random API keys' do
token1 = accounts(:stouset).api_keys.generate!
token2 = accounts(:stouset).api_keys.generate!
refute_equal token1, token2
end
test '::generate creates hashed keys that are different from their stored value' do
token = accounts(:stouset).api_keys.generate!
key = ApiKey.authenticate!(token)
refute_equal token, key.authenticator
end
test '::authenticate! raises when no matching key is found' do
assert_raises(ActiveRecord::RecordNotFound) { ApiKey.authenticate!('abcd') }
end
test '::authenticate! touches #last_used_at' do
token = accounts(:stouset).api_keys.generate!
Timecop.freeze(1.week.from_now) do
assert_equal Time.current, ApiKey.authenticate!(token).last_used_at
end
end
test '::authenticate! pulls the correct account for a key' do
stouset = accounts(:stouset)
other = Account.create!
stouset_key = stouset.api_keys.generate!
other_key = other.api_keys.generate!
assert_equal stouset, ApiKey.authenticate!(stouset_key).account
assert_equal other, ApiKey.authenticate!(other_key).account
end
end
# frozen_string_literal: true
require 'test_helper'
class SecretTokenTest < ActiveSupport::TestCase
def master_key
@_master_key ||= SecretToken.generate(64, self.context, username: self.username)
end
def context
@_context ||= self.gen_context
end
def username
@_username ||= self.gen_username
end
def gen_hex(entropy = 16)
SecureRandom.hex(entropy)
end
def gen_bytes(entropy = 16)
SecureRandom.bytes(entropy)
end
def gen_context
Faker::Internet.slug
end
def gen_username
Faker::Internet.username
end
def generate(
entropy,
context = self.gen_context,
username: self.gen_username
)
SecretToken.generate(entropy, context, username: username)
end
test 'cannot be instantiated with less than 128 bits of entropy' do
assert_raises(ArgumentError) { self.generate(15) }
assert_raises(ArgumentError) { SecretToken.from_bytes self.gen_bytes(15), 'ctx' }
assert_raises(ArgumentError) { SecretToken.from_hex self.gen_hex(15), 'ctx' }
end
test 'cannot be instantiated with invalid hex' do
assert_raises(ArgumentError) { SecretToken.from_hex('af01bg', 'ctx') }
assert_raises(ArgumentError) { SecretToken.from_hex('af01b', 'ctx') }
end
test 'cannot be instantiated with an empty context' do
assert_raises(ArgumentError) { self.generate(16, '') }
assert_raises(ArgumentError) { self.generate(16, nil) }
assert_raises(ArgumentError) { SecretToken.from_bytes self.gen_bytes, '' }
assert_raises(ArgumentError) { SecretToken.from_bytes self.gen_bytes, nil }
assert_raises(ArgumentError) { SecretToken.from_hex self.gen_hex, '' }
assert_raises(ArgumentError) { SecretToken.from_hex self.gen_hex, nil }
end
test '::generate creates tokens of the given length' do
token1 = self.generate(16)
token2 = self.generate(16)
token3 = self.generate(32)
token4 = self.generate(64)
assert_equal 16, token1.length
assert_equal 16, token2.length
assert_equal 32, token3.length
assert_equal 64, token4.length
end
test '::generate creates unique tokens with identical inputs' do
tokens = Array.new(32) { self.generate(16, self.context, username: self.username) }
copies = tokens.dup
tokens.each do |token|
copies.shift
copies.each { |other| refute_equal(token, other) }
end
end
test '::generate creates tokens with the provided attributes' do
token = self.generate(128, self.context, username: self.username)
assert_equal self.context, token.context
assert_equal self.username, token.username
end
test '#authenticator creates a token dervied from the provided token' do
token = self.generate(16)
authenticator = self.master_key.authenticator(token)
assert_equal token.length, authenticator.length
assert_equal token.username, authenticator.username
assert_equal token.context, authenticator.context.delete_suffix('/authenticator')
refute_equal token.to_bytes, authenticator.to_bytes
end
test '#authenticator returns the same authenticator given identical inputs' do
token = self.generate(32)
assert_equal(
self.master_key.authenticator(token),
self.master_key.authenticator(token),
)
end
test '#authenticator raises if the master key and token are from unrelated domains' do
domain = Faker::Internet.domain_name
token = self.generate(16)
master_key = SecretToken.from_bytes(self.gen_bytes, self.gen_context, domain: domain)
assert_raises(StandardError) { master_key.authenticator(token) }
end
test '#authenticate! returns true if the authenticator matches the token' do
token = self.generate(16)
authenticator = self.master_key.authenticator(token)
assert self.master_key.authenticate!(token, authenticator)
end
test '#authenticate! raises if the authenticator does not match the token' do
token1 = self.generate(16, self.context, username: self.username)
token2 = self.generate(16, self.context, username: self.username)
token3 = SecretToken.from_bytes(token1.to_bytes, self.gen_context, username: self.username)
token4 = SecretToken.from_bytes(token1.to_bytes, self.context, username: self.gen_username)
authenticator = self.master_key.authenticator(token1)
assert_raises(ArgumentError) { self.master_key.authenticate!(token2, authenticator) }
assert_raises(ArgumentError) { self.master_key.authenticate!(token3, authenticator) }
assert_raises(ArgumentError) { self.master_key.authenticate!(token4, authenticator) }
end
test '#authenticate! raises if the master key, token, and authenticator are provided out-of-order' do
token = self.generate(16)
authenticator = self.master_key.authenticator(token)
assert_raises(ArgumentError) { self.master_key.authenticate!(authenticator, token) }
assert_raises(ArgumentError) { token.authenticate!(master_key, authenticator) }
assert_raises(ArgumentError) { token.authenticate!(authenticator, master_key) }
assert_raises(ArgumentError) { authenticator.authenticate!(master_key, token) }
assert_raises(ArgumentError) { authenticator.authenticate!(token, master_key) }
end
test '#subkey creates identical keys for identical inputs' do
key1 = self.master_key.subkey(32, self.context, username: self.username, salt: 'salt')
key2 = self.master_key.subkey(32, self.context, username: self.username, salt: 'salt')
assert_equal key1, key2
end
test '#subkey creates different keys for different contexts' do
key1 = self.master_key.subkey(32, self.gen_context)
key2 = self.master_key.subkey(32, self.gen_context)
refute_equal key1, key2
end
test '#subkey creates different keys for different usernames' do
key1 = self.master_key.subkey(32, self.context, username: self.gen_username)
key2 = self.master_key.subkey(32, self.context, username: self.gen_username)
refute_equal key1, key2
end
test '#subkey creates different keys from different salts' do
key1 = self.master_key.subkey(32, self.context, username: self.username, salt: self.gen_hex)
key2 = self.master_key.subkey(32, self.context, username: self.username, salt: self.gen_hex)
refute_equal key1, key2
end
test '#inspect redacts private key material' do
token = self.generate(32)
inspect = token.inspect
secret = inspect.scan(%r{ [\w\-\[\]_.]+ }x)[2]
assert_equal '[REDACTED]', secret
end
end
@stouset
Copy link
Author

stouset commented Apr 16, 2021

I hereby license this under the MIT license. I make no assertions as to its fitness for any purpose. While I am reasonably confident that there are no glaring security errors, I make no guarantees to that effect.

@stouset
Copy link
Author

stouset commented Apr 16, 2021

Examples of use:

$ ./bin/rails c
Loading development environment (Rails 6.1.3.1)
[1] [app][development] pry(main)> account = Account.create!
=> #<Account:0x00007ff409274408 id: "775cbf1c-9efb-11eb-9bb3-77052e986e8f", created_at: Fri, 16 Apr 2021 21:34:04.118969000 UTC +00:00, updated_at: Fri, 16 Apr 2021 21:34:04.118969000 UTC +00:00>
[2] [app][development] pry(main)> api_key = a.api_keys.generate!
=> "secret-token://306c973e-2e5c-421a-83d2-0f2b99f9ec70:[REDACTED]@example.com/api_key"
[3] [app][development] pry(main)> account.api_keys.first.authenticator
=> "secret-token://306c973e-2e5c-421a-83d2-0f2b99f9ec70:[REDACTED]@example.com/api_key/authenticator"
[4] [app][development] pry(main)> Account.authenticate_by_api_key!(api_key)
=> #<Account:0x00007ff3d80d3080 id: "775cbf1c-9efb-11eb-9bb3-77052e986e8f", created_at: Fri, 16 Apr 2021 21:34:04.118969000 UTC +00:00, updated_at: Fri, 16 Apr 2021 21:34:04.118969000 UTC +00:00>

@azdrenymeri
Copy link

Hello @stouset,
Huge thanks for providing these code snippets.
Could you possibly update the next_bytes method to read on line #201 since the latest version of hkdf has renamed it

Commit: jtdowney/hkdf@4e9576e

@stouset
Copy link
Author

stouset commented Jan 26, 2022

I have updated the snippet for this. I have also improved the invocation of HKDF to be more conformant with the intended use of its input parameters. This mostly is of theoretical value, but it's important all the same.

This will cause existing subkeys and password authenticators which used the prior HDKF scheme to become invalid. They will need to be regenerated. If you cannot do so, I can't personally recommend staying on the prior implementation but that is also an option.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment