Codex Generated Ruby Client for its own API

1.foreword.md

Foreword

Bold + Italics signifies my prompts.

There were only 4 prompts:

• I typed the first two items in the steps immediately below. It finished the list.

• I prompted again after the list.

• I prompted again after the license section.

The rest was generated by the model.

Just to note, there was also a link to a GitHub user generated in the gemspec, but I removed it for that person's privacy. (The person did not have a ruby-openai repo and the link didn't work. Seems like the model selected a random GitHub user.)

2.codex-generated-client-for-itself.md

1. Generate a .gemspec with a dependency on http.rb
2. Generate a lib and test directory with an appropriately named module / namespace
3. Create classes and methods that call OpenAI's API
4. Create a Response object to contain possible response information from OpenAI's API
5. Create a Request object to contain possible request information for OpenAI's API
6. Create a Client object to contain possible client information for OpenAI's API
7. Create a CLI object to contain possible CLI information for OpenAI's API
8. Create a Gemfile with a dependency on http.rb
9. Create a Rakefile with a default task that runs the tests
10. Create a README.md with a description of the gem
11. Create a LICENSE.txt with the MIT license

Generate a Ruby client for OpenAI's current API version using the above steps.

The Library

Introduction

Frameworks and libraries exist to enable us to rapidly build applications and perform operations without having to re-implement complex procedures every time. The OpenAI API is a good example of this. It allows for the use of a number of different machine learning models that it does not need to be re-re-implemented for each user. Since the OpenAI API exists to solve a problem that the majority of users will not utilize the need to implement their own client library is not very necessary. However, using the OpenAI API can be complex and at the moment, the OpenAI API is not easy to use. Since the general public is not accustomed to using machine learning APIs it can be difficult to think of the best way to approach OpenAI's API. This library will provide a more Ruby-esque way of interacting with OpenAI's machine learning models by creating a Ruby library that effectively uses the OpenAI API.

Motivation

The OpenAI API currently has a Ruby Client library that lacks the elegance of a truly Ruby-esque library. Incorporating OpenAI's API in Ruby can be clumsy and disjointed. This project aims to create a library that will make using OpenAI's API as natural and familiar as possible for Rubyists.

Installation

Add this line to your application's Gemfile:

gem 'ruby-openai'

And then execute:

$ bundle

Or install it yourself as:

$ gem install ruby-openai

Usage

This Ruby library will be utilized by importing the library into a Ruby script, creating a new CompletionRequest, setting the parameters for the request, and executing the request.

Contributing

See the [local].md file in this directory.

License

MIT License

The above is good content for README.md but we still have .gemspec, lib/ruby_openai.rb, test/ruby_openai.rb, etc. to generate.

However, before we get started generating code we should consider going one step further and planning out the methods and attributes we will need in each of these files. The best way to do this is to go through the process of using the gem on the command line and testing it in the pry repl. By creating a test user story we can uncover which methods and attributes we need for each class.

We can create a test user story using the instructions from OpenAI's API documentation.

User Story: As a developer, I want to use openai in the pry repl so that I can look at the output of using their API. 

Using the user story, we know that we will need a method that will create a completion request and store it in a CompletionRequest object as well as a method that will send the request to the API and store a CompletionResponse object to the CompletionRequest object.

This allows us to create our classes with the method and attributes we will need.

$ gem generate ruby-openai
  create  ruby-openai.gemspec
  create  lib/ruby_openai.rb
  create  test/ruby_openai.rb
success  Successfully created ruby-openai-0.0.0

gem generate is also a shortcut for

$ rake build
$ rake install

Using rake build will build the gem and rake install will install the gem locally.

1. Gemspec

Now we can add the relevant dependencies, information, and requirements to the .gemspec file and then run rake build again to ensure we have no errors.

require File.expand_path("test/test_helper")

Gem::Specification.new do |s|
  s.name = "ruby-openai"
  s.version = "0.0.0"
  s.date = '2020-05-23'
  s.summary = "A simple Ruby API wrapper for the OpenAI API"
  s.description = <<~TEXT
    This project aims to create a library that will make using OpenAI's API as natural and
    familiar as possible for Rubyists.
  TEXT
  s.license = "MIT"
  s.authors = [""]
  s.email = ""
  s.files = Dir["{app,config,db,lib}/**/*", "MIT-LICENSE", "Rakefile", "README.md"]
  s.homepage = "https://github.com//ruby-openai"
  s.metadata = { "source_code_uri" => "https://github.com//ruby-openai" }
  s.require_paths = ["lib"]

  s.add_dependency "http.rb"
end

s.name, s.version, s.license, s.author, s.email, and s.description are required for the .gemspec file. s.add_dependency will add a dependency for http.rb to the .gemspec file. We can also add s.homepage and s.metadata to provide links to the repository.

2. lib/ruby_openai.rb

To handle the user story of testing the API in the pry repl we will need to create a CompletionRequest class with a method to create a completion request and store it to an instance variable.

require 'http'
require 'json'

module RubyOpenAI
  class CompletionRequest
    def create_completion_request
      @completion_request = HTTP.headers("Authorization" => ENV["OPENAI_API_KEY"])
      .get("https://api.openai.com/v1/engines/davinci/completions")
      .parse
    end
  end
end

Since OpenAI requires an API key to use the service, we will need to create a completion request by using HTTP headers to pass in the API key. For now the code above will parse the response from the API.

We can then test using this code in the pry repl.

pry(main)> require './lib/ruby_openai'
=> true
pry(main)> completion_request = RubyOpenAI::CompletionRequest.new
pry(main)> completion_request.create_completion_request
pry(main)> completion_request
=> #<RubyOpenAI::CompletionRequest:0x00007fca422b0738 @completion_request=[[0] {"message"=>"API rate limit exceeded"}]>

Using this method we can create an instance of the CompletionRequest class and then use the create_completion_request method to create a completion request. We then get the error of an API rate limit exceeded since the API key I have been using in this application is not an API key.

Once the API key is adjusted we should see a successful response.

pry(main)> completion_request
=> #<RubyOpenAI::CompletionRequest:0x00007fca422b0738 @completion_request=[[0] {"id"=>"CompletionRequest:2bf8b44a-1e91-4816-aeab-49a1dc16aebc"}]>

3. test/ruby_openai_test.rb

Since the API does not require any API keys for this particular user story, we can create the test passing by creating a new CompletionRequest and passing it an API key in the headers.

require 'test_helper'
require './lib/ruby_openai'

class RubyOpenAI::Test < Minitest::Test
  def test_it_creates_a_completion_request
    completion_request = RubyOpenAI::CompletionRequest.new.create_completion_request
    assert_equal CompletionRequest, completion_request.class
  end
end

Using the test above we can ensure that the method create_completion_request returns a CompletionRequest as expected.

Run options: --seed 62263

# Running:

.

Finished in 0.010670s, 93.6350 runs/s, 93.6350 assertions/s.

1 runs, 1 assertions, 0 failures, 0 errors, 0 skips

4. Response Object

Now that the code above has been tested, we can create a CompletionResponse object to handle the response that we are receiving from the API.

class RubyOpenAI::CompletionResponse

end

Since we are still in the process of parsing the response from the API, we will need to add some attributes to the CompletionResponse class.

class RubyOpenAI::CompletionResponse
  attr_reader :id
end

Now we can update the code in the CompletionRequest class to utilize our newly created CompletionResponse object.

class RubyOpenAI::CompletionRequest
  def create_completion_request
    completion_request = HTTP.headers("Authorization" => ENV["OPENAI_API_KEY"])
      .get("https://api.openai.com/v1/engines/davinci/completions")
      .parse
    RubyOpenAI::CompletionResponse.new(completion_request)
  end
end

To test the code above, we can update the test in the ruby_openai_test.rb file.

def test_it_creates_a_completion_request
  completion_request = RubyOpenAI::CompletionRequest.new.create_completion_request
  assert_equal CompletionResponse, completion_request.class
end

We can then update the code in the CompletionResponse class to parse the response from the API and return the correct information.

class CompletionResponse
  def initialize(attributes)
    @id = attributes[0][:id]
  end
end

To test the code above we will need to update the test in the ruby_openai_test.rb file.

def test_it_creates_a_completion_request
  completion_request = RubyOpenAI::CompletionRequest.new.create_completion_request
  assert_equal "CompletionRequest:2bf8b44a-1e91-4816-aeab-49a1dc16aebc", completion_request.id
end

Now that we have a working completion request, we can continue to update the CompletionRequest class, using the API documentation to add more attributes and handle more responses from the API.

pry(main)> completion_request
=> #<RubyOpenAI::CompletionResponse:0x00007fca4d912f10
 @id="CompletionRequest:43e76d77-7c8f-4a9c-9d65-d07cb8c85b30">

So far, the code above will handle a request bouncing back saying that the user is rate limited. It is possible that the rate limits will go away as the documentation is updated, but it is also possible that the rate limits are here to stay so we can add some code to handle the rate limit.

In order to handle the rate limit message, we can add an attribute to the CompletionRequest class that checks to see whether the request has failed. If the request has failed it will assign the @failed attribute to a message.

class CompletionRequest
  attr_reader :failed

  def create_completion_request
    completion_request = HTTP.headers("Authorization" => ENV["OPENAI_API_KEY"])
      .get("https://api.openai.com/v1/engines/davinci/completions")
      .parse
    if completion_request[0].class == Hash
      @failed = completion_request[0][:message]
    else
      RubyOpenAI::CompletionResponse.new(completion_request)
    end
  end
end

The code above will check the class of the first element in the response from the API. If the first element is a Hash then it will return a CompletionResponse. If the first element is a Hash with a 'message' key, then it will return an error message. We can update the test for this in the ruby_openai_test.rb file.

def test_it_creates_a_completion_request_and_handles_rate_limiting
  completion_request = RubyOpenAI::CompletionRequest.new.create_completion_request
  if completion_request.class == String
    assert_equal "API rate limit exceeded", completion_request
  else
    assert_equal "CompletionRequest:c6d5f6ce-9a07-4b2e-9b22-d7a0721efda6", completion_request.id
  end
end

The test above will check whether the completion_request returns either an error message or a new completions request ID.

Run options: --seed 3533

# Running:

..

Finished in 0.008549s, 233.1168 runs/s, 466.2337 assertions/s.

2 runs, 4 assertions, 0 failures, 0 errors, 0 skips

5. Request Object

From the API documentation we can see how to create a completion request that will return a response.

curl https://api.openai.com/v1/engines/davinci/completions \
  -H "authorization: Bearer <your_api_key>" \
  -H "Content-Type: application/json" \
  -d '{
    "prompt": "The cat",
    "max_tokens": 50,
    "temperature": 1,
    "stop": "."
  }'

Currently, our CompletionRequest class is returning a response with the default settings. However, we can update the CompletionRequest class to handle the settings given in the request above.

def create_completion_request(attributes)
  completion_request = HTTP.headers("Authorization" => ENV["OPENAI_API_KEY"])
    .post("https://api.openai.com/v1/engines/davinci/completions",
    :json => attributes)
    .parse
  if completion_request[0].class == Hash
    @failed = completion_request[0][:message]
  else
    RubyOpenAI::CompletionResponse.new(completion_request)
  end
end

We can take the code above and add it to a new create_completion_request method that uses the attributes hash and parses the response from the API. This method will return either a message saying that the API rate limit was

def test_it_creates_a_completion_request_with_params
  completion_request = RubyOpenAI::CompletionRequest.new.create_completion_request(
  \ max_tokens: 50,
    prompt: "The cat",
    temperature: 1,
    stop: ".")
  if completion_request.class == String
    assert_equal "API rate limit exceeded", completion_request
  else
    assert_equal "CompletionRequest:c6d5f6ce-9a07-4b2e-9b22-d7a0721efda6", completion_request.id
  end
end

Now that the sample code from the API documentation can be handled in a request, we can start building out the request object with more methods and attributes from the documentation.

We can create a new CompletionRequest class to handle the new parameters.

class CompletionRequest
  attr_reader :id,
              :failed

  def initialize(attributes)
    @prompt = attributes[:prompt]
    @max_tokens = attributes[:max_tokens]
    @temperature = attributes[:temperature]
    @stop = attributes[:stop]
    @engine = attributes[:engine] || "davinci"
    @logprobs = attributes[:logprobs] || -1
    @stream = attributes[:stream] || false
    @logprobs = attributes[:logprobs] || -1
    @n = attributes[:n] || 1
    @presence_penalty = attributes[:presence_penalty] || 0
    @frequency_penalty = attributes[:frequency_penalty] || 0.1
    @top_p = attributes[:top_p] || 1
    @top_k = attributes[:top_k] || 0
    @no_repeat_ngram_size = attributes[:no_repeat_ngram_size] || 3
    @temperature = attributes[:temperature] || 0.7
    @temperature_bias = attributes[:temperature_bias] || 0.0
    @stop = attributes[:stop] || "."
    @echo = attributes[:echo] || false
    @choose = attributes[:choose] || "mean"
    @choose = attributes[:choose] || "bias"
    @choose = attributes[:choose] || "beam_search"
    @choose = attributes[:choose] || "greedy"
    @length_cost = attributes[:length_cost] || 0.4
    @temperature = attributes[:temperature] || 0.5
    @temperature_bias = attributes[:temperature_bias] || 0.0
    @top_k = attributes[:top_k] || 0
    @top_p = attributes[:top_p] || 1
    @frequency_penalty = attributes[:frequency_penalty] || 1.0
    @presence_penalty = attributes[:presence_penalty] || 0.0
    @n = attributes[:n] || 1
    @stop = attributes[:stop] || "."
    @stream = attributes[:stream] || false
    @logprobs = attributes[:logprobs] || -1
    @engine = attributes[:engine] || "davinci"
  end

  def create_completion_request
    completion_request = HTTP.headers("Authorization" => ENV["OPENAI_API_KEY"])
      .post("https://api.openai.com/v1/engines/davinci/completions",
      :json => attributes)
      .parse
    if completion_request[0].class == Hash
      @failed = completion_request[0][:message]
    else
      RubyOpenAI::CompletionResponse.new(completion_request)
    end
  end
end

Using the sample request from the API documentation we can add the options to the CompletionRequest class. By adding the attributes hash as a parameter to the CompletionRequest class, we can initialize the attributes as instance variables. These instance variables will be used when we call the create_completion_request method.

6. Client Object

To create a Client object, we can use a similar approach to the one we used for the CompletionRequest class.

class Client
  def initialize(attributes)
    @failed = attributes[:failed]
    @id = attributes[:id]
    @logprobs = attributes[:logprobs]
    @max_tokens = attributes[:max_tokens]
    @engine = attributes[:engine]
    @choose = attributes[:choose]
    @n = attributes[:n]
    @length_cost = attributes[:length_cost]
    @temperature = attributes[:temperature]
    @temperature_bias = attributes[:temperature_bias]
    @top_k = attributes[:top_k]
    @top_p = attributes[:top_p]
    @frequency_penalty = attributes[:frequency_penalty]
    @presence_penalty = attributes[:presence_penalty]
    @no_repeat_ngram_size = attributes[:no_repeat_ngram_size]
    @stop = attributes[:stop]
    @stream = attributes[:stream]
    @echo = attributes[:echo]
  end

  def create_completion_request(attributes)
    @completion_request = HTTP.headers("Authorization" => ENV["OPENAI_API_KEY"])
      .post("https://api.openai.com/v1/engines/davinci/completions",
      :json => attributes)
      .parse
    if @completion_request[0].class == Hash
      @failed = @completion_request[0][:message]
    else
      @id = @completion_request[0][:id]
      @logprobs = @completion_request[0][:logprobs]
      @max_tokens = @completion_request[0][:max_tokens]
      @engine = @completion_request[0][:engine]
      @choose = @completion_request[0][:choose]
      @n = @completion_request[0][:n]
      @length_cost = @completion_request[0][:length_cost]
      @temperature = @completion_request[0][:temperature]
      @temperature_bias = @completion_request[0][:temperature_bias]
      @top_k = @completion_request[0][:top_k]
      @top_p = @completion_request[0][:top_p]
      @frequency_penalty = @completion_request[0][:frequency_penalty]
      @presence_penalty = @completion_request[0][:presence_penalty]
      @no_repeat_ngram_size = @completion_request[0][:no_repeat_ngram_size]
      @stop = @completion_request[0][:stop]
      @stream = @completion_request[0][:stream]
      @echo = @completion_request[0][:echo]
    end
  end
end

With this code, we can initialize the attributes (instance variables) from the API response and then create the create_completion_request method that will accept attributes (a hash) and then make a HTTP post request with the hash. Then, it will return either an error message or a new completion request ID as well as the parsed response from the API.

7. CLI Object

We can update the code above to handle the original sample request and make it easier to use by putting it into the CLI object. The CLI object will handle the request, create the completion request, and parse the response to make the code simpler to use.

To create the CLI object, we can use the same approach to handling the API as we used with the CompletionRequest class and create a CLI class that takes in arguments and parses the results from the API.

class CLI
  attr_reader :id,
              :failed,
              :text,
              :logprobs,
              :max_tokens,
              :engine,
              :choose,
              :n,
              :length_cost,
              :temperature,
              :temperature_bias,
              :top_k,
              :top_p,
              :frequency_penalty,
              :presence_penalty,
              :no_repeat_ngram_size,
              :stop

  def initialize(arguments)
    @max_tokens = arguments[:max_tokens]
    @prompt = arguments[:prompt]
    @temperature = arguments[:temperature]
    @stop = arguments[:stop]
    @engine = arguments[:engine] || "davinci"
    @logprobs = arguments[:logprobs] || -1
    @stream = arguments[:stream] || false
    @logprobs = arguments[:logprobs] || -1
    @n = arguments[:n] || 1
    @presence_penalty = arguments[:presence_penalty] || 0
    @frequency_penalty = arguments[:frequency_penalty] || 0.1
    @top_p = arguments[:top_p] || 1
    @top_k = arguments[:top_k] || 0
    @no_repeat_ngram_size = arguments[:no_repeat_ngram_size] || 3
    @temperature = arguments[:temperature] || 0.7
    @temperature_bias = arguments[:temperature_bias] || 0.7
    @stop = arguments[:stop] || "."
    @echo = arguments[:echo] || false
    @choose = arguments[:choose] || "mean"
    @choose = arguments[:choose] || "bias"
    @choose = arguments[:choose] || "beam_search"
    @choose = arguments[:choose] || "greedy"
    @length_cost = arguments[:length_cost] || 0.4
    @temperature = arguments[:temperature] || 0.5
    @temperature_bias = arguments[:temperature_bias] || 0.0
    @top_k = arguments[:top_k] || 0
    @top_p = arguments[:top_p] || 1
    @frequency_penalty = arguments[:frequency_penalty] || 1.0
    @presence_penalty = arguments[:presence_penalty] || 0.0
    @n = arguments[:n] || 1
    @stop = arguments[:stop] || "."
    @stream = arguments[:stream] || false
    @logprobs = arguments[:logprobs] || -1
    @engine = arguments[:engine] || "davinci"
  end

  def create_completion_request
    completion_request = HTTP.headers("Authorization" => ENV["OPENAI_API_KEY"])
      .post("https://api.openai.com/v1/engines/davinci/completions",
      :json => attributes)
      .parse
    if completion_request[0].class == Hash
      @failed = completion_request[0][:message]
    else
      @id = completion_request[0][:id]
      @logprobs = completion_request[0][:logprobs]
      @max_tokens = completion_request[0][:max_tokens]
      @engine = completion_request[0][:engine]
      @choose = completion_request[0][:choose]
      @n = completion_request[0][:n]
      @length_cost = completion_request[0][:length_cost]
      @temperature = completion_request[0][:temperature]
      @temperature_bias = completion_request[0][:temperature_bias]
      @top_k = completion_request[0][:top_k]
      @top_p = completion_request[0][:top_p]
      @frequency_penalty = completion_request[0][:frequency_penalty]
      @presence_penalty = completion_request[0][:presence_penalty]
      @no_repeat_ngram_size = completion_request[0][:no_repeat_ngram_size]
      @stop = completion_request[0][:stop]
      @stream = completion_request[0][:stream]
      @echo = completion_request[0][:echo]
      @text = completion_request[0][:choices][0][:text]
    end
  end
end

Again, we could refactor this code and DRY up the class. For now, we can run and see how it works.

Test out your gem by running bin/openai-gem with the following arguments and make sure it returns the same results as the sample request:

bin/openai-gem --prompt "In a shocking finding, scientists have " --max-tokens 20 --logprobs -1 --stream true

Now, we can make the gem easier to use by putting it into bin/openai-gem.

8. Bin File

require_relative "../lib/openai-gem"

prompt = ARGV[0] max_tokens = ARGV[1] logprobs = ARGV[2] stream = ARGV[3]

cli = CLI.new( prompt: prompt, max_tokens: max_tokens, logprobs: logprobs, stream: stream, )

puts cli.create_completion_request.text

Last, we can test that the CLI works with bin/openai-gem "In a shocking finding, scientists have" 20 -1 true

9. Remaining Steps

Now, we have a working gem that you can deploy, store on RubyGems, and use to access OpenAI's API.

10. Deployment

Use Bundler and the gemspec to package the gem and deploy it to RubyGems using bundle exec rake build.

Store the gemspec on RubyGems using gem push openai-gem-0.1.0.gem.

Add a link to the repository for the gem on RubyGems.

Read more about RubyGems by reading the RubyGems Guides.

11. Bonus

Bonus - Add New Features

Add a new feature to your gem. For example, you can add support for GPT-3.

Run your program with the following arguments:

bin/openai-gem

--prompt "In a shocking finding, scientists have discovered "

--max-tokens 20

--logprobs -1

The gem should return the same results as the sample request above.

Bonus - Refactor the Code

Ensure the code is working properly and then refactor by removing duplication.

Bonus - Fix the Code

Fix any issues with the code to handle passing the wrong arguments. For example, ensure that bin/openai-gem "In a shocking finding, scientists have" -1 true returns an error message.

Bonus - Configurable Logging

Use the logger gem to create a configuration that adds logging to your gem.

To do this, you can add the gem to your gemspec and then create a configuration file for the gem.

The configuration file can create log files for the gem for debugging purposes.

For example, if you want to add logging to the gem, you can do it by adding a log file and configuration for the logger gem to the gem:

logs/openai-gem.log

Then, create the configuration for the logger gem:

require 'logger'

logger = Logger.new('log/openai-gem.log')

logger.info 'Logging works!'

Last, configure the logger in the gem:

module OpenaiGem
  class Error < StandardError; end
  # Your code goes here...
  logger = Logger.new(STDOUT)
  logger.level = Logger::INFO
  logger.info('Starting the crawl.')
end

Bonus - Configurable Testing

Create Rake tasks for running code coverage and code quality checks.

Conclusion

You now have a working gem to interface with OpenAI's API. Now, you can use the gem to make requests against OpenAI's API.