SDKs

api.md

API

Here are the concerns:

• Routes: How do I get to the necessary resource?
• Configuration: What options do I want for my connection or requests
• Connection: How do I connect to the application? Typically this will be HTTP(S).
• Logging: We will have specific logging for the api, the requests, and applications.
• Errors: We will have specific exceptions we raise if any issues arise.
• Requests: These use the routes and connections to make a request to the service. The return from this is a response.
• Response: The response from a request is typically a schema that aligns with content negotiation (Content-Type or Accept)
• Model: This is the schema/model definition for the resource returned from the API. This is typically represented as a collection and instance for the resource.

Example: Sales Transactions

There is a defined API resource of Sales Transactions. According the schema I can also send in query string parameters or a request body to perform more filtering.

Service::Client.configure do |c|
  c.logger = Service::Logger.new(Service::Client.root.join('logs/application.log'))
  c.request_logger = Service::Logger.new(Service::Client.root.join('logs/requests.log'))
end

# Writes to logs/application.log
>> Service::Client.logger.info('List Transactions')
=> true

# Verify connection
>> Service::Client.connection
=> #<Service::Connection>

# Retrieve the route definition (URI Template) for the transactions. Some APIs may return a root dictionary. If they don't, we define them in our own routes file.
>> route = Service::Client.routes.find_by_rel('transactions')
=> #<Service::Route>

>> route.url_for(page: 1, per_page: 20, q: 'search term')
=> "/transactions?page=1&per_page=20&q=search+term"

# Issue a request to retrieve the transactions
>> request = Service::Requests::Transactions.list
=> #<Service::Response>

# Truncated for brevity. There could be many transactions.
>> request.body.to_s
=> "{ "data": [{ "id": "123", "created_date": "2023-02-23", "amount": "23.99"}] }"

# Utilize the extracted schema to build out model. The collection model is an Enumerable. The instance model may be a Comparable (Forwardable, Delegator).
>> transactions = Service::Models::Transactions.new(request.body.to_s)
=> #<Service::Models::Transactions>

>> transactions.count
=> 1

# Retrieve a specific transaction
>> transaction = transactions.retrieve('123')
=> #<Service::Models::Transaction>

# Returns the ID
>> transaction.id
=> '123'

# Returns a casted Date object
>> transaction.created_date
=> 2023-02-23

# Returns a casted RubyUnits::Unit delegated object. This permits us to convert and do math as needed.
>> transaction.amount_unit
=> 23.99 USD

The boundaries and separation of conerns permit us to inspect at each level.

screen_scraping.md

Screen Scraping

Here are the concerns:

• Routes: How do I get to the necessary Page?
• Configuration: What options do I want for my connection or requests
• Connection: How do I connect to the application? Typically this will be HTTP(S).
• Logging: We will have specific logging for the scraper, the requests, and applications.
• Errors: We will have specific exceptions we raise if any issues arise. This is very important for screen scraping as it may detect that a DOM structure has changed and we need to manually review.
• Requests: These use the routes and connections to make a request to the service. The return from this is a response.
• Response: The response from a request is typically an HTML document from an HTTP(S) resource.
• Page: This takes the HTML response and uses a parser (Nokogiri, in Ruby) to query and traverse the DOM as needed. The page may include many resources, so it's not a 1-1 relationship with the resource.
• Model: This is the schema/model definition for the resource(s) extracted from a page. The page model extracts the resources and permits us to create collections and instances of those resources.

Example: Sales Transactions

There may be a resource that lists out all sales transactions. The page itself includes a table (or grid), pagination, search, or other. With this information, we can do the following:

Service::Client.configure do |c|
  c.logger = Service::Logger.new(Service::Client.root.join('logs/application.log'))
  c.request_logger = Service::Logger.new(Service::Client.root.join('logs/requests.log'))
end

# Writes to logs/application.log
>> Service::Client.logger.info('List Transactions')
=> true

# Verify connection
>> Service::Client.connection
=> #<Service::Connection>

# Retrieve the route definition (URI Template) for the transactions
>> route = Service::Client.routes.find_by_rel('transactions')
=> #<Service::Route>

>> route.url_for(page: 1, per_page: 20, q: 'search term')
=> "/transactions?page=1&per_page=20&q=search+term"

# Issue a request to retrieve the transactions
>> request = Service::Requests::Transactions.list
=> #<Service::Response>

>> request.body.to_s
=> "<html><title>Transactions</title>..."

# Parse a page of transactions
>> page = Service::Pages::Transactions.new(request.body.to_s)
=> #<Service::Pages::Transactions>

# Extract the Table. Typically we will adhere to a pattern of {resource}_attributes, which may have a corresponding model
>> page.transactions_attributes
=> [{ id: '123', created_date: "2023-02-23", amount: "$23.99" }]

# Utilize the extracted schema to build out model. The collection model is an Enumerable. The instance model may be a Comparable (Forwardable, Delegator).
>> transactions = Service::Models::Transactions.new(page.transactions_attributes)
=> #<Service::Models::Transactions>

>> transactions.count
=> 1

# Retrieve a specific transaction
>> transaction = transactions.retrieve('123')
=> #<Service::Models::Transaction>

# Returns the ID
>> transaction.id
=> '123'

# Returns a casted Date object
>> transaction.created_date
=> 2023-02-23

# Returns a casted RubyUnits::Unit delegated object. This permits us to convert and do math as needed.
>> transaction.amount_unit
=> 23.99 USD

The boundaries and separation of conerns permit us to inspect at each level.