.description.sbds_tests.txt

sbds_tests - Please have a look at the README.md file.

.gitignore

.rvmrc

README.md

• Title: sbds_tests - Steem Blockchain Data Service Tests
• Tags: radiator ruby steem steemdev sbds
• Notes:

Overview

Here comes tests for sbds (Steem Blockchain Data Service), which is a project that is currently pre-release software and thus, not yet suitable for production use. See:

https://github.com/steemit/sbds

Even though it's just pre-release, it's already a great way to ingest the STEEM blockchain into a local database that any app can utilize.

These tests focus strictly on API access to verify ingestion. Current known routes offered by sbds:

• GET https://sbds.steemitdev.com/health
  • Returns the current state of sbds blockchain ingestion.
• GET https://sbds.steemitdev.com/api/v1/blockchainStats
  • Returns basic statistics.
• POST https://sbds.steemitdev.com/
  • JSON-RPC methods
    • sbds.count_operations
      • Not currently being tested.
    • sbds.get_custom_json_by_tid
      • Returns matching custom_json operations.
      • Accepts custom_json.id, e.g.:"params": {"tid": "follow"}
    • sbds.get_random_operation_block_nums
      • Returns random block_num values associated with the op_type.
    • sbds.get_random_operations
      • Returns random operations associated with the operation type.

sbds.get_random_operations

The most important tests, at the moment, focus on this method. One tests takes all known op_type values and asks sbds for each of them. The random operations that are returned by sbds are then directly compared to the blockchain by querying the API with radiator.

This ensures that sbds a) correctly ingests the operation, and b) correctly reconstructs the operation from the database.

So far, it does this perfectly.

---

Install

To use these Radiator tests:

Linux

$ sudo apt-get install ruby-full git openssl libssl1.0.0 libssl-dev
$ gem install bundler

macOS

$ gem install bundler

I've tested it on various versions of ruby. The oldest one I got it to work was:

ruby 2.0.0p645 (2015-04-13 revision 50299) [x86_64-darwin14.4.0]

First, clone this gist and install the dependencies:

$ git clone git@gist.github.com:86303fa429b6e0ea0cc2ec94a4b451ae.git sbds_tests
$ cd sbds_tests
$ bundle install

---

Upgrade

Typically, you can upgrade to the latest version by this command, from the original directory you cloned into:

$ git pull

Usually, this works fine as long as you haven't modified anything. If you get an error, try this:

$ git stash --all
$ git pull --rebase
$ git stash pop

If you're still having problems, I suggest starting a new clone.

---

Usage

To run the tests:

$ rake

Example output:

Run options: --seed 54197

# Running:

..S.....

Fabulous run in 117.877599s, 0.0679 runs/s, 20.3601 assertions/s.

8 runs, 2400 assertions, 0 failures, 0 errors, 1 skips

You have skipped tests. Run with --verbose for details.

The tests will also look for ENV overrides, for example, this will effectively switch testing targets over to production:

$ BASE_URL=https://sbds.steemit.com NODE=https://steemd.steemit.co rake

And this will test a docker instance running on port 32769:

$ BASE_URL=http://localhost:32769 rake

---

Check here to see an updated version of these tests:

https://gist.github.com/inertia186/86303fa429b6e0ea0cc2ec94a4b451ae

---


See my previous Ruby How To posts in: #radiator #ruby

Get in touch!

If you're using sbds_tests, I'd love to hear from you. Drop me a line and tell me what you think! I'm @inertia on STEEM and SteemSpeak.

License

I don't believe in intellectual "property". If you do, consider sbds_tests as licensed under a Creative Commons License.

blockchain_stats_test.rb

require 'test_helper'

class BlockchainStatsTest < SbdsTest
  def test_basic_get
    response = open("#{BASE_URL}/api/v1/blockchainStats")
    
    refute_nil response, 'did not expect nil response'
    result = JSON[response.read]
    
    %w(post_counts comment_counts account_creates votes labels payments).each do |key|
      assert result[key], "expect #{key}"
      next if key == 'labels'
      
      refute_equal 0, result[key], "did not expect zero for #{key}; regression on https://github.com/steemit/sbds/issues/53"
    end
  end
end

Gemfile

source 'https://rubygems.org'
gem 'radiator'
gem 'pry'
gem 'minitest'
gem 'minitest-line'
gem 'byebug'

Gemfile.lock

GEM
  remote: https://rubygems.org/
  specs:
    bitcoin-ruby (0.0.10)
    byebug (9.0.6)
    coderay (1.1.1)
    ffi (1.9.17)
    hashie (3.5.5)
    json (1.8.6)
    little-plugger (1.1.4)
    logging (2.2.2)
      little-plugger (~> 1.1)
      multi_json (~> 1.10)
    method_source (0.8.2)
    minitest (5.10.1)
    minitest-line (0.6.3)
      minitest (~> 5.0)
    multi_json (1.12.1)
    net-http-persistent (2.9.4)
    pry (0.10.4)
      coderay (~> 1.1.0)
      method_source (~> 0.8.1)
      slop (~> 3.4)
    radiator (0.2.1)
      bitcoin-ruby (= 0.0.10)
      ffi (= 1.9.17)
      hashie (~> 3.5, >= 3.5.5)
      json (~> 1.8, >= 1.8.6)
      logging (~> 2.2, >= 2.2.0)
      net-http-persistent (~> 2.9, >= 2.9.4)
    slop (3.6.0)

PLATFORMS
  ruby

DEPENDENCIES
  byebug
  minitest
  minitest-line
  pry
  radiator

BUNDLED WITH
   1.14.3

get_custom_json_by_tid_test.rb

require 'test_helper'

class GetCustomJsonByTidTest < SbdsTest
  def setup
    @from = '2017-03-28 00:23:27'
    @to = '2017-03-28 00:23:27'
  end
  
  def test_basic_get
    response = json_rpc do
      {
        method: 'sbds.get_custom_json_by_tid',
        params: {
          tid: 'com.steemit.community',
          from: @from,
          to: @to
        }
      }
    end
    
    refute_nil response, 'did not expect nil response'
    assert_equal '200', response.code, "expect response code 200 (OK): #{response.body}"
    result = JSON[response.body]['result']
    
    refute_nil result, "did not expect nil result, response body was: #{response.body}"  
    assert_equal Array, result.class, 'expect array'
    assert_equal 1, result.size, 'expect single element'
    
    custom_json = result.first
    refute_nil custom_json['required_posting_auths'], 'did not expect missing required_posting_auths; regression on https://github.com/steemit/sbds/issues/33'
  end
  
  def test_multiple_type_get
    tids = [
      # "follow",
      "",
      "reblog",
      "account_history",
      "opinion",
      "bookchain",
      "testchain",
      "test",
      "task",
      "com.steemit.community",
      "message",
      "charlieshrem",
      "booyah",
      "test_community"
    ]
    
    tids.each do |tid|
      response = json_rpc do
        {
          method: 'sbds.get_custom_json_by_tid',
          params: {
            tid: tid
          }
        }
      end
      
      refute_nil response, 'did not expect nil response'
      assert_equal '200', response.code, "expect response code 200 (OK): #{response.body}"
      result = JSON[response.body]['result']
      
      next if result.nil? # No operations for this tid or timeout?
      
      # refute_nil result, "did not expect nil result, response body was: #{response.body}"  
      assert_equal Array, result.class, 'expect array'
    end
  end
end

get_random_operation_block_nums_test.rb

require 'test_helper'

class GetRandomOperationBlockNumsTest < SbdsTest
  def test_basic_get
    response = json_rpc do
      {
        method: 'sbds.get_random_operation_block_nums',
        params: {
          op_type: 'pow',
          count: 1
        }
      }
    end
    
    refute_nil response, 'did not expect nil response'
    assert_equal '200', response.code, "expect response code 200 (OK): #{response.body}"
    result = JSON[response.body]['result']
    
    refute_nil result, "did not expect nil result, response body was: #{response.body}"  
    assert_equal Array, result.class, 'expect array'
    assert_equal 1, result.size, 'expect single element'
  end
end

get_random_operations_test.rb

require 'test_helper'

class GetRandomOperationsTest < SbdsTest
  def test_basic_get
    count = 1
    
    response = json_rpc do
      {
        method: 'sbds.get_random_operations',
        params: {
          op_type: 'pow',
          count: count
        }
      }
    end
    
    refute_nil response, 'did not expect nil response'
    assert_equal '200', response.code, "expect response code 200 (OK): #{response.body}"
    result = JSON[response.body]['result']
    
    refute_nil result, "did not expect nil result, response body was: #{response.body}"
    assert_equal Array, result.class, 'expect array'
    assert_equal count, result.size, 'expect single element'
    
    result.each do |op|
      %w(
        transaction_num nonce type timestamp props work block_id block_num
        worker_account operation_num
      ).each do |key|
        assert op[key], "expect #{key}"
      end
    end
  end
  
  def test_basic_get_uningested
    count = 1
    
    response = json_rpc do
      {
        method: 'sbds.get_random_operations',
        params: {
          op_type: 'decline_voting_rights',
          count: count
        }
      }
    end
    
    refute_nil response, 'did not expect nil response'
    assert_equal '200', response.code, "expect response code 200 (OK): #{response.body}"
    result = JSON[response.body]['result']
    
    skip 'Not supported yet.' if result.nil?
    fail 'Correct support detected, please update test.'
    refute_nil result, "did not expect nil result, response body was: #{response.body}"
    assert_equal Array, result.class, 'expect array'
    assert_equal count, result.size, 'expect single element'
  end
  
  def test_compare_all_ops_types
    count = 10
    blocks = {}
    
    Radiator::Operation::IDS.map(&:to_s).each do |op_type|
      response = json_rpc do
        {
          method: 'sbds.get_random_operations',
          params: {
            op_type: op_type,
            count: count
          }
        }
      end
      
      refute_nil response, 'did not expect nil response'
      assert_equal '200', response.code, "expect response code 200 (OK): #{response.body}"
      result = JSON[response.body]['result']
      
      next if result.nil? # No operations for this type?
      
      assert_equal Array, result.class, 'expect array'
      
      result.each do |op|
        block_num = op['block_num']
        
        block = if blocks.keys.include? block_num
          blocks[block_num]
        else
          response = api.get_block(block_num)
          blocks[block_num] = response.result
        end
        
        tx = block.transactions[op['transaction_num'] - 1]
        type, operation = tx.operations[op['operation_num'] - 1]
        
        assert_equal op_type, type, 'expect same operation type'
        
        operation.keys.each do |key|
          assert_equal operation[key], op[key], "expect equal #{key} for #{operation} in block: #{block_num}"
        end
      end
    end
  end
end

health_test.rb

require 'test_helper'

class HealthTest < SbdsTest
  def test_basic_get
    response = begin
      open("#{BASE_URL}/health")
    rescue OpenURI::HTTPError => e
      # this is normal when sbds is behind
      skip
    end
    
    refute_nil response, 'did not expect nil response'
    result = JSON[response.read]
    
    %w(last_irreversible_block timestamp diff last_db_block).each do |key|
      assert result[key], "expect #{key}"
    end
    
    assert result['last_irreversible_block'] >= result['last_db_block'], 'expect last_irreversible_block greater than or equal to last_db_block'
    assert_equal result['last_irreversible_block'] - result['last_db_block'], result['diff'], 'expect last_irreversible_block - last_db_block = diff'
    assert result['diff'] < 10, 'expect diff less than 10'
    assert Time.parse(result['timestamp']), 'expect timestamp'
  end
end

Rakefile

require 'rake/testtask'

Encoding.default_external = Encoding::UTF_8
Encoding.default_internal = Encoding::UTF_8

Rake::TestTask.new(:test) do |t|
  t.libs << '.'
  t.test_files = FileList['*_test.rb']
  t.ruby_opts << if ENV['HELL_ENABLED']
    '-W2'
  else
    '-W1'
  end
end

task default: :test

test_helper.rb

require 'radiator'
require 'minitest/autorun'
require 'pry'
require 'open-uri'
require 'json'
require 'net/http'

if ENV["HELL_ENABLED"]
  require "minitest/hell"
  
  class Minitest::Test
    parallelize_me!
  end
else
  require "minitest/pride"
end

BASE_URL = ENV['BASE_URL'] || 'https://sbds.steemitdev.com'
CHAIN = (ENV['CHAIN'] || 'steem').to_sym
NODE = ENV['NODE'] || 'https://steemd.steemitdev.com'

class SbdsTest < MiniTest::Test
  def api
    @api ||= Radiator::Api.new(chain: CHAIN, url: NODE)
  end
  
  def json_rpc(id = 0, &block)
    uri = URI("#{BASE_URL}/")
    https = Net::HTTP.new(uri.host, uri.port)
    https.use_ssl = true if BASE_URL =~ /^https/
    request = Net::HTTP::Post.new(uri.path, 'Content-Type' => 'application/json')
    request.body = {id: id, jsonrpc: '2.0'}.merge(yield).to_json
    begin
      https.request(request)
    rescue Net::ReadTimeout => e
      puts "Read Timeout for #{request.body}: #{e}"
    end
  end
end