Skip to content

Instantly share code, notes, and snippets.

@michaeltelford
Last active March 21, 2024 15:46
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save michaeltelford/b90d5e062da383be503ca2c3a16e9164 to your computer and use it in GitHub Desktop.
Save michaeltelford/b90d5e062da383be503ca2c3a16e9164 to your computer and use it in GitHub Desktop.
An example Wgit development setup
# Connects to the local `toys db start` mongo instance.
WGIT_CONNECTION_STRING="mongodb://rubyapp:abcdef@localhost/crawler"
require 'dotenv'
require 'byebug' # Call 'byebug' anywhere in the code to debug.
### SETUP ###
# Define a method to facilitate the reloading of code changes.
def reload
original_verbose = $VERBOSE
$VERBOSE = nil # Suppress warning messages (from reloading CONSTANTS).
load 'load.rb' # (Re)load all code changes.
$VERBOSE = original_verbose # Activate warning messages again globally.
Wgit.logger.level = Logger::DEBUG
true
end
# Load the most recent code into the session and include modules etc.
reload
# Load the .env vars. Used to connect to the DB etc.
Dotenv.load
### FIXTURES ###
def in_memory_db
@in_memory_db ||= Database::InMemory.new
end
def db
return @db if @db.instance_of?(Database.adapter_class)
# Avoid creating a new in-memory DB everytime we switch between adapters.
@db = if Database.adapter_class == Database::InMemory
in_memory_db
else
Database.adapter_class.new
end
end
def url
@url ||= Url.new(
'http://txti.es/',
crawled: true,
date_crawled: Time.now,
crawl_duration: 0.567890
)
end
def doc
@doc ||= Document.new(
'http://www.mytestsite.com/home',
File.read('test/mock/fixtures/test_doc.html')
)
end
### HELPERS ###
def use_mongo
Database.adapter_class = Database::MongoDB
use_database(db)
end
def use_memory
Database.adapter_class = Database::InMemory
use_database(db)
end
def index_wiki(&block)
wiki = 'https://github.com/michaeltelford/wgit/wiki'
opts = {
follow: :default,
allow_paths: 'michaeltelford/wgit/wiki/*',
disallow_paths: 'michaeltelford/wgit/wiki/*/_history'
}
index_site(wiki, **opts, &block)
end
def url_with_fragment(doc, query)
fragment = doc.nearest_fragment(query, &:last)
return nil unless fragment
doc.url.join(fragment).to_s
end
def mock_fixtures
load 'test/mock/fixtures.rb'
end
# Print some basic usage information.
def info
puts 'To load your code changes type: reload'
puts 'Available fixture vars include: crawler, url, doc, db'
puts 'Some helpful database manipulator methods include:'
puts 'empty_db!, index, index_site, search, db.num_records etc.'
puts 'To see this information again type: info'
puts "When you're finished type: exit (or press Ctrl+D)"
end
# Set the default DB adapter to use.
use_mongo
puts
info
@michaeltelford
Copy link
Author

michaeltelford commented Dec 28, 2019

For a full development console, do the following:

  • bundle exec toys setup
  • Copy the contents of the files above into the same files at the root of the repo
  • bundle exec toys db build
  • bundle exec toys db start
  • bundle exec toys console

Done.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment