Ruby applications which make HTTP calls to external services such as Websolr or Bonsai Elasticsearch may find it beneficial to use the net-http-spy gem to intercept and log HTTP calls made to that service. This is a very thorough way to log all communication between your application and the external service, which is very helpful when you need to troubleshoot an issue.
gem 'net-http-spy'| > mkdir test1 | |
| > echo foo > test1/foo | |
| > cp -lr test1 test2 | |
| > echo bar > test2/foo | |
| > cat test1/foo | |
| foo | |
| > cat test2/foo | |
| bar |
So Heroku routes their requests to your dynos randomly. While you're enhancing your middleware to log that dyno-level queue time, I have an optimization for you that should help fix the issue outright!
gem 'rack-timeout'
Incomplete brain-dump in progress. Thoughts and feedback welcome via Twitter (@nz_) or IM.
Sunspot is in a state of second-system syndrome. It constructed with a solid set of highly abstracted constructs to represent its DSL and eventually translate those operations into Solr operations. I hypothesize that Sunspot's design would become more flexible and intuitive if modeled off of Solr's concepts.
As a thought experiment: Explaining Sunspot's current design would make one a better programmer, with deeper knowledge of many techniques and object-oriented design abstractions involved in building a DSL for a Rails application. This explanation would not necessarily lend itself to a deeper understanding of Solr.
On the other hand, explaining Solr's concepts ought to help lend an intuitive understanding to Sunspot's own architecture. Implementation work to support a DSL should be built on top of those concepts in order to interface with the syntax and concepts that curre
Bonsai production plans are now available, which you can review at http://addons.heroku.com/bonsai
If your index is created on our older architecture (before November 2012), you will need to remove and replace the addon to upgrade. We'll preserve the existing index that you have now when you remove the addon, to help make sure the transition happens smoothly.
heroku config | grep BONSAI_INDEX_URL
| <schema> | |
| <types> | |
| <!-- add a location fieldtype, which splits a string into component subfields --> | |
| <fieldType name="location" class="solr.LatLonType" subFieldSuffix="_coordinate"/> | |
| </types> | |
| <fields> | |
| <!-- add the location field, and the coordinate subtype --> | |
| <field name="location" type="location" indexed="true" /> | |
| <dynamicField name="*_coordinate" type="tdouble" indexed="true" stored="false"/> | |
| </fields> |
| # Still a WIP while I read up on what all the API attributes are. Suggestions welcome. ;) | |
| require 'fog' | |
| require 'fog/aws/cloud_watch/alarm' | |
| include Fog::AWS::CloudWatch | |
| conn = Fog::Compute.new provider: 'AWS' | |
| conn.servers.each do |srv| | |
| Alarm.new({ |
| require 'net/http' | |
| require 'elastictastic' | |
| class Rubygem | |
| include Elastictastic::Document | |
| field :id, type: 'string' | |
| field :version, type: 'string' | |
| field :name, type: 'string' | |
| end |
For our initial beta launch, we designed our Heroku addon integration to automatically create one index per Heroku app. However, feedback from users such as yourself showed us that there are some ElasticSearch use cases where it is necessary to have the ability to create and destroy your own indexes. Furthermore, some applications or ElasticSearch clients required more than one index per application.
Rather than pre-create one index per application, our new architecture provides a URL that behaves like an ElasticSearch cluster endpoint, where you may create, update and destroy indexes within your account. We find this is a lot more useful in a handful of situations, and along the way we've been able to overhaul and improve our architecture to accommodate this design.
To upgrade to the latest architecture, you will need to remove the addon, and add it again. You will also need to create a new index, either with curl, or as
| task :reindex do | |
| batch_size = 20 | |
| models = [Foo, Bar, Baz] | |
| models.each do |model| | |
| model.find_in_batches(batch_size).each do |batch| | |
| retries = 1 | |
| begin | |
| Sunspot.index(batch) |