I was recently asked to add Blacklight Advanced Search to a Hyku app for the US Department of Transportation. It was a little tricky, so I’m documenting the process in the hopes of making life easier for the next person who has to do this.
Many thanks to Dean Farrell at UNC Libraries for pointing me at UNC’s hy-c implementation, where blacklight advanced search is installed into Hyrax: https://github.com/UNC-Libraries/hy-c/pull/421/files
David Kinzer’s Blacklight Search Notes are also excellent background reading: https://gist.github.com/dkinzer/4f6dbb4634dbbdc99255dbea6305ccae
As always, start with a test. I like to write a high-level feature test as a way of keeping myself focused on what I’m trying to deliver, and then I fill in unit tests for specific methods as needed. Here is my nearly-empty high level feature test. Note that I don’t really know what advanced search is going to look like yet, so I’m not spending lots of time on detail yet. However, it’s still helpful to have this test first. It gives me a way to quickly iterate and prove to myself that the changes I’m making are moving me in the right direction, and it lets me build the test up over time as I’m implementing the feature.
# frozen_string_literal: true
require 'rails_helper'
include Warden::Test::Helpers
RSpec.feature 'Advanced Search', type: :feature, js: true, clean: true do
context 'an unauthenticated user' do
scenario 'advanced search basic sanity check' do
visit '/advanced'
fill_in('Title', with: 'ambitious aardvark')
find('#advanced-search-submit').click
expect(page).to have_content('ambitious aardvark')
expect(page).to have_content('No results found for your search')
end
end
end
I follow the blacklight_advanced_search basic instructions:
Add to your application's Gemfile:
gem "blacklight_advanced_search"
then run 'bundle install'. Then run:
rails generate blacklight_advanced_search
Note: It will offer to generate a new basic search partial for you. You do NOT want it. It will break the basic search in Hyku. If you end up with a new app/views/catalog/_search_form.html.erb just delete it and add by hand anywhere you want a link to advanced search.
So, I follow the installation instructions, and then I run my feature spec. The first time I do, I get this error:
undefined method facets_for_advanced_search_form for Hyrax::CatalogSearchBuilder (NoMethodError)`
So, clearly, I’m missing some configuration.
I drop a byebug in at the line indicated by the stack trace. In this case: blacklight-6.23.0/lib/blacklight/search_builder.rb:147
[142, 151] in /usr/local/rvm/gems/ruby-2.5.8/gems/blacklight-6.23.0/lib/blacklight/search_builder.rb
142: #
143: # @return a params hash for searching solr.
144: def processed_parameters
145: request.tap do |request_parameters|
146: byebug
=> 147: processor_chain.each do |method_name|
148: send(method_name, request_parameters)
149: end
150: end
151: end
(byebug) request_parameters
{"facet.field"=>[], "facet.query"=>[], "facet.pivot"=>[], "fq"=>[], "hl.fl"=>[]}
(byebug) processor_chain
[:default_solr_parameters, :add_query_to_solr, :add_facet_fq_to_solr, :add_facetting_to_solr, :add_solr_fields_to_query, :add_paging_to_solr, :add_sorting_to_solr, :add_group_config_to_solr, :add_facet_paging_to_solr, :add_access_controls_to_solr_params, :filter_models, :only_active_works, :add_access_controls_to_solr_params, :show_works_or_works_that_contain_files, :show_only_active_records, :filter_collection_facet_for_access, :facets_for_advanced_search_form]
That : facets_for_advanced_search_form at the end of the processor chain seems like a likely culprit, so I’m going to remove it and see if that gets my form to render. Initially, I’m just going to add a next right in place, to see if this fixes the problem, before I go to the trouble of figuring out the right way to do it:
def processed_parameters
request.tap do |request_parameters|
processor_chain.each do |method_name|
next if method_name == :facets_for_advanced_search_form
send(method_name, request_parameters)
end
end
end
And now my form renders!
So I’ve proven to myself that the problem is this missing facets_for_advanced_search_form method, but how to fix that in a maintainable way?
Let’s go look at the class mentioned in the error message: Hyrax::CatalogSearchBuilder
Here is the source of that method: https://github.com/samvera/hyrax/blob/130c4e600318a9194725b35dd0fd6e19e5108dd9/app/search_builders/hyrax/catalog_search_builder.rb
Notice the excellent guidance provided here:
# If you plan to customize the base catalog search builder behavior (e.g. by
# adding a mixin module provided by a blacklight extension gem), inheriting this
# class, customizing behavior, and reconfiguring `CatalogController` is the
# preferred mechanism.
Sounds good to me. Let’s try that!
So, I’m going to start by writing a test, of course. I go look at how the search builders are set up in hyrax, and I make this very basic test in spec/search_builders/ntl_search_builder_spec.rb (note that this is adapted from the test setup at https://github.com/samvera/hyrax/blob/130c4e600318a9194725b35dd0fd6e19e5108dd9/spec/search_builders/hyrax/collection_search_builder_spec.rb... I did not attempt to write this off the top of my head.)
# frozen_string_literal: true
RSpec.describe NtlSearchBuilder do
let(:scope) do
double(blacklight_config: CatalogController.blacklight_config,
current_ability: ability)
end
let(:user) { create(:user) }
let(:ability) { ::Ability.new(user) }
let(:access) { :read }
let(:builder) { described_class.new(scope).with_access(access) }
it "can be instantiated" do
expect(builder).to be_instance_of(described_class)
end
end
This test doesn’t do anything except verify that the class is set up correctly, but that is better than nothing, and it gives us a place where we can flesh out behavior as we need it. I run it, and of course it fails, for the expected reasons: NameError: uninitialized constant NtlSearchBuilder
I then define the class, by making this file at app/search_builders/ntl_search_builder.rb:
class NtlSearchBuilder < Hyrax::CatalogSearchBuilder
end
And now my test passes!
Now, we set CatalogController to use this new class:
# Use locally customized NtlSearchBuilder so we can enable blacklight_advanced_search
config.search_builder_class = NtlSearchBuilder
And I add the facets_for_advanced_search_form method to my NtlSearchBuilder class, just copying the method (https://github.com/projectblacklight/blacklight_advanced_search/blob/master/lib/blacklight_advanced_search/advanced_search_builder.rb#L77-L96)
I also add the lines that the Blacklight advanced search generator installed into app/models/search_builder.rb. This is the bit that actually combines the Advanced Search query capabilities with the Hyrax provided search capabilities (e.g., search gated by permissions, workflow, group membership, etc.)
Now my class looks like this:
##
# A locally defined search builder, which will allow us to customize the search
# behavior of this application. In particular, this is needed to allow us to
# use blacklight_advanced_search.
class NtlSearchBuilder < Hyrax::CatalogSearchBuilder
include Blacklight::Solr::SearchBuilderBehavior
include BlacklightAdvancedSearch::AdvancedSearchBuilder
self.default_processor_chain += [:add_advanced_parse_q_to_solr, :add_advanced_search_to_solr]
# A Solr param filter that is NOT included by default in the chain,
# but is appended by AdvancedController#index, to do a search
# for facets _ignoring_ the current query, we want the facets
# as if the current query weren't there.
#
# Also adds any solr params set in blacklight_config.advanced_search[:form_solr_parameters]
def facets_for_advanced_search_form(solr_p)
# ensure empty query is all records, to fetch available facets on entire corpus
solr_p["q"] = '{!lucene}*:*'
# explicitly use lucene defType since we are passing a lucene query above (and appears to be required for solr 7)
solr_p["defType"] = 'lucene'
# We only care about facets, we don't need any rows.
solr_p["rows"] = "0"
# Anything set in config as a literal
if blacklight_config.advanced_search[:form_solr_parameters]
solr_p.merge!(blacklight_config.advanced_search[:form_solr_parameters])
end
end
end
And now my advanced search page renders and my feature test for advanced search passes.
One last step: I flesh out my advanced search feature spec a bit, including loading some actual client provided data, ensuring the advanced search feature really does work as expected. Here is the feature spec with some of that added:
# frozen_string_literal: true
require 'rails_helper'
include Warden::Test::Helpers
RSpec.feature 'Advanced Search', type: :feature, js: true, clean: true do
context 'empty solr index' do
scenario 'basic search sanity check' do
visit '/'
fill_in('q', with: 'ambitious aardvark')
click_button('Go')
expect(page).to have_content('ambitious aardvark')
expect(page).to have_content('No results found for your search')
end
scenario 'advanced search basic sanity check' do
visit '/advanced'
fill_in('Title', with: 'ambitious aardvark')
find('#advanced-search-submit').click
expect(page).to have_content('ambitious aardvark')
expect(page).to have_content('No results found for your search')
end
end
##
# Load solr data from Department of Transportation import testing and ensure advanced
# search works reasonably well with this data.
context 'with data' do
let!(:admin_set_collection_type) { FactoryBot.create(:admin_set_collection_type) }
let!(:user_collection_type) { FactoryBot.create(:user_collection_type) }
before do
solr = Blacklight.default_index.connection
sample_records = JSON.parse(File.open(File.join(fixture_path, 'solr', 'dot-sample-data.json')).read)
docs = sample_records["response"]["docs"]
docs.each do |doc|
doc.delete("_version_")
doc.delete("score")
solr.add(doc)
end
solr.commit
end
scenario 'basic search sanity check' do
visit '/'
click_button('Go')
number_of_search_results = find('span.page_entries').find_all('strong').last.text
expect(number_of_search_results).to eq('17')
end
it "searches by title" do
visit '/advanced'
fill_in('Title', with: 'habitat')
find('#advanced-search-submit').click
number_of_search_results = find('span.page_entries').find_all('strong').last.text
expect(number_of_search_results).to eq('1')
end
end
end