Testing CreateIssuePagesJob retry

retry-job.md

Getting started, pre-requisites:

1. Clone the issue-pdf-composition branch of newspaper_works
2. Make sure your current working directory is the root of the newspaper_works GEM.
3. Set up the gem and test app:

bundle install
bundle exec rake engine_cart:generate

Step 1: clean your (test environment) Fedora and Solr

From the root of your newspaper_works GEM, run:

fcrepo_wrapper --config config/fcrepo_wrapper_test.yml clean
solr_wrapper --config config/solr_wrapper_test.yml clean

Step 2: run wrappers after cleaning, using test configuration:

fcrepo_wrapper --config config/fcrepo_wrapper_test.yml
solr_wrapper --config config/solr_wrapper_test.yml

Step 4: Make sure sidekiq is running in your .internal_test_app in the test environment:

cd .internal_test_app

Add the following to your .internal_test_app/Gemfile:

gem 'sidekiq'

Then run bundle install to ensure that sidekiq is installed for your app.

Configure your app to use the :sidekiq queue adapter:

...by editing config/application.rb. Then run sidekiq:

bundle exec sidekiq --environment test

Step 3: Open rails console test in .internal_test_app, ingest an NDNP issue from newspaper_works_fixtures as follows:

require 'newspaper_works_fixtures'
ndnp_path = File.join(NewspaperWorksFixtures.file_fixtures, 'ndnp')
Hyrax.config.whitelisted_ingest_dirs.push(ndnp_path)
issue_path = File.join(ndnp_path, 'batch_local/sn85058233/17082901001/1935080201/1935080201.xml')
issue_data = NewspaperWorks::Ingest::NDNP::IssueIngest.new(issue_path)
# We need data with no pre-made PDF derivative, because we want job to create
#   this for us so we can test retry because we are waiting on page PDF to be
#   made.  To do this, modify the NDNP batch data inline to remove
#   the pre-made PDF from the data.
issue_data.each do |page|
  # modify page.files, so that PageIngester, via IssueIngester won't see them:
  page.files.reject! { |path| path.end_with?('pdf') }
end
ingester = NewspaperWorks::Ingest::NDNP::IssueIngester.new(issue_data)
ingester.ingest 

Step 4: verify job start

Watch sidekiq console, looking for indication that the following job is started:

• ComposeIssuePDFJob

_It is likely that the ComposeIssuePDFJob will fail on first attempt, and retry. You should also see CharacterizeJob and CreateDerivativesJob fire from Hyrax, for each page. Until those two jobs are done for each of the two pages of the issue being ingested, ComposeIssuePDFJob will fail (slated for retry) due to exception.`

Step 5: verify job retires

Verify that ComposeIssuePDFJob started more than once in sidekiq log.

Step 6: Check final PDF in rails console:

issue = ingester.target
issue.reload!
NewspaperWorks::Data::WorkDerivatives.of(issue).exist?('pdf') # => true