The tests that fail in smart-answers when I run with seed 56021

test-output.txt

Chriss-MacBook-Air-2:~/Code/alphagov/smart-answers$ rake test TESTOPTS="--seed=56021"
  * DEFERRED: sequence of two questions a question raises an error should permit custom error messages per question. 
Run options: --seed=56021

# Running:

.....................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................EEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEE.................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................

Finished in 256.681458s, 15.2329 runs/s, 80.9564 assertions/s.

  1) Error:
SmartAnswersControllerTest#test_: GET /<slug> a response has been accepted should link back to change the response. :
RuntimeError: Unexpected modification of flow registry options, got {:show_drafts=>true, :preload_flows=>true} and had {:preload_flows=>true, :show_drafts=>true, :show_transitions=>true}
    lib/smartdown_adapter/registry.rb:9:in `instance'
    app/controllers/smart_answers_controller.rb:115:in `smartdown_registry'
    app/controllers/smart_answers_controller.rb:119:in `smartdown_question'
    app/controllers/smart_answers_controller.rb:79:in `find_smart_answer'
    test/functional/smart_answers_controller_test.rb:418:in `block (3 levels) in <class:SmartAnswersControllerTest>'


  2) Error:
SmartAnswersControllerTest#test_: GET /<slug> a response has been accepted should show response summary. :
RuntimeError: Unexpected modification of flow registry options, got {:show_drafts=>true, :preload_flows=>true} and had {:preload_flows=>true, :show_drafts=>true, :show_transitions=>true}
    lib/smartdown_adapter/registry.rb:9:in `instance'
    app/controllers/smart_answers_controller.rb:115:in `smartdown_registry'
    app/controllers/smart_answers_controller.rb:119:in `smartdown_question'
    app/controllers/smart_answers_controller.rb:79:in `find_smart_answer'
    test/functional/smart_answers_controller_test.rb:418:in `block (3 levels) in <class:SmartAnswersControllerTest>'


  3) Error:
SmartAnswersControllerTest#test_: GET /<slug> a response has been accepted should show the next question. :
RuntimeError: Unexpected modification of flow registry options, got {:show_drafts=>true, :preload_flows=>true} and had {:preload_flows=>true, :show_drafts=>true, :show_transitions=>true}
    lib/smartdown_adapter/registry.rb:9:in `instance'
    app/controllers/smart_answers_controller.rb:115:in `smartdown_registry'
    app/controllers/smart_answers_controller.rb:119:in `smartdown_question'
    app/controllers/smart_answers_controller.rb:79:in `find_smart_answer'
    test/functional/smart_answers_controller_test.rb:418:in `block (3 levels) in <class:SmartAnswersControllerTest>'


  4) Error:
SmartAnswersControllerTest#test_: GET /<slug> date question no response given format=json should give correct canonical url. :
RuntimeError: Unexpected modification of flow registry options, got {:show_drafts=>true, :preload_flows=>true} and had {:preload_flows=>true, :show_drafts=>true, :show_transitions=>true}
    lib/smartdown_adapter/registry.rb:9:in `instance'
    app/controllers/smart_answers_controller.rb:115:in `smartdown_registry'
    app/controllers/smart_answers_controller.rb:119:in `smartdown_question'
    app/controllers/smart_answers_controller.rb:79:in `find_smart_answer'
    test/functional/smart_answers_controller_test.rb:52:in `submit_json_response'
    test/functional/smart_answers_controller_test.rb:211:in `block (5 levels) in <class:SmartAnswersControllerTest>'


  5) Error:
SmartAnswersControllerTest#test_: GET /<slug> date question no response given format=json should show an error message. :
RuntimeError: Unexpected modification of flow registry options, got {:show_drafts=>true, :preload_flows=>true} and had {:preload_flows=>true, :show_drafts=>true, :show_transitions=>true}
    lib/smartdown_adapter/registry.rb:9:in `instance'
    app/controllers/smart_answers_controller.rb:115:in `smartdown_registry'
    app/controllers/smart_answers_controller.rb:119:in `smartdown_question'
    app/controllers/smart_answers_controller.rb:79:in `find_smart_answer'
    test/functional/smart_answers_controller_test.rb:52:in `submit_json_response'
    test/functional/smart_answers_controller_test.rb:217:in `block (5 levels) in <class:SmartAnswersControllerTest>'


  6) Error:
SmartAnswersControllerTest#test_: GET /<slug> date question no response given should redisplay question. :
RuntimeError: Unexpected modification of flow registry options, got {:show_drafts=>true, :preload_flows=>true} and had {:preload_flows=>true, :show_drafts=>true, :show_transitions=>true}
    lib/smartdown_adapter/registry.rb:9:in `instance'
    app/controllers/smart_answers_controller.rb:115:in `smartdown_registry'
    app/controllers/smart_answers_controller.rb:119:in `smartdown_question'
    app/controllers/smart_answers_controller.rb:79:in `find_smart_answer'
    test/functional/smart_answers_controller_test.rb:41:in `submit_response'
    test/functional/smart_answers_controller_test.rb:200:in `block (4 levels) in <class:SmartAnswersControllerTest>'


  7) Error:
SmartAnswersControllerTest#test_: GET /<slug> date question no response given should show an error message. :
RuntimeError: Unexpected modification of flow registry options, got {:show_drafts=>true, :preload_flows=>true} and had {:preload_flows=>true, :show_drafts=>true, :show_transitions=>true}
    lib/smartdown_adapter/registry.rb:9:in `instance'
    app/controllers/smart_answers_controller.rb:115:in `smartdown_registry'
    app/controllers/smart_answers_controller.rb:119:in `smartdown_question'
    app/controllers/smart_answers_controller.rb:79:in `find_smart_answer'
    test/functional/smart_answers_controller_test.rb:41:in `submit_response'
    test/functional/smart_answers_controller_test.rb:205:in `block (4 levels) in <class:SmartAnswersControllerTest>'


  8) Error:
SmartAnswersControllerTest#test_: GET /<slug> date question should accept question input and redirect to canonical url. :
RuntimeError: Unexpected modification of flow registry options, got {:show_drafts=>true, :preload_flows=>true} and had {:preload_flows=>true, :show_drafts=>true, :show_transitions=>true}
    lib/smartdown_adapter/registry.rb:9:in `instance'
    app/controllers/smart_answers_controller.rb:115:in `smartdown_registry'
    app/controllers/smart_answers_controller.rb:119:in `smartdown_question'
    app/controllers/smart_answers_controller.rb:79:in `find_smart_answer'
    test/functional/smart_answers_controller_test.rb:41:in `submit_response'
    test/functional/smart_answers_controller_test.rb:170:in `block (3 levels) in <class:SmartAnswersControllerTest>'


  9) Error:
SmartAnswersControllerTest#test_: GET /<slug> date question should display collapsed question, and format number. :
RuntimeError: Unexpected modification of flow registry options, got {:show_drafts=>true, :preload_flows=>true} and had {:preload_flows=>true, :show_drafts=>true, :show_transitions=>true}
    lib/smartdown_adapter/registry.rb:9:in `instance'
    app/controllers/smart_answers_controller.rb:115:in `smartdown_registry'
    app/controllers/smart_answers_controller.rb:119:in `smartdown_question'
    app/controllers/smart_answers_controller.rb:79:in `find_smart_answer'
    test/functional/smart_answers_controller_test.rb:227:in `block (3 levels) in <class:SmartAnswersControllerTest>'


 10) Error:
SmartAnswersControllerTest#test_: GET /<slug> date question should display question. :
RuntimeError: Unexpected modification of flow registry options, got {:show_drafts=>true, :preload_flows=>true} and had {:preload_flows=>true, :show_drafts=>true, :show_transitions=>true}
    lib/smartdown_adapter/registry.rb:9:in `instance'
    app/controllers/smart_answers_controller.rb:115:in `smartdown_registry'
    app/controllers/smart_answers_controller.rb:119:in `smartdown_question'
    app/controllers/smart_answers_controller.rb:79:in `find_smart_answer'
    test/functional/smart_answers_controller_test.rb:162:in `block (3 levels) in <class:SmartAnswersControllerTest>'


 11) Error:
SmartAnswersControllerTest#test_: GET /<slug> date question should not error if passed blank response. :
RuntimeError: Unexpected modification of flow registry options, got {:show_drafts=>true, :preload_flows=>true} and had {:preload_flows=>true, :show_drafts=>true, :show_transitions=>true}
    lib/smartdown_adapter/registry.rb:9:in `instance'
    app/controllers/smart_answers_controller.rb:115:in `smartdown_registry'
    app/controllers/smart_answers_controller.rb:119:in `smartdown_question'
    app/controllers/smart_answers_controller.rb:79:in `find_smart_answer'
    test/functional/smart_answers_controller_test.rb:41:in `submit_response'
    test/functional/smart_answers_controller_test.rb:175:in `block (3 levels) in <class:SmartAnswersControllerTest>'


 12) Error:
SmartAnswersControllerTest#test_: GET /<slug> date question should not error if passed string response. :
RuntimeError: Unexpected modification of flow registry options, got {:show_drafts=>true, :preload_flows=>true} and had {:preload_flows=>true, :show_drafts=>true, :show_transitions=>true}
    lib/smartdown_adapter/registry.rb:9:in `instance'
    app/controllers/smart_answers_controller.rb:115:in `smartdown_registry'
    app/controllers/smart_answers_controller.rb:119:in `smartdown_question'
    app/controllers/smart_answers_controller.rb:79:in `find_smart_answer'
    test/functional/smart_answers_controller_test.rb:41:in `submit_response'
    test/functional/smart_answers_controller_test.rb:180:in `block (3 levels) in <class:SmartAnswersControllerTest>'


 13) Error:
SmartAnswersControllerTest#test_: GET /<slug> date question valid response given format=json should give correct canonical url. :
RuntimeError: Unexpected modification of flow registry options, got {:show_drafts=>true, :preload_flows=>true} and had {:preload_flows=>true, :show_drafts=>true, :show_transitions=>true}
    lib/smartdown_adapter/registry.rb:9:in `instance'
    app/controllers/smart_answers_controller.rb:115:in `smartdown_registry'
    app/controllers/smart_answers_controller.rb:119:in `smartdown_question'
    app/controllers/smart_answers_controller.rb:79:in `find_smart_answer'
    test/functional/smart_answers_controller_test.rb:52:in `submit_json_response'
    test/functional/smart_answers_controller_test.rb:187:in `block (5 levels) in <class:SmartAnswersControllerTest>'


 14) Error:
SmartAnswersControllerTest#test_: GET /<slug> date question valid response given format=json should set correct cache control headers. :
RuntimeError: Unexpected modification of flow registry options, got {:show_drafts=>true, :preload_flows=>true} and had {:preload_flows=>true, :show_drafts=>true, :show_transitions=>true}
    lib/smartdown_adapter/registry.rb:9:in `instance'
    app/controllers/smart_answers_controller.rb:115:in `smartdown_registry'
    app/controllers/smart_answers_controller.rb:119:in `smartdown_question'
    app/controllers/smart_answers_controller.rb:79:in `find_smart_answer'
    test/functional/smart_answers_controller_test.rb:52:in `submit_json_response'
    test/functional/smart_answers_controller_test.rb:192:in `block (5 levels) in <class:SmartAnswersControllerTest>'


 15) Error:
SmartAnswersControllerTest#test_: GET /<slug> debugging should not render debug information on the page when not enabled. :
RuntimeError: Unexpected modification of flow registry options, got {:show_drafts=>true, :preload_flows=>true} and had {:preload_flows=>true, :show_drafts=>true, :show_transitions=>true}
    lib/smartdown_adapter/registry.rb:9:in `instance'
    app/controllers/smart_answers_controller.rb:115:in `smartdown_registry'
    app/controllers/smart_answers_controller.rb:119:in `smartdown_question'
    app/controllers/smart_answers_controller.rb:79:in `find_smart_answer'
    test/functional/smart_answers_controller_test.rb:458:in `block (3 levels) in <class:SmartAnswersControllerTest>'


 16) Error:
SmartAnswersControllerTest#test_: GET /<slug> debugging should render debug information on the page when enabled. :
RuntimeError: Unexpected modification of flow registry options, got {:show_drafts=>true, :preload_flows=>true} and had {:preload_flows=>true, :show_drafts=>true, :show_transitions=>true}
    lib/smartdown_adapter/registry.rb:9:in `instance'
    app/controllers/smart_answers_controller.rb:115:in `smartdown_registry'
    app/controllers/smart_answers_controller.rb:119:in `smartdown_question'
    app/controllers/smart_answers_controller.rb:79:in `find_smart_answer'
    test/functional/smart_answers_controller_test.rb:451:in `block (3 levels) in <class:SmartAnswersControllerTest>'


 17) Error:
SmartAnswersControllerTest#test_: GET /<slug> format=json should render content without layout. :
RuntimeError: Unexpected modification of flow registry options, got {:show_drafts=>true, :preload_flows=>true} and had {:preload_flows=>true, :show_drafts=>true, :show_transitions=>true}
    lib/smartdown_adapter/registry.rb:9:in `instance'
    app/controllers/smart_answers_controller.rb:115:in `smartdown_registry'
    app/controllers/smart_answers_controller.rb:119:in `smartdown_question'
    app/controllers/smart_answers_controller.rb:79:in `find_smart_answer'
    test/functional/smart_answers_controller_test.rb:437:in `block (3 levels) in <class:SmartAnswersControllerTest>'


 18) Error:
SmartAnswersControllerTest#test_: GET /<slug> meta description in translation file should be shown. :
RuntimeError: Unexpected modification of flow registry options, got {:show_drafts=>true, :preload_flows=>true} and had {:preload_flows=>true, :show_drafts=>true, :show_transitions=>true}
    lib/smartdown_adapter/registry.rb:9:in `instance'
    app/controllers/smart_answers_controller.rb:115:in `smartdown_registry'
    app/controllers/smart_answers_controller.rb:119:in `smartdown_question'
    app/controllers/smart_answers_controller.rb:79:in `find_smart_answer'
    test/functional/smart_answers_controller_test.rb:98:in `block (4 levels) in <class:SmartAnswersControllerTest>'
    test/helpers/i18n_test_helper.rb:7:in `using_translation_file'
    test/functional/smart_answers_controller_test.rb:97:in `block (3 levels) in <class:SmartAnswersControllerTest>'


 19) Error:
SmartAnswersControllerTest#test_: GET /<slug> money question should display question. :
RuntimeError: Unexpected modification of flow registry options, got {:show_drafts=>true, :preload_flows=>true} and had {:preload_flows=>true, :show_drafts=>true, :show_transitions=>true}
    lib/smartdown_adapter/registry.rb:9:in `instance'
    app/controllers/smart_answers_controller.rb:115:in `smartdown_registry'
    app/controllers/smart_answers_controller.rb:119:in `smartdown_question'
    app/controllers/smart_answers_controller.rb:79:in `find_smart_answer'
    test/functional/smart_answers_controller_test.rb:300:in `block (3 levels) in <class:SmartAnswersControllerTest>'


 20) Error:
SmartAnswersControllerTest#test_: GET /<slug> money question should show a validation error if invalid input. :
RuntimeError: Unexpected modification of flow registry options, got {:show_drafts=>true, :preload_flows=>true} and had {:preload_flows=>true, :show_drafts=>true, :show_transitions=>true}
    lib/smartdown_adapter/registry.rb:9:in `instance'
    app/controllers/smart_answers_controller.rb:115:in `smartdown_registry'
    app/controllers/smart_answers_controller.rb:119:in `smartdown_question'
    app/controllers/smart_answers_controller.rb:79:in `find_smart_answer'
    test/functional/smart_answers_controller_test.rb:41:in `submit_response'
    test/functional/smart_answers_controller_test.rb:306:in `block (3 levels) in <class:SmartAnswersControllerTest>'


 21) Error:
SmartAnswersControllerTest#test_: GET /<slug> money question suffix_label in translation file should show the label after the question input. :
RuntimeError: Unexpected modification of flow registry options, got {:show_drafts=>true, :preload_flows=>true} and had {:preload_flows=>true, :show_drafts=>true, :show_transitions=>true}
    lib/smartdown_adapter/registry.rb:9:in `instance'
    app/controllers/smart_answers_controller.rb:115:in `smartdown_registry'
    app/controllers/smart_answers_controller.rb:119:in `smartdown_question'
    app/controllers/smart_answers_controller.rb:79:in `find_smart_answer'
    test/functional/smart_answers_controller_test.rb:314:in `block (5 levels) in <class:SmartAnswersControllerTest>'
    test/helpers/i18n_test_helper.rb:7:in `using_translation_file'
    test/functional/smart_answers_controller_test.rb:313:in `block (4 levels) in <class:SmartAnswersControllerTest>'


 22) Error:
SmartAnswersControllerTest#test_: GET /<slug> multiple choice question format=json no response given should show an error message. :
RuntimeError: Unexpected modification of flow registry options, got {:show_drafts=>true, :preload_flows=>true} and had {:preload_flows=>true, :show_drafts=>true, :show_transitions=>true}
    lib/smartdown_adapter/registry.rb:9:in `instance'
    app/controllers/smart_answers_controller.rb:115:in `smartdown_registry'
    app/controllers/smart_answers_controller.rb:119:in `smartdown_question'
    app/controllers/smart_answers_controller.rb:79:in `find_smart_answer'
    test/functional/smart_answers_controller_test.rb:52:in `submit_json_response'
    test/functional/smart_answers_controller_test.rb:403:in `block (5 levels) in <class:SmartAnswersControllerTest>'


 23) Error:
SmartAnswersControllerTest#test_: GET /<slug> salary question a response has been accepted should have cache headers set to 30 mins for inner pages. :
RuntimeError: Unexpected modification of flow registry options, got {:show_drafts=>true, :preload_flows=>true} and had {:preload_flows=>true, :show_drafts=>true, :show_transitions=>true}
    lib/smartdown_adapter/registry.rb:9:in `instance'
    app/controllers/smart_answers_controller.rb:115:in `smartdown_registry'
    app/controllers/smart_answers_controller.rb:119:in `smartdown_question'
    app/controllers/smart_answers_controller.rb:79:in `find_smart_answer'
    test/functional/smart_answers_controller_test.rb:376:in `block (4 levels) in <class:SmartAnswersControllerTest>'


 24) Error:
SmartAnswersControllerTest#test_: GET /<slug> salary question a response has been accepted should show response summary. :
RuntimeError: Unexpected modification of flow registry options, got {:show_drafts=>true, :preload_flows=>true} and had {:preload_flows=>true, :show_drafts=>true, :show_transitions=>true}
    lib/smartdown_adapter/registry.rb:9:in `instance'
    app/controllers/smart_answers_controller.rb:115:in `smartdown_registry'
    app/controllers/smart_answers_controller.rb:119:in `smartdown_question'
    app/controllers/smart_answers_controller.rb:79:in `find_smart_answer'
    test/functional/smart_answers_controller_test.rb:376:in `block (4 levels) in <class:SmartAnswersControllerTest>'


 25) Error:
SmartAnswersControllerTest#test_: GET /<slug> salary question error message not overridden in translation file should show a generic message. :
RuntimeError: Unexpected modification of flow registry options, got {:show_drafts=>true, :preload_flows=>true} and had {:preload_flows=>true, :show_drafts=>true, :show_transitions=>true}
    lib/smartdown_adapter/registry.rb:9:in `instance'
    app/controllers/smart_answers_controller.rb:115:in `smartdown_registry'
    app/controllers/smart_answers_controller.rb:119:in `smartdown_question'
    app/controllers/smart_answers_controller.rb:79:in `find_smart_answer'
    test/functional/smart_answers_controller_test.rb:41:in `submit_response'
    test/functional/smart_answers_controller_test.rb:358:in `block (4 levels) in <class:SmartAnswersControllerTest>'


 26) Error:
SmartAnswersControllerTest#test_: GET /<slug> salary question error message overridden in translation file should show a validation error if invalid amount. :
RuntimeError: Unexpected modification of flow registry options, got {:show_drafts=>true, :preload_flows=>true} and had {:preload_flows=>true, :show_drafts=>true, :show_transitions=>true}
    lib/smartdown_adapter/registry.rb:9:in `instance'
    app/controllers/smart_answers_controller.rb:115:in `smartdown_registry'
    app/controllers/smart_answers_controller.rb:119:in `smartdown_question'
    app/controllers/smart_answers_controller.rb:79:in `find_smart_answer'
    test/functional/smart_answers_controller_test.rb:41:in `submit_response'
    test/functional/smart_answers_controller_test.rb:346:in `block (5 levels) in <class:SmartAnswersControllerTest>'
    test/helpers/i18n_test_helper.rb:7:in `using_translation_file'
    test/functional/smart_answers_controller_test.rb:345:in `block (4 levels) in <class:SmartAnswersControllerTest>'


 27) Error:
SmartAnswersControllerTest#test_: GET /<slug> salary question should accept responses as GET params and redirect to canonical url. :
RuntimeError: Unexpected modification of flow registry options, got {:show_drafts=>true, :preload_flows=>true} and had {:preload_flows=>true, :show_drafts=>true, :show_transitions=>true}
    lib/smartdown_adapter/registry.rb:9:in `instance'
    app/controllers/smart_answers_controller.rb:115:in `smartdown_registry'
    app/controllers/smart_answers_controller.rb:119:in `smartdown_question'
    app/controllers/smart_answers_controller.rb:79:in `find_smart_answer'
    test/functional/smart_answers_controller_test.rb:41:in `submit_response'
    test/functional/smart_answers_controller_test.rb:371:in `block (3 levels) in <class:SmartAnswersControllerTest>'


 28) Error:
SmartAnswersControllerTest#test_: GET /<slug> salary question should display question. :
RuntimeError: Unexpected modification of flow registry options, got {:show_drafts=>true, :preload_flows=>true} and had {:preload_flows=>true, :show_drafts=>true, :show_transitions=>true}
    lib/smartdown_adapter/registry.rb:9:in `instance'
    app/controllers/smart_answers_controller.rb:115:in `smartdown_registry'
    app/controllers/smart_answers_controller.rb:119:in `smartdown_question'
    app/controllers/smart_answers_controller.rb:79:in `find_smart_answer'
    test/functional/smart_answers_controller_test.rb:337:in `block (3 levels) in <class:SmartAnswersControllerTest>'


 29) Error:
SmartAnswersControllerTest#test_: GET /<slug> salary question should show a validation error if invalid period. :
RuntimeError: Unexpected modification of flow registry options, got {:show_drafts=>true, :preload_flows=>true} and had {:preload_flows=>true, :show_drafts=>true, :show_transitions=>true}
    lib/smartdown_adapter/registry.rb:9:in `instance'
    app/controllers/smart_answers_controller.rb:115:in `smartdown_registry'
    app/controllers/smart_answers_controller.rb:119:in `smartdown_question'
    app/controllers/smart_answers_controller.rb:79:in `find_smart_answer'
    test/functional/smart_answers_controller_test.rb:41:in `submit_response'
    test/functional/smart_answers_controller_test.rb:365:in `block (3 levels) in <class:SmartAnswersControllerTest>'


 30) Error:
SmartAnswersControllerTest#test_: GET /<slug> should 503 if content_api times out. :
RuntimeError: Unexpected modification of flow registry options, got {:show_drafts=>true, :preload_flows=>true} and had {:preload_flows=>true, :show_drafts=>true, :show_transitions=>true}
    lib/smartdown_adapter/registry.rb:9:in `instance'
    app/controllers/smart_answers_controller.rb:115:in `smartdown_registry'
    app/controllers/smart_answers_controller.rb:119:in `smartdown_question'
    app/controllers/smart_answers_controller.rb:79:in `find_smart_answer'
    test/functional/smart_answers_controller_test.rb:134:in `block (2 levels) in <class:SmartAnswersControllerTest>'


 31) Error:
SmartAnswersControllerTest#test_: GET /<slug> should accept responses as GET params and redirect to canonical url. :
RuntimeError: Unexpected modification of flow registry options, got {:show_drafts=>true, :preload_flows=>true} and had {:preload_flows=>true, :show_drafts=>true, :show_transitions=>true}
    lib/smartdown_adapter/registry.rb:9:in `instance'
    app/controllers/smart_answers_controller.rb:115:in `smartdown_registry'
    app/controllers/smart_answers_controller.rb:119:in `smartdown_question'
    app/controllers/smart_answers_controller.rb:79:in `find_smart_answer'
    test/functional/smart_answers_controller_test.rb:41:in `submit_response'
    test/functional/smart_answers_controller_test.rb:413:in `block (2 levels) in <class:SmartAnswersControllerTest>'


 32) Error:
SmartAnswersControllerTest#test_: GET /<slug> should cope with no artefact found. :
RuntimeError: Unexpected modification of flow registry options, got {:show_drafts=>true, :preload_flows=>true} and had {:preload_flows=>true, :show_drafts=>true, :show_transitions=>true}
    lib/smartdown_adapter/registry.rb:9:in `instance'
    app/controllers/smart_answers_controller.rb:115:in `smartdown_registry'
    app/controllers/smart_answers_controller.rb:119:in `smartdown_question'
    app/controllers/smart_answers_controller.rb:79:in `find_smart_answer'
    test/functional/smart_answers_controller_test.rb:145:in `block (2 levels) in <class:SmartAnswersControllerTest>'


 33) Error:
SmartAnswersControllerTest#test_: GET /<slug> should display first question after starting. :
RuntimeError: Unexpected modification of flow registry options, got {:show_drafts=>true, :preload_flows=>true} and had {:preload_flows=>true, :show_drafts=>true, :show_transitions=>true}
    lib/smartdown_adapter/registry.rb:9:in `instance'
    app/controllers/smart_answers_controller.rb:115:in `smartdown_registry'
    app/controllers/smart_answers_controller.rb:119:in `smartdown_question'
    app/controllers/smart_answers_controller.rb:79:in `find_smart_answer'
    test/functional/smart_answers_controller_test.rb:107:in `block (2 levels) in <class:SmartAnswersControllerTest>'


 34) Error:
SmartAnswersControllerTest#test_: GET /<slug> should display landing page if no questions answered yet. :
RuntimeError: Unexpected modification of flow registry options, got {:show_drafts=>true, :preload_flows=>true} and had {:preload_flows=>true, :show_drafts=>true, :show_transitions=>true}
    lib/smartdown_adapter/registry.rb:9:in `instance'
    app/controllers/smart_answers_controller.rb:115:in `smartdown_registry'
    app/controllers/smart_answers_controller.rb:119:in `smartdown_question'
    app/controllers/smart_answers_controller.rb:79:in `find_smart_answer'
    test/functional/smart_answers_controller_test.rb:65:in `block (2 levels) in <class:SmartAnswersControllerTest>'


 35) Error:
SmartAnswersControllerTest#test_: GET /<slug> should have cache headers set to 30 mins. :
RuntimeError: Unexpected modification of flow registry options, got {:show_drafts=>true, :preload_flows=>true} and had {:preload_flows=>true, :show_drafts=>true, :show_transitions=>true}
    lib/smartdown_adapter/registry.rb:9:in `instance'
    app/controllers/smart_answers_controller.rb:115:in `smartdown_registry'
    app/controllers/smart_answers_controller.rb:119:in `smartdown_question'
    app/controllers/smart_answers_controller.rb:79:in `find_smart_answer'
    test/functional/smart_answers_controller_test.rb:75:in `block (2 levels) in <class:SmartAnswersControllerTest>'


 36) Error:
SmartAnswersControllerTest#test_: GET /<slug> should have meta robots noindex on question pages. :
RuntimeError: Unexpected modification of flow registry options, got {:show_drafts=>true, :preload_flows=>true} and had {:preload_flows=>true, :show_drafts=>true, :show_transitions=>true}
    lib/smartdown_adapter/registry.rb:9:in `instance'
    app/controllers/smart_answers_controller.rb:115:in `smartdown_registry'
    app/controllers/smart_answers_controller.rb:119:in `smartdown_question'
    app/controllers/smart_answers_controller.rb:79:in `find_smart_answer'
    test/functional/smart_answers_controller_test.rb:119:in `block (2 levels) in <class:SmartAnswersControllerTest>'


 37) Error:
SmartAnswersControllerTest#test_: GET /<slug> should not have noindex tag on landing page. :
RuntimeError: Unexpected modification of flow registry options, got {:show_drafts=>true, :preload_flows=>true} and had {:preload_flows=>true, :show_drafts=>true, :show_transitions=>true}
    lib/smartdown_adapter/registry.rb:9:in `instance'
    app/controllers/smart_answers_controller.rb:115:in `smartdown_registry'
    app/controllers/smart_answers_controller.rb:119:in `smartdown_question'
    app/controllers/smart_answers_controller.rb:79:in `find_smart_answer'
    test/functional/smart_answers_controller_test.rb:70:in `block (2 levels) in <class:SmartAnswersControllerTest>'


 38) Error:
SmartAnswersControllerTest#test_: GET /<slug> should respond with 404 if not found. :
RuntimeError: Unexpected modification of flow registry options, got {:show_drafts=>true, :preload_flows=>true} and had {:preload_flows=>true, :show_drafts=>true, :show_transitions=>true}
    lib/smartdown_adapter/registry.rb:9:in `instance'
    app/controllers/smart_answers_controller.rb:115:in `smartdown_registry'
    app/controllers/smart_answers_controller.rb:119:in `smartdown_question'
    app/controllers/smart_answers_controller.rb:79:in `find_smart_answer'
    test/functional/smart_answers_controller_test.rb:60:in `block (2 levels) in <class:SmartAnswersControllerTest>'


 39) Error:
SmartAnswersControllerTest#test_: GET /<slug> should send slimmer analytics headers. :
RuntimeError: Unexpected modification of flow registry options, got {:show_drafts=>true, :preload_flows=>true} and had {:preload_flows=>true, :show_drafts=>true, :show_transitions=>true}
    lib/smartdown_adapter/registry.rb:9:in `instance'
    app/controllers/smart_answers_controller.rb:115:in `smartdown_registry'
    app/controllers/smart_answers_controller.rb:119:in `smartdown_question'
    app/controllers/smart_answers_controller.rb:79:in `find_smart_answer'
    test/functional/smart_answers_controller_test.rb:139:in `block (2 levels) in <class:SmartAnswersControllerTest>'


 40) Error:
SmartAnswersControllerTest#test_: GET /<slug> should send the artefact to slimmer. :
RuntimeError: Unexpected modification of flow registry options, got {:show_drafts=>true, :preload_flows=>true} and had {:preload_flows=>true, :show_drafts=>true, :show_transitions=>true}
    lib/smartdown_adapter/registry.rb:9:in `instance'
    app/controllers/smart_answers_controller.rb:115:in `smartdown_registry'
    app/controllers/smart_answers_controller.rb:119:in `smartdown_question'
    app/controllers/smart_answers_controller.rb:79:in `find_smart_answer'
    test/functional/smart_answers_controller_test.rb:128:in `block (2 levels) in <class:SmartAnswersControllerTest>'


 41) Error:
SmartAnswersControllerTest#test_: GET /<slug> should show outcome when smart answer is complete so that 'smartanswerOutcome' JS event is fired. :
RuntimeError: Unexpected modification of flow registry options, got {:show_drafts=>true, :preload_flows=>true} and had {:preload_flows=>true, :show_drafts=>true, :show_transitions=>true}
    lib/smartdown_adapter/registry.rb:9:in `instance'
    app/controllers/smart_answers_controller.rb:115:in `smartdown_registry'
    app/controllers/smart_answers_controller.rb:119:in `smartdown_question'
    app/controllers/smart_answers_controller.rb:79:in `find_smart_answer'
    test/functional/smart_answers_controller_test.rb:114:in `block (2 levels) in <class:SmartAnswersControllerTest>'


 42) Error:
SmartAnswersControllerTest#test_: GET /<slug> value question label in translation file should show the label text before the question input. :
RuntimeError: Unexpected modification of flow registry options, got {:show_drafts=>true, :preload_flows=>true} and had {:preload_flows=>true, :show_drafts=>true, :show_transitions=>true}
    lib/smartdown_adapter/registry.rb:9:in `instance'
    app/controllers/smart_answers_controller.rb:115:in `smartdown_registry'
    app/controllers/smart_answers_controller.rb:119:in `smartdown_question'
    app/controllers/smart_answers_controller.rb:79:in `find_smart_answer'
    test/functional/smart_answers_controller_test.rb:264:in `block (5 levels) in <class:SmartAnswersControllerTest>'
    test/helpers/i18n_test_helper.rb:7:in `using_translation_file'
    test/functional/smart_answers_controller_test.rb:263:in `block (4 levels) in <class:SmartAnswersControllerTest>'


 43) Error:
SmartAnswersControllerTest#test_: GET /<slug> value question should accept question input and redirect to canonical url. :
RuntimeError: Unexpected modification of flow registry options, got {:show_drafts=>true, :preload_flows=>true} and had {:preload_flows=>true, :show_drafts=>true, :show_transitions=>true}
    lib/smartdown_adapter/registry.rb:9:in `instance'
    app/controllers/smart_answers_controller.rb:115:in `smartdown_registry'
    app/controllers/smart_answers_controller.rb:119:in `smartdown_question'
    app/controllers/smart_answers_controller.rb:79:in `find_smart_answer'
    test/functional/smart_answers_controller_test.rb:41:in `submit_response'
    test/functional/smart_answers_controller_test.rb:252:in `block (3 levels) in <class:SmartAnswersControllerTest>'


 44) Error:
SmartAnswersControllerTest#test_: GET /<slug> value question should display collapsed question, and format number. :
RuntimeError: Unexpected modification of flow registry options, got {:show_drafts=>true, :preload_flows=>true} and had {:preload_flows=>true, :show_drafts=>true, :show_transitions=>true}
    lib/smartdown_adapter/registry.rb:9:in `instance'
    app/controllers/smart_answers_controller.rb:115:in `smartdown_registry'
    app/controllers/smart_answers_controller.rb:119:in `smartdown_question'
    app/controllers/smart_answers_controller.rb:79:in `find_smart_answer'
    test/functional/smart_answers_controller_test.rb:257:in `block (3 levels) in <class:SmartAnswersControllerTest>'


 45) Error:
SmartAnswersControllerTest#test_: GET /<slug> value question should display question. :
RuntimeError: Unexpected modification of flow registry options, got {:show_drafts=>true, :preload_flows=>true} and had {:preload_flows=>true, :show_drafts=>true, :show_transitions=>true}
    lib/smartdown_adapter/registry.rb:9:in `instance'
    app/controllers/smart_answers_controller.rb:115:in `smartdown_registry'
    app/controllers/smart_answers_controller.rb:119:in `smartdown_question'
    app/controllers/smart_answers_controller.rb:79:in `find_smart_answer'
    test/functional/smart_answers_controller_test.rb:246:in `block (3 levels) in <class:SmartAnswersControllerTest>'


 46) Error:
SmartAnswersControllerTest#test_: GET /<slug> value question suffix_label in translation file should show the label text after the question input. :
RuntimeError: Unexpected modification of flow registry options, got {:show_drafts=>true, :preload_flows=>true} and had {:preload_flows=>true, :show_drafts=>true, :show_transitions=>true}
    lib/smartdown_adapter/registry.rb:9:in `instance'
    app/controllers/smart_answers_controller.rb:115:in `smartdown_registry'
    app/controllers/smart_answers_controller.rb:119:in `smartdown_question'
    app/controllers/smart_answers_controller.rb:79:in `find_smart_answer'
    test/functional/smart_answers_controller_test.rb:276:in `block (5 levels) in <class:SmartAnswersControllerTest>'
    test/helpers/i18n_test_helper.rb:7:in `using_translation_file'
    test/functional/smart_answers_controller_test.rb:275:in `block (4 levels) in <class:SmartAnswersControllerTest>'


 47) Error:
SmartAnswersControllerTest#test_: GET /<slug> without a valid artefact should have cache headers set to 5 seconds. :
RuntimeError: Unexpected modification of flow registry options, got {:show_drafts=>true, :preload_flows=>true} and had {:preload_flows=>true, :show_drafts=>true, :show_transitions=>true}
    lib/smartdown_adapter/registry.rb:9:in `instance'
    app/controllers/smart_answers_controller.rb:115:in `smartdown_registry'
    app/controllers/smart_answers_controller.rb:119:in `smartdown_question'
    app/controllers/smart_answers_controller.rb:79:in `find_smart_answer'
    test/functional/smart_answers_controller_test.rb:90:in `block (3 levels) in <class:SmartAnswersControllerTest>'


 48) Error:
SmartAnswersControllerTest#test_: GET /<slug> without a valid artefact should still return a success response. :
RuntimeError: Unexpected modification of flow registry options, got {:show_drafts=>true, :preload_flows=>true} and had {:preload_flows=>true, :show_drafts=>true, :show_transitions=>true}
    lib/smartdown_adapter/registry.rb:9:in `instance'
    app/controllers/smart_answers_controller.rb:115:in `smartdown_registry'
    app/controllers/smart_answers_controller.rb:119:in `smartdown_question'
    app/controllers/smart_answers_controller.rb:79:in `find_smart_answer'
    test/functional/smart_answers_controller_test.rb:85:in `block (3 levels) in <class:SmartAnswersControllerTest>'

3910 runs, 20780 assertions, 0 failures, 48 errors, 0 skips
Coverage report Rcov style generated for Functional Tests to /Users/chrisroos/Code/alphagov/smart-answers/coverage/rcov
rake aborted!
Command failed with status (1): [ruby -I"lib:test" -I"/Users/chrisroos/Code/alphagov/smart-answers/.bundle/gems/gems/rake-10.4.2/lib" "/Users/chrisroos/Code/alphagov/smart-answers/.bundle/gems/gems/rake-10.4.2/lib/rake/rake_test_loader.rb" "test/functional/duplicate_flow_test.rb" "test/functional/routing_test.rb" "test/functional/smart_answers_controller_test.rb" "test/integration/engine/calendar_export_test.rb" "test/integration/engine/changing_answer_test.rb" "test/integration/engine/checkbox_questions_test.rb" "test/integration/engine/country_and_date_questions_test.rb" "test/integration/engine/data_partials_test.rb" "test/integration/engine/input_validation_test.rb" "test/integration/engine/money_and_salary_questions_test.rb" "test/integration/engine/multi_choice_and_value_questions_test.rb" "test/integration/engine/precalculations_test.rb" "test/integration/engine/report_a_problem_test.rb" "test/integration/healthcheck_test.rb" "test/integration/smart_answer_flows/additional_commodity_code_test.rb" "test/integration/smart_answer_flows/adoption_calculator_test.rb" "test/integration/smart_answer_flows/am_i_getting_minimum_wage_test.rb" "test/integration/smart_answer_flows/am_i_getting_minimum_wage_v2_test.rb" "test/integration/smart_answer_flows/appeal_a_benefits_decision_test.rb" "test/integration/smart_answer_flows/apply_for_probate_test.rb" "test/integration/smart_answer_flows/apply_tier_4_visa_test.rb" "test/integration/smart_answer_flows/benefit_cap_calculator_test.rb" "test/integration/smart_answer_flows/benefits_abroad_test.rb" "test/integration/smart_answer_flows/benefits_if_you_are_abroad_test.rb" "test/integration/smart_answer_flows/calculate_agricultural_holiday_entitlement_test.rb" "test/integration/smart_answer_flows/calculate_child_maintenance_test.rb" "test/integration/smart_answer_flows/calculate_married_couples_allowance_test.rb" "test/integration/smart_answer_flows/calculate_redundancy_pay_test.rb" "test/integration/smart_answer_flows/calculate_redundancy_pay_v3_test.rb" "test/integration/smart_answer_flows/calculate_state_pension_test.rb" "test/integration/smart_answer_flows/calculate_statutory_sick_pay_test.rb" "test/integration/smart_answer_flows/calculate_your_holiday_entitlement_test.rb" "test/integration/smart_answer_flows/check_uk_visa_test.rb" "test/integration/smart_answer_flows/check_uk_visa_v2_test.rb" "test/integration/smart_answer_flows/childcare_costs_for_tax_credits_test.rb" "test/integration/smart_answer_flows/energy_grants_calculator_test.rb" "test/integration/smart_answer_flows/estimate_self_assessment_penalties_test.rb" "test/integration/smart_answer_flows/find_a_british_embassy_test.rb" "test/integration/smart_answer_flows/help_if_you_are_arrested_abroad_test.rb" "test/integration/smart_answer_flows/inherits_someone_dies_without_will_test.rb" "test/integration/smart_answer_flows/legalisation_document_checker_test.rb" "test/integration/smart_answer_flows/marriage_abroad_test.rb" "test/integration/smart_answer_flows/marriage_abroad_v2_test.rb" "test/integration/smart_answer_flows/maternity_calculator_test.rb" "test/integration/smart_answer_flows/minimum_wage_calculator_employers_test.rb" "test/integration/smart_answer_flows/minimum_wage_calculator_employers_v2_test.rb" "test/integration/smart_answer_flows/overseas_passports_test.rb" "test/integration/smart_answer_flows/paternity_calculator_test.rb" "test/integration/smart_answer_flows/pip_checker_test.rb" "test/integration/smart_answer_flows/plan_adoption_leave_test.rb" "test/integration/smart_answer_flows/register_a_birth_test.rb" "test/integration/smart_answer_flows/register_a_death_test.rb" "test/integration/smart_answer_flows/request_for_flexible_working_test.rb" "test/integration/smart_answer_flows/simplified_expenses_checker_test.rb" "test/integration/smart_answer_flows/simplified_expenses_checker_v2_test.rb" "test/integration/smart_answer_flows/state_pension_through_partner_test.rb" "test/integration/smart_answer_flows/state_pension_topup_test.rb" "test/integration/smart_answer_flows/student_finance_calculator_test.rb" "test/integration/smart_answer_flows/towing_rules_test.rb" "test/integration/smart_answer_flows/uk_benefits_abroad_test.rb" "test/integration/smart_answer_flows/uk_benefits_abroad_v2_test.rb" "test/integration/smart_answer_flows/vat_payment_deadlines_test.rb" "test/integration/smartdown_flows/example_plugins_test.rb" "test/integration/smartdown_flows/question_types_test.rb" "test/unit/age_related_allowance_chooser_test.rb" "test/unit/calculators/agricultural_holiday_entitlement_calculator_test.rb" "test/unit/calculators/arrested_abroad_calculator_test.rb" "test/unit/calculators/birth_calculator_test.rb" "test/unit/calculators/child_maintenance_calculator_test.rb" "test/unit/calculators/childcare_cost_calculator_test.rb" "test/unit/calculators/commodity_code_calculator_test.rb" "test/unit/calculators/holiday_entitlement_test.rb" "test/unit/calculators/maternity_benefits_calculator_test.rb" "test/unit/calculators/maternity_paternity_calculator_test.rb" "test/unit/calculators/minimum_wage_calculator_test.rb" "test/unit/calculators/minimum_wage_calculator_v2_test.rb" "test/unit/calculators/night_work_hours_test.rb" "test/unit/calculators/passport_and_embassy_data_query_test.rb" "test/unit/calculators/pip_dates_test.rb" "test/unit/calculators/plan_adoption_leave_test.rb" "test/unit/calculators/plan_maternity_leave_test.rb" "test/unit/calculators/rates_query_test.rb" "test/unit/calculators/redundancy_calculator_test.rb" "test/unit/calculators/registrations_data_query_test.rb" "test/unit/calculators/registrations_data_query_v2_test.rb" "test/unit/calculators/self_assessment_penalties_test.rb" "test/unit/calculators/state_pension_amount_calculator_test.rb" "test/unit/calculators/state_pension_topup_calculator_test.rb" "test/unit/calculators/static_data_query_test.rb" "test/unit/calculators/statutory_sick_pay_calculator_test.rb" "test/unit/calculators/translator_links_test.rb" "test/unit/calculators/vat_payment_deadlines_test.rb" "test/unit/calculators/workplace_pension_calculator_test.rb" "test/unit/calendar_state_test.rb" "test/unit/calendar_test.rb" "test/unit/checkbox_question_test.rb" "test/unit/country_select_question_test.rb" "test/unit/date_helper_test.rb" "test/unit/date_question_test.rb" "test/unit/flow_registration_presenter_test.rb" "test/unit/flow_registry_test.rb" "test/unit/flow_test.rb" "test/unit/friendly_time_diff_test.rb" "test/unit/govspeak_presenter_test.rb" "test/unit/graph_presenter_test.rb" "test/unit/ics_renderer_test.rb" "test/unit/married_couple_allowance_calculator_test.rb" "test/unit/money_question_test.rb" "test/unit/money_test.rb" "test/unit/multiple_choice_question_test.rb" "test/unit/node_presenter_test.rb" "test/unit/optional_date_test.rb" "test/unit/phrase_list_test.rb" "test/unit/predicates_test.rb" "test/unit/question_base_test.rb" "test/unit/salary_question_test.rb" "test/unit/salary_test.rb" "test/unit/smartdown_adapter/plugin_factory_test.rb" "test/unit/smartdown_adapter/presenter_test.rb" "test/unit/smartdown_adapter/registry_test.rb" "test/unit/smartdown_adapter/text_question_presenter_test.rb" "test/unit/smartdown_content/smartdown_scenarios_test.rb" "test/unit/smartdown_plugins/animal-example-multiple_test.rb" "test/unit/smartdown_plugins/extendables/data_partial_test.rb" "test/unit/smartdown_plugins/landlord-immigration-check_test.rb" "test/unit/smartdown_plugins/pay-leave-for-parents-v2_test.rb" "test/unit/smartdown_plugins/pay-leave-for-parents_adoption_test.rb" "test/unit/smartdown_plugins/pay-leave-for-parents_test.rb" "test/unit/state_test.rb" "test/unit/value_question_test.rb" "test/unit/working_days_test.rb" "test/unit/world_location_test.rb" "test/unit/worldwide_organisation_test.rb" --seed=56021]
/Users/chrisroos/Code/alphagov/smart-answers/lib/tasks/minitest.rake:12:in `block in <top (required)>'
Tasks: TOP => test:all
(See full trace by running task with --trace)