Skip to content

Instantly share code, notes, and snippets.

@sandbergja

sandbergja/notes

Created October 14, 2022 23:49
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save sandbergja/47bdc5fa46e767aa21ebe6ce6abdd30f to your computer and use it in GitHub Desktop.
Save sandbergja/47bdc5fa46e767aa21ebe6ce6abdd30f to your computer and use it in GitHub Desktop.
Download ZIP
Notes from 2022-10-14 code review of queued ingest
Raw
notes
ingest.queued.abort_on_error - this is useful when you are working directly in the database (like if you are making a large change to indexing and want to try it out until you hit an error). However, if you are working with the coordinator, it will justt log it at a different level
ingest.queued.all= true is the same as ingest.queued.biblio.all=true and ingest.queued.authority.all=true. These are additive, you can't take them away by, for example, ingest.queued.all= true and ingest.queued.authority.all=false.
The authority queueing (ingest.queued.authority.all and ingest.queued.authority.propagate) could be especially helpful for sites that do a lot of authority loading
ingest.queued.*.*.* are specific exceptions to the rules (like ingest.queued.biblio.insert.marc_edit_inline is an exception to ingest.queued.biblio.insert)
it seems like it might be helpful to have a diagram of the decision tree on these global flags
What should somebody do if they see failures?
* Enter a transaction
* Set the global flags needed
* Try the reingest function in the db, you will get really nice detailed feedback that way
What is the best way to monitor this? This feature gives us more visibility of the ingest errors, how do we make the most of these data for observability?
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment