Enabling Solr Document Versioning.md

Enabling Solr Document Versioning

1. What is it?
   • Optimistic concurrency / Document centric versioning constraints
   • Solr rejects older documents with a 409 error.
2. Why?
   • If you are ingesting multiple versions of a document but you don't have a guarantee of the order in which they will be ingested.
3. How?
   • Simply add some configuration to your solrconfig.xml

       <config>
     
       	<!-- ... -->
     
       	<updateRequestProcessorChain name="DocCentricVersioningOnDate">
       		<processor class="solr.ParseDateFieldUpdateProcessorFactory">
       			<str name="defaultTimeZone">Etc/UTC</str>
       			<arr name="format">
       				<str>yyyy-MM-dd HH:mm:ss Z</str>
       				<str>yyyy-MM-dd HH:mm:ss</str>
       			</arr>
       		</processor>
       		<processor class="solr.DocBasedVersionConstraintsProcessorFactory">
       			<str name="versionField">record_update_date</str>
       			<bool name="ignoreOldUpdates">false</bool>
       		</processor>
       		<processor class="solr.LogUpdateProcessorFactory" />
       		<processor class="solr.RunUpdateProcessorFactory" />
       	</updateRequestProcessorChain>
     
       </config>
     

   • Make sure to index configured field with appropriate format
   • This is a good opportunity to fine tune the meaning of what is considered the latest update time
4. ✨ ✨ Demo ✨ ✨
5. Caveats and mitigations:
   • Solr errors out do to max skipped.
     • Allow unlimited skips provide "solr_writer.max_skipped", -1
   • But there's still a performance hit and traject still exits with -1:
     • Or use custom SolrJsonWriter: provide "writer_class_name", "CobIndex::SolrJsonWriter"