error_resolver.md

Error Resolver

The error resolver process is triggered when an erroneous Imports::Line is updated with a group of resolved_attributes.

The resolved_attributes contain values which should have been computed by the parser, but the parser was unable to compute these values and produced an Imports::Error instead.

The interface will allow the user to resolve one or multiple attributes related to a common Imports::Line, the Error Resolver will treat those resolved_attributes separately.

The main purposes of the error resolver is to :

• break a Imports::Line fix into several Imports::Error fixes
• find all the Imports::Error similar to the one being fixed
• fix all those errors by :
  • updating the relevant property on their associated Imports::Line
  • delete the error
  • recompute the error counters on their associated Imports::Line
  • update the status of their associated Imports::Line if there is no more error
• recompute the status of the Import record (currently done in Imports::Line::Update )

This must be done in an acceptable delay (less than 1sec) on imports associated with 500 000 lines. Looping over each line is therefore not an acceptable approach.

Blank values : do not propagate

If the initial raw value of an attribute is "blank", fixing the error must not be propagated to other lines have the same "blank" value.

blank values are :

• """ (an empty string)
• /\s+/ (a string formed of only blank characters)
• nil (the absence of a value)
• "Unknown" (case insensitive)
• "N/A" (case insensitive)

Similar errors :

When providing a resolved attribute, we currently try to guess what could have been the initial error. This is due to the fact that several different errors could lead to the same attribute being resolved.

For instance : channel_slug_not_found and channel_slug_combination_not_found both lead to a resolved attribute channel_slug.

Sometimes we also do "partial resolution", for instance when a period_start and a period_end are missing, and we fix only one of the two.

This strategy has created a deep fragility and I suggest changing it to a smaller list of errors, each of which must have a predetermined type of resolution, possibly composed of multiple values.

By doing so, the resolution could target specifically which error is being resolved and finding the similar errors would mean finding the other error of the exact same kind as the one being resolved with the exact same error_value.

Suggested new list of error and resolutions

• sellable_not_found : [sellable_type, sellable_id]

• channel_not_found : [channel_slug]

• country_not_found : [country_code]

• currency_not_found : [currency_code]

• invalid_quantity : [quantity]

• invalid_period : [period_start_at, period_end_at]

• invalid_gross_sdrm_amount : [gross_sdrm_amount_subunit]

• invalid_net_received_amount : [net_received_amount_subunit]

• invalid_ppd_amount : [ppd_amount_subunit]

• invalid_retail_price_amount : [retail_price_amount_subunit]

• invalid_unit_price_gross_amount : [unit_price_gross_amount_subunit]

Failure

If a fix cannot be applied, the whole error resolver process must fail and no change must have occured in database. The server must return an error code and an error message which will be handled by the UI.

Attribute behaviour : Breakdown

sellable_not_found

The sellable association of the line must be updated with the provided sellable found with [sellable_type, sellable_id] If no Sellable is found, the whole process must fail.

Possible sellable includes : Release, Track, Youtube Videos, Merchandising item

This fix is propagatable to similar errors

channel_not_found

The channel_slug field of the line must be updated with the provided [channel_slug] If the provided value is not valid, the whole process must fail

Valid slugs are found in Sales::ChannelType :

• 'digital-global'
• 'streaming'
• 'download'
• 'ringtone'
• 'physical-global'
• 'cd'
• 'vinyl'
• 'dvd'
• 'cassette'
• 'compilation-global'
• 'intern-compilation'
• 'extern-compilation'
• 'neighboring-rights'
• 'video'
• 'direct-sales-to-consumer'
• 'artist-sales'
• 'synchronization'
• 'kiosk-sales'
• 'clubs-sales'
• 'premium-sales'
• 'budget-sales'
• 'mid-sales'
• 'discount-sales'
• 'mail-order-sales'
• 'sample'
• 'merchandising'
• 'stems'
• 'special-products'
• 'brands'

This fix is propagatable to similar errors

country_not_found

The country field of the line must be updated with the provided country If the provided value is not valid, the whole process must fail.

This fix is propagatable to similar errors

currency_not_found

The original_currency field of the line must be updated with the provided currency_code If the provided value is not valid, the whole process must fail.

This fix is propagatable to similar errors

IMPORTANT : The missing currency has prevented all the amount parsers to run. Therefore, fixing the currency and deleting those errors may leave a line record with no error but still in an invalid state which could only be revealed by running the AmountParser.

For this reason, the error resolver cannot set a processed status to all the lines without error after having fixed a bunch of errors. The error resolver must rerun the line through a parsing job.

This parsing job should skip all the already set attributes, because they may be the result of manual fixing.

invalid_quantity

The quantity field of the line must be updated with the provided quantity If the provided value is not valid, the whole process must fail.

This fix is NOT propagatable to similar errors

invalid_period

The period_start_at and period_end_at fields of the line must be updated with the provided [period_start_at, period_end_at] If any of the provided value are not valid, the whole process must fail

This fix is NOT propagatable to similar errors

invalid_xxxx_amount :

The amount field must be updated according to provided xxx_amount_subunit and line.original_currency ! If the provided value is not valid, the whole process must fail.

This fix is NOT propagatable to similar errors