Skip to content

Instantly share code, notes, and snippets.

@RSijelmass

RSijelmass/gist:d1a1f6a849a57e4d03c3532e1b44d22c

Last active January 17, 2023 10:32
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save RSijelmass/d1a1f6a849a57e4d03c3532e1b44d22c to your computer and use it in GitHub Desktop.
Save RSijelmass/d1a1f6a849a57e4d03c3532e1b44d22c to your computer and use it in GitHub Desktop.
Download ZIP
Investigation Retried Top Ups in Failed/Succeeded State
Raw
gistfile1.md

Potentially Paid Top Ups That aren't on the Card

TL;DR

We have recently moved some scheduled tasks to a new scheduling system in Sidekiq, where we accidentally passed in the top_up_id for failed top ups that we wanted to retry. By passing the existing failed top up ID it meant that we were retrying a top up that was already in a failed state instead of initiating a new top up object. By retrying a failed top up it meant that the top up could never move out of a top_up_failed state, but the payment was still attempted.

This failed top up had a new payment instance created for it that has overwritten the old one. This payment seems to be in an odd state of processing_payment, and it's currently unclear whether a) that payment has been taken from the user without giving them a top up, and if so b) how we may give the user a top up without charging them again (possible solution: goodwill top ups) and placing the attached payment in a correct state.

There seem to be at time of writing 1,014 top ups in a failed state with a processing_payment payment attached; breakdown can be found here.

Breakdown

For the top ups I’ve had a look at in the Rollbar they all have:

  1. at least one payment decision attached that says attempt_payment
  2. a payment attached that has a state of processing_payment → it’s a bit of a ? what this means as it seems to be stuck in a transient state. Does this mean the payment is taken, or not?

Example payment:

=> #<CleoBank::Payment:0x00007f14ecaf0a98
 id: "61ded1e8-54c1-42e9-b6fc-e9fdbd2dd36d",
 user_id: 4279161,
 payment_wallet_transaction_id: 87552010,
 transfer_wallet_transaction_id: nil,
 owner_type: "CleoBank::TopUp",
 owner_id: "fef10551-8b5d-4de4-be48-e89edf0eac34",
 amount_in_cents: 1000,
 created_at: Tue, 13 Dec 2022 15:00:29.456914000 UTC +00:00,
 updated_at: Tue, 13 Dec 2022 15:01:23.163219000 UTC +00:00,
 state: "processing_payment",
 transfer_amount_in_cents: 1000,
 payment_succeeded_at: nil>

To answer the above questions I’m now looking at the wallet transactions attached to the payment:

[cleo][PRODUCTION] main:0> payment.payment_wallet_transaction
=> #<WalletTransaction:0x00007f4c146b9d08
 id: 87552010,
 wallet_id: 4271100,
 wallet_provider_ref: "ch_3MEZyvG1pNatukaD0DrGUgUy",
 amount: 0.1e2,
 fees: 0.0,
 transaction_type: "direct_transfer",
 status: "SUCCEEDED",
 result_code: "authorized/",
 result_message: "Payment complete.",
 created_at: Tue, 13 Dec 2022 15:01:20.966485000 UTC +00:00,
 updated_at: Tue, 13 Dec 2022 15:01:23.133581000 UTC +00:00,
 exchange_rate: 0.0,
 fees_currency: "USD",
 category: "credit_builder_card_payment",
 initiator_id: nil,
 initiator_type: nil,
 tag: "credit_builder_card_payment",
 sent_at: Tue, 13 Dec 2022 15:01:20.966125000 UTC +00:00,
 currency_code: "USD",
 subscription_id: nil,
 payment_method_fingerprint: nil,
 loan_id: nil,
 extra: nil,
 payment_source_type: "CARD",
 payment_destination_type: "BALANCE",
 payment_source_ref: {"external_id"=>"pm_1LPwGeG1pNatukaDF9L4XpIM"},
 payment_destination_ref: {"external_id"=>"master-account"},
 refund_reason_code: nil,
 disputed_reason: nil,
 status_updated_at: Tue, 13 Dec 2022 15:01:23.133402000 UTC +00:00,
 reward_transaction_id: nil,
 user_id: 4279161>

[cleo][PRODUCTION] main:0> payment.transfer_wallet_transaction
=> nil

So it seems that the payment has:

  • a payment_wallet_transaction attached with a SUCCEEDED status, and
  • no transfer_wallet_transaction.

The async submission log attached to the payment however seems to be in a failed state, meaning that the PaymentWalletTransactionListener won’t listen to any updates and won’t move the payment into a different state. So they’ll kind of forever be stuck in this processing_payment, and now I’m still not sure if the payment has actually been taken from their account.

Async submission log:

[cleo][PRODUCTION] main:0> payment.async_submission_log_for_payment
=> #<AsyncSubmissionLog:0x00007f4c09fdf258
 id: "810f9803-cd18-4ad3-a6e5-ca565bf73626",
 category: "credit_builder_card_payment",
 status: "failed",
 user_id: 4279161,
 extra:
  {"errors"=>
    [{"code"=>"yep_that_didn_t_work_mind_giving_that_another_go",
      "field"=>"base",
      "detail"=>"Yep, that didn’t work. Mind giving that another go 🙏",
      "source"=>{"pointer"=>"/data/attributes/base"},
      "status"=>422,
      "full_code"=>"base_yep_that_didn_t_work_mind_giving_that_another_go"}],
   "job_name"=>"CleoBank::CreditBuilderCard::ExecuteCardPaymentWorker",
   "failure_reason"=>"exception",
   "queue_override"=>"bulk_enqueue_processing",
   "ip_address_event_id"=>nil,
   "cleo_bank_payment_id"=>"61ded1e8-54c1-42e9-b6fc-e9fdbd2dd36d"},
 created_at: Tue, 13 Dec 2022 15:00:29.461798000 UTC +00:00,
 updated_at: Tue, 13 Dec 2022 15:01:23.183431000 UTC +00:00,
 submittable_type: "CleoBank::Payment",
 submittable_id: "61ded1e8-54c1-42e9-b6fc-e9fdbd2dd36d",
 parent_id: "4103effc-fba3-4a7a-be3e-335c156347ac",
 position: 0>

Granting the right users a goodwill top up

In the end the conclusion is that yes, we have taken a user's payment, but no, we have not granted them a top up. For this reason we have summed up the total value each user is due, considering that the linked wallet transaction also has a SUCCEEDED status attached. The full list of user IDs can be found below, sorted by amount to return in dollars:

1470077,750.00000000
5718705 1930699,225.00000000
6329398 5982186 6205847 5965947 5494193 4880843 6194686,150.00000000
6160285 5539275 4942634 6223223 5839972 5395753,100.00000000
6162188 4952625 6268506 6091613 5385697 4905040 6032714 5834193 6328885 6320607 5296061 6172975,90.00000000
1421198 2036970 3799418 4907145 3109435 5623063 5375747,75.00000000
5400547 6093605 5051581 5364614 6326158 5875727 2910083 6138786 6202508 2479208 6181515 6149936 4268034 6246530 2239508 4488110 5967518 4337447 5638955 3627503,60.00000000
6339374 4270890 6320135 5925736 6355649 5918842 6100003 3855983 4840548 5134513 6304850,50.00000000
6268223 4734879 4370122,40.00000000
5519354 2355544 6232471 4233859 5983928 5360952 6333181 6346108 1388788 2974755 5977019 5915021 5907389 6236468 4390513 6330428 5881638 5236786 5821086 5998523 4177872 5108367 6280169 3726370 6123975 5937691 6265670 5165087 5667046 5648791 6033937 4247580 6174763 4851995 6003158,30.00000000
6219390,25.00000000
6344276 4428560 6298091 6327714 5110682 1786294 6096275 6354388 1280879 6037878 6308328 3809544 6073557 5340511 6046616 6069028 6101797 6265122,20.00000000
6141027,15.00000000
6267043 5809493 4131908 6155461 5049603 6172864 5225397 4279161 5556159 6099842 4853650 4602740 6300357 4660445 5248527 5866526 6313807 6121292 5785539 5222318 6314286 5902371 1334263 6115607 6312720 5752380 6019703 4703284 5941162 6194061 2685986 6347091,10.00000000
6178090,5.00000000
4751992,3.00000000
5882741,2.00000000
5555406 5954676 3534856 6085340 6183503,1.00000000

Results

When having a look at the above users, one can see that they have all now a goodwill top up with the associated top up value granted on the 20th of December when thist work was done. As an example, we can see user 5718705 who should have received a top up of $225 - they have a goodwill top up with the reason "correctly attributing retried top ups".

This should be enough confirmation that the reconciliation has been completed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment