1_bad.md

Running with patch

$ QUEUE_ADAPTER=async ./fix-preview-image-race-condition.rb 

Yields

Creating a post and uploading a video
Setting queue adapter to async
================================================================================

Record counts before doing anything (created after 2024-01-31 18:21:12 UTC):
  - 0 Posts []
  - 0 Visuals []
  - 0 ActiveStorage::Blobs []
  - 0 ActiveStorage::Attachments []
  - 0 ActiveStorage::VariantRecords []

================================================================================

Record counts after attaching '/Users/justin/code/searls/grog/test/fixtures/files/video.mp4' (created after 2024-01-31 18:21:12 UTC):
  - 1 Post [39]
  - 1 Visual [44]
  - 1 ActiveStorage::Blob [168]
  - 1 ActiveStorage::Attachment [152]
  - 0 ActiveStorage::VariantRecords []

================================================================================

Record counts after letting preprocess jobs run (created after 2024-01-31 18:21:12 UTC):
  - 1 Post [39]
  - 1 Visual [44]
  - 5 ActiveStorage::Blobs [168, 169, 170, 171, 172]
  - 5 ActiveStorage::Attachments [152, 153, 154, 155, 156]
  - 2 ActiveStorage::VariantRecords [79, 80]

================================================================================

Record counts after destroying post (created after 2024-01-31 18:21:12 UTC):
  - 0 Posts []
  - 0 Visuals []
  - 5 ActiveStorage::Blobs [168, 169, 170, 171, 172]
  - 4 ActiveStorage::Attachments [153, 154, 155, 156]
  - 2 ActiveStorage::VariantRecords [79, 80]

================================================================================

Record counts after letting purge jobs run (created after 2024-01-31 18:21:12 UTC):
  - 0 Posts []
  - 0 Visuals []
  - 2 ActiveStorage::Blobs [170, 172]
  - 2 ActiveStorage::Attachments [154, 156]
  - 1 ActiveStorage::VariantRecord [80]

As the above shows, 5 database records remain after purge (and 2 blobs are left in storage).

2_good.md

Running with patch

$ QUEUE_ADAPTER=async ./fix-preview-image-race-condition.rb

Yields

Creating a post and uploading a video
Setting queue adapter to async
================================================================================

Record counts before doing anything (created after 2024-01-31 17:11:50 UTC):
  - 0 Posts []
  - 0 Visuals []
  - 0 ActiveStorage::Blobs []
  - 0 ActiveStorage::Attachments []
  - 0 ActiveStorage::VariantRecords []

================================================================================

Record counts after attaching '/Users/justin/code/searls/grog/test/fixtures/files/video.mp4' (created after 2024-01-31 17:11:50 UTC):
  - 1 Post [38]
  - 1 Visual [43]
  - 1 ActiveStorage::Blob [166]
  - 1 ActiveStorage::Attachment [150]
  - 0 ActiveStorage::VariantRecords []

================================================================================

Record counts after letting preprocess jobs run (created after 2024-01-31 17:11:50 UTC):
  - 1 Post [38]
  - 1 Visual [43]
  - 2 ActiveStorage::Blobs [166, 167]
  - 2 ActiveStorage::Attachments [150, 151]
  - 0 ActiveStorage::VariantRecords []

================================================================================

Record counts after destroying post (created after 2024-01-31 17:11:50 UTC):
  - 0 Posts []
  - 0 Visuals []
  - 2 ActiveStorage::Blobs [166, 167]
  - 1 ActiveStorage::Attachment [151]
  - 0 ActiveStorage::VariantRecords []

================================================================================

Record counts after letting purge jobs run (created after 2024-01-31 17:11:50 UTC):
  - 0 Posts []
  - 0 Visuals []
  - 0 ActiveStorage::Blobs []
  - 0 ActiveStorage::Attachments []
  - 0 ActiveStorage::VariantRecords []

As the above shows, the purge jobs are able to remove everything

fix-preview-image-race-condition.rb

#!/usr/bin/env ruby

require_relative "../config/environment"

puts "Creating a post and uploading a video"

now = Time.zone.now
PRE_EXISTING_IDS = {
  ActiveStorage::VariantRecord => ActiveStorage::VariantRecord.all.map(&:id)
}

QUEUE_ADAPTER = Rails.application.config.active_job.queue_adapter = if ENV["QUEUE_ADAPTER"]
                  puts "Setting queue adapter to #{ENV["QUEUE_ADAPTER"]}"
                  ENV["QUEUE_ADAPTER"].to_sym
                else
                  puts "Using test queue adapter"
                  :test
                end

def hr
  "=" * 80
end

def record_counts_after(time)
  [
    Post,
    Visual,
    ActiveStorage::Blob,
    ActiveStorage::Attachment,
    ActiveStorage::VariantRecord
  ].map { |klass|
    rel = if PRE_EXISTING_IDS.key?(klass)
      klass.where.not(id: PRE_EXISTING_IDS[klass]).order("id ASC")
    else
      klass.where("created_at > ?", time).order("created_at ASC")
    end
    "  - #{rel.count} #{klass.name.pluralize(rel.count)} [#{rel.map(&:id).join(", ")}]"
  }.join("\n")
end

def print_summary(title, time)
  puts <<~MSG
    #{hr}

    Record counts #{title} (created after #{time}):
    #{record_counts_after(time)}

  MSG
end

def blob_for(file_path)
  file = Pathname.new(file_path)
  ActiveStorage::Blob.create_before_direct_upload!(
    key: nil,
    filename: file_path,
    byte_size: file.size,
    checksum: OpenSSL::Digest::MD5.file(file).base64digest,
    content_type: Marcel::MimeType.for(file),
    record: nil
  ).tap do |blob|
    ActiveStorage::Blob.service.upload(blob.key, file.open)
  end
end

def run_test_enqueued_jobs!
  puts ActiveJob::Base.queue_adapter.enqueued_jobs.each { |job|
    args = job[:args].map { |arg|
      if arg.is_a?(Hash) && arg.key?("_aj_globalid")
        GlobalID::Locator.locate(arg["_aj_globalid"])
      elsif arg.is_a?(Hash) && arg.key?("_aj_symbol_keys")
        arg.without("_aj_symbol_keys")
      else
        arg
      end
    }
    job[:job].new.perform(*args)
  }
end

def clear_test_enqueued_jobs!
  ActiveJob::Base.queue_adapter.enqueued_jobs.clear
end

print_summary("before doing anything", now)

user = User.first

file = ENV["FILE"] || Rails.root.join("test/fixtures/files/video.mp4")
post = Post.create!(
  user: user,
  published_at: now,
  slug: "my-first-post",
  title: "My first post",
  caption: "This is my first post",
  visuals: [
    Visual.new(file: blob_for(file))
  ]
)

print_summary("after attaching '#{file}'", now)

if QUEUE_ADAPTER == :test
  run_test_enqueued_jobs!
  clear_test_enqueued_jobs!
elsif QUEUE_ADAPTER != :inline
  sleep 5
  print_summary("after letting preprocess jobs run", now)
end

if ENV["INSPECT"]
  puts "Hitting debugger to inspect state before destroying post"
  binding.irb
end

post.destroy!
print_summary("after destroying post", now)

if QUEUE_ADAPTER == :test
  run_test_enqueued_jobs!
  clear_test_enqueued_jobs!
elsif QUEUE_ADAPTER != :inline
  sleep 5
end
print_summary("after letting purge jobs run", now)

if ENV["INSPECT"]
  blobs = ActiveStorage::Blob.where("created_at > ?", now)
  attachments = ActiveStorage::Attachment.where("created_at > ?", now)
  variant_records = ActiveStorage::VariantRecord.where.not(id: PRE_EXISTING_IDS[ActiveStorage::VariantRecord])
  binding.irb
end