wiki-error.md

From the debug log:

[DBQuery] LocalFileDeleteBatch::addOlds [0.005s] mw-gw1-db-cluster.aws.arena.net: SELECT  oi_archive_name  FROM `oldimage`    WHERE oi_name = 'Ascalon_child_f.jpg'  
[DBQuery] LocalFileDeleteBatch::getHashes [0.001s] mw-gw1-db-cluster.aws.arena.net: SELECT  oi_archive_name,oi_sha1  FROM `oldimage`    WHERE oi_archive_name IN ('20100713135207!Ascalon_child_f.jpg','20100713135214!Ascalon_child_f.jpg','','20100713135240!Ascalon_child_f.jpg','20100714213745!Ascalon_child_f.jpg','20200720171715!Ascalon_child_f.jpg')  AND oi_name = 'Ascalon_child_f.jpg'  
[FileOperation] FileBackendStore::ingestFreshFileStats: Could not stat file mwstore://local-backend/local-public/archive/6/69/
[DBQuery] FileDeleteForm::doDelete [0.029s] mw-gw1-db-cluster.aws.arena.net: ROLLBACK

Now, running the above two SQL queries:

mysql> SELECT  oi_archive_name  FROM `oldimage`    WHERE oi_name = 'Ascalon_child_f.jpg';
+------------------------------------+
| oi_archive_name                    |
+------------------------------------+
| 20100713135207!Ascalon_child_f.jpg |
| 20100713135214!Ascalon_child_f.jpg |
|                                    |
| 20100713135240!Ascalon_child_f.jpg |
| 20100714213745!Ascalon_child_f.jpg |
| 20200720171715!Ascalon_child_f.jpg |
+------------------------------------+
6 rows in set (0.00 sec)

mysql> SELECT  oi_archive_name,oi_sha1  FROM `oldimage`    WHERE oi_archive_name IN ('20100713135207!Ascalon_child_f.jpg','20100713135214!Ascalon_child_f.jpg','','20100713135240!Ascalon_child_f.jpg','20100714213745!Ascalon_child_f.jpg','20200720171715!Ascalon_child_f.jpg')  AND oi_name = 'Ascalon_child_f.jpg';
+------------------------------------+---------------------------------+
| oi_archive_name                    | oi_sha1                         |
+------------------------------------+---------------------------------+
| 20100713135207!Ascalon_child_f.jpg | 8lq44q4npa98fh65lgy8k9715lsg95g |
| 20100713135214!Ascalon_child_f.jpg | 81fn5ilr0zuf40x1164dkhl1kws52ea |
|                                    | 81fn5ilr0zuf40x1164dkhl1kws52ea |
| 20100713135240!Ascalon_child_f.jpg | 81fn5ilr0zuf40x1164dkhl1kws52ea |
| 20100714213745!Ascalon_child_f.jpg | 81fn5ilr0zuf40x1164dkhl1kws52ea |
| 20200720171715!Ascalon_child_f.jpg | cef8mn9b045xs8alvcblzbfhzizwnu5 |
+------------------------------------+---------------------------------+
6 rows in set (0.00 sec)

I've verified that the above filenames are the only ones that exist in $IP/images/archive/6/69:

/var/www/images/gww-en/archive/6/69 # ls -l *Ascalon_child_f.jpg
-rw-r--r-- 1 www-data www-data 12524 Jun 19 07:18 '20100713135207!Ascalon_child_f.jpg'
-rw-r--r-- 1 www-data www-data 11621 Jul 20 11:37 '20100713135214!Ascalon_child_f.jpg'
-rw-r--r-- 1 www-data www-data 11621 Jun 19 07:18 '20100713135240!Ascalon_child_f.jpg'
-rw-r--r-- 1 www-data www-data 11621 Jun 19 07:18 '20100714213745!Ascalon_child_f.jpg'
-rw-r--r-- 1 www-data www-data 32582 Jul 20 10:17 '20200720171715!Ascalon_child_f.jpg'

Thus, Could not stat file mwstore://local-backend/local-public/archive/6/69/ is really trying to stat a file with a blank filename and failing, causing the overall failure of the image page deletion.

So looking at the first four entries for this image's file history, each entry's oi_timestamp is in the oi_archive_name of the preceding entry (except for the first). So it seems that if I were to update the 3rd entry with the blank oi_archive_name to be a new name that falls between the first two, and update the 4th entry's timestamp to match the value in the 3rd's oi_archive_name, that may be enough to fix it, assuming there aren't other references in the database that this could mess up.

Note that the 3rd and 4th entries are identical right now except for the oi_archive_name, so the timestamp would need to be distinct to prevent a duplicate row, as I'm not sure if that would break things in a different way.

mysql> select * from oldimage where oi_name = 'Ascalon_child_f.jpg'\G
*************************** 1. row ***************************
          oi_name: Ascalon_child_f.jpg
  oi_archive_name: 20100713135207!Ascalon_child_f.jpg
          oi_size: 12524
         oi_width: 127
        oi_height: 475
          oi_bits: 8
oi_description_id: 611939
         oi_actor: 43819
     oi_timestamp: 20100713135113
      oi_metadata: 0
    oi_media_type: BITMAP
    oi_major_mime: image
    oi_minor_mime: jpeg
       oi_deleted: 0
          oi_sha1: 8lq44q4npa98fh65lgy8k9715lsg95g
*************************** 2. row ***************************
          oi_name: Ascalon_child_f.jpg
  oi_archive_name: 20100713135214!Ascalon_child_f.jpg
          oi_size: 11621
         oi_width: 127
        oi_height: 475
          oi_bits: 8
oi_description_id: 611939
         oi_actor: 43819
     oi_timestamp: 20100713135207
      oi_metadata: 0
    oi_media_type: BITMAP
    oi_major_mime: image
    oi_minor_mime: jpeg
       oi_deleted: 0
          oi_sha1: 81fn5ilr0zuf40x1164dkhl1kws52ea
*************************** 3. row ***************************
          oi_name: Ascalon_child_f.jpg
  oi_archive_name: 
          oi_size: 11621
         oi_width: 127
        oi_height: 475
          oi_bits: 8
oi_description_id: 611939
         oi_actor: 43819
     oi_timestamp: 20100713135214
      oi_metadata: 0
    oi_media_type: BITMAP
    oi_major_mime: image
    oi_minor_mime: jpeg
       oi_deleted: 0
          oi_sha1: 81fn5ilr0zuf40x1164dkhl1kws52ea
*************************** 4. row ***************************
          oi_name: Ascalon_child_f.jpg
  oi_archive_name: 20100713135240!Ascalon_child_f.jpg
          oi_size: 11621
         oi_width: 127
        oi_height: 475
          oi_bits: 8
oi_description_id: 611939
         oi_actor: 43819
     oi_timestamp: 20100713135214
      oi_metadata: 0
    oi_media_type: BITMAP
    oi_major_mime: image
    oi_minor_mime: jpeg
       oi_deleted: 0
          oi_sha1: 81fn5ilr0zuf40x1164dkhl1kws52e