gistfile1.txt

15:48:02 -!- Irssi: #ubuntu-mirrors: Total of 87 nicks [8 ops, 0 halfops, 0 voices, 79 normal]
15:48:03 -!- Channel #ubuntu-mirrors created Sun Nov 26 15:42:48 2006
15:48:03 -!- Irssi: Join to #ubuntu-mirrors was synced in 1 secs
15:48:18 < limeburst> good morning
15:50:26 < limeburst> I've been getting some hash sum mismatch errors recently
15:50:59 < limeburst> http://archive.ubuntu.com/ubuntu/dists/xenial-updates/main/binary-amd64/by-hash/SHA256/a0645de208f8bb005d4a3764253b3230209ca29ed2b0fb0acd114ee52b395b19
15:51:16 < limeburst> Today I got a mismatch error at that address
15:51:31 < limeburst> http://archive.ubuntu.com/ubuntu/dists/xenial-updates/main/binary-amd64/by-hash/SHA256/6c97fc80d170dd2153d8c323914968dee6ce7801dc155a35bbb84f6a935590c9
15:52:02 < limeburst> Few days ago at that address, althought that resource gives a 404
15:53:27 < limeburst> I'm located in Korea, and kr.archive.ubuntu.com seems to be providing good files, no hash mismatch
16:06:30 < sarnold> limeburst: the first url gives me the right contents; the second url gives me a 404
16:07:22 < sarnold> limeburst: are you using squid-deb-proxy, or squid, or apt-cacher-ng? I used to get hash-sum mismatches with apt-cahcer-ng all the time and switched to squid-deb-proxy. friends have told me the same story, but switching from squid-deb-proxy to 
                    apt-cacher-ng. ;)
16:10:16 < limeburst> sarnold: In South Korea your government snoops your HTTP traffic ;) but I'm not behind any proxy and caching mechanisms as far as I can tell
16:12:24 < sarnold> limeburst: hrm, the site that I've used before to help people find intrusive caches has died :(
16:14:50 < limeburst> ah, I just tried diffing the hash matching and the mismatching a0645
16:14:51 < limeburst> 63475c63475
16:14:51 < limeburst> < Phased-Update-Percentage: 20
16:14:51 < limeburst> ---
16:14:51 < limeburst> > Phased-Update-Percentage: 30
16:16:12 < limeburst> https://bugs.launchpad.net/ubuntu/+bug/1459618
16:16:59 < limeburst> https://wiki.ubuntu.com/PhasedUpdates
16:18:26 < sarnold> limeburst: oh my.
16:18:48 < sarnold> limeburst: you've got both a good file and a bad file and the only difference after decompressing them is the phased update percentage for  apackage?
16:18:59 < limeburst> yeah
16:19:46 < limeburst> Is that the intended behavior? intentionally breaking apt-get update?
16:20:47 < limeburst> if it is intended, then so much for 'StableReleaseUpdates'
16:21:08 < sarnold> limeburst: please attach both files to that bug
16:21:25 < sarnold> wgrant: ^^^ any suggestions who to assign this to?
16:25:03 < wgrant> sarnold, limeburst: It's intended behaviour that packages files may differ only by Phased-Update-Percentage. Those files are named by hash, so if you're getting the wrong hash then your ISP is buggy.
16:25:11 < wgrant> There's not much we can do about that -- you'll need to complain to your service provider.
16:26:36 < sarnold> wgrant: so you're reasonably convinced this is transparent proxy gone awry?
16:26:52 < wgrant> sarnold: There's no viable mechanism by which the wrong content could appear on our side.
16:26:59 < limeburst> Are those files ever published with differing percentage with the same filename?
16:27:29 < sarnold> wgrant: pfew :) that's a relief
16:27:53 < wgrant> limeburst: That's why the by-hash mechanism was introduced ~xenial. The Release file specifes the hash, and then the client downloads from the by-hash dir so there is no race.
16:28:07 < wgrant> limeburst: a0645de... is the SHA-256 of the file contents.
16:28:26 < limeburst> I'm aware of that
16:28:46 < wgrant> (so no, the content can't change without the filename changing, unless we've managed to find a vulnerability in SHA-256...)
16:29:07 < limeburst> But I still find it strange that my ISP or whatever would change just those Phased-Update-Percentage lines
16:29:28 < limeburst> and highly improbable
16:30:22 -!- KingPin [kingpin@bela.kpsn.org] has quit [Ping timeout: 268 seconds]
16:30:37 < wgrant> limeburst: My own ISP has done similar things in the past.
16:30:58 < wgrant> limeburst: If the file size was the same, the file was over 5MiB, and the first 8KiB were identical, it matched an existing cache regardless of filename(!) or domain(!!!)
16:31:59 < wgrant> (I noticed it with Ubuntu cloud-images, which of course start out very similarly and are of fixed size)
16:34:10 -!- Spads [spacehobo@unaffiliated/spads] has joined #ubuntu-mirrors
16:34:11 < limeburst> now that looks like a sad, but a very good explanation
16:34:13 < wgrant> So it's not completely outside the realms of possibility that your ISP is almost as dodgy as mine, and decided the file looked similar enough to one it had seen previously.
16:34:34 < sarnold> wgrant: holy cow, that's crazyness
16:34:39 < wgrant> Yes, yes it was.