Small concise report of my experience with BTRFS filesystem recovery so I do "remember" it if I need in the future.
I'm in no way a btrfs expert. What's written here is what I got trying to figure out an incident with my data; it worked for me but it may be wrong and it may eat your data.
After a few power oscilation / a few sequential outages, Linux fail to boot due failure to mount BTRFS filesystem.
Error message on mount attempt:
[132657.733892] BTRFS warning (device bcache0): checksum verify failed on 3706342326272 wanted 0x60b2b4ff found 0x9a653891 level 1
[132657.733941] BTRFS warning (device bcache0): couldn't read tree root
[132657.739490] BTRFS error (device bcache0): open_ctree failed
Filesystem is over a bcache device, with a ssd cache in writethrough mode and a software RAID5.
Getting the non-corrupted data from the filesystem was possible by using the procedure describe in this page from the btrfs wiki (ironically described as deprecated): https://btrfs.wiki.kernel.org/index.php/Restore
tl; dr:
$ sudo btrfs-find-root /dev/bcache0
Couldn't read tree root
Superblock thinks the generation is 120819
Superblock thinks the level is 1
Well block 3706532397056(gen: 120817 level: 1) seems good, but generation/level doesn't match, want gen: 120819 level: 1
(...)
$ sudo btrfs restore -vimoxS -t 3706532397056 /dev/bcache0 /mntbtrfs restore will make a copy of the recoverable data, so the target must have enough space to hold it.
Everything I tried besides the described above. Particularly:
btrfs check: complained about the failure to open the root tree and did nothing else;btrfs rescue zero-log: also complained and did nothing. Found later this was for a specific and already fixed bug, and is needed when there's a kernel OOPS with a specifc stacktrace;btrfs rescue super-recover: told me my superblocks where all ok;btrfs chunk-recover: Took a long time and didn't fix the failure to mount the drive. Found later this one is also for a specifc already fixed bug, which generates log messages like missing chunk or whatever;
There're a few docs around the net showing commands which the official documentation says one shouldn't use unless a better analysis shows it should be used. If the data is important it seems it would be best to look for someone with knowledge of the inner workings of the filesystem. There're also one or two reports around of people who decided to ignore these warnings and killed their data.
That all said, I've tried to reach the btrfs mailling list and some antispam somewhere seems to have eaten my e-mail.
The better your backups are, the less stressful will be your mental health when your filesystem blows (when, not if). In this case almost all of the data have been recovered, but backup has been very useful for the corrupted part.
After the recovery finished, I've tried to toy around with a few of the "dangerous" commands - like btrfs check --repair, --init-csum-tree - to see if I could get the original filesystem mounted. None of them gave any output indicative of having worked, filesystem still unmountable and the same command used to restore tha data before seems to miss a whole subvolume now, so it seems I've broken something.
I can't say for sure if this problem happened due to btrfs or something lower level - bcache, md-raid, lying physical disks.
I can say I'm going to give a rest to my btrfs experiments and going back to good, old, stable ext4. It's not that I think it wouldn't have gotten some data corrupted in the same circunstances, but it just recovers itself - lost data is lost, good data is there. Getting an unmountable filesystem, "running around" the net trying to find how to fix it, needing enough extra space to a new copy of the data and a truckload of hours and disk-io-stress to copy everything out seemed very messy to me.