Skip to content

Instantly share code, notes, and snippets.

@cetinajero

cetinajero/ddrescue.md

Last active April 10, 2024 03:46
Show Gist options
  • Star 36 You must be signed in to star a gist
  • Fork 4 You must be signed in to fork a gist
  • Save cetinajero/78417647c0b7c89691b921e5327a3493 to your computer and use it in GitHub Desktop.
Save cetinajero/78417647c0b7c89691b921e5327a3493 to your computer and use it in GitHub Desktop.
Download ZIP
Guide for using Ddrescue to recover data
Raw
ddrescue.md

Guide for using Ddrescue to recover data

Start command:

ddrescue -d /dev/sda output.img output.mapfile

NOTE: If direct disc access is not available in your system, try raw devices.

Read your system documentation to find how to bind a raw device to a regular block device. Some OSs provide raw access through especial device names, like /dev/rdisk.

Define the starting position:

ddrescue -i30GiB /dev/sda output.img output.mapfile

Define the maximum recovery size of the execution:

ddrescue -s10GiB /dev/sda output.img output.mapfile

Only copy using certain phases:

ddrescue --cpass=1,3-4 /dev/sda output.img output.mapfile

Enable retrying phase:

ddrescue -r3 /dev/sda output.img output.mapfile

Input size unknown command:

input_size=500123456789; start=199000000000; ddrescue -i$start -s$(($input_size - $start)) /dev/sda output.img output.mapfile

Mount img as read-only:

hdiutil attach output.img -shadow output.shadow

Here’s what this does:

  • -d tells ddrescue to use direct disk access and ignore the kernel’s cache.

  • -i bytes defines the starting position of the rescue domain in infile, in bytes. Defaults to 0. This is not the point from which ddrescue starts copying. (For example, if you pass the option '--reverse' to ddrescue, it starts copying from the end of the rescue domain).

  • -s bytes defines the maximum size of the rescue domain in bytes. It limits the amount of input data to be copied. If ddrescue can't determine the size of the input file, you may need to specify it with this option. Note that this option does not specify the size of the resulting outfile.

  • --cpass selects what pass(es) to run during the copying phase. Valid pass values range from 1 to 5. To run only the given pass(es), specify also --no-trim and --no-scrape. --cpass=0 skips the copying phase entirely.

  • -r3 tells ddrescue to retry bad sectors 3 times before giving up. Note: On a failing drive you may want to eliminate this option the first time so as to not waste time hammering on bad sectors and risking drive failure. You can always use the mapfile to go back and retry the bad sectors after you get an image from the first sweep.

  • /dev/sda is the drive we are rescuing…the whole disk. Naturally, if you just wanted a specific partition, you would use something like /dev/sda1 instead.

  • output.img is the name of the image file.

  • output.mapfile is the name of the mapfile. Always use a mapfile. This allows you to resume an interrupted image at the point you left off, or to retry bad sectors after an initial pass. Without a mapfile, you will have to start over again!

Algorithm

Only non-tried areas are read in large blocks at Copying phase. Trimming, scraping, and retrying are done sector by sector.

Each sector is tried at most two times; the first in the Copying phase as part of a large block read, the second in one of the phases below as a single sector read.

1. Copying phase

Copying is done in up to five passes. The copying direction is reversed after each pass until all the rescue domain is tried.

The purpose of the multiple passes is to delimit large bad areas fast, recover the most promising areas first, keep the mapfile small, and produce good starting points for trimming.

  • First pass: The first pass reads the non-tried parts of the input file, marking the failed blocks as non-trimmed and skipping beyond them.

  • Second pass: The second pass delimits the blocks skipped by the first pass. The first two passes also skip beyond slow areas.

  • Third and fourth passes: The third and fourth passes read the blocks skipped due to slow areas (if any) by the first two passes, in the same direction that each block was skipped. For each block, passes 2 to 4 skip the rest of the block after finding the first error in the block.

  • Fifth pass: The last pass is a sweeping pass, with skipping disabled.

2. Trimming phase

Trimming is done in one pass. For each non-trimmed block, read forwards one sector at a time from the leading edge of the block until a bad sector is found. Then read backwards one sector at a time from the trailing edge of the block until a bad sector is found.

Then mark the bad sectors found (if any) as bad-sector, and mark the rest of the block as non-scraped without trying to read it. If any edge is already adjacent to a bad sector, it is considered as already trimmed and is not trimmed again.

3. Scraping phase

Scrape together the data not recovered by the copying or trimming phases. Scraping is done in one pass. Each non-scraped block is read forwards, one sector at a time. Any bad sectors found are marked as bad-sector.

4. Retrying phase (disabled by default)

Try to read again the bad sectors until the specified number of retry passes is reached. The direction is reversed after each pass. Every bad sector is tried only once in each pass.

Ddrescue can't know if a bad sector is unrecoverable or if it will be eventually read after some retries.

Statically compiling ddrescue for an ESXI hypervisor v5.5

Install a GNU/Linux OS based on the 2.4 Linux Kernel (e.g. CentOS 3.9).

Environment:

Software Version
Linux kernel 2.4.21-50
gcc v3.2.3 (20030502)
glibc v2.3.2-95.50
make v3.79.1

Download the ddrescue source code.

Compile the binary using the static linked libraries flag:

./configure LDFLAGS="-static"
make
make install

File carving

TestDisk & PhotoRec

TestDisk & PhotoRec's Source code

Run:

brew install testdisk
photorec

Foremost

Foremost's Source code

Run:

brew install foremost
foremost -i /dev/disk2s1
foremost -i /dev/disk2s1 -a               # include partially recovered files
foremost -i /dev/disk2s1 -t gif,pdf
foremost -i /dev/disk2s1 -c foremost.conf

foremost.conf:

# extension   case-sensitive-header?  max-file-size   header  optional-footer
bak    y       3000000000    \x54\x41\x50\x45

Encrypting files

Strongest symmetric-key algorithm (AES-256)

Encrypt:

openssl enc -aes-256-cbc -e -in plaintext -out encrypted

Decrypt:

openssl enc -aes-256-cbc -d -in encrypted -out decrypted

Hashing files

Create an MD5 hash

md5 file.img > file.md5

Compressing files

ZIP

Compress:

zip -vr file.zip directory/ -x "*.DS_Store"

Uncompress:

unzip file.zip

Tar gzip

Compress:

# normal
tar czvf file.tar.gz directory

# with progress bar
tar cf - /directory -P | pv -s $(($(du -sk /directory | awk '{print $1}') * 1024)) | gzip > file.tar.gz

Uncompress:

# normal
tar xzvf file.tar.gz

# with progress bar
pv file.tar.gz | tar xzf -

Splitting files

Split

split -b 1G file.iso file.iso.

Join

cat file.iso.* > file.iso

Binary hex editor

Open file in binary mode

vi -b file.img

Binary to hex

:%!xxd

Hex back to binary

:%!xxd -r
@lorenzojlamas
Copy link

Thank you so much. Great job

@AGenchev
Copy link

AGenchev commented Jan 28, 2024 •
edited

I used it like
ddrescue -d -r3 /dev/sda output.img output.mapfile
and no additional passes or retries were observed. It went only one pass forward and didn't make more sophisticated efforts to re-read the failed 27 sectors. Still I'm lucky that only 27 4k sectors went missing. Now I'm rewriting the disk with badblocks to trigger its auto-recovery.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment