Skip to content

Instantly share code, notes, and snippets.

@Mart-Bogdan
Forked from fmarani/md5chunks.py
Last active October 18, 2020 12:20
Show Gist options
  • Save Mart-Bogdan/49baccf29a2faca02e57a63b3e2126a9 to your computer and use it in GitHub Desktop.
Save Mart-Bogdan/49baccf29a2faca02e57a63b3e2126a9 to your computer and use it in GitHub Desktop.
calculate md5 hashes of chunks of a file, configurable block size
#!/usr/bin/env python
from optparse import OptionParser
from pathlib import Path
import hashlib
import sys
parser = OptionParser()
parser.add_option("-b", "--blocksize", dest="blocksize", type=int, default=1024,
help="Specify blocksize", metavar="blocksize")
(options, args) = parser.parse_args()
if len(args) == 0:
print ("Please specify a filename")
sys.exit(1)
f = open(args[0], 'rb')
c = 0
file_size = Path(args[0]).stat().st_size
total_blocks = file_size / options.blocksize
while 1:
block = f.read(options.blocksize)
if not block:
print("processed %8.d blocks out of %8.d total progress is 100%%"%(c-1,total_blocks), file=sys.stderr )
break
m = hashlib.md5()
m.update(block)
md5 = m.hexdigest()
print ("%d,%s" % (c, md5))
if c % 10240 == 0:
print("processed %8.d blocks out of %8.d total progress is %.2f%%"%(c,total_blocks, 100*(float(c)/float(total_blocks))), file=sys.stderr )
c += 1
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment