Create a gist now

Instantly share code, notes, and snippets.

What would you like to do?
James Baker, Processing Workflow for Digital Media, This&THATCamp Sussex Humanities Lab, 19-20 May 2016

Processing Workflow for Digital Media

James Baker, This&THATCamp Sussex Humanities Lab, 19-20 May 2016


Why digital forensics?

In just 25 years most people in Britain and worldwide have come to create and organise information in a new way. In so doing, the paper archive has been replaced by the hard disk, a new format that requires historians to think and act afresh.

Digital forensics allows you to collect, secure and preserve digital historical records held on data storage devices like hard disks without changing them and without removing the context of their creation. So alongside archiving plenty of 'proper' documents, you also retain file structures, software, logs, caches, deleted files, and operating systems. Preservation of the latter in the case of a hard disk, enables you to use your forensic capture like a disk image, to virtually boot up someone else's software environment and browse their files in the same place they were made and preserved.

The BitCurator Environment is a popular means of undertaking this work. It is built on a stack of free and open source digital forensics tools and associated software libraries, modified and packaged for increased accessibility and functionality for collecting institutions. The BitCurator software is freely distributed under an open source license. It can be installed as a Linux environment; run as a virtual machine on top of most contemporary operating systems; or run as individual software tools, packages, support scripts, and documentation.

Features of BitCurator include:

  • Pre-imaging data triage
  • Forensic disk imaging
  • File system analysis and reporting
  • Identification of private and individually identifying information
  • Export of technical and other metadata

Preparatory work

  • Setup BitCurator OS (either as a Virtual Machine or native install; this document assumes the former).
  • Source appropriate hardware write-blockers (eg Wiebetech ComboDock; Sussex Humanities Lab have two)
  • Source drives to read media (floppy, CD, DVD).
  • Source flash drive as temporary storage.
  • Adjust power save settings to ensure that machine does not switch off or hibernate during processing (note: this can kill the process and waste lots of time!)

Forensic Capture

  • Undertake logical sort of all source media. Depending on collection, this might include:
    • Sort source media by container type.
    • Sort source media by file system. For disks, this might include an attempt to read write-protected source media on a Windows or Mac OS machine.
    • Set aside for storage (but not capture) installer/software disks.
    • Any difficult or unusual media set to one side for bespoke processing after business as usual.
  • Capture these decisions.
  • Start BitCurator (process assumes use as virtual machine with latest BitCurator Virtual Machine (download) and VirtualBox default settings.
  • If you are working with a large disk, connect external flash drive as writeable temporary storage for forensic captures (otherwise, there should be sufficient disk space on the BitCurator VM). First mount 'read-only' using Forensic Tools / BitCurator Mounter. Note the path for this disk in the 'raw device' column. Then in Terminal do sudo mount –w /dev/xxxx /media/xxxx where 'xxxx' is the value after '/' in the previous 'raw device column' (eg sdb1).
  • Connect source media. Ensure media to be processed is write-protected when connecting media to workstation. BitCurator sets mount policy to 'read-only' by default. For different drive types following connection is advised:
  • Hard drive: use hardware write blocker. For bare drives, also ensure that drive is placed on a surface on which it will not overheat (for example a metal sheet).
    • Floppy drive: use on-disk write blocker tag.
    • USB stick: no hardware write blocker required (Kessler & Carlton, 2014)
    • Other media: case-by-case basis.
  • Open Forensics Tools / BitCurator Mounter to ensure media is mounted correctly.
  • Create forensic disk image in E01 format using Guymager and store in appropriate folder in file system:
    • Open Imaging Tools / Guymager.
    • Select media, right-click, select 'Acquire Image'.
    • In 'Acquire Image Screen', enter as follows then hit Start:
      • File Format: Expert Witness Format.
      • Split size: 2047 MiB.
      • Case Number: Something that represents the collection
      • Evidence Number: Something that represents the item
      • Examiner: A user ID that makes sense to you.
      • Description: Recommend something like {case number}-{evidence number}-{examiner}.
      • Notes: recommend media type (e.g. cdd, fdd, hdd).
      • Image directory: select location to save disk image (for see external flash drive see the /media folder).
      • Image filename: recommend {case number}{evidence number}.
    • Hash calculation / verification:
      • Calculate MD5: yes.
      • Calculate SHA-1: yes.
      • Calculate SHA-256: yes.
      • Re-read source after acquisition: no.
      • Verify image after acquisition: yes.
  • When process is complete (Guymager should report 'Finished – Verified & Ok' in the 'State' column), make a copy of capture and metadata in a backup folder.

Reporting

  • To create a directory and file listing for your disk image, open Forensic Tools / BitCurator Reporting Tool
  • Select Fiwalk XML tab and enter:
    • Image File: location of captured image (filename {case number}{evidence number}.E01).
    • Output XML File: location of captured image with filename {case number}{evidence number}.xml.
  • Hit Run. When output reads 'Success!!!' check .xml output

Basic Image Access

  • To access files on your disk image, open Forensic Tools / BitCurator Disk Image Access
  • Select 'Open disk image'
  • Navigate to forensic capture (the .E01 file) and choose open
  • Browse the disk image and check the box for files you want to export from the image. When you are done, hit 'Export selection' and choose a location to export the files to (such as /media/XXXX/)
  • Open files in your normal PC environment for curation and description. Note that specialist software may be required for legacy file formats.

Advanced Image Access

BitCurator includes range of tools that enable you to scan a disk image for documents that contains certain patterns and features. See 'Regular Expressions in bulk extractor' for more information.


Post-processing

  • Original digital media stored or decommissioned as appropriate.
  • Optional: Photography of digital media and hardware undertaken if deemed appropriate and stored in appropriate folder alongside forensic capture and metadata (this is useful when the media contains handwritten notes, for example)

Things to consider for the future

  • Investigate private shared network for deposit of outputs.
  • Investigate acquisition of further forensic hardware and software tools:
    • For source media processing: AccessData FTK Imager; KryoFlux.
    • For curatorial arrangement and description: Quick View Plus.
  • Complex source media (including Mac and legacy Linux OS filesystems) may cause problems.
  • Integrate virus and malware check steps.

Bibliography

John, Jeremy Leighton, Ian Rowlands, Peter Williams, and Katrina Dean. 'Digital Lives: Personal Digital Archives for the 21st Century', 2010. http://britishlibrary.typepad.co.uk/files/digital-lives-synthesis01a.pdf.

Kessler, Gary C., and Greg H. Carlton. 'A Study of Forensic Imaging in the Absence of Write-Blockers'. Journal of Digital Forensics, Security and Law 9, no. 3 (30 September 2014): 51–58.

Lee, Cal. 'Bringing Bits to the User: BitCurator and BitCurator Access'. presented at the Coalition for Networked Information (CNI) Membership Meeting, 14 December 2015. https://www.cni.org/wp-content/uploads/2016/01/lee-cni-20151214.pdf.

Lee, Christopher A., Porter Olsen, Alexandra Chassanoff, Kam Woods, Matthew Kirschenbaum, and Sunitha Misra. 'From Code to Community: Building and Sustaining BitCurator through Community Engagement', 30 September 2014. http://www.bitcurator.net/wp-content/uploads/2014/11/code-to-community.pdf.

Olsen, Porter. 'Write Blocking and Le Mal d'Légal'. Two Ideas at the Same Time, 20 February 2016. https://porterolsen.wordpress.com/2016/02/20/forensic-write-blockers-and-the-interstitial-space-of-the-digital-archive/.

Redwine, Gabriela, and Neil Beagrie. 'Personal Digital Archiving'. Technology Watch Report. Digital Preservation Coalition, 1 December 2015. http://dx.doi.org/10.7207/twr15-01.

Ries, Thorsten. 'Harddrive Philology: Analysing the Writing Process on Thomas Kling's Archived Laptops', 2014. http://dharchive.org/paper/DH2014/Paper-786.xml.

Rosenthal, David. 'The Architecture of Emulation on the Web'. DSHR's Blog, 13 April 2016. http://blog.dshr.org/2016/04/the-architecture-of-emulation-on-web.html.

Thomas, Susan. 'Receiving and Managing Email Archives at the Bodleian Library: A Case Study', 12 August 2011. http://e-records.chrisprom.com/receiving-and-managing-email-archives-at-the-bodleian-library-a-case-study-susan-thomas/.


Some admin...

Creative Commons Licence
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment