Skip to content

Instantly share code, notes, and snippets.

@timrobertson100
Created September 21, 2014 10:21
Show Gist options
  • Save timrobertson100/86799f2770a0c4dc04ac to your computer and use it in GitHub Desktop.
Save timrobertson100/86799f2770a0c4dc04ac to your computer and use it in GitHub Desktop.
/**
* An end to end test that writes some random files and ensures that when deflated separately, merged and inflated
* they represent the same byte sequence as a concatenation of the original files.
*/
@Test
public void testParallelCompress() throws IOException {
// generate the uncompressed files and create a merged version
List<File> parts = Lists.newArrayList();
for (int i = 0; i < NUMBER_PARTS; i++) {
parts.add(randomFileOfSize(PART_SIZE_IN_BYTES));
}
File original = File.createTempFile("original-", ".txt", new File("/tmp/d2"));
merge(parts, original);
// compress each file separately, and add an extra file which will mark a close of the deflation stream
List<File> deflated = Lists.newArrayList();
for (File f : parts) {
File compressedPart = File.createTempFile("comp-", D2Utils.fileExtension, new File("/tmp/d2"));
D2Utils.compress(new FileInputStream(f), new FileOutputStream(compressedPart));
deflated.add(compressedPart);
}
// merge and mergeAndDecompress
File decompressed = File.createTempFile("inflated-", ".txt", new File("/tmp/d2"));
D2Utils.decompress(inputStreamsToFiles(deflated), new FileOutputStream(decompressed));
Assert.assertTrue("Content of files should be identical", FileUtils.contentEquals(original, decompressed));
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment