|# Decompressor/compressor for files in Mozilla's "mozLz4" format. Firefox uses this file format to|
|# compress e. g. bookmark backups (*.jsonlz4).|
|# This file format is in fact just plain LZ4 data with a custom header (magic number [8 bytes] and|
|# uncompressed file size [4 bytes, little endian]).|
|# This Python 3 script requires the LZ4 bindings for Python, see: https://pypi.python.org/pypi/lz4|
|# Copyright (c) 2015, Tilman Blumenbach|
|# All rights reserved.|
|# Redistribution and use in source and binary forms, with or without modification, are permitted|
|# provided that the following conditions are met:|
|# 1. Redistributions of source code must retain the above copyright notice, this list of conditions|
|# and the following disclaimer.|
|# 2. Redistributions in binary form must reproduce the above copyright notice, this list of|
|# conditions and the following disclaimer in the documentation and/or other materials provided|
|# with the distribution.|
|# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR|
|# IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND|
|# FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR|
|# CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL|
|# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,|
|# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER|
|# IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT|
|# OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.|
|from argparse import ArgumentParser|
|def __init__(self, msg):|
|self.msg = msg|
|if file_obj.read(8) != b"mozLz40\0":|
|raise InvalidHeader("Invalid magic number")|
|compressed = lz4.compress(file_obj.read())|
|return b"mozLz40\0" + compressed|
|if __name__ == "__main__":|
|argparser = ArgumentParser(description="MozLz4a compression/decompression utility")|
|"-d", "--decompress", "--uncompress",|
|help="Decompress the input file instead of compressing it."|
|help="Path to input file."|
|help="Path to output file."|
|parsed_args = argparser.parse_args()|
|in_file = open(parsed_args.in_file, "rb")|
|except IOError as e:|
|print("Could not open input file `%s' for reading: %s" % (parsed_args.in_file, e), file=sys.stderr)|
|out_file = open(parsed_args.out_file, "wb")|
|except IOError as e:|
|print("Could not open output file `%s' for writing: %s" % (parsed_args.out_file, e), file=sys.stderr)|
|data = decompress(in_file)|
|data = compress(in_file)|
|except Exception as e:|
|print("Could not compress/decompress file `%s': %s" % (parsed_args.in_file, e), file=sys.stderr)|
|except IOError as e:|
|print("Could not write to output file `%s': %s" % (parsed_args.out_file, e), file=sys.stderr)|
You sir are a life saver! I recently wrote a script to clean up the search engines used by our organisations users and found to my horror that this is no longer possible due to this compression Mozilla have implemented. Thanks to your lovely script I can now decompress the file, parse the JSON and recompress saving me hours of work! :)
Can you please write readme? I'm not familiar with python.
Maybe I do something wrong. I've installed lz4 with pip and my python version is 2.7.12.
P.S. When I tried with python 3 on a different computer I had a different error: "module 'lz4' has no attribute 'decompress'". :(
I'm using Fedora 26. Python2 and Python3 are installed. It seems the default for python was python2.
If I entered "python mozlz4.py -d search.json.mozlz4 search.json" it would error out.
plus didn't have to mod the import lz4 line.
A similar script to list all tabs: https://gist.github.com/tmonjalo/33c4402b0d35f1233020bf427b5539fa
LZ4 is not bad in itself, but LZ4 for Firefox certainly is a bad idea.
@ATRescue, what you're missing is the fact that in order to be up-to-date, session files need to be constantly updated. That's a lot of writes if you have significant number of tabs open! Without compression session storage would be regularly outdated on HDDs and would wear out consumer SSDs faster.
Sure, LZ4 in standard frame format would be better. SQLite db in WAL mode (provided it could be updated incrementally) would be much better. But you can't say for sure that "there is absolutely no need for LZ4" here.
Nope. ext* don't have it, XFS doesn't have it, HFS on older OS X required use of Apple-specific APIs for compression to work (i.e. it wasn't transparent). Besides, apps generally can't just tell OS to transparently compress arbitrary files, and asking end user to go X and toggle Y to make browser responsive or less write-intensive would definitely not help its market share.
God, if I could upvote this a hundred times I would. Why would anyone design a NEW file format (jsonlz4) for something that we already have so many STANDARD, well-defined formats for? Why does Mozilla require end-users to download, compile, test different potential hacks to get their data out of a custom format? Just use a STANDARD format!