独自実装Deflateで最適化を頑張ってみた。

deflate-time-1.txt

https://bitbucket.org/7shi/deflatelib

* テスト対象 Ruby.exe: ruby 1.6.8 (2002-12-24) [i686-mingw32]
Length: 1748481

* DeflateStream (MS製)
Length: 942160, Time: 00:00:00.1875000

* F# (独自実装・Releaseビルド)
Length: 850186, Time: 00:00:00.2500000

** 最適化の過程（Debugビルド）

1. FSharpZip
Length: 890510, Time: 00:00:01.1250000

2. CRC削除
Length: 890510, Time: 00:00:01.0000000

3. 他の方式を削除
Length: 890510, Time: 00:00:00.8594135

4. ハッシュの読み出しバグを修正して圧縮率を向上
Length: 850174, Time: 00:00:00.7969056

5. バッファ移動時の再キャッシュを抑制
Length: 850186, Time: 00:00:00.7500240

6. 長さ・距離をテーブル化
Length: 850186, Time: 00:00:00.7031385

7. WriteBitの引数をboolに変更
Length: 850186, Time: 00:00:00.6718879

8. WriteBitsをテーブルで高速化
Length: 850186, Time: 00:00:00.6093867

deflate-time-2.txt

* F#

[target file]
Length: 1748481

[myimpl (file I/O)]
Length: 850186, Time: 00:00:00.4062500

[myimpl (on memory)]
[1:OK] Length: 850186, Time: 00:00:00.2656250
[2:OK] Length: 850186, Time: 00:00:00.2656250
[3:OK] Length: 850186, Time: 00:00:00.2500000
[4:OK] Length: 850186, Time: 00:00:00.2500000
[5:OK] Length: 850186, Time: 00:00:00.2656250
[1 - 5] Time: 00:00:00.2590000

[DeflateStream (on memory)]
[1:OK] Length: 942160, Time: 00:00:00.1562500
[2:OK] Length: 942160, Time: 00:00:00.1718750
[3:OK] Length: 942160, Time: 00:00:00.1718750
[4:OK] Length: 942160, Time: 00:00:00.1562500
[5:OK] Length: 942160, Time: 00:00:00.1718750
[1 - 5] Time: 00:00:00.1660000


* VB.NET

[target file]
Length: 1748481

[myimpl (file I/O)]
Length: 850186, Time: 00:00:00.2968750

[myimpl (on memory)]
[1:OK] Length: 850186, Time: 00:00:00.2656250
[2:OK] Length: 850186, Time: 00:00:00.2656250
[3:OK] Length: 850186, Time: 00:00:00.2656250
[4:OK] Length: 850186, Time: 00:00:00.2656250
[5:OK] Length: 850186, Time: 00:00:00.2656250
[1 - 5] Times: 00:00:00.2660000

[DeflateStream (on memory)]
[1:OK] Length: 942160, Time: 00:00:00.1718750
[2:OK] Length: 942160, Time: 00:00:00.1562500
[3:OK] Length: 942160, Time: 00:00:00.1562500
[4:OK] Length: 942160, Time: 00:00:00.1718750
[5:OK] Length: 942160, Time: 00:00:00.1718750
[1 - 5] Times: 00:00:00.1660000


* C#

[target file]
Length: 1748481

[myimpl (file I/O)]
Length: 850186, Time: 00:00:00.2343750

[myimpl (on memory)]
[0:OK] Length: 850186, Time: 00:00:00.2343750
[1:OK] Length: 850186, Time: 00:00:00.2343750
[2:OK] Length: 850186, Time: 00:00:00.2343750
[3:OK] Length: 850186, Time: 00:00:00.2343750
[4:OK] Length: 850186, Time: 00:00:00.2187500
[1 - 5] Times: 00:00:00.2310000

[DeflateStream (on memory)]
[0:OK] Length: 942160, Time: 00:00:00.1562500
[1:OK] Length: 942160, Time: 00:00:00.1718750
[2:OK] Length: 942160, Time: 00:00:00.1718750
[3:OK] Length: 942160, Time: 00:00:00.1718750
[4:OK] Length: 942160, Time: 00:00:00.1718750
[1 - 5] Times: 00:00:00.1690000


* C++ (VC)

[VC++ 2010] compress time: 0.125000
[gcc-4.6.1 -O0] compress time: 0.312000
[gcc-4.6.1 -O1] compress time: 0.187000
[gcc-4.6.1 -O2] compress time: 0.141000
[gcc-4.6.1 -O3] compress time: 0.125000


* Andromeda

compress time: 00:00:01.0620000


* VBA

1: 6.250000047
2: 6.269999687
3: 6.310000224
4: 6.259999867
5: 6.279999507
[1 - 5] 6.273999866

deflate-time-3.txt

* テスト対象 heasarc_hipparcos.tdat
Size: 85,234,422

* DeflateStream (MS製)
Size: 24,039,736
compress time: 00:00:05.8281250
compress time: 00:00:05.8437500
compress time: 00:00:05.8281250

* F# 最適化の過程（Debugビルド）

1. FSharpZip
Size: 29,257,542
compress time: 00:00:17.9062500
compress time: 00:00:18.0156250
compress time: 00:00:31.3281250

2. CRC削除
Size: 29,257,542
compress time: 00:00:15.9218750
compress time: 00:00:15.9687500
compress time: 00:00:29.5468750

3. 他の方式を削除
Size: 29,257,542
compress time: 00:00:14.8593750
compress time: 00:00:14.8593750
compress time: 00:00:28.4062500

4. ハッシュの読み出しバグを修正して圧縮率を向上
Size: 28,494,611 -
compress time: 00:00:15.7500000
compress time: 00:00:15.7500000
compress time: 00:00:27.2968750

5. バッファ移動時の再キャッシュを抑制
Size: 28,495,287 +
compress time: 00:00:13.3593750
compress time: 00:00:13.3437500
compress time: 00:00:25.5468750

6. 長さ・距離をテーブル化
Size: 28,495,287
compress time: 00:00:11.6250000
compress time: 00:00:11.6093750
compress time: 00:00:22.7968750

7. WriteBitの引数をboolに変更
Size: 28,495,287
compress time: 00:00:10.9843750
compress time: 00:00:10.9843750
compress time: 00:00:21.8906250

8. WriteBitsをテーブルで高速化
Size: 28,495,287
compress time: 00:00:09.4531250
compress time: 00:00:09.4218750
compress time: 00:00:18.5625000

deflate-time-4.txt

VC++ 2010 (Release)
compress time: 4.563000

VC++ 2010 (Debug)
compress time: 16.750000

g++ 4.6.1 -O3
compress time: 5.109000

g++ 4.6.1 -O2
compress time: 5.782000

g++ 4.6.1 -O1
compress time: 6.609000

g++ 4.6.1 -O0
compress time: 12.047000

VBA
所要時間（秒） 252.5100001

F# (Release)
compress time: 00:00:09.5312500

F# (Debug)
compress time: 00:00:19.3906250

C# (Release)
compress time: 00:00:09.1406250

C# (Debug)
compress time: 00:00:16.9062500

VB.NET (Release)
compress time: 00:00:10.1093750

VB.NET (Debug)
compress time: 00:00:18.4687500

Andromeda
compress time: 00:00:39.8130000

Program.fs

open System
open System.IO

let sin = new FileStream(@"foo", FileMode.Open)
let sout = new FileStream(@"foo.deflate", FileMode.Create)
let t1 = DateTime.Now
let w = new Deflate.Writer(sin)
w.Compress sout
sout.Close()
sin.Close()
let ts = DateTime.Now - t1
printfn "compress time: %A" ts