Skip to content

Instantly share code, notes, and snippets.

@ph4r05

ph4r05/booltest-halving-demo.md

Last active December 18, 2019 14:55
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save ph4r05/5a1f10607445c30273ea30ae86a3d4f5 to your computer and use it in GitHub Desktop.
Save ph4r05/5a1f10607445c30273ea30ae86a3d4f5 to your computer and use it in GitHub Desktop.
Download ZIP
Booltest halving demo test
Raw
booltest-halving-demo.md

Here is a simple demo for the BoolTest vs BoolTest-halving:

I was testing a simple PNG image with the BoolTest. Here are the differences presented: Tested PNG: https://i.kym-cdn.com/entries/icons/original/000/005/908/idontwanttoliveonthisplanet.jpg

Summary

NormalBooltest:

- best poly zscore  12.33883, expp: 0.0625, exp:     227, obs:     407, diff: -79.2951542 %, poly: [[114, 170, 154, 178]]
Ref samples: 40005, min-zscrore: 4.838657, max-zscore: 7.835336, best observed: 12.33882879511837, rejected: True, alpha: 2.4996875390576178e-05

Halving BoolTest:

- best poly zscore  14.20215, expp: 0.0625, exp:     113, obs:     260, diff: -129.0748899 %, poly: [[218, 226, 210, 234]]
Ref samples: 40005, min-zscrore: 4.838657, max-zscore: 7.835336, best observed: 14.202148707155084, rejected: True, alpha: 2.4996875390576178e-05

# Second half test:
Binomial dist [0], two-sided pval: 0.025636235, poly pst: 0.0625, ntrials: 1816, succ: 137, poly: ((218, 226, 210, 234),)

Conclusion:

  • Normal BoolTest rejected uniformity hypothesis with alpha=2.5e-5 (zscore 12.34).
  • Halving BoolTest processed first half of the same input, picks the best distinguisher which also rejects the first half uniformity hypothesis with alpha=2.5e-5 (zscore 14.20).
  • Halving BoolTest processed another half with the best distinguisher from the first half and did not reject the hypothesis, pvalue for the second phase is 0.025636235, which is much higher than 2.5e-5 from the first phase.

Results

Normal BoolTest:

booltest --degree 2 --block 256 --combine-deg 2 --json-out --json-nice --json-top 5  --debug https://i.kym-cdn.com/entries/icons/original/000/005/908/idontwanttoliveonthisplanet.jpg

Output:

2019-12-18 15:54:14 phx.local booltest.booltest_main[771] DEBUG Input polynomials length: 0
2019-12-18 15:54:14 phx.local urllib3.connectionpool[771] DEBUG Starting new HTTPS connection (1): i.kym-cdn.com:443
2019-12-18 15:54:14 phx.local urllib3.connectionpool[771] DEBUG https://i.kym-cdn.com:443 "HEAD /entries/icons/original/000/005/908/idontwanttoliveonthisplanet.jpg HTTP/1.1" 200 0
2019-12-18 15:54:14 phx.local booltest.booltest_main[771] INFO Basic settings, deg: 2, blocklen: 256, comb deg: 2, TV size: None, rounds: 0
2019-12-18 15:54:14 phx.local booltest.booltest_main[771] DEBUG Initialising
2019-12-18 15:54:14 phx.local booltest.booltest_main[771] INFO Testing input object: https://i.kym-cdn.com/entries/icons/original/000/005/908/idontwanttoliveonthisplanet.jpg, size: 113 kB
2019-12-18 15:54:14 phx.local booltest.booltest_main[771] WARNING Input data size not aligned to the block size. Input bytes: 116232, block bits: 256, rem: 64
2019-12-18 15:54:14 phx.local booltest.booltest_main[771] INFO Updating TV to 116224
2019-12-18 15:54:14 phx.local booltest.booltest_main[771] INFO Using reference data file /Users/dusanklinec/.pyenv/versions/3.7.1/lib/python3.7/site-packages/booltest/assets/pval_db.json
2019-12-18 15:54:14 phx.local booltest.booltest_main[771] DEBUG Initializing test
2019-12-18 15:54:14 phx.local booltest.booltest_main[771] DEBUG Initializing HWanalysis
2019-12-18 15:54:14 phx.local booltest.booltest_main[771] DEBUG Precomputing term mappings
2019-12-18 15:54:14 phx.local booltest.booltest_main[771] INFO Pval db loaded, samples: 40005, min: 4.838657, max: 7.835336
2019-12-18 15:54:14 phx.local booltest.booltest_main[771] INFO BlockLength: 256, deg: 2, terms: 32640
2019-12-18 15:54:14 phx.local urllib3.connectionpool[771] DEBUG Starting new HTTPS connection (1): i.kym-cdn.com:443
2019-12-18 15:54:14 phx.local urllib3.connectionpool[771] DEBUG https://i.kym-cdn.com:443 "GET /entries/icons/original/000/005/908/idontwanttoliveonthisplanet.jpg HTTP/1.1" 200 116232
2019-12-18 15:54:14 phx.local booltest.booltest_main[771] INFO Analyzing input, deg: 2, blocklen:  256, tvsize:   116224 =   113.50 kB =     0.11 MB, num-blocks: 3632, round: 0, process: 929792 bits
2019-12-18 15:54:14 phx.local booltest.booltest_main[771] INFO Evaluating all terms, bitlen: 929792, bytes: 116224
2019-12-18 15:54:14 phx.local booltest.booltest_main[771] INFO Done: [0, 256, 32640]
2019-12-18 15:54:14 phx.local booltest.booltest_main[771] INFO HWS merged - move
2019-12-18 15:54:14 phx.local booltest.booltest_main[771] INFO Probabilities: [1.0, 0.5, 0.25], expected count: [3632.0, 1816.0, 908.0]
2019-12-18 15:54:14 phx.local booltest.booltest_main[771] INFO Find best with allsort start deg: 1
2019-12-18 15:54:14 phx.local booltest.booltest_main[771] INFO Sorting...
2019-12-18 15:54:14 phx.local booltest.booltest_main[771] INFO Sorted... len: 256
2019-12-18 15:54:14 phx.local booltest.booltest_main[771] INFO Stats computed [1], mean zscore: 1.820572302365235
 - zscore[deg=1]: -5.11067, +5.11067, observed: 00001662, expected: 00001816   idx:    160, term: [160]
 - zscore[deg=1]: -4.77881, +4.77881, observed: 00001672, expected: 00001816   idx:    217, term: [217]
 - zscore[deg=1]: -4.34739, +4.34739, observed: 00001685, expected: 00001816   idx:     83, term: [83]
 - zscore[deg=1]: -4.31420, +4.31420, observed: 00001686, expected: 00001816   idx:    192, term: [192]
 - zscore[deg=1]: -4.31420, +4.31420, observed: 00001686, expected: 00001816   idx:    227, term: [227]
 - zscore[deg=1]: -4.24783, +4.24783, observed: 00001688, expected: 00001816   idx:    144, term: [144]
 - zscore[deg=1]: -4.14827, +4.14827, observed: 00001691, expected: 00001816   idx:     40, term: [40]
 - zscore[deg=1]: -4.14827, +4.14827, observed: 00001691, expected: 00001816   idx:    232, term: [232]
 - zscore[deg=1]: -4.04871, +4.04871, observed: 00001694, expected: 00001816   idx:     48, term: [48]
 - zscore[deg=1]: -4.04871, +4.04871, observed: 00001694, expected: 00001816   idx:    196, term: [196]
 - zscore[deg=1]: -3.81641, +3.81641, observed: 00001701, expected: 00001816   idx:     76, term: [76]
 - zscore[deg=1]: -3.81641, +3.81641, observed: 00001701, expected: 00001816   idx:    224, term: [224]
 - zscore[deg=1]: -3.78322, +3.78322, observed: 00001702, expected: 00001816   idx:    235, term: [235]
 - zscore[deg=1]: -3.71685, +3.71685, observed: 00001704, expected: 00001816   idx:     80, term: [80]
 - zscore[deg=1]: -3.65048, +3.65048, observed: 00001706, expected: 00001816   idx:    216, term: [216]
2019-12-18 15:54:14 phx.local booltest.booltest_main[771] INFO Comb...
2019-12-18 15:54:14 phx.local booltest.booltest_main[771] INFO Stats...
Mean zscore[deg=1]: 1.820572302365235
2019-12-18 15:54:14 phx.local booltest.booltest_main[771] INFO Find best with allsort start deg: 2
2019-12-18 15:54:14 phx.local booltest.booltest_main[771] INFO Sorting...
2019-12-18 15:54:14 phx.local booltest.booltest_main[771] INFO Sorted... len: 32640
2019-12-18 15:54:14 phx.local booltest.booltest_main[771] INFO Stats computed [2], mean zscore: 1.0152215735227708
 - zscore[deg=2]: +6.09289, -6.09289, observed: 00001067, expected: 00000908   idx:  31224, term: [202, 218]
 - zscore[deg=2]: +5.93961, -5.93961, observed: 00001063, expected: 00000908   idx:   2536, term: [10, 42]
 - zscore[deg=2]: +5.70969, -5.70969, observed: 00001057, expected: 00000908   idx:   8116, term: [34, 42]
 - zscore[deg=2]: +5.55641, -5.55641, observed: 00001053, expected: 00000908   idx:  31960, term: [218, 242]
 - zscore[deg=2]: +5.40313, -5.40313, observed: 00001049, expected: 00000908   idx:   2528, term: [10, 34]
 - zscore[deg=2]: +5.32649, -5.32649, observed: 00001047, expected: 00000908   idx:  31612, term: [210, 218]
 - zscore[deg=2]: +5.28817, -5.28817, observed: 00001046, expected: 00000908   idx:   8244, term: [34, 170]
 - zscore[deg=2]: +5.17321, -5.17321, observed: 00001043, expected: 00000908   idx:   2512, term: [10, 18]
 - zscore[deg=2]: +5.17321, -5.17321, observed: 00001043, expected: 00000908   idx:   6320, term: [26, 42]
 - zscore[deg=2]: +5.13489, -5.13489, observed: 00001042, expected: 00000908   idx:  22684, term: [114, 170]
 - zscore[deg=2]: +5.09657, -5.09657, observed: 00001041, expected: 00000908   idx:    676, term: [2, 170]
 - zscore[deg=2]: +5.09657, -5.09657, observed: 00001041, expected: 00000908   idx:  31952, term: [218, 234]
 - zscore[deg=2]: +5.05825, -5.05825, observed: 00001040, expected: 00000908   idx:  22732, term: [114, 218]
 - zscore[deg=2]: +4.98161, -4.98161, observed: 00001038, expected: 00000908   idx:   6312, term: [26, 34]
 - zscore[deg=2]: +4.94329, -4.94329, observed: 00001037, expected: 00000908   idx:   4444, term: [18, 26]
2019-12-18 15:54:14 phx.local booltest.booltest_main[771] INFO Comb...
2019-12-18 15:54:14 phx.local booltest.booltest_main[771] INFO Stats...
Mean zscore[deg=2]: 1.0152215735227708
2019-12-18 15:54:14 phx.local booltest.booltest_main[771] INFO Combining 128 terms in 2 degree, total = 8128 evals, keep best limit: None
2019-12-18 15:54:15 phx.local booltest.booltest_main[771] INFO Evaluating
 - best poly zscore  12.33883, expp: 6.2500e-02, exp:     227, obs:     407, diff: -79.2951542 %, poly: [[114, 170, 154, 178]]
 - best poly zscore  12.06463, expp: 6.2500e-02, exp:     227, obs:     403, diff: -77.5330396 %, poly: [[10, 42, 18, 26]]
 - best poly zscore  12.06463, expp: 6.2500e-02, exp:     227, obs:     403, diff: -77.5330396 %, poly: [[10, 18, 26, 42]]
 - best poly zscore  12.06463, expp: 6.2500e-02, exp:     227, obs:     403, diff: -77.5330396 %, poly: [[10, 18, 42, 202]]
 - best poly zscore  12.06463, expp: 6.2500e-02, exp:     227, obs:     403, diff: -77.5330396 %, poly: [[10, 26, 18, 42]]
 - best poly zscore  12.06463, expp: 6.2500e-02, exp:     227, obs:     403, diff: -77.5330396 %, poly: [[18, 42, 10, 202]]
 - best poly zscore  11.92753, expp: 6.2500e-02, exp:     227, obs:     401, diff: -76.6519824 %, poly: [[18, 26, 210, 242]]
 - best poly zscore  11.92753, expp: 6.2500e-02, exp:     227, obs:     401, diff: -76.6519824 %, poly: [[26, 210, 18, 242]]
 - best poly zscore  11.85899, expp: 6.2500e-02, exp:     227, obs:     400, diff: -76.2114537 %, poly: [[10, 18, 42, 58]]
 - best poly zscore  11.65334, expp: 6.2500e-02, exp:     227, obs:     397, diff: -74.8898678 %, poly: [[210, 218, 226, 234]]
 - best poly zscore  11.65334, expp: 6.2500e-02, exp:     227, obs:     397, diff: -74.8898678 %, poly: [[218, 226, 210, 234]]
 - best poly zscore  11.65334, expp: 6.2500e-02, exp:     227, obs:     397, diff: -74.8898678 %, poly: [[194, 218, 210, 234]]
 - best poly zscore  11.58479, expp: 6.2500e-02, exp:     227, obs:     396, diff: -74.4493392 %, poly: [[10, 42, 18, 242]]
 - best poly zscore  11.58479, expp: 6.2500e-02, exp:     227, obs:     396, diff: -74.4493392 %, poly: [[218, 242, 210, 234]]
 - best poly zscore  11.58479, expp: 6.2500e-02, exp:     227, obs:     396, diff: -74.4493392 %, poly: [[210, 218, 234, 242]]
 - best poly zscore  11.58479, expp: 6.2500e-02, exp:     227, obs:     396, diff: -74.4493392 %, poly: [[10, 18, 42, 242]]
 - best poly zscore  11.58479, expp: 6.2500e-02, exp:     227, obs:     396, diff: -74.4493392 %, poly: [[218, 234, 210, 242]]
 - best poly zscore  11.58479, expp: 6.2500e-02, exp:     227, obs:     396, diff: -74.4493392 %, poly: [[18, 42, 10, 242]]
 - best poly zscore  11.58479, expp: 6.2500e-02, exp:     227, obs:     396, diff: -74.4493392 %, poly: [[154, 170, 178, 194]]
 - best poly zscore  11.51624, expp: 6.2500e-02, exp:     227, obs:     395, diff: -74.0088106 %, poly: [[10, 42, 26, 34]]
 - best poly zscore  11.51624, expp: 6.2500e-02, exp:     227, obs:     395, diff: -74.0088106 %, poly: [[34, 42, 10, 26]]
 - best poly zscore  11.51624, expp: 6.2500e-02, exp:     227, obs:     395, diff: -74.0088106 %, poly: [[10, 34, 26, 42]]
 - best poly zscore  11.51624, expp: 6.2500e-02, exp:     227, obs:     395, diff: -74.0088106 %, poly: [[170, 218, 18, 242]]
 - best poly zscore  11.51624, expp: 6.2500e-02, exp:     227, obs:     395, diff: -74.0088106 %, poly: [[18, 218, 170, 242]]
 - best poly zscore  11.44769, expp: 6.2500e-02, exp:     227, obs:     394, diff: -73.5682819 %, poly: [[10, 42, 34, 170]]
 - best poly zscore  11.44769, expp: 6.2500e-02, exp:     227, obs:     394, diff: -73.5682819 %, poly: [[10, 42, 2, 26]]
 - best poly zscore  11.44769, expp: 6.2500e-02, exp:     227, obs:     394, diff: -73.5682819 %, poly: [[10, 42, 34, 202]]
 - best poly zscore  11.44769, expp: 6.2500e-02, exp:     227, obs:     394, diff: -73.5682819 %, poly: [[34, 42, 10, 18]]
 - best poly zscore  11.44769, expp: 6.2500e-02, exp:     227, obs:     394, diff: -73.5682819 %, poly: [[34, 42, 10, 202]]
 - best poly zscore  11.44769, expp: 6.2500e-02, exp:     227, obs:     394, diff: -73.5682819 %, poly: [[10, 34, 18, 42]]
2019-12-18 15:54:15 phx.local booltest.booltest_main[771] INFO Ref samples: 40005, min-zscrore: 4.838657, max-zscore: 7.835336, best observed: 12.33882879511837, rejected: True, alpha: 2.49969e-05
2019-12-18 15:54:15 phx.local booltest.booltest_main[771] INFO Not aligned block read, terminating. Data left: 64 bits, block size: 256 bits
2019-12-18 15:54:15 phx.local booltest.booltest_main[771] INFO Finished processing: https://i.kym-cdn.com/entries/icons/original/000/005/908/idontwanttoliveonthisplanet.jpg 
2019-12-18 15:54:15 phx.local booltest.booltest_main[771] INFO Data read: 116232 B
2019-12-18 15:54:15 phx.local booltest.booltest_main[771] INFO Read data SHA1:   5f2bed43e507222e038fc9cd8f2436247a98bedf 
2019-12-18 15:54:15 phx.local booltest.booltest_main[771] INFO Read data SHA256: e4bda1a1b278a897addd9c729208996666b6b284457f45e8f94066d05173b2d6 
2019-12-18 15:54:15 phx.local booltest.booltest_main[771] INFO Processing finished in 1.2692430019378662 sec
{
  "blocklen": 256,
  "degree": 2,
  "comb_degree": 2,
  "top_k": 128,
  "input_poly": [],
  "offset": 0,
  "halving": false,
  "inputs": [
    {
      "iobj": "https://i.kym-cdn.com/entries/icons/original/000/005/908/idontwanttoliveonthisplanet.jpg",
      "size": 116232,
      "offset": 0,
      "sha1": "5f2bed43e507222e038fc9cd8f2436247a98bedf",
      "sha256": "e4bda1a1b278a897addd9c729208996666b6b284457f45e8f94066d05173b2d6",
      "tvsize": 116224,
      "blocks": 3632,
      "res": [
        {
          "round": 0,
          "dists": [
            {"expp": 0.0625, "exp_cnt": 227.0, "obs_cnt": 407, "diff": -79.29515418502203, "zscore": 12.33882879511837, "poly": [[114, 170, 154, 178]]},
            {"expp": 0.0625, "exp_cnt": 227.0, "obs_cnt": 403, "diff": -77.5330396475771, "zscore": 12.064632599671297, "poly": [[10, 42, 18, 26]]},
            {"expp": 0.0625, "exp_cnt": 227.0, "obs_cnt": 403, "diff": -77.5330396475771, "zscore": 12.064632599671297, "poly": [[10, 18, 26, 42]]},
            {"expp": 0.0625, "exp_cnt": 227.0, "obs_cnt": 403, "diff": -77.5330396475771, "zscore": 12.064632599671297, "poly": [[10, 18, 42, 202]]},
            {"expp": 0.0625, "exp_cnt": 227.0, "obs_cnt": 403, "diff": -77.5330396475771, "zscore": 12.064632599671297, "poly": [[10, 26, 18, 42]]}
          ],
          "ref_samples": 40005,
          "ref_alpha": 2.4996875390576178e-05,
          "ref_minmax": [
            4.838657,
            7.835336
          ],
          "rejects": true
        }
      ]
    }
  ],
  "time_elapsed": 1.269366979598999,
  "time_data_read": 0.06740880012512207,
  "time_data_bins": 8.416175842285156e-05,
  "time_process": 0.8681261539459229
}

Halving BoolTest:

booltest --degree 2 --block 256 --combine-deg 2 --json-out --json-nice --json-top 5 --halving --halving-top 5 --debug https://i.kym-cdn.com/entries/icons/original/000/005/908/idontwanttoliveonthisplanet.jpg

Result:

2019-12-18 15:54:57 phx.local booltest.booltest_main[792] DEBUG Input polynomials length: 0
2019-12-18 15:54:57 phx.local urllib3.connectionpool[792] DEBUG Starting new HTTPS connection (1): i.kym-cdn.com:443
2019-12-18 15:54:57 phx.local urllib3.connectionpool[792] DEBUG https://i.kym-cdn.com:443 "HEAD /entries/icons/original/000/005/908/idontwanttoliveonthisplanet.jpg HTTP/1.1" 200 0
2019-12-18 15:54:57 phx.local booltest.booltest_main[792] INFO Basic settings, deg: 2, blocklen: 256, comb deg: 2, TV size: None, rounds: 0
2019-12-18 15:54:57 phx.local booltest.booltest_main[792] DEBUG Initialising
2019-12-18 15:54:57 phx.local booltest.booltest_main[792] INFO Testing input object: https://i.kym-cdn.com/entries/icons/original/000/005/908/idontwanttoliveonthisplanet.jpg, size: 113 kB
2019-12-18 15:54:57 phx.local booltest.booltest_main[792] WARNING Input data size not aligned to the block size. Input bytes: 116232, block bits: 256, rem: 32
2019-12-18 15:54:57 phx.local booltest.booltest_main[792] INFO Updating TV to 116224
2019-12-18 15:54:57 phx.local booltest.booltest_main[792] INFO Using reference data file /Users/dusanklinec/.pyenv/versions/3.7.1/lib/python3.7/site-packages/booltest/assets/pval_db.json
2019-12-18 15:54:57 phx.local booltest.booltest_main[792] DEBUG Initializing test
2019-12-18 15:54:57 phx.local booltest.booltest_main[792] DEBUG Initializing HWanalysis
2019-12-18 15:54:57 phx.local booltest.booltest_main[792] DEBUG Precomputing term mappings
2019-12-18 15:54:57 phx.local booltest.booltest_main[792] INFO Pval db loaded, samples: 40005, min: 4.838657, max: 7.835336
2019-12-18 15:54:57 phx.local booltest.booltest_main[792] INFO BlockLength: 256, deg: 2, terms: 32640
2019-12-18 15:54:57 phx.local urllib3.connectionpool[792] DEBUG Starting new HTTPS connection (1): i.kym-cdn.com:443
2019-12-18 15:54:57 phx.local urllib3.connectionpool[792] DEBUG https://i.kym-cdn.com:443 "GET /entries/icons/original/000/005/908/idontwanttoliveonthisplanet.jpg HTTP/1.1" 200 116232
2019-12-18 15:54:57 phx.local booltest.booltest_main[792] INFO Analyzing input, deg: 2, blocklen:  256, tvsize:    58112 =    56.75 kB =     0.06 MB, num-blocks: 1816, round: 0, process: 464896 bits
2019-12-18 15:54:57 phx.local booltest.booltest_main[792] INFO Evaluating all terms, bitlen: 464896, bytes: 58112
2019-12-18 15:54:57 phx.local booltest.booltest_main[792] INFO Done: [0, 256, 32640]
2019-12-18 15:54:57 phx.local booltest.booltest_main[792] INFO HWS merged - move
2019-12-18 15:54:57 phx.local booltest.booltest_main[792] INFO Probabilities: [1.0, 0.5, 0.25], expected count: [1816.0, 908.0, 454.0]
2019-12-18 15:54:57 phx.local booltest.booltest_main[792] INFO Find best with allsort start deg: 1
2019-12-18 15:54:57 phx.local booltest.booltest_main[792] INFO Sorting...
2019-12-18 15:54:57 phx.local booltest.booltest_main[792] INFO Sorted... len: 256
2019-12-18 15:54:57 phx.local booltest.booltest_main[792] INFO Stats computed [1], mean zscore: 3.167931967631542
 - zscore[deg=1]: -7.18065, +7.18065, observed: 00000755, expected: 00000908   idx:    160, term: [160]
 - zscore[deg=1]: -7.08678, +7.08678, observed: 00000757, expected: 00000908   idx:     40, term: [40]
 - zscore[deg=1]: -6.89905, +6.89905, observed: 00000761, expected: 00000908   idx:    192, term: [192]
 - zscore[deg=1]: -6.80519, +6.80519, observed: 00000763, expected: 00000908   idx:    184, term: [184]
 - zscore[deg=1]: -6.47666, +6.47666, observed: 00000770, expected: 00000908   idx:    217, term: [217]
 - zscore[deg=1]: -6.33586, +6.33586, observed: 00000773, expected: 00000908   idx:    104, term: [104]
 - zscore[deg=1]: -6.19507, +6.19507, observed: 00000776, expected: 00000908   idx:     48, term: [48]
 - zscore[deg=1]: -6.19507, +6.19507, observed: 00000776, expected: 00000908   idx:    216, term: [216]
 - zscore[deg=1]: -6.14813, +6.14813, observed: 00000777, expected: 00000908   idx:    232, term: [232]
 - zscore[deg=1]: -6.00734, +6.00734, observed: 00000780, expected: 00000908   idx:     80, term: [80]
 - zscore[deg=1]: -5.91347, +5.91347, observed: 00000782, expected: 00000908   idx:    112, term: [112]
 - zscore[deg=1]: -5.86654, +5.86654, observed: 00000783, expected: 00000908   idx:     96, term: [96]
 - zscore[deg=1]: -5.81961, +5.81961, observed: 00000784, expected: 00000908   idx:    144, term: [144]
 - zscore[deg=1]: -5.77268, +5.77268, observed: 00000785, expected: 00000908   idx:     56, term: [56]
 - zscore[deg=1]: -5.77268, +5.77268, observed: 00000785, expected: 00000908   idx:    168, term: [168]
2019-12-18 15:54:57 phx.local booltest.booltest_main[792] INFO Comb...
2019-12-18 15:54:57 phx.local booltest.booltest_main[792] INFO Stats...
Mean zscore[deg=1]: 3.167931967631542
2019-12-18 15:54:57 phx.local booltest.booltest_main[792] INFO Find best with allsort start deg: 2
2019-12-18 15:54:57 phx.local booltest.booltest_main[792] INFO Sorting...
2019-12-18 15:54:57 phx.local booltest.booltest_main[792] INFO Sorted... len: 32640
2019-12-18 15:54:57 phx.local booltest.booltest_main[792] INFO Stats computed [2], mean zscore: 2.0547448779669386
 - zscore[deg=2]: +7.58699, -7.58699, observed: 00000594, expected: 00000454   idx:   8116, term: [34, 42]
 - zscore[deg=2]: -6.82829, +6.82829, observed: 00000328, expected: 00000454   idx:  28136, term: [160, 217]
 - zscore[deg=2]: -6.77410, +6.77410, observed: 00000329, expected: 00000454   idx:  11280, term: [48, 217]
 - zscore[deg=2]: -6.28636, +6.28636, observed: 00000338, expected: 00000454   idx:   4160, term: [16, 217]
 - zscore[deg=2]: -6.23217, +6.23217, observed: 00000339, expected: 00000454   idx:   9596, term: [40, 217]
 - zscore[deg=2]: -6.17798, +6.17798, observed: 00000340, expected: 00000454   idx:  30625, term: [192, 194]
 - zscore[deg=2]: +6.12378, -6.12378, observed: 00000567, expected: 00000454   idx:  22684, term: [114, 170]
 - zscore[deg=2]: -6.12378, +6.12378, observed: 00000341, expected: 00000454   idx:  24344, term: [126, 216]
 - zscore[deg=2]: -6.01540, +6.01540, observed: 00000343, expected: 00000454   idx:   3681, term: [14, 217]
 - zscore[deg=2]: -6.01540, +6.01540, observed: 00000343, expected: 00000454   idx:   9568, term: [40, 189]
 - zscore[deg=2]: -6.01540, +6.01540, observed: 00000343, expected: 00000454   idx:  27750, term: [156, 217]
 - zscore[deg=2]: -5.96121, +5.96121, observed: 00000344, expected: 00000454   idx:  17376, term: [80, 217]
 - zscore[deg=2]: +5.90701, -5.90701, observed: 00000563, expected: 00000454   idx:   6312, term: [26, 34]
 - zscore[deg=2]: -5.90701, +5.90701, observed: 00000345, expected: 00000454   idx:   7848, term: [32, 217]
 - zscore[deg=2]: -5.90701, +5.90701, observed: 00000345, expected: 00000454   idx:  10623, term: [45, 184]
2019-12-18 15:54:58 phx.local booltest.booltest_main[792] INFO Comb...
2019-12-18 15:54:58 phx.local booltest.booltest_main[792] INFO Stats...
Mean zscore[deg=2]: 2.0547448779669386
2019-12-18 15:54:58 phx.local booltest.booltest_main[792] INFO Combining 128 terms in 2 degree, total = 8128 evals, keep best limit: None
2019-12-18 15:54:58 phx.local booltest.booltest_main[792] INFO Evaluating
 - best poly zscore  14.20215, expp: 6.2500e-02, exp:     113, obs:     260, diff: -129.0748899 %, poly: [[218, 226, 210, 234]]
 - best poly zscore  14.00826, expp: 6.2500e-02, exp:     113, obs:     258, diff: -127.3127753 %, poly: [[18, 42, 2, 10]]
 - best poly zscore  14.00826, expp: 6.2500e-02, exp:     113, obs:     258, diff: -127.3127753 %, poly: [[162, 170, 82, 90]]
 - best poly zscore  13.81438, expp: 6.2500e-02, exp:     113, obs:     256, diff: -125.5506608 %, poly: [[10, 42, 18, 114]]
 - best poly zscore  13.62049, expp: 6.2500e-02, exp:     113, obs:     254, diff: -123.7885463 %, poly: [[26, 34, 18, 42]]
 - best poly zscore  13.52355, expp: 6.2500e-02, exp:     113, obs:     253, diff: -122.9074890 %, poly: [[34, 42, 210, 234]]
 - best poly zscore  13.42660, expp: 6.2500e-02, exp:     113, obs:     252, diff: -122.0264317 %, poly: [[34, 42, 2, 10]]
 - best poly zscore  13.23272, expp: 6.2500e-02, exp:     113, obs:     250, diff: -120.2643172 %, poly: [[42, 122, 210, 234]]
 - best poly zscore  13.03883, expp: 6.2500e-02, exp:     113, obs:     248, diff: -118.5022026 %, poly: [[26, 34, 10, 42]]
 - best poly zscore  12.94189, expp: 6.2500e-02, exp:     113, obs:     247, diff: -117.6211454 %, poly: [[34, 42, 226, 234]]
 - best poly zscore  12.94189, expp: 6.2500e-02, exp:     113, obs:     247, diff: -117.6211454 %, poly: [[226, 234, 42, 210]]
 - best poly zscore  12.84495, expp: 6.2500e-02, exp:     113, obs:     246, diff: -116.7400881 %, poly: [[42, 122, 114, 130]]
 - best poly zscore  12.84495, expp: 6.2500e-02, exp:     113, obs:     246, diff: -116.7400881 %, poly: [[90, 98, 82, 162]]
 - best poly zscore  12.74800, expp: 6.2500e-02, exp:     113, obs:     245, diff: -115.8590308 %, poly: [[18, 42, 218, 234]]
 - best poly zscore  12.65106, expp: 6.2500e-02, exp:     113, obs:     244, diff: -114.9779736 %, poly: [[34, 42, 218, 234]]
 - best poly zscore  12.65106, expp: 6.2500e-02, exp:     113, obs:     244, diff: -114.9779736 %, poly: [[34, 42, 18, 114]]
 - best poly zscore  12.65106, expp: 6.2500e-02, exp:     113, obs:     244, diff: -114.9779736 %, poly: [[114, 170, 82, 90]]
 - best poly zscore  12.65106, expp: 6.2500e-02, exp:     113, obs:     244, diff: -114.9779736 %, poly: [[218, 234, 42, 210]]
 - best poly zscore  12.55412, expp: 6.2500e-02, exp:     113, obs:     243, diff: -114.0969163 %, poly: [[34, 42, 114, 170]]
 - best poly zscore  12.55412, expp: 6.2500e-02, exp:     113, obs:     243, diff: -114.0969163 %, poly: [[114, 170, 82, 162]]
 - best poly zscore  12.55412, expp: 6.2500e-02, exp:     113, obs:     243, diff: -114.0969163 %, poly: [[26, 34, 2, 10]]
 - best poly zscore  12.55412, expp: 6.2500e-02, exp:     113, obs:     243, diff: -114.0969163 %, poly: [[18, 42, 66, 114]]
 - best poly zscore  12.55412, expp: 6.2500e-02, exp:     113, obs:     243, diff: -114.0969163 %, poly: [[10, 42, 66, 114]]
 - best poly zscore  12.45717, expp: 6.2500e-02, exp:     113, obs:     242, diff: -113.2158590 %, poly: [[114, 170, 18, 42]]
 - best poly zscore  12.45717, expp: 6.2500e-02, exp:     113, obs:     242, diff: -113.2158590 %, poly: [[162, 170, 66, 114]]
 - best poly zscore  12.45717, expp: 6.2500e-02, exp:     113, obs:     242, diff: -113.2158590 %, poly: [[146, 162, 18, 114]]
 - best poly zscore  12.36023, expp: 6.2500e-02, exp:     113, obs:     241, diff: -112.3348018 %, poly: [[114, 170, 218, 226]]
 - best poly zscore  12.36023, expp: 6.2500e-02, exp:     113, obs:     241, diff: -112.3348018 %, poly: [[26, 34, 226, 234]]
 - best poly zscore  12.36023, expp: 6.2500e-02, exp:     113, obs:     241, diff: -112.3348018 %, poly: [[106, 114, 210, 234]]
 - best poly zscore  12.36023, expp: 6.2500e-02, exp:     113, obs:     241, diff: -112.3348018 %, poly: [[162, 170, 90, 98]]
2019-12-18 15:54:58 phx.local booltest.booltest_main[792] INFO Ref samples: 40005, min-zscrore: 4.838657, max-zscore: 7.835336, best observed: 14.202148707155084, rejected: True, alpha: 2.49969e-05
2019-12-18 15:54:58 phx.local booltest.booltest_main[792] INFO Halving, setting the best poly: [((218, 226, 210, 234),), ((18, 42, 2, 10),), ((162, 170, 82, 90),), ((10, 42, 18, 114),), ((26, 34, 18, 42),)]
2019-12-18 15:54:58 phx.local booltest.booltest_main[792] DEBUG Initializing HWanalysis
2019-12-18 15:54:58 phx.local booltest.booltest_main[792] DEBUG Precomputing term mappings
2019-12-18 15:54:58 phx.local booltest.booltest_main[792] INFO Pval db loaded, samples: 40005, min: 4.838657, max: 7.835336
2019-12-18 15:54:58 phx.local booltest.booltest_main[792] INFO Analyzing input, deg: 2, blocklen:  256, tvsize:    58112 =    56.75 kB =     0.06 MB, num-blocks: 1816, round: 1, process: 464896 bits
 - zscore[idx00]: +2.27816, observed: 00000137, expected: 00000113   idx:      0, poly: ((218, 226, 210, 234),)
 - zscore[idx01]: +2.08427, observed: 00000135, expected: 00000113   idx:      1, poly: ((18, 42, 2, 10),)
 - zscore[idx04]: +1.79345, observed: 00000132, expected: 00000113   idx:      4, poly: ((26, 34, 18, 42),)
 - zscore[idx02]: +1.40567, observed: 00000128, expected: 00000113   idx:      2, poly: ((162, 170, 82, 90),)
 - zscore[idx03]: +1.30873, observed: 00000127, expected: 00000113   idx:      3, poly: ((10, 42, 18, 114),)
2019-12-18 15:54:59 phx.local booltest.booltest_main[792] INFO Binomial dist [0], two-sided pval: 2.56362e-02, poly pst: 6.2500e-02, ntrials: 1816, succ: 137, poly: ((218, 226, 210, 234),)
2019-12-18 15:54:59 phx.local booltest.booltest_main[792] INFO Binomial dist [1], two-sided pval: 4.15595e-02, poly pst: 6.2500e-02, ntrials: 1816, succ: 135, poly: ((18, 42, 2, 10),)
2019-12-18 15:54:59 phx.local booltest.booltest_main[792] INFO Binomial dist [2], two-sided pval: 1.59710e-01, poly pst: 6.2500e-02, ntrials: 1816, succ: 128, poly: ((162, 170, 82, 90),)
2019-12-18 15:54:59 phx.local booltest.booltest_main[792] INFO Binomial dist [3], two-sided pval: 1.90472e-01, poly pst: 6.2500e-02, ntrials: 1816, succ: 127, poly: ((10, 42, 18, 114),)
2019-12-18 15:54:59 phx.local booltest.booltest_main[792] INFO Binomial dist [4], two-sided pval: 8.06853e-02, poly pst: 6.2500e-02, ntrials: 1816, succ: 132, poly: ((26, 34, 18, 42),)
2019-12-18 15:54:59 phx.local booltest.booltest_main[792] DEBUG Initializing HWanalysis
2019-12-18 15:54:59 phx.local booltest.booltest_main[792] DEBUG Precomputing term mappings
2019-12-18 15:54:59 phx.local booltest.booltest_main[792] INFO Pval db loaded, samples: 40005, min: 4.838657, max: 7.835336
2019-12-18 15:54:59 phx.local booltest.booltest_main[792] INFO Not aligned block read, terminating. Data left: 64 bits, block size: 256 bits
2019-12-18 15:54:59 phx.local booltest.booltest_main[792] INFO Finished processing: https://i.kym-cdn.com/entries/icons/original/000/005/908/idontwanttoliveonthisplanet.jpg 
2019-12-18 15:54:59 phx.local booltest.booltest_main[792] INFO Data read: 116232 B
2019-12-18 15:54:59 phx.local booltest.booltest_main[792] INFO Read data SHA1:   5f2bed43e507222e038fc9cd8f2436247a98bedf 
2019-12-18 15:54:59 phx.local booltest.booltest_main[792] INFO Read data SHA256: e4bda1a1b278a897addd9c729208996666b6b284457f45e8f94066d05173b2d6 
2019-12-18 15:54:59 phx.local booltest.booltest_main[792] INFO Processing finished in 1.8014929294586182 sec
{
  "blocklen": 256,
  "degree": 2,
  "comb_degree": 2,
  "top_k": 128,
  "input_poly": [],
  "offset": 0,
  "halving": true,
  "inputs": [
    {
      "iobj": "https://i.kym-cdn.com/entries/icons/original/000/005/908/idontwanttoliveonthisplanet.jpg",
      "size": 116232,
      "offset": 0,
      "sha1": "5f2bed43e507222e038fc9cd8f2436247a98bedf",
      "sha256": "e4bda1a1b278a897addd9c729208996666b6b284457f45e8f94066d05173b2d6",
      "tvsize": 116224,
      "blocks": 3632,
      "res": [
        {
          "round": 0,
          "dists": [
            {"expp": 0.0625, "exp_cnt": 113.5, "obs_cnt": 260, "diff": -129.07488986784142, "zscore": 14.202148707155084, "poly": [[218, 226, 210, 234]]},
            {"expp": 0.0625, "exp_cnt": 113.5, "obs_cnt": 258, "diff": -127.31277533039648, "zscore": 14.008262717978905, "poly": [[18, 42, 2, 10]]},
            {"expp": 0.0625, "exp_cnt": 113.5, "obs_cnt": 258, "diff": -127.31277533039648, "zscore": 14.008262717978905, "poly": [[162, 170, 82, 90]]},
            {"expp": 0.0625, "exp_cnt": 113.5, "obs_cnt": 256, "diff": -125.55066079295155, "zscore": 13.81437672880273, "poly": [[10, 42, 18, 114]]},
            {"expp": 0.0625, "exp_cnt": 113.5, "obs_cnt": 254, "diff": -123.7885462555066, "zscore": 13.620490739626549, "poly": [[26, 34, 18, 42]]}
          ],
          "ref_samples": 40005,
          "ref_alpha": 2.4996875390576178e-05,
          "ref_minmax": [
            4.838657,
            7.835336
          ],
          "rejects": true
        },
        {
          "round": 1,
          "dists": [
            {"expp": 0.0625, "exp_cnt": 113.5, "obs_cnt": 137, "diff": -20.704845814977972, "zscore": 2.2781603728200994, "poly": [[218, 226, 210, 234]]},
            {"expp": 0.0625, "exp_cnt": 113.5, "obs_cnt": 135, "diff": -18.94273127753304, "zscore": 2.0842743836439195, "poly": [[18, 42, 2, 10]]},
            {"expp": 0.0625, "exp_cnt": 113.5, "obs_cnt": 132, "diff": -16.29955947136564, "zscore": 1.7934453998796516, "poly": [[26, 34, 18, 42]]},
            {"expp": 0.0625, "exp_cnt": 113.5, "obs_cnt": 128, "diff": -12.775330396475772, "zscore": 1.4056734215272961, "poly": [[162, 170, 82, 90]]},
            {"expp": 0.0625, "exp_cnt": 113.5, "obs_cnt": 127, "diff": -11.894273127753303, "zscore": 1.3087304269392062, "poly": [[10, 42, 18, 114]]}
          ],
          "halvings": [
            {"nsamples": 1816, "nsucc": 137, "pval": 0.025636234543933885, "poly": [[218, 226, 210, 234]]},
            {"nsamples": 1816, "nsucc": 135, "pval": 0.041559545918785974, "poly": [[18, 42, 2, 10]]},
            {"nsamples": 1816, "nsucc": 128, "pval": 0.15971032208437713, "poly": [[162, 170, 82, 90]]},
            {"nsamples": 1816, "nsucc": 127, "pval": 0.1904719077553753, "poly": [[10, 42, 18, 114]]},
            {"nsamples": 1816, "nsucc": 132, "pval": 0.08068532504703282, "poly": [[26, 34, 18, 42]]}
          ]
        }
      ]
    }
  ],
  "time_elapsed": 1.8015429973602295,
  "time_data_read": 0.013880252838134766,
  "time_data_bins": 9.608268737792969e-05,
  "time_process": 0.7150542736053467
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment