One might need to get a better grasp at what Browsertrix is performing in terms of trafic and where it is being blocked.
You might use the following docker compose stack to run:
- a mitmweb web proxy to intercept all Browsertrix traffic and dump them on file
- a screencasting Browsertrix crawler to get a grasp on crawler behavior
If you open http://localhost:9037, you will see Browsertrix screencasting of the browser.
mitmdump are placed in the output/mitmdump
folder, with one file per minute.