Skip to content

Instantly share code, notes, and snippets.

🌴
On vacation until July 8

Philip Jägenstedt foolip

🌴
On vacation until July 8
Block or report user

Report or block foolip

Hide content and notifications from this user.

Learn more about blocking users

Contact Support about this user’s behavior.

Learn more about reporting abuse

Report abuse
View GitHub Profile
View hasExtension-bodies.csv
We can't make this file beautiful and searchable because it's too large.
page,url,body,truncated
http://www.wetafx.co.nz/,https://www.wetafx.co.nz/themes/weta/js/main_desktop_extended.js,"/*!
* jQuery JavaScript Library v2.1.4
* http://jquery.com/
*
* Includes Sizzle.js
* http://sizzlejs.com/
*
* Copyright 2005, 2014 jQuery Foundation, Inc. and other contributors
View hasExtension.csv
page url
http://www.baotuyetmobile.vn/ https://live.vipchat.vn/static/js/5.fc7599ee.chunk.js
http://www.politicsandwar.com/ https://cdn.plot.ly/plotly-latest.min.js
http://www.merchology-canada.myshopify.com/ https://cdn.shopify.com/s/files/1/1600/4693/t/20/assets/script.js?556945129885721949
http://www.data.world/ https://assets.data.world/js/app.20a06785d2d8599e3d7e.js
http://www.angular.cn/ https://www.angular.cn/worker-basic.min.js
http://www.facturaelectronicagfa.mx/ https://www.facturaelectronicagfa.mx/worker-basic.min.js
http://www.facturaelectronicagfa.mx/ https://www.facturaelectronicagfa.mx/worker-basic.min.js?_sw-precache=df10515adec5f5e556a4db44114b5f28
http://www.merchology.com/ https://cdn.shopify.com/s/files/1/0312/6537/t/137/assets/script.js?14395824414243321746
http://www.dare2compete.com/ https://dare2compete.com/worker-basic.min.js
View majestic-sample-analysis.md

Use of Accept-Language

Sites were sampled from the Majestic Million, 10 site per order of magnitude. This was done based on the suspicion that use of the header is different in the long tail of sites.

Testing was done in Sweden with different Accept-Language headers based on informed guesses about what languages the sites might support. Between any two attempts using the same site cookies were cleared to avoid previous language being remembered, but it is still possible the server had state based on IP that affected results.

Summary

By rank (buckets of 10):

  • 10-99: 2/10 sites fully localized, 2/10 localized GDPR prompts, 2/10 used physical location/IP.
View majestic-sample.txt
# output of https://gist.github.com/foolip/d18bd25bd2547c225b4a677dbda31e4b (first revision, with two manual replacements after)
1
forbes.com
mit.edu
godaddy.com
itunes.apple.com
t.co
weebly.com
live.com
View accept-language.py
#!/usr/bin/python3
import csv
import math
import random
rows_by_logrank = {}
with open('majestic_million.csv', 'r') as csvfile:
majestic = csv.DictReader(csvfile)
View update.js
'use strict';
const compareVersions = require('compare-versions');
const fs = require('fs');
const path = require('path');
const uaParser = require('ua-parser-js');
const overrides = require('./overrides').filter(Array.isArray);
function findEntry(bcd, path) {
View update.js
'use strict';
const compareVersions = require('compare-versions');
const fs = require('fs');
const path = require('path');
const uaParser = require('ua-parser-js');
const overrides = require('./overrides').filter(Array.isArray);
function findEntry(bcd, path) {
View update.js
'use strict';
const compareVersions = require('compare-versions');
const fs = require('fs');
const path = require('path');
const uaParser = require('ua-parser-js');
const overrides = require('./overrides').filter(Array.isArray);
function findEntry(bcd, path) {
View update.js
'use strict';
const compareVersions = require('compare-versions');
const fs = require('fs');
const path = require('path');
const uaParser = require('ua-parser-js');
const overrides = require('./overrides').filter(Array.isArray);
function findEntry(bcd, path) {
View update.js
'use strict';
const compareVersions = require('compare-versions');
const fs = require('fs');
const path = require('path');
const uaParser = require('ua-parser-js');
const overrides = require('./overrides').filter(Array.isArray);
function findEntry(bcd, path) {
You can’t perform that action at this time.