Skip to content

Instantly share code, notes, and snippets.

View Fitblip's full-sized avatar

Ryan Fitblip

View GitHub Profile
@Fitblip
Fitblip / tests.py
Created October 27, 2016 19:23
Django test for un-run migtraionts
'''
It's worth noting that we use the built-in User model, but also have a tweak that enforces email
uniqueness (which IMO is a pretty large oversight by the django folks), so that's why it's acceptable
that `Migrations for 'auth'` is in the payload.
'''
class TestForNonGeneratedMigrations(TestCase):
def test_for_migrations_to_run(self):
stdout = StringIO.StringIO()
call_command('makemigrations', no_color=True, dry_run=True, stdout=stdout)
@Fitblip
Fitblip / keybase.md
Created January 25, 2017 22:25
keybase.md

Keybase proof

I hereby claim:

  • I am fitblip on github.
  • I am fitblip (https://keybase.io/fitblip) on keybase.
  • I have a public key ASDrBgOsT14rT3qzqYUK2ypD6G-W9vHelq5PZY8wnbEPXQo

To claim this, I am signing this object:

import requests
import json
import locale
locale.setlocale(locale.LC_ALL, 'en_US')
ctl_log = requests.get('https://www.gstatic.com/ct/log_list/log_list.json').json()
total_certs = 0
human_format = lambda x: locale.format('%d', x, grouping=True)
// curl -s 'https://ct1.digicert-ct.com/log/ct/v1/get-entries?start=0&end=0' | jq .
{
"entries": [
{
"leaf_input": "AAAAAAFIyfaldAAAAAcDMIIG/zCCBeegAwIBAgIQBnEXzRy4ia2KEDMQEa+lMjANBgkqhkiG9w0BAQsFADB1MQswCQYDVQQGEwJVUzEVMBMGA1UEChMMRGlnaUNlcnQgSW5jMRkwFwYDVQQLExB3d3cuZGlnaWNlcnQuY29tMTQwMgYDVQQDEytEaWdpQ2VydCBTSEEyIEV4dGVuZGVkIFZhbGlkYXRpb24gU2VydmVyIENBMB4XDTE0MDkzMDAwMDAwMFoXDTE1MTAwNTEyMDAwMFowggEOMR0wGwYDVQQPDBRQcml2YXRlIE9yZ2FuaXphdGlvbjETMBEGCysGAQQBgjc8AgEDEwJVUzEVMBMGCysGAQQBgjc8AgECEwRVdGFoMRUwEwYDVQQFEww1Mjk5NTM3LTAxNDIxEjAQBgNVBAkTCVN1aXRlIDUwMDEkMCIGA1UECRMbMjYwMCBXZXN0IEV4ZWN1dGl2ZSBQYXJrd2F5MQ4wDAYDVQQREwU4NDA0MzELMAkGA1UEBhMCVVMxDTALBgNVBAgTBFV0YWgxDTALBgNVBAcTBExlaGkxFzAVBgNVBAoTDkRpZ2lDZXJ0LCBJbmMuMRwwGgYDVQQDExNjdDEuZGlnaWNlcnQtY3QuY29tMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAyysV+OCPSn5MnCri+xQk7v+iOuA8EoSNUJqvTA95F1OEPzK0szO7seL6NlHPS4/Qkza9/l4d5AyZN7z7GHz1T955WiOmFOxlt3x5wrPUnsUJoEcv9fGfb6rjRdPKxgTJP+0EVup/jV0uvokyRkNzABhZOHi023gMkuudh75yOiYUYllUij0mQR/ZBMRg7Iigj2o9t/zKyHHY
from construct import Struct, Byte, Int16ub, Int64ub, Enum, Bytes, Int24ub, this, GreedyBytes, GreedyRange, Terminated, Embedded
MerkleTreeHeader = Struct(
"Version" / Byte,
"MerkleLeafType" / Byte,
"Timestamp" / Int64ub,
"LogEntryType" / Enum(Int16ub, X509LogEntryType=0, PrecertLogEntryType=1),
"Entry" / GreedyBytes
)
@Fitblip
Fitblip / schema.json
Created May 17, 2017 22:01
BigQuery Schema
[
{
"name": "url",
"type": "STRING",
"mode": "REQUIRED"
},
{
"mode": "REQUIRED",
"name": "cert_index",
"type": "INTEGER"
@Fitblip
Fitblip / bigquery.sql
Created May 17, 2017 22:18
BigQuery Punycode Example
SELECT
all_dns_names
FROM
[ctl-lists:certificate_data.scan_data]
WHERE
(REGEXP_MATCH(all_dns_names,r'\b?xn\-\-'))
AND NOT all_dns_names CONTAINS 'cloudflare'
@Fitblip
Fitblip / CoinbaseBigQuery.sql
Last active May 17, 2017 22:26
Finding all coinbase sites using BigQuery
SELECT
all_dns_names
FROM
[ctl-lists:certificate_data.scan_data]
WHERE
(REGEXP_MATCH(all_dns_names,r'.*\.coinbase.com[\s$]?'))
SELECT
url,
COUNT(*) AS total_certs
FROM
[ctl-lists:certificate_data.scan_data]
WHERE
(REGEXP_MATCH(all_dns_names,r'.*flowers-to-the-world.*'))
GROUP BY
url
ORDER BY
@Fitblip
Fitblip / find_needle.py
Created June 24, 2017 05:24
Find a specific keyword (paypal) in any certificates sent by Certstream
import certstream
import base64
NEEDLE = "paypal"
# Search for domains with a keyword in them and write the corresponding certificate to a file
def certstream_callback(message):
if message['message_type'] == "certificate_update":
all_domains = message['data']['leaf_cert']['all_domains']
if NEEDLE.lower() in " ".join(all_domains).lower():