copyleftdev

PALIMPSEST — Deterministic Crawl Kernel

Same input + same seed = identical crawl, identical artifacts, identical replay. Everything bends around this. If we lose this, we've built noise — not history.

What This Is

A crawl kernel, not a crawler. Provides deterministic scheduling, reproducible fetch execution, verifiable artifact generation, replay-grade normalization, and temporal indexing as a first-class primitive. CLI, UI, dashboards are thin skin. The kernel is the product.

copyleftdev

//! palimpsest-envelope: The execution envelope.
//!
//! Every fetch runs inside a sealed context capturing:
//! seed, timestamp, headers, DNS resolution snapshot, TLS fingerprint,
//! and browser config (if applicable).
//!
//! The envelope is immutable after construction. It is the critical
//! abstraction that makes both determinism (Law 1) and replay fidelity
//! (Law 5) possible.

copyleftdev

//! Determinism verification harness.
//!
//! Crawls a SimulatedWeb twice with the same seed and asserts
//! bit-identical results. Any divergence is a Law 1 violation.

use std::path::Path;
use std::sync::Arc;

use chrono::Utc;
use palimpsest_core::hash::ContentHash;

copyleftdev

// Palimpsest Determinism Layer (Law 1)
// Injected BEFORE page navigation to ensure all JS execution
// is deterministic for a given seed.

// Freeze Date.now() to the envelope timestamp.
const __palimpsest_base_time = ENVELOPE_TIMESTAMP_MS;
let __palimpsest_time_offset = 0;
Date.now = function() {
    __palimpsest_time_offset += 1;
    return __palimpsest_base_time + __palimpsest_time_offset;

copyleftdev

//! Core domain types shared across all crates.

use rand_chacha::ChaCha8Rng;
use rand_chacha::rand_core::SeedableRng;
use serde::{Deserialize, Serialize};
use url::Url;

use crate::hash::ContentHash;
use crate::time::CaptureInstant;

copyleftdev

## The Six Laws (Never Break These)

1. **Determinism** — Frontier ordering is seed-driven. Retry logic is explicit. No hidden randomness anywhere. No `rand` in core paths — seeded PRNG only.
2. **Idempotence** — Same URL + same execution context = identical artifact hash.
3. **Content Addressability** — All artifacts are BLAKE3 hash-addressed. Deduplication is structural.
4. **Temporal Integrity** — Every capture binds wall clock + logical clock + crawl context + dependency chain.
5. **Replay Fidelity** — Stored artifacts must be sufficient to reconstruct the HTTP exchange, DOM state, and resource dependency graph.
6. **Observability as Proof** — Every decision is queryable. Every failure is replayable. Every artifact is verifiable.

copyleftdev

Supplemental IOC & Infrastructure Intelligence

Toll Booth Phishing Campaign — Deep Dive Follow-Up

Context: This supplements Ben Polonsky's writeup with net-new findings from active infrastructure reconnaissance conducted 2026-03-28. All probing was passive/non-invasive against already-identified phishing infrastructure.

---

1. Backend Framework Identification

The article identified OpenResty 1.29.2.1 as the web server. Behind it sits a second layer:

copyleftdev

#!/usr/bin/env python3
"""Fetch GitHub traffic stats for all repositories using gh CLI."""

import json
import subprocess
import sys
from concurrent.futures import ThreadPoolExecutor, as_completed

def run_gh(args: list[str]) -> dict | list | None:
    """Run gh api command and return parsed JSON."""

copyleftdev

🔧 The Ultimate JQ Cheat Sheet

jq — lightweight, flexible command-line JSON processor

📋 Table of Contents

• Basics
• Navigation
• Arrays
• Objects
• Filtering & Selection

copyleftdev

[
    {
        "id": "cards_structure_seed_100_low_entropy",
        "category": "structure",
        "description": "Verify minimal card count at low entropy",
        "url": "https://proteus-target.vercel.app/lab/cards?seed=100&entropy=0.1&profile=structure",
        "parameters": {
            "seed": 100,
            "entropy": 0.1,
            "profiles": [