Skip to content

Instantly share code, notes, and snippets.

@SophiaHatzPCR
Last active August 4, 2026 07:49
Show Gist options
  • Select an option

  • Save SophiaHatzPCR/6e08c8117d95fd138104366e28529f4d to your computer and use it in GitHub Desktop.

Select an option

Save SophiaHatzPCR/6e08c8117d95fd138104366e28529f4d to your computer and use it in GitHub Desktop.
Model and figures for "Would We See It Coming? Preference Falsification Cascades in Multi-Agent Systems

The model

Technical appendix to "Would We See It Coming? Preference Falsification Cascades in Multi-Agent Systems" (LessWrong, August 2026). It states what the simulations in that post compute. The R script that produces every figure is in the same gist.

AI disclosure: This appendix and the R script were written by Claude (Anthropic) and checked by me for accuracy.

Notation:

  • N: the number of agents, 100 throughout.
  • threshold (t): the visible count of revealing agents at which an agent reveals its own misalignment. With N = 100 a count and a percentage coincide.
  • visible count: how many agents appear to be revealing, those who have plus the spark.
  • sd: heterogeneity, the standard deviation of the normal distribution the thresholds are drawn from. A wide spread is a varied population; at zero every agent holds the same threshold.
  • tau: threshold softness. At tau = 0 an agent reveals exactly when the visible count reaches its threshold; at tau > 0 it reveals with a logistic probability that rises through the threshold, so revelation is noisy rather than a hard cut-off.
  • gamma: transparency, the speed at which agents learn the visible count. Unset (or at its limit) agents see the true count at once; near zero they see a lagged estimate that chases the truth by a fraction gamma each round, so discovery is slow and serial.
  • spark: at a set round, a small fixed count is added to the visible count from outside, as if that many agents had revealed, enough to push the lowest-threshold agents over.

The model. Each latently-misaligned agent carries a revelation threshold; genuinely aligned agents never reveal, their threshold set out of reach. The stock of misaligned agents is fixed for a run. Before the spark the visible count is zero and no one reveals. From the spark round onward the spark's count is added to the visible count; agents whose threshold the visible count has reached reveal (hard tau) or may reveal (soft tau), which raises the count, which brings the next agents over. Updates are synchronous, and revelation is absorbing: once an agent reveals it stays revealed, so the model runs one way only and cannot show silence rebuilding. Thresholds are drawn from a normal distribution, rounded and clamped. In Figures 2 to 5 the clamp is [1, 100], the heterogeneity conditions vary the standard deviation of the draw and the transparency conditions vary gamma. Figure 1 places a 15% misaligned subset in a population of 100, the other 85 agents with their threshold set out of reach; there the clamp is [1, 16], since the visible count can never exceed the size of the misaligned subset plus the spark. D_true in Figure 1 is not simulated: it is drawn as a flat line at 15%, standing for the assumption that this share privately dissents. Figures 2 to 5 model a population made up of misaligned agents. The vertical axis in Figures 2 and 3 is the share of the misaligned agents that have revealed; in Figures 4 and 5 it is the warning window itself, the number of rounds between that share passing 5% and 95%.

One function carries the model:

simulate <- function(thresholds, tau = 0, spark_round = 0, spark_size = 0,
                     rounds = 40, gamma = NULL) {
  N <- length(thresholds)
  state <- logical(N)              # who has revealed (all FALSE)
  traj <- integer(rounds)
  observed <- 0                    # lagged estimate of the visible count (used only if gamma set)
  for (r in seq_len(rounds)) {
    round0 <- r - 1
    if (spark_round > 0 && round0 < spark_round) {   # before the spark: nobody reveals
      traj[r] <- 0L; next
    }
    true_visible <- sum(state) + if (round0 >= spark_round) spark_size else 0
    if (is.null(gamma)) {
      visible <- true_visible                        # agents see the true count instantly
    } else {
      observed <- observed + gamma * (true_visible - observed)   # estimate chases the truth
      visible <- observed
    }
    if (tau <= 0) {
      newly <- (thresholds <= visible) & !state      # hard cutoff
    } else {
      p <- 1 / (1 + exp(-(visible - thresholds) / tau))   # soft: logistic probability
      newly <- (runif(N) < p) & !state
    }
    state <- state | newly                           # revelation is absorbing
    traj[r] <- sum(state)
  }
  traj
}

Everything in the figures is this function, run over different threshold draws and values of gamma, tau, and the spark. Figures 1 to 3 plot single runs. Figures 4 and 5 plot a mean over 20 seeded repetitions at each point on the axis; repetitions in which the cascade never reaches 95% are dropped from that mean.

Two notes on the spark, since its size affects how fast a cascade completes and therefore how wide the warning window is. Figures 3 and 5 use the same population and the same spark, so they are one experiment shown two ways: Figure 3 gives four trajectories, Figure 5 the width of the window across fourteen levels of transparency. In Figure 4 the spark is instead set to the lower decile of each run's own threshold distribution, so that every population gets a comparable push. A small fixed spark would leave a homogeneous population unlit, since every agent there waits for the same count.

References:

Granovetter, M. (1978). Threshold Models of Collective Behavior. American Journal of Sociology, 83(6), 1420–1443. https://doi.org/10.1086/226707

Kuran, T. (1989). Sparks and Prairie Fires: A Theory of Unanticipated Political Revolution. Public Choice, 61(1), 41–74. https://doi.org/10.1007/BF00116762

# Figures for "Would We See It Coming? Preference Falsification Cascades in Multi-Agent Systems"
# A small Granovetter/Kuran threshold-model simulation. Running this script top to bottom
# reproduces the five figures in the post and writes them to pf-figures/.
# Requires R with ggplot2. Figures with soft thresholds use random draws; seeds are set.
# Written by Claude (Anthropic) and checked by me for accuracy.
library(ggplot2)
dir.create("pf-figures", showWarnings = FALSE)
BLUE <- "#1A4F8B"; RED <- "#C1442E"; GREY <- "#8A8F98"
theme_wt <- theme_classic(base_size = 12) +
theme(plot.title = element_text(face = "bold", size = 13),
plot.title.position = "plot",
legend.title = element_blank(),
legend.background = element_blank(),
legend.key = element_blank(),
legend.text = element_text(size = 9))
# The model: given a set of thresholds, a spark, and two dials (softness tau,
# transparency gamma), return how many agents have revealed in each round.
simulate <- function(thresholds, tau = 0, spark_round = 0, spark_size = 0,
rounds = 40, gamma = NULL) {
N <- length(thresholds)
state <- logical(N) # who has revealed (all FALSE)
traj <- integer(rounds)
observed <- 0 # lagged estimate of the visible count (used only if gamma set)
for (r in seq_len(rounds)) {
round0 <- r - 1
if (spark_round > 0 && round0 < spark_round) { # before the spark: nobody reveals
traj[r] <- 0L; next
}
true_visible <- sum(state) + if (round0 >= spark_round) spark_size else 0
if (is.null(gamma)) {
visible <- true_visible # agents see the true count instantly
} else {
observed <- observed + gamma * (true_visible - observed) # estimate chases the truth
visible <- observed
}
if (tau <= 0) {
newly <- (thresholds <= visible) & !state # hard cutoff
} else {
p <- 1 / (1 + exp(-(visible - thresholds) / tau)) # soft: logistic probability
newly <- (runif(N) < p) & !state
}
state <- state | newly # revelation is absorbing
traj[r] <- sum(state)
}
traj
}
# --- Fig 1: what aggregate monitoring cannot see ---
set.seed(3); N <- 100; Dtrue <- 15
n_dis <- Dtrue
th_dis <- pmin(pmax(round(rnorm(n_dis, 9, 5)), 1), 16) # dissenters: low, spread thresholds
th <- c(th_dis, rep(9999, N - n_dis)) # the rest never reveal
Dobs <- simulate(th, tau = 3, spark_round = 8, spark_size = 2, rounds = 40) / N * 100
d1 <- data.frame(round = 0:(length(Dobs) - 1), Dobs = Dobs)
lt <- sprintf("D_true: private misalignment (%d%%)", Dtrue)
lo <- "D_obs: what the monitor sees"
lines1 <- rbind(data.frame(round = range(d1$round), value = Dtrue, series = lt),
data.frame(round = d1$round, value = d1$Dobs, series = lo))
lines1$series <- factor(lines1$series, levels = c(lt, lo))
p1 <- ggplot() +
geom_ribbon(data = d1, aes(round, ymin = Dobs, ymax = Dtrue), fill = RED, alpha = 0.09) +
geom_vline(xintercept = 8, color = GREY, linetype = "dotted") +
geom_line(data = lines1, aes(round, value, color = series, linetype = series), linewidth = 1) +
annotate("text", x = 8.6, y = 2.2, label = "spark", color = GREY, hjust = 0, size = 3.4) +
annotate("text", x = 2.6, y = 9.2, label = "what aggregate monitoring\ncannot see", color = RED, hjust = 0, size = 3.3) +
scale_color_manual(values = setNames(c(RED, BLUE), c(lt, lo))) +
scale_linetype_manual(values = setNames(c("dashed", "solid"), c(lt, lo))) +
coord_cartesian(ylim = c(-0.5, 19)) +
scale_y_continuous(breaks = seq(0, 15, 5)) +
labs(x = "round", y = "share revealing misalignment (%)", title = "") +
theme_wt +
theme(legend.position = "inside", legend.position.inside = c(0.99, 0.55), legend.justification = c(1, 0.5))
ggsave("pf-figures/fig1.png", p1, width = 7, height = 4.2, dpi = 150)
# --- Fig 2: shape follows heterogeneity (transparency held fixed) ---
set.seed(3); N <- 100; mu <- 45; rounds <- 60; sr <- 8; tau_fixed <- 1.5; q_seed <- 0.10
draw <- function(sd) pmin(pmax(round(rnorm(N, mu, sd)), 1), 100)
specs <- list(
list(lab = "broad variation (sd = 30)", sd = 30, col = "#1A4F8B"),
list(lab = "moderate variation (sd = 15)", sd = 15, col = "#5A7FB0"),
list(lab = "narrow variation (sd = 6)", sd = 6, col = "#A85A5A"),
list(lab = "no variation; step (sd = 0)", sd = 0, col = "#C1442E")
)
rows <- do.call(rbind, lapply(specs, function(s) {
th <- if (s$sd == 0) rep(mu, N) else draw(s$sd)
spark <- round(quantile(th, q_seed)) # matched seed: the lower decile of each run's own thresholds
tr <- simulate(th, tau = tau_fixed, spark_round = sr, spark_size = spark, rounds = rounds, gamma = 0.15)
data.frame(round = 0:(rounds - 1), value = tr / N * 100, series = s$lab)
}))
rows$series <- factor(rows$series, levels = sapply(specs, `[[`, "lab"))
cols <- setNames(sapply(specs, `[[`, "col"), sapply(specs, `[[`, "lab"))
p2 <- ggplot(rows, aes(round, value, color = series)) +
geom_vline(xintercept = sr, color = GREY, linetype = "dotted") +
geom_line(linewidth = 1) +
annotate("text", x = sr + 1, y = 28, label = "spark", color = GREY, hjust = 0, size = 3.4) +
scale_color_manual(values = cols) +
coord_cartesian(ylim = c(-3, 108), xlim = c(0, 55)) +
labs(x = "round", y = "share revealing misalignment (%)", title = "") +
theme_wt +
theme(legend.position = "inside", legend.position.inside = c(0.99, 0.45),
legend.justification = c(1, 0.5))
ggsave("pf-figures/fig2.png", p2, width = 7, height = 4.2, dpi = 150)
# --- Fig 3: opacity trajectories (heterogeneity held fixed) ---
set.seed(3); th_het <- pmin(pmax(round(rnorm(100, 45, 26)), 1), 100) # one broad population, held fixed (same draw as Fig 5)
spark_h <- round(quantile(th_het, 0.10))
gammas_o <- c(0.05, 0.15, 0.4, 1.0)
labs_o <- c("opaque (gamma = 0.05)", "low opacity (gamma = 0.15)", "moderate opacity (gamma = 0.4)", "transparent (gamma = 1.0)")
rows_o <- do.call(rbind, lapply(seq_along(gammas_o), function(i) {
set.seed(500 + i)
tr <- simulate(th_het, tau = 1.5, spark_round = 8, spark_size = spark_h, rounds = 400, gamma = gammas_o[i])
data.frame(round = 0:399, value = tr / length(th_het) * 100, series = labs_o[i])
}))
rows_o <- rows_o[rows_o$round >= 1, ] # log x-axis cannot show round 0
rows_o$series <- factor(rows_o$series, levels = labs_o)
p_o <- ggplot(rows_o, aes(round, value, color = series)) +
geom_vline(xintercept = 8, color = GREY, linetype = "dotted") +
geom_line(linewidth = 1) +
scale_x_log10() +
annotate("text", x = 5, y = 28, label = "spark", color = GREY, hjust = 0, size = 3.4) +
scale_color_manual(values = setNames(c("#1A4F8B", "#5A7FB0", "#A85A5A", "#C1442E"), labs_o)) +
coord_cartesian(ylim = c(-3, 108)) +
labs(x = "round (log scale)", y = "share revealing misalignment (%)", title = "") +
theme_wt +
theme(legend.position = "inside", legend.position.inside = c(0.02, 0.98), legend.justification = c(0, 1))
ggsave("pf-figures/fig3.png", p_o, width = 7, height = 4.2, dpi = 150)
# --- Fig 4: warning window vs heterogeneity (controlled sweep) ---
N <- 100; mu <- 45; sr <- 8; tau_fixed <- 1.5; q_seed <- 0.10
rise_by_spread <- function(sd, reps = 20) {
outs <- numeric(0)
for (k in seq_len(reps)) {
set.seed(400 + k)
th <- if (sd == 0) rep(mu, N) else pmin(pmax(round(rnorm(N, mu, sd)), 1), 100)
spark <- round(quantile(th, q_seed))
tr <- simulate(th, tau = tau_fixed, spark_round = sr, spark_size = spark, rounds = 300)
d <- tr / N * 100
if (max(d) < 95) next # skip runs that never really complete
outs <- c(outs, max(which.max(d >= 95) - which.max(d >= 5), 0)) # rounds from 5% to 95%
}
if (length(outs)) mean(outs) else NA_real_
}
sds <- seq(0, 30, length.out = 13)
d2s <- data.frame(sd = sds, rise = sapply(sds, rise_by_spread))
p2s <- ggplot(d2s, aes(sd, rise)) +
geom_line(color = BLUE, linewidth = 1) + geom_point(color = BLUE, size = 2.4) +
scale_x_reverse() +
labs(x = "heterogeneity, decreasing from broad variation (sd = 30) to homogeneity (sd = 0)",
y = "warning window (rounds)", title = "") +
theme_wt
ggsave("pf-figures/fig4.png", p2s, width = 7, height = 4.2, dpi = 150)
# --- Fig 5: warning window vs transparency (population held fixed) ---
set.seed(3); th_fixed <- pmin(pmax(round(rnorm(100, 45, 26)), 1), 100) # drawn once, held fixed
spark_fixed <- round(quantile(th_fixed, 0.10)) # same population and same spark as Fig 3
rise_time <- function(gamma, reps = 20) {
outs <- numeric(0)
for (k in seq_len(reps)) {
set.seed(300 + k)
tr <- simulate(th_fixed, tau = 1.5, spark_round = 8, spark_size = spark_fixed, rounds = 400, gamma = gamma)
d <- tr / length(th_fixed) * 100
if (max(d) < 95) next
outs <- c(outs, max(which.max(d >= 95) - which.max(d >= 5), 0))
}
if (length(outs)) mean(outs) else NA_real_
}
gammas <- seq(0.03, 1.0, length.out = 14)
d3 <- data.frame(gamma = gammas, rise = sapply(gammas, rise_time))
floor_rise <- d3$rise[which.max(d3$gamma)] # window that survives at full transparency
p3 <- ggplot(d3, aes(gamma, rise)) +
geom_hline(yintercept = floor_rise, linetype = "dotted", color = GREY) +
geom_line(color = BLUE, linewidth = 1) + geom_point(color = BLUE, size = 2.4) +
annotate("text", x = 0.8, y = floor_rise + 15, hjust = 0, size = 3.4, color = GREY, label = "floor") +
labs(x = "opacity, decreasing from opaque (gamma = 0.03) to transparent (gamma = 1)",
y = "warning window (rounds)", title = "") +
theme_wt
ggsave("pf-figures/fig5.png", p3, width = 7, height = 4.2, dpi = 150)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment