davidsvaughn

Yep — you can keep a plain CausalLM behind vLLM and still get a tunable yes/no cutoff with zero architecture changes. You’ve got two clean patterns that work with the OpenAI-compatible server:

---

Option A (recommended): Score → threshold in your client

1. Ask vLLM for the next-token logprobs (no real decoding—max_tokens=1, temperature=0).
2. Read the logprob of the “yes” token and the “no” token at that step.
3. Use a threshold on the logprob difference $\Delta = \log p(\text{yes}) - \log p(\text{no})$.

davidsvaughn

from keras.layers import Recurrent
import keras.backend as K
from keras import activations
from keras import initializers
from keras import regularizers
from keras import constraints
from keras.engine import Layer
from keras.engine import InputSpec