Change what a model does, and know that the change was real.
We study representation steering in large language models and vision-language models: how a behaviour is shifted at runtime, and how you know the shift held.
Editing a running model is possible. Knowing it worked is the hard part.
A language model reads. A vision-language model reads and sees. Teams already ask both to be more careful, more factual, or more specific. Retraining is slow, and editing the running model is possible.
The retina study asks whether a highlighted region earned the grade. This work asks the matching question of a steered model. A gain over the untouched model is a start. A gain that survives a strong alternative, and whose cost in capability or compute has been counted, is the claim worth keeping.
The same standard covers models that only read and models that also see. A result that holds for one and is untested on the other will be labelled that way.
STEER: one description for every intervention.
Whether the model handles text, or images and text together, each intervention is described the same way.
Working on steering or model editing?
We’re open to collaboration on evaluation, especially for vision-language models.