Method
Summary-as-State
ReViSe operates analogously to a recurrent neural network: it maintains a compact state that propagates information from previous rounds to the VLM, without re-admitting raw frames or conversation history. This is the load-bearing design choice that keeps the context compact even as evidence accumulates.
Each round, the agent receives sampled video frames, the question, and its current summary
state. Every response begins with a transient <think> reasoning trace,
then commits a structured summary in the POHR format inside
<summarize>:
| P Previously seen | What frames have been inspected so far |
| O Observations | What was just observed in the current frames |
| H Hypotheses | How observations update the current belief |
| U Uncertainties | What remains unclear |
| R Reasons | Why specific new frames are needed, or why the question is now answerable |
On a non-final round the agent appends a <select> request for new frames;
on the final round it emits only <think> and <answer>,
reusing the last committed summary. Only the <summarize> state persists
between rounds — the <think> trace is transient and raw frames are
never re-admitted — keeping the context compact.
The ReViSe agent loop: sample frames → update the POHR summary → request targeted frames or commit an answer.
1 · Plug-and-play
Wraps any VLM — including proprietary APIs like GPT-4o — as a frozen black box. No parameter updates needed.
2 · RL fine-tuning
GRPO with the EAGER reward, combining three annotation-free terms:
- Confidence gain — reward the increased log-odds gap to the correct option after adding frames.
- Summary sufficiency — re-ask using only the committed summary and reward success.
- Correct-and-early stop — reward answering correctly within a small turn budget.
