Selective visual inference with observable decisions.
Agentic Vision explores a simple idea: visual intelligence does not have to treat every frame as a completely new problem.
A lightweight scene descriptor identifies when the observed environment remains semantically stable and when a new interpretation is warranted. When the descriptor matches, the previous interpretation can be reused. When the scene changes materially, perception can be refreshed.
The result is a measurable tradeoff between inference cost and perceptual sensitivity — with the decision itself preserved as part of the system’s evidence.
The architecture originated in an agentic DOOM environment, where thousands of compiled agents interacted with a live game through a Model Context Protocol gateway. Broker agents observed different aspects of the scene — entities, motion, zones, and HUD state — and invoked vision selectively as those observations changed.
Agentic Vision carries that architecture into real-world imagery.
Road footage extends the architecture into a setting where perception itself is uncertain, allowing different scene descriptors to be evaluated under real-world visual variation.
The game build is still running, if you want to see the architecture in its original habitat: agentarena.ca.
Across 1,712 contiguous dashcam frames sampled at 5 Hz, progressively richer scene descriptors produced a clear engineering tradeoff.
| Scene descriptor | Model calls skipped | Material changes hidden |
|---|---|---|
| Class counts | 43.6% | 28.5% |
| + spatial band | 37.1% | 23.5% |
| + spatial band + scale | 22.1% | 9.1% |
A class-count descriptor skipped 43.6% of candidate model calls. Adding spatial and scale information reduced reuse, and reduced the material changes requiring refreshed perception from 28.5% to 9.1%.
The important result is not a single optimum. It is that the tradeoff is measurable.
Scene representation can therefore become an explicit engineering parameter: richer descriptors spend more computation distinguishing states; simpler descriptors maximize reuse.
A second perception channel provides an independent view of the same frames.
This captures changes that may not be represented by the detector defining the primary scene state. Cross-detector evaluation therefore provides a stronger measurement of selective perception, and makes disagreement between perception channels observable.
| Scene descriptor | Hidden, single channel | Hidden, cross-detector |
|---|---|---|
| Class counts | 21.5% | 28.5% |
| + spatial band | 15.5% | 23.5% |
| + spatial band + scale | 1.0% | 9.1% |
In this run, the two channels disagreed about the presence of vulnerable road users on 9.5% of frames.
That disagreement is useful evidence. Instead of disappearing inside the perception pipeline, it becomes something the system can measure, label, and reason about.
A scene fingerprint is a descriptor. Reusing an existing interpretation is an admission decision made against that descriptor.
Agentic Vision applies the Agentic Datasets model to continuously arriving perception:
Sources declare what evidence they provide; experiments declare what evidence they require; admission is evaluated before execution.
| Source | Temporal experiment | Descriptor collisions | Live imagery |
|---|---|---|---|
| Academic stills | requires track identity | admitted | requires live pixels |
| Academic tracks | admitted | admitted | requires live pixels |
| Dashcam capture | admitted (derived) | admitted (derived) | admitted |
| Simulation capture | admitted | admitted | admitted |
A temporal experiment requiring track identity is admitted only for a source capable of supplying tracks. Derived channels remain explicitly identified as derived. Live-image experiments require pixels.
This makes the relationship between data, capability, and execution explicit.
Selective inference therefore becomes more than an optimization: it becomes an observable, measurable, and governable part of the perception architecture.
Agentic Vision is an independent technical project initiated and maintained by Alexander Chernov. It is an application of Agentic Datasets to visual perception.
The current release focuses on reproducible measurement of the selective-inference decision layer. Its recorded pipeline evaluates 1,712 frames using two independently derived perception channels, allowing descriptor reuse, perceptual change, and cross-detector disagreement to be measured directly.
The next stage extends the same architecture from replay measurement to live model invocation and adaptive perception.
Maintainer: Alexander Chernov · ORCID 0009-0007-3198-2712