Provider: deepseek · Model: deepseek-chat · Generated On: 2026-07-05T11:29:47+00:00 · Stage: success · Confidence: high
This PR introduces a VLM-primary scene graph where image-grounded VLM calls own relational/semantic edges, while geometry retains only metric robot-actionable facts (reachable_by). It also adds stable object identity via cross-tick re-binding and soft eviction, preventing object ID churn and observation count resets. The old RelationEngine is removed, and the scene graph store now supports two writer slices (fast geometric loop + slow LLM builder). New env knobs and unit tests are added for identity stability and image relation inference.
PR Changes
system/pilot/src/planner.rs: Added extract_json_object helper to tolerate prose preamble around JSON in RTDL responses; updated prompt to require JSON-only replies.
system/scene/README.md: Documented new env vars for merge/identity tuning, VLM-primary relations, and object TTL.
system/scene/scene_service/ingest/perception_concept_graphs.py: Added cross-tick re-bind, soft eviction, same-class proximity collapse, merge class groups, and many new env overrides for merge/identity knobs.
system/scene/scene_service/scene_graph/geometric_loop.py: New file: fast geometric loop producing reachable_by edges with debounce hysteresis.
system/scene/scene_service/scene_graph/geometry.py: New file: deterministic geometric predicates and jitter-stable geometry signature.
system/scene/scene_service/scene_graph/image_relations.py: New file: image-grounded VLM relation inference with projection, annotation, and parsing.
system/scene/scene_service/scene_graph/llm_client.py: Added multimodal image support and reasoning_effort forwarding.
system/scene/scene_service/scene_graph/prompts.py: Added semantic-only relation prompt and helper functions.
system/scene/scene_service/scene_graph/relations.py: Added semantic-only inference, relation normalization, and increased LLM timeout.
system/scene/scene_service/scene_graph/store.py: Refactored to two-slice store (geometric + semantic) with geometry-signature-based cache invalidation.
system/scene/scene_service/scene_graph/types.py: Added reachable_by to RELATION_TYPES and geometric method tag.
system/scene/scene_service/service.py: Wired geometric loop and scene graph store; removed old RelationEngine.
system/scene/scene_service/web.py: Updated web UI to show relations as text list instead of on-canvas dashed lines.
system/scene/scripts/start.sh: Added env var passthrough for new merge/identity knobs.
system/scene/tests/test_identity.py: New file: unit tests for object identity stability.
system/scene/tests/test_image_relations.py: New file: unit tests for image-grounded relation inference.
system/scene/tests/test_persistence.py: Updated test helper to set new attributes.
system/scene/tests/test_scene_graph.py: Added many new tests for geometry, cache invalidation, store composition, debounce, semantic inference, normalization, and LLM client.
Test Result
No machine-readable scenario summary was produced; logs are embedded below. The CI ran Webots integration tests but no pass/fail data is available in the provided context.
Suggested Fix
Monitor the new VLM-primary relation pass in production: ensure the image-grounded VLM call does not time out (VLM_REASONING_EFFORT=minimal may help). Watch for object identity stability under the new soft-eviction TTL (SCENE_OBJECT_TTL_SEC).
Risks Or Watchouts
Image-grounded VLM call may be slow (~30 s) and could time out if reasoning_effort is not set appropriately.
Soft-eviction TTL (default 30 s) may need tuning if objects briefly leave view for longer periods.
The new geometry-signature-based cache invalidation may cause occasional recomputation on bucket boundaries.
Removal of the old RelationEngine may affect any external consumers that relied on its output format.
Each scenario step is one VLM planning round. The tree below is the RTDL returned for that round; every do leaf has its own expected runtime contract and output checks.
No RTDL plan trees were recorded in the summary.
Logs
Build stdout is embedded as sim_logs/build-webots-deployment.log when the build step starts. The deploy supervisor output with the Robonix boot banner is embedded as rbnx-boot.log.
Quick logs:
Select a log
Loading embedded Ace viewer; fallback log viewer is available if scripts are blocked.
-No log selected.
Runtime Contract Coverage
Contracts observed by the scenario runner in executor leaf results. This shows which runtime contract ids were exercised by this run; it is not a full API inventory.