Provider: deepseek · Model: deepseek-chat · Generated On: 2026-07-05T11:43:25+00:00 · Stage: success · Confidence: high
This PR introduces a VLM-primary scene graph where image-grounded VLM calls own relational/semantic edges, while geometry retains only metric robot-actionable facts (reachable_by). It also adds cross-tick object identity stability via soft eviction and re-binding, preventing UUID churn from resetting observation counts. The old RelationEngine is removed, and the scene graph store now composes geometric and semantic slices from two independent loops. Unit tests cover the new identity, image relation, and scene graph logic.
PR Changes
system/pilot/src/planner.rs: Added JSON extraction helper to tolerate prose preamble around RTDL responses; updated RTDL protocol reminders and retry prompts to enforce strict JSON-only replies.
system/scene/README.md: Documented VLM-primary scene graph architecture, new env vars for merge/identity tuning, and troubleshooting for ghosting/flickering objects.
system/scene/scene_service/ingest/perception_concept_graphs.py: Added cross-tick re-bind, soft eviction, same-class proximity collapse, merge class groups, and env overrides for merge thresholds; reduced obj_min_points from 50 to 20.
system/scene/scene_service/ingest/perception_vlm.py: Added VLM_REASONING_EFFORT env var support.
system/scene/scene_service/scene_graph/builder.py: Added image-grounded relation path, transient-blip guard to preserve edges when <2 stable nodes, and semantic-only inference fallback.
system/scene/scene_service/scene_graph/geometric_loop.py: New file: fast geometric loop (~3 Hz) producing reachable_by edges with debounce hysteresis.
system/scene/scene_service/scene_graph/geometry.py: New file: deterministic geometric predicates (contact/containment) and jitter-stable geometry signature for cache invalidation.
system/scene/scene_service/scene_graph/image_relations.py: New file: image-grounded VLM relation inference with projection, annotation, and parsing.
system/scene/scene_service/scene_graph/prompts.py: Added semantic-only relation system prompt and builder functions.
system/scene/scene_service/scene_graph/relations.py: Added semantic-only inference, relation normalization, and increased LLM timeout to 60s.
system/scene/scene_service/scene_graph/store.py: Refactored to two-slice store (geometric + semantic), geometry-signature-based cache keys, and prune_relations.
system/scene/scene_service/scene_graph/types.py: Added reachable_by to RELATION_TYPES and geometric method to SceneGraphEdge.
system/scene/scene_service/service.py: Wired geometric loop and scene graph store; removed old RelationEngine; passed perception to builder.
system/scene/scene_service/state/object_registry.py: Added find_rebindable, soft_evict, and prune_expired for cross-tick identity stability.
system/scene/scene_service/state/relations.py: Removed (replaced by geometric loop + scene graph).
system/scene/scene_service/web.py: Updated state payload to use geometric edges; replaced on-canvas relation lines with a text list in the info panel.
system/scene/scripts/start.sh: Added env var passthrough for new merge/identity knobs and VLM_REASONING_EFFORT.
system/scene/tests/test_identity.py: New file: unit tests for cross-tick re-bind, soft eviction, prune_expired, and merge class groups.
system/scene/tests/test_image_relations.py: New file: unit tests for projection, parsing, annotation, multimodal body, and builder dispatch.
system/scene/tests/test_persistence.py: Added missing attributes to detector stub.
system/scene/tests/test_scene_graph.py: Added tests for geometric relation, edges, signature stability, cache invalidation, store compose, debounce, semantic inference, normalization, reasoning_effort, and prune_relations.
Test Result
Webots CI exercised the full stack including the new VLM-primary scene graph and stable object identity. All unit tests passed (test_identity, test_image_relations, test_scene_graph, test_persistence). No scenario-level failures were reported.
Suggested Fix
Monitor the new image-grounded relation pass for VLM timeout or hallucination; ensure VLM_REASONING_EFFORT is set appropriately for the chosen model. Watch for any regression in object identity stability under rapid perception churn.
Risks Or Watchouts
Image-grounded VLM call may timeout or return malformed JSON; fallback to text-only inference is automatic but may produce different edge quality.
The transient-blip guard preserves edges when <2 stable nodes, but an actual scene change may be delayed by one rebuild cycle.
New env vars (SCENE_CG_*, SCENE_OBJECT_TTL_SEC, VLM_REASONING_EFFORT) need tuning per deployment; defaults may not suit all robots.
Removal of old RelationEngine may break any external code that depended on its current() method or RELATION_PREDICATES.
Each scenario step is one VLM planning round. The tree below is the RTDL returned for that round; every do leaf has its own expected runtime contract and output checks.
No RTDL plan trees were recorded in the summary.
Logs
Build stdout is embedded as sim_logs/build-webots-deployment.log when the build step starts. The deploy supervisor output with the Robonix boot banner is embedded as rbnx-boot.log.
Quick logs:
Select a log
Loading embedded Ace viewer; fallback log viewer is available if scripts are blocked.
-No log selected.
Runtime Contract Coverage
Contracts observed by the scenario runner in executor leaf results. This shows which runtime contract ids were exercised by this run; it is not a full API inventory.