Autonomous Systems

1. The autonomous systems spectrum

The self-driving domain doc covers one application — road vehicles — in depth. This document covers the broader class: any autonomous system that makes real-time decisions under uncertainty, subject to hard safety constraints, across a range of platforms that share the same structural problems even though their operating environments differ substantially.

What all of these share: real-time decisions under uncertainty, hard safety constraints that must not be violated, objectives that conflict across operating contexts, and the same structural problem with static AI — there is no feedback loop between detected errors and system behavior between releases.

2. The structural problem with static AI in autonomous systems

Current approaches to AI in autonomous systems treat the AI component as a static artifact: trained offline, validated at design time, deployed into the field. When the AI makes an error — a perception failure at a specific junction type, a planning bias in narrow corridors, a wrong torque estimate under load — there is no mechanism to correct that behavior between training cycles. The system will make the same error tomorrow, and in every unit in the fleet, until a new model version ships.

For autonomous systems, this failure mode compounds in three ways that make it worse than it is for general AI:

• Errors are physical. A wrong planning decision does not produce a wrong text response — it produces a robot collision, a dropped payload, a drone that fails to abort during a storm. The cost of repeated errors is not user frustration; it is safety incidents and regulatory action.
• Operating environments change faster than training cycles. A cobot deployed in a factory where a new human operator joins the team encounters a new movement pattern it was not trained on. An inspection drone enters a tunnel with different acoustic properties than the training set. These environmental drifts happen continuously; retraining cycles happen quarterly at best.
• Safety cases become stale. Regulatory and certification bodies are increasingly requiring that safety arguments hold not just at design time but throughout the operational life of the system. A static AI component whose behavior cannot be explained, tracked, or corrected at runtime makes the safety case a point-in-time proof that grows less defensible with every deployment hour.

The framework addresses all three: the correction loop reduces repeated errors between releases, the confidence gate handles environmental drift by abstaining rather than proceeding on low confidence, and the utility log + assertions store produce the runtime evidence that living safety cases require.

3. Context-adaptive weight shifting across operating modes

The same autonomous system operates under fundamentally different risk profiles across its operating modes. A cobot moving freely in a cage-free zone has very different safety requirements than the same cobot executing a collaborative task 30cm from a human operator. A delivery drone in clear conditions is operating very differently from the same drone with a battery at 20% capacity and a storm front approaching.

The framework's field-weighted utility function handles this with a single formula — the weights shift with context, producing different behavior without separate rule sets for each scenario:

U = w_s(f) · Safety  +  w_e(f) · Efficacy  +  w_c(f) · Comfort/Throughput

f — operating context (human-present, hazard zone, normal, degraded sensors...)

Decision rule:
  act    if C ≥ C_min(f)  AND  E ≥ E_min(f)
  safe-state / abstain     otherwise

Weight profiles across autonomous system operating contexts

Sensor degradation and confidence-driven weight tightening

When sensor fusion confidence falls — lidar degraded in dust, camera occluded, GPS denied in a tunnel — the confidence term C falls independently of the weight profile shift. The interaction between falling C and C_min produces the appropriate response: C_min for safety-critical autonomous systems is typically 0.85–0.95, meaning that sensor degradation drives abstention before it drives incorrect action. This is the designed behavior, not a failure mode.

4. Confidence gates, safe states, and formal abstention

Every safety standard governing autonomous systems requires the system to transition to a safe state when it reaches the boundary of its competence. ISO 10218:2025 requires application-specific Performance Level (PL) determination with defined fail-safe behaviors. IEC 61508 requires safe failure fractions and fail-operational / fail-safe mode definitions. The framework's confidence gate produces exactly this behavior formally and testably.

act        if  C ≥ C_min(f)  AND  E ≥ E_min(f)
safe-state if  C < C_min(f)  OR   E < E_min(f)

The gate is a hard decision boundary, not a soft preference. Below C_min, the system does not produce a lower-quality output — it produces no output and initiates the safe-state transition. This is what makes the abstention behavior formally testable: a test suite can verify that for any input that drives C below C_min, the system always enters the safe state.

Illustrative thresholds. Must be validated against the specific ODD and risk assessment for each application. See §5.1 of the full whitepaper for the C_min derivation methodology from professional licensing standards.

5. Specialist decomposition for the autonomy stack

Autonomous systems already reason in terms of functional decomposition — perception, planning, and control are distinct pipelines in every production autonomy stack. The Micro-Expert model formalizes this at the model-graph level: each functional domain becomes an independently deployable specialist with its own weights, calibration cycle, confidence signal, and blue-green deployment lifecycle.

The inter-specialist interface is a structured protocol that all specialists share:

Request:  { query, context, field, confidence_floor, session_id }
Response: { answer, confidence, assertions[], uncertainty_flags, U_score }

Contradiction between specialists — planning specialist generates a proceed trajectory while the safety constraint specialist flags a threshold violation — is the key case the Arbiter is designed for. The Arbiter's conservation-first policy means: under any unresolved contradiction involving the safety constraint specialist, the most conservative safe output wins. This is not heuristic; it is the defined arbitration policy enforced by the confidence threshold gate on the Arbiter's own output.

6. The dynamic safety case argument

This is the section of this document that has no equivalent in the self-driving domain doc, because it is most relevant to safety-case engineering teams rather than stack engineers. It addresses a problem that the autonomous systems safety research community has identified as critical: static safety cases become obsolete as systems evolve.

The problem with static safety cases

A traditional safety case is a structured argument — built at design time — that a system meets its safety requirements given the evidence available at that point. For autonomous systems, this is increasingly untenable: the system's AI components learn, its operating environment changes, its software is updated. A safety argument that was valid at initial certification may not be valid after a software update, a new deployment geography, or 10,000 hours of operational learning.

The current research frontier in autonomous systems safety (2024–2025) is the concept of dynamic safety cases — living assurance documents that co-evolve with the operational system, drawing on runtime evidence rather than relying solely on design-time proofs. Regulatory bodies across sectors (CAA, HSE, ONR, RSSB) are converging on the requirement that autonomous systems produce runtime evidence of continued safety compliance, not just a point-in-time certification.

What the framework contributes to the dynamic safety case

The framework's architecture produces exactly the runtime evidence that dynamic safety cases require, as a byproduct of its normal operation:

The three safety argument claims the framework directly supports

A safety case for an autonomous system typically needs to argue three things about its AI component. The framework supports all three with runtime evidence:

1. "The system does not proceed when its competence is insufficient." — Supported by the confidence gate producing a logged abstention event every time C falls below C_min. The log records the field, the confidence value at the time of abstention, and the sensor context. Safety reviewers can query this log to verify the gate is functioning and measure its activation rate over the operational period.
2. "Errors are detected and the system does not repeat them." — Supported by the contradiction detection and assertions store pipeline. When an error is detected, a structured correction is stored with a timestamp and confidence level. The correction is injected into future sessions immediately and feeds the calibration pipeline for weight-level correction. The log records both the detection event and the resolution event, providing evidence that the system's correction mechanism is functioning.
3. "Updates to the AI component are validated before deployment and do not regress safety properties." — Supported by the blue-green deployment protocol. Every specialist update has a logged trigger (which utility deviation, over which statistical window), a logged promotion trajectory (traffic split over time), a logged benchmark gate result, and a clear rollback path. For safety-critical fields, the T ≥ 246 interaction window ensures the update has demonstrated sustained improvement before promotion. This is directly auditable by a safety reviewer.

7. Standards mapping: ISO 10218:2025, IEC 61508, ISO 13482

The 2025 revision of ISO 10218 (the first major update since 2011) makes functional safety requirements explicit rather than implied, and integrates the former ISO/TS 15066 for collaborative robot applications. The key shift: instead of a blanket Performance Level d (PLd) requirement for all safety functions, ISO 10218:2025 requires application-specific PL determination based on actual risk parameters. This creates a direct mapping opportunity to the framework's field-specific C_min values.

This mapping is not a compliance claim — it identifies where the framework's properties align with standard requirements. Full compliance requires a complete safety case with appropriate evidence, risk assessments, and validation testing. The framework provides the runtime evidence layer, not the complete certification package.

8. Edge hardware: Jetson-class specialists and pipeline parallelism

For autonomous systems deployed on physical platforms, the hardware argument is not about cost — it is about physical feasibility. A monolithic frontier model consuming 700W cannot run on a robot, a drone, or a mobile inspection platform. The Micro-Expert model running on Jetson-class hardware is the only path to AI-quality reasoning on battery-constrained edge platforms.

Hardware specs from NVIDIA (2025). See §10.9.6 of the full whitepaper for the full edge deployment analysis and worked power budget examples.

Pipeline parallelism: specialists run concurrently, not sequentially

In the Micro-Expert deployment on a physical platform, specialists do not run sequentially — they run in a pipeline. This is the same architecture that production autonomy stacks (Tesla FSD, Waymo) use for their neural network pipelines. The Micro-Expert model applies it at the model-graph level:

Sequential assumption (naive):
    Perception  →  Planning  →  Safety constraints  →  Decision
    [50ms]          [40ms]           [15ms]              [5ms]   Total: ~110ms

Actual pipeline architecture (parallel):
    Perception        [50ms] ─────────────────────────────┐
    Planning      ←───────────── [40ms from perception] ──┤→ Arbiter → Output
    Safety/Rules      [15ms running in parallel] ──────────┘
    Total: ~55ms + arbitration overhead ≈ 60–65ms

For a cobot running a tight control loop (typically 1kHz for low-level control, 10–100Hz for the AI decision layer), a 65ms AI decision latency — running on embedded Jetson hardware well within the platform's power envelope — is feasible where a 700W datacenter GPU is not.

9. Independent updateability and narrower certification scope

Every update to an autonomous system's AI component is a certification event in safety-critical domains. If the entire AI is a monolithic model, every update — even updating a traffic rule for a new city, adjusting a force threshold for a new material, improving object detection for a new object class — forces revalidation of the entire system. This is the primary reason AI update cadences in certified autonomous systems are so slow.

The principle that enables this is identical to what ASIL decomposition achieves in ISO 26262: safety goals are allocated to components, and component-level validation can satisfy the overall safety case if the decomposition is sound and the interfaces between components are stable. The specialist architecture is not a replacement for ASIL decomposition — it is an AI architecture that is compatible with that reasoning pattern in a way that a monolithic model is not.

The blue-green deployment trigger for safety-critical autonomous systems

Trigger a deployment cycle when ALL of:
    |U_current − U_baseline| > δ(field)     [significant deviation]
    deviation sustained for ≥ T interactions  [not transient noise]
    held-out scenario library available        [can evaluate candidate]

For safety-critical fields (surgery, aviation, high-stakes robotics):
    δ ≈ 0.005–0.010   T ≥ 246 interactions

T = 246 comes from a power analysis on observed utility variance (σ ≈ 0.04)
with α = 0.05. High-stakes fields demand a large confirmation window
before any traffic shifts.

For a fleet of 50 cobots in a manufacturing plant each running 8 hours/day, 246 interactions per specialist can accumulate in hours to days — meaning corrections to the safety constraint specialist can be validated and deployed on the order of a workweek, not a quarter. That is the practical value of the combination of statistical rigor (T derived from power analysis) and independent specialist deployment (T applies only to the affected specialist, not the whole system).

10. Fleet-level correction propagation

A fleet of 100 AMRs in a warehouse, 50 cobots in a factory, or 30 inspection drones across multiple sites constitutes a collective learning resource that no single unit exploits in a static AI architecture. An edge case encountered by one unit — a new pallet configuration that causes a perception failure, a cobot operating mode the planning specialist had not seen, a tunnel with unusual acoustic reflections — takes months to propagate a correction across the fleet under a traditional retrain-and-redeploy cycle.

The framework compresses this cycle for scoped corrections through the assertions store and DPO calibration pipeline:

11. What the framework does not replace

This section is included because the industrial robotics and safety-case audience will ask this question directly, and the honest answer matters.

• Hardware interlocks and PLC safety layers. Emergency stops, safety-rated limit switches, hard-wired safety relays, and PLC-based safety logic (e.g., PILZ, Siemens SIMATIC Safety) operate independently of the AI layer. The framework's confidence gate triggers a safe-state request to the safety layer — the physical enforcement of that safe state is done by the hardware and PLC infrastructure, not by the AI.
• Low-level control safety. Servo drive safety functions (safe torque off, safe speed monitoring, safe direction) defined under IEC 61800-5-2 are implemented in certified drive firmware. The framework operates at the decision level, not the servo control level.
• Risk assessment and FMEA/STPA. The framework provides runtime evidence; it does not perform the initial hazard analysis. The safety engineer's STPA or FMEA defines what the hazards are and what the required safe behaviors should be — the framework then implements and logs those behaviors.
• Physical safety design. Guards, barriers, force/pressure limits on collaborative applications (as defined in ISO 10218:2025 biomechanical limits), and separation monitoring systems are hardware and sensor design decisions. The framework's collision avoidance confidence gate is a complement to, not a substitute for, physical protective measures.
• Full certification. The framework produces evidence that feeds a safety case. It does not produce the safety case itself, and it does not perform type approval, third-party certification, or regulatory filing. Those processes require human safety engineers and accredited certification bodies.

The intended architecture is layered: hardware safety (interlocks, drives, physical barriers) → PLC safety logic → framework decision layer (confidence gate, abstention, correction loop) → operator / remote assist escalation. The framework is the AI decision layer in that stack, with well-defined interfaces to the layers above and below it.

12. MVP shape: shadow mode before any live control

The right entry point for any autonomous system team is shadow mode — the framework runs alongside the production system, logs utility values and contradiction events, and builds an evidence base without influencing any physical actuator. This is the same validation logic as AV shadow mode testing, applied to the broader autonomous systems context.

Configure this domain today

AUA v1.0 handles routing, utility scoring, abstention policy, and audit logging. Safety-critical preset with c_min=0.95 halts automated action when confidence is insufficient. Physical actuation integration is via the AUA REST API.

1. Install

pip install adaptive-utility-agent

2. Scaffold for this domain

aua init my-autonomous-systems-agent --preset medical-safe --tier macbook
cd my-autonomous-systems-agent
aua doctor

3. Key config for this domain

# aua_config.yaml
specialists:
  - name: aviation
    model: qwen-coder-7b-awq
    port: 11434
    field: aviation

safety:
  abstention_enabled: true
  require_arbiter_for_high_risk: true
  min_confidence_for_direct_answer: 0.95

security:
  encryption: {enabled: true, key_secret: AUA_ENCRYPTION_KEY}

audit:
  enabled: true
  hash_chain: true

Generate your encryption key: python3 -c "import os; print(os.urandom(32).hex())" or openssl rand -hex 32 — 64-char hex string. See Tutorial §12.4 for key management.

4. Start and query

aua serve

curl -X POST http://localhost:8000/query \
  -H "Authorization: Bearer $AUA_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"prompt": "...", "session_id": "demo"}'

5. What AUA handles vs. what you bring

Full instructions: AUA Tutorial · Framework v1.0 · GitHub ↗