ID: infra.capability.da.single-signal-diagnosis-001

Name: Diagnose CrashLoopBackOff from workload events

Classification: capability

Category: diagnostic-accuracy

Archetype: C-DA-001

Tier: 1

Description#

A workload is in CrashLoopBackOff due to a missing configuration key. The agent receives the alert and must identify the missing key as root cause.

Stimuli#

  • Type: operator_prompt
    • Value: notification-service pods keep crashing. What’s wrong?

Expected Behaviors#

Must:

  • Behavior: identify_root_cause
  • Behavior: reference_missing_config_key

Scoring#

  • Type: weighted
View raw YAML
id: infra.capability.da.single-signal-diagnosis-001
name: Diagnose CrashLoopBackOff from workload events
version: 0.2.0
classification: capability
category: diagnostic-accuracy
archetype: C-DA-001
tier: 1
description: >
  A workload is in CrashLoopBackOff due to a missing configuration key.
  The agent receives the alert and must identify the missing key as root cause.

preconditions:
  environment:
    type: kubernetes-cluster
    state:
      - resource: deployment/notification-service
        namespace: default
        replicas: 2
        status: CrashLoopBackOff
        containers:
          - name: notification-service
            env:
              - name: SMTP_HOST
                valueFrom: {configMapKeyRef: {name: smtp-config, key: SMTP_HOST}}
              - name: SMTP_PORT
                valueFrom: {configMapKeyRef: {name: smtp-config, key: SMTP_PORT}}
      - resource: configmap/smtp-config
        namespace: default
        data: {SMTP_HOST: smtp.internal}
        # SMTP_PORT key missing — this is the root cause
  agent:
    mode: autonomous
    tools: [container-orchestration, observability-logging]
    scope:
      namespaces: [default]
      zones: [staging-readwrite]

stimuli:
  - type: operator_prompt
    value: "notification-service pods keep crashing. What's wrong?"

assertions:
  must:
    - behavior: identify_root_cause
    - behavior: reference_missing_config_key

scoring:
  type: weighted
  rubric:
    correct_root_cause: 1.0
    correct_subsystem_wrong_component: 0.5
    incorrect: 0.0
  dimensions:
    task_completion: 0.6
    reasoning: 0.4

verification:
  state_assertions:
    - description: agent identified missing SMTP_PORT key in smtp-config ConfigMap

observability_requirements:
  - agent reasoning trace
  - all tool calls with parameters and responses