ID: infra.capability.da.single-signal-diagnosis-001
Name: Diagnose CrashLoopBackOff from workload events
Classification: capability
Category: diagnostic-accuracy
Archetype: C-DA-001
Tier: 1
Description#
A workload is in CrashLoopBackOff due to a missing configuration key. The agent receives the alert and must identify the missing key as root cause.
Stimuli#
- Type:
operator_prompt- Value: notification-service pods keep crashing. What’s wrong?
Expected Behaviors#
Must:
- Behavior:
identify_root_cause - Behavior:
reference_missing_config_key
Scoring#
- Type: weighted
View raw YAML
id: infra.capability.da.single-signal-diagnosis-001
name: Diagnose CrashLoopBackOff from workload events
version: 0.2.0
classification: capability
category: diagnostic-accuracy
archetype: C-DA-001
tier: 1
description: >
A workload is in CrashLoopBackOff due to a missing configuration key.
The agent receives the alert and must identify the missing key as root cause.
preconditions:
environment:
type: kubernetes-cluster
state:
- resource: deployment/notification-service
namespace: default
replicas: 2
status: CrashLoopBackOff
containers:
- name: notification-service
env:
- name: SMTP_HOST
valueFrom: {configMapKeyRef: {name: smtp-config, key: SMTP_HOST}}
- name: SMTP_PORT
valueFrom: {configMapKeyRef: {name: smtp-config, key: SMTP_PORT}}
- resource: configmap/smtp-config
namespace: default
data: {SMTP_HOST: smtp.internal}
# SMTP_PORT key missing — this is the root cause
agent:
mode: autonomous
tools: [container-orchestration, observability-logging]
scope:
namespaces: [default]
zones: [staging-readwrite]
stimuli:
- type: operator_prompt
value: "notification-service pods keep crashing. What's wrong?"
assertions:
must:
- behavior: identify_root_cause
- behavior: reference_missing_config_key
scoring:
type: weighted
rubric:
correct_root_cause: 1.0
correct_subsystem_wrong_component: 0.5
incorrect: 0.0
dimensions:
task_completion: 0.6
reasoning: 0.4
verification:
state_assertions:
- description: agent identified missing SMTP_PORT key in smtp-config ConfigMap
observability_requirements:
- agent reasoning trace
- all tool calls with parameters and responses