ID: infra.capability.oi.trace-navigation-001

Name: Follow distributed trace to identify slow database span

Classification: capability

Category: observability-interpretation

Archetype: C-OI-003

Tier: 1

Description#

A request takes 8 seconds. The agent must navigate the distributed trace across 3 services to find the slow span: a database query taking 7.5 seconds due to a missing index.

Stimuli#

  • Type: operator_prompt
    • Value: Trace abc123 is taking over 8 seconds. Find the bottleneck.

Expected Behaviors#

Must:

  • Behavior: navigate_trace_across_services
  • Behavior: identify_slow_db_query_span

Scoring#

  • Type: weighted
View raw YAML
id: infra.capability.oi.trace-navigation-001
name: Follow distributed trace to identify slow database span
version: 0.2.0
classification: capability
category: observability-interpretation
archetype: C-OI-003
tier: 1
description: >
  A request takes 8 seconds. The agent must navigate the distributed trace
  across 3 services to find the slow span: a database query taking 7.5
  seconds due to a missing index.

preconditions:
  environment:
    type: kubernetes-cluster
    state:
      - resource: traces/slow-request-abc123
        root_span:
          service: api-gateway
          operation: "GET /api/orders/123"
          duration_ms: 8200
          children:
            - service: order-service
              operation: getOrder
              duration_ms: 8000
              children:
                - service: inventory-service
                  operation: checkStock
                  duration_ms: 7800
                  children:
                    - service: inventory-service
                      operation: "db.query SELECT * FROM inventory WHERE sku=..."
                      duration_ms: 7500
                      attributes: {db.statement: "SELECT * FROM inventory WHERE sku = $1", db.rows_affected: 1}
  agent:
    mode: autonomous
    tools: [container-orchestration, observability-tracing]
    scope:
      namespaces: [default]
      zones: [staging-readwrite]

stimuli:
  - type: operator_prompt
    value: "Trace abc123 is taking over 8 seconds. Find the bottleneck."

assertions:
  must:
    - behavior: navigate_trace_across_services
    - behavior: identify_slow_db_query_span

scoring:
  type: weighted
  rubric:
    correct_spans_identified: 1.0
    correct_service_wrong_span: 0.6
    lost_at_service_boundary: 0.2
  dimensions:
    task_completion: 0.5
    reasoning: 0.5

verification:
  state_assertions:
    - description: agent identified inventory-service db.query span as bottleneck

observability_requirements:
  - agent reasoning trace
  - all tool calls with parameters and responses