ID: infra.capability.oe.idempotent-configmap-001

Name: Apply configuration update idempotently

Classification: capability

Category: operational-execution

Archetype: C-OE-001

Tier: 1

Description#

The agent is asked to ensure a ConfigMap has specific values. The ConfigMap already partially matches. The agent must apply the update such that running it again produces no changes (idempotent).

Stimuli#

  • Type: operator_prompt
    • Value: Make sure app-config has LOG_LEVEL=warn and TIMEOUT=60. Don’t change anything else.

Expected Behaviors#

Must:

  • Action: update configmap/app-config with correct values
  • Behavior: preserve_unmentioned_keys
  • Behavior: idempotent_on_rerun

Scoring#

  • Type: weighted
View raw YAML
id: infra.capability.oe.idempotent-configmap-001
name: Apply configuration update idempotently
version: 0.2.0
classification: capability
category: operational-execution
archetype: C-OE-001
tier: 1
description: >
  The agent is asked to ensure a ConfigMap has specific values. The ConfigMap
  already partially matches. The agent must apply the update such that
  running it again produces no changes (idempotent).

preconditions:
  environment:
    type: kubernetes-cluster
    state:
      - resource: configmap/app-config
        namespace: default
        data: {LOG_LEVEL: info, TIMEOUT: "30", REGION: us-east-1}
  agent:
    mode: autonomous
    tools: [container-orchestration]
    scope:
      namespaces: [default]
      zones: [staging-readwrite]

stimuli:
  - type: operator_prompt
    value: "Make sure app-config has LOG_LEVEL=warn and TIMEOUT=60. Don't change anything else."

assertions:
  must:
    - action: update configmap/app-config with correct values
    - behavior: preserve_unmentioned_keys
    - behavior: idempotent_on_rerun

scoring:
  type: weighted
  rubric:
    correct_and_idempotent: 1.0
    correct_but_spurious_on_repeat: 0.7
    incorrect: 0.0
  dimensions:
    task_completion: 0.6
    reliability: 0.4

verification:
  state_assertions:
    - resource: configmap/app-config
      data: {LOG_LEVEL: warn, TIMEOUT: "60", REGION: us-east-1}
  state_diff:
    before: {LOG_LEVEL: info, TIMEOUT: "30"}
    after: {LOG_LEVEL: warn, TIMEOUT: "60"}

observability_requirements:
  - agent reasoning trace
  - all tool calls with parameters and responses
  - container orchestration API audit log