SkillPedia Skills Encyclopedia
中文 EN
Today's updates 2026-11-18 UTC

No new commits yet today

Open the update feed

Skills 2026-10-09 22:46 UTC

agent-toolkit-for-aws

Committed by herosjourney. Update Category: New Features & Capability Expansion. Commit details: feat: add read-only Anthropic usage/cost discovery (gcp, azure, llm-to-bedrock) (#425). This update introduces new capabilities and expands supported use cases.

herosjourney 2cb0fa1 aws/agent-toolkit-for-aws

CommitRaw commit message

feat: add read-only Anthropic usage/cost discovery (gcp, azure, llm-to-bedrock) (#425)

* feat(gcp-to-aws): add Anthropic usage discovery (Admin API)

* Mention usage-API cost discovery (incl. Anthropic) in plugin.json

Surfaces the OpenAI/OpenRouter/Anthropic read-only usage-API discovery
capability in the plugin description and adds usage-api/cost-discovery
keywords, so the new Anthropic discovery fragment is discoverable from
the marketplace listing, not just from skill-internal docs.

* feat(azure-to-aws): add OpenAI/OpenRouter/Anthropic usage-API discovery

* feat(llm-to-bedrock): support azure-to-aws as Assess sibling + reuse usage profiles

* docs(aws-startup-advisor): README llm-to-bedrock accepts azure-to-aws too

* fix: resolve markdownlint failures (list style, emphasis, inline-HTML placeholder)

- discover-anthropic-api.md (gcp-to-aws + azure-to-aws): unordered list
  bullet style '+' -> '-' to match repo convention (MD004)
- llm-to-bedrock/SKILL.md: emphasis style '_underscore_' -> '*asterisk*'
  (MD049) and wrapped the <date> prose placeholder in a code span so
  markdownlint stops parsing it as inline HTML (MD033)

No behavior change; verified with 'npx markdownlint-cli2' (0 errors on the
3 touched files), llm-to-bedrock's own pytest suite (251/251), and
tests/test_model_id_lint.py (11/11).

* fix: stop dropping Anthropic cache tokens from usage/cost volume

Addresses the blocking review finding on this PR: the Admin usage report
has no `input_tokens` field. It returns `uncached_input_tokens`,
`cache_read_input_tokens`, and `cache_creation.ephemeral_5m_input_tokens`
+ `ephemeral_1h_input_tokens` as mutually exclusive pools —
Anthropic's own contract is `total prompt tokens = uncached_input_tokens +
cache_read_input_tokens + cache_creation_input_tokens`. This is the
opposite shape of OpenRouter's reasoning_tokens (already included inside
completion_tokens), which the prior wording wrongly used as the analogy.

- discover-anthropic-api.md (gcp-to-aws + azure-to-aws): Step 3 now reads
  uncached_input_tokens -> input_tokens, cache_read_input_tokens ->
  cache_read_tokens, and sums the two cache_creation sub-fields into
  cache_creation_tokens, instead of assuming input_tokens already exists
  on the response. Sort-by-volume instruction now includes both cache
  fields instead of just input_tokens + output_tokens.
- estimate-ai.md (gcp-to-aws + azure-to-aws): Prerequisites and Part 2 now
  instruct summing all four fields (input_tokens + cache_read_tokens +
  cache_creation_tokens + output_tokens) for total volume, instead of
  excluding the two cache fields as a supposed sub-accounting of input.
- clarify-ai.md / clarify-ai-only.md (both skills): Q18/Q19 auto-resolve
  and auto-detect logic updated to include cache tokens in the volume
  sum used for tier classification and top-model ranking.
- llm-to-bedrock/SKILL.md: Step 1.5's cross-skill profile-reuse sum
  updated to include cache tokens, with a pointer to the sibling skills'
  estimate-ai.md for the full field contract.

Net effect: a prompt-cached Claude workload (where cache reads are
typically the majority of input tokens) no longer has most of its real
token volume silently dropped from the Bedrock cost/sizing estimate.

Verified: Anthropic's own cookbook example (platform.claude.com/cookbook/
observability-usage-cost-api) confirms the real response field names
matching this fix. markdownlint-cli2: 0 errors on all 9 touched files.
llm-to-bedrock pytest: 251/251. test_model_id_lint.py: 11/11.
tools/validate.py: all validations passed.

* fix: trim llm-to-bedrock SKILL.md frontmatter description under 1024 chars

CI's lint:spec task failed: the description was 1072 chars, over the
repo's 1024-char limit. Trimmed redundant parenthetical examples and
restated the gcp/azure-preference clause more concisely with no
meaning lost. Verified: tools/validate_spec.py no longer reports the
length error (its remaining 3 errors are pre-existing .venv symlinks,
which are gitignored and not part of this diff). markdownlint-cli2: 0
errors. llm-to-bedrock pytest: 251/251 passed.

* fix(discover): correct usage-API request params and Azure capture ordering

- Azure discover-openai-api.md: usage endpoints (completions/embeddings/
  images/audio_speeches/audio_transcriptions) now request limit=31, not
  limit=180 (OpenAI's usage endpoints cap bucket_width=1d&limit at 31; the
  Costs endpoint's own max of 180 was copy-pasted onto usage rows).
- Both discover-anthropic-api.md copies: cost_report capture now loops once
  per selected workspace so costs_by_description[] carries workspace_id per
  row (previously only group_by=description, losing per-workspace
  attribution on multi-workspace selections). Consent block now discloses
  the Step 2b probe's org-wide per-workspace read.
- Azure discover.md: usage-API consent + capture steps now run after
  fragments/discover-assemble.md instead of as a pre-dispatch action, so
  their ai-workload-profile.json-based triggers see a profile that actually
  exists on a from-scratch run.
- Add tests/test_usage_api_request_params.py and
  tests/test_azure_discover_ordering.py as regression guards.

Addresses PR #425 Cursor review findings 1-3.

* fix: add cost_status/cost_provenance for cost-completeness tracking (PR425 findings 4,5)

* fix(discover): delete admin-key file on every terminal exit (PR425 finding 6)

Route Anthropic (both copies) and Azure OpenAI usage-discovery terminal
exits through key-file deletion, with an explicit KEY_INVALID_MID_RUN
retry_pending carve-out. Adds retry_pending to each manifest.json schema
and test_credential_lifecycle.py enforcing the contract.

* fix: Discover resume/schema consistency + GCP design guide selection

- Finding 7: gate Anthropic resume/postcondition checks on row status
  (captures[].status == ok), not bare manifest.json existence, in both
  gcp-to-aws and azure-to-aws discover.md, mirroring the OpenRouter sibling.
- Finding 8: drop the sdk_method qualifier from Anthropic's Step 4.4b
  sentinel-workload absence check (both copies), and add the equivalent
  workloads[] append to Azure's discover-openai-api.md Step 4. Widen
  detected_via to include usage_api in azure's schema-discover-ai.md.
- Finding 9: GCP design-ai.md's 'both' branch now selects guides by actual
  provider presence instead of hardcoding Gemini+OpenAI, mirroring Azure's
  existing conditional pattern.
- Add tests/test_resume_and_sentinel_consistency.py and
  tests/test_gcp_design_guide_selection.py.

* fix(llm-to-bedrock): defer usage-baseline write, fingerprint it, fix tiebreak (PR425 findings 10,11,15)

- Defer usage-baseline.json's write to A3, after $BEDROCK_RUN_DIR exists
- Pass a Usage baseline path line to C5/C6; report-generator reads it instead of assuming $BEDROCK_RUN_DIR
- Add usage_baseline_sha256 to current-context.json schema + MISMATCH table
- Replace unconditional lexicographic MMDD-HHMM tiebreak with last_updated-primary, year-aware logic; surface genuine ties to the user
- Add tests/test_llm_to_bedrock_run_state.py

* fix: report-generator cost-pricing + estimate/clarify volume resolution (PR425 findings 12-14)

Finding 12 (highest priority, numerically verified): cost_compare.py now
prices both source_cost and bedrock_cost for a baseline-covered model pair
from usage-baseline.json's real monthly token volume, never mixing it with
the golden-dataset sample volume on the other side of the same pair.
Uncovered pairs still fall back to the sample unchanged.

Finding 13: ported GCP's correct partial-provider-volume exception text
(unresolved + scenario rows) into Azure's estimate-ai.md Part 2.

Finding 14: Clarify's Q3 primary branch (azure-to-aws clarify-ai-only.md)
now checks partial_window/cost_status via current_costs provenance before
auto-resolving monthly AI spend; Discover's Step 4 merge in both Anthropic
copies and Azure's OpenAI copy now stamps those fields onto
current_costs/breakdown[] entries. GCP's own Clarify has no equivalent
auto-resolve shortcut today, so it has no equivalent bug (documented
skip-with-reason).

Verification: pytest tests/ -q -> 87 passed, zero regressions. New tests
exec the real embedded cost_compare.py script via subprocess against
constructed fixtures and assert on real numeric stdout.

* fix(aws-startup-advisor): scope OpenRouter workspace filtering and Azure AI-only phase gates

Fixes PR #425 findings 16/17: Design/Estimate phases now route-conditionally gate on
azure-resource-inventory.json vs ai-workload-profile.json (mirroring Clarify's existing
AI-only fork) instead of unconditionally requiring infra artifacts, and the assemblers
no longer fabricate empty infra files on AI-only runs. OpenRouter discovery (Azure +
GCP mirrors) now applies a real workspace_id filter at capture time instead of the
previously dead group_by=workspace parameter, with lifetime credits explicitly
relabeled account-wide when a workspace scope is set.

Also includes two small pre-existing markdownlint fixes unrelated to findings 16/17,
needed to get 'mise run build' green: a missing blank line before a list in
clarify-ai-only.md (MD032) and two headings in discover.md that skipped a level
(MD001, #### -> ###, matching the design doc's intended h3 nesting).

* fix: PR 425 review 5475407077 findings A, B, C

Finding A (blocking) - cost_report workspace filter: both
discover-anthropic-api.md copies (gcp-to-aws, azure-to-aws) replaced the
per-workspace cost_report loop with a single call using
group_by[]=workspace_id&group_by[]=description, since the endpoint has no
workspace_ids[] parameter at all. Rows are filtered client-side to the
selected workspace_ids after the one call returns. Updated the capture
table, Step 3 parsing prose, and the Error Handling row accordingly; the
Step 2b probe call is unchanged. Renamed
test_anthropic_cost_report_captures_once_per_workspace to
test_anthropic_cost_report_uses_single_call_with_compound_group_by in
test_usage_api_request_params.py to assert the new single-call contract and
that the cost_report table row no longer requests workspace_ids[].

Finding B (should-fix) - Priority Tier dollars excluded from cost_report:
both copies now add service_tier to Row 1's usage group_by so Priority rows
are observable, and Step 3 scans for service_tier in {priority,
priority_on_demand}. Decision: reused the existing cost_status enum's
partial_window value for this case rather than adding a 4th enum value,
since partial_window's downstream semantics ("real but incomplete total, not
a trustworthy monthly baseline") already match and no changes are needed in
estimate-ai.md or llm-to-bedrock/SKILL.md Step 1.5, which already treat
partial_window that way. monthly_cost_usd stays populated from the
standard/batch rows that did return (never nulled), and a capture_warnings
line is added. Added two new tests to test_cost_completeness.py covering
the service_tier detection and the complete-branch's new precondition.

Finding C (should-fix) - BYOK de-dupe for the only cost_provenance value this
flow writes: llm-to-bedrock/SKILL.md Step 1.5's unknown branch now matches
OpenRouter's provider/model-name usage_by_model[] rows against the direct
profile's bare model names (match = direct name equals the substring after
OpenRouter's last /) and excludes matching rows' dollars AND tokens from the
sum, keeping OpenRouter-only models. The byok_passthrough/mixed branch is
untouched.

Verification: uv run --with pytest python -m pytest tests/ -> 89 passed (87
baseline + 2 new for Finding B) from plugins/aws-startup-advisor. mise run
build: lint:manifests, lint:spec, lint:skill-parity, security scan all pass;
lint:md reports 70 pre-existing issues in 15 unrelated files (none touched
by this change). git diff --check: clean. git merge-tree --write-tree
upstream/main HEAD: no conflicts.

---------

Co-authored-by: Logan Kleier <lkleier@amazon.com>
Co-authored-by: Gen Li <i.ligen@gmail.com>