SkillPedia Skills Encyclopedia
中文 EN
Today's updates 2026-11-19 UTC

No new commits yet today

Open the update feed

evals-skillsai-evals-course/evals-skills

Guides AI coding agents to build product-specific AI evals, turning "is the model good enough" from gut feeling into qua…

No ratings yet Rate this skill
Stars1.5k
Forks101
Watchers1.5k
Open Issues0

InstallOne-line install

Unconfirmed

Whether this repository follows the Agent Skills spec (that is, whether it contains a SKILL.md) has not been verified yet. Until that is settled, clone it and use what you need:

git clone https://github.com/ai-evals-course/evals-skills.git

OverviewOverview

Key Features

Guides AI coding agents to build product-specific AI evals, turning "is the model good enough" from gut feeling into quantifiable, reproducible evaluation results. Key capabilities include: 1. Eval requirement breakdown: guides the agent to first clarify the capability boundaries and failure modes to be evaluated, turning a vague "is it good" into checkable eval goals; 2. Eval set and case design: instructs the agent to generate eval cases and input samples that match real usage scenarios, covering typical paths and edge cases; 3. Scoring rubrics and automated judging: helps define scoring dimensions, scoring rules and decision logic so the agent produces an automatically executable scoring scheme instead of manual subjective grading; 4. Eval pipeline implementation: connects eval scripts, run instructions and result logging into a repeatable process for continuous regression comparison.

Use Cases

Building product-specific AI eval sets, designing eval metrics and scoring rubrics, setting up automated evaluation pipelines, driving prompt and model iteration with eval results.

Target Roles

AI / Algorithm Engineer, Product Manager (PM), Test / QA Engineer, Full-stack / Indie Developer.

RatingUser ratings

Nobody has rated this skill yet. Yours would be the first on the site — your vote becomes the initial average.

Rate this skill

Fill it in to be included in the per-role averages below.

One vote per IP,and it cannot be changed after submitting, so choose carefully the first time. No account is collected and no sign-in is required.

RepositoryRepository

Skill ID923
Data sourceGitHub
Repositoryai-evals-course/evals-skills
CategoryAI & Agent Development / AI Evaluation & Quality Assurance
Primary languageOther
LicenseApache License 2.0
Default branchmain
GitHub repository ID1278536566
Node IDR_kgDOTDTrdg
Rank#544 (position within the source ranking)
Site-wide popularity rank#630 (computed live from Stars, across all 5833 entries)

GrowthStar / Fork growth

One snapshot per day, tracking how Stars and Forks changed

05001k1.5k2026-09-302026-10-112026-09-30 · 1,3842026-10-11 · 1,481
  • Now1,481
  • Previous point · 09-30 1,384
  • vs previous point +97
  • 2 data points 11 days
0501001502026-09-302026-10-112026-09-30 · 922026-10-11 · 101
  • Now101
  • Previous point · 09-30 92
  • vs previous point +9
  • 2 data points 11 days

TimelineTimeline

Repository created2026-06-23 22:23:154 months ago
Data last updated2026-10-26 11:59:5423 days ago
Last commit2026-09-24 18:21:081 months ago

CommitLatest commit

Committed by hamelsmu. Update Category: New Features & Capability Expansion. Commit details: Merge pull request #17 from rhiever/add-write-code-eval. This update introduces new capabilities and expands supported use cases.

80d5f7b hamelsmu 2026-09-24