evals-skillshamelsmu/evals-skills
A companion eval skill set for the course "AI Evals For Engineers & PMs", turning LLM evaluation from concept into runna…
No ratings yet
Rate this skill
Stars1.7k
Forks172
Watchers1.7k
Open Issues2
InstallOne-line install
UnconfirmedWhether this repository follows the Agent Skills spec (that is, whether it contains a SKILL.md) has not been verified yet. Until that is settled, clone it and use what you need:
git clone https://github.com/hamelsmu/evals-skills.git
OverviewOverview
Key Features
A companion eval skill set for the course "AI Evals For Engineers & PMs", turning LLM evaluation from concept into runnable eval code and process. Key capabilities include: 1. Eval dataset construction: provides methods and templates for building eval datasets, breaking vague "works well" into individually checkable test samples; 2. Grader authoring: offers a practical path for writing graders and eval metrics so model outputs can be scored automatically instead of by subjective human judgment; 3. Eval workflow execution: organizes hands-on eval steps following the course chapters, so engineers and PMs can advance evaluation against one shared standard and reuse the conclusions.
Use Cases
Designing eval plans for LLM applications, building eval datasets, writing automated scoring logic, and comparing quality across iterations and before launch.
Target Roles
AI / Algorithm Engineer, Product Manager (PM), Test / QA Engineer, Data Analyst / Data Engineer.
RatingUser ratings
Nobody has rated this skill yet. Yours would be the first on the site — your vote becomes the initial average.
RepositoryRepository
| Skill ID | 819 |
|---|---|
| Data source | GitHub |
| Repository | hamelsmu/evals-skills |
| Category | AI Engineering & Evaluation / LLM Evaluation & Quality Assurance |
| Primary language | Other |
| License | MIT License |
| Default branch | main |
| GitHub repository ID | 1170210179 |
| Node ID | R_kgDORb_9gw |
| Rank | #505 (position within the source ranking) |
| Site-wide popularity rank | #588 (computed live from Stars, across all 5833 entries) |
GrowthStar / Fork growth
One snapshot per day, tracking how Stars and Forks changed
- Now1,676
- Previous point · 09-25 1,673
- vs previous point +3
- 2 data points 16 days
- Now172
- Previous point · 09-25 171
- vs previous point +1
- 2 data points 16 days
TimelineTimeline
Repository created2026-03-01 21:20:098 months ago
Data last updated2026-10-26 11:59:1923 days ago
Last commit2026-08-16 05:28:253 months ago
CommitLatest commit
Committed by hamelsmu. Update Category: Refactoring & Structural Improvements. Commit details: Point deprecation notice at renamed repo. This update restructures internal code for better maintainability, without behavior changes.