SkillPedia Skills Encyclopedia
中文 EN
Today's updates 2026-11-18 UTC

No new commits yet today

Open the update feed

OpenJudgeagentscope-ai/OpenJudge

Provides reusable grader capabilities for LLM and agent outputs, turning "is it good" into computable scores and rationa…

No ratings yet Rate this skill
Stars851
Forks73
Watchers851
Open Issues14

InstallOne-line install

Unconfirmed

Whether this repository follows the Agent Skills spec (that is, whether it contains a SKILL.md) has not been verified yet. Until that is settled, clone it and use what you need:

git clone https://github.com/agentscope-ai/OpenJudge.git

OverviewOverview

Key Features

Provides reusable grader capabilities for LLM and agent outputs, turning "is it good" into computable scores and rationales. Key capabilities include: 1. Multi-dimensional scoring: scores model responses item by item across correctness, relevance and safety, with a stated rationale for each judgment, replacing manual line-by-line review; 2. Agent trajectory evaluation: performs process-level judging of multi-step agent execution (tool calls, intermediate decisions) to locate the failing step rather than only checking the final answer; 3. Reward signal generation: outputs scores/preference data usable for reward model training and RLHF preference alignment, feeding evaluation results directly into the training loop; 4. Skill-based packaging: organizes judging rules and prompts as skills so grading criteria can be reused and swapped across evaluation tasks.

Use Cases

pre-release response quality spot checks, agent task success rate and process compliance evaluation, reward model training set construction, RLHF preference data annotation and validation, automated grading and regression comparison of evaluation sets.

Target Roles

AI / Algorithm Engineer, Data Analyst / Data Engineer, Test / QA Engineer.

TopicsTopics

RatingUser ratings

Nobody has rated this skill yet. Yours would be the first on the site — your vote becomes the initial average.

Rate this skill

Fill it in to be included in the per-role averages below.

One vote per IP,and it cannot be changed after submitting, so choose carefully the first time. No account is collected and no sign-in is required.

RepositoryRepository

Skill ID1104
Data sourceGitHub
Repositoryagentscope-ai/OpenJudge
CategoryAI & Agent Skills / Model Evaluation & Alignment
Primary languagePython
LicenseApache License 2.0
Default branchmain
GitHub repository ID1015852565
Node IDR_kgDOPIyuFQ
Rank#733 (position within the source ranking)
Site-wide popularity rank#977 (computed live from Stars, across all 5833 entries)

GrowthStar / Fork growth

Still collecting First record taken on 2026-09-23. Two days of data are needed before a trend line can be drawn.

TimelineTimeline

Repository created2025-07-08 06:17:012025-07-08
Data last updated2026-10-26 12:05:1323 days ago
Last commit2026-09-11 07:11:092 months ago

CommitLatest commit

Committed by chr6192. Update Category: New Features & Capability Expansion. Commit details: feat(skills): reorganize skills/ into domain suites (#189). This update introduces new capabilities and expands supported use cases.

d1e0642 chr6192 2026-09-11

Next stepsNext steps