56 results available

phoenix-evals

github

·

Last changed

Hardened

This skill has been hardened by Chainguard

Last changed

This version passed the hardening gate. Hardening applied 1 targeted change and preserved all other text. Classified findings before hardening: 2. Classified findings closed: 0. Classified findings remaining: 2. Unclassified scanner findings remaining: 0. Review findings closed: 0. Review findings remaining: 11.

Open before hardening

2

Low

Open after hardening

2

Low

Expand	File	Group	Subtype	Severity	Status
	references/setup-python.md	Supply Chain & Install-Time Trust	Install-time download and execute	Low	Open
	references/setup-python.md	Malicious Code & Command Execution	—	Low	Open

2 findings

SKILL.md

Phoenix Evals

Build evaluators for AI/LLM applications. Code first, LLM for nuance, validate against humans.

Quick Reference

Task	Files
Setup	setup-python, setup-typescript
Decide what to evaluate	evaluators-overview
Choose a judge model	fundamentals-model-selection
Use pre-built evaluators	evaluators-pre-built
Build code evaluator	evaluators-code-python, evaluators-code-typescript
Build LLM evaluator	evaluators-llm-python, evaluators-llm-typescript, evaluators-custom-templates
Batch evaluate DataFrame	evaluate-dataframe-python
Understand experiments	experiments-overview
Run experiment	experiments-running-python, experiments-running-typescript
Create dataset	experiments-datasets-python, experiments-datasets-typescript
Generate synthetic data	experiments-synthetic-python, experiments-synthetic-typescript
Validate evaluator accuracy	validation, validation-evaluators-python, validation-evaluators-typescript
Sample traces for review	observe-sampling-python, observe-sampling-typescript
Analyze errors	error-analysis, error-analysis-multi-turn, axial-coding
RAG evals	evaluators-rag
Avoid common mistakes	common-mistakes-python, fundamentals-anti-patterns
Production	production-overview, production-guardrails, production-continuous

Workflows

Starting Fresh: observe-tracing-setup → error-analysis → axial-coding → evaluators-overview

Building Evaluator: fundamentals → common-mistakes-python → evaluators-{code|llm}-{python|typescript} → validation-evaluators-{python|typescript}

RAG Systems: evaluators-rag → evaluators-code-* (retrieval) → evaluators-llm-* (faithfulness)

Production: production-overview → production-guardrails → production-continuous

Reference Categories

Prefix	Description
`fundamentals-*`	Types, scores, anti-patterns
`observe-*`	Tracing, sampling
`error-analysis-*`	Finding failures
`evaluators-*`	Code, LLM, RAG evaluators
`experiments-*`	Datasets, running experiments
`validation-*`	Validating evaluator accuracy against human labels
`production-*`	CI/CD, monitoring

Key Principles

Principle	Action
Error analysis first	Can't automate what you haven't observed
Custom > generic	Build from your failures
Code first	Deterministic before LLM
Validate judges	>80% TPR/TNR
Binary > Likert	Pass/fail, not 1-5

Install

Install with chainctl


chainctl skill install public/github/phoenix-evals:latest

Get started with chainctl

View source

Harden your own skills

Bring Chainguard's hardening pipeline to your first-party skills — custom rulesets, automated remediation, full audit trail.

Provenance

Upstream sourcehttps://github.com/github/awesome-copilot

Upstream commitc73936e05eea186dd9970db9e97c104d7e77a644

OCI reference

skills.cgr.dev/public/github.com/github/skills/phoenix-evals:latest

License

MIT

Files

HARDENING.md

LICENSE

SKILL.md

references/axial-coding.md

references/common-mistakes-python.md

references/error-analysis-multi-turn.md

references/error-analysis.md

references/evaluate-dataframe-python.md

references/evaluators-code-python.md

references/evaluators-code-typescript.md

references/evaluators-custom-templates.md

references/evaluators-llm-python.md

references/evaluators-llm-typescript.md

references/evaluators-overview.md

references/evaluators-pre-built.md

references/evaluators-rag.md

references/experiments-datasets-python.md

references/experiments-datasets-typescript.md

references/experiments-overview.md

references/experiments-running-python.md

references/experiments-running-typescript.md

references/experiments-synthetic-python.md

references/experiments-synthetic-typescript.md

references/fundamentals-anti-patterns.md

references/fundamentals-model-selection.md

references/fundamentals.md

references/observe-sampling-python.md

references/observe-sampling-typescript.md

references/observe-tracing-setup.md

references/production-continuous.md

references/production-guardrails.md

references/production-overview.md

references/setup-python.md

references/setup-typescript.md

references/validation-evaluators-python.md

references/validation-evaluators-typescript.md

references/validation.md

The trusted source for open source

Talk to an expert

Privacy

Terms

© 2026 Chainguard, Inc. All Rights Reserved.
Chainguard® and the Chainguard logo are registered trademarks of Chainguard, Inc. in the United States and/or other countries.
The other respective trademarks mentioned on this page are owned by the respective companies and use of them does not imply any affiliation or endorsement.