Langsmith Code Eval

langchain-ai/lca-skills/skills/langsmith-code-eval

Multi-file

ai-mlOfficial

Official Provider SkillView repo

Creates code-based evaluators for LangSmith-traced agents. Use when building custom evaluation logic, testing tool usage patterns, or scoring agent outputs programmatically. Triggers on requests to evaluate agents, create evaluators, or run experiments against LangSmith datasets.

Files4 files

EVALUATOR_REFERENCE.md98 lines

Loading editor…

Install

Recommended

One command — your agent picks it up automatically.

Select AI agent

Select an AI agent above to see the install command.

Manual Install

More steps

Download the archive and add the files to your project manually.

Skill details

Versionv1.0.0

AuthorLangChain AI

Categoryai-ml

Skill IDlangchain-ai/lca-skills/skills/langsmith-code-eval

Files4 files

Related skills

Langsmith FetchFetches LangSmith traces for debugging agent behavior. Use when troubleshooting agent issues, reviewing conversation history, or investigating tool calls.Cuml Machine LearningUse for GPU-accelerated machine learning on tabular data using NVIDIA cuML. Triggers when tasks involve classification, regression, clustering, dimensionality reduction, or model training on datasets.Continual LearningNightly refinement of an existing per-repo review-style prompt using this reviewer's own finding outcomes. Read confirmed (resolved-by-commit / thumbs-up) and dismissed (thumbs-down) findings, promote the bug patterns the team actually fixes, demote the false-positive patterns, reconcile against the current prompt, and save the refined version. Use this once outcomes exist; use bootstrap-repo-analysis for a cold-start repo.Deep Agents CoreINVOKE THIS SKILL when building ANY Deep Agents application. Covers create_deep_agent(), harness architecture, SKILL.md format, and configuration options.Deep Agents MemoryINVOKE THIS SKILL when your Deep Agent needs memory, persistence, or filesystem access. Covers StateBackend (ephemeral), StoreBackend (persistent), FilesystemMiddleware, and CompositeBackend for routing.Ecosystem PrimerINVOKE FIRST for any LangChain / LangGraph / Deep Agents agent building project before consulting other skills or writing any agent code. Required starting point for up to date info on framework selection (LangChain vs LangGraph vs Deep Agents vs hybrid composition), agent patterns, install, environment setup, and which skill to load next.