nemo-evaluator-sdk
Evaluates LLMs across 100+ benchmarks from 18+ harnesses (MMLU, HumanEval, GSM8K, safety, VLM) with multi-backend execution. Use when needing scalable evaluation on local Docker, Slurm HPC, or cloud platforms. NVIDIA's enterprise-grade platform with container-first architecture for reproducible benchmarking.
Installation
Follow the repository instructions for this skill and your agent. Review the files and dependencies before adding it to your workflow.
View source ↗GuidesSkill instructions
Features and purpose
Evaluates LLMs across 100+ benchmarks from 18+ harnesses (MMLU, HumanEval, GSM8K, safety, VLM) with multi-backend execution. Use when needing scalable evaluation on local Docker, Slurm HPC, or cloud platforms. NVIDIA's enterprise-grade platform with container-first architecture for reproducible benchmarking.
Quick start
Follow the repository instructions for this skill and your agent. Review the files and dependencies before adding it to your workflow.
Guides ↗Examples
Read the usage examples in the source repository.
View source ↗Source history
- Indexed on
- 2026-01-20
- Index updated
- 2026-02-13
- Branch
- main
Index and sync dates describe this directory. View the source repository for code history.
GitHub ↗chat Comments (0)
Sign in to join the discussion and leave a comment.
No comments yet. Be the first to share your thoughts!