Evaluation, Testing and Optimization at Scale
Proving and improving Claude solutions: evaluation strategy, benchmarks, A/B tests, optimization of quality, latency and cost.
CLA-403 12 lessons planned ▶ Video lessons coming soon
Offered by HSM University · Claude Certification Prep
Course content
- 1Evaluation strategy for enterprisesComing
- 2Defining success metricsComing
- 3Building and maintaining evaluation suitesComing
- 4Model-graded evaluation at scaleComing
- 5A/B testing models and promptsComing
- 6Optimizing qualityComing
- 7Optimizing latencyComing
- 8Optimizing costComing
- 9Migrating between model versionsComing
- 10Monitoring quality in productionComing
- 11Reporting results to stakeholdersComing
- 12Scenario: an evaluation program for a support assistantComing
Requirements
Recommended background. You can start any course at any time.
Description
Proving and improving Claude solutions: evaluation strategy, benchmarks, A/B tests, optimization of quality, latency and cost.