Skip to main content
AdOpenFree logoPromote your productReach more potential users and drive product growth and revenue.Advertise

Tools tagged with "Evaluation"

Favicon of Warp

Warp

Free ListingStars: 65.2K

Open infrastructure for cloud software factories

Coding Agents

Warp is an agentic development environment, born out of the terminal, offering open infrastructure to build, measure, and interact with agents across the SDLC so teams ship more and spend less.

Favicon of nanochat

nanochat

Free ListingStars: 60.1K

Minimal experimental harness for training LLMs

Language Models

nanochat is a minimal, hackable harness for training LLMs end to end on a single GPU node, covering tokenization, pretraining, finetuning, evaluation, and inference. It is pitched as the best ChatGPT that $100 can buy.

Favicon of Langfuse

Langfuse

Free ListingStars: 35.1K

Open Source Agent Evals & Observability

Evaluation & Benchmarks

Open source platform for tracing, evaluating, and improving AI agents. Use production data to understand behavior, collaborate on fixes, and ship better quality at lower cost and latency.

Favicon of Laya

Laya

Free ListingStars: 34K

Typed decisions with calibrated probabilities in one pass

Language Models

Laya is a non-autoregressive System 1 decision engine for typed choice, score, and yes/no questions over text. It returns calibrated probabilities in one forward pass across 100+ languages and routes requests to the right checkpoint.

Favicon of Mastra

Mastra

Free ListingStars: 28.9K

TypeScript framework for AI agents and applications

Agent Frameworks

Mastra is an open-source TypeScript framework for building AI-powered applications and agents, with agents, workflows, memory, workspaces, and observability for moving from prototype to production.

Favicon of promptfoo

promptfoo

Free ListingStars: 26.1K

Automated testing and red teaming for LLM apps

Security Testing & Red Teaming

Promptfoo is an open source CLI and library for evaluating and red-teaming LLM-based applications. It provides automated testing to find and fix AI risk, compare model providers, and test prompts, agents, and RAGs during development.

Favicon of Agent Development Kit (ADK)

Agent Development Kit (ADK)

Free ListingStars: 22K

Open-source framework for building production AI agents

Agent Frameworks

ADK is an open-source, code-first framework for building, evaluating, debugging, and deploying reliable AI agents at enterprise scale. It is available in Python, TypeScript, Go, Java, and Kotlin.

Favicon of Arize Phoenix

Arize Phoenix

Free ListingStars: 11.6K

AI Observability and Evaluation

Evaluation & Benchmarks

Arize Phoenix is an open-source AI observability platform for tracing, evaluating, and improving LLM applications. It provides tracing via OpenTelemetry, LLM-based evaluations, prompt management, and experiments to compare changes. Built

Favicon of Kev

Kev

Free ListingStars: 9.6K

Small Jev-like decision models to train and run yourself.

Language Models

Kev is a family of small decision models built on Qwen3.5 and Qwen3.8, based on Jev's architecture. Use pretrained weights or train your own, with yes/no, multiple-choice, and rating questions in one request.

Favicon of Harbor

Harbor

Free ListingStars: 5.6K

Evaluate and optimize sandboxed agents and models.

Agent Frameworks

Harbor is a framework from the creators of Terminal-Bench for evaluating and optimizing agents and language models in container environments. It supports arbitrary agents, custom benchmarks, parallel cloud experiments, and rollouts for RL

Favicon of ADR

ADR

Free ListingStars: 2.1K

Enterprise security for AI agents

Security Testing & Red Teaming

ADR (Agentic AI Detection and Response) is an enterprise security system for AI agents. It discovers unsanctioned AI tools, observes agent activity, benchmarks defenses, and detects threats across employee-facing and customer-facing agents.

Favicon of Folkbench

Folkbench

Free ListingStars: 2K

AI relay station benchmarks and rankings

Evaluation & Benchmarks

Folkbench benchmarks AI API relay stations and publishes availability, response time, and pricing rankings, plus Folkbench Switch, an open-source desktop client for Claude Code, Codex, and similar tools.

Favicon of Overmind

Overmind

Free ListingStars: 0.7K

The training platform for specialized agents

Training & Fine-Tuning

Overmind is a training platform for specialized AI agents that turns production traces into datasets, evaluations, fine-tuned models, and deployments you own, with hosted or self-hosted operation.

Ad
OpenFree logoPromote your product

Reach more potential users and drive product growth and revenue.

Advertise