# Observability & evals for AI agents

Traces of what it did, and a score for whether that was any good.

6 options tracked · 4 open source · 4 self-hostable · required

## All options

| Option | What it does | Licence | Self-host | Page |
| --- | --- | --- | --- | --- |
| Langfuse | Open-source tracing, prompt management and evals. Self-hostable in one compose file. | open source | yes | https://newagent.build/c/langfuse |
| LangSmith | Tracing, datasets and evals from the LangChain team. | proprietary | no | https://newagent.build/c/langsmith |
| Braintrust | Eval-first platform — scorers, datasets and a playground for prompt iteration. | proprietary | yes | https://newagent.build/c/braintrust |
| Arize Phoenix | OpenTelemetry-native tracing and evals you can run locally. | open source | yes | https://newagent.build/c/phoenix |
| Pydantic Logfire | OpenTelemetry observability with first-class Python and Pydantic AI support. | open source | no | https://newagent.build/c/logfire |
| promptfoo | Local eval and red-team harness that runs in CI. No account needed. | open source | yes | https://newagent.build/c/promptfoo |

## Machine interfaces

- `GET https://newagent.build/api/registry` — every component as JSON
- `GET https://newagent.build/api/vendors` — vendors, with the repo and releases feed to watch
- `GET https://newagent.build/api/advise?q=<plain english>` — recommended stack for a description
- `GET https://newagent.build/api/stack?<layer>=<id>&format=sh|json|md|yml|agents|env` — a stack as files
- `GET https://newagent.build/llms.txt` — the whole registry in one fetch

Any page here also returns markdown if you send `Accept: text/markdown`.