Comet review

Comet’s Opik is an LLM observability and evaluation platform: a hosted cloud service or open-source software teams can self-host. It sits beside an LLM application or agent, capturing traces of activity such as context retrieval, tool selection and user feedback. Teams use those records to inspect behavior, group recurring issues, test changes and monitor agents in production.

Opik also provides Test Suites with pass/fail results and dataset evaluations with LLM-as-a-judge metrics. Ollie analyzes traces and recommends fixes. These functions make Opik an operational layer around an application, not a model provider or a replacement for the application’s runtime. Comet’s broader product set includes MLOps tools for comparing training runs, model versions and datasets. The homepage also describes Cost Intelligence for Claude Code and Codex usage. Opik is available through Comet Cloud or as self-hosted open-source software, so teams can choose where the observability service runs.

The integration surface shown by Comet is a Python package and framework integrations, rather than a documented REST API workflow. The homepage’s Python example imports track from opik and uses decorators to instrument functions. It also shows a LlamaIndex integration configured with set_global_handler("opik"). The vendor says Opik supports 60+ integrations, but teams should confirm that their framework and SDK versions are covered in the Opik documentation.

Comet also offers an MCP server. The homepage presents it as a way for a coding agent to access trace information and says Diagnostics uses it to surface recurring problems. That puts traces into an agent-assisted debugging workflow without making MCP a substitute for application instrumentation. The homepage links to GitHub and shows code examples, but does not establish a CLI or enumerate supported programming languages beyond its Python examples. Confirm SDK support, deployment instructions and integration-specific setup in the vendor’s docs before choosing an instrumentation approach.

Opik Open Source costs $0 per month for teams that self-host and includes observability and agent-testing features. Free Cloud is also $0, but is capped at 10 team members, 25,000 spans per month and 60-day data retention. Pro Cloud starts at $19 per month for up to 50 members and 100,000 spans, with 60-day retention; the plan allows customizable monthly span limits and retention.

Usage above Pro’s included span allowance costs $5 per additional 100,000 spans. Enterprise has unlimited members and custom usage plans, but its price is not published. Ollie’s coding harness has a free trial followed by token purchases; token prices are not published. The optional MLOps platform add-on is also unpriced. The pricing page lists monthly prices but does not state annual billing or discounts. Budget around spans first, then confirm custom limits, retention and any add-ons for the expected production workload.

Comet describes Opik as open-source software and offers a $0 Open Source plan for teams that self-host. That gives teams a route to run the observability platform outside Comet Cloud. The vendor does not name the licence on the vendor's site available here, so confirm the current licence and its terms before adopting or modifying the software.

The cloud and self-hosted options reduce dependence on a single hosting model, but they do not by themselves establish how easily traces, datasets or evaluations can be exported. Ask Comet about export formats, API access and migration paths before storing long-lived operational history in Opik. If you self-host, the team takes on operating the service and its underlying infrastructure. The open-source option may reduce hosting lock-in, but a migration still depends on the portability of your data and evaluation workflows.

Opik’s production features include dashboards, alerts, trace analysis and evaluation, and Comet says it can monitor agents across their production footprint. Its homepage says traces appear ready for debugging almost instantly, including at high volumes; treat that as a vendor claim and validate latency against your workload. Free Cloud and Pro Cloud both include 60-day retention by default. Pro allows teams to customize monthly span limits and retention, while Enterprise offers custom usage plans and flexible deployments.

The vendor does not publish request-rate limits on the pricing page, so ask about ingestion quotas, burst behavior and what happens when a span allowance is reached. Comet describes its platform as built for enterprise reliability and security, and lists SSO and dedicated support with SLAs for Enterprise. Confirm the applicable SLA, security controls, deployment architecture and compliance requirements with the vendor. Also check how alerts behave during ingestion delays and how usage is measured before putting the service on a critical debugging path.