Companies in the category 'LLMOps'

Langfuse is an open-source observability platform for LLM applications. It allows users to debug, analyze, and improve their language model applications with traces, spans and metrics. Langfuse serves enterprises and is built on agents, evals, and tracing technologies.

Portkey is a unified control plane for production AI that enables engineering teams to manage, monitor, and optimize large language model deployments at scale. The platform provides a high-performance AI gateway with built-in observability, routing, guardrails, and cost management, sitting directly in the path of AI traffic to enforce policy in real time. Portkey supports over 1,600 models across 60+ providers and processes more than 120 million daily requests, serving over 24,000 organizations including Fortune 500 enterprises.

Agenta is an open-source LLMOps platform that provides developers and product teams with integrated tools for prompt management, evaluation, human annotation, and observability of large language model applications. The platform enables teams to version and compare prompts, run automated and human-in-the-loop evaluations, trace production requests, and collaborate across engineering and domain-expert roles—all from a single interface.

TraceLoop provides an open-source LLM monitoring and observability platform, enabling comprehensive tracing and debugging of LLM applications in production. Built on technologies like OpenTelemetry, it works with most major LLMs and vector databases and provides tools for building structured evaluation frameworks for AI applications. TraceLoop primarily serves enterprise clients.

WhyLabs provides an AI observability platform that helps organizations monitor and manage the health, security, and performance of their AI applications. It offers tools for real-time data quality monitoring, model performance tracking, and LLM security.

AgentOps provides an agent evaluation and observability platform. It allows users to visualize agent behavior, catch regressions, and deploy confidently using advanced evals and metrics. Their platform serves enterprise clients.