Open Source · v0.0.1

One SDK.
Every model.
Full observability.

Provider-agnostic LLM analytics. Cost, latency, quality, usage — powered by the Countly platform. Self-hosted. GDPR compliant.

Explore Adapters ↓ GitHub →

Adapters

799

Tests

16+

Models

@countly-ai/openai

See everything behind
your GPT calls

Wrap your OpenAI client with one line. Token usage, cost, latency, tool calls, streaming — all captured automatically. Zero config.

Streaming Tool Calls Reasoning Tokens Cost Tracking

npm install @countly-ai/openai

app.ts

import OpenAI from "openai";
import { observeOpenAI } from "@countly-ai/openai";

const openai = observeOpenAI(new OpenAI(), {
  appKey: "YOUR_APP_KEY",
  url: "https://your-countly.com",
});

// Use exactly as before — observability is automatic
const res = await openai.chat.completions.create({
  model: "gpt-4o",
  messages: [{ role: "user", content: "Hello" }],
});

@countly-ai/anthropic

Understand what Claude is thinking — and costing

Full observability for Anthropic's Claude. Cache tokens, tool use blocks, streaming — every metric captured with zero friction.

Cache Tokens Tool Use Message Streaming TTFT Tracking

npm install @countly-ai/anthropic

app.ts

import Anthropic from "@anthropic-ai/sdk";
import { observeAnthropic } from "@countly-ai/anthropic";

const anthropic = observeAnthropic(new Anthropic(), {
  appKey: "YOUR_APP_KEY",
  url: "https://your-countly.com",
});

const msg = await anthropic.messages.create({
  model: "claude-sonnet-4-20250514",
  max_tokens: 1024,
  messages: [{ role: "user", content: "Hello" }],
});

@countly-ai/ai-sdk

Zero-config tracing
for the AI SDK

Drop in a SpanProcessor. Every Vercel AI SDK call traced via OpenTelemetry. No code changes to your generateText or streamText calls.

OpenTelemetry Span Processing Auto-trace

npm install @countly-ai/ai-sdk

instrumentation.ts

import { CountlySpanProcessor } from "@countly-ai/ai-sdk";
import { NodeTracerProvider } from "@opentelemetry/sdk-trace-node";

const provider = new NodeTracerProvider();
provider.addSpanProcessor(
  new CountlySpanProcessor({
    appKey: "YOUR_APP_KEY",
    url: "https://your-countly.com",
  })
);
provider.register();

@countly-ai/google-genai

Gemini observability,
out of the box

Wrap Google's GenAI client. Every generateContent and stream call tracked with token counts, costs, and latency — no configuration.

generateContent Streaming Safety Filters Function Calls

npm install @countly-ai/google-genai

app.ts

import { GoogleGenAI } from "@google/genai";
import { observeGoogleGenAI } from "@countly-ai/google-genai";

const client = observeGoogleGenAI(
  new GoogleGenAI({ apiKey: "..." }), {
    appKey: "YOUR_APP_KEY",
    url: "https://your-countly.com",
  }
);

const res = await client.models.generateContent({
  model: "gemini-2.5-pro",
  contents: "Explain quantum computing",
});

@countly-ai/langchain

Every chain, every
callback, every metric

Drop in a callback handler. LLM calls, tool executions, chain completions — all lifecycle events captured automatically.

Callback Handler Tool Lifecycle Concurrent Runs

npm install @countly-ai/langchain

● agent.ts

import { CountlyCallbackHandler } from "@countly-ai/langchain";

const handler = new CountlyCallbackHandler({
  appKey: "YOUR_APP_KEY",
  url: "https://your-countly.com",
});

// Works with any chain, agent, or LLM
const result = await chain.invoke(
  input, { callbacks: [handler] }
);

@countly-ai/mastra

Full trace visibility for
Mastra agents

Export traces from Mastra's telemetry pipeline. Spans grouped by trace, aggregated into LLM metrics, shipped to Countly.

Trace Aggregation Span Processor Observability

npm install @countly-ai/mastra

● mastra.config.ts

import { Mastra } from "@mastra/core";
import { CountlyMastraExporter,
  CountlySpanProcessor } from "@countly-ai/mastra";

const mastra = new Mastra({
  telemetry: {
    exporters: [new CountlyMastraExporter({
      appKey: "YOUR_APP_KEY",
      url: "https://your-countly.com",
    })],
    processors: [new CountlySpanProcessor()],
  },
});

@countly-ai/cohere

Command your Cohere
costs and quality

Observe both v1 and v2 chat APIs. Token counts, tool calls, finish reasons — all normalized into Countly's unified data model.

V1 + V2 API Tool Calls Cost Tracking

npm install @countly-ai/cohere

app.ts

import { CohereClient } from "cohere-ai";
import { observeCohere } from "@countly-ai/cohere";

const cohere = observeCohere(
  new CohereClient({ token: "..." }), {
    appKey: "YOUR_APP_KEY",
    url: "https://your-countly.com",
  }
);

const res = await cohere.v2.chat({
  model: "command-r-plus",
  messages: [{ role: "user", content: "Hello" }],
});

@countly-ai/llamaindex

Index your LLM
performance, not just data

Attach an event handler to LlamaIndex. LLM and tool lifecycles captured with camelCase and snake_case format support.

Event Handler Multi-Format Concurrent

npm install @countly-ai/llamaindex

handler.ts

import { CountlyEventHandler } from "@countly-ai/llamaindex";

const handler = new CountlyEventHandler({
  appKey: "YOUR_APP_KEY",
  url: "https://your-countly.com",
});

import { Settings } from "llamaindex";
Settings.callbackManager.on(
  "llm-end",
  handler.handleLLMEnd.bind(handler)
);

Ship with confidence.
Observe everything.

Your LLM data stays on your infrastructure. SOC 2, HIPAA, GDPR compliant. No vendor lock-in.

Get Started on GitHub Learn about Countly →

npm install @countly-ai/openai @countly-ai/anthropic @countly-ai/google-genai ...

One SDK.Every model. Full observability.

See everything behindyour GPT calls

Understand what Claude is thinking — and costing

Zero-config tracing for the AI SDK

Gemini observability,out of the box

Every chain, everycallback, every metric

Full trace visibility for Mastra agents

Command your Coherecosts and quality

Index your LLMperformance, not just data

Ship with confidence. Observe everything.

One SDK.
Every model.
Full observability.

See everything behind
your GPT calls

Zero-config tracing
for the AI SDK

Gemini observability,
out of the box

Every chain, every
callback, every metric

Full trace visibility for
Mastra agents

Command your Cohere
costs and quality

Index your LLM
performance, not just data

Ship with confidence.
Observe everything.