COMING SOON · LLM PRODUCTION WORKLOADS

Cut your LLM token
costs by 80-99%.

The complete toolkit for prompt caching, KV cache optimization, and intelligent token management — without sacrificing quality.

Get Early Access View on GitHub

Input token savings

via prompt caching

Fewer output tokens

efficient tool use

Cost reduction

model routing

At 100k calls/month: save $17,900 · 96% fewer tokens · 8× faster

Real customer data

Daily API spend, before vs after

Anonymized production traffic. 30 days. Same workload.

Before smart-token

After smart-token

$681

Peak daily spend (before)

$47

Peak daily spend (after)

93%

Total cost reduction

$19k saved

Over 30 days

Capabilities

Everything you need to run LLMs efficiently

Battle-tested optimization techniques from the frontier of production LLM serving.

Prompt Caching

Automatic cache-header injection across Anthropic, OpenAI, and self-hosted models. Zero code changes.

90% ↓ input tokens

KV Cache Optimization

PagedAttention, RadixAttention, disaggregated prefill/decode. 3× throughput on the same hardware.

3× throughput

Smart Model Routing

Route by complexity: Haiku for simple, Opus for hard. Learned from your traffic patterns.

60-95% ↓ costs

Cost Observability

Real-time token accounting, per-endpoint budgets, and anomaly detection out of the box.

Full visibility

Prompt Compression

LLMLingua-style compression that preserves semantics — safely shrink context before it hits the model.

2-6× smaller

Batch API Integration

Automatically queue non-urgent work to batch endpoints. 50% off on eligible calls.

50% ↓ cost

Works with your existing LLM stack

Anthropic

OpenAI

vLLM

SGLang

LangChain

LiteLLM

Built for

Every LLM workload that matters

Agent Workflows

10-step agents, 50k tokens each. Caching = massive win.

RAG Pipelines

Repeated context across queries. Prefix cache = 90% off.

Chatbots

Multi-turn context accumulates. Smart truncation saves cash.

Batch Inference

Nightly jobs, offline eval. Batch API = 50% off input.

Calculate your savings

Try it yourself. See what you'd save.

Drag the sliders. Numbers update in real time using published API rates.

Requests per month 100,000

1K100K10M

Avg tokens per request 12,000

50010K200K

Model Opus

You would save

$16,830

per month · 93% cost reduction

Without smart-token $18,000

With smart-token $1,170

Annualized savings $201,960 / yr

Based on published Anthropic API rates · assumes 65% cache hit rate + 20% routing to smaller models

Early adopters

Teams shipping with smart-token

Real numbers from teams running LLMs in production.

“

We were burning $40k/month on Opus for our support agent. smart-token dropped that to under $3k without touching a line of application code. The prompt caching alone paid for itself in a week.

-92% monthly LLM spend

Maya Krishnan

Staff Eng · Fintech unicorn

“

Our RAG chatbot's TTFT went from 2.4s to 240ms after enabling KV cache reuse. Users noticed immediately. Support tickets about "slow responses" went to zero.

10× faster time-to-first-token

Daniel Rossi

CTO · Legal AI startup

“

Model routing is what sold us. 80% of our traffic doesn't need Opus — smart-token figured that out from our patterns and routed automatically. Same quality, fraction of the price.

$47k saved in Q1

Jamie Tan

Head of AI · Dev-tools SaaS

Pricing

Pay for what you save, not what you send

Start free. Scale only when you're already saving money.

Free

$0 forever

Perfect for side projects and hobbyists exploring token optimization.

Up to 100k tokens/month
Prompt caching & basic routing
Community support
Open-source SDK

Start free

Common questions

Not seeing your question? Just ask.

Do I need to rewrite my LLM code?

No. smart-token is a drop-in replacement for the Anthropic and OpenAI SDKs. Change one import line and every optimization — prompt caching, routing, cost tracking — is on by default. For advanced control, opt in per-request with cache: true or model: "auto".

Which providers and models are supported?

All major API providers (Anthropic, OpenAI, Google, xAI, Groq, Together, Fireworks) plus any self-hosted vLLM or SGLang endpoint. Model routing picks between Opus, Sonnet, Haiku, GPT-4o, GPT-4o-mini, and their equivalents based on your task complexity.

How does prompt caching actually save money?

Cached input tokens are charged at 10% of the base rate (Anthropic) or ~50% off (OpenAI). smart-token automatically injects cache breakpoints at optimal positions in your system prompt and few-shot context, and it deduplicates repeated prefixes across requests. Typical savings: 90% off input tokens for RAG and agent workloads.

Will my prompts or data ever leave my infrastructure?

On the Enterprise plan, smart-token runs entirely inside your VPC — nothing leaves your network. On Free and Pro, only anonymized token counts and latency metrics are sent for observability (never prompt contents). SOC 2 Type II and HIPAA BAA available for Enterprise.

What if my traffic is bursty or unpredictable?

Pro billing is metered on actual tokens processed, not peak capacity. Combined with routing (fallback to smaller models under load) and batch queuing (defer non-urgent work to 50%-off batch endpoints), you pay less during spikes, not more.

How is this different from just using LiteLLM?

LiteLLM unifies APIs. smart-token unifies APIs and actively optimizes them — caching, routing, compression, and observability all live in one layer. Think of us as the optimization layer that sits above your gateway.

Get notified

Be first to try smart-token

Join the waitlist. We'll email you when the SDK ships — no spam, one message.

Cut your LLM token costs by 80-99%.