Remaining time after subtracting client, network, auth, prompt, and post-processing overheads from total SLA.

How is TTFT calculated?

User sets TTFT percentage of LLM budget; max TTFT = LLM budget × TTFT%.

What does traffic light mean?

Green: LLM budget >500ms, Yellow: 200-500ms, Red: <200ms (infeasible).

Total user-facing SLA (ms)

Client-side processing 50

Network RTT 80

🌍 region preset:

Auth / middleware 30

Prompt construction 20

Post‑processing / parsing 30

🧠 LLM budget (after overheads) 1790 ms

🔹 TTFT % 40%

Max TTFT: 716 ms Gen TPS: —

Output tokens budget:

🚦 Feasibility: GREEN P95 ×1.5

📋 Compatible models (by TTFT) LLM budget 1790ms

Class	Models	TTFT range	Status
⚡ Fast	Haiku, GPT-4o-mini, Gemini Flash	<200ms	✅
🔶 Medium	Sonnet, GPT-4o	200–500ms	✅
🐢 Slow	Opus, o1	>500ms	⚠️

* based on max TTFT from LLM budget & TTFT%