Systems & architecture
6 posts tagged #llm on DiCodeWeb, from fundamentals to production, newest first.
6 postsUpdated Oct 24, 2026
A practical threat model for applications that put an LLM between tools and private data: direct and indirect prompt injection, the lethal trifecta, and the design patterns that keep the damage contained, mapped against the OWASP Top 10 for LLM Applications.
What happens to an LLM feature once real traffic arrives: where latency actually comes from, a per-feature token and cost meter with tenant budgets, structuring prompts for provider prompt caching, exact and semantic caching, model routing, trimming context, batch APIs, parallel calls, retries with fallback and circuit breakers, and the dashboard I watch.
Building a golden dataset from real usage, deterministic checks first, LLM-as-judge with rubrics and its biases, pairwise comparison, running evals in CI on prompt or model changes, tracking cost and latency alongside quality, online signals, tracing, and how to avoid overfitting your eval set.
Tool calling is really a loop that your code drives. This post covers defining tools with schemas, an agent loop with a step limit, handling invalid arguments, a message list with typed parts, streaming partial arguments, rendering tool calls as components, and making users confirm tools with side effects.
The frontend half that AI chat tutorials tend to skip: picking a transport, parsing SSE by hand when events are split across chunks, a Route Handler that keeps the API key safe and propagates cancellation upstream, message-parts state, batching renders with requestAnimationFrame, safe incremental markdown, autoscroll, accessibility and error handling.
How I build product features on LLMs that return typed JSON: structured output modes in the major APIs, defining schemas with Zod, handling refusals and truncated output, retry with repair, versioning prompts in code, realistic expectations about temperature, and designing schemas models handle well.