Domain
AI Systems
Building a model demo takes an afternoon. Building an AI system that holds up under real traffic, real cost constraints, and real failure modes is a different discipline — seven patterns that recur, in some combination, in nearly every production LLM/agent architecture. This domain covers each one in depth, and — the angle the rest of this notebook cares about — how the pattern's shape changes when the model runs on, or alongside, embedded hardware instead of purely as a cloud service.
Some patterns exist mainly to improve output quality (RAG, Fine-tuning, Agentic Evals). Others exist mainly to control cost and risk once quality is acceptable (Caching, Guardrails & Defensive UX, Observability). User Feedback sits in the middle and feeds both over time. A strong system design names this trade-off explicitly instead of treating every pattern as a free win — the same "why, and when" discipline this notebook already applies to CAN vs. SPI vs. I2C.
Topics
- RAG — grounding generation in retrievable, verifiable contextwritten
- Fine-tuning — specializing weights when prompting hits a ceilingwritten
- Caching — cutting latency and per-token cost with reusewritten
- Guardrails & Defensive UX — enforcing safety at the boundarywritten
- User Feedback — closing the loop into a self-improving flywheelwritten
- Agentic Evals — scoring trajectories, not just answerswritten
- LLM Observability — tracing and monitoring live trafficwritten
Planned
prompt engineering model serving on-device inference / TinyML
These seven patterns were first popularized for cloud-hosted LLM products, and most references stop there. An embedded engineer building anything AI-adjacent — a voice-controlled appliance, an edge-vision camera, a fleet of NPU-equipped gateway devices, an LLM-assisted diagnostic tool talking to a vehicle's own ECUs — hits every one of these patterns too, just with a different budget: milliwatts instead of a GPU cluster, a flash partition instead of a vector database with unlimited storage, and a real actuator on the other end of a "friction by design" confirmation step instead of a web form. Each page here ends with an Embedded applications section making that translation explicit.