GitHub - calmrocks/ai-engineer-notebooks: Hands-on, framework-free Colab notebooks for the AI Engineer / Forward Deployed Engineer (FDE) skill set — model APIs, structured output, tool calling, RAG, evals-as-the-spine, agents (loop from scratch, tool design, guardrails, MCP, Skills), fine-tuning vs LoRA, prompt-injection/security, LLMOps, and customer craft. Runs on the free Groq API.

Learn the applied-LLM stack the way you'll actually be interviewed on it: framework-free, on a free API, from prompting all the way to serving, fine-tuning, and a red-team benchmark. Runnable Colab notebooks for the AI Engineer / Forward Deployed Engineer (FDE) skill set. You build working systems on top of foundation models (model APIs, RAG, evals, agents, adaptation, serving) using raw APIs, not frameworks. What makes this different Framework-free, on purpose. You write the agent loop, RAG, and evals from raw API calls first, so you understand what LangChain/LlamaIndex actually do before you reach for them (and can judge when not to). Patterns are durable; wrappers churn. Evals are the spine. "Measure before you tune" is installed early and returns in every section. It's the habit that separates an engineer who shipped a system from one who built a demo. Free to run, end to end. Everything runs on the free Groq API (no credit card). The two topics Groq can't host, LoRA fine-tuning (06) and self-hosted serving (09), are concept-first with optional, fenced Colab-GPU appendices that were verified on a real Colab T4. Real case studies, not toy demos. Three end-to-end case studies show the skills combined under real constraints: a support assistant debugged in production, a pipeline-vs-agent cost showdown, and a red-team robustness benchmark. OpenAI-compatible throughout, so every pattern transfers directly to OpenAI and (with small changes) Anthropic. Swap the base URL and the skills carry over. Built as the hands-on companion to Plan: Transitioning to Forward Deployed Engineer / AI Engineer. The plan explains what to learn and why; these notebooks are where you run it. Who this is for Backend or full-stack engineers moving into AI Engineer, FDE, Applied AI, or Solutions Engineer (AI) roles. Different titles, largely the same job. You can ship production code; you want the applied-model layer on top. Learning order Work top to bottom. Each notebook is self-contained (installs its own dependencies, reads API keys from Colab secrets) and ends with exercises. 00 — Setup Notebook What you'll learn Environment & cost hygiene API keys via Colab secrets, spend guards, model picking 01 — Model APIs Notebook What you'll learn Prompting fundamentals Clear instructions, few-shot, output-format specs, step-by-step reasoning: the cheapest lever, each shown moving a number Structured output Getting reliable JSON out of a model, and where it breaks Tool calling Function/tool calling end to end, error paths included Streaming Streaming responses and what UIs need from them Context & caching Context-window budgeting, prompt caching, batch vs real-time pricing 02 — Evals I: measuring outputs Notebook What you'll learn Measuring outputs Golden sets and metrics on the section-01 task; install the "measure before you tune" habit before building anything you'd need to tune. Evals is the spine; it returns in every section after this 03 — RAG Notebook What you'll learn What is RAG? The retrieve → augment → generate loop, why RAG beats a plain LLM, and why RAG isn't the same as embeddings, with a 15-line working demo Embeddings & retrieval Embedding choice, vector search, similarity pitfalls. Get retrieval working first Hybrid & reranking Keyword + vector hybrid retrieval, rerankers, when each earns its cost Chunking Chunking strategies on a real messy corpus, revisited last once you can judge them against retrieval Why RAG fails Diagnosing bad answers: retrieval quality, not generation, is usually the bottleneck 04 — Evals II: the differentiator Notebook What you'll learn Golden sets Building a golden set for the RAG system from section 03 LLM as judge Judge prompts, agreement with humans, and the judge's own failure modes Regression evals Evals as CI: catching quality regressions when you change a prompt or model 05 — Agents Notebook What you'll learn Agent loop from scratch A working agent loop in raw API calls, no framework Tool design Designing tools the model can actually use well Guardrails & budgets Stopping conditions, cost/latency budgets, when a pipeline beats an agent MCP & the tool ecosystem Concept: what the Model Context Protocol standardizes, how it maps to the raw tool loop, and when to reach for it Skills & progressive disclosure Concept: packaging reusable know-how an agent loads on demand. The SKILL.md pattern, the context-budget payoff, and Tools/MCP/Skills as one story Harness engineering Synthesis: the scaffold around the call: context assembly & compaction, tool-result shaping, and verification loops. Names the discipline the section has been teaching piece by piece 06 — Adapting the model Notebook What you'll learn Fine-tune vs RAG vs prompt When to change the model's weights vs its inputs; what LoRA/QLoRA are and cost; the argument you'll have in the room, plus an optional real LoRA fine-tune on a free GPU 07 — Security Notebook What you'll learn Prompt injection & the trust boundary Direct & indirect prompt injection, output handling, PII, excessive agency. The OWASP LLM Top 10 risks, failing live then defended 08 — Operations Notebook What you'll learn Observability & LLMOps Tracing every call, safe prompt logging, cost/latency/error metrics, drift detection, and the observe→eval feedback loop Reliability & fallbacks Retries with backoff, timeouts, fallback models, output validation, circuit breakers, graceful degradation Experiment tracking & registry MLflow end to end: log runs/params/metrics from the section-04 eval harness, register and version a model, and promote by stage: the tooling that turns "I ran an eval" into a tracked, reproducible workflow 09 — Serving & inference performance Where the free Groq API can't run the topic (these frameworks need a GPU), the notebook teaches it concept-first and fences an optional Colab-GPU appendix, the same pattern as the section-06 LoRA appendix. Notebook What you'll learn Serving frameworks The serving stack an AI engineer actually picks between (vLLM, TGI, Triton, TensorRT-LLM): what each optimizes, how they map onto the raw API you've been calling, and when to reach for which Inference performance The levers behind throughput and latency: continuous batching, the KV cache, quantization, and the throughput-vs-latency trade, with the napkin math to size a deployment 10 — ML system design & performance Notebook What you'll learn Designing an inference service Concept: the ML system design interview, worked end to end: QPS/VRAM/latency/cost estimation, replica scaling, queueing, caching, and the SLA trade-offs, on a realistic LLM-serving prompt 11 — Customer craft (the FDE differentiator) Notebook What you'll learn Scoping & discovery Turn a vague customer ask into a scoped, evaluable system: discovery questions, a one-page scoping doc, the demo discipline: the customer-scenario interview round most engineers can't evidence 12 — Case Studies & Capstone Where the skills come together into projects. First a case study (one realistic scenario worked end to end, runnable), then the capstone: the deployed repo you build yourself. (Section overview.) Notebook What you'll learn Case study A — Customer-support assistant One scenario scoped → built → served → debugged in production: a vague ask becomes a deployed, evaluated RAG+agent assistant, then a live quality regression (a stale index after a corpus migration) that you diagnose and fix. A build-to-debug arc threading sections 02–11 Case study B — Contract extraction: pipeline vs agent The judgment call interviewers love: build the same extraction task as both an agent and a pipeline, then prove with accuracy + token cost that the pipeline wins when the steps are known Case study C — Red-team robustness benchmark A different kind of system, a harness that evaluates a model instead of serving one: an attacker→target→judge (PAIR) loop that measures attack success rate, composing the agent loop, LLM-judge, security, and evals Capstone: the brief for the deployed project that goes on your resume, a real repo with a serving component and an eval report. Case studies are for learning; the capstone is for hiring. Conventions Raw model APIs, no frameworks. Patterns are durable; wrappers churn. One shared corpus (data/) across RAG and eval sections, so evals measure the retrieval you actually built. Self-contained notebooks. First cell installs, second cell calls from aien import setup; client, MODEL = setup() to load your key from Colab secrets (or a local env var). No hidden state between notebooks. aien is the tiny shared-setup package in this repo (one place to change credential loading), installed automatically by the first cell. Every notebook ends with exercises. Do them before moving on. Setup Get a free API key at console.groq.com, no credit card required. In Colab: the key icon in the left sidebar → add GROQ_API_KEY as a secret, and toggle notebook access on. Open any notebook via its badge and run top to bottom. Running locally instead: pip install -r requirements.txt && pip install -e . (the second installs the aien setup helper), export GROQ_API_KEY=..., open with Jupyter. Related reading Plan: Transitioning to FDE / AI Engineer is the roadmap these notebooks implement Guide: Building a Real LLM Project for Your Resume sets the capstone's requirements bar Walkthrough: Designing a RAG System covers the systems view of section 03 Walkthrough: Designing an AI Agent Orchestration System covers the systems view of section 05