Applications & Agents¶
What this page solves
Turning a model into a system that solves real problems: connect your own documents (retrieval augmentation), connect tools (agents), and prove it works (evaluation). Required practice: much day-to-day LLM work in industry lives here.
How far to go¶
| Level | Learn | Done when |
|---|---|---|
| Orientation | Prompts, context, RAG, tool calls, agents | One QA flow with retrieval and a tool drawn |
| Foundation | Chunking, embeddings, retrieval, source display, test sets | F2 done; retrieval and generation failures told apart |
| Advanced | Agent loops, structured output, multi-step planning, automated evaluation | One read-only tool added to F2 and evaluated |
Core knowledge¶
Prompt and context engineering: the model only sees its context. System prompts set role and rules, few-shot examples set format, relevant material sets the source of facts. Longer context costs more and dilutes attention.
Retrieval-augmented generation (RAG): chunk documents, embed and index them; at question time retrieve the most relevant chunks into the context and answer from them. Knowledge stays updatable and answers traceable.
Chunking and retrieval quality: chunks too large add noise, too small lose context; hybrid retrieval (BM25 keywords + vectors) and rerankers improve recall markedly. Most RAG problems are retrieval problems, not generation problems.
Tool calling: the model emits a structured call (name + arguments), the program executes it and returns the result. The model executes nothing itself; permissions and safety live in the program.
Agents: a "think → call tool → observe → continue" loop until the task is done. Powerful but brittle: more steps, more accumulated error; limit steps and validate each output.
Structured output: ask for JSON matching a schema, or use constrained decoding to guarantee the format, so downstream code parses reliably.
Evaluation: a fixed question set (including unanswerable and adversarial ones), human or model grading, statistics by failure type. Without a test set every "improvement" is a feeling.
Debate · long context vs. retrieval¶
Long context: windows reach a million tokens, so put everything in — simpler, nothing missed. Retrieval: cheaper, faster, traceable, and information in the middle of long contexts gets overlooked. In practice both: retrieve to narrow down, then use a long-enough context for the relevant material.
Curated resources¶
-
Practice · LlamaIndex Starter
The smallest document QA flow: load, chunk, index, retrieve, generate.
-
Agent concepts · HF Agents Course
Tool calling, agent loops and evaluation, from concepts to small examples.
Follow-along experiment¶
-
F2 · QA over your website
Index your own Markdown, show sources, evaluate on 20 questions, compare plain QA with RAG.
Self-check¶
A RAG answer is wrong. How do you tell retrieval from generation?
Check whether the retrieved chunks contain the answer: if not, it is retrieval (chunking, embeddings, top-k); if yes but wrong, it is generation (prompt, model, context too long).
Why include questions the documents cannot answer?
To test whether the model admits it does not know instead of inventing an answer — the most common hallucination scenario.
Why do agents get less reliable with more steps?
Each step has some error rate and errors propagate; ten steps at 95% each gives about 60% overall.
Where is the safety boundary of tool calling?
The model only requests calls; the program executes them, validates arguments, enforces permissions and requires human confirmation for side effects.