Tech World Software SchoolTWSS
0x60Lesson 7 of 11

Generate answers grounded in sources

Build the prompt, order the context, demand citations, and let the model say “I don’t know”.

22 min 6-question quiz 2 code exercises
By the end of this lesson you can
  • Write a prompt that separates instructions, sources and question
  • Order context so the model actually uses it
  • Check that every claim is cited and supported

Retrieval found good chunks - now the model has to use them faithfully. A solid RAG prompt has clear parts:

  1. Instructions: answer only from the sources; cite the source ID after each claim; if the sources don’t answer the question, say so.
  2. Sources: each chunk with its ID, clearly delimited (for example in <source id="returns#2">…</source> tags), and treated as data - never as instructions to follow.
  3. The question, at the end.

Abstaining is a feature. “I couldn’t find that in our documentation” is far better than a confident guess, and it tells you which documents are missing.

Where you put evidence matters too. The “Lost in the Middle” study (Liu et al., TACL 2024) found that models use information at the start and end of a long context far better than information in the middle - accuracy dropped by more than 30% in some tests. So:

  • send fewer, better chunks (that’s what reranking is for), and
  • place the strongest evidence at the beginning and end.

Try it

Up to 15 XP

Find the unsupported claims

Read the sources, then check the generated answer. Click every claim that isn’t supported by the source it cites - or that has no source at all.

Returns question (round 1 of 2)Flags found 0/0

Click every suspicious section. There are 3.

SOURCES [s1] Returns are accepted within 30 days of delivery. Items must be unused. [s2] Refunds go to the original payment method within 5 business days. QUESTION How do returns and refunds work? ANSWER You can return unused items within 30 days of delivery [s1]. Refunds go back to your original payment method within 5 business days [s2]. Return shipping is free [s2]. Most customers see the money in 2 days. Damaged items can be exchanged in store.
prompt_template.py
sources = {"returns#2": "Opened electronics: 14 days, 10% fee."}
blocks = "\n".join(f'<source id="{key}">{text}</source>' for key, text in sources.items())
print(blocks)
Output
<source id="returns#2">Opened electronics: 14 days, 10% fee.</source>

Key takeaways

  • Separate instructions, delimited sources and the question; ask for citations and allow “I don’t know”.

  • Put the strongest evidence at the start and end; send fewer, better chunks.

  • A citation must actually support its claim - check them.

Lesson quiz

6 questions · pass with 5 correct · up to 50 XP

Passing this quiz completes the lesson and keeps your streak going. Questions you miss come back in review sessions later.

Practice: write Python

Write Python in the editor and run it against sample inputs. Python runs locally in your browser using a WebAssembly runtime.

Exercise 1

Put the best evidence at the edges

+25 XP

Each input line lists chunk IDs in relevance order (best first). Reorder them so the best chunks sit at the start and end: 1st goes first, 2nd goes last, 3rd second, 4th second-to-last, and so on. Print each reordered list.

  • Five and four chunks
main.py
Loading editor…

Python runs in a sandboxed browser worker with a 60 second time limit. Its runtime loads from the Pyodide CDN; your code stays in this browser.

Exercise 2

Check an answer’s citations

+25 XP

Lines before --- are sources: id|text. Lines after are answer sentences, each possibly ending with citations like [s1]. For each sentence (with the citations removed), print:

  • UNCITED: sentence if it has no citation;
  • BAD SOURCE id: sentence if it cites an unknown source;
  • UNSUPPORTED n: sentence if a number in it appears in none of its cited sources (first such number);
  • otherwise ok: sentence.
  • Five sentences
main.py
Loading editor…

Python runs in a sandboxed browser worker with a 60 second time limit. Its runtime loads from the Pyodide CDN; your code stays in this browser.

Questions about this lesson

Stuck? Ask. Figured something out? Share it. Explaining is one of the best ways to learn.

Loading posts…