Represent documents with word counts
Build a bag-of-words view and weigh words by how much they tell documents apart.
- Turn a document into a bag of word counts
- Calculate TF-IDF and explain what it rewards
- Name what count-based representations miss
A program can’t compare sentences directly, but it can compare numbers. The simplest way to turn a document into numbers is to count its words. This bag-of-words representation keeps which words occur and how often, and throws away their order - as if you tipped the words into a bag and shook it.
from collections import Counter
words = "cats chase cats".split()
print(Counter(words))Counter({'cats': 2, 'chase': 1})Not every word is equally informative
Raw counts reward words like the, which appear everywhere. TF-IDF fixes that by multiplying two numbers:
- Term frequency (tf): how much of this document the term makes up - its count divided by the document’s length.
- Inverse document frequency (idf): how rare the term is across all documents: , where is the number of documents and df how many contain the term.
A word in every document gets ln(N/N) = ln(1) = 0, so it carries no weight at all. A word that is frequent here but rare elsewhere gets the highest weight - it is what makes this document different.
Try it
Up to 15 XPTF-IDF explorer
Click terms and watch the three numbers under each document.
- Try the: why is its TF-IDF zero everywhere, even though it is the most common word?
- Try cat and then purr: which one is the better clue to the first document, and why?
- Each card shows its top term - is it a good one-word summary?
Pick a term
Pets
The cat sat by the fire. The cat began to purr.
- tf
- 0.273
- idf = ln(3/3)
- 0.000
- tf × idf
- 0.000
Top term here: sat
Weather
The rain fell all day. The storm brought rain and wind.
- tf
- 0.182
- idf = ln(3/3)
- 0.000
- tf × idf
- 0.000
Top term here: rain
Animals
The dog chased the cat across the field.
- tf
- 0.375
- idf = ln(3/3)
- 0.000
- tf × idf
- 0.000
Top term here: dog
“the” appears in 3 of 3 documents - everywhere, so its idf is 0 and it can’t tell documents apart.
Key takeaways
Bag of words = word counts without order: simple, fast, and blind to “dog bites man” vs “man bites dog”.
TF-IDF = how frequent a term is here × how rare it is overall. Words found everywhere get weight 0.
Counts capture topics well, but not meaning, negation or word order.
Lesson quiz
7 questions · pass with 5 correct · up to 50 XP
Passing this quiz completes the lesson and keeps your streak going. Questions you miss come back in review sessions later.
Practice: apply NLP with Python
Try each text-processing idea in Python, run it against sample inputs, and use the results to see where the method works or falls short.
Count the words
Read text, lowercase it, extract alphabetic words, then print each distinct word and count in alphabetical order as word: count.
- Repeated words
- Three tokens
Python runs in a sandboxed browser worker with a 60 second time limit. Its runtime loads from the Pyodide CDN; your code stays in this browser.
Calculate TF-IDF
The first line is a term. The second is the number of documents N, followed by N lines of documents. Treat each document as its lowercase alphabetic words (re.findall(r"[a-z]+", line.lower())).
For each document, print its TF-IDF for the term rounded to 3 decimals, using and . If no document contains the term, print 0.0 for each.
- A distinctive word
- A word in every document
Python runs in a sandboxed browser worker with a 60 second time limit. Its runtime loads from the Pyodide CDN; your code stays in this browser.
Questions about this lesson
Stuck? Ask. Figured something out? Share it. Explaining is one of the best ways to learn.
Loading posts…