Chunking in Practice

# AI For the Curious: RAG Tools

Companion code for the **[AI For the Curious](https://sambessey.com/articles)** series on [sambessey.com](https://sambessey.com).

This repo covers the **Chunking for RAG** article: how to break long documents into smaller pieces ready for embedding into a vector database, and how the strategy you pick affects retrieval quality.

---

## What is chunking and why does it matter?

Before an LLM can answer questions about your documents, those documents need to be turned into vectors and stored in a database. The problem: you can't embed a 50-page PDF as one blob. You slice it into chunks first, embed each chunk, and store them individually.

The chunks you retrieve at query time are what the LLM actually reads. Get chunking wrong and you get:

- **Missed answers** (the relevant content was split across a boundary)
- **Noisy answers** (too much irrelevant text crammed into one chunk)
- **Broken context** (a sentence retrieved mid-thought, with no surrounding information to anchor it)

Chunking strategy directly controls recall, precision, and coherence.

---

## The running example

The code works on a synthetic airline policy corpus: eight internal documents covering topics like irregular operations, fee waivers, loyalty tier entitlements, and passenger rights. Each document includes structured metadata fields (Policy ID, Owner, Audience, Sensitivity, Topics) modelled on how real enterprise knowledge bases are tagged.

The corpus lives in `data/policies/simple_example/`.

---

## Three chunkers, iterated

The code is structured as three progressively more capable chunkers. Each one builds on the last.

### chunker-1: structure-first splitting

Splits the document at every H2 heading (`##`). One section, one chunk. No metadata. No size control.

This is the simplest possible approach and the right starting point for understanding the problem. It fails when sections are wildly different lengths.

### chunker-2: adding metadata

Same H2-based split, but now each chunk carries:

- **headingPath**: the chain of headings leading to this section (document title + H2 heading)
- **metadata**: structured fields extracted from the policy header block (policy ID, owner, classification, audience, sensitivity, topics, and more)

A chunk without metadata is just text. A chunk with metadata becomes addressable: you can filter by audience, sensitivity level, or policy owner before the LLM ever sees it.

### chunker-3: token budget and overlap

Extends chunker-2 with two important mechanics:

**Token budget**: if a section exceeds the target token count (default: 500), it gets split into smaller sub-chunks at paragraph boundaries. Splitting on blank lines rather than raw character positions keeps paragraphs, list items, and table rows intact.

**Overlap**: the tail of each chunk is copied to the start of the next. This prevents a sentence or key fact from being stranded at a boundary where neither the preceding nor the following chunk contains enough context to understand it.

Token estimation uses a simple 4-chars-per-token approximation. The comments in the code explain how to swap in a real tokenizer (tiktoken, `@anthropic-ai/tokenizer`) for production use.

---

## Running the code

**Prerequisites:** Node.js 18+ and a package manager that supports `tsx`.

```bash
npm install
```

Run a chunker against one of the sample documents:

```bash
npx tsx index.ts chunker-1 10_irrops_reaccommodation.md
npx tsx index.ts chunker-2 10_irrops_reaccommodation.md
npx tsx index.ts chunker-3 10_irrops_reaccommodation.md
```

The first argument is the chunker (`chunker-1`, `chunker-2`, `chunker-3`). The second is any file in `data/policies/simple_example/`.

Output is printed with `console.dir` so nested objects are shown in full.

---

## File structure

```
.
├── chunking/
│   ├── chunker-1.ts      # H2-based split, minimal output
│   ├── chunker-2.ts      # Adds heading path and metadata extraction
│   └── chunker-3.ts      # Adds token budget enforcement and overlap
├── data/
│   └── policies/
│       └── simple_example/
│           ├── 00_readme.md
│           ├── 10_irrops_reaccommodation.md
│           ├── 20_fee_waivers_and_fare_rules.md
│           ├── 30_partner_airline_guidelines.md
│           ├── 40_service_recovery_vouchers.md
│           ├── 50_loyalty_tier_entitlements.md
│           ├── 60_flight_disruption_faq.md
│           ├── 70_passenger_rights_and_compensation.md
│           └── contents.md
├── index.ts              # Entry point: picks the chunker and file from CLI args
└── package.json
```

---

## Part of a series

This repo sits inside the RAG track of the *AI For the Curious* series:

| Article | What it covers |
|---|---|
| Primer: Databases | The main database technologies and how they relate to LLMs |
| RAG and MCP 101 | A quick introduction to two key concepts underpinning the series |
| pgvector Setup for Airline RAG | Setting up pgvector locally and running similarity queries |
| **Chunking for RAG** | **This repo: breaking documents into chunks ready for embedding** |

Read the full article at [sambessey.com/articles](https://sambessey.com/articles).