# Code and data for "Weaviate Tutorial: Hybrid Search Over a Crawled Docs Site"

Everything here produced the numbers in the post. Python 3.11, versions in `requirements.txt` (`pip-freeze.txt` is the full venv).

## Layout

Scripts expect `../data/chunks.jsonl` and `../data/ingest.py` (a copy of `ingest.py` is in this folder). `chunks.jsonl` is not included (304 chunks with 768-float vectors). It comes from the shared ingest pipeline: crawl the WebCrawlerAPI docs, split by headings, embed with local Ollama `nomic-embed-text`. Fields per row: `id, url, heading, text, embedding, model, dim`. Documents were embedded as `"search_document: " + heading + "\n" + text`, queries as `"search_query: " + question`. The scripts also write to `../logs` and `../render` (scratch folders), which are not part of this folder.

## Run order

```bash
docker compose -f docker-compose.yml -p kb-weaviate up -d   # ports bound to 127.0.0.1
python load.py                                              # create DocsChunk, insert 304 chunks with vectors
python search.py name_not_resolved                          # three query modes on one question
python make_questions.py && python check_overlap.py         # draft eval set, overlap check (overlap.json)
# freeze: copy eval_set_draft.json to eval_set.json, write its sha256 to eval_set.sha256, write params.json
python evaluate.py > evaluate.out                           # 6 configs x 3 repeats, writes results.json
python report.py                                            # results.md
python chart.py                                             # c1-recall.png
docker compose -p kb-weaviate down -v                       # removes only this project's container, network and volume
```

## Freeze record

| What | When (UTC) | Hash or value |
|---|---|---|
| Parameters (`params.json`): image tag, client version, embedding prefixes, metrics, fixed comparison, alpha list | 2026-11-02 08:39:47 | n/a |
| Eval set (`eval_set.json`, 28 questions) | 2026-11-02 08:39:47 | sha256 `b0a68f3c3e410ccb36276dd406e02f4cdd72fb95256d4a6f8ce060072c88da9e` |
| First and only scored run (`results.json`) | 2026-11-02 08:40:38 | n/a |

`evaluate.py` refuses to run if `eval_set.json` no longer matches `eval_set.sha256`. A first draft of question 17 shared 4 salient words with its gold chunk (limit: 2), so it was reworded before the freeze. No retrieval score was computed before the freeze. Nothing was tuned after the run.

## Setup facts

- Weaviate image `cr.weaviate.io/semitechnologies/weaviate:1.40.3`, compose file adapted from the Weaviate Docker installation page read 2026-11-02 (ports on 127.0.0.1, telemetry off, no vectorizer module).
- `weaviate-client` 4.20.0. `HybridFusion` has two members, `RANKED` and `RELATIVE_SCORE`. Default fusion in the server is relative score (Weaviate docs, since v1.24).
- BM25 defaults from the Weaviate docs: `word` tokenization, k1 1.2, b 0.75, operator `or`.
- BM25 and the keyword leg of hybrid both search `heading` and `text` (`query_properties`).

## Labels

Questions and gold labels were written by an LLM (Claude, model id `claude-sonnet-5-5`) with the answer chunks visible. Gold is page level. The paraphrase overlap check is `check_overlap.py` (`overlap.json`). `hand_check.json` records that 14 gold chunks were re-read by the same model. No human reviewed the labels.