Docs · Database

Sharded DBs

Monolith broken extremely well — each version/source its own file.

Manifest — 7 shards today

data/db/manifest.json — single source of truth (see tools/bible/split_db.py SHARDS)

default_bundle: core.db, verses.lxxe.db, interlinear.lxx.db, interlinear.gnt.db, lexicon.db (5 files, ~82 MB measured — ~70 MB design in docs/DB_SHARDING.md)

buried not shipped: interlinear.mt.db (233k), verses.nkjv.db (31k → WEB) — bundle:false, on-demand only

Planned next: verses.web.db (WEB PD) + verses.kjv.db (KJV PD/Crown) via scripts/import-translation.js — replaces NKJV.

Shard	Filter	Rows	Primacy
core.db	core tables only	5+3+0+4	core
verses.lxxe.db	verses.version='LXXE'	27,058	primary
verses.nkjv.db	verses.version='NKJV'	31,102	buried → WEB
interlinear.lxx.db	interlinear.source='LXX'	432,676	primary
interlinear.gnt.db	interlinear.source='GNT'	138,994	primary
interlinear.mt.db	interlinear.source='MT'	233,343	buried
lexicon.db	lexicon only	6,338	shared

Dry-run (#7 shards) → /tmp/metanoia-shard-test/manifest.json validated: source_bytes 123318272 (118 MB), source_sha256 8a6539a7…, per-shard sha256 in manifest, default_bundle 5 files ~82 MB (core 76K + lxxe 5.2M + lxx 57.2M + gnt 18.7M + lex 0.4M). Matches tools/bible/split_db.py:31–92 SHARDS order + keep_sql predicates. See docs/DB_SHARDING.md:30–65 for design vs measured.

Script: tools/bible/split_db.py --source data/bible.db --out data/db — VACUUM INTO + DELETE WHERE NOT (keep_sql filter) + VACUUM. Each shard is a valid SQLite file with its own CREATE TABLE. Zig opens via ATTACH DATABASE 'data/db/<file>' AS schema with fallback to legacy data/bible.db if manifest absent. See docs/DB_SHARDING.md + Learn → English for NKJV→WEB rationale.

Back: Architecture

Next: TTS + Whisper