Repository tour
This page is the directory map, the one to keep open while you find your way around. It assumes you have read the System map, so it tells you where code lives rather than what it does. Layers and import contracts says which of these directories may import which.
packages/aizk/ ├── src/aizk/ the engine, one installable Python package ├── src/eval/ the benchmark and diagnostic harness, a second package ├── src/services/ the GLiNER sidecar, its own container ├── src/web/ the SvelteKit app ├── src/deploy/ compose file, Dockerfiles, Caddy, observability ├── docs/ this site └── tests/ one directory per engine packagesrc/aizk/, the engine
Section titled “src/aizk/, the engine”Twenty-nine top-level modules, grouped here by what they are for rather than by layer.
| Group | Modules | What lives there |
|---|---|---|
| transports | mcp/, api/, cli.py, commands/, client/ |
the four ways in |
| the shared service | memory.py |
find, keep and share for one caller |
| the store | store/ |
models, mixins, DDL, migrations, engine, identity |
| the write path | extract/, artifacts/, serving/ |
ingest, uploads, model clients |
| derived knowledge | graph/, ontology/ |
extraction, grounding, communities, vocabulary |
| the read path | retrieval/ |
lanes, fusion, reranking, packing, templates |
| autonomy | background/ |
queue adapters, jobs and the scheduler |
| operations | ops/, admin.py, backup.py, export.py, status.py |
doctor, probes, dumps, usage reports |
| foundations | config/, types.py, exceptions.py, provenance.py, common/ |
settings and leaf vocabulary |
| glue | runtime.py, auth.py, storage.py, integrations/, usage.py |
composition, identity, bytes, sidecar clients |
A few of those deserve a sentence.
store/ is the largest and the most structured. models/tables/ has one file per table,
models/views/ has the security-invoker views such as live_fact.py, mixins/ assembles every
table from reusable pieces, ddl/ holds custom SQLAlchemy DDL constructs for extensions, grants
and views, and identity/ holds User and Organization. migrations/versions/ holds the
PostgreSQL revision history. migrations/cockroachdb/versions/ holds the fused CockroachDB
baseline.
serving/ is where the model clients live, one subpackage per lane, so embed/, rerank/,
gate/, extract/ and chunk/. Every one of them talks to a container over HTTP and none of
them loads a model in-process.
integrations/ is the same idea for non-model services, with clamav/, docling/ and logto/.
Each has a client.py and typed models.py, so a sidecar’s wire format never leaks into the
engine.
retrieval/ splits into lanes/, find/ for the orchestrator, rerank/, packing/ for the
budget walk, models/ for the candidate and result types, and templates/ for the single Jinja
template that renders the answer.
graph/ is flat and each file is one step or one pass, which makes it the easiest package to read
end to end. build.py runs the per-chunk projection, grounding.py accepts only the proposed facts it can
tie back to the source text, dedupe.py and consolidation.py fold what survives into what is
already known, and communities.py, raptor.py, profiles.py, insight.py, decay.py, promote.py and
reembed.py are the scheduled passes.
background/ is small on purpose. queue.py selects PgQueuer for PostgreSQL or the portable
queue for CockroachDB behind the typed QueueJob and QueuePayload bases. jobs/ holds the job
families for conversion, projection and maintenance. schedule.py binds them to a worker and fans
scoped passes out over each exact scope set that has stored memory.
The other three source trees
Section titled “The other three source trees”src/eval/ is a separate installable package with its own entrypoint, aizk-eval. It holds the
corpus builders, the retrieval and extraction runners, the metrics and statistics code, and a
small FastAPI service for running plans. It imports the engine and the engine never imports it.
How we evaluate covers it properly.
src/services/gliner/ is two files, app.py and a Dockerfile. It is the only model server we
write ourselves, because GLiNER has no vLLM-compatible serving image. Everything else rides on
vLLM.
src/web/ is the SvelteKit app. src/lib/api/ is generated from openapi.json, which itself
comes from the FastAPI app, so a change to a browser API response type is regenerated rather than
hand-edited. src/routes/app/ has one directory per screen, and those directories are where the
user-facing renaming shows up, since findings are facts, subjects are entities and themes are
communities. The web app has the detail.
src/deploy/ holds the container builds and Compose profile. docker-compose.yml defines the
self-hosted services, Dockerfile builds both container and Lambda targets, and cockroachdb/
holds the cloud database setup. infra/aws/ declares the AIZK ECR, Lambda, S3, Scheduler,
Parameter Store, logging and budget resources.
docs/ and tests/
Section titled “docs/ and tests/”docs/ is this Astro and Starlight site. Pages live under src/content/docs/docs/, the sidebar
and integrations are in astro.config.mjs, interactive diagrams are Svelte components in
src/components/, the marketing landing page is src/pages/index.astro with its parts in
src/components/marketing/, and scripts/check-pages.mjs is the gate that fails the build on a
long page, a page with no diagram, or a broken link.
Writing these docs is the contract.
tests/ mirrors the engine, so tests/store/, tests/graph/, tests/retrieval/ and so on, with
shared fixtures in conftest.py and factories in factories.py and strategies.py. Tests marked
integration and benchmark are excluded by default. Coverage is gated at 100 percent.
Testing explains the layout and the markers.
If you want to change X, open Y
Section titled “If you want to change X, open Y”| You want to change | Open |
|---|---|
| a table, a column, a policy | src/aizk/store/models/tables/ then add a migration |
| what an MCP tool accepts or returns | src/aizk/mcp/server.py |
| what the web app can ask for | src/aizk/api/app.py, then regenerate the TS client |
| how a file becomes text | src/aizk/artifacts/ and src/aizk/integrations/docling/ |
| how text becomes chunks | src/aizk/serving/chunk/ |
| which chunks get extracted | src/aizk/serving/gate/ and src/aizk/extract/ |
| how facts are grounded or merged | src/aizk/graph/grounding.py, consolidation.py |
| a retrieval lane | src/aizk/retrieval/lanes/ |
| how results are ranked or packed | src/aizk/retrieval/rerank/, packing/ |
| the wording of a find response | src/aizk/retrieval/templates/find.md.j2 |
| a scheduled pass or its priority | src/aizk/background/jobs/, schedule.py |
| a setting or its default | src/aizk/config/settings.py |
| which services run | src/deploy/docker-compose.yml |
| a CLI command | src/aizk/commands/ |
| a Python dependency | pyproject.toml, then refresh uv.lock |
What it is built on
Section titled “What it is built on”AIZK uses patos for typed base models and SQL field helpers. rls, distributed as
rlsalchemy, owns the generic row level security machinery. AIZK registers its tables with it and
keeps only the scope lattice locally. OpenTelemetry records graph and Find spans without a separate
hardware profiling dependency.
Python dependencies live in pyproject.toml and exact resolved versions live in uv.lock.
The development setup explains the one pinned SQLAlchemy fork installation step.
- Design principles explains why the tree is shaped this way.
- Development setup gets a working environment.
- The data model is the right first stop inside
store/.