Fusion and reranking

Ranking happens twice. PostgreSQL fuses three chunk rankings into one score inside the find statement, and Python reorders the whole candidate list afterward with a cross encoder. This page assumes you have read the lanes, since the first half is what SourceLane calls. The code is src/aizk/store/models/tables/chunk.py and src/aizk/retrieval/rerank/rescore.py.

  dense (distance under the floor) ─┐
  bm25  (tokenize, to_bm25query)   ─┼─▶ RRF, sum of 1 / (rrf_k + rank)
  title (named in query, longest)  ─┘             │
                                                  ▼
                        + promoted_bonus, + 1.0 when named
                                                  │
                                                  ▼
                 at most find_per_document per document, then k
                                                  │
                                                  ▼
                  cross encoder scores the first rerank_depth
                                                  │
                                                  ▼
           direct and unshadowed first, then score, then evidence id

Chunk.fused unions three rankings

Each ranking produces (id, document_id, rank) and each is cut at fusion_depth before the lanes merge. The dense and lexical rankings reach that cut through Chunk.ranking, which is what keeps each one an index walk instead of a scan.

Dense ranks embedding @ qvec ascending, guarded by embedding IS NOT NULL, distance < find_max_distance and Document.is_active(). The floor is what keeps an off-corpus question from returning its least bad match.

Lexical is BM25 through VectorChord. The query goes through to_bm25query('ix_chunk_bm25', tokenize(:qtext, 'aizk_bm25')) and ranks with the <&> operator against a bm25 column. That column and its index exist only in the migration and never on the model, which is why the code reaches for them with sqlalchemy.column("bm25"). Scores come back negative, so raw_rank < 0 filters out the rows that matched nothing.

Title is the exact identity ranking. Document.named_in_query() lowercases both the title and the qtext bind, replaces every non-alphanumeric run with a space, pads both with spaces and asks whether the padded title occurs inside the padded query, requiring at least three characters. That padding is what makes it a whole-token match rather than a substring accident. Its chunks rank by length(title) DESC first, so the most specific named title wins, then by chunk ord.

The three are unioned and grouped, and each contributes 1 / (rrf_k + rank) with rrf_k defaulting to 60. A chunk found by two rankings collects both votes. Fusing positions rather than scores is the point, since a cosine distance and a BM25 score are not comparable numbers.

Chunk.ranking keeps the chunk indexes in play

Ranking chunks joined to document looks harmless and is not. Under row security the planner gives up on both chunk indexes, loops over every visible chunk and sorts the lot, which on a production snapshot of 22,290 chunks cost 80 ms for the dense ranking and 245 ms for the lexical one, and meant the two largest indexes in the database, a 66 MB vchordrq and a 343 MB bm25, were maintained on every write and read by nothing.

So each of the two rankings runs over chunk alone inside a MATERIALIZED common table expression, and joins document outside it to drop expired sources. MATERIALIZED is load bearing, because without it the planner folds the window back into the join and rebuilds the same scan. Row security hides every chunk whose document the caller cannot read, so the window already sees exactly what the join would have, and the only predicate the join still owns is Document.is_active(). Because that join discards rows after the window is taken, the window reaches fusion_depth * fusion_overfetch deep and spends the difference as slack. The same snapshot carries an expiry on 5 of 1,168 documents with 1 expired, so at three deep the slack is never close to spent. Measured on that snapshot the dense ranking fell to 7.5 ms and the lexical one to 10 ms, both walking their index, and the fused statement went from 320 ms to 21 ms with byte-identical output over nine query and vector combinations.

An owned query keeps the join inside the ranking. Its Document.scopes = :qscopes predicate is selective, so the planner drives from the few matching documents and stays fast on its own, while a global window would spend its over-fetch on documents the share could never carry. On a scope holding 2 percent of the corpus the windowed shape returned 31 and 10 rows where the joined shape returned the full 50 in 4.7 ms and 10.8 ms.

Where this comes from

Fusing ranks instead of raw scores is Reciprocal Rank Fusion. The references map traces every mechanism back to its source.

Chunk.hybrid scores, caps and cuts

hybrid joins the fused CTE to document and builds one score.

source_score = (
    fused.c.rrf_score
    + case((promoted, bindparam("promoted_bonus", type_=Float)), else_=0.0)
    + case((Document.named_in_query(), literal(1.0)), else_=0.0)
)

promoted_bonus defaults to 0.01, which is a nudge, since a promoted document is evidence somebody already found worth keeping. The named-title bonus is a hard-coded 1.0 and is not configurable, which is deliberate because it is roughly two orders of magnitude larger than any RRF sum and so functions as a class rather than as a weight.

A row_number() partitioned by document_id and ordered by score then enforces find_per_document, three by default, so one long document cannot fill the whole lane. The survivors order by score and cut at k. The lane also projects named_in_query() as direct, which is the only place that flag is set.

merit_order reorders everything

merit_order takes the statement’s rows in their lane-priority order and scores the first rerank_depth of them, 50 by default, sending each candidate’s rendered line to RerankClient. Note that it scores against the raw query, not the speaker-rewritten string that went to the embedder. The scores are zipped with strict=True and kept in a dict keyed by evidence_id, which is exactly why Candidate carries that excluded field. trace() returns those same scores.

reordered sorts the scored candidates by a three-part key.

key = (
    -(candidate.direct and candidate.direct_title not in shadowed),
    -scores[candidate.evidence_id],
    candidate.evidence_id,
)

First comes the identity group, then merit inside it, then evidence_id so ties break exactly as the statement ordered them. Candidates past the scoring depth are not sorted at all. They keep the statement’s order and are appended after the scored block, so the reranker changes the head of the list and leaves the tail alone.

Title shadowing

_shadowed_titles returns every named title that is strictly contained in another named title.

Without it, a question naming the fictional document Atlas Migration Weekly Plan also directly names the document titled Atlas Migration, and both would land in the authoritative group. The broader document then competes on equal footing with the one the question actually asked for. Shadowing drops the contained title out of the identity group while leaving it in the ranking, so it can still win on merit. Two unrelated titles named in the same question shadow neither, since neither contains the other, and they stay peers.

The shadow test runs on Candidate.direct_title, which is the casefolded source_title and only exists when direct is true.

Next

• Budget packing turns this order into the response.
• Retrieval tuning has rrf_k, fusion_depth and rerank_depth.
• Retrieval results has what these choices measure.