← Back to Docs

Search & Query Reference

Polylogue uses a query-first grammar over archive units. Bare tokens are full-text terms, field clauses narrow the selected unit, explicit sessions/messages/actions/blocks/files/assertions/runs/observed-events/context-snapshots/delegations where ... forms opt into Boolean predicates, and trailing CLI verbs render or mutate the selected session set. The same query semantics — filters, retrieval lanes, ranking policy, and typed response payloads — apply across the CLI, MCP, Python API, and daemon HTTP surfaces.

Quick links:

• Retrieval Lanes — dialogue, actions, hybrid, semantic, and the default lexical auto lane.
• Terminal Unit Queries — messages/actions/blocks/ files/assertions/runs/observed-events/context-snapshots/delegations where ... row results.
• Ranking Policy — current mixed-bm25-rrf-vector policy and version contract.
• SearchEnvelope Contract — the typed response shape shared across surfaces (#1266).
• FTS5 Syntax — boolean, phrase, and prefix queries.
• Searchable Content Coverage — exactly which block fields FTS indexes, and which (e.g. Write/Edit tool file bodies) it does not.
• Facets (Scoped vs Global) — aggregate counts with both views (#1269).

Grammar

The canonical parser is the Lark grammar in polylogue/archive/query/expression.py. It has two entry shapes that lower through the same typed AST and query planner:

compact-query      ::= compact-clause*
boolean-query      ::= ["sessions" "where"] predicate
unit-query         ::= ("messages" | "actions" | "blocks" | "files" | "assertions" | "runs" | "observed-events" | "context-snapshots" | "delegations") "where" predicate
pipeline-query     ::= unit-query ("|" pipeline-stage)+
                     | "sessions" "where" predicate "|" unit-query ("|" pipeline-stage)*

compact-clause     ::= field-clause
                     | quoted-text | bare-text
                     | "-" quoted-text | "-" bare-text
                     | count-comparison | count-range
                     | date-comparison | date-range

predicate          ::= predicate "OR" predicate
                     | predicate "AND" predicate
                     | "NOT" predicate
                     | "(" predicate ")"
                     | field-clause
                     | count-comparison | count-range
                     | date-comparison | date-range
                     | fts-leaf | semantic-leaf
                     | "exists" structural-unit "(" predicate ")"
                     | "seq" "(" sequence-step "->" sequence-step+ ")"

structural-unit    ::= "message" | "action" | "block" | "file" | "assertion"
sequence-step      ::= action-field-clause ("AND" action-field-clause)*
action-field-clause ::= "action:" value | "tool:" value | "command:" value
                      | "path:" value | "output:" value | "text:" value
pipeline-stage     ::= "sort" "by" "time" ["asc" | "desc"]
                     | "group" "by" field
                     | "count"
                     | "sort" "by" ("count" | "key") ["asc" | "desc"]
                     | "limit" integer
                     | "offset" integer

Compact queries select sessions:

polylogue repo:polylogue tag:active -"stale plan" "query envelope"
polylogue origin:(codex-session|claude-code-session) messages:>=10 words:<=2000
polylogue session:codex-session:abc123
polylogue near:id:codex-session:abc123

Explicit Boolean session queries also select sessions:

polylogue sessions where '(repo:polylogue OR repo:sinex) AND NOT tag:stale'
polylogue sessions where 'session:codex-session:abc123'
polylogue sessions where 'exists message(role:assistant AND text:timeout)'
polylogue sessions where 'seq(action:file_edit -> action:shell)'
polylogue sessions where 'seq(action:file_edit -> action:shell AND output:failed -> action:file_edit)'

Sequence predicates match an ordered subsequence of action rows inside a session. Each step is an action-unit predicate, so the sequence can describe real coding workflows instead of only action categories:

polylogue sessions where 'seq(action:file_edit -> action:shell AND output:failed -> action:file_edit)'
polylogue sessions where 'seq(tool:edit AND path:polylogue/archive -> tool:bash AND command:pytest)'

The SQL lowerer applies every step predicate to its own ordered action row and then enforces message/block order between steps. Unsupported fields in a sequence step fail typed as action-predicate errors; they do not broaden into a plain text search.

Unit queries select terminal rows instead of sessions:

polylogue messages where 'role:assistant AND text:timeout'
polylogue actions where 'session.repo:polylogue AND action:file_edit AND path:polylogue/archive'
polylogue blocks where 'type:code AND text:sqlite'
polylogue files where 'action:file_edit AND path:polylogue/archive/query'
polylogue assertions where 'kind:decision AND status:active AND text:review'
polylogue runs where 'role:subagent AND status:completed AND agent:Explore'
polylogue observed-events where 'delivery_state:acted_on AND text:#2100'
polylogue context-snapshots where 'boundary:session_start AND session.repo:polylogue'
polylogue delegations where 'mapping_state:resolved AND instruction:review'

Pipeline stages can either decorate a direct terminal row query or scope one through a session source stage:

polylogue 'messages where role:assistant | sort by time desc | limit 10'
polylogue 'actions where action:file_edit | limit 20 | offset 40'
polylogue 'sessions where repo:polylogue AND origin:claude-code-session | messages where role:assistant'
polylogue 'sessions where origin:(codex-session|claude-code-session) | actions where action:file_edit'
polylogue 'sessions where repo:polylogue | files where action:file_edit | group by path | count'
polylogue 'sessions where ~"timeout failure" | messages where role:user'
polylogue 'sessions where exists block(type:code) | messages where role:user'
polylogue 'sessions where seq(action:file_edit -> action:shell) | messages where role:user'
polylogue 'sessions where lineage:id:chatgpt-export:ext-child | messages where role:user'
polylogue 'sessions where repo:polylogue | messages where role:assistant | limit 10 | offset 20'
polylogue 'sessions where repo:polylogue | messages where role:assistant | sort by time desc | limit 10'

When a pipeline starts with sessions where ..., the left stage is lowered into predicates on the terminal unit query, so it uses the same row executor as direct messages/actions/files/... pipelines. Field/count/date predicates become session.<field> row predicates. FTS (~"..."), exists ..., sequence (seq(...)), and lineage:id:<session-id> stages lower through the existing session Boolean lowerers against the terminal row's owning session. Semantic vector predicates still reject in a session pipeline stage until terminal row queries have explicit ranked-result semantics. Terminal pipeline stages currently support sort by time [asc|desc], group by FIELD | count, aggregate sort by count|key [asc|desc], limit N, and offset N for SQL-backed terminal rows (messages, actions, blocks, files, assertions). Query-string limits narrow the surface limit instead of expanding caller/API caps, and query-string offsets are added to the caller offset. runs, observed-events, context-snapshots, and delegations are SQL-backed terminal rows over source-derived archive relations plus any non-duplicate materialized projection enrichments. Main runs, session_started events, tool-finished events, and session-start context snapshots are computed directly from sessions and blocks; richer digest-derived rows can still come from session_runs, session_observed_events, and session_context_snapshots. These units support sort by time, limit, and offset and are usable as exists run(...) / exists observed-event(...) / exists context-snapshot(...) session selectors. They do not yet expose group by | count aggregation (no aggregate lowerer), so aggregate stages on these units fail with a typed error rather than broadening. Pipeline syntax is terminal-row syntax: session-selector surfaces reject piped queries, including aggregate stages like group by role | count, instead of dropping those stages and widening the query to exists message(...).

--explain --format json reports terminal pipelines as a unit_source AST with ordered pipeline_stages. Session-scoped pipelines include a session_scope stage carrying the original session predicate, followed by any terminal sort/limit/offset stages. Consumers should use that stage list rather than inferring pipeline behavior from incidental limit, offset, or sort fields.

Unsupported forms raise typed ExpressionCompileErrors and must not broaden into looser full-text search. In particular, reserved unit prefixes such as messages where are errors when malformed; they are not treated as ordinary text terms.

DSL Fields

Field values support quoted strings and in-field alternatives:

polylogue 'origin:(codex-session|claude-code-session) title:"query DSL"'
polylogue 'tool:bash AND NOT tag:stale'

Negation is supported for fields that are semantically safe to negate, such as origin, tag, tool, and action.

Comparisons And Ranges

Readable comparisons are supported for message counts, word counts, numeric durations/tokens, and dates:

polylogue 'messages >= 5 AND messages <= 20'
polylogue 'words between 100 and 500'
polylogue 'sessions where duration_ms >= 60000'
polylogue 'messages where input_tokens >= 1000 AND duration_ms between 2000 and 3000'
polylogue 'date between 2026-06-01 and 2026-06-17'

Explicit Boolean queries can also compare the session aggregate columns already maintained by the archive:

polylogue 'sessions where user_messages >= 2 AND assistant_words between 500 and 2000'
polylogue 'sessions where authored_user_messages >= 2 AND authored_user_words >= 100'
polylogue 'sessions where system_messages = 0 AND tool_messages = 0'
polylogue 'sessions where tool_use_messages >= 1 AND paste_messages = 0'

Supported aggregate count fields are messages, words, user_messages, authored_user_messages, assistant_messages, system_messages, tool_messages, user_words, authored_user_words, assistant_words, tool_use_messages, thinking_messages, and paste_messages. user_messages and user_words are provider-role counts: they preserve source envelope truth and may include runtime protocol/context rows from providers such as Claude Code. Use authored_user_messages and authored_user_words when the question is human-authored prompt material. The tool_use_messages, thinking_messages, and paste_messages fields count messages carrying those signals, not raw block rows. These fields are SQL-backed session predicates; use them in sessions where ... Boolean queries or as session.<field> predicates inside terminal unit queries.

Supported non-count numeric fields are duration_ms on sessions and messages, plus message-scoped input_tokens, output_tokens, cache_read_tokens, and cache_write_tokens. Session duration_ms uses the archive's reported session duration; message duration_ms uses the stored per-message duration. Message numeric fields are accepted in messages where ... terminal queries and exists message(...) structural predicates.

Compact count syntax is equivalent where available:

polylogue messages:>=5 words:<=500

Structural Predicates

exists <unit>(...) keeps the selected unit as sessions but requires at least one child row matching the nested predicate.

Structural units also accept session.<field> predicates for the owning session fields that their lowerer can evaluate. SQL-backed units can use the full session filter surface, including action/tool/path/feature predicates. runs, observed-events, context-snapshots, and delegations are SQL-backed over the materialized run-projection tables, so they too accept the full session filter surface (e.g. session.repo, session.tool, session.path) alongside their own fields above. Count and date session fields accept compact comparison prefixes such as session.messages:>=2, session.words:<=500, and session.date:>=2026-01-02. This lets a unit query carry its session scope inline instead of splitting selection between the query string and parallel parameters.

Boolean and terminal predicates preserve comparison semantics: >, >=, <, <=, and = are distinct for count and numeric fields, while date and terminal time accept the four ordered comparisons plus between. The legacy flat root filters (messages > 10, words < 500) still compile into inclusive SessionQuerySpec min/max bounds because that older request object has no exclusive-bound fields.

Examples:

polylogue sessions where 'exists action(session.repo:polylogue AND tool:bash AND text:pytest)'
polylogue sessions where 'exists file(session.repo:polylogue AND action:file_edit AND path:archive/query)'
polylogue sessions where 'exists block(type:code AND text:timeout)'

FTS And Semantic Leaves

Inside Boolean predicates, ~ marks an explicit FTS leaf and semantic: / near:text: mark semantic-vector leaves:

polylogue sessions where '~"null pointer" AND repo:polylogue'
polylogue sessions where 'semantic:"query compiler failure"'
polylogue sessions where 'near:text:timeout'

Semantic leaves require embeddings to be configured and available. When vectors are unavailable, the query fails with a typed semantic/vector availability error instead of falling back to broad lexical search.

Terminal Unit Queries

Most query expressions select sessions. Explicit unit sources select terminal rows instead:

polylogue --format json messages where role:assistant AND text:timeout
polylogue --format ndjson actions where session.repo:polylogue AND action:file_edit AND path:polylogue/archive
polylogue --format yaml blocks where type:code AND text:sqlite
polylogue --format json files where action:file_edit AND path:polylogue/archive/query
polylogue --format json assertions where kind:decision AND status:active AND text:review
polylogue --format json runs where role:subagent AND status:completed AND agent:Explore
polylogue --format json observed-events where delivery_state:acted_on AND text:#2100
polylogue --format json context-snapshots where boundary:session_start AND session.repo:polylogue
polylogue --format json delegations where mapping_state:resolved AND instruction:review
polylogue --format json messages where text:timeout | group by role | count

The row shape is the shared QueryUnitEnvelope used by CLI JSON/NDJSON/YAML, Python Polylogue.query_units(), MCP query_units, and daemon GET /api/query-units?expression=.... Plain and CSV CLI output are transport-specific renderings of the same message/action/block/file/assertion/run/ observed-event/context-snapshot row payloads. Aggregate pipelines over SQL-backed terminal rows use the sibling QueryUnitAggregateEnvelope and currently support group by FIELD | count over closed unit fields such as message role, action tool/action, block type, file path, assertion kind/status, and owning-session origin/repo. Aggregate rows can be ordered with sort by count [asc|desc] or sort by key [asc|desc] before limit/offset; the default order is count descending, then key. Those surfaces share the same session-scoping filters for the row source where applicable, such as origin, tag, repo, title, date bounds, message-type and tool/paste/thinking feature filters.

Use session.<field> inside the expression when the unit rows should be scoped by their owning session:

polylogue messages where session.origin:claude-code-session AND role:assistant
polylogue actions where session.repo:polylogue AND action:file_edit
polylogue blocks where session.since:7d AND session.words:<=500 AND type:code
polylogue files where session.repo:polylogue AND action:file_edit AND path:archive/query
polylogue assertions where session.repo:polylogue AND kind:caveat
polylogue runs where session.repo:polylogue AND role:subagent AND status:completed
polylogue observed-events where session.origin:codex-session AND object_ref:github-review
polylogue context-snapshots where session.messages:>=2 AND session.date:>=2026-01-02 AND boundary:subagent_start

files where ... is an affected-path evidence projection over action rows with non-empty tool_path. It returns one row per owning session and normalized path, with action counts and first/last tool-use refs. It is not a global filesystem inventory and does not claim that a file still exists on disk.

runs, observed-events, context-snapshots, and delegations are SQL-backed row sources over source-derived archive relations, with materialized projection tables used only for richer non-duplicate rows that are not cheap local projections of sessions and blocks. They are both terminal unit sources (runs where ...) and exists run(...) / exists observed-event(...) / exists context-snapshot(...) session selectors, and they accept the full SQL-backed session filter surface (session.action, session.tool, session.path, session.has, …). They support sort by time, limit, and offset; they do not yet expose group by | count aggregation, so aggregate stages on these units fail with a typed error rather than broadening.

Session filters such as --origin, --tag, --repo, --since, and --until still narrow the owning sessions before rows are returned. Session-only actions and result-shaping modes that do not have row semantics yet, such as delete, open, stats, count, --cursor, and custom sort/reverse modes, are rejected instead of silently coercing row queries back to session queries.

Assertion Candidate Judgment

Any non-user-authored assertion write (agent, transform, detector, blackboard post, or any future automated writer) lands with status:candidate by default -- enforced once, inside upsert_assertion itself, not per-writer (37t.15). Two narrow exceptions: an assertion_id that already carries a terminal judgment outcome (accepted/rejected/deferred/superseded/deleted) keeps that outcome instead of being resurrected to candidate, and session tags (AssertionKind.TAG) opt out entirely via require_promotion=False since tags are categorization, not epistemic claims, and have no judgment-queue path today. Candidate rows are private, carry context_policy.inject=false, and keep promotion_required=true until an operator makes an explicit judgment. They can be inspected like any other assertion row, or through the review list that keeps pending candidates separate from already judged lifecycle rows:

polylogue assertions where 'status:candidate AND target:session:codex-session:abc123' --format json
polylogue mark candidates list --target-ref session:codex-session:abc123 --format json
polylogue mark candidates review --target-ref session:codex-session:abc123 --format json

The judgment workflow lives under polylogue mark candidates and writes back into the same user.db assertion substrate:

polylogue mark candidates accept assertion:candidate-id --reason "confirmed by transcript" --format json
polylogue mark candidates reject assertion:candidate-id --reason "unsupported by evidence" --format json
polylogue mark candidates defer assertion:candidate-id --reason "needs another source" --format json
polylogue mark candidates supersede assertion:candidate-id --kind summary --body "replacement claim" --format json

The closed assertion status vocabulary is active, candidate, accepted, rejected, deferred, superseded, deleted, and inactive; unknown values fail at the typed boundary instead of being rendered as generic strings. accept and supersede create an active assertion whose evidence includes the candidate assertion ref and whose supersedes lineage points at the candidate. reject writes status:rejected, and defer writes status:deferred; both preserve a judgment assertion with the reason and leave a durable lifecycle record. Judged candidates remain in mark candidates review with disabled action reasons such as candidate_already_accepted, candidate_already_rejected, candidate_deferred, or candidate_superseded, while mark candidates list only shows candidates still awaiting judgment. No candidate assertion is injected into compiled context unless a later surface asks for candidates explicitly.

Pending candidate judgments also appear in the operator debt cockpit:

polylogue ops debt list --kind assertion-candidate --only-actionable --format json

Public Ref Resolution

Query/read payloads carry public object and evidence refs so agents and the web shell can jump from a row, context image, or assertion back to the exact archive object it cites. Resolve refs through the shared resolver rather than turning them into broad text search:

polylogue read session:codex-session:abc123 --format json
polylogue read message:codex-session:abc123:m1 --format json
polylogue read block:codex-session:abc123:m1:0 --format json
polylogue read assertion:assertion-id --format json
polylogue read delegation:claude-code-session:parent:message:0 --format json

The same PublicRefResolutionPayload is exposed by Polylogue.resolve_ref(), MCP resolve_ref, and daemon GET /api/refs/resolve?ref=.... The resolver supports session, message, block, assertion, delegation, and runtime projection refs (run, observed-event, context-snapshot) when the addressed object exists. Unsupported or missing refs return a bounded unresolved payload with caveats; they never widen into a session search.

Delegation refs resolve to an explicit evidence card. The card preserves the complete recorded instruction while separately bounding the parent-side dispatch result, actual child-session excerpt, parent context, and parent follow-up with per-window/per-excerpt truncation markers. Ordinary delegations where ... rows expose only previews, SHA-256 hashes, structural mapping/result states, and evidence refs; they do not claim child success, utility, or parent use.

Outbound OTel Projection

Polylogue has two telemetry directions that should not be confused:

• inbound OTLP receiver routes store external telemetry in ops.db for local correlation;
• outbound OTel projection exports Polylogue archive evidence as an observability-shaped view.

Outbound projection is not archive authority. Trace/span ids are stable export ids, while Polylogue refs remain the navigation surface back to canonical sessions, messages, runs, context snapshots, observed events, assertions, and evidence refs.

The Python API exposes the first bounded projection surface:

payload = await archive.export_otel(
    source_ref="session:codex-session:abc123",
    expressions=(
        "runs where session.id:codex-session:abc123",
        "observed-events where session.id:codex-session:abc123",
        "context-snapshots where session.id:codex-session:abc123",
    ),
)

OtelProjectionPayload currently emits OTLP-like JSON (format="otlp-json") over existing query-unit row payloads. Runs and actions become spans; messages, observed events, and context snapshots become log/event records. Tool outputs and absolute local paths are omitted by default; the payload instead carries output length/presence, redaction flags, and refs that clients can resolve deliberately when the operator wants to inspect source evidence.

Filters

Identity and content

Content type

Message stats

Semantic actions

Time and scope

Tags

Retrieval

Verbs

Verbs determine the action applied to the matched session set.

Retrieval Lanes

Lane selection lives on the query as retrieval_lane. The resolved value appears in the SearchEnvelope.retrieval_lane field on every response so consumers can tell what actually ran (which matters because auto resolves to a concrete lane — see below).

Implementation: polylogue/storage/search_providers/fts5.py, polylogue/storage/search_providers/hybrid.py, polylogue/storage/search_providers/sqlite_vec_support.py.

Lane Semantics

dialogue (FTS5 lexical)

• Backed by SQLite FTS5's BM25 implementation against messages_fts.
• Tokenizer is unicode61 remove_diacritics 2 — case-insensitive for ASCII, Unicode-aware tokenization, and folds ordinary combining-mark diacritics (ó->o, ż->z, ą->a, ...) so a plain-ASCII query finds accented content and vice versa. Porter stemming is not available in this SQLite build, so refactor and refactoring are distinct tokens. Use prefix queries (refactor*) when you want morphological breadth.
• ł/Ł (Latin L with stroke) has no Unicode decomposition, so remove_diacritics cannot reach it. pl_fold (polylogue/storage/fts/pl_fold.py) closes that specific gap: it folds ł/Ł into indexed text before FTS insertion and into MATCH query text in escape_fts5_query, so latwo/zrobilem finds seeded łatwo/zrobiłem (polylogue-9jsi). The same tokenizer and fold apply to the threads_fts and session_work_events_fts insight-search surfaces, including full rebuild, missing-row repair, and dangling-row repair paths. A trigram fallback lane for further recall (beyond word-boundary tokens) is deliberately not part of this fold — see polylogue-xul7 (tracked follow-up) for a measured, benchmarked trigram lane before any such lane ships or is defaulted on.
• Raw score is BM25: lower is better in SQLite FTS5, values are typically negative, and they are not comparable across queries.
• Match evidence: matched_terms, snippet, match_surface="message", message_id, and target_ref point at the hit message.

actions (FTS5 over action blocks)

• Same FTS5 mechanics as dialogue, but the query is restricted to tool_use and tool_result blocks inside messages_fts. The normalized actions view remains the structured action surface for filters and analytics.
• Current public action-lane hits preserve rank and action match surface but do not expose the underlying action FTS BM25 score in the shared SearchEnvelope; consumers should treat score_kind=null as the contract for action-only hits until the action evidence path is widened.
• Useful when you remember an action ("the session where I edited connection_profile.py") rather than its prose.

hybrid (RRF fusion)

• Runs both dialogue (FTS5) and semantic (vector) lanes, then fuses with Reciprocal Rank Fusion at k=60: fused_score = Σ 1 / (k + rank_in_lane).
• Tie-breaking is deterministic: descending fused score, then ascending session_id. This makes cursor and offset pagination stable across runs even when scores tie.
• Reported score_kind is "rrf". Higher fused scores indicate stronger cross-lane consensus.
• Lane contributions (per-lane rank and per-lane RRF contribution) are preserved end-to-end on each hit's score_components (#1267): every lane that contributed adds a <lane>_rank (1-based rank within that lane) and a matching <lane>_rrf (the 1 / (k + rank) contribution that was summed into the fused score). Lane names are text (FTS5 dialogue), action (FTS5 action blocks), and vector (semantic). A hit that appeared only in the lexical lane carries {text_rank, text_rrf} and nothing else; a hit that survived both lanes carries the full (text|action|vector)_(rank|rrf) set, so consumers can show "ranked high in both lexical and semantic lanes" without re-running the search.

semantic (vector-only)

• Pure k-nearest-neighbor over Voyage-4 1024-dim embeddings via sqlite-vec's vec0 virtual table.
• Triggered by --similar <text> or --semantic (which promotes the positional query string into similar_text; no FTS leg runs).
• Score kind is "vector_distance" — lower means closer in embedding space. Like BM25, distances are not directly comparable across different query embeddings.
• Requires embeddings to be enabled and populated; see docs/architecture.md § Embedding Pipeline.
• Use polylogue ops embed status to check whether vector retrieval is disabled, missing an API key, pending backlog catch-up, partially usable, or complete. polylogue ops embed status --detail performs exact pending-message and retrieval-band accounting; the default status path stays cheap and reports the latest persisted catch-up run.

Embedding Activation And Catch-Up

Semantic search stays unavailable until embeddings are both enabled and materialized. The activation path is deliberately bounded:

1. polylogue ops embed status shows config state, key presence, coverage, configured model/dimension, monthly cost cap, backlog, latest catch-up progress, and next_action (code, reason, command) for automation.
2. polylogue ops embed preflight --max-sessions 10 estimates the next bounded window without contacting Voyage. Use --format json for the scriptable form: it reports the exact window, pricing assumptions, effective cost cap, and a ready-to-run backfill_args list for the same bounded catch-up slice.
3. polylogue ops embed enable --yes enables the daemon stage when a Voyage key is already configured, or polylogue ops embed enable --voyage-api-key ... records the key and enables the stage.
4. polylogue ops embed backfill --max-sessions 10 runs an explicit bounded catch-up batch; after enablement, polylogued also processes bounded daemon batches for new or stale sessions.

--max-messages is a hard message-count window and uses live message counts rather than potentially stale session_stats. --max-sessions may use materialized session stats so small first batches remain fast on large archives.

MCP clients should use embedding_status for readiness/next-action state and embedding_preflight for the same no-provider-call cost window. Both tools are read-only and return the canonical JSON payloads used by the CLI, so agents do not need to scrape terminal output before deciding whether semantic search is actually usable.

Python API clients use the same contracts through Polylogue.embedding_status(detail=False) and Polylogue.embedding_preflight(...).

Auto Lane

When retrieval_lane=auto (the default), ordinary text queries run on the lexical dialogue lane. The planner does not inspect embeddings.db before a default search, so keyword lookup stays fast and predictable on large archives.

The resolved lane is always echoed back in SearchEnvelope.retrieval_lane so callers do not have to re-derive what ran.

Two ergonomic overrides on the root CLI surface (#1217):

• --lexical — force the FTS-only lane; useful when you want deterministic keyword matches regardless of embedding state.
• --semantic — promote the query string into a vector-only similarity probe; no FTS leg, no boolean operators applied.

Ranking Policy

Every SearchEnvelope declares its ranking_policy and ranking_policy_version. The current policy identifier is mixed-bm25-rrf-vector (version 1):

• dialogue orders hits by FTS5 BM25 (lower is better; raw scores are usually negative).
• actions orders through the action FTS read model, but the public action-lane hit payload does not currently expose a numeric action score.
• hybrid fuses dialogue + action + semantic lanes with RRF at k=60 and orders by fused score, breaking ties on (−fused_score, session_id).
• semantic orders by ascending vector distance.

Consumers should pin the ranking_policy_version they validate against and treat any change as a contract event. The version is intentionally exposed so external pipelines can detect ordering shifts without diffing raw scores. See docs/openapi/search.yaml (x-polylogue-ranking-policy) for the machine-readable declaration.

SearchEnvelope Contract

All ranked surfaces (CLI --format json, MCP search/list_sessions, daemon GET /api/sessions?query=…, Python API) return the typed SearchEnvelope defined in polylogue/surfaces/payloads.py and emitted via the daemon under docs/openapi/search.yaml (#1266).

Each hit's match (a SessionSearchMatchPayload) carries:

Score-kind cheatsheet

• bm25 — lower = better match in SQLite FTS5. Values are typically negative. Never display raw BM25 as a percent or compare across queries; rank position is the durable signal.
• rrf — higher = better; bounded by Σ 1/(k+1) over contributing lanes. Per-lane decomposition lives in score_components as <lane>_rank / <lane>_rrf pairs, so consumers can explain "this hit appeared in both lexical and semantic lanes" without re-running the query (#1267).
• vector_distance — lower = closer in embedding space. Not comparable across different query embeddings.
• null — identity-bearing match (e.g. attachment identity lane); no numeric score, only rank.

Per-hit explanations (#1267)

Every ranked hit carries deterministic why-this-matched evidence on its match payload. The exact field set depends on the lane that produced the hit:

Tokenization for matched_terms strips FTS5 boolean operators (AND / OR / NOT / NEAR), quote/colon/paren punctuation, and trailing * prefix markers, then deduplicates case-insensitively. The result is the literal set of tokens a reader can expect to see highlighted in the snippet.

Hybrid score_components are the load-bearing surface: each contributing lane adds two entries that explain its share of the fused score. For example, a hit at lexical rank 1 and vector rank 2 carries:

"score_components": {
  "text_rank": 1.0,
  "text_rrf": 0.0163934426,
  "vector_rank": 2.0,
  "vector_rrf": 0.0161290323
},
"score": 0.0325224749

Consumers can read score_components directly to render a "matched in both lanes" badge or to debug ranking drift without re-running the search.

Pagination

For ranked queries, prefer next_cursor over offset. Cursor pagination encodes the rank tie-breaker ((rank, score, session_id, retrieval_lane)) and is stable under archive growth between page fetches. Offset pagination is supported for non-ranked list paths and as a best-effort fallback for ranked paths.

The cursor is an opaque URL-safe base64 token (a versioned JSON envelope; see :class:polylogue.surfaces.payloads.SearchCursor). Consumers MUST treat it as opaque and pass it back unchanged.

# Page 1
polylogue "sqlite" read --all --format json --limit 25
# Read .next_cursor from the response, then ask for page 2:
polylogue "sqlite" read --all --format json --limit 25 \
    --cursor "$NEXT_CURSOR"

MCP search and the daemon /api/sessions endpoint accept the same cursor parameter; the Python API exposes Polylogue.search_envelope( query, cursor=...). The cursor carries the retrieval lane it was minted in, so a dialogue cursor passed back to a hybrid request is rejected up-front rather than silently changing ranking policy mid-walk.

Stability guarantees (#1268):

• No duplicates: any hit returned on page N is filtered out on page N+1 even when new rows were ingested between requests.
• No gaps: any hit that sorts strictly after the anchor (under the lane's natural ordering: BM25 lower-is-better, RRF higher-is-better, vector distance lower-is-better) survives the cursor trim.
• Restart-stable: cursors are self-contained tokens with no server-side state; they survive daemon restart.

FTS5 Syntax

The messages_fts virtual table uses SQLite's FTS5 with the unicode61 tokenizer. Prefix queries use *:

polylogue "refactor*"

Phrase queries use quotes:

polylogue '"null pointer exception"'

Boolean operators combine terms:

polylogue "refactor AND schema NOT test"

Column filters restrict matches:

polylogue 'text:css {session_id claude-code}: refactor'

Searchable Content Coverage

blocks.search_text — the generated column FTS5 indexes — concatenates a fixed subset of block fields, not the whole block. Anything outside that subset is invisible to the ranked dialogue/actions/hybrid lanes, --contains, contains:, and bare-text queries, even though the underlying data is stored and readable through other surfaces (read, select, direct block/action reads). The live definition lives in polylogue/storage/sqlite/archive_tiers/index.py (blocks.search_text GENERATED ALWAYS AS (...)); this table is drift-checked against that expression by tests/unit/pipeline/test_search_text_coverage_contract.py.

Dedicated action-evidence lane: when you know an agent wrote or edited a specific string into a file body, use the query DSL's action text predicate. It searches the action's raw tool_input JSON (as well as normalized action fields and output), so it covers Write's content and Edit's old_string/new_string without putting those bodies in FTS:

polylogue 'actions where tool:write AND text:"needle"'
polylogue 'sessions where exists action(tool:edit AND text:"needle")'

This is an unindexed LIKE filter, not FTS; constrain it by action, path, session scope, or time on large archives. For direct index.db inspection, the equivalent JSON-aware SQL probe is:

SELECT block_id, session_id, tool_name,
       json_extract(tool_input, '$.file_path') AS file_path
FROM blocks
WHERE tool_name IN ('Write', 'Edit')
  AND (
    json_extract(tool_input, '$.content') LIKE '%needle%'
    OR json_extract(tool_input, '$.old_string') LIKE '%needle%'
    OR json_extract(tool_input, '$.new_string') LIKE '%needle%'
  );

Extending search_text itself to cover $.content was considered and deliberately deferred (see polylogue-013x) because Write bodies can be large enough to meaningfully bloat the FTS index, and that tradeoff needs a size probe against a live archive plus a derived-tier index rebuild before it is decided, not a silent schema bump.

Output Formats

Set format with -f / --format on a verb:

polylogue "sqlite locking" read --all --format json
polylogue --since yesterday read --all --format ndjson

Facets (Scoped vs Global)

Facets summarize the archive as aggregate counts. Polylogue exposes the same shape across daemon HTTP (GET /api/facets), MCP (facets), CLI (polylogue facets), and the Python API (Polylogue.facets); see #1269 (slice D of #873).

A facets response carries both views explicitly:

• scoped — counts rolled from the current query/filter set. Empty buckets if the filter chain narrows away every value.
• global — counts over the unfiltered archive. Always populated.
• scoped_to_query — true whenever any filter narrowed the view.
• complete_families / deferred_families / family_status — route-state metadata for each facet family. HTTP readers use this to distinguish an actually-empty bucket from a family intentionally not materialized in the current response.
• generated_at, stale, stale_age_s, budget_exceeded — freshness and budget metadata surfaced to the web workbench.
• idf — inverse-document-frequency per facet value, computed against the global universe. Higher = rarer = stronger signal; near zero = value appears in almost every session. Disable with --no-idf on the CLI.

Top-level fields (origins, tags, total_sessions etc.) mirror the active view (scoped when filtered, global otherwise) for backward compatibility with surfaces written before #1269. Consumers that need both views should read scoped and global directly.

Daemon HTTP keeps expensive archive-wide families lazy for workbench first paint. GET /api/facets includes origin/tag/count buckets and marks repos plus action_types as deferred_by_default. Clients that need the full set can call GET /api/facets?include_deferred=1&budget_ms=5000 or GET /api/facets?families=repos,action_types; a budget overrun returns the existing bucket fields plus budget_exceeded=true and per-family deferred reasons instead of blocking the page indefinitely. HTTP workbench requests attach request IDs and bounded AbortController timeouts; the daemon observes closed client sockets inside facet SQLite progress handlers so cancelled/timed-out browser requests can interrupt already-started scans.

polylogue facets                     # global only (no filters)
polylogue facets -o chatgpt-export   # scoped to ChatGPT exports + global side-by-side
polylogue facets -q "vector store"   # scoped to FTS hits
polylogue facets -f json --no-idf    # FacetsResponse, no IDF weighting

Empty Result Diagnostics

When a query returns no results:

1. Check origin spelling: polylogue --origin claude-code-session read --all (not claude_code)
2. Expand the time window: --since 2024-01 instead of --since yesterday
3. Verify the archive has data: polylogue analyze --count (no filters)
4. Check FTS index health: polylogued status shows fts_readiness
5. Run polylogue ops doctor for schema and index integrity
6. If using --similar, ensure embeddings are built (check polylogue ops embed status --detail for embedding readiness/coverage)
7. If you know an agent wrote or edited a specific string into a file (a Write/Edit tool body), FTS will not find it. Use the action-evidence query path documented in Searchable Content Coverage, such as actions where tool:write AND text:"needle".