What platform search indexes¶
Every entity is either mapped below or excluded with a reason. The mapping is the contract the indexer implements; nothing reflects over a model, so a column added later cannot reach the index by existing.
Entities¶
A field is worth 2 as a title and 1 as body, which is where the mapping's
judgement lives: email→title on user is the decision that finding
a person by address ranks with finding them by name. A warehouse table's
columns weigh 1, as body does.
| Entity | Fields | Chunking | Facets | Route |
|---|---|---|---|---|
asset_check |
name→title, expr→body, asset→body |
whole | - | /orchestration?asset={asset} |
asset_run (only state=failed) |
asset→title, error→body, logs→body |
split 2000/200 | status, verdict, created_at | /orchestration?run={parent} |
audit_event |
action→title, target_id→body, target_type→body |
whole | verdict | /settings/audit |
conversation |
title→title, summary→body |
whole | created_at | /conversations/{id} |
dashboard (only deleted_at=None) |
title→title, name→title |
whole | status | /dashboards/{id} |
data_source |
name→title, description→body, source_type→body |
whole | status | /warehouse/sources/{id} |
derivation (only deleted_at=None) |
name→title, question→body, source→body, narrative→body, assumptions_json→body (json), attestation_json→body (json) |
field | verdict, created_at, data_hash | /derivations/{id} |
feature_entity |
name→title, description→body |
whole | - | /features |
feature_view (only deleted_at=None) |
name→title, description→body, source→body, contract_json→body (json) |
whole | - | /features/{id} |
llm_profile |
name→title |
whole | - | /settings/models |
materialization_schedule |
name→title, selection→body, dataset→body |
whole | - | /orchestration |
message |
content→body |
split 2000/200, late | created_at | /conversations/{parent}?message={id} |
metric (only deleted_at=None) |
name→title, manifest_json→body (json) |
whole | created_at | /metrics?metric={id} |
metric_monitor |
name→title, target→body, method→body |
whole | created_at | /monitors?monitor={id} |
monitor_incident |
reason→body |
whole | incident_state, created_at | /monitors?incident={id} |
notebook (only deleted_at=None) |
name→title |
whole | created_at | /notebooks/{id} |
notebook_cell |
source→body, outputs_json→body (json) |
split 2000/200, late | - | /notebooks/{parent} |
notebook_folder (only deleted_at=None) |
name→title |
whole | - | /notebooks?folder={id} |
orchestration_run |
cause→title |
whole | status, created_at | /orchestration?run={id} |
saved_query (only deleted_at=None) |
name→title, sql→body |
whole | - | /explore?query={id} |
setting |
key→title |
whole | - | /settings |
training_set |
name→title, label→body |
whole | - | /features |
workflow |
name→title |
whole | - | /orchestration?workflow={id} |
Why each is chunked that way¶
late means each chunk is embedded in the context of its whole row rather
than on its own, so a fragment keeps what surrounded it. It needs a
SpanEmbedder; the static fallback embedder degrades to per-chunk.
asset_check: whole, a name and a short descriptionasset_run: split 2000/200, a failed run's log is long, and the error is anywhere in itaudit_event: whole, a name and a short descriptionconversation: whole, a name and a short descriptiondashboard: whole, a name and a short descriptiondata_source: whole, a name and a short descriptionderivation: field, prose, code and structured assumptions score separatelyfeature_entity: whole, a name and a short descriptionfeature_view: whole, a name and a short descriptionllm_profile: whole, a name and a short descriptionmaterialization_schedule: whole, a name and a short descriptionmessage: split 2000/200, late, a message can be arbitrarily long prosemetric: whole, a name and a short descriptionmetric_monitor: whole, a name and a short descriptionmonitor_incident: whole, a name and a short descriptionnotebook: whole, a name and a short descriptionnotebook_cell: split 2000/200, late, cell source and its printed output are both unboundednotebook_folder: whole, a name and a short descriptionorchestration_run: whole, a name and a short descriptionsaved_query: whole, a name and a short descriptionsetting: whole, a name and a short descriptiontraining_set: whole, a name and a short descriptionworkflow: whole, a name and a short description
Sources with no table behind them¶
| Entity | Read from | Title | Other text | Facets |
|---|---|---|---|---|
warehouse_table |
WarehouseService.tables() joined to WarehouseService.columns() | the warehouse table name | its column names, from the persisted catalog (IP-17) | created_at |
model |
the MLflow model registry, via ModelService.registry() | the registered model name | its description and the tags a person set | verdict, created_at |
Excluded, and why¶
| Table | Reason |
|---|---|
Budget |
a numeric limit, filtered on rather than searched |
ComputeUsage |
usage accounting, no searchable text |
DashboardSubscription |
a delivery rule with no searchable text |
DashboardVersion |
its label is only ever 'saved' or 'published', never written |
DataSource |
superseded by ExternalDataSource for connections |
DerivationRun |
a run of an indexed derivation |
ExternalDataSchema |
the warehouse_table document is assembled from it |
FeatureDriftRow |
computed over the data |
FeatureExpectationRow |
computed over the data |
FeatureStatisticsRow |
profiles and samples of the data |
InferenceEvent |
production request and prediction payloads |
JobRow |
its label is an indexed derivation's name, and nothing more |
MetricSnapshot |
a numeric observation, filtered on rather than searched |
MetricVersion |
a version of an indexed metric |
Notification |
a delivery of an event whose artifact is indexed |
NotificationPreference |
per-event-type switches, with no prose of their own |
OnlineFeatureRow |
the feature values themselves |
PromotedQuery |
promoting authors a derivation, and that gets indexed |
RetrainPolicy |
settings on an indexed model, with no prose of its own |
Secret |
holds an encrypted credential |
WarehouseColumn |
the warehouse_table document is assembled from it |
Size at a realistic corpus¶
A mature single-team project. These row counts and text lengths are stated assumptions, not measurements: argue with a number rather than with the conclusion.
| Entity | Rows | Chunks/row | Chunks | Assumption |
|---|---|---|---|---|
audit_event |
50,000 | 1 | 50,000 | the highest-count entity in the corpus |
derivation |
1,000 | 5 | 5,000 | question, source, narrative, two JSON |
message |
5,000 | 1 | 5,000 | a chat turn, occasionally much longer |
orchestration_run |
5,000 | 1 | 5,000 | one per materialization run |
notebook_cell |
2,400 | 1 | 2,400 | ~12 cells a notebook, source + outputs |
asset_run |
250 | 5 | 1,250 | 5% of ~5k runs fail; only those are indexed |
conversation |
500 | 1 | 500 | - |
saved_query |
300 | 1 | 300 | SQL text, short enough to stay whole |
notebook |
200 | 1 | 200 | - |
monitor_incident |
200 | 1 | 200 | - |
metric |
100 | 1 | 100 | - |
asset_check |
100 | 1 | 100 | - |
warehouse_table |
100 | 1 | 100 | columns are a field, not documents |
dashboard |
50 | 1 | 50 | - |
metric_monitor |
50 | 1 | 50 | - |
setting |
50 | 1 | 50 | - |
feature_view |
40 | 1 | 40 | - |
notebook_folder |
30 | 1 | 30 | - |
training_set |
30 | 1 | 30 | - |
materialization_schedule |
30 | 1 | 30 | - |
model |
30 | 1 | 30 | - |
feature_entity |
20 | 1 | 20 | - |
workflow |
20 | 1 | 20 | - |
data_source |
20 | 1 | 20 | - |
llm_profile |
5 | 1 | 5 | - |
| total | 70,525 |
At 384 dimensions and 4 bytes a float, 70,525 chunks is 108 MB of vectors.
An exact array_cosine_similarity scan over 200k documents at this width
measures 127 ms, and 14 ms once a filter narrows it. A corpus this
size is therefore answered by scanning a column: a dedicated vector store
would buy latency that is already there. The parent's no-vector-store
decision assumed as much, and this validates it.
One number decides it¶
audit_event is 50,000 of those 70,525 chunks, 71% of the
whole corpus, and the only entity with no natural ceiling. It is also the one
whose text is mostly identifiers rather than prose, so it gains least from
being embedded at all.
Excluding it, or indexing it lexically without a vector, leaves 20,525 chunks and 32 MB. That is the decision worth taking deliberately before the indexer is built, rather than discovering it when the audit log has grown.