Elasticsearch / Intermediate: models, pipelines and relevance evidence / Lesson 8
Objects, nested records and denormalization
Preserve relationships inside document arrays and choose a search model with understood update costs.
On this page
Lesson goals
- Explain cross-object matches in ordinary object arrays.
- Use nested scope and evaluate denormalization against joins.
An array contains pairs, not independent bags
An artifact can have conservation assignments: curator A is reviewer and curator B is restorer. The business asks whether curator A restored the artifact. If an ordinary object array is flattened into multi-valued curator and role fields, independent conditions can match A from one object and restorer from another. The source JSON still looks paired, which makes the defect easy to miss in manual inspection. Specify whether conditions must hold on the same array element. A database-style expectation of row association does not automatically survive a document indexβs ordinary object mapping.
Keep a deliberate cross-match fixture
The synthetic document intentionally contains no assignment where curator-a is a restorer. A correct same-assignment query must reject it for that question while accepting a query for curator-b as restorer. This negative fixture is more informative than two assignments whose curator and role happen to satisfy every combination. Preserve source identity and exact controlled role values. Case normalization and analysis should not create another ambiguity in this relationship; keyword fields make the equality contract explicit. The example is source data only and does not establish its mapping.
{"artifact_id":"art-101","assignments":[{"curator":"curator-a","role":"reviewer"},{"curator":"curator-b","role":"restorer"}]}No single assignment contains curator-a and restorer together. An ordinary object mapping can lose that association.
Enter one nested scope for both conditions
Map assignments as nested when each object must remain independently searchable, with curator and role as keyword subfields. The nested query places both term conditions inside one nested scope. Two separate nested queries can match different child objects, so their apparent similarity is not enough. inner_hits can show which nested records matched, but returning many of them increases response work. Nested objects are represented by additional internal documents and carry indexing and query costs. They are a semantic tool for genuine same-object requirements, not a default representation for every arbitrary metadata key.
{"query":{"nested":{"path":"assignments","query":{"bool":{"filter":[{"term":{"assignments.curator":"curator-a"}},{"term":{"assignments.role":"restorer"}}]}},"score_mode":"none"}}}Assuming the stated nested mapping, this query does not match the fixture because no same nested record satisfies both filters.
Denormalization trades read convenience for propagation
Embedding a department label in every artifact avoids a search-time join, but renaming the department requires updating every affected search document. Keep the authoritative department identity alongside the copied label and define propagation lag. A partial update of one parent document can still require reindexing its underlying Lucene representation; do not infer relational row-update costs from the small JSON patch. Parent-child joins provide another model with routing and performance constraints, and Elastic recommends careful use rather than treating them as a relational join engine. Choose a model from query frequency, update frequency and consistency needs.
Nested counts have a different unit
A nested aggregation counts nested records inside its scope, not automatically distinct root artifacts. reverse_nested can return to the parent context for suitable summaries. If one artifact has three matching assignments, a dashboard can show three assignments but only one artifact. Label those metrics and reconcile both against a tiny fixture. Arbitrary key-value metadata may fit flattened fields when same-object rich querying is unnecessary, avoiding an unbounded nested population. Every choice still needs limits on source size, array length and user-controlled query breadth. Semantic correctness and resource safety should be reviewed together.
Common errors and fixes
A curator appears to have the wrong role
- Why it happens
- Conditions matched values from different ordinary object-array elements.
- How to fix it
- Use a reviewed nested model and place same-record conditions in one scope.
- Prevent it next time
- Keep a cross-match negative fixture in model validation.
Two nested clauses still cross-match
- Why it happens
- Each clause independently selected a different nested child.
- How to fix it
- Combine the conditions inside one nested query when one child must satisfy both.
- Prevent it next time
- Review scope boundaries explicitly rather than just field names.
A dashboard overcounts artifacts
- Why it happens
- Nested record counts were labelled as root-document counts.
- How to fix it
- Return to parent scope where needed and label both units.
- Prevent it next time
- Test one parent with several matching children.
Check your understanding
1. Should curator-a plus restorer match the source fixture?
Show worked solution
No under same-assignment semantics. curator-a is reviewer and curator-b is restorer. Map assignments as nested and combine both filters inside one nested scope. An ordinary object mapping can return a false cross-element match despite the paired appearance of _source.
2. What must a denormalized department label design say about a rename?
Show worked solution
It must name the authoritative identity, affected-document discovery, update propagation and acceptable lag. Retain reconciliation evidence so stale labels can be repaired. Avoid promising immediate consistency across many documents from a single label change in the source system.
Key takeaways
- Object-array association requires an explicit search model.
- Document modeling includes update propagation and counting units.