Elasticsearch / Basic: document and query contracts / Lesson 6
Sorting, pagination and result contracts
Return stable catalog pages while controlling result shape, deep pagination and changing search views.
On this page
Lesson goals
- Build a total sort order using suitable field types.
- Explain search_after, point-in-time and total-hit semantics.
A page includes more than a size
A catalog page needs an eligibility query, selected response fields, a total ordering and a continuation rule. Sorting by year alone leaves ties, so a stable unique artifact_id keyword can complete the order for a fixed data set. Do not sort on analyzed title text when an exact keyword representation is intended. Sorting directly on _id has restrictions and is not a general substitute for a doc-values-enabled business identifier field. Explain missing-year placement and ascending or descending direction explicitly. A user changing filters must start a new continuation sequence rather than reusing an unrelated cursor.
Read the returned sort values
The body below requests two artifacts in year and identifier order. In a fixed synthetic population ordered as (1880, art-100), (1890, art-101), (1890, art-102), the first two rows end at (1890, art-101). Search hits include their sort values when explicit sorting is used. The application should preserve the server-returned values and their types rather than reconstructing a cursor from formatted display text. Source filtering limits the returned source fields, but does not establish field-level authorization or eliminate all internal document work.
{"size":2,"query":{"match_all":{}},"sort":[{"year":"asc"},{"artifact_id":"asc"}],"_source":["artifact_id","title","year"]}For the fixed three-document teaching order, the last returned sort array is [1890,"art-101"]. This is reasoned fixture behavior.
search_after continues the same order
search_after asks for hits following the supplied sort tuple. Keep the query and sort definition unchanged between requests. The second body continues after the synthetic last tuple and therefore reaches art-102 in the fixed fixture. Without a stable search view, concurrent insertions, updates or refreshes can still change the population. A point-in-time context can preserve a consistent search view across requests when supported and correctly maintained. Use the latest returned PIT identifier and include all required tie-breaker values; do not assume an example two-value cursor applies unchanged when PIT adds a shard-document tie-breaker.
{"size":2,"query":{"match_all":{}},"sort":[{"year":"asc"},{"artifact_id":"asc"}],"search_after":[1890,"art-101"],"_source":["artifact_id","title","year"]}Continuation for the fixed-data ordinary-search example, without PIT. It is not a snapshot-consistency guarantee under concurrent changes.
Deep offsets ask shards to retain more candidates
from plus size is convenient for shallow pages, but each shard may need to collect candidates for earlier offsets as well as the visible page. Raising a result-window limit does not remove this cost. Prefer a supported continuation pattern for long traversals and define whether the user actually needs arbitrary page numbers. PIT contexts also retain resources and need bounded keep-alive and cleanup, so they are not free permanent bookmarks. Scroll serves particular bulk traversal scenarios and should not be chosen merely because its name resembles UI scrolling. Match the mechanism to a clear consistency and interaction requirement.
Read the total-hit relation
A total-hit value can carry relation eq for an exact count or gte for a lower bound under the tracking policy. Displaying every returned value as an exact catalog count can mislead users. track_total_hits can request stronger counting behavior at additional cost where appropriate. Search responses can also report timed_out and failed shards; a nonempty hit list is not proof of complete results. Define whether the application accepts partial search or displays an explicit incomplete state. Keep continuation tokens bound to the callerโs allowed filters so a copied cursor cannot bypass access restrictions.
Common errors and fixes
Pages repeat artifacts with equal years
- Why it happens
- The sort lacks a stable unique tie-breaker.
- How to fix it
- Add a suitable keyword identity and preserve returned sort tuples.
- Prevent it next time
- Test multiple artifacts with identical primary sort values.
search_phase_execution_exception
- Why it happens
- A requested sort or result window conflicts with field behavior or limits.
- How to fix it
- Inspect the detailed cause and use mapped sort fields and an appropriate continuation.
- Prevent it next time
- Review pagination with realistic depth before release.
A lower-bound hit count is displayed as exact
- Why it happens
- The application ignored hits.total.relation.
- How to fix it
- Preserve the relation or request the justified exact-count policy.
- Prevent it next time
- Include partial, timed-out and lower-bound responses in UI fixtures.
Check your understanding
1. After sort tuple [1890,"art-101"], which fixture row follows in ascending order?
Show worked solution
The row (1890, art-102) follows because the year ties and the keyword identifier is greater. Preserve the exact tuple types. This answer assumes a fixed population; add a correctly managed PIT when the product requires a consistent view across concurrent refreshes.
2. Why is increasing the maximum result window not a complete deep-pagination strategy?
Show worked solution
It permits more retained candidates without removing per-shard collection and memory costs. Define the traversal requirement and evaluate search_after with a suitable consistency context. Keep a resource lifetime budget and handle partial responses rather than equating a larger limit with a scalable user experience.
Key takeaways
- Stable pages require a total order and a defined search view.
- Counts and partial-result indicators belong in the result contract.