Elasticsearch / Basic: document and query contracts / Lesson 1

Documents, indices and search visibility

Place a museum search request inside Elasticsearch and distinguish indexing acknowledgment from search visibility.

About 4 min readContent reviewed 2026-10-28

On this page

Lesson goals

Explain document identity, index boundaries and shards.
Distinguish refresh, durability and real-time retrieval.

A search index is a derived application model

A museum catalog may keep authoritative acquisition records in a transactional system while Elasticsearch serves searchable artifact descriptions. A document is a JSON representation optimized for that search task, not automatically a complete replacement for the system of record. Decide how the search copy is built, how updates arrive and which lag the product can tolerate. Store the stable artifact identity so a hit can link back to authoritative detail. A search result saying an object is available is only as current as its ingestion path; the application must recheck critical booking or ownership decisions at the proper authority.

An index and an identifier define the target

An index contains related documents with a mapping that defines field behavior. The same _id can exist in different indices, so retain the index or logical collection context when resolving a hit. The illustrative document body belongs to PUT museum-artifacts-v1/_doc/art-101. Explicit identity makes replacement semantics reviewable; automatically generated identifiers suit some append-only events but make duplicate delivery harder to recognize. Elasticsearch distributes an index through primary shards and replica copies in versioned clusters. A shard is a physical distribution unit, not a business table relationship or an application authorization boundary.

json Illustrative technical example; logical results are reasoned, not engine-tested.

{"artifact_id":"art-101","title":"Bronze navigation compass","status":"displayed","year":1890}

A synthetic source document with a stable catalog identifier. It is a request body only and is not sent to a service.

Acknowledgment is not immediate search visibility

Elasticsearch search is near real-time. A successful indexing response does not imply that a following search already sees the document; refresh opens recent changes for search. A real-time GET by identifier has different visibility behavior from a search. Refresh is also different from flush and from the durability guarantees of the write path. Do not use these terms interchangeably when explaining a delayed result. In the 8.19 teaching baseline, ordinary active indices commonly refresh periodically, but settings and deployment models alter timing. The course never promises a fixed one-second end-to-end indexing deadline.

Read the response as evidence with scope

A response may include _index, _id, _version, _seq_no, _primary_term and shard information. These fields identify the accepted operation and support concurrency control, but an example response is not a universal constant. The simplified response below deliberately omits deployment-dependent metadata. A result of created differs from updated, and a timeout can leave the client uncertain whether the server accepted work. Preserve the intended document identity before retrying. A generated new identifier on every retry can produce duplicates even if each individual request is valid JSON.

json Illustrative technical example; logical results are reasoned, not engine-tested.

{"_index":"museum-artifacts-v1","_id":"art-101","result":"created"}

A schematic successful first-write response, not captured Elasticsearch output. Later replacement normally reports updated.

Separate the server model from a managed project

This course uses versioned 8.19 reference semantics for ordinary document indices and typeless APIs. It does not claim that 8.19 is the newest release or prescribe its support lifetime. Elastic Cloud Serverless is a continuously updated project model with different infrastructure responsibilities and supported settings; do not copy node or shard administration into it. Self-managed, Hosted, ECK and ECE deployments also divide responsibilities differently. Identify the actual server version and subscription before applying an example. Every request remains runnable:false, and no cluster, cloud account or configured endpoint is contacted.

Common errors and fixes

A new document is missing from search

Why it happens: The write was accepted but no relevant refresh has made it searchable yet.
How to fix it: Compare identity, target index and visibility expectations before changing data.
Prevent it next time: Define the search freshness contract and avoid forcing a refresh after every event.

index_not_found_exception

Why it happens: The request names an absent or inaccessible intended index.
How to fix it: Verify the documented alias or index and authorized principal.
Prevent it next time: Keep environment-specific target names explicit.

Retry creates duplicate artifacts

Why it happens: Each replay allocated another identifier for the same business item.
How to fix it: Reconcile by stable artifact identity and use deliberate write semantics.
Prevent it next time: Include duplicate delivery in ingestion design reviews.

Check your understanding

1. Why can GET by identifier find an artifact that a title search does not yet find?

Show worked solution

The operations have different visibility contracts. A real-time document GET can observe the accepted write before refresh exposes it to search. Verify the same index and identifier, then check refresh expectations and query mapping separately. Do not conclude that the source document was lost from a single early search.

2. Does a primary shard represent one museum department?