Terms
- Client request. The GraphQL request a client sends the router.
- Client response. What the router sends back to the client. It is built from the results of all fetches.
- Subgraph request. A request the router sends to a subgraph to resolve part of a client request.
- Subgraph response. What a subgraph sends back to the router. The cache stores data from subgraph responses, never client responses.
- Fetch. One subgraph request and its subgraph response. One client request can need several fetches.
- Root fetch. A fetch for the root
Queryfields of an operation, such astopProducts. - Entity fetch. A fetch that asks a subgraph for more fields of objects another subgraph returned. The router sends them in an
_entitiesquery. - Representation. How the router identifies one object in an entity fetch: its type and key fields, such as
{"__typename":"Product","upc":"1"}. - Entry. One cached answer. A root fetch is stored as one entry. An entity fetch is stored as one entry per object.
- Key. The name an entry is stored under in Redis. Two fetches that send the same subgraph request get the same key, so the second one finds the first one’s entry.
- Hit and miss. A fetch is a hit when every entry it needs is in the cache, so the subgraph is not called. Otherwise it is a miss.
- Lifetime. How long an entry is kept. It comes from the
Cache-ControlorExpiresheader of the subgraph response. Redis deletes the entry when it runs out. - Tag. A label stored next to an entry, such as
subgraph-products. An invalidation names a tag to delete every entry that carries it.
An example
A client sends this query.topProducts, name and price come from the products subgraph. reviews comes from the reviews subgraph.
products:
data object of that answer as one entry.
2. An entity fetch to reviews, for the two products the first fetch returned:
Reusing an entry from another query
A different query can reuse an entry when it leads to the same subgraph request. This query asks for one product:- A root fetch to
products:{ product(upc: "1") { __typename upc name } }. This request differs fromtopProducts, so it is a miss. Its answer is stored as a new entry. - An entity fetch to
reviewsfor{"__typename":"Product","upc":"1"}, asking for... on Product { reviews { body rating } }. This is the same request the first query made for product1. It finds the entry9f2c…:5feb…, andreviewsis not called.
When the selection changes
An entity entry is only reused when the same fields are asked for. The fields an entity fetch asks for are its selection set:reviews { body rating } in the example.
A query that asks for one field less gets a different request, even for the same product:
reviews fetch now asks for ... on Product { reviews { body } }. That is a different selection, so it is a miss, even though the cached entry already holds body. The router stores a second entry for product 1 next to the first one. The same happens when an argument changes, such as reviews(first: 5) instead of reviews(first: 10). Each distinct selection is cached separately. The cache never answers a request from part of an entry that holds more fields.
How the router turns a request into a key is covered next.
Keys
A key says which subgraph request an entry answers. The key does not hold the subgraph request itself, which can be large. It holds SHA-256 hashes of its parts, called digests. Two subgraph requests with the same parts produce the same digests, so they get the same key. A key has two required parts and up to two optional ones:key_prefix and e: in front, for example cosmo_response_cache:e:2:9f2c…:5feb….
- Version. Changes when the router changes how it builds keys or stores entries. Entries under an older version are no longer read and expire on their own.
- Entity part. Which object, or which whole request, the entry is about.
- Selection part. What was asked about it: the query text and variables of the request.
- Private part. Only for
privatesubgraph responses. See Private and shared entries. - Vary part. Only for subgraph responses with a
Varyheader. See Varying responses.
Root fetches
A root fetch is one entry, so its key covers the whole request:- Entity part: a digest of the full request the router sends the subgraph. That is the query text, the variable values, the static headers configured for the subgraph and the subgraph URL.
- Selection part: a digest of the subgraph’s id.
Entity fetches
An entity fetch is one entry per object, so each object gets its own key:- Entity part: a digest of the object’s representation, such as
{"__typename":"Product","upc":"1"}. - Selection part: a digest of the rest of the request: the
_entitiesquery text, which lists the fields asked for, and the variables other than the representations.
5feb…, because the router asked for the same fields of both. Their entity parts differ.
A later fetch for the same fields of the same object reuses the entry, wherever the object appears in an operation. Asking for other fields of the same object is a different selection, so it is a different entry.
Keys in merged entity fetches
Withengine.enable_multi_fetch, which is on by default, the router can combine entity fetches to the same subgraph that run at the same step of the plan into one request. Each part of that request is an _entities field under its own alias, such as a0: _entities(...) and a1: _entities(...).
The selection part covers the whole request, aliases included. A merged request therefore has a different selection part from a plain _entities request for the same objects and fields. The two do not share entries. See Limitations.
What else is part of the key
- Client extensions. Extensions the client sends, other than
persistedQuery, are forwarded to subgraphs and are part of the selection digest. - Undefined variables. A variable that was omitted is keyed apart from one sent as
null, because the subgraph receives different requests. - Private id. A
privatesubgraph response is stored under a key that adds a digest of the client request’sprivate_id. - Vary. A subgraph response with a
Varyheader is stored under a variant key derived from the values of the named headers. See Varying responses.
Vary.
Request lifecycle
The router plans an operation into subgraph fetches. The cache acts on each fetch separately.- Inbound deduplication. Identical client requests in flight at the same time share one execution. Only the request that executes reads and writes the cache. The others receive its client response. See Request Deduplication.
- Key building. The router renders each fetch’s subgraph request and derives one key per entry from it. See Keys.
- Lookup. All keys of a fetch are read from the store in one call, or two when the entry varies. See Varying responses.
- Origin. A fetch that is not a full hit is sent to the subgraph. With
only-if-cached, it fails instead. Identical subgraph requests in flight share one call. - Store decision. The status, body and headers of the subgraph response decide whether it is stored and for how long. See What is stored.
- Write. Stored entries are written before the fetch completes. The write is synchronous: the client response waits for it. See Storage.
- Client response headers. The router turns the lifetimes of the fetches into the
Cache-Controlof the client response. It also builds theCache-Tagheader when that is enabled. See Client Cache-Control.
only-if-cached. A failed lookup is a miss. A failed write is logged and counted, and the client response is sent as usual.
Which fetches are cached
Fetches inside
@defer groups are cached like any other fetch of a query. The client response to a @defer query always carries Cache-Control: no-cache.
Lookups
A lookup reads every key a fetch needs from Redis in one call.All or nothing per fetch
An entity fetch is served from the cache only when every one of its objects is found. If one object is missing, the router sends the whole fetch to the subgraph, including the objects that were cached. The fetch counts as amiss.
In the example, if the reviews of product 1 are cached but those of product 2 are not, the reviews subgraph is asked for both.
Each object is still stored on its own. A later fetch made only of cached objects is a hit.
Merged entity fetches
A merged fetch is looked up per alias. An alias whose objects are all cached is answered from the cache and left out of the request. The router rebuilds the request with only the other aliases and sends that. Each alias is still all or nothing. The fetch counts as apartial_hit when some aliases came from the cache, and a hit when all did.
When the rebuilt request fails, the aliases answered from the cache keep their data. Fetches that depend on the merged fetch are still skipped, and their fields are null without an error. See Limitations.
Private and shared entries
Most entries are shared: every fetch with the same key gets the same answer. Aprivate subgraph response is stored under a key with an extra part for the user or tenant it belongs to, called the private id.
A client request with a private id looks up both keys of every object in one call. The private entry wins when both exist.
- The private part of the key is a SHA-256 digest of the id. Raw ids are never written to Redis.
- A client request without a private id does not store or read private entries. The subgraph is asked every time.
- Invalidation by subgraph, type or tag also removes private entries.
- A
setresponse header rule onCache-Controlcan replaceprivatein the client response. It does not change how the entry is stored.
Objects that do not exist
An object the subgraph returns asnull is not stored, so a fetch that includes it is never a hit. See Limitations.
Varying responses
A subgraph whose answer depends on a request header, such asAccept-Language, names it in a Vary response header. The router keeps one entry per combination of values it sent.
- The first subgraph response stores the names from
Varyas a record at the entry’s key, and the body under a variant key. - A later lookup reads the record, digests the values of those headers in the current subgraph request, and reads the variant.
- Values are taken from the request the router builds, after header propagation. A named header the router does not set counts as empty.
Vary: *is not stored. Neither is aVarywith more than 100 names, nor a response whose varied header was changed after the router built the request, for example by a custom module.Accept,Accept-EncodingandContent-Typeare not checked. The entry is keyed by the value the router built, not the one sent. A module that changes them based on other input can cause wrong hits.- A header with a value per caller, such as
CookieorAuthorization, yields one entry per caller. Such entries are rarely reused. Useprivateinstead. - When the subgraph changes the names it varies on, the new set is added to the record. A record keeps up to 8 sets, newest first. An older variant stays reachable while both it and the record live. The record takes the newest response’s lifetime, and a response without
Varyreplaces it.
What is stored
The decision is made from the raw subgraph response, before any response header rules run. Acache_control_policy value or a set rule on Cache-Control changes the client response. It never changes what is stored.
The checks run in this order. The first that applies decides. In a merged entity fetch, headers are checked once and the rest per alias. It reports stored when any alias was stored.
The store decision is reported per fetch. See Store decision.
How the subgraph’s Cache-Control is read
These rules apply to theCache-Control header of the subgraph response. The Cache-Control a client sends in its request is covered under Request directives.
- Directive names are case-insensitive.
- Several
Cache-Controlheader lines are read as one, joined by commas. - When a lifetime directive appears twice, the first one counts. The second is ignored, even if invalid.
- An invalid value in a known directive, such as
max-age=abcormax-age=-1, makes the whole header invalid. Nothing is stored, even ifno-storeis also present. - Qualified
no-cache, such asno-cache="Set-Cookie", does not prevent storing. Entries hold no response headers, so there is nothing to strip.no-cache=""counts as unqualified. privatein any form, includingprivate="Set-Cookie", makes the whole subgraph response private.- A subgraph request that carried
Authorizationcan make the subgraph response private. See Authorization.
Authorization
A subgraph request that carries anAuthorization header produces a private response unless the response says public, s-maxage or must-revalidate. Without a private_id, such a response is not stored.
This applies only when Authorization is propagated to the subgraph. A subgraph that serves the same data to every authenticated caller should say public.
- The miss response does not carry
privatefrom this rule. Its hits do. To keep such responses out of a CDN:- Send
privatefrom the subgraph. - Configure a cache control policy.
- Make sure no
setrule replacesprivate.
- Send
- Only
Authorizationtriggers the rule. Other credentials, such asCookieor a custom header, do not make a response private.
Lifetime
The stored lifetime is the first of these that is present in the subgraph response, minus itsAge header:
s-maxagemax-ageExpires, measured against theDateheader, or against the current time if there is noDatefallback_ttl, but only when the header has one ofpublic,private,must-revalidate,proxy-revalidate,stale-if-error=<seconds>,stale-while-revalidate=<seconds>or a qualifiedno-cache
Cache-Control and no Expires is not stored. Neither is one whose Cache-Control holds only directives the cache does not use, such as immutable or no-transform. fallback_ttl does not apply to either.
A zero lifetime is not stored and does not fall through to the next source. max-age=60, s-maxage=0 is not stored. max-age=0, s-maxage=60 is stored for 60 seconds.
public is not required. max-age=60 on its own is stored for 60 seconds.
The cache does not revalidate entries and does not serve expired entries. stale-while-revalidate, stale-if-error and must-revalidate only make fallback_ttl apply.
Errors and partial data
- A single or batch fetch whose subgraph response has any GraphQL error stores nothing, including the entities that resolved.
- In a merged fetch, an error whose path starts at an alias excludes that alias only. An error with no alias in its path excludes every alias.
Cache-Control of cached fetches
TheCache-Control of the client response is built by the cache control policy, which merges the Cache-Control of every fetch. The cache changes what a cached fetch contributes to that merge. Without a policy, the cache adds no Cache-Control to the client response.
A fetch that missed and was stored reports the subgraph’s directives with the lifetimes capped at the stored lifetime, minus the Age header of the subgraph response. When the subgraph gave no max-age and no Expires:
- a non-private subgraph response that is
publicor hass-maxagegets the stored lifetime ass-maxage - any other gets it as
max-age
max-ageis the remaining lifetime for browsers. It is never longer than the entry’s own remaining lifetime.s-maxageis the remaining lifetime of the entry, when it differs frommax-age. A private entry gets nos-maxage.- An entry stored only from
s-maxage, or frompublicwithfallback_ttl, is shared-only: its lifetime is meant for shared caches such as a CDN, not for browsers. Its hit reportss-maxagealone. publicis reported only if every entry served was stored as public.privateis reported if any came from a private entry.- A remaining lifetime of zero is reported as
no-cache. - No
Ageheader is sent. The lifetimes already exclude the time spent in the cache.
Request directives
A client can control the cache for its own request with theCache-Control request header.
max-age greater than zero, min-fresh and max-stale are ignored. So is the Pragma header. A Cache-Control request header that cannot be parsed is ignored.
Requests with no-cache, max-age=0 or only-if-cached skip inbound deduplication.
Storage
Redis
The cache keeps two kinds of Redis key:
Entries. Each entry is a string with the entry’s lifetime as its Redis expiry, so Redis deletes it when the lifetime ends. The value holds:
- the subgraph data: the
dataobject of a root fetch, or one object of an entity fetch - the entry’s tags, so a hit can rebuild the cache tag header
- whether the entry is public, and whether it is shared-only
- how much shorter the browser lifetime is than the stored lifetime
- a token naming the write that stored it, so a writer only ever changes its own entry
Cache-Control, are not stored. For a subgraph response with a Vary header, the entry at the key holds the list of header names instead, and the data sits under a second key that adds the vary part.
Tag sets. An invalidation needs to find every entry with a given tag. Each tag therefore has a sorted set listing the keys of its entries. A set member’s score is the time that entry expires. Members more than five minutes past expiry are removed on the next write. The set itself expires five minutes after its last member.
Reads. A lookup sends GET and PTTL for each key in one batch. PTTL returns the time the entry has left, which becomes the lifetime reported to the client.
Writes. An entry without tags is one SET with its lifetime. An entry with tags takes up to three round trips, so that an invalidation running at the same time cannot be missed:
- The entry is added to each tag set. It is written with a short lifetime of up to 10 seconds, called its lease.
- Each tag set is checked again. If an invalidation ran in the meantime, the entry is deleted.
- The entry’s lifetime is extended to its full value, or the entry is deleted if a tag was invalidated since step 1.