Response Cache (Experimental)

Experimental feature. Response caching is in alpha and is subject to change, and should not be used in production environments.

The router caches two kinds of subgraph response. Entity fetches. An entity fetch is the request the router sends to a subgraph through the _entities root field to resolve fields of an entity that another subgraph owns. Entries are keyed per entity and per selection set, so asking for different fields of the same entity does not share an entry. Root query fetches. A root query fetch is the request the router sends a subgraph for the root fields the client selected. The whole answer is stored as one entry. The key covers the operation text and every variable value. Any difference in either is a miss, and the request goes to the subgraph as it would have. Only queries are cached. Mutations and subscriptions are never cached. Response caching is disabled by default.

What gets cached

A subgraph response is cached only when its Cache-Control header asks to be. The rules apply in this order:

no-store refuses caching.
no-cache or private refuses caching, in any form. This includes the qualified no-cache="Set-Cookie" form.
s-maxage sets the lifetime and takes precedence over max-age.
max-age sets the lifetime when s-maxage is absent.
When neither lifetime is present, fallback_ttl applies if the header contains public, must-revalidate, proxy-revalidate, stale-if-error, or stale-while-revalidate. The stale directives require a valid duration, such as stale-if-error=60.

public is optional. For example, Cache-Control: max-age=60 and Cache-Control: s-maxage=60 both allow caching for 60 seconds. A value of 0 for the selected lifetime refuses caching. It does not fall through to the next rule. For example, max-age=60, s-maxage=0 is not cached, while max-age=0, s-maxage=60 is cached for 60 seconds. An invalid duration in a recognized directive also prevents caching. A missing or empty Cache-Control header is never cached. Headers containing only unrecognized directives, such as cdn-cache-control=60 or immutable, are also never cached. fallback_ttl does not apply to these responses. Revalidation and stale-response directives opt into the fallback lifetime. The response cache does not revalidate entries or serve expired entries. A response carrying GraphQL errors is never cached, whatever its Cache-Control header says.

Configuring the response cache does not make anything cacheable on its own. Subgraphs opt in with a recognized caching directive, such as Cache-Control: max-age=60 or Cache-Control: public. If nothing appears to be cached, check the subgraph response headers first.

Enable it

Redis

Every router replica shares one cache and entries outlive the process. Define a storage provider, then reference it by provider_id. Redis 7.0 or newer is required to use this feature.

storage_providers:
  redis:
    - id: 'my_redis'
      cluster_enabled: false
      urls:
        - 'redis://localhost:6379'

response_cache:
  enabled: true
  fallback_ttl: 30s
  key_prefix: 'cosmo_response_cache:'
  storage:
    provider: 'redis'
    provider_id: 'my_redis'

An unknown provider_id fails startup.

Router memory

Each replica holds its own cache, nothing survives a restart, and no Redis is needed.

response_cache:
  enabled: true
  fallback_ttl: 30s
  storage:
    provider: 'memory'
    max_entries: 2048

provider defaults to redis, so caching in memory has to be asked for by name. This prevents a missing provider_id from quietly turning one shared cache into one cache per replica.

Configuration reference

config.yaml

response_cache:
  # Enable response caching. Default: false.
  enabled: true
  # Lifetime for a response with a recognized caching directive but no lifetime.
  # A response naming its own max-age or s-maxage gets that instead.
  # Minimum 1s. Default: 30s.
  fallback_ttl: 30s
  # Prepended to every cache key so entries cannot collide with anything else
  # sharing the Redis instance. Redis only, ignored by the memory provider.
  # Default: cosmo_response_cache:
  key_prefix: 'cosmo_response_cache:'
  storage:
    # redis or memory. Default: redis.
    provider: 'redis'
    # ID of a provider declared under storage_providers.redis.
    # Required unless provider is memory.
    provider_id: 'my_redis'
    # Memory provider only. Default: 10000.
    max_entries: 10000
  invalidation:
    # Which secondary indexes are built over cached entries, so an entry can
    # later be found by what it is about. Each defaults to true.
    # Index under the cache tags a subgraph declared.
    cache_tag: true
    # Index under the subgraph that answered.
    subgraph: true
    # Index entities under their __typename.
    type: true
    endpoint:
      # Serve the invalidation endpoint. Default: false.
      enabled: false
      # Its own address, not a route on the router's port. Default: localhost:5027.
      listen_addr: 'localhost:5027'
      # Default: /invalidation.
      path: '/invalidation'
      # Sent as the Authorization header, no Bearer prefix. At least 32
      # characters. Required when the endpoint is enabled.
      shared_key: ''

storage is required when enabled is true. Environment variables bypass the config schema validation. A fallback_ttl of zero or less fails startup either way.

Limitations

These apply to the current alpha. A batch is served from the cache only when every entity in it is present. One miss sends the whole batch to the subgraph, including the entities that were already cached. Entities are stored individually, so a later batch made up only of cached entities is a full hit. Forwarded request headers are not part of the cache key. The key covers the request the router renders for the subgraph, which is the operation text and its variables. Headers added by header propagation are applied after that and never reach the key. Two requests that differ only by a propagated header share one entry, so the first response is served to the second caller. A subgraph whose answer varies by request header must use private or no-store. Omitting public does not prevent caching. Cache hits produce no subgraph telemetry. A hit skips the subgraph fetch, so no subgraph span or metric is recorded for it. Subgraph request counts fall as the hit rate rises. Use them to confirm the cache is working, not to measure traffic. enable_multi_fetch disables caching for merged entity fetches. When engine.enable_multi_fetch is on, the router merges entity fetches to the same subgraph within one parallel wave into a single request. A merged fetch is not cached, and nothing reports that it was skipped. An entity fetch that has nothing to merge with is still cached, and root query fetches are unaffected. Support for caching merged fetches is planned.

Cache tags

A subgraph names what its response is about by returning a cache tag extension. Tags are recorded as a secondary index over the cached entries, so an entry can be found by what it is about rather than only by the key it happens to sit under. The shape depends on what was fetched, because an entity fetch caches one entry per entity while a root query fetch caches its whole answer as one. Entity fetches return apolloEntityCacheTags, an array of arrays holding one list per entity, in the same order as the _entities answered:

{
  "data": { "_entities": [{ "__typename": "User", "id": "1" }, { "__typename": "User", "id": "2" }] },
  "extensions": { "apolloEntityCacheTags": [["users", "user-1"], ["users", "user-2"]] }
}

Root query fetches return apolloCacheTags, one flat array:

{
  "data": { "homepage": { "title": "Hello" } },
  "extensions": { "apolloCacheTags": ["homepage", "featured"] }
}

An outer array whose length does not match the entities answered costs the whole response its tags, rather than being zipped as far as the shorter of the two. Either extension is consumed by the router and never forwarded to clients.

Invalidation

Entries leave the cache on their own, when the TTL expires. However users might want to invalidate the cached data before this happens. In order to do this the router provides an invalidation endpoint on a different port, which can get called to invalidate entries.

Tags may not be invalidated. This is a known bug we are working on a fix for, essentially under certain circumstances certain entries belonging to a tag may not be invalidated, which leads to an issue of correctness.

Enable the endpoint

config.yaml

response_cache:
  invalidation:
    endpoint:
      enabled: true
      shared_key: 'a-random-string-of-at-least-32-characters'

Send a request

POST a JSON array to the endpoint, with the shared key as the Authorization header. There is no Bearer prefix.

curl -X POST "http://localhost:5027/invalidation" \
  -H "Authorization: ${INVALIDATION_SHARED_KEY}" \
  -H "Content-Type: application/json" \
  -d '[
    { "kind": "subgraph", "subgraph": "products" },
    { "kind": "type", "subgraph": "products", "type": "Product" },
    { "kind": "cache_tag", "subgraphs": ["products", "reviews"], "cache_tag": "product-42" }
  ]'

Each element names one of three kinds.

`kind`	Removes	Fields
`subgraph`	Every entry the subgraph answered	`subgraph`
`type`	Every entity of that `__typename` the subgraph answered	`subgraph`, `type`
`cache_tag`	Every entry the listed subgraphs tagged with it	`subgraphs`, `cache_tag`

A successful request answers 202 with the number of cache keys removed:

{ "count": 12 }

The whole array is validated before anything is removed. One bad element refuses the request with 400 and removes nothing, so it can be corrected and resent as a whole. Each error names the element it is about:

{ "errors": [{ "index": 1, "kind": "type", "message": "a \"type\" request requires a type" }] }

Restrictions

Tags can be outdated. With the Redis provider, an entry cached again under different tags stays listed under its earlier tags until it expires. Invalidating a tag the entry no longer carries still removes it. With the memory provider, an entry is listed under the tags of its latest write only. Everything is scoped to a subgraph. A type or a cache tag is indexed per subgraph that answered, so every request names one. There is no way to invalidate a type or a tag across all subgraphs in one element. List the subgraphs instead. The memory provider is per replica. Each router holds its own cache, so an invalidation reaches only the replica it was sent to. We do not recommend using the in memory cache with multiple instances, and also in production.

Observability

The router logs once at startup when the cache is enabled. With the Redis provider:

INFO  Response cache enabled  {"fallback_ttl": "30s", "storage_provider": "redis", "key_prefix": "cosmo_response_cache:", "storage_provider_id": "my_redis"}

With the memory provider:

INFO  Response cache enabled  {"fallback_ttl": "30s", "storage_provider": "memory", "max_entries": 2048}

Nothing is logged when the cache is disabled. Cache failures never fail a request. When a read or a write fails, the router serves from the subgraph and logs a warning:

WARN  Response cache degraded, serving from the subgraph instead

That warning is sampled to one line per second. An unreachable cache produces one failure per cacheable fetch of every request in flight, and logging it unsampled would compound the outage.

Storage Providers defines the Redis instance the redis provider references.
Cache Warmer is a different cache. It pre-populates the operation plan cache and does not cache subgraph responses.

Getting Started

Concepts

Cosmo Connect

Cosmo ConnectRPC

Router

Studio

Control Plane

GraphQL / Federation

Deployments and Hosting

Response Cache (Experimental)

What gets cached

Enable it

Redis

Router memory

Configuration reference

Limitations

Cache tags

Invalidation

Enable the endpoint

Send a request

Restrictions

Observability

​What gets cached

​Enable it

​Redis

​Router memory

​Configuration reference

​Limitations

​Cache tags

​Invalidation

​Enable the endpoint

​Send a request

​Restrictions

​Observability

​Related

What gets cached

Enable it

Redis

Router memory

Configuration reference

Limitations

Cache tags

Invalidation

Enable the endpoint

Send a request

Restrictions

Observability

Related