How to ensure fast, reliable, selective cache updates when Content Fragments (CF) and Experience Fragments (XF) change

Modern AEM implementations rely heavily on caching to deliver high‑performance digital experiences. Dispatcher (the AEM cache layer) and Content Delivery Networks (CDNs) together accelerate content delivery and reduce compute load on Publish instances. However, when dynamic content like Content Fragments (CFs) and Experience Fragments (XFs) change, any page rendering those items must be refreshed, or users will see stale data.

This article outlines a robust strategy—aligned with business requirements—to handle cache invalidation efficiently across Dispatcher and CDN layers.

Why Caching Is Complex for CFs & XFs

CFs and XFs are often embedded inside multiple pages. Since Dispatcher caches the fully rendered HTML, any CF/XF change does not automatically refresh cached pages referencing them. This is a widely known issue—XF updates do not propagate unless pages are explicitly invalidated.

Similarly, CFs may be included via components or fetched dynamically (e.g., GraphQL or Sling Models). While CFs themselves are not cached individually, updating many CFs can trigger excessive invalidation if not managed properly.

Our Requirements Recap

We are trying to find a solution which must be:

1. Fast and reliable
2. No negative impact on authoring or end‑user experience
3. Invalidate only necessary page caches
4. Handle both Dispatcher & CDN layers
5. Support Any number of dependent pages per fragment

The Page Calculator Process

All strategies described depend on a Page Calculator Process → a service that:

1. Detects which CF/XF was modified.
2. Identifies all pages referencing it (via references API or custom search query).
3. Returns the list of impacted pages.

This list becomes the input for Dispatcher and CDN invalidation operations.

Caching Strategies & Evaluation

Below are the four approaches evaluated.

Option 1 — Invalidate from Author (Dispatcher Flush + CDN Purge)

On activation of CF/XF:

1. The Transport or Event Handler calls the Page Calculator.
2. Dispatcher flush triggered from Author.
3. CDN purged for each affected page.

✔ Pros

Simple to implement.

✘ Cons

• Requires knowing the list of Dispatcher instances (fragile for blue‑green or auto‑scaled topologies).
• In cloud deployments, invalidation initiated from Author has architectural inconsistencies.
• CDN purge from Author is supported, but full consistency becomes difficult.
• Doesn’t scale well for high‑volume invalidation.

Option 2 — Dispatcher Flush from Publisher, CDN Purge from Author

Flow:

• Author triggers CDN purge.
• Publisher event handler triggers page-level Dispatcher flush (more precise).

✔ Pros

• More accurate targeting of Dispatcher invalidation.
• Better alignment with actual rendering environment.

✘ Cons

• Page Calculator runs twice (Author + Publisher).
• Extra compute overhead on both tiers.

Option 3 — Invalidate from Publisher (Dispatcher Flush + CDN Purge)

This approach uses a Publisher‑side event handler:

1. Detect CF/XF modification event.
2. Run Page Calculator (once).
3. Flush Dispatcher locally (no instance list needed).
4. Trigger CDN purge using the same page list.

✔ Pros

• Most efficient and scalable approach.
• No reliance on Author knowing Dispatcher topology.
• One execution of Page Calculator.
• “Local flush” on Publish is more reliable.
• Aligns with modern AEMaaCS architecture, where Dispatcher sits directly in front of Publish and CDN purges can be handled similarly. [experience….adobe.com]
• CDN purging supports single URL or surrogate-key methods for batching. [experience….adobe.com]

✔ Best Fit for Requirements

• Fast ✓
• Low end-user impact ✓
• Selective invalidation ✓
• Works for Dispatcher + CDN ✓
• Scales to large number of pages per fragment ✓

Option 4 — Replication API from Author + CDN Purge

Flow:

• Author triggers CDN purge.
• Author calls Replication API to “republish” all affected pages.
• Publishing event triggers Dispatcher invalidation automatically.

✔ Pros

• Lowest development effort.
• Reuses existing replication infrastructure.

✘ Cons

• High load on replication agents.
• Replicates pages that authors may not intend to publish.
• Significant risk in large deployments.
• Not aligned with modern caching architecture.
• Can cause cache storms.

Detailed Caching Strategies

Dispatcher Caching Strategies

A. Traditional Activation-based Invalidation

Dispatcher uses statfiles and invalidation rules to determine which cache files to expire. However, this may invalidate too much or too little.

B. Selective Page-level Flush (Recommended)

Using the Page Calculator output, flush only specific paths:

• /dispatcher/invalidate.cache endpoint
• Limits clearing to only necessary pages
• Reduces performance impact vs broad invalidation

C. TTL-based Caching

Use TTL headers (Cache-Control, Surrogate-Control) so cached HTML expires automatically after a configurable time.

• Good for content that’s updated but not urgently time-sensitive
• Should be combined with selective invalidation for CF/XF-based pages

D. Sling Dynamic Include (SDI) for Experience Fragments

SDI transforms embedded XF HTML into a server-side include so the XF is cached separately from the page.
On XF update, only the XF file must be invalidated—not all pages using it [aemconcepts.com]

This dramatically reduces the number of pages needing invalidation.

CDN Caching Strategies

CDN caching layers (Fastly, Cloudflare, Akamai) typically cache based on:

• URL-based entries
• Surrogate keys
• TTL expiration

A. URL-based Purge

Purge each affected page individually.
Supported with PURGE calls in AEMaaCS. [experience….adobe.com]

B. Surrogate-key Purge

Group pages using a surrogate key (ideal when one CF/XF maps to many pages).

• Must set Surrogate-Key headers on responses
• Purge once per surrogate key

C. Soft Purge vs Hard Purge

• Soft purge keeps stale content available to users until backend fetch completes.
• Hard purge removes content immediately, causing higher load.

D. TTL Strategy

CDN respects Cache-Control and Surrogate-Control headers from AEM.
TTL helps ensure stale content eventually disappears, even without purging.

Recommended Solution (Option 3), why?

• Minimizes cross-environment dependencies.
• Avoids replication overhead.
• Avoids author-side invalidation complexity.
• Scales for large number of dependent pages easily.
• Aligns with documented best practices:
  • Dispatcher local invalidation recommended for precision.
  • CDN requires explicit API-based purge for fresh content.

Consequences & Mitigation

Risk: High volume of invalidations → Increased load on Publish

Especially if pages are large or expensive to render.

Mitigation: Implement Cache Re-fetch (Warming)

• After invalidation, programmatically fetch all affected URLs.
• This warms both Dispatcher and CDN caches.
• Prevents “thundering herd” effect.
• Aligns with best practices for safe cache refresh.