Sling Content Distribution (SCD): a practical guide for cloud & on‑prem

Apache Sling Content Distribution (SCD) lets you move content (Sling resources) across Sling/AEM instances using a path‑level API and configurable distribution agents. It works equally well on classic on‑prem topologies and in modern cloud deployments (including AEM as a Cloud Service), where publishing is powered by SCD under the hood. [sling.apache.org], [sling.apache.org], [experience….adobe.com] [stacknowledge.in]

What SCD is (and why it matters)

• Path‑level API + agents. The SCD API operates on repository paths; agents package paths, queue them, transport them, and import them on the target.
• Works on cloud and on‑prem. AEM as a Cloud Service uses SCD (with an external pipeline service) instead of classic replication agents; on‑prem/AMS still use the same SCD bundles locally.
• Three core models (+ one variant): forward push, reverse pull, and sync with a coordinator; plus multi‑datacenter sync, a variant of sync using a coordinator per DC.
• Built from OSGi configs. Agents, importers/exporters, queues, and credentials are all created via OSGi factory configurations (e.g., ForwardDistributionAgentFactory).

If you’re targeting very large cloud topologies, the Journal‑based SCD implementation offers cloud‑friendly scale using a persisted log and shared blob store for large binaries (binary‑less packages).

Architectural building blocks

• Bundles: org.apache.sling.distribution.api, …core, optional serializers (Kryo/Avro), and sample/ITs modules.
• Agents & endpoints: agents connect to importer/exporter HTTP endpoints on the opposite side.
• Queues & events: package lifecycle events include created, queued, distributed, dropped (source), and imported (target). You can observe these via OSGi event topics.

Distribution scenarios

1) Forward distribution (push)

Push content from a source (e.g., Author) to one or many targets (e.g., Publish).

Trigger via HTTP:

# add/update
curl -u admin:admin \
  -d "action=ADD" -d "path=/content/sample1" \
  http://localhost:4502/libs/sling/distribution/services/agents/publish

# delete
curl -u admin:admin \
  -d "action=DELETE" -d "path=/content/sample1" \
  http://localhost:4502/libs/sling/distribution/services/agents/publish

Events you’ll see (source/target): …/package/created, …/queued, …/distributed, …/dropped (source); …/imported (target).

2) Reverse distribution (pull)

Targets pull from remote sources—useful for consolidating changes or user‑generated content.

Typical flow (HTTP):

# on the "remote" side, queue content to be pulled
curl -u admin:admin \
  -d "action=ADD" -d "path=/content/sample1" \
  http://localhost:4503/libs/sling/distribution/services/agents/reverse

# on the consolidating instance, pull it
curl -u admin:admin \
  -d "action=PULL" \
  http://localhost:4502/libs/sling/distribution/services/agents/publish-reverse

3) Sync distribution (coordinate)

Synchronize content across multiple instances using a coordinating instance (often Author).

4) Multi‑datacenter sync

A sync variant with one coordinator per datacenter for cross‑DC consistency.

Configuring agents via OSGi (cloud & on‑prem)

SCD is configured through OSGi factory configs (JSON .cfg.json). The file name follows the factoryPid~name.cfg.json convention (Configuration Installer).

Forward agent on the source (Author):
/apps/myproj/osgiconfig/config.author/org.apache.sling.distribution.agent.impl.ForwardDistributionAgentFactory~publish.cfg.json

{
  "name": "publish",
  "enabled": true,
  "packageImporter.endpoints": [
    "http://publish:4503/libs/sling/distribution/services/importers/default"
  ],
  "serviceName": "myagent"   // binds to the secret provider below
}

Importer on each target (Publish):
org.apache.sling.distribution.packaging.impl.importer.LocalDistributionPackageImporterFactory~default.cfg.json

{
  "name": "default"
}

Transport credentials (target):
com.adobe.granite.distribution.transport.impl.SecretTransportAuthenticationProviderFactory~myagent.cfg.json

{
  "serviceName": "myagent",
  "username": "distribution-user",
  "password": "••••••••"
}

(Ensure the same serviceName is referenced by the agent.)

Cloud note. In AEM as a Cloud Service, you still provide OSGi configs (as code, in ui.config) and environment variables; Cloud Manager applies them by runmode. You don’t hand‑configure replication agents in Cloud; distribution is handled by SCD’s pipeline. [experience….adobe.com], [experience….adobe.com]

“On/Off Time” auto‑replication

If authors set On Time/Off Time on pages, you can enable Auto Replicate so publish/unpublish fires automatically at those timestamps via an OSGi config named On Off Trigger Configuration.

Using the Java API

Sling’s Distribution API provides Distributor, DistributionRequest and request types like ADD, DELETE, and INVALIDATE (useful for cache invalidation).

Example: invalidate cache for specific paths

import org.apache.sling.distribution.DistributionRequest;
import org.apache.sling.distribution.DistributionRequestType;
import org.apache.sling.distribution.SimpleDistributionRequest;
import org.apache.sling.distribution.DistributionResponse;
import org.apache.sling.distribution.Distributor;
import org.apache.sling.api.resource.ResourceResolver;
import org.osgi.service.component.annotations.Component;
import org.osgi.service.component.annotations.Reference;

@Component(service = MyCacheInvalidator.class)
public class MyCacheInvalidator {

    @Reference
    private Distributor distributor;  // OSGi-injected SCD Distributor

    public DistributionResponse invalidate(ResourceResolver resolver,
                                           boolean isDeep,
                                           String... pathsToInvalidate) {

        DistributionRequest req = new SimpleDistributionRequest(
                DistributionRequestType.INVALIDATE,   // supported by modern SCD APIs
                isDeep,
                pathsToInvalidate
        );

        // "publish" = agent name configured in OSGi (ForwardDistributionAgentFactory~publish)
        return distributor.distribute("publish", resolver, req);
    }
}

DistributionRequestType.INVALIDATE is available in current AEMaaCS SDKs; upgrade if your SDK lacks it.

Example: add/delete content programmatically

DistributionRequest addReq =
    new SimpleDistributionRequest(DistributionRequestType.ADD, true, "/content/site/en");

DistributionRequest deleteReq =
    new SimpleDistributionRequest(DistributionRequestType.DELETE, false, "/content/site/en/old-page");

distributor.distribute("publish", resolver, addReq);
distributor.distribute("publish", resolver, deleteReq);

(Ensure the publish agent and credentials are configured as shown earlier.)

HTTP endpoints (for tooling & ops)

SCD exposes HTTP endpoints under /libs/sling/distribution/services/agents/<agentName>. POST with action=ADD|DELETE|PULL and path=/content/... to trigger operations; use importer/exporter endpoints for the opposite side of the transfer.

Observability & events

Subscribe to SCD OSGi event topics to log/audit distribution. For example, the forward flow emits:

• org/apache/sling/distribution/agent/package/created
• …/queued
• …/distributed
• …/dropped
• org/apache/sling/distribution/importer/package/imported (on targets)

SCD doesn’t update cq:lastReplicated* properties automatically (unlike classic replication). If you need them, listen for SCD events and set them yourself.

@Component(service = org.osgi.service.event.EventHandler.class,
  property = {"event.topics=org/apache/sling/distribution/event/DistributionEvent/END"})
public class ReplicationStampListener implements EventHandler {
  public void handleEvent(Event event) {
    // read 'path' from event, update cq:lastReplicated*
  }
}

Sling Content Invalidation: SCD vs Replication API (and limits)

• Preferred: Use SCD’s INVALIDATE request type to invalidate dispatcher/CDN surfaces tied to paths you changed—this plays nicely with the SCD event model in both cloud and on‑prem.
• Classic replication‑based flush: Traditional Dispatcher Flush agents (on on‑prem/AMS) send invalidation requests when content is replicated; this works but can queue up and isn’t recommended for high‑volume, cloud‑native setups. Cloud replaces agents with SCD and encourages TTL‑based caching & rule‑driven flush.

In practice, replication‑API‑driven flushes can become a bottleneck at scale (queues, back‑pressure) and are discouraged for broad/blanket invalidations; prefer TTL headers and targeted rules (e.g., ACS AEM Commons Dispatcher Flush Rules) when you truly must invalidate.

End‑to‑end samples

1) Forward distribution: minimal OSGi configs (source & target)

// Author: Forward agent -> Publish importer(s)
// org.apache.sling.distribution.agent.impl.ForwardDistributionAgentFactory~publish.cfg.json
{
  "name": "publish",
  "enabled": true,
  "packageImporter.endpoints": [
    "http://publish:4503/libs/sling/distribution/services/importers/default"
  ],
  "serviceName": "myagent"
}

// Publish: Local importer
// org.apache.sling.distribution.packaging.impl.importer.LocalDistributionPackageImporterFactory~default.cfg.json
{ "name": "default" }

// Publish: secret provider (credentials used by Author's agent)
// com.adobe.granite.distribution.transport.impl.SecretTransportAuthenticationProviderFactory~myagent.cfg.json
{
  "serviceName": "myagent",
  "username": "distribution-user",
  "password": "••••••••"
}

2) Triggering via cURL (dev/test)

# push a page subtree
curl -u admin:admin -d "action=ADD" -d "path=/content/site/en" \
  http://localhost:4502/libs/sling/distribution/services/agents/publish

3) Auto publish/unpublish when On/Off time hits

Enable Auto Replicate in the On Off Trigger Configuration OSGi config; authors set times in Page Properties, and SCD handles the timed action.

Cloud vs on‑prem quick notes

Tips & references

• Binary‑less distribution: For large assets, use Vault binary‑less packages with a shared blob store. [github.com]
• Naming matters: avoid spaces in agent names to prevent queue issues in some UIs. [aurx.co]
• Version your APIs: the Distribution API is released independently (latest 0.7.x); ensure your SDK/bundles expose INVALIDATE if you plan to use it. [mvnrepository.com], [experience….adobe.com]

Wrap‑up

SCD gives you a unified, OSGi‑configurable way to deliver content between Sling/AEM instances—forward, reverse, and sync—with a clean path‑level API and solid observability. In cloud, it’s the underpinning of publish; on‑prem, it helps you move beyond fragile point‑to‑point replication. For cache management, prefer SCD INVALIDATE + TTLs + targeted rules over broad replication‑driven flushes.

If you want, I can turn this into a runnable reference project (OSGi configs + a small “invalidate by path” service) you can drop into your codebase.