Enterprise Integration Patterns

Advanced System Design & Software Architecture · Lesson 47 · Microservices and Enterprise Architecture

Lesson Overview

Checkout can publish OrderAuthorized and fulfillment can read it, which is the log from Message Queues and Event Streaming. That is one pattern: a message on a channel. The enterprise around TechLearningPro is not one channel. A warehouse system wants one message per physical line, not one message per order. A digital-entitlement worker wants only the lines that are course licenses. Finance wants one settlement when every line has a result, not a partial. The legacy ERP speaks material numbers and plant codes, and it will not grow a Jackson annotation because a new service wishes it would. Someone has to route, split, collect, and translate. Those jobs have names, and the names are older than Kafka.

Gregor Hohpe and Bobby Woolf’s Enterprise Integration Patterns is the catalog. A message channel carries messages. A content-based router sends a message to one of several channels based on its content. A splitter breaks one message into many. An aggregator combines many back into one, using a correlation id and a completeness condition. A message translator changes the data format. A channel adapter connects a system that does not speak your messaging API. An anti-corruption layer, the DDD name for the translator you put at a boundary you do not own, keeps the ERP’s model out of the order aggregate. A canonical data model is the tempting idea that one enterprise schema removes all translators, and it is usually how every team waits on one XSD.

This lesson maps those patterns onto tools you can actually run. RabbitMQ’s exchanges are the textbook routers: a topic exchange routes by routing key, a headers exchange routes by header values, a fanout publishes to every bound queue. Kafka is a log of partitions, not an exchange. A content-based router in Kafka is a consumer that reads one topic and writes others, or a producer that chooses the topic. Spring Integration is the Java library whose DSL is these patterns: IntegrationFlow, route, split, and aggregate, with CorrelationStrategy and ReleaseStrategy. The mapping is the lesson. Copying an EIP diagram onto a single shared database is not.

Module 5’s notification platform is a splitter and a set of channel adapters. The e-commerce platform is a router between digital and warehouse fulfillment plus an aggregator for “order complete.” The payment case study is a translator at the acquirer boundary, not a canonical payment object forced on the acquirer. You will name the pattern before you name the broker feature, because the broker feature is narrower than the word “messaging” suggests.

Learning Objectives

By the end of this lesson, you should be able to:

1. Name a message channel, a router, a splitter, an aggregator, a translator, and an adapter, and point at which component in checkout owns each one.
2. Explain why Kafka does not provide a broker-side content-based router, and how a small router service or a chosen topic approximates one without pretending partitions are bindings.
3. Use a RabbitMQ topic or headers exchange when the pattern you want is routing at the broker, and say what the competing consumer on a queue deletes.
4. Implement a splitter and an aggregator with an explicit correlation id, a completeness condition, and a timeout, including what you do with a partial group.
5. Put an anti-corruption layer at the ERP boundary so material numbers never appear on the order aggregate or in a public API.
6. Reject a single enterprise canonical model when the real need is a published language between two contexts.
7. Keep idempotent receivers, dead-letter channels, and claim checks in the design when messages are large or poison.
8. Recognize the same patterns in a Module 5 notification fan-out and in an order-to-warehouse flow, and say which pattern must not live inside the domain aggregate.

Prerequisites

• Lesson 9, Event-Driven Architecture Fundamentals: a domain event is not automatically an integration message. This lesson is the integration edge.
• Lesson 33, Message Queues and Event Streaming: topics, consumer groups, and dead letters. This lesson does not re-teach offsets.
• Lesson 8, Domain-Driven Design Fundamentals: bounded contexts are why a canonical Order for the whole company fails.
• Lesson 32, Distributed Transactions and the Saga Pattern: a process manager is the stateful cousin of a router. An aggregator is not a saga.
• Comfort reading a routing key and a JSON field. You do not need to have operated an ESB.

Detailed Core Concepts

Channels and endpoints

A message is a header plus a body. The header carries what the infrastructure needs and what routers look at: a correlation id, a content type, a routing key. The body carries the business payload. Mixing them, by forcing the router to deserialize a private domain object, ties the router to the publisher’s class version. Prefer a small explicit header or a stable field.

A message channel is the logical pipe. A point-to-point channel delivers a message to one consumer. A publish-subscribe channel delivers it to every subscriber. In RabbitMQ the point-to-point case is a queue with competing consumers: the broker deletes the message when one consumer acknowledges. The publish-subscribe case is a fanout exchange with one queue per subscriber, each queue getting a copy. In Kafka both shapes are the same log. One consumer group is the point-to-point case, with the caveat Lesson 33 insisted on: reading does not delete, retention does. Several consumer groups are publish-subscribe. If you need “delete on success” and a per-subscriber copy, say so and pick the broker feature that matches. Calling every Kafka topic a queue will make someone acknowledge a record and expect it to vanish for finance.

A message endpoint is the code in the application that sends or receives. Spring’s KafkaTemplate and @KafkaListener, and Spring AMQP’s RabbitTemplate and @RabbitListener, are endpoints. The domain service should not build AMQP headers in the middle of a pricing rule. The endpoint sits at the edge, which is the hexagonal boundary you already have.

Content-based router

A content-based router examines the message and emits it to exactly one downstream channel (or to a dead-letter channel if nothing matches). “Digital SKU goes to entitlement, physical SKU goes to the warehouse” is a router. It is not a reason for both workers to receive every order and ignore half. Filtering after the fact still makes the warehouse pay for digital traffic and, on a queue, still makes acknowledgement someone’s problem.

On RabbitMQ the router can be the broker. A topic exchange routes by pattern: binding order.digital.# receives digital, order.physical.# receives physical. A headers exchange routes on header values rather than a key, which is the pattern people mean when the decision is type=license rather than a key you can encode in a string. The publisher must set the key or the header honestly. A wrong key is a misroute, not a broker bug.

On Kafka the broker will not read your JSON and choose a topic. You have two honest designs. The producer chooses the topic (fulfillment.digital or fulfillment.physical) because it already knows the line type. That pushes routing to the publisher and removes a hop. Or one topic order.lines is consumed by a router service that publishes to the downstream topics. Use the router service when the rule changes more often than the producer, or when several publishers should not each embed the rule. Do not describe a consumer group as a content-based router. Every member of the group sees a partition of the same topic, not a filtered subset, unless you have a separate topic or a stream processor that actually writes another topic.

A router has a default: the dead-letter channel or an explicit review channel. A line with a type you do not recognize must not vanish and must not be forced into the warehouse. Lesson 33’s dead letter is the same idea at the failure boundary. Here it is also the “no route” boundary. Count those messages. A growing review channel means the catalog invented a type the router does not know.

Splitter and aggregator

A splitter takes one message and emits several, correlated. An order with three lines becomes three line messages, each with orderId and a lineCount of 3, or a line id the aggregator can count. The splitter is not allowed to forget the correlation. Downstream idempotency keys are per line (orderId + lineId), not only per order, or a redelivery of one line looks like a new line.

An aggregator is the inverse, and it is stateful. It stores parts until a completeness condition holds: group.size() == lineCount, or “we have a terminal status for every line id we recorded at split time.” The second form survives a split that does not know the count in advance only if the set of ids was stored with the group. A timeout is mandatory. Warehouse lines that never return must not pin the group in memory until the process restarts and forgets them. On timeout you either emit a partial result with a clear complete=false or you send the group to a dead-letter channel. Silently completing a shipment that is missing a line is how a learner receives a course and not the book they also bought, while the metric says success.

Correlation is orderId for “all lines of this order,” and it must be the same value the splitter wrote. A correlation that uses a random id generated inside the aggregator will never complete. Groups should expire on completion so a duplicate line does not reopen a finished order. Spring Integration’s aggregator is an in-memory (or message-store-backed) implementation of this state. If you run two aggregator instances, an in-memory store splits the group and neither side reaches lineCount. Use a shared MessageGroupStore or a single active aggregator (or build the aggregate in a database table keyed by orderId) when you have more than one pod. This is the same ownership point as any other state. Kafka partitions do not aggregate for you. A consumer that sees one partition sees only the lines whose key hashed there. Key the line messages by orderId if one consumer must see every line of an order, and accept that the order’s lines are then sequential on that partition.

A resequencer is the related pattern when parts must be delivered in order to the next step. Do not confuse it with the aggregator. Resequencing restores order. Aggregating produces one message. A saga process manager from Lesson 32 remembers business state and emits commands. It may use an aggregator’s technical trick, but a timeout on a message group is not a compensation policy. If a line fails, the domain decides whether to cancel the order or ship the rest. The aggregator only knows whether the set is complete.

Translators, adapters, and the model you refuse to share

A message translator converts format and schema: JSON in, the ERP’s flat record out, or the reverse. A channel adapter is the connector at the edge: a file poller, a JDBC poller, an HTTP inbound gateway, a Kafka source. The ERP does not become a Kafka application because you want it to. An adapter on your side speaks the ERP’s interface (a file drop, a SOAP endpoint they already have, a table they already write) and publishes a message your side understands.

An anti-corruption layer is that translator with a policy. The warehouse says material M-8831 at plant P2. The order context says SKU BOOK-ARCH-1 at warehouse blr-1. The mapping table lives in the integration component, not in Order as a field called plantCode “just so we don’t lose it.” Once plantCode is on the aggregate, every API and every pact carries it, and the ERP’s reorganization becomes your migration. The layer can be a Spring Integration flow or a plain service. The pattern is the boundary, not the framework.

A canonical data model tries to skip translators by defining one Order for checkout, warehouse, finance, and the ERP. It works for a tiny estate and stalls a large one, because every new field is a meeting. Prefer a published language between two parties: the OrderAuthorized JSON you already version, and a separate WarehouseShipment JSON the adapter emits. Two small contracts beat one enterprise schema that contains every party’s accidents. If a field is not needed on the other side, it is not in the contract. Claim check is how you keep the contract small when the payload is a PDF or a video: the message carries an object-storage key, and the consumer fetches the bytes if it is authorized. The key is not a public URL with the bucket wide open. Lesson 44’s workload identity is how the fetcher authenticates.

What you must not rebuild

An integration flow that starts updating the order tables directly is a second writer. The order service remains the only writer of its schema. The router publishes commands or the warehouse publishes facts (LineShipped) that the order service consumes and applies. That is the ownership rule from the microservices lessons, expressed as “the aggregator does not UPDATE orders.”

Do not use the integration layer as a hidden workflow engine for every future requirement. A content-based router with a growing decision table is fine. A router that calls five services synchronously and holds locks is an ESB with a new logo. If the business process has compensation, it is a saga with an owner, not a chain of filters nobody is on call for.

Architecture and Design Diagrams

Physical lines and digital lines leave one order and come back as one completion, or as a review when a line type is unknown. The ERP is behind an adapter. The order database has one writer.

mermaidCopy
flowchart LR
  O[order-service] -->|OrderAuthorized| Ch[(order.authorized)]
  Ch --> Split[splitter]
  Split --> R{content-based router}
  R -->|digital| Ent[entitlement worker]
  R -->|physical| Wh[warehouse adapter]
  R -->|unknown| Review[review channel]
  Wh --> ERP[legacy ERP]
  Ent -->|LineGranted| Back[(line.results)]
  Wh -->|LineShipped| Back
  Back --> Agg[aggregator]
  Agg -->|OrderComplete or timeout| O

The sequence for three lines, one of them slow. The timeout path is part of the design, not an afterthought.

mermaidCopy
sequenceDiagram
  participant O as Order service
  participant S as Splitter
  participant E as Entitlement
  participant W as Warehouse
  participant A as Aggregator

  O->>S: OrderAuthorized lines=3
  S->>E: line 1 digital
  S->>E: line 2 digital
  S->>W: line 3 physical
  E->>A: LineGranted 1
  E->>A: LineGranted 2
  Note over A: size 2, lineCount 3, wait
  W->>A: LineShipped 3
  A->>O: OrderComplete

Kafka versus RabbitMQ for the same router. Pick the row that matches deletion, replay, and who evaluates the predicate.

mermaidCopy
flowchart TD
  Q[Need publish-subscribe plus replay?] -->|yes| K[Kafka topics and groups]
  Q -->|no, delete on ack| RMQ[RabbitMQ queue per worker]
  K --> P{Who evaluates the route?}
  P -->|producer already knows| T[Publish to the specific topic]
  P -->|rule changes independently| RS[Router consumer writes downstream topics]
  RMQ --> EX[Topic or headers exchange]

Technical Explanation

A router that is a function, not a framework miracle

Spring Integration’s route takes the payload type and a function that returns the channel key. The mapping is configuration you can test. Unknown keys go to a review channel you named, not to a null channel that drops.

javaCopy
@Bean
IntegrationFlow routeLines() {
    return IntegrationFlow.from(MessageChannels.direct("lines.in"))
        .route(LineMessage.class, LineMessage::fulfillment,
            spec -> spec
                .channelMapping("digital", "fulfillment.digital")
                .channelMapping("physical", "fulfillment.physical")
                .defaultOutputChannel("fulfillment.review"))
        .get();
}

LineMessage.fulfillment() returns digital or physical and nothing else the mapping did not plan for. The method does not query the ERP. The router stays side-effect free so a unit test can pass a line and assert the channel. If evaluating the route needs a database, you have a service, and the flow calls that service. You do not hide a JDBC call inside a routing expression nobody load-tests.

The RabbitMQ equivalent, when the broker should route, binds queues to a topic exchange:

javaCopy
@Bean
Binding digitalBinding(Queue digitalQueue, TopicExchange orders) {
    return BindingBuilder.bind(digitalQueue).to(orders).with("order.digital.#");
}

The publisher sets the routing key. A test publishes order.digital.license and expects one message on digitalQueue and none on the warehouse queue. That test is the router’s contract. A headers exchange is the same idea when the predicate does not fit a key; the binding uses where("fulfillment").matches("physical") on a HeadersExchange. Use headers when the key would be a lossy encoding of several fields. Do not use headers as a junk drawer of domain objects.

Split on the way out, aggregate on the way back

The splitter’s output is a message per line. split with a function that returns the collection is the DSL. Each emitted message needs the headers the aggregator will require. Set them in a transform immediately after the split, or put them on the line object and copy them to headers before the aggregate. Headers survive a channel. A field buried in a payload the next translator replaces does not.

javaCopy
@Bean
IntegrationFlow splitOrder() {
    return IntegrationFlow.from(MessageChannels.direct("orders.in"))
        .split(OrderMessage.class, OrderMessage::lines)
        .enrichHeaders(h -> h.headerFunction("orderId", m -> ((LineMessage) m.getPayload()).orderId())
                              .headerFunction("lineCount", m -> ((LineMessage) m.getPayload()).lineCount()))
        .channel("lines.in")
        .get();
}

The aggregator completes only when the group size matches lineCount. expireGroupsUponCompletion(true) drops the group so a duplicate delivery does not append to a finished order. groupTimeout is the warehouse’s silence. sendPartialResultOnExpiry(false) means a timeout does not look like success. The timeout handler publishes a OrderIncomplete the order service can show to support. If you set sendPartialResultOnExpiry(true), the downstream must read a header that says the group expired, and the order service must not treat that message as OrderComplete. Pick one and test it.

javaCopy
@Bean
IntegrationFlow aggregateLines() {
    return IntegrationFlow.from(MessageChannels.direct("line.results"))
        .aggregate(a -> a
            .correlationStrategy(m -> m.getHeaders().get("orderId"))
            .releaseStrategy(group -> {
                Object expected = group.getOne().getHeaders().get("lineCount");
                return expected instanceof Integer count && group.size() == count;
            })
            .groupTimeout(Duration.ofMinutes(30).toMillis())
            .sendPartialResultOnExpiry(false)
            .expireGroupsUponCompletion(true))
        .channel("orders.complete")
        .get();
}

Two pods with the default in-memory store will each hold a fragment of the same orderId. Use JdbcMessageStore or RedisMessageStore as the MessageGroupStore, or key Kafka records by orderId and run a single logical aggregator consumer per partition so all lines of an order land in one place. A database table line_result (order_id, line_id, status) with a unique constraint is often clearer than a message store, and the “completeness” query is SQL you can read during an incident. The pattern is the same either way. The store has to be shared if the workers are not.

Kafka mapping, stated so you do not invent broker features

Partition key is not a routing key. The key decides order and which consumer in a group sees the record. It does not select a different consumer group. If digital and physical must scale independently, they are two topics, not two keys on one topic.

The ERP adapter as an anti-corruption layer

The adapter’s public method on your side accepts a WarehouseShipment you defined. Inside, it maps SKU to material number using a table the integration team owns, and it calls the ERP the way the ERP already works. The order service never imports the ERP client. A test of the adapter uses a recorded ERP response and asserts the mapping. A test of the order service uses LineShipped and never a plant code. When the ERP changes a code, one mapping row changes. When you have no mapping, the line goes to the review channel rather than out as a blank material number the ERP will accept and mis-pick.

Idempotency at this boundary is the receiver’s job. The ERP may acknowledge slowly and the adapter may retry. The request carries the line id the ERP can treat as an idempotency key if their API allows it. If their API does not, the adapter records “sent line id” in its own store before the call and does not send a second create. That store is the adapter’s, not a new column on orders called erp_sent_flag written by two processes.

Practical Implementation or Design Walkthrough

Problem

An order can contain course licenses and a physical book. Entitlement must not wait for the warehouse. The warehouse ERP must receive material numbers. Finance must hear that the order is complete only when every line has a terminal result, or must hear that it is incomplete after 30 minutes. A poison line must not block the partition. The order service remains the only writer of orders.

Requirements

• Split OrderAuthorized into line messages with orderId, lineId, lineCount, and fulfillment.
• Route digital and physical. Anything else goes to fulfillment.review and pages when the rate is non-zero.
• Correlate completions on orderId. Complete only when the count matches. On timeout, emit OrderIncomplete, not a fake completion.
• Translate SKU to material number only inside the warehouse adapter.
• Kafka for the durable facts (order.authorized, line.results) so finance can replay. The router may be a Spring Integration flow in a small service or a Kafka consumer. Do not require RabbitMQ unless you specifically need broker-side routing and deletion. If the platform standard is already Kafka, stay on Kafka and own the router. Adding a second broker for one pattern is an operational cost you have to justify.
• Idempotent handling on lineId in each worker. Dead-letter the poison line and continue the partition, as in Lesson 33.

Steps

1. Define the line message fields and the two downstream topic names. Publish one pact or schema for the line message. fulfillment is an enum, exact-matched in the contract.
2. Implement the splitter as a pure function tested without Spring: one order in, N lines out, correlation present. Then put it behind the endpoint.
3. Implement the router function the same way. A table test covers digital, physical, and unknown.
4. Decide aggregation state. If you run one consumer and key by orderId, document that parallelism for a single order is one. If you run many pods, put group state in a store keyed by orderId with a unique line id.
5. Write the adapter mapping as data, not as a switch statement of plant codes inside the order module. Unmapped SKU: review channel, no ERP call.
6. Integration-test with Testcontainers Kafka: publish one order, assert two topics received the right lines, feed two results plus a timeout for a third, assert OrderIncomplete and no OrderComplete.
7. Confirm the order service applies OrderComplete through its own consumer and its own transaction. The aggregator process does not hold the order JDBC password.

Failure handling

A crash after emitting two of three line messages is a partial split. The consumer of OrderAuthorized must be idempotent, and the splitter must tolerate a redelivery of the whole order: line ids are stable, so downstream unique constraints turn the duplicate lines into no-ops. A crash inside the aggregator after writing two results must not lose them. That is why the group store is a database, not a heap, unless you accept re-aggregation from the line.results log, which Kafka retention makes possible if you key and compact carefully. Replaying the log is the recovery. An in-memory group is a cache of the log, not the record.

An ERP timeout is a retryable failure of the adapter, with a limit, then a dead letter on that line. It does not roll back the digital entitlements that already succeeded. The incompleteness shows up in the aggregator timeout. Compensation, if the business wants the courses revoked because the book cannot ship, is a saga decision in the order service, triggered by OrderIncomplete. It is not a filter in the router.

Expected result

A mixed order produces independent digital progress and a warehouse request in the ERP’s language. Finance sees one completion or one explicit incompleteness. An unknown SKU is visible in the review channel. The order aggregate’s schema still has no plant code. You can point at each box and name the pattern, the store of its state, and the owner.

Enterprise Real-World Scenarios

Notifications as a splitter and a set of adapters

Design a Notification Platform accepts one “learner should hear this” command and must deliver email, push, and sometimes SMS. That is a splitter, not three copies of the accept API. Each channel is an adapter with its own timeout, its own idempotency key, and its own dead letter. The aggregator, if the product needs it, is “all required channels accepted or one failed,” and the completeness rule is the product’s, not “whatever returned first.” A canonical “Notification” object that includes every provider’s private response fields will leak vendor specifics into the accept API. Keep the accept contract small. Let each adapter translate. The claim-check pattern shows up when the body is a rendered attachment: the message carries the key in object storage, and the email adapter fetches it with the workload identity from Lesson 44. Do not put the PDF on the Kafka record and call it simple. The record will hit the size limit, and the log will retain the document for the topic’s retention.

Warehouse and ERP beside an e-commerce order

Design a Large-Scale E-Commerce Platform that still settles inventory in an ERP is this lesson’s walkthrough at a larger SKU count. The content-based router is how a marketplace seller’s line goes to a different adapter than a first-party book. Adding a seller must be a new route and a new mapping table, not a change to checkout’s domain. The aggregator’s timeout is a customer-visible “part of your order is delayed,” which means OrderIncomplete is a product event, not only an integration detail. The case study should draw the anti-corruption layer as a box with a mapping store, and it should say the order service does not call the ERP client. If the diagram shows checkout connected to the ERP by a synchronous arrow “for simplicity,” the design has pushed a slow, poorly available system into the purchase path. Route asynchronously unless a business rule truly requires the ERP’s answer before the HTTP response, and then the SLI includes the ERP.

Best Practices

• Name the pattern in the design review before the product. “Router” and “aggregator” tell you where state lives. “We’ll use Kafka” does not.
• Keep routers pure. Put side effects in endpoints and adapters.
• Put correlation id and completeness on purpose. Always set a group timeout and define the partial outcome.
• Share aggregator state if more than one pod can see the same correlation id, or partition so only one pod can.
• Translate at the edge. Do not leak ERP codes, acquirer codes, or provider response bodies into the core schema.
• Prefer two published languages to one canonical enterprise model.
• Use a claim check for large payloads. Authenticate the fetch.
• Idempotent receivers on the split piece’s id, not only the parent id.
• Dead-letter both poison messages and unroutable messages. Alert on both depths.
• Leave the system of record as the only writer. Integration publishes facts. It does not update another context’s tables.

Common Mistakes and Anti-Patterns

Treating Kafka partitions as a content-based router. A key chooses a partition inside one topic. Both digital and physical still land in the same consumer if there is one group. Use separate topics or a router that writes them.

An aggregator with no timeout. A lost line pins the group, memory grows, and the order never completes and never fails. Support sees silence. The timeout is the product behavior.

In-memory aggregation behind a horizontally scaled Deployment. Each pod holds a fragment, no group completes, and a redeploy wipes the fragments. Use a store or a partition key that co-locates the group.

Plant codes on the order aggregate. The anti-corruption layer was skipped for a deadline. The next ERP change is a checkout migration. Map at the edge.

One canonical Order XSD. Every team blocks on the schema committee, and the document accumulates optional fields nobody can delete. Published languages between pairs stay small enough to version, which Lesson 40 already asked you to do.

The integration service writes the order database. Two writers, no transaction, and the order team cannot explain a row. Publish OrderComplete and let the order service apply it.

Synchronous ERP calls inside checkout. The router became a distributed transaction. Latency and availability of the ERP become the purchase SLI. Split asynchronously unless the business accepts that coupling in the SLI.

Troubleshooting and Design Review

Symptom: digital entitlements arrive and the order never completes. Cause: the warehouse line’s orderId header does not match, lineCount is missing, or the aggregator store is split across pods. Check: one order’s three results and the group size in the store. Fix: the correlation contract, then the store. Do not “fix” it by completing on the first result.

Symptom: the warehouse receives course licenses. Cause: the routing key or the fulfillment field is wrong at the producer, or the default route points at the warehouse. Fix: unknown goes to review, and a contract test locks the enum. A default of “physical” is a silent mis-pick.

Symptom: ERP shows duplicate shipments after a timeout. Cause: the adapter retried a create the ERP does not idempotent, and the adapter has no sent-line store. Fix: the idempotency record before the call, or the ERP’s idempotency key if the API offers one. Do not retry blindly because the HTTP client retries.

Symptom: Kafka consumer lag grows on one partition whenever a single order is large. Cause: the splitter’s output is keyed so one order’s lines share a partition, which you wanted for aggregation, and one order has thousands of lines. Fix: accept the hotspot or aggregate from a table so the worker can be stateless and the key can be the line id. Say which hotspot you chose.

Symptom: a PDF blows the broker’s max.message.bytes. Cause: the claim check was skipped. Fix: store the object, send the key, restrict who can fetch it.

Design review. Where is the router’s decision tested without a broker? Where does an unknown type go? What is the aggregator’s completeness rule and timeout outcome? Where is its state when there are two pods? Which codes are forbidden on the order aggregate? Who is the only writer of each table? What happens to a line the ERP never acknowledges?

Interview Questions and Answers

Q1. What is a content-based router, and why is a Kafka consumer group not one?

A content-based router sends each message to one of several channels based on the message content, and it has an explicit destination for “no match.” A Kafka consumer group divides partitions of a single topic among its members. Every message on the topic is still in that topic. Nothing in the group subscription filters by a JSON field. To route, the producer writes to the chosen topic, or a process reads the input topic and writes to downstream topics. RabbitMQ’s topic and headers exchanges do evaluate a key or a header inside the broker. That is a router. A consumer group is a work-sharing mechanism.

Q2. What does an aggregator need that a splitter does not?

State, a correlation id, a completeness condition, and a timeout policy. The splitter can be a pure function: one order becomes N line messages if the line ids and the count are on each message. The aggregator must remember which lines have arrived, across crashes if completion matters, and it must decide what a missing line means after the timeout. Without a timeout the group waits forever. Without shared state, two pods never agree that the group is complete.

Q3. Why must split line messages carry a stable line id?

Redelivery is normal. The entitlement worker and the ERP adapter need an idempotency key smaller than the order, or a retry of line 2 creates a second entitlement and a second shipment. The aggregator needs the same id so a duplicate result does not increment the group size twice. orderId alone is the correlation, not the idempotency key of the piece.

Q4. How do you map publish-subscribe and point-to-point onto Kafka and RabbitMQ without lying?

Kafka: one group is one set of competing readers, and other groups are further subscribers; the log is retained, so this is not deletion. RabbitMQ: one queue with competing consumers is point-to-point and ack deletes; a fanout or topic exchange with a queue per service is publish-subscribe. If the business needs replay, the log is the fit. If the business needs “exactly one worker takes this job and the broker forgets it,” the queue is the fit. Lesson 33’s enrollment-mail versus order-fact split is the same choice.

Q5. What is an anti-corruption layer, and where does the mapping table live?

It is a translator at the boundary of a system you do not control, so that system’s model does not enter your aggregate and your APIs. The SKU-to-material and warehouse-to-plant tables live with the adapter, owned by the integration team. The order service publishes and consumes your published language (LineShipped with a SKU). A plant-code field on Order means the layer was bypassed. Tests of the order service do not mention the ERP type.

Q6. Why is a company-wide canonical order model a risk?

Every context’s change becomes everyone’s schema change. Optional fields accumulate, versioning stalls, and the model still fails to stop translators because the ERP will not adopt it. A published language per relationship stays small enough to version and to pact-test. Share a model only where two contexts are actually the same domain and the same team. “We might need it later” is how the XSD grows.

Q7. What is a claim check, and when do you use it?

The message carries a reference, not the heavy body. You use it when the payload is a document, an image, or anything that would dominate broker retention and message size limits. The consumer fetches the bytes with its own authorization. A public bucket URL in the message is not a claim check. It is a data leak with extra steps. The broker then retains the key, which is what you can stand to replay.

Q8. How does this differ from a saga?

A router and a splitter do not own business compensation. An aggregator knows whether a set of messages is complete. A saga process manager knows what the business should do when a step fails: release the reservation, reverse the capture, keep the digital line. OrderIncomplete can be the signal that starts a saga in the order service. The integration flow should not also cancel payments on its own. One owner of the business state.

Q9. You scaled the aggregator to three pods and completions stopped. What happened?

The group state was in memory, local to each pod, and the three line results landed on three different instances. No instance ever saw lineCount. A redeploy made it worse by dropping the partial groups. Fix by storing groups in a shared MessageGroupStore or a table, or by partitioning the input so one consumer owns a given orderId. Then add the timeout you also need, because the scale fix does not create the warehouse’s missing line.

Q10. Where do these patterns appear in the notification and e-commerce case studies?

The notification platform splits one command into channel adapters, isolates vendor payloads in those adapters, dead-letters a single channel without losing the others, and uses a claim check for attachments. Completeness is a product rule. The e-commerce platform routes first-party and marketplace lines to different adapters, keeps ERP codes out of the order aggregate, and emits an explicit incomplete event when the warehouse is silent. Neither case study puts a synchronous ERP or email-provider call on the purchase hot path unless the SLI names that dependency. Neither one updates another service’s database from the integration process.

Hands-On Exercises

Exercise 1: Design the mixed-line flow

Problem. Specify the integration for an order that contains licenses and a physical book, plus an ERP that speaks material numbers.

Tasks.

1. Name each pattern and the component that owns it. State who writes the order tables.
2. Define the line message fields, the correlation id, and the idempotency key.
3. Choose Kafka topics or RabbitMQ exchanges for the router, and justify the choice with replay or deletion, not with fashion.
4. Write the completeness condition and the timeout outcome in one sentence each.
5. Say where aggregator state lives when there are two pods.
6. Draw the anti-corruption boundary: which types may exist on which side of it.
7. List two failures (duplicate line, unknown type) and the channel or constraint that absorbs each.

Deliverables. A diagram, the field list, the routing decision, and the timeout behavior.

Acceptance. An unknown fulfillment type cannot reach the ERP. A redelivered line cannot double-ship. The order schema contains no plant code. A missing warehouse result becomes OrderIncomplete, not a silent wait.

Exercise 2: Review an “ESB” proposal

The proposal puts checkout, the ERP client, and payment capture in one Spring Integration flow that updates all three databases, uses an in-memory aggregator with no timeout, routes unknown types to the warehouse binding, and embeds PDFs in the Kafka record. Write the review. Name the pattern being violated and the replacement.

Rubric.

Lesson Summary

Integration is a set of named patterns with owners, not a broker slogan. Channels are point-to-point or publish-subscribe, and Kafka and RabbitMQ implement those differently: retention and groups on one side, exchanges and ack-deletion on the other. A content-based router evaluates a predicate and has a review destination; a consumer group does not. A splitter emits correlated pieces with stable ids. An aggregator is shared state plus a completeness rule plus a timeout that cannot look like success by accident. Translators and adapters, built as an anti-corruption layer, keep ERP and provider models out of the core. A canonical schema for the whole company usually recreates the coupling you meant to leave behind. Large bodies become claim checks. The order service stays the only writer of its tables. Notification fan-out and mixed warehouse fulfillment in Module 5 are these patterns with different adapters, not new theory.

Further Reading

• Gregor Hohpe and Bobby Woolf, Enterprise Integration Patterns — channel, router, splitter, aggregator, translator, adapter, dead-letter channel, claim check. The online catalog is the index.
• Spring Integration, Java DSL and the aggregator — IntegrationFlow, route, split, CorrelationStrategy, ReleaseStrategy, MessageGroupStore.
• RabbitMQ, exchanges — direct, topic, fanout, and headers, and what a binding matches.
• Apache Kafka documentation on topics and consumer groups — what the broker will not do: it will not route by JSON content.
• Eric Evans, Domain-Driven Design, the anti-corruption layer and published language — the modeling half of the translator.
• Lesson 33 in this course, Message Queues and Event Streaming — dead letters and why a group is not a queue that deletes.
• Next: Architecture Migration and Modernization, where these adapters are also how a new service talks to what you have not extracted yet.
• Module 5: Design a Notification Platform and Design a Large-Scale E-Commerce Platform.