Multi-Destination-Cast
Multiple-Destination-Cast (MDC) is a capability that allows Aeron Transport to deliver data to multiple simultaneous destinations from a single Publication. Multiple-Destination-Cast is an advanced feature of Aeron Transport, and this guide touches on the bare basics of how to develop a simple sample.
MDC Publications¶
Note
MDC Publications are able to provide roughly equivalent behavior to UDP multicast in environments in which there is no support for UDP multicast. Note that it is not true multicast - data is sent individually to each subscription, however, flow control capabilities are supported.
MDC Publications can run in either a dynamic and manual mode. With dynamic MDC Publications, a Subscription adds itself to the Publication dynamically at runtime. With manual MDC Publications, the Subscription must be explicitly added to the Publication.
Dynamic MDC Publications work much like standard Aeron Publication, although the configuration is inverted.
Type | Publication | Subscription |
---|---|---|
Standard | Channel points to Subscription | Channel/port on localhost |
Dynamic Multi-Destination-Cast Publication | Channel/port on localhost | Channel points to Publication |
Sample Dynamic MDC Publication¶
A full, multi-host sample can be found on GitHub. This sample creates a single publisher with a dynamic MDC Publication, along with two clients which subscribe to the MDC Publication. Each process is in a dedicated Docker container.
Media Driver Configuration¶
The Media Driver requires no specific configuration, however, you can optionally enable spiesSimulateConnection
if you require Publications to be able to publish data without any Subscriptions connected. The sample code does this as follows:
1 2 3 4 5 6 |
|
Publication¶
The MDC dynamic Publication uses a specific channel configuration, which at minimum contains:
aeron:udp?control-mode=dynamic|control=MDC_HOST:MDC_CONTROL_PORT
The control-mode=dynamic
tells Aeron that this is a Dynamic MDC Publication. The MDC_HOST
in the sample above must be the host on which the Publication
is running. Behind the scenes, Subscriptions connect to this host using the control port to construct the runtime flow, so the control port must be a port known to the clients.
The sample code constructs the Publication channel as follows:
1 2 3 |
|
In addition to the basic configuration, the flow control strategy can also be configured via the Channel. This includes flow control strategies with group based limits, where a minimum number of nodes must be connected before the Publication will move to connected status.
Subscription¶
Connecting an MDC Subscription to a dynamic Publication is simple - the channel configuration must include the MDC Publication host and a local endpoint to which the remote host will publish. The minimal configuration is:
aeron:udp?endpoint=LOCALHOST:0|control=MDC_HOST:MDC_CONTROL_PORT|control-mode=dynamic
As with the Publication, the control-mode=dynamic
tells Aeron that this is a dynamic MDC connection. The endpoint=LOCALHOST:0
refers to an ephemeral port on the localhost to which the MDC Dynamic Publication will publish data - this can be a predefined port as well. Finally, the control=MDC_HOST:MDC_CONTROL_PORT
tells Aeron how to connect to the remote dynamic MDC Publication.
The sample code constructs the Subscription channel as follows:
1 2 |
|
The sample runs two instances of the clients, each in a dedicated Docker host.
Sample output¶
The following is the lightly edited output of a sample run via a docker-compose up
:
mdc-subscriber-2_1 | 16:47:00.945 [main] INFO MultiDestinationSubscriberAgent - launching media driver
mdc-publisher_1 | 16:47:00.946 [main] INFO MultiDestinationPublisherAgent - launching media driver
mdc-subscriber-1_1 | 16:47:01.066 [main] INFO MultiDestinationSubscriberAgent - launching media driver
mdc-subscriber-2_1 | 16:47:01.082 [main] INFO MultiDestinationSubscriberAgent - connecting aeron; media driver directory /dev/shm/aeron-root
mdc-publisher_1 | 16:47:01.083 [main] INFO MultiDestinationPublisherAgent - launching aeron
mdc-subscriber-2_1 | 16:47:01.093 [main] INFO MultiDestinationSubscriberAgent - adding the subscription
mdc-subscriber-2_1 | 16:47:01.093 [main] INFO MultiDestinationSubscriberAgent - detected ip4 address as 10.1.0.3
mdc-publisher_1 | 16:47:01.093 [main] INFO MultiDestinationPublisherAgent - Media Driver directory is /dev/shm/aeron-root
mdc-publisher_1 | 16:47:01.095 [main] INFO MultiDestinationPublisherAgent - detected ip4 address as 10.1.0.2
mdc-publisher_1 | 16:47:01.107 [main] INFO MultiDestinationPublisherAgent - creating publication
mdc-subscriber-2_1 | 16:47:01.128 [mdc-subscriber] INFO MultiDestinationSubscriberAgent - starting
mdc-subscriber-1_1 | 16:47:01.162 [main] INFO MultiDestinationSubscriberAgent - connecting aeron; media driver directory /dev/shm/aeron-root
mdc-publisher_1 | 16:47:01.165 [mdc-publisher] INFO MultiDestinationPublisherAgent - Starting up
mdc-publisher_1 | 16:47:01.166 [mdc-publisher] INFO MultiDestinationPublisherAgent - appended 1
mdc-subscriber-1_1 | 16:47:01.170 [main] INFO MultiDestinationSubscriberAgent - adding the subscription
mdc-subscriber-1_1 | 16:47:01.170 [main] INFO MultiDestinationSubscriberAgent - detected ip4 address as 10.1.0.4
mdc-subscriber-1_1 | 16:47:01.201 [mdc-subscriber] INFO MultiDestinationSubscriberAgent - starting
mdc-publisher_1 | 16:47:03.167 [mdc-publisher] INFO MultiDestinationPublisherAgent - appended 2
mdc-subscriber-2_1 | 16:47:03.170 [mdc-subscriber] INFO MultiDestinationSubscriberFragmentHandler - received 2
mdc-subscriber-1_1 | 16:47:03.171 [mdc-subscriber] INFO MultiDestinationSubscriberFragmentHandler - received 2
mdc-publisher_1 | 16:47:05.167 [mdc-publisher] INFO MultiDestinationPublisherAgent - appended 3
mdc-subscriber-1_1 | 16:47:05.171 [mdc-subscriber] INFO MultiDestinationSubscriberFragmentHandler - received 3
mdc-subscriber-2_1 | 16:47:05.170 [mdc-subscriber] INFO MultiDestinationSubscriberFragmentHandler - received 3
...
Notes:
- the samples make use of a relaxed
IdleStrategy
configuration, which explains why there is a 3 to 4 millisecond gap between theappend
andreceive
. - the
mdc-publisher_1
has a log entry where itappended 1
, but the two clients do not show a correspondingreceived 1
log. This is because themdc-publisher_1
Media Driver hasspiesSimulateConnection
set to true, and themdc-subscriber-*
processes were not yet connected. They do receive 2 onwards.
Flow Control¶
The example above raises a question: what should the mdc-publisher
do if mdc-subscriber-1
starts to fall behind mdc-subscriber-2
? This behavior is configurable in Aeron by way of flow control.
Flow control can be configured to a default value within the Media Driver and then customized per channel.
Flow Control Types¶
Type | Description |
---|---|
max | Publication will be limited by the fastest Subscription. Slow consumers may lose data packets. This is the default in Aeron. |
min | Publication will be limited by the slowest Subscription. |
tagged | Publication will be limited by the slowest tagged Subscription within a group |
If the Publication is producing data at a pace beyond the level as defined by the flow control strategy, the Publication will be subjected to back pressure.
Max Flow Control¶
Configure max flow control by setting fc=max
on the Channel configuration, for example:
aeron:udp?control-mode=dynamic|control=MDC_HOST:MDC_CONTROL_PORT|fc=max
Or set the default with the Media Driver Context (note that multicastFlowControlSupplier
is used for both UDP multicast and Multi-Destination-Cast):
1 2 3 4 |
|
Min Flow Control¶
Configure min flow control by setting fc=min
on the Channel configuration, for example:
aeron:udp?control-mode=dynamic|control=MDC_HOST:MDC_CONTROL_PORT|fc=min
Or set the default with the Media Driver Context:
1 2 3 4 |
|
Additional options that can be set include group size. The example below sets a group size of 5 with g:/5
.
Group size controls the connected state of the Publication — for example, with a group size of 5, the Publication will only be considered connected if 5 Subscriptions are connected.
You can then have the system functional with at least 5 connected Subscriptions, the slowest of which sets the pace for all connected Subscriptions.
aeron:udp?control-mode=dynamic|control=MDC_HOST:MDC_CONTROL_PORT|fc=min,g:/5
This can also be defaulted in the Media Driver Context (here setting group size to 5):
1 2 3 4 5 |
|
Tagged Flow Control¶
Sometimes you need a finer level of control within Multi-Destination-Cast — for example, you may have a group of subscribers which should not lose data packets, but you may also have other subscribers connected that are not impacted by data loss and you do not want these loss accepting subscribers to hold all Subscriptions back with a min
flow control strategy. The tagged flow control strategy enables this.
The following configuration sets the flow control to tagged
and the group to 101
for a Publication channel.
aeron:udp?control-mode=dynamic|control=MDC_HOST:MDC_CONTROL_PORT|fc=tagged,g:101
For a Subscription to join the tagged
flow control with the 101
tag (and thus be subject to flow control, much like the min
strategy), it would need gtag
set as follows:
aeron:udp?endpoint=LOCALHOST:0|control=MDC_HOST:MDC_CONTROL_PORT|control-mode=dynamic|gtag=101
If the Subscription needs to join the same MDC Publication, but is not concerned with data loss (i.e. is not subject to flow control), then the gtag
can be removed:
aeron:udp?endpoint=LOCALHOST:0|control=MDC_HOST:MDC_CONTROL_PORT|control-mode=dynamic
The following is the equivalent for setting the Media Driver defaults:
1 2 3 4 5 |
|
As with the min
flow control strategy, a minimum group size can be set for the tagged
flow control. In this Publication channel example, the tag is set to 101, and the group size is set to 5:
aeron:udp?control-mode=dynamic|control=MDC_HOST:MDC_CONTROL_PORT|fc=tagged,g:101/5