---
abstract: 'We present the Epidemic routing protocol implementation in ns-3. It is a full-featured DTN protocol in that it supports the message abstraction and store-and-haul behavior. We compare the performance of our Epidemic routing ns-3 implementation with the existing implementation of Epidemic in the ONE simulator, and discuss the differences.'
author:
- 'Justin P. Rohrer and Andrew N. Mauldin'
bibliography:
- 'wns3.bib'
- '../bib/jprohrer.bib'
- '../bib/master.bib'
- '../bib/rfc.bib'
title: |
    Implementation of Epidemic Routing with\
    IP Convergence Layer in ns-3
---

&lt;ccs2012&gt; &lt;concept&gt; &lt;concept\_id&gt;10003033.10003039.10003045.10003046&lt;/concept\_id&gt; &lt;concept\_desc&gt;Networks Routing protocols&lt;/concept\_desc&gt; &lt;concept\_significance&gt;500&lt;/concept\_significance&gt; &lt;/concept&gt; &lt;concept&gt; &lt;concept\_id&gt;10010147.10010341.10010342&lt;/concept\_id&gt; &lt;concept\_desc&gt;Computing methodologies Model development and analysis&lt;/concept\_desc&gt; &lt;concept\_significance&gt;500&lt;/concept\_significance&gt; &lt;/concept&gt; &lt;/ccs2012&gt;

Introduction
============

In this work we present the implementation of the Epidemic routing protocol [@vahdat:2000:ERF] in ns-3 [@ns3:www]. While several routing protocols are available in ns-3, it does not currently include any protocols, and development of such protocols typically proceeds with other simulators, such as The ONE simulator [@Keranen:2009:OSD:1537614.1537683]. In various routing simulation efforts [@pospischil:2017:MRO; @rohrer:2017:EOG; @rohrer:2016:GAR] we have encountered the need to simulate both s and s in the same simulator, and so have developed an architecture for integrating the two. Since there are no other DTN routing protocols in ns-3, we evaluate our results by comparing against the same simulation run in the simulator.

routing protocols typically deal with “messages” as opposed to “packets”. Messages may range in size from a few kB to hundreds of MB, which means they need a mechanism for fragmenting and reassembling these messages on a hop-by-hop basis to meet lower layer requirements. routers also buffer messages if a next-hop is not available, and may replicate messages to multiple next hops. These features differentiate routing protocols from or traditional routing protocols. This also leads to various design architectures. s are often implemented as an application-overlay network with Convergence Layer Adapters (CLAs) for particular network types, instead of a native layer-3 routing protocol. The application-layer approach (e.g. the Bundling Protocol [@rfc5050]) has advantages in ease of development and flexibility, but prevents the overlay from interoperating with traditional IP applications. Due to these limitations, we implement routing protocols as native ns-3 routing protocols so they can be used with the existing ns-3 application classes. In addition we have designed an convergence architecture that supports messages made up of multiple packets. Our implementation includes application classes to generate such messages. All of this code is available from our website[^1], and we are in the process of contributing it to the ns-3 distribution.

Prior Work {#sec:Background}
==========

This is not the first attempt at implementing the Epidemic routing protocol in ns-3. In 2015 Alenazi [*et al*. ]{}published their ns-3 implementation of Epidemic [@Alenazi:2015:ERP:2756509.2756523], the code for which began the review process for inclusion in ns-3 in 2013, however that process appeared to stall in 2015, and the authors did not respond to our inquiries about the status. The existing code however was available online and we attempted to use this version for our DTN simulations, however limitations in this implementation rapidly became apparent and necessitated developing our own implementation. Since Alenazi’s implementation routes only atomic packets, not complete messages, we will refer to his implementation as PacketEpidemic in the remainder of the paper, for ease of reference.

The PacketEpidemic implementation for implements the Epidemic logic discussed in [@vahdat:2000:ERF], however it possesses many limitations. As mentioned, PacketEpidemic handles individual packets instead of messages. PacketEpidemic’s node discovery mechanism results in incorrect operation for nodes with large buffers, where the nodes repeatedly resend messages that have already been transferred. Also, PacketEpidemic does not support control message fragmentation, so control messages are limited to the size of one packet. Large message buffers can generate summary vectors that exceed the size of a packet and when this happens the simulation crashes. The lack of control packet fragmentation restricts the number of messages that a node can handle. Our Epidemic implementation addresses the need for control packet fragmentation and as a result, the control packet headers, node discovery, and data handling, are significantly different from PacketEpidemic.

Code Structure {#ssec:CodeStruct}
==============

We note that implementing Epidemic routing was not our end goal, but a stepping stone to the implementation of many other DTN routing protocols in ns-3, so some design decisions take into account not just epidemic, but other protocols as well. We borrowed the top-level code structure from Alenazi’s Epidemic [@Alenazi:2015:ERP:2756509.2756523], so all routing protocols are broken into four main classes. Our Epidemic implementation forwards groups of data packets called messages instead of atomic packets. Section \[sec:MessageGeneration\] describes how the protocols generate messages and segment them into individual packets. This section provides an overview of the code structure. The routing protocol contains a group of packet classes, packet queue class, queue entry class, and routing protocol class. The diagram in Figure \[fig:DTNUML\] shows the relationships functions and attributes used by all Epidemic protocols to implement logic. We also borrow from our prior experience implementing DSDV for  [@narra:2011:DSD].

\[t\]

[IPv4 Routing Protocol]{}[0,0]{}

[Routing Class]{}[0,-1.75]{}

[Packet Queue Class]{}[8,0]{}

[Queue Entry]{}[12,0]{}

[Packet Classes]{}[4,0]{}

Packet Classes {#sec:packetclass}
--------------

The packet classes are the packet header declarations used by the protocols. Each packet header is its own class because the packet header is a data structure that defines a packet. The packet classes inherit ’s `Header` class. The `Header` class defines a packet and provides the interface for other classes to interact with the packets. The routing class and packet queue class interact with the packet class to read and write packet headers. Since packet headers have various types of information, they have their own accessors and mutators. However, all packet header declarations require a `Serialize` and `Deserialize` function because of the `Header` class. The `Serialize` function writes the header information to a packet buffer, and the `Deserialize` function reads from a packet buffer. These functions are required because passes packets between nodes as byte arrays. When a node receives a packet, it uses the `Deserialize` function to obtain the header information. The `Print` function permits another class to print the header to the screen or log file.

Packet Queue and Queue Entry Classes {#sec:packetqueue}
------------------------------------

The Packet Queue Class manages a node’s message buffer. The Routing Protocol class interacts with the Packet Queue class to manage messages and generate control packets. The Packet Queue class implements a protocol’s buffer management scheme, something which is much more significant in other DTN protocols. The `m_BufferSize` attribute defines the maximum size of the message buffer in bytes. `Enqueue` adds messages to the buffer, and `Dequeue` removes messages from the buffer. After a node adds a message to the message buffer, `DropExpiredMessages` removes expired messages. If the message buffer is full, `Purge` removes messages according to the protocol’s queue management scheme. `FindDisjointMessages` generates the list of messages to replicate according to a protocol’s message priority. The `Drop` function removes a selected message from the buffer. The `GetSize` function returns the number of messages in the buffer.

Since messages are groups of packets, the message buffer requires a data structure to group packets. The `m_queue` is a map matching a message ID to a queue entry defined by the Queue Entry class. A queue entry is a data structure that stores the packets belonging to a message, the header, expiration time, and message ID. The `GetMessageByteSize` returns the number of packets contained in message. The `GetMessagePacketTotal` returns the total number of packets belonging to a message. The `GetCurrentPktCnt` returns the number of packets currently contained in the queue entry. The `GetPacketSize` returns the size a data packet in a message. `AddPacket` adds a packet to the queue entry. `GetPackets` returns all of the packets contained in a message. The Packet Queue uses the Queue Entry functions to generate control packets, manage the message buffer, and retrieve messages for the Routing Protocol class.

Routing Protocol Class {#sec:routingclass}
----------------------

The Routing Protocol class inherits from ’s `Ipv4RoutingProtocol`. The Routing Protocol class implements the control logic of the protocols. The `Recv<Protocol>` function executes the control packet exchange based on the packet headers defined in the packet class. The Routing Class initializes the packet queue. The routing class interacts with the packet queue class to generate control packets, and the routing protocol class interacts with the packet queue class to store and retrieve messages. The `SendBeacons` function transmits beacon packets at the specific interval. The `SendDisjointMessages` calls the `FindDisjointMessages` from the Packet Queue class to generate the list of message to transmit to another node. Then `SendDisjointMessages` calls `SendMessageFromQueue` to transmit the messages from the generated message list. When a node receives a packet, `RouteInput` determines the interface, buffer, or function to execute. `RouteInput` handles the logic for buffering incomplete messages and acknowledging messages described in Section \[sec:MessageGeneration\]. `RouteOutput` handles packets leaving a node.

Message Generation and Handling {#sec:MessageGeneration}
===============================

The and handle node data differently. This section discusses how and the implement message handling and generation. The discussion includes the differences between the PacketEpidemic implementation and our Epidemic implementation to handle messages, and application-layer message-traffic generation.

 Behavior
---------

In the , traffic generation uses messages. The does not have packets like networks, so the does not have packet header definitions. Messages are similar to bundles from the Bundling Protocol [@rfc5050] because messages are the base unit of data. Unlike packets, messages can be any size. The handles messages as a single object. The does not fragment messages, and the does not permit partial messages to propagate throughout the network. If a node does not completely receive a message, then the node drops the partially received message. In the , messages do not carry control instructions, and nodes share control information by directly accessing another node’s data structures in memory. The does not include the overhead of exchanging control instructions in its simulation results.

 Behavior
---------

Since implements the entire network stack, our protocols define groups of packets generated by one source node destined to another node as a message. A message is equivalent to RFC-5050’s bundles as the base unit of data. The packets used to share routing information between nodes are control packets. Control packets are not messages because they are routing protocol specific, so they are not the base unit of data. Our protocols assume an -based convergence layer. Unlike the , cannot generate a message as a single object of any size, so large messages are segmented into groups of individual packets. Each packet is assigned a header with a custom identifier that associates each packet with the rest of the message. This architecture integrates with the existing code base, and does not require the modification of protocols below the routing layer. The following subsections discuss how defines and handles messages. The discussion includes the differences from PacketEpidemic message handling, and our message-traffic generation.

Message Definition
------------------

Our data packet header shown in Figure \[fig:MsgHdr\] is the custom header that identifies a packet belonging to a message. The Bundling Protocol ( 5050) [@rfc5050] influenced our data packet header design, but our protocols do not implement the Bundling Protocol. The PacketEpidemic implementation does not have a header for message layer, instead, it uses only packets as the base unit of data with the address identifying source and destination nodes.

Each data packet header contains a Message Identification Number, last hop, packet count, and packet index. The Message Identification Number in Figure \[fig:MsgID\] is a 64-bit unsigned integer derived from the source Node Identification Number and timestamp. The first 16-bits of the Message Identification Number are the source Node Identification Number. The last 48-bits are the message’s generation timestamp in microseconds. The source Node Identification Number used by the packet header is the node number from , which automatically assigns a unique integer to every node in the simulation.

[32]{}\
&\
\

[32]{}\
\
& [ 0 ]{}\
[ 0 ]{} & [ 0 ]{}\
[ 0 ]{}\

The range of scenarios we expect to simulate require a node to generate several messages in a millisecond at most, but will not require generation of more than one message in a microsecond. The timestamp is in microseconds to ensure that every Message Identification Number is unique. While a per-node sequential message counter would also create a unique identifier, our protocols require message generation time and source node identification for routing decisions, so this information is dual-purpose. Some scenarios studied would exceed the largest time in microseconds represented by a 32-bit timestamp, therefore we use 48-bits for the timestamp. The last hop field is the 16-bit Node Identification Number of the last node that forwarded the message. Nodes use the last hop to buffer incomplete messages for message reconstruction. The 32-bit total number of packets and the 32-bit packet index support reordering during message reassembly.

Message Handling
----------------

In PacketEpidemic, nodes pushed all packets selected for transmission to the link layer immediately after completing the control packet exchange. ’s link layer contains a packet queue that has a limited size. This link-layer packet queue is and cannot be manipulated by higher layers in the network stack, they can only add packets [@NS3Manual]. When a node’s message buffer equals or exceeds the size of the packet queue, then the node will fill the packet queue and any new packets sent to the full queue are dropped. The link layer will attempt to transmit the packets in order regardless if the destination node is connected. If the first node moves out of range, then the link layer will still transmit the packets. As a result, the node wastes available bandwidth and meeting opportunities. This behavior is not a requirement of the Epidemic routing protocol, but is a limitation specific to the PacketEpidemic implementation.

In order to improve link utilization, `ACK` packets in Figure \[fig:DTNAck\] control the message exchange sequence. `ACK` packets do not acknowledge messages reaching their final destination, rather, they are a hop-by-hop acknowledgement of messages transmitted between two connected nodes. When a node receives a complete message from another node, it sends an acknowledgement packet consisting of the 64-bit Message Identification Number, 16-bit Node Identification Number, and 16-bit Message Status. The Message Identification Number is the received message’s message ID. The Node Identification Number is the node that received the message. The Message Status block permits adding reliability in future work. The Message Status indicates if a node received a message successfully. If the message transfer is unsuccessful, then Message Status is zero. We do not retransmit such messages, but such reliability may be a desirable future enhancement and is supported by the header.

[32]{}\
\
&\

After the transmitting node determines message priority, the node fetches the first message. A node transmits the complete message and waits for an `ACK` before sending the next message. The receiving node queues the message’s packets in a message reception buffer. The receiving node maintains a message reception buffer for each node that is connected. Each connection’s receiving buffer can buffer only one message. When a message is complete, the receiving node transmits an `ACK`. `RouteInput` handles message reconstruction and `ACK` generation. Upon receiving the `ACK`, the transmitting node transmits the next message. The `Recv<Protocol>` function handles `ACK` reception. If the connection breaks, then the node resets the message reception buffer for that connection. A node considers a connection broken when the node does not receive any packets from that neighbor for two beacon intervals. When a node does not receive a complete message, it deletes the partial message. As a result, nodes do not forward partial messages.

Deleting partial messages may seem wasteful, but dropped messages due to a lost packet occurs infrequently ([*e*.*g*. ]{}less than 0.1% of the time in the Helsinki scenario). While `ACK`s improve link efficiency, they also have limitations. First, a node requires a large link layer buffer because the buffer must store all of the packets contained in a message. When the link layer’s packet buffer exceeds the packet limit, the node removes the newest packets from the queue. As a result, those packets fail to transfer causing messages to drop. ’s link layer limits the number of packets that can be stored in the buffer, so the scenario configuration file must increase the link layer’s buffer to accommodate the expected message sizes. Second, the link layer buffer deletes packets based on a time-limit. When a packet exceeds the time-limit, the link layer buffer deletes the packet. For the purposes of this work, the example scenarios set the link layer buffer to the size of the node’s message buffer. The scenarios set the packet queue time-limit to two beacon intervals because two beacon intervals correspond to a dropped connection.

Message Generation
------------------

The Application generates message traffic using the Packet Header, and is adapted from the On-Off Application. The Application generates packets with the Packet Header. The message size parameter in bytes and packet size determines the number of generated packets. The Application generates one message according to the entered parameters. The Message Identification Number uses the source Node Identification Number and message generation time. If a scenario requires more than one message, then the scenario must include multiple Application message generators.

Since future work may use packets instead of messages for data traffic, our protocol implementation is backwards compatible with raw packets. When a node generates a packet that does not have the Packet Header, the DTN router on that node creates a Packet Header for that data packet. `RouteInput` handles Packet Header generation for standard packets. The Message Identification Number uses the packet’s generation time in microseconds and the packet’s source Node Identification Number to generate the Message Identification Number. A node then treats each packet as an individual message, permiting ’s default packet generators to work with the protocols.

Node Discovery {#sec:NodeDiscovery}
==============

DTNs do not have constant connectivity, so nodes must discover other nodes to initiate a connection. Unlike , the simulator handles node discovery independently of any routing protocol. As a result, the DTN protocols must include a node discovery mechanism. In PacketEpidemic, nodes transmit `BEACON` control packets at a specific time interval for node discovery. The `BeaconInterval` defines the frequency of beacon transmission. To ensure that all nearby nodes can receive the beacon, the node broadcasts a beacon using the network’s broadcast address. Nodes are likely to be synchronized in sending broadcasts, resulting in lost beacons due to collisions. In order to prevent synchronization among nodes, a uniform random variable staggers the beacons. The `BeaconRandomness` variable defines the upper bound of the uniform random distribution to add to the base beacon interval. Since each beacon starts the exchange process between nodes, nodes with many packets buffered may already be exchanging packets when the next beacon interval occurs. The `HostRecentPeriod` prevents hosts from re-exchanging redundant control packets [@Alenazi:2015:ERP:2756509.2756523], however nodes with large buffers will exceed the `HostRecentPeriod` used PacketEpidemic resulting in the nodes re-exchanging control packets and messages while already transferring messages. This causes messages to be sent and re-sent multiple times wasting bandwidth.

Our Epidemic implementation also uses beacons for node discovery. A node broadcasts beacons at a set interval plus a random delay to minimize beacon collisions. However, we do not use a `HostRecentPeriod` to prevent hosts from re-exchanging redundant control packets, instead each node remembers the neighbors with which it is currently in contact. Every time a node receives a control packet or data packet from another node; it updates its record of connected nodes. A node considers a connection broken when the node does not receive anything from the neighbor for two beacon intervals.

A node can have more than one radio interface and address, so address is not a sufficient identifier to prevent two nodes from restarting a connection when they detect the second radio interface. The `BEACON` control packet is a `MessageType` header with the field set to `BEACON`. The `MessageType` header field indicates the type of control packet. Since a node does not know whether another node has more than one address, the beacon must include the Node Identification Number. Therefore, the `MessageType` header includes the 16-bit Node Identification Number. Figure \[fig:DTNMtype\] illustrates the `MessageType` Header. provides a unique number to every node in a simulation. A node uses the Node Identification Number to determine whether a node is considered connected. If a node is not connected, then the node adds the Node Identification Number with a timestamp to its list of connected nodes. Then, the node continues the control packet exchange. If the Node Identification Number is in the list of connected nodes and the connected timestamp does not exceed two beacon intervals, then the node ignores the `BEACON`. However, every data packet and control packet other than `BEACON`s update the connection timestamp. A `BEACON` changes the timestamp only when the timestamp exceeds two beacon intervals because it is considered as a new connection.

[24]{}\
&\

Epidemic Logic {#sec:EpidemicDTN}
==============

The Epidemic control logic performs four main steps as shown in Figure \[fig:EpidemicDTNConM\]. After receiving a `BEACON`, nodes exchange `REPLY` and `REPLY_BACK` control packets. `REPLY` and `REPLY_BACK` control packets are the summary vectors discussed in [@vahdat:2000:ERF]. The goal of sending message summaries is to avoid sending messages that the other node already contains in its buffer. Since both nodes are likely to send a response to a beacon at the same time, an anti-entropy session prevents node responses from colliding. The node with the lower address sends its message summary first as a `REPLY` packet. When the node with the higher address receives the `REPLY` packet, it sends a response using the `REPLY_BACK` packet. After a node receives the list of messages that the other node contains, the node sends messages that the other node does not have in its buffer.

(0,0) circle \[radius=1.25\]; at (0,0) [A]{}; (6,0) circle \[radius=1.25\]; at (6,0) [B]{}; (.9,1.0) – (5.1,1.0); at (3,1.25) [`BEACON`]{}; (1.25,0.33) – (4.75,0.33); at (3,0.58) [`REPLY`]{}; (1.25,-0.33) – (4.75,-0.33); at (3,-0.08) [`REPLY_BACK`]{}; (.9,-1.0) – (5.1,-1.0); at (3,-0.75) [Messages]{};

Message Identification and Limits
---------------------------------

Since our implementation manages messages instead of packets, we use a different `EpidemicHeader` than PacketEpidemic [@Alenazi:2015:ERP:2756509.2756523]. Figure \[fig:EpidemicDTNEpcHdr\] illustrates the `EpidemicHeader`. First, the 64-bit Message Identification Number from Section \[sec:MessageGeneration\] replaced the 32-bit packet identification number. Since the Message Identification Number includes the source node and timestamp of message generation, we do not use a separate timestamp field.

[32]{}\
\
\

The uses hop limits and message to reduce network resource consumption. The lower 48-bit portion of the Message Identification Number permits the routing protocol to remove expired messages from a node’s buffer. When a node receives a message, the node checks the timestamp against the maximum age of a message. If the timestamp exceeds the amount of time a message can live, then the node discards the message. The `EpidemicHeader` 32-bit hop count field is used limit the number of hops that a message can traverse. When nodes generate a message, they initialize the hop count to the maximum number of hops allowed. At each hop, the node decrements the hop count and when the hop count reaches zero, the message is discarded.

Control Packet Identification
-----------------------------

The `MessageType` Header in Figure \[fig:DTNMtype\] identifies control packets. Since the simulator uses shared data structures to exchange routing information it did not implement control packets. In our implementation, control packets are not messages (the do not have a DTN message header). Control packets share routing specific information between two neighboring nodes. The 8-bit Message Type field indicates the type of control packet. For Epidemic, control packets are `BEACON`, `ACK`, `REPLY`, and `REPLY_BACK`. The `MessageType` Header encapsulates the control packets in order to identify the control packet. When a node receives a packet, it checks the Message Type field for the packet type to call the appropriate packet header class.

Summary Vector
--------------

While Epidemic uses the same control packet sequence as PacketEpidemic, the `SummaryVectorHeader` in Figure \[fig:EpidemicDTNSMVHdr\] differs. The `SummaryVectorHeader` defines `REPLY` and `REPLY_BACK` control packets. We use a 64-bit Message Identification Number in place of packetEpidemic’s 32-bit packet identification number. PacketEpidemic’s 32-bit Summary Vector Length counts all of the packets held by a node’s buffer. In our Epidemic implementation, the 16-bit Summary Vector Length counts the number of Message Identification Numbers in the `SummaryVectorHeader`. A 16-bit unsigned integer is large enough cover a `SummaryVectorHeader`. Since the `MessageType` Header encapsulates the `SummaryVectorHeader`, a node identifies the other node using the Node Identification Number instead of the address because nodes can have more than one address.

[32]{}\
&\
\
\
\

PacketEpidemic did not support control packet fragmentation. In order to support fragmentation, the Fragmentation Block identifies whether there are more `SummaryVectorHeader` packets. When the Fragmentation Block is one, more `SummaryVectorHeader` packets remain. When the Fragmentation Block is zero, that packet is the last `SummaryVectorHeader`. Once a node receives one with the Fragmentation Block set to zero, the node continues the message exchange sequence. The control packet fragmentation can tolerate packet loss if the lost packet had the fragmentation block set to one, but the fragmentation protocol does not support retransmission of control packets. If the lost control packet had the fragmentation block set to zero, then the control packet sequence would stop until a beacon restarts the exchange sequence.

Evaluation
==========

Our primary interest in implementing DTN protocols in ns-3 is to gain fidelity and account for the cost associated with control message exchanges and the overhead of layer-2 protocols, which can not be simulated in the and other popular DTN simulators. With this in mind, our reference point is the performance of the same protocol in the simulator. We note that this comparison would be impossible without support for segmenting messages into multiple packets, due to the use of relatively large messages as the base unit of transmission in the and other DTN simulators. We compare the protocols using three different mobility scenarios, all of which are run in the , which outputs and ns-2 mobility trace which in turn is input to ns-3. In this way, while there is still randomness in the mobility, there is no difference in the mobility of the nodes between the two simulators. Each parameter set is run 10 times with different seeds for the random number generator. The Helsinki mobility scenario is an urban environment, which is the default scenario for the simulator and hence appears commonly in DTN simulation literature. Bold Alligator is our interpretation of a US Marine Corps exercise, and Omaha is our interpretation of the historic amphibious assault at Omaha Beach during World War II. While we do not have room here to fully specify each scenario, they are presented in detail in LT Mauldin’s master’s thesis [@mauldin:2017:CAO].

Helsinki
--------

Table \[tab:HelEpdPerfDiff\] shows that ’s Epidemic returns lower , higher average latency, and lower message replication overhead. Both simulators show that Epidemic’s increases as buffer size increases. Link layer overhead and control packets reduce available bandwidth to share messages. The does not include the time or bandwidth consumed by control packets, so the version increases message replication because nodes have more time to share messages. ’s increased sensitivity to transmission speed and higher latency highlights the impact of the added overhead.

Bold Alligator
--------------

In and the , Epidemic’s and average latency increase asymptotically with buffer size. Epidemic’s average hop count increases from the 5 to 10 buffer, but Epidemic shows minimal change in average hop count for larger message buffers in both simulators. Message replication overhead in the and follow the same trend. Larger message buffers return minimal change in message replication overhead.

While both versions of Epidemic share trends, consistently returns lower , higher average latency, and lower message replication overhead. To illustrate this observation, Table \[tab:BAEpdPerfDiff\] contains the percent difference between and the . ’s is 66% to 74% percent lower than the . Average latency is 52% to 108% higher. As buffer size increases, the difference in average latency between and the increases. Message replication overhead is 19% to 31% lower in . Larger message buffers and higher speed radios increase the message replication overhead performance gap between simulators.

Omaha
-----

Epidemic’s and average latency increase asymptotically with buffer size. Message replication overhead decreases with larger message buffers, the shows a steep decrease compared to ’s shallow decrease in message replication overhead. Both versions of Epidemic show that average latency increases asymptotically with larger message buffers. Epidemic’s performance in is close to the in Table \[tab:OmEpdPerfDiff\]. ’s is within 35% of the , and the simulators match at several data points. At small buffer sizes, has lower latency than the , but large message buffers show has higher latency. With respect of message replication overhead, ’s message replication overhead is 60% to 83% lower than the . Higher transmission speeds increase the performance gap in message replication overhead between and the .

Aggregate Scenario Performance
------------------------------

This section compares the overall performance of and the using , average latency, average hop count, and message replication overhead. The bar graphs are an average of all buffer sizes, transmission speeds, and protocol data points for each scenario. Each bar graph uses a 95% confidence interval. Figure \[fig:SimAggMDR\] compares aggregate protocol and scenario between the and . Within Bold Alligator, delivers 61% fewer messages than the , and 24% fewer messages in Omaha. delivers 17% fewer messages in Helsinki. Across all scenarios and protocols, delivers 31% fewer messages than the .

Figure \[fig:SimAggLat\] compares aggregate average latency for protocols and scenarios between simulators. Epidemic’s average latency is 21% higher in than the . In relation to network overhead in Figure \[fig:SimAggOvd\], protocols with higher message replication overhead tend to return a smaller difference between simulators. Helsinki’s average latency in is 202% higher than the . Bold Alligator’s average latency is 160% higher in and Omaha’s average latency is 53% higher in . Across all scenarios and protocols, ’s average latency is 119% higher than the .

Unlike and average latency, message replication overhead in Figure \[fig:SimAggOvd\] displays different trends between military and urban scenarios. returns lower message replication overhead than the by 61%. However, ’s message replication overhead is 88% higher in Bold Alligator and 2.5% higher in Omaha. We note that the protocols increase message replication overhead due to messages circling within clusters of nodes. The protocols, except for Epidemic, do not have this behavior because nodes check whether a message traversed the other node. If the message already traversed the other nodes, then the node does not transmit the message. Helsinki does not form node clusters, so ’s Helsinki does not return the higher message replication overhead.

Messages circling within clusters does occur with the ’s Epidemic, so the ’s message replication overhead is 65% lower than the . Across all scenarios and protocols, ’s message replication overhead is 5% higher than the .

In summary, delivers fewer messages and experiences higher average latency than the . Protocols that share more data to make routing decisions tend to deliver fewer messages in than the . Message replication overhead depends on node mobility due to implementation differences. Scenarios that form clusters of nodes return higher message replication overhead in . Scenarios that do not form clusters of nodes return lower message replication overhead in .

Conclusions
===========

and the employ different levels of abstraction to simulate network protocols. The focuses on simulating the behavior of opportunistic routing protocols, so the abstracts everything below the routing layer. In contrast, simulates the entire network stack. The routing protocols require packets for messages, node discovery, and sharing routing information between nodes. The does not include link layer overhead, packet header overhead, or control packet overhead.

The sends data as a single object called a message, and the shares routing information by directly accessing communicating nodes’ memory data structures. Our protocols assume an convergence layer adapter by defining groups of packets that compose a message. Control packets share routing information between nodes in . When comparing the effective throughput of messages transmitted between two connected nodes, the addition of packet headers and link layer overhead reduces effective throughput by 40% to 70% relative to the ’s radio bandwidth. Depending on the scenario and protocol, packet headers added during message segmentation in make up 2% to 5.5% of all transmitted data. Depending on the routing protocol and scenario, ’s control packets can consume a significant portion of transmitted data. Control packets make up 0.1% to 33% of all transmitted data in . Protocols that share less information for routing decisions transmit fewer/smaller control packets. While control packets contribute to network overhead, message replication represents a larger fraction of network overhead. For protocols that use the same message limit algorithm, protocols with more control packets consume more power.

When comparing the and protocols, the protocols returned 31% lower and 119% higher average latency aggregated across all protocols and scenarios. Message replication overhead varies between scenarios. In Helsinki, the protocols return lower message replication overhead than the . Both simulators demonstrate sensitivity to buffer size. Larger message buffers return higher . In Helsinki, larger message buffers reduce average latency. Bold Alligator and Omaha show that larger message buffers increase average latency. shows greater sensitivity to transmission speed than the . In , higher transmission speeds return higher , lower average latency, and increase message replication overhead. Higher transmission speeds in the return small changes in , slightly reduces average latency, and increases message replication overhead. ’s increased sensitivity to transmission speed is due to control packets, packet headers, and link layer overhead.

Based on our findings, our future protocol development will continue in instead of the . While the permits faster development of new protocols, and ’s simulations take 25 to 50 times longer than the , and requires a separate program to analyze the large trace files; the ’s abstraction may not reflect actual protocol performance. The sharing of routing information via shared data structures subsidizes performance of protocols with very large control packets. Considering control packets transmission impacts protocol performance analysis and link layer overhead significantly, and also significantly reduces message replication, as shown in our results.

The authors would like to thank Dr. Robert Beverly for his feedback and contribution to improving the rigor of this work. We also thank the WNS3 reviewers for their constructive analysis that strengthened this contribution.

The work was funded in part by the United States Marine Corps and United States Navy. Views and conclusions are those of the authors and should not be interpreted as representing the official policies or position of the U.S. government.

[^1]: <https://tancad.net/projects/dtn-simulation/>
