embedded TCP/IP Packet forwarding leading to a lot of retransmits
<p>My goal is to forward packets received from gbit ethernet to a serial interface with help of a multicore microcontroller. I got it working, but the performance when forwarding TCP traffic is poor. I am looking for ideas on what might cause the poor TCP peformance.</p>

<p><strong>Current State</strong></p>

<p>The gbit-ethernet functionality has been implemented based on a library provided by the microcontrollers manufacturer. I did some minor adjustments only (removing mac filtering).
The Serial Interface has been written from scratch, is fully software based and reaches a line data rate of about 19 MBit/s. The packet error rate when transmitting ethernet packets is &lt;0.05%.</p>

<p>Both interfaces work based on packets and provide buffering for receiving and transmitting packets. They are connected via a bridge thread. On receiving a packet on Interface A, the bridge thread is notified, grabs the packet and forwards it to Interface B for transmission. Time measurements on all threads show that these operation take significantly less time than packet transmits on the physical layer.</p>

<p><strong>Poor Performance</strong></p>

<p>Using two microcontrollers, it is possible to build an ethernet-ethernet bridge with serial communication in between. Establishing connection to a DHCP server works well over the bridge. Speed tests show results of 1,1/0,3 MBit/s (Ookla) and ~5/~3 MBit/s (T-Online) Down/Upload on an average ping of 20 ms. When trying to load web sites, the page may finish loading in the next 2-3 minutes or not finish loading at all (connection is established tho, as the first few files of a website are always loaded). </p>

<p>When connecting two PCs over the bridge and running iPerf/jPerf for Bandwith tests, i expected to run into the systems physical limitation due to the 19 MBit serial link. I expected to achieve a TCP bandwidth > 10 Mbit/s. However, the bandwidth is low and instable, with a maximum of about 1 MBit/s, dropping down to &lt; 60 KBit/s.</p>

<p><em>When examining network traffic with help of wireshark i see a lot (a loooooot) of retransmissions and duplicate ACKs, but I have no idea what might be the cause of this.</em></p>

<p>I am certain the ethernet library works well (open source, provided by chip manufacturer). When artificially creating packets on one microcontroller and sending them over the serial interface into the network, wireshark displays all generated packets correctly, verifying the functionality of the serial interface. The por performance must be caused by the way TCP works in combination with the hardware setup, but i am out of ideas. Any suggestions?</p>

<p>I can provide wireshark files if needed</p>