1. Evaluating the Networking Performance of Linux-based
Home Router Platforms for Multimedia Services
Ingo Kofler, Robert Kuschnig, Hermann Hellwagner
Institute for Information Technology
¨
Alpen-Adria-Universitat Klagenfurt, Austria
WoMAN’11, Barcelona, July 2011
Ingo Kofler Linux-based Home Router Platform Evaluation

2. Motivation
Popularity of Linux-based network devices
No real-time OS but sufficient for home network deployments
Low development costs
Mature platform and code base
Availability of services and applications
Open platform for running additional multimedia services
What performance can be expected in terms of...
computing performance
memory throughput
application-layer networking throughput
... and where are the bottlenecks?
Ingo Kofler Linux-based Home Router Platform Evaluation

3. Examples of Multimedia Services
In-Network Adaptation of Scalable Video Content
Media-Aware Network Element (MANE) for H.264/SVC
Adapt scalable video directly on network device
Adaptation must be signaling-aware and stateful
Realized as proxy running as process in user space
Proxy adapts video bit rate according to wireless link conditions
Previous work → ACM NOSSDAV’08, ACM NOSSDAV’11
Ingo Kofler Linux-based Home Router Platform Evaluation

4. Examples of Multimedia Services
Serving Multimedia Content in the Home Network
Deployment as always-on home server
Storage attached via USB port
Serving videos, music and pictures to clients in the network
Open source server implementations available
Support for USB-based DVB devices for serving live content (DVB-T/S/C)
Ingo Kofler Linux-based Home Router Platform Evaluation

5. Examples of Multimedia Services
Sharing Social Content from Home
Using server at own premises to serve social content
To overcome privacy and copyright concerns in social networks
Recent work by Marcon et al. @ ACM NOSSDAV’11
Private content is stored on USB storage and served via HTTP
Proof-of-concept study indicates feasibility
Ingo Kofler Linux-based Home Router Platform Evaluation

6. Home Router Platforms
Software
openWrt Linux
Popular Linux distribution for embedded systems
Replacement for stock firmware
Supports a variety of network devices/models
Provides SDK for cross-compiling software
Large set of available software packages
Common software platform for comparison
Most recent release: Backfire 10.03 based on Linux 2.6.32 kernel
Deployed on representative router platforms
Running performance and network benchmarks
Ingo Kofler Linux-based Home Router Platform Evaluation

7. Home Router Platforms
Hardware Specifications
Vendor/Model Linksys TP-Link UBNT
WRT54GL WR1043ND RS Pro
System-on-a-chip BCM5352EL AR9132 AR7161
CPU MIPS32 4Kc MIPS 24Kc MIPS 24Kc
CPU clock 200 MHz 400 MHz 720 MHz
Inst./Data-Cache 16 KB / 8 KB 64 KB / 32 KB 64 KB / 32 KB
SDRAM size/clock 16 MB / 100 MHz 32 MB / 400 MHz 128 MB / 360 MHz
Flash capacity 4 MB 8 MB 16 MB
Networking 100 Mbps 1 Gbps 1 Gbps
Wireless 802.11b/g 802.11b/g/n 802.11a/b/g/n
Misc - USB USB, 3x mini-PCI
Ingo Kofler Linux-based Home Router Platform Evaluation

8. Platform Evaluation
Processing and memory performance
Determine processing performance and memory throughput
Comparison of results to modern system (Core i7 M620)
CoreMark benchmark
Focus on CPU performance using synthetic workload
List operation, matrix multiplication, state machine and CRC
Algorithms executed in a loop
Metric: Number of loop iterations per second
STREAM copy
Part of the STREAM benchmark suite
Copies array content element-by-element
Metric: Megabytes per second
Ingo Kofler Linux-based Home Router Platform Evaluation

9. Platform Evaluation
Processing and memory performance
Vendor/Model Clock CoreMark CoreMark STREAM copy
[MHz] absolute normalized [MB/sec]
WRT54GL 200 333 1.67 59
TLWR 400 701 1.75 231
RSPRO 720 1263 1.75 381
Intel Core 2 Duo E8400 3000 8,651 2.88 4,055
Intel Core i7 M620 2667 9,457 3.55 6,830
CoreMark: 7.5 to 28 times slower than modern desktop CPU (single core)
STREAM copy: 18 to 116 times slower
Ingo Kofler Linux-based Home Router Platform Evaluation

10. Platform Evaluation
Networking performance
Focus on application-layer performance
Evaluation using different traffic scenarios (UDP/TCP, incoming/outgoing)
Investigated impact of different parameters (block size, socket buffer)
Results obtained using iperf and netperf
Experiments performed with firewall rules disabled
Ingo Kofler Linux-based Home Router Platform Evaluation

11. Platform Evaluation
Outgoing and Incoming TCP
500
500
WRT54 WRT54
q TLWR q TLWR
RSPRO RSPRO
400
400
Throughput [Mbps]
Throughput [Mbps]
300
300
q q
q
200
200
q
q
qq q
q q q q
q q
q
100
100
q
q
q
0
0
0 50000 100000 150000 200000 250000 0 50000 100000 150000 200000 250000
Block size [Byte] Block size [Byte]
Ingo Kofler Linux-based Home Router Platform Evaluation

12. Platform Evaluation
Outgoing and Incoming UDP
500
300
WRT54 WRT54
q TLWR q TLWR
RSPRO RSPRO
250
400
200
Throughput [Mbps]
Throughput [Mbps]
300
150
q
200
q
q qq
q
q qq q
100
qq
q q
q q
q q q
q
q
q
100
50
q
0
0
400 600 800 1000 1200 1400 0 50 100 150 200 250 300
Block size [Byte] Transmit rate [Mbps]
Ingo Kofler Linux-based Home Router Platform Evaluation

13. Platform Evaluation
Observations
Choice of block size significantly influences throughput
Cannot be observed on typical PC-based systems
High overhead per system call
Conclusion: avoid small packet sizes (UDP) and block sizes (TCP)
Application-layer throughput far below nominal link speed
WRT54 only achieves 29 Mbps outgoing UDP throughput
418 Mbps outgoing UDP throughput on RSPRO
Ingo Kofler Linux-based Home Router Platform Evaluation

14. Platform Evaluation
Observations
Choice of block size significantly influences throughput
Cannot be observed on typical PC-based systems
High overhead per system call
Conclusion: avoid small packet sizes (UDP) and block sizes (TCP)
Application-layer throughput far below nominal link speed
WRT54 only achieves 29 Mbps outgoing UDP throughput
418 Mbps outgoing UDP throughput on RSPRO
Identify performance bottleneck using ftrace framework
Debugging and tracing facility in Linux kernel
Useful to determine function execution time
Execution traces of UDP sendto system call
Ingo Kofler Linux-based Home Router Platform Evaluation

15. Platform Evaluation
Impact of netfilter
UDP sendto causes around 200 function calls in kernel
Nearly half of the functions caused by the netfilter component
Re-run evaluations using kernel without netfilter
Ingo Kofler Linux-based Home Router Platform Evaluation

16. Platform Evaluation
Impact of netfilter
UDP sendto causes around 200 function calls in kernel
Nearly half of the functions caused by the netfilter component
Re-run evaluations using kernel without netfilter
600
WRT54 WRT54
WRT54 w/o netfilter WRT54 w/o netfilter
800
q TLWR q TLWR
500
q TLWR w/o netfilter q TLWR w/o netfilter
RSPRO RSPRO
RSPRO w/o netfilter RSPRO w/o netfilter
400
600
Throughput [Mbps]
Throughput [Mbps]
300
400
q q q
q
q q
q q
200
q
q q
q
q q q
q q
q q
200
q q q q
q
q q
100
q q
q q
q
q q
q
0
0
0 50000 100000 150000 200000 250000 400 600 800 1000 1200 1400
Block size [Byte] Block size [Byte]
Ingo Kofler Linux-based Home Router Platform Evaluation

17. Platform Evaluation
ftrace Results
Impact of netfilter
1/3 of execution time spent for netfilter code
Throughput increases up to 50 percent
netfilter used for many (often relevant) features like NAT, firewall, . . .
Ingo Kofler Linux-based Home Router Platform Evaluation

18. Platform Evaluation
ftrace Results
Impact of netfilter
1/3 of execution time spent for netfilter code
Throughput increases up to 50 percent
netfilter used for many (often relevant) features like NAT, firewall, . . .
Further investigations
Measuring time-consuming functions of UDP sendto syscall
Aggregate function to certain blocks/task of networking stack
Evaluations performed with disabled netfilter code
Ingo Kofler Linux-based Home Router Platform Evaluation

19. Platform Evaluation
Dissecting the UDP sendto syscall
20
Socket API
Buffer Handling
UDP
Significant part required
IP
Bridge Tx for buffer handling (buffer
Hardware Tx
15
allocation, copy from user
to kernel space)
Duration [usecs]
Checksum calculation
10
done in software
Potential for
5
improvements by
offloading
0
400 1450
UDP payload size [Byte]
Ingo Kofler Linux-based Home Router Platform Evaluation

20. Conclusions
Evaluation of Linux-based router platforms
Interesting platform for multimedia services
Deployment as Media-Aware Network Element
Modest application layer performance
Far below nominal link speed
Use large packet sizes (UDP) and large buffers (TCP)
Disable netfilter component if not really required
No hardware offloading, potential for improvements
Ingo Kofler Linux-based Home Router Platform Evaluation