This document discusses performance testing for Voice over LTE (VoLTE) networks. It outlines challenges in ensuring high quality VoLTE service and describes key aspects of the network that require testing, including the radio access network, evolved packet core, and IMS core. Specific tests are recommended to evaluate quality of service for VoLTE by analyzing call establishment, quality, handover performance, and more. Large-scale emulation is suggested to simulate real-world user and traffic conditions for comprehensive VoLTE performance testing.

1. Performance Test VoLTE
diversifEye
Large Scale Tunnel Requests
Use real phone Credentials
Stateful Application Requests Millions of GTP tunnels
with real application requests
MME
Emulated server
S11 Side responses
eNodeB S1u S5 SGi
SGW PDN GW
Fully load S1 - S1u interfaces
Delivering high quality Voice over LTE networks (VoLTE)
Introduction
One of the major challenges associated with Voice over LTE (VoLTE) for carriers is defining the Quality of Service (QoS)
features for the LTE network.
VoLTE is positioned as a high quality voice service enabling call quality equivalent to or better than a 2.5G/3G cellular call,
except the network is now all IP. The challenge for the carrier is to ensure each element of the network from the User Equip-
ment (UE), Radio Access Network (RAN), Evolved Packet Core (EPC) and IP Multimedia Subsystem (IMS) core are optimized
correctly to deliver voice with an uncompromising level of quality. In addition it will most likely be the first time all compo-
nents are tested with a mission critical application.
The Challenge
Voice over LTE will play a key part in determining a subscriber’s
Quality of Experience (QoE) on a 4G network. However, a QoS test
plan cannot be solely focused on voice and must include other
services, which include video and data. The VoLTE application will
compete with other applications for bearer paths and bandwidth.
A common reason for 4G carriers to test performance at an appli-
cation layer level is to ensure the QoS prioritization of the voice
element is accurate, but to also ensure that there is no negative
performance impact on other revenue generating sources which
include IMS enablers of presence and location.
With the proliferation of smartphones and tablet computing
devices, more applications are appearing which are enabled by Figure 1: VoLTE call quality is dependant on the LTE
the Session Initiation Protocol (SIP). The increasing capacity network handling thousands of concurrent SIP sessions
requirement on the management layer, plus the addition of appli- per second. Dedicated QoS for Voice in LTE will impact
Data applications running on the same limited band-
cations means QoS optimization for VoLTE is a two dimensional
width resources.
challenge in which both layers need attention.

2. VOLTE
The successful delivery of any application on a 4G network is dependent on the smooth transition of packets between
distinct elements on the 4G bearer path. With the focus on VoLTE, a number of test requirements immediately appear when
considering QoS.
At a basic level, performance testing will include call establishment, call billing and call termination procedures. However, to
understand the impact that variances in LTE network QoS settings have, carriers need to look to the actual subscriber or
Quality of Experience (QoE) performance measurement. An example subscriber QoE measurement which is impacted by
QoS settings is the individual call quality rating, which must be measured in real-time. This is achieved using subjective call
analysis, giving an insight into the actual audio quality over the duration of the call.
Defining a test strategy for VoLTE
4G network performance testing requires multiple approaches. This application note focuses in on the IP aspect. At an IP
packet level, there are many performance tests required with varying degrees of complexity. These complex test challenges
are overcome by using per flow emulation with stateful application flows.
The advantages of per flow stateful application traffic emulation
and performance testing are highlighted when it comes to testing
VoLTE QoS performance on mobility or in call handover scenarios.
To identify some basic LTE QoS performance test requirements an
example approach is to step through each element on the 4G
network bearer path:
Radio Access Network
enodeBs manage a number of complex processes which include
bandwidth management of the subscriber applications, manage-
ment of the User Equipment (UE) measurement reports, communi-
cation with the Mobility Management Entity (MME) and Serving
Gateway (SGW).
A critical factor in any cellular network is call continuity when roam-
ing. Call continuity success is based on the ability of the enodeBs to
implement call handovers. A basic LTE QoS test ensures connectiv-
ity between enodeBs, alongside packet latency measurements
between enodeBs. Tight control over QoS settings covering the
delivery of packets passing between enodeBs is critical, further
testing will include scaling to include multiple device handovers or
increased traffic on the enodeB interconnect path (X2).
Figure 2: 4G/LTE radio access networks are
LTE subscribers will have multiple bearer paths assigned to them, so smarter, with more functionality based on the
QoE performance tests should include as many different applica- reach of IP to the tower. QoS for VoLTE is more than
tions as possible. Per flow performance testing enables carriers analyzing power of the radio network, analysis
measure actual voice quality and to also show how the focused QoS must be performed at a packet level. Testing is not
settings on the dedicated voice bearer, enforced by the enodeB, limited to the call itself but must also include
impacts bandwidth management on the other application types. security attack mitigation type tests, assessing
The aim is to find a reliable QoS bearer path setting for VoLTE, whilst known vulnerabilities associated with IP.
maintaining connectivity for all other applications running.
Performance TESTING

3. UE Performance test eNodeB, SGW, PDN GW
UE eNodeB & mobile DPI devices :Test per application
flow QoE for voice, video and data.
UE emulator
Serving
diversifEye : Emulate multiple Gateway
applications per emulated UE Packet Data Network
Gateway + DPI devices
Stateful Applications
(Video, Voice, Web, Email, etc)
INTERNET
UE IPv4/6 UE IPv4/6
GTP TEID GTP TEID
diversifEye : Emulate application
diversifEye server responses or connect to
external 3rd party servers
Figure 3 : diversifEye provides large scale emulation of stateful application flows, by accurately representing UEs (IPv4/IPv6)
and parameters such as IMSI and user authentication details.
diversifEye per flow architecture and large scale test capabilities are not only used to test the scalability of the LTE EPC, but
provide vital details on a per subscriber Quality of Experience basis.
Evolved Packet Core (EPC)
A function of the LTE EPC is to scale to meet the increasing demands made by subscribers and their applications. In most
instances each subscriber device will have multiple bearer paths assigned, dependant on the application, each bearer path
will be assigned a QoS priority level.
Initial VoLTE application testing will assess that the application QoS quality level is correct and the network core is prioritiz-
ing VoLTE flows. Following on from that, it’s possible to examine performance on other applications.
Examination of QoS is not limited to a single user entity. The level of scalability that is achievable in the EPC is dependent on
the number of sessions the MME will establish, closely linked with the volume of traffic flows the SGW can cope with.
QoS performance testing has a knock on effect in terms of utilization on the control/signalling plane in LTE. See Shenick’s
companion note for more details http://www.shenick.com/media_lib/files/Shenick_Optimizing_4G_networks.pdf. Essen-
tially, QoS is put in place to handle the millions of session requests accessing unique applications connecting to global web
services. Large scale activity testing emphasizes the need for emulation testing in the EPC.
Scale is not the only limiting factor to VoLTE; performance testing needs to examine the Session Initiation Protocol (SIP), in
all instances there will be a large volume of concurrent SIP sessions. Detailed testing should outline the overhead associ-
ated with maintaining each of those sessions.
VoLTE Performance TESTING

4. As in the RAN, roaming UEs place additional resource requirements on the EPC. Once the enodeBs and the UE have finalized
handover, it is up to the target enodeB to communicate this message with the MME. It’s critical that the latency and the avail-
able memory buffers on interlinking paths (S1-MME) are set correctly. With packet latency performance, it’s often the case
that buffer sizes are set too aggressively which has a negative impact on QoS performance especially during a period of low
traffic requests. It only requires tens of milliseconds of latency to perceive a VoLTE call as poor quality.
Testing the management paths with varied traffic load scenarios will result in an optimized QoS setting, preserving VoLTE
call continuity.
UE SIP Metrics Description
The EPC MME manages both the interconnected enodeBs andInaRTP Bits/sec number of gateways in which the bearer traffic
UE varying The number of RTP bits/second received in by this UE.
passes through. Returning to the roaming subscriber’s call; call continuity
UE Out RTP Bits/sec is further dependent on the speed in which the
The number of RTP bits/second sent out by this UE.
UE In RTP Packets/sec
UE
to the user plane The number of RTP packets/second receivedby by this UE.
MME updates the SGW, with a message indicating the change Out RTP Packets/sec path for the UE. Latency measurements
The number of RTP packets/second sent out
in
this UE.
again are an important parameter to maintaining the VoLTE callRTP Out of Sequence
UE quality.
The number of packets out of sequence sent out by this UE.
Packets
UE RTP Dropped Packets The number of packets dropped by this UE.
Another important QoS feature in the LTE EPC concerns the speed in which the address redirection request in by this UE. out, it
UE Duplicate RTP Packets The number of duplicate packets received is carried
must be managed by the SGW so as to minimize the volume UE Out Calls Attempted
of traffic heading backnumber of calls out attempted by this UE. sample QoS
The to the source enodeB. A
performance optimization test will cover bandwidth utilization, Out Calls Established
UE
anCalls Rejected test isThe number ofan excessivethis UE.
UE Out
example to force calls established by number of redirec-
The number of calls rejected by this UE.
tion updates for the MME to handle. UE In Calls Attempted
The number of incoming calls that this UE attempted to
receive.
UE In Calls Established The number of incoming calls established by this UE.
As each request increases the CPU cycles required in the MME, this has an effect of The number ofthe MME’s abilitythis UE.
UE In Calls Rejected slowing incoming calls rejected by to manage the
SGWs. To determine what impact these loads have on performance, analysis is done covering the number of packets still
UE Calls Errored The number of calls with errors logged by this UE.
arriving to the UE address after the UE handover request has beenOut Messages by theThe number of SIP messages sent out byby this UE.
UE SIP
initiated
UE SIP Messages Resent
source enodeB.
The number of SIP messages resent out
this UE.
UE SIP In Messages The number of SIP messages received by this UE.
UE In RTCP Packets The number of RTCP packets received in by this UE.
UE Out RTCP Packets The number of RTCP packets sent out by this UE
UE SIP Metrics Description
UE Registrations Attempted The number of registrations attempted by this UE.
UE In RTP Bits/sec The number of RTP bits/second received in by this UE.
UE Registrations Successful The number of successful registrations by this UE.
UE Out RTP Bits/sec The number of RTP bits/second sent out by this UE.
UE In RTP Packets/sec The number of RTP packets/second received in by this UE. UE Registrations Rejected The number of registrations rejected by this UE.
UE Out RTP Packets/sec The number of RTP packets/second sent out by this UE. UE Registrations Errored The number of registrations with errors logged by this UE.
UE RTP Out of Sequence UE Calls Received Ringing The number of ringing calls received in by this UE.
The number of packets out of sequence sent out by this UE.
Packets UE Mean Time to Ringing (ms) The average time for incoming calls to this UE to ring.
UE RTP Dropped Packets The number of packets dropped by this UE.
UE Min Time to Ringing (ms) The minimum time for incoming calls to this UE to ring.
UE Duplicate RTP Packets The number of duplicate packets received in by this UE.
UE Max Time to Ringing (ms) The maximum time for incoming calls to this UE to ring.
UE Out Calls Attempted The number of calls out attempted by this UE.
The number of messages with RTP packets received in by
UE Out Calls Established The number of calls established by this UE. UE Calls Received RTP Packet
this UE.
UE Out Calls Rejected The number of calls rejected by this UE.
UE Mean Time to RTP Packet
The number of incoming calls that this UE attempted to The Mean time for this UE to receive the first RTP packet.
UE In Calls Attempted (ms)
receive.
UE Min Time to RTP Packet
UE In Calls Established The number of incoming calls established by this UE. The Minimum time for this UE to receive the first RTP packet.
(ms)
UE In Calls Rejected The number of incoming calls rejected by this UE. UE Max Time to RTP Packet
The Maximum time for this UE to receive the first RTP packet
UE Calls Errored The number of calls with errors logged by this UE. (ms)
UE SIP Out Messages The number of SIP messages sent out by this UE. UE RTP Jitter (RFC 3350) ms The Jitter per ms.
UE SIP Messages Resent The number of SIP messages resent out by this UE.
UE RTP Max Jitter (RFC 3350) The maximum Jitter per ms.
UE SIP In Messages The number of SIP messages received by this UE.
UE In RTCP Packets The number of RTCP packets received in by this UE.
UE Out RTCP Packets The number of RTCP packets sent out by this UE
UE Registrations Attempted The number of registrations attempted by this UE.
UE Registrations Successful The number of successful registrations by this UE.
UE Registrations Rejected The number of registrations rejected by this UE.
UE Registrations Errored The number of registrations with errors logged by this UE.
UE Calls Received Ringing The number of ringing calls received in by this UE.
Table 1 : Dependanttime forthe SIPcalls to this UE to ring. (typically 9 to 16 SIP messages per session) the signalling plane will
UE Mean Time to Ringing (ms) The average on incoming configuration
UE Min Time to require tight management. To identify how ring.
Ringing (ms) The minimum time for incoming calls to this UE to configuration optimization impacts quality a unique set of metrics is
UE Max Time to Ringing (ms)detailing each step in the VoLTE call process. Highlighted above are a sample set of metrics.
required The maximum time for incoming calls to this UE to ring.
The number of messages with RTP packets received in by
UE Calls Received RTP Packet
this UE.
UE Mean Time to RTP Packet
The Mean time for this UE to receive the first RTP packet.
(ms)
UE Min Time to RTP Packet
The Minimum time for this UE to receive the first RTP packet.
(ms)
UE Max Time to RTP Packet
(ms)
UE RTP Jitter (RFC 3350) ms
The Maximum time for this UE to receive the first RTP packet
The Jitter per ms.
VoLTE Performance TESTING

5. IMS Core
The IMS core contains a number of elements, for the purpose of VoLTE QoS performance testing, it’s crucial to examine the
core in terms of the SIP functionality and handling.
Focusing on the Call Session Control Function (CSCF), the dual purpose of this element is to handle SIP messaging and
manage the IP address assignment requests from the UE (DHCP). The CSCF may also use encryption (IPSec or TLS) for secu-
rity purposes on its ports.
QoS performance testing aimed at VoLTE in the IMS core should cover all the protocols in use. As a minimum, VoLTE QoE
tests in the IMS core must cover:
- UE IP address assignment timing performance
- SIP registration performance
- Actual voice quality analysis in real-time
- Call release and teardown performance
An example performance test is to see how quickly an IP address is assigned to the UE or how quickly that device can send
the first SIP “Invite” message. This scaled to several thousand users will give an accurate representation on how well the IMS
core is primed.
The IMS Core will pass millions of SIP messages a second. This puts the rates of concurrent SIP sessions in the order of tens
of thousands. Clearly it’s not feasible to try to actively test these numbers with a group of friendly users. To create large
volumes of concurrent SIP sessions requires high performance, scalable emulation tools that can analyze each and every
session in real-time.
Figure 4 : Optimization is a key requirement for LTE as each
SIP session enabled on the carrier costs revenue. IMS
enablers such as Presence and Location are one way in
which carriers can counteract revenue loss to OTT content
providers and benefit from an all IP network. For these
services to a success carriers must performance test in the
application layer.
VoLTE Performance TESTING

6. VoLTE Performance TESTING
diversifEye
diversifEye™ is a test and performance measurement solution for next generation networks such as 4G-LTE and Cloud Com-
puting. diversifEye provides emulation, test and performance measurements of individual application flows such as Video,
Voice and Data applications running in the LTE network. diversifEye’s per flow architecture with real time analysis provides
the granularity necessary to determine any Quality of Experience performance issues on a per application basis relating to
poorly configured QoS settings on the 4G network.
As an emulation tool diversifEye saves considerable time when testing 4G network performance with its pre-configured
applications and emulated UE end-points. Testing includes performance analysis of SIP, SIP over IPSec, SIP over GTP, etc.
Furthermore diversifEye provides IMS type applications such as SIP enabled RTSP or SIP enabled HTTP flows.
diversifEye provides large scale emulation of stateful application flows, by accurately representing UEs and parameters such
as IMSI and user authentication details. diversifEye’s emulated traffic flows are used to load various interfaces on the 4G path.
diversifEye is not limited to solely loading the LTE network paths but also enables the delivery of real user activity on a per
UE basis. diversifEye’s per flow architecture means multiple application flow requests per registered UE, representing a real
subscriber’s smartphone activity. diversifEye enables testing utilization performance of the LTE EPC.
diversifEye’s per flow architecture provides performance measurements on each and every emulated application flow asso-
ciated with each and every emulated UE, enabling users to quantify how LTE QoS feature changes impact a subscriber’s
VoLTE service.
Figure 5 : Dedicated VoLTE performance windows provide real-time views on
QoE. Capture live performance issues when changing core LTE QoS settings.
Summary
Optimizing 4G networks to save costs has been greatly simplified by Shenick’s per-flow emulation and test approach. With
per-flow emulation and real time analysis it’s possible to limit the impact to customer QoS. Using Shenick’s solutions, service
providers and network equipment manufacturers are given the necessary granular view of end-user quality in real time. Any
subtle changes in quality are picked up by Shenick’s thresholding facility, enabling service providers to quickly judge, in real
time, if a decision to change or optimize a device setting is the correct one. This win-win situation ensures service providers
continue to deliver services at the right price!
About Shenick Network Systems
Shenick is an award winning provider of per-flow IP communications test systems, enabling converged IP network service providers and
communications equipment vendors to test the complete service delivery life cycle from lab to live deployment. Established in 2000,
Shenick has deployed its diversifEye™ integrated network, application and security attack emulation and performance monitoring
systems to converged IP-oriented network service providers, communications equipment manufacturers, large enterprise and govern-
ments globally. diversifEye is a registered trademark of Shenick Network Systems.
For more information contact: info@shenick.com
US : 533 Airport Boulevard, Burlingame, CA 94010, USA Tel: +1-650-288-0511
EU : Brook House, Corrig Avenue, Dun Laoghaire, Co Dublin, Ireland Tel: +353-1-236-7002