IPTV QoE Monitoring

IPTV QoS
Monitoring

Fast Forward Your Development www.dsp-ip.com

Overview
• QoS and QoE
• Measurement and Monitoring Goals
▫ Lab, pre-deployment, Monitoring, failure analysis
• Measurement Methods
▫ Objective, Subjective, payload/codec based
• Monitoring Levels
▫ Transport, Transaction, content
• Monitoring Domains
▫ Head-end, network, Home


Challenges in providing QoE
Name Challenge Answer
network Overcome Packet loss, jitter, Improved protocols
network congestion QoS mapping schemes

Bandwidth No bandwidth in the network and at increased capacity inside the
the home home and the network level

Monitoring Measure hundreds of video streams Video over IP monitoring,
in Gb Eth & IP environment new protocols & devices

CRM costs Lower MTBF, call center calls and Automatic maintenance,
“truck rolls” Automatic firmware update
Home network monitoring


User quality measurements
• What does subscribers want?
▫ Content
 Content quality
 Content pricing
Technical  Content availability
Factors
▫ Viewing experience
 Video Quality
 Channel Zapping delay
▫ Integration of services
 Usability
 One bill
 New services
▫ Customer services


Voice/Video Quality?
We can look at quality in different ways:

1. Perceptual subjective Quality (MOS)
2. Numerical Quality parameters (PSNR)
3. How to connect (relation matrix between 1
and 2)


MEASUREMENT &
MONITORING
In the Lab & In The Fields
For Pre-Deployment/monitoring/Failure
Analysis


Measurement & Monitoring phases

Design

Lab
Testing

Deployment
Phase
Pre Analysis
Deployment Problem Solving
Testing Tuning

Pre- 24/7
Deployment Monitoring
Phase


Measurement & Monitoring phases
• Design & Lab testing
▫ Simulation and Emulation of the network
▫ Lab and testing tools
• Pre Deployment Stage
▫ Work on actual network
▫ Load testing
▫ Lab, testing, diagnosing and monitoring tools
• Deployment (production) Phase
▫ Mostly monitoring (probes) equipment, management
systems, data filtering and diagnostics equipment


MEASUREMENT TAXONOMY
Subjective
Objective
Payload based, codec aware, codec anaware


Measurement methods review
• Subjective
▫ Accurate
▫ Expensive, not for monitoring
• Objective
▫ Repeatable
▫ For both testing and monitoring


Multimedia monitoring methods
Broadcast HSI and
World Data World

Subjective Objective

Network
MOS BT500 Codec aware Monitoring
(Voice) (Video) Packet based
Delay, Jitter
Payload Packet loss
VQS
Full Telchemy
Codec independent
Reference Reduced Packet based
V-Factor
Reference

J.144 PSNR No
Reference VQI MDI

Testing Monitoring


Objective methods

Objective

Payload Codec aware Codec independent Network
Packet based Packet based Monitoring


Payload Based Methods

Payload

Full
Reference
Reduced
Reference

J.144 PSNR No
Reference


Full Reference: Video Quality Assessment
ITU-T J.144 and ITU-R BT.1683

• Full-reference perceptual models
• Digital TV
• Rec. 601 image resolution (PAL/NTSC)
• Bit rates: 768 kbps ~ 5 Mbps
• Compression errors


Voice Quality Assessment – with/out reference
• ITU-T P.862 (Feb 2001) - Full Reference
▫ Full-reference perceptual model (PESQ)
▫ Signal-based measurement
▫ Narrow-band telephony and speech codecs
▫ P.862.1 provides output mapping for prediction on
MOS scale
• ITU-T P.563 (May 2004)
▫ No-reference perceptual model
▫ Signal-based measurement
▫ Narrow-band telephony applications


Voice Quality Assessment

• ITU-T P.862.2 (Nov 2005):
▫ Extension of ITU-T P.862
▫ Wide-band telephony and speech codecs (5 ~7Khz)
• ITU-T P.VTQ (on-going):
▫ Targeted at VoIP applications
▫ Minimum performance framework for no-reference
packet-based measurement
▫ Models analyze packet statistics; speech payload is
assumed
▫ Uses P.862 as a measurement reference


Codec Aware Methods

Codec aware
Packet based

VQS
Telchemy

V-Factor

VQI


Packet – Codec Aware
• Monitoring technique
• Codec dependent
• Incorporates network parameters data with
codec behavior data
• Scales- could monitor thousands of channels
• Examples: The need a codec aware metrics
35
▫ VQS (Telchemy) 30
▫ VQI(Brix) 25
Robust
PSNR (dB)

▫ V-Factor (QoSMetrics) 20

15
codec
10
Problem area
5

0
0 “Raw” 5 10 15 20
codec Packet Loss (%)


Packet – Codec aware
100
Packet Loss/Discard Rate

80 Packet loss/discard typically
occurs in high density periods
60
40
20
0
0 10 20 30 40 50
Base quality level 5 Time
depends on frame rate,
Mean Opinion Score

codec type, bit rate 4

3
Average can be Impact of Burst of
misleading 2
Packet Loss
1 Subjective
0 5 10 15 20
compensation for
Poor quality 5-8 Time variance between
during burst of seconds 15-30 human and testing
loss/discards
seconds equipment view of loss


Example V-Factor
• Based on MPQM (Moving Picture Quality
Metrics) – high quality video measurement
standard
• V = f(QER, PLR, R)
▫ QER – relative video codec quality
▫ PLR – Packet loss ratio (based on actual packet
loss, jitter data and jitter buffer model)
▫ R – Image complexity factor (2-3)
• Adopted by Spirnet


Packet – Codec Independent
• Monitoring only
• Codec independent
• Based on network parameters data only
• Scales - could monitor thousands of channels
• Examples:
▫ MDI
 IneoQuest
 standardized by IETF


MONITORING LEVELS


Measurement Levels
• Transport Level
• Service (transaction) Level
• Media Quality Level

Add Your Text
Content Quality

Add Your Text
Video Stream Quality

Add Your Text
Transport Quality

Add Your Text
Transaction Quality


Transport Level
• Example:
▫ Packet Loss
▫ Loss Patterns
▫ Jitter
▫ Delay
• Well understood
• Defined by ITU and IETF


Transaction Level
• Examples
▫ Post-dial delay in PSTN/mobile networks
▫ Video start time
• Requires understanding in both network
monitoring and signaling (IGMP, SIP) and in
media coding (analysis of the media to discover
dial tone or I frame)


Channel zapping delay
• Multicast saves bandwidth but creates
signaling delays:
▫ Multicast Leave +
▫ Multicast Join +
▫ First I Frame +
▫ Up to 2 seconds buffering time
First frame
viewed
Total Channel zapping Latency

Leave latency First I Buffering
Frame latency
Join latency
Signaling Latency Media
Latency


Microsoft improved channel zapping
• Unicast server is used to serve the stream till the
second multicast stream has an “I Frame”
▫ Immediate I Frame
▫ Shorter buffering time

I Frame unicast First frame
viewed I Frame Multicast
Total Channel zapping Latency

Leave latency unicast stream

Join latency
Multicast stream
Shorter buffering time


VoD Signaling

• Unicast server is used to serve the stream till the
second multicast stream has an “I Frame”
▫ Immediate I Frame
▫ Shorter buffering time

First frame
viewed

Total VoD Signaling Latency

Play to server Buffering latency


Content Level
• Content quality is a payload based measurement.
▫ Requires decoding of the video stream
▫ Understanding of the buffering and error concealment algorithms
of the decoder
• CPU intensive – Does not scale
• Accurate
• Used mostly is Lab equipment and diagnostic equipment
• Examples:
▫ PSNR
▫ ITU-T J.144
• Usually requires the reference (original) stream
• Tests:
▫ Source artifacts
▫ Source quality


Standardization landscape

ATIS ITU ITU DSL

IIF – IPTV Study Group 12 VQEG – Video DSL Forum
Interoperability Quality Expert
Forum Algorithms for Group TR-64, TR-69
end-to-end LAN and WAN
Video performance
QoS Metrics transmission measurement based monitoring
Standardization performance on Subjective tests standards
Database

Used for Used in
Monitoring Diagnostics /
Lab


Example: ATIS IIF Quality Metrics
VSTQ - Video Service Transmission Quality
▫ Transmission Quality - codec/ content independent
▫ Based on the rate and distribution of effective packet loss and discard
VSPQ - Video Service Picture Quality
▫ Estimated viewing quality
▫ Considers the impact of VSTQ, video codec type and rate, resolution
VSAQ - Video Service Audio Quality
▫ audio listening quality
▫ Considers the impact of VSTQ, audio codec type, sample rate, …..
VSMQ - Video Service Multimedia Quality
▫ overall user experience
▫ Combined effect of VSPQ, VSAQ, audio-video synchronization..
VSCQ - Video Service Control (Plane) Quality
▫ Considers responsiveness and reliability of control plane (trick play)


Monitoring levels
• J.144 and PSNR examines the video content only (payload
measurements)
• TR101290 examines only transport stream data and coherence without
examining the video content
• V-Factor and VQS looks at packet loss, jitter and loss patterns data and
incorporate it with codec information and video header information
• MDI – Examines only packet loss and packet loss patterns without
considering the codec or video information

TR101290
MPEG2TS
MDI V-Factor, VQS
Headers

J.144, PSNR
Video
payload


MEASUREMENT DOMAINS
Headend
Network
VoD
Home


DVB TR 101290
• DVB Transport stream monitoring standard
• Alerts and KPI:
▫ TS sync loss ▫ CRC error
▫ Sync byte error ▫ PCR error
▫ PAT error ▫ PCR
▫ Continuity counter errors discontinuity
▫ PMT error ▫ PCR accuracy
▫ PID error error
▫ Transport error ▫ PTS error


Network measurement points

• Head-end • Network Probe interface
▫ VoD Servers  GBE, OC3/12, STM1/4
▫ Encoders • Raw Video input
▫ TS muxers  MPEG2 TS - DVB ASI, DVBoIP
• Network core  Digital and analog video
▫ Routers
• Network edge
▫ ATM switches
▫ GBE Switches
• Access
▫ DSLAM / BRAS
• CPE
▫ STBs
▫ Home network
Source: Brix-Video Quality Measurement Algorithms WP


Headend
• Video source quality
• Video encoding quality
• MPEG2 Transport stream parameters
▫ TR100290
 1st-3rd level priority alerts
• Inputs
▫ Raw analog video in a variety of forms (SD/HD)
▫ DVB-ASI
▫ DVBoIP
▫ DVBoATM (AAL1, AAL5)


VoD
• Pre-deployment
▫ RTSP signaling latency (delay)
▫ Performance under stress (user load)
▫ Trick Mode (FF, REW)
 Signaling delay
 Video quality
 Buffer state (overflow and underflow)
• Monitoring
▫ Loss of data (under stress)
• Interfaces:
▫ GBE
▫ 10GBE


Home network monitoring challenges
• Scalability – Millions of STBs
▫ Cost – Monitoring hardware and/or software costs
must not add to STB costs (multiply by number of
STBs)
▫ Data – Large amounts of data which needs RT
processing, filtering, logging and analysis
• Unclear network structure
• Non “standard” equipment
• Installation, support and debugging problems


Home network monitoring roots
• Home network monitoring is currently a minor
feature in IPTV “Support Automation”
• Support Automation handles:
▫ Installation
▫ Preventive maintenance
▫ STB and CPE remote management (TR69 and more)
▫ Service verification for field technician
▫ CRM support
▫ Interaction logging
▫ Home Network monitoring and repair


CPE network management
• 2Wire solution -Three data views
 OSS/BSS
 Customer care
 Data Analysis
• Other companies in this field
▫ SupportSoft
▫ Enure
▫ Motive


Low level network monitoring


Vendors specialization
Lab Testing Monitoring

Incumbent Telecom Cable Telecom Cable
Agilent Rhode & Agilent Thomson
Ixia Schwartz JDSU/ Snell&Wilcox
Spirnet Acterna
Spirnet

New Telecom Telecom
entrants Shenick Telchemy
Bridge Technologies
iBrix
QoSMetrix


DSP-IP Contact information
For Course materials & lecture request contact:
Adi Yakov
Training and Courses Manager
adi@dsp-ip.com
+972-9-8651933
For projects development services contact:
Alona Ashkenazi
Technology Management Services :
alona@dsp-ip.com
+972-9-8850956 www.dsp-ip.com
Mail : info@dsp-ip.com
Phone: +972-9-8850956,
Fax : +972-50- 8962910


IPTV QoE Monitoring

Recomendados

Recomendados

Mais conteúdo relacionado

Mais procurados

Mais procurados (20)

Destaque

Destaque (20)

Semelhante a IPTV QoE Monitoring

Semelhante a IPTV QoE Monitoring (20)

Mais de Yoss Cohen

Mais de Yoss Cohen (20)

Último

Último (20)

IPTV QoE Monitoring

Notas do Editor