Hardware performance monitoring facilities such as counters can provide invaluable information about system behavior. In recent years, Linux 'perf' has become the standard tool for managing these facilities and interpreting data that they generate. In this talk we will discuss changes to Xen and Linux that will allow PV guests (including dom0) use perf for profiling themselves and, in the case of dom0, the hypervisor.

1. Xen PMU
Boris Ostrovsky, Oracle
Xen Developer Summit
Oct 24-25, 2013
1Copyright © 2013, Oracle and/or its affiliates. All rights reserved.

2. Agenda

Profiling methods

Existing tools

Why perf

Design requirements

High-level design

Status

Next steps
2Copyright © 2013, Oracle and/or its affiliates. All rights reserved.

3. Profiling Methods

Timing

Counting



Software: counting probes
Hardware: performance counters
Sampling



Software: timer-based
Hardware: HW counters, IBS, PEBS
Tracing

Software events: recording probes

Hardware events (Intel BTS)
3Copyright © 2013, Oracle and/or its affiliates. All rights reserved.

4. Existing Tools

Guests

SW-based methods: any guest, most tools

HW-based methods:



HVM: Counters, BTS: most tools
PV: xenoprofile
Hypervisor

SW-based methods: xentrace, xenalyze, xentop, xenperf

HW-based methods: xenoprofile
4Copyright © 2013, Oracle and/or its affiliates. All rights reserved.

5. Why perf

Why not (xen)oprofile



not inthe tree
not in active development
perf

in the tree

active development

lots of features
5Copyright © 2013, Oracle and/or its affiliates. All rights reserved.

6. Desired Properties

Use HVM VPMU for both PV and HVM

As little as possible non-Xen changes in Linux kernel


Staged development
Possibility of other tools using the same framework
6Copyright © 2013, Oracle and/or its affiliates. All rights reserved.

7. (Very) High-Level Design Features

Manage PMU in current VPMU



VPMU states
VPMU context (i.e. registers)
On PMU interrupt stop all counters

Emulate PMU in Linux to avoid traps on MSR accesses

Domain re-enables PMU via hypercall when done processing
the interrupt

Global profile mode


Dom0 profiles everyone
APIC and NMI mode
7Copyright © 2013, Oracle and/or its affiliates. All rights reserved.

8. Status

Limitations


Only profile hypervisor CPUs where dom0 is running

Only counters are supported (no IBS/PEBS)


PV VCPUs should be pinned
No backtrace support (for the hypervisor)
V3 for Xen in progress

4.4 is unlikely

coordinate with Linux patches

perf toolstack changes (not Xen-specific code)
8Copyright © 2013, Oracle and/or its affiliates. All rights reserved.

9. Future Enhancements

Address limitations from stage 1

No pinning requirement

#PCPUs > #VCPUs in dom0

Tracing multiple guests in global mode

New HW features (PEBS, IBS)

Tracing (xentrace?)
9Copyright © 2013, Oracle and/or its affiliates. All rights reserved.

10. Example (perf record/report)
10Copyright © 2013, Oracle and/or its affiliates. All rights reserved.

11. Example (perf kvm record/report)
11Copyright © 2013, Oracle and/or its affiliates. All rights reserved.