My PhD defense talk. Please see www.ashishgupta.org for more details.

1. Black Box Methods for Inferring Parallel Applications' Properties in Virtual Environments Ashish Gupta March 2008 PhD Final Talk Committee: Prof. Peter Dinda Prof. Fabian Bustamante Prof. Yan Chen Prof. Dongyan Xu (Purdue University)

2. Introduction

5. Adaptation in Virtuoso Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments Input for adaptation problem formalization in my colleague Ananth Sundararaj’s thesis Different input components Many more inferable properties VM/application demands Resource availability User constraints

8. Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments Components of my dissertation Virtual Topology and Traffic Inference Framework Black box metrics for Absolute Performance Ball in The Court principles to compute application slowdown Global Bottlenecks using time decomposition Increasing Application Performance In Virtual Environments through Run-time Inference and Adaptation Free Network Measurement For Adaptive Virtualized Distributed Computing Inference Adaptation 1 2 3 4 5 6

9. Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments Topics I cover Virtual Topology and Traffic Inference Framework Black box metrics for Absolute Performance Ball in The Court principles to compute application slowdown Global Bottlenecks using time decomposition Increasing Application Performance In Virtual Environments through Run-time Inference and Adaptation Free Network Measurement For Adaptive Virtualized Distributed Computing Inference Adaptation 2 3 4 5 A B

12. Patterns application’s capabilities Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments 2-D Mesh 3-D Toroid 3-D Hypercube Reduction Tree All-to-All

14. Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments Different contributions of my dissertation Virtual Topology and Traffic Inference Framework Black box metrics for Absolute Performance Ball in The Court principles to compute application slowdown Global Bottlenecks using time decomposition Increasing Application Performance In Virtual Environments through Run-time Inference and Adaptation Free Network Measurement For Adaptive Virtualized Distributed Computing Inference Adaptation 2 3 4 5 A B

15. Collective Inference for Topology: VTTIF

16. Goal of VTTIF Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments Low Level Traffic Monitoring ? An online topology inference framework for a VM environment Application Topology

17. Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments Traffic Analyzer Rate based Change detection Traffic Matrix Query Agent VM Network Scheduling Agent VNET daemon VM VNET overlay network To other VNET daemons Physical Host VTTIF Architecture

18. Inferred topology Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments Parallel Integer Sort

19. Black Box Measures of performance

24. Inter send-packet delay Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments 176 Receive from 175 176 Send to 175 176 Send to 175 176 Send to 175 176 Receive from 175 176 Send to 175

25. Inter send-packet delay Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments Traffic trace for Patterns Message size = 4000 bytes Computation per iteration: 100 MFlops Clustering based on inter-send delays Count in cluster matches actual iteration rate output by application (325)

26. Patterns: clusters without load Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments Actual: 568 Reported: 569

27. Patterns: clusters with external CPU load Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments Actual: 142 Reported: 153 Actual execution time ratio: 3.922 Ratio from reported iteration rate: 3.72 Within 5%

31. RIR time series Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments For dynamic applications, RIR changes with time Need a time series for RIR over the trace

32. Outputting RIR time series - Workflow Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments Sniff Packets Send packets that obey the conditions Slide a 1 second window over these send packets Sliding interval =  t Get a new time series denoting RIR for each 1 second sliding window Sampling duration = T Derivative Metrics from the above time series Average RIR CDF Power Spectrum Super-phase period Using tcpdump/libpcap for the VM traffic Send packets satisfying the two conditions 1 sec Slide by  t i 1 , i 2, i 3, i 4, i 5, …. i n Each number represents number of iterations for a particular 1 sec window instance

35. Effectiveness of RIR avg Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments For MG application, running under different load conditions (100% and 60% CPU load), predicted execution time error rates were 13% and 7% respectively (completely black box) Value of c = 1.1 here (c*RTT factor)

36. RIR time series graph Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments A super-step

40. A sample power spectrum for 4 processes Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments Consistency across processes

41. Example for MG Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments Significant frequency separation Exec time = 19.44 seconds Super-phase period = 4.1 seconds (1/0.244) Number of super-phases ~4

43. Ball In the Court Methods

48. Approach to computing the BIC delay Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments Let’s compute the time differential for event pairs, for *.176 Some computation (6 ms) Some computation (6ms)

49. With Load, Some BIC Delays Get Larger Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments Loaded process Unloaded process 1. Sending an ack  process’s responsibility 2. This responsibility was hugely inflated in the loaded case ( 64 us to 23822 us) 3. BIC delays for other receives are similar (receive  BIC for other processes)

54. Using BIC Delays to Measure Application Imbalance Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments Compute BIC delay Imbalance Algorithm Slowdown

56. Global BIC algorithm Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments

57. Evaluation with the IS application Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments Balanced case Total runtime = 14.36 seconds Total runtime = 152.67 seconds Loaded case Imbalance between 176 and 175 = 124.49 seconds Actual difference = 138.41 seconds Using completely black box means, we could determine, how slow is the application running with load, without knowing about the unloaded case

61. Global bottlenecks and Time Decomposition

64. Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments Different contributions of dissertation Virtual Topology and Traffic Inference Framework Black box metrics for Absolute Performance Ball in The Court principles to compute application slowdown Global Bottlenecks using time decomposition Increasing Application Performance In Virtual Environments through Run-time Inference and Adaptation Free Network Measurement For Adaptive Virtualized Distributed Computing Inference Adaptation 2 3 4 5 A B * An API also defined in the dissertation

66. Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments Effect on BSP Application Throughput For high Compute/Communicate Ratio, Migration + Topology dramatically improves performance Adapting to External Load Imbalance External load removed, but I/O still dominates, so topology helps External load removed, can drive higher I/O

67. Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments Free Network Measurement For Adaptive Virtualized Distributed Computing

72. Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments Components of my dissertation Virtual Topology and Traffic Inference Framework Black box metrics for Absolute Performance Ball in The Court principles to compute application slowdown Global Bottlenecks using time decomposition Increasing Application Performance In Virtual Environments through Run-time Inference and Adaptation Free Network Measurement For Adaptive Virtualized Distributed Computing Inference Adaptation 1 2 3 4 5 6

73. Thank you !

74. Backup slides

79. Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments Effect on BSP Application Throughput Adapting to Compute/Communicate Ratio For high Compute/Communicate Ratio, Migration + Topology dramatically improves performance Less time spent in I/O, so migration alone is enough Since I/O dominates, drop in latency improves performance Even for small amount of I/O, it takes up significant time

80. Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments TPCW Throughput (WIPS) With Image Server Facing External Load No Topology Topology No Migration 1.216 1.76 Migration 1.4 2.52

82. Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments What is WREN ? Developed by my colleague Marcia Zangrilli from WM College 1. Observes incoming/outgoing packets 2. Online analysis to derive latency/bandwidth information for all host pair connections 3. Answers network queries for any pair of hosts What does it do ?

83. Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments An important Contribution: Problem Formalization

84. Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments Two approaches to adaptation

85. VTTIF in a dynamic topology environment Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments Parameters: Smoothing Window  sliding window duration over which updates are aggregated Update Rate Detection Threshold

88. Bin

89. Evidence of Adaptation Driven by Inference

92. Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments Effect on BSP Application Throughput For high Compute/Communicate Ratio, Migration + Topology dramatically improves performance Adapting to External Load Imbalance External load removed, but I/O still dominates, so topology helps External load removed, can drive higher I/O

93. Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments Evaluation Scenario 2 : Large 256 host topology. 32 potential hosts, 8 Virtual Machines Results for Multi Constraint Cost Function : Bandwidth and Latency Annealing easy to adapt and finds good mappings compared to heuristic

94. Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments The New Adaptation Process

95. Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments Two approaches to adaptation

96. Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments Evaluation Scenario 2 : Large 256 host topology. 32 potential hosts, 8 Virtual Machines Results for Multi Constraint Cost Function : Bandwidth and Latency Annealing easy to adapt and finds good mappings compared to heuristic

101. Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments Summary of contributions Virtual Topology and Traffic Inference Framework Black box metrics for Absolute Performance Ball in The Court principles to compute application slowdown Global Bottlenecks using time decomposition Increasing Application Performance In Virtual Environments through Run-time Inference and Adaptation Free Network Measurement For Adaptive Virtualized Distributed Computing

102. Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments Summary of contributions Virtual Topology and Traffic Inference Framework Black box metrics for Absolute Performance Ball in The Court principles to compute application slowdown Global Bottlenecks using time decomposition Increasing Application Performance In Virtual Environments through Run-time Inference and Adaptation Free Network Measurement For Adaptive Virtualized Distributed Computing

109. Citations [167] VALIANT, L. A bridging model for parallel computation. Communications of the ACM 33 , 8 (Aug. 1990), 103–111.

116. Dynamic Topology Inference by VTTIF Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments 1. Fast updates Smoothed Traffic Matrix 2. Low Pass Filter Aggregation 3. Threshold change detection Topology change output VNET Daemons on Hosts VNET Daemon at Proxy Aggregated Traffic Matrix

119. Solving the issues Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments

121. Effect of load on the RIR time series Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments 1. Expansion effect 2. Basic structure of time series is same 3. Each phase more irregular (more spikes) 4. Captures the time dynamics of the application

123. The RIR-CDF metric Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments CDF indicates the RIR region application spends most of its time in RIR-CDF can be used to predict which application will be more affected by external load (for dynamic applications) I show how we can predict slowdown for MG vs IS Slowdown mapping

125. Effect of load on different RIR areas Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments Difference of performance over 12.46 times at the extremes under full load Applications with higher iteration rates always seem to be more drastically affected by external load then those with lower iteration rates.

127. Scheduling using the CDF Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments For the IS and MG applications we have seen till now, which application may be hurt more if one of the processes from each application shares the physical host with an external computational load? The impact: we can now determine in advance , the impact of external load if we must choose one of these applications to be influenced by the load. Map each RIR value to its slowdown RIR value from the mapping Slowdown CDF

128. Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments Slowdown CDF For MG Slowdown CDF For IS More time spent in low regions Avg = 0.103 Avg = 0.065