1. HP Restricted
CMU Performance Monitoring tool - Prototype
Intern Name:
Goutam
Adwant
What does the tool do?
Purpose of this tool is to run jobs that would extract core,disk,memory & network-
detail + summary data fromCMU( using collect & Scripts ) fora specified time
frame and then store it in MySQL DBforthe purpose of reporting/graphing &
analysis of the test results.

2. HP Restricted
CMU Performance Monitoring tool - Prototype
Intern Name:
Goutam
Adwant
Technologies:
Front end:
•HTML
•CSS
•JQuery
ServerSide:
•PHP/PHPSECLIB– Used to exec/run monitoring dump commands on mgmt. server& connect to MySQL dBto
store metadata of the test run.
•Python – Used to extract & read dump file and connect to MySQL dB& insert the data to its corresponding detail &
summary table.
Tools:
•HeidiSQL – To display mgmt. serverMySQL data in a tabularform
•CMU – To view the data of the defined metrics – Core,Disk,Memory & Network.
Build a simple front end screen through which usercan input test details & time frame.

3. HP Restricted
CMU Performance Monitoring tool - Prototype
Intern Name:
Goutam
Adwant
Component functional view :
Test Description
Start Time
End Time
Submit
Front End Screen
PHP
ScriptCMU
Python
Script
Excel/CSV
Charts
Reporting
Connect &
Generates dump
file
Returns
Dump file
Triggers
Exec .py scripts
Reads the dump file
& inserts into
respective tables
MySQL
Entire code deployed on tomcat serverof mgmt. node.
AAA
File
CPU & Disk
Scripts
/proc/stat
Cpucore0-39
Disk1-13
Request is hit only once
and the result is stored in
cache so that all the core &
diskdata are retrieved
fromthat.

4. HP Restricted
CMU Performance Monitoring tool - Prototype
Intern Name:
Goutam
Adwant
Schema Flow diagram:
Test Execution
Test metrics
summary
cpu_util_all disk_metrics
Detail Level Data 
Summary Level Data 
1 row pereach test execution
1 row/node during a test run
1 row/detail of node during a
test run
Meta Data 

5. HP Restricted
CMU Performance Monitoring tool - Prototype
Intern Name:
Goutam
Adwant
Test Execution data – captures metadata foreach test execution
DBSnapshotSchema
CREATE TABLE `test_execution` (
`test_id` VARCHAR(45) NOT NULL,
`start_time` DATETIME NULL DEFAULT NULL,
`end_time` DATETIME NULL DEFAULT NULL,
`test_desc` VARCHAR(45) NULL DEFAULT NULL,
`test_result` VARCHAR(45) NULL DEFAULT NULL,
`test_user` VARCHAR(45) NULL DEFAULT NULL,
`metrics_calc` TINYINT(1) NULL DEFAULT NULL,
`results_captrured` TINYINT(1) NULL DEFAULT NULL,
`stdout_captured` TINYINT(1) NULL DEFAULT NULL,
PRIMARY KEY (`test_id`));
Test id: Programgenerated metric. It is unique forevery job executed. Acts as a foreign key to the meta & Summary table/data.
start_time: Userinput data – start time of the data that needs to be extracted.
End_time : Userinput data – end time of the data that needs to be extracted.
Test_desc: Userinput data – test description of the test run that the usersubmits.
Test_result,test_user,metrics_calc,results_captured,stdout_captured: not yet populated – to be decided.

6. HP Restricted
CMU Performance Monitoring tool - Prototype
Intern Name:
Goutam
Adwant
Test Execution data – Metric Details
Metric Name Source
Test_id Program generated metric- unique for
each test run
Start_time User input data
End_time User input data
Test_desc User input data

7. HP Restricted
CMU Performance Monitoring tool - Prototype
Intern Name:
Goutam
Adwant
Test Metrics Summary data –
Captures node level summary data. It contains core,disk,network& memory
information of each node for5sec time span.
Schema :
CREATE TABLE `test_metrics_summary` (
`test_id` INT(11) NULL DEFAULT NULL,
`servername` VARCHAR(45) NOT NULL, `date_time` VARCHAR(45) NOT NULL,
`interval_time` DATETIME NOT NULL, `data_disks_read_mb` DECIMAL(10,2) NULL DEFAULT NULL,
`data_disks_write_mb` DECIMAL(10,2) NULL DEFAULT NULL, `data_disks_read_iops` VARCHAR(45) NULL DEFAULT NULL,
`data_disks_write_iops` VARCHAR(45) NULL DEFAULT NULL,`total_cpu` INT(11) NULL DEFAULT NULL,
`sys_disks_read_mb` VARCHAR(45) NULL DEFAULT NULL,`sys_disks_write_mb` VARCHAR(45) NULL DEFAULT NULL,
`sys_disks_read_iops` VARCHAR(45) NULL DEFAULT NULL,`sys_disks_write_iops` VARCHAR(45) NULL DEFAULT NULL,
`nw_pkt_in` VARCHAR(45) NULL DEFAULT NULL,`nw_pkt_out` VARCHAR(45) NULL DEFAULT NULL,
`memory_used` VARCHAR(45) NULL DEFAULT NULL,`total_avail_memory` VARCHAR(45) NULL DEFAULT NULL,
`memory_free` VARCHAR(45) NULL DEFAULT NULL, PRIMARY KEY (`servername`, `date_time`, `interval_time`));

8. HP Restricted
CMU Performance Monitoring tool - Prototype
Intern Name:
Goutam
Adwant
Metric Name Source
test_id Program generated metric- unique for each test run
start_time, end_time User input data
data_disks_read_mb, data_disks_write_mb /opt/cmu/tools/cmu_get_det_disk -n disk –rs
/opt/cmu/tools/cmu_get_det_disk -n disk -ws
data_disks_read_iops, data_disks_write_iops /opt/cmu/tools/cmu_get_det_disk -n disk –rc
/opt/cmu/tools/cmu_get_det_disk -n disk -wc
system_disks_read_mb, system_disks_write_mb /opt/cmu/tools/cmu_get_det_disk -n sys –rs
/opt/cmu/tools/cmu_get_det_disk -n sys -ws
system_disks_read_iops, system_disks_write_iops /opt/cmu/tools/cmu_get_det_disk -n sys–rc
/opt/cmu/tools/cmu_get_det_disk -n sys -wc
total_cpu /opt/cmu/tools/cmu_get_det_cpu -c -n cpu
nw_out_mbps, nw_in_mbps COLLECTL (nettotals.kbout) / 1024
COLLECTL (nettotals.kbin) / 1024
total_memory COLLECTL (meminfo.tot)
memory_free COLLECTL (meminfo.tot) - (meminfo.used)
memory_used COLLECTL (meminfo.used) / (meminfo.tot) * 100
Test Metrics Summary data – Metric Details

9. HP Restricted
CMU Performance Monitoring tool - Prototype
Intern Name:
Goutam
Adwant
CPU/Core - Detail Data – captures core level detail
DBSnapshotSchema
CREATE TABLE `cpu_util_all` (
`test_id` INT(11) NOT NULL,
`interval_time` INT(11) NOT NULL,
`servername` VARCHAR(45) NOT NULL,
`date_time` VARCHAR(45) NOT NULL,
`cpu_user` VARCHAR(45) NOT NULL,
`cpu_core_value` VARCHAR(45) NULL DEFAULT NULL,
PRIMARY KEY (`test_id`, `interval_time`,
`servername`, `date_time`, `cpu_user`));
Test id: Programgenerated metric. It is unique forevery job executed. Acts as a foreign key to the meta & Summary table/data.
Interval_time: Programgenerated metric. Fora particulartest_id, it increments forevery 5seconds of test result data. ( e.g.. For
1min of test data – we have 12 interval_time id’s )
Servername: Generated through the dump file.
Data_time: Generated through the dump file.
cpu_user: Metric data generated through the AAA file.
cpu_core_value: Metric data generated through the AAA file.

10. HP Restricted
CMU Performance Monitoring tool - Prototype
Intern Name:
Goutam
Adwant
Metric Name Source
test_id Program generated metric- unique for each test run
Interval_time Program generated metric. For a particular test_id, it
increments for every 5seconds of test result data
Servername Generated through the dump file.
date_time Generated through the dump file. Increments with
5secs from start time to end time.
cpu_user Generated through detail script in AAA. cpucore_00 to
cpucore_39
cpu_core_value /opt/cmu/tools/cmu_get_det_cpu -n cpu00-39
CPU/Core data – Metric Details

11. HP Restricted
CMU Performance Monitoring tool - Prototype
Intern Name:
Goutam
Adwant
Disk– Detail Data:
DBSnapshotSchema
CREATE TABLE `disk_stats_all` (
`test_id` INT(11) NOT NULL,
`interval_time` INT(11) NOT NULL,
`servername` VARCHAR(45) NOT NULL,
`date_time` VARCHAR(45) NOT NULL,
`disk_name` VARCHAR(45) NOT NULL,
`disk_read_mbps` VARCHAR(45) NULL DEFAULT NULL,
`disk_write_mbps` VARCHAR(45) NULL DEFAULT NULL,
`disk_reads` VARCHAR(45) NULL DEFAULT NULL,
`disk_writes` VARCHAR(45) NULL DEFAULT NULL,
PRIMARY KEY (`test_id`, `interval_time`, `servername`,
`date_time`, `disk_name`));
Test id: Programgenerated metric. It is unique forevery job executed. Acts as a foreign key to the meta & Summary table/data.
Interval_time: Programgenerated metric. Fora particulartest_id, it increments forevery 5seconds of test result data. ( e.g.. For
1min of test data – we have 12 interval_time id’s )
Servername: Generated through the dump file.
Data_time: Generated through the dump file.
Disk_name: Inserted through the python script forthe numberof disks present fora particularnode.
disk_read_mbps, disk_write_mbps,disk_reads,disk_writes : Metric data generated through the AAA file.

12. HP Restricted
CMU Performance Monitoring tool - Prototype
Intern Name:
Goutam
Adwant
Metric Name Source
test_id Program generated metric- unique for each test run
Interval_time Program generated metric. For a particular test_id, it
increments for every 5seconds of test result data
Servername Generated through the dump file.
date_time Generated through the dump file. Increments with
5secs from start time to end time.
disk_name Generated through detail script in AAA.disk-1 to disk-
13
disk_read_mbps,disk_write_mbps /opt/cmu/tools/cmu_get_det_disk -n disk1-13 –rs
/opt/cmu/tools/cmu_get_det_disk -n disk1-13 –ws
disk_reads,disk_writes /opt/cmu/tools/cmu_get_det_disk -n disk1-13 -rc
/opt/cmu/tools/cmu_get_det_disk -n disk1-13 –wc
Diskdata – Metric Details

13. HP Restricted
CMU Performance Monitoring tool - Prototype
Intern Name:
Goutam
Adwant
• Select `cmudata`.`test_metrics_summary`.`test_id` AS `test_id`,`cmudata`.`test_metrics_summary`.`interval_time` AS `interval_time`,avg(`cmudata`.`test_metrics_summary`.`total_cpu`) AS
`avg(total_cpu)`,sum(`cmudata`.`test_metrics_summary`.`data_disks_read_mb`) AS `sum(data_disks_read_mb)`,sum(`cmudata`.`test_metrics_summary`.`data_disks_write_mb`) AS
`sum(data_disks_write_mb)`,sum(`cmudata`.`test_metrics_summary`.`data_disks_read_iops`) AS `sum(data_disks_read_iops)`,sum(`cmudata`.`test_metrics_summary`.`data_disks_write_iops`) AS
`sum(data_disks_write_iops)`,sum(`cmudata`.`test_metrics_summary`.`sys_disks_read_mb`) AS `sum(sys_disks_read_mb)`,sum(`cmudata`.`test_metrics_summary`.`sys_disks_write_mb`) AS
`sum(sys_disks_write_mb)`,sum(`cmudata`.`test_metrics_summary`.`sys_disks_read_iops`) AS `sum(sys_disks_read_iops)`,sum(`cmudata`.`test_metrics_summary`.`sys_disks_write_iops`) AS
`sum(sys_disks_write_iops)`,sum(`cmudata`.`test_metrics_summary`.`nw_pkt_in`) AS `sum(nw_pkt_in)`,sum(`cmudata`.`test_metrics_summary`.`nw_pkt_out`) AS
`sum(nw_pkt_out)`,sum(`cmudata`.`test_metrics_summary`.`memory_used`) AS `sum(memory_used)`,sum(`cmudata`.`test_metrics_summary`.`total_avail_memory`) AS
`sum(total_avail_memory)`,sum(`cmudata`.`test_metrics_summary`.`memory_free`) AS `sum(memory_free)` from `cmudata`.`test_metrics_summary` where
(`cmudata`.`test_metrics_summary`.`servername` like 'f02wn%') group by `cmudata`.`test_metrics_summary`.`test_id`,`cmudata`.`test_metrics_summary`.`interval_time` orderby
`cmudata`.`test_metrics_summary`.`test_id`,`cmudata`.`test_metrics_summary`.`interval_time
Views:
test_metrics_north_aggregate : Retrieves the aggregate level data filtered only for
workernodes and ordered by test_id & interval id.
test_metrics_south_aggregate : Retrieves the aggregate level data filtered only fordata
nodes and ordered by test_id & interval id.
• Select `cmudata`.`test_metrics_summary`.`test_id` AS `test_id`,`cmudata`.`test_metrics_summary`.`interval_time` AS `interval_time`,avg(`cmudata`.`test_metrics_summary`.`total_cpu`) AS
`avg(total_cpu)`,sum(`cmudata`.`test_metrics_summary`.`data_disks_read_mb`) AS `sum(data_disks_read_mb)`,sum(`cmudata`.`test_metrics_summary`.`data_disks_write_mb`) AS
`sum(data_disks_write_mb)`,sum(`cmudata`.`test_metrics_summary`.`data_disks_read_iops`) AS `sum(data_disks_read_iops)`,sum(`cmudata`.`test_metrics_summary`.`data_disks_write_iops`) AS
`sum(data_disks_write_iops)`,sum(`cmudata`.`test_metrics_summary`.`sys_disks_read_mb`) AS `sum(sys_disks_read_mb)`,sum(`cmudata`.`test_metrics_summary`.`sys_disks_write_mb`) AS
`sum(sys_disks_write_mb)`,sum(`cmudata`.`test_metrics_summary`.`sys_disks_read_iops`) AS `sum(sys_disks_read_iops)`,sum(`cmudata`.`test_metrics_summary`.`sys_disks_write_iops`) AS
`sum(sys_disks_write_iops)`,sum(`cmudata`.`test_metrics_summary`.`nw_pkt_in`) AS `sum(nw_pkt_in)`,sum(`cmudata`.`test_metrics_summary`.`nw_pkt_out`) AS
`sum(nw_pkt_out)`,sum(`cmudata`.`test_metrics_summary`.`memory_used`) AS `sum(memory_used)`,sum(`cmudata`.`test_metrics_summary`.`total_avail_memory`) AS
`sum(total_avail_memory)`,sum(`cmudata`.`test_metrics_summary`.`memory_free`) AS `sum(memory_free)` from `cmudata`.`test_metrics_summary` where
(`cmudata`.`test_metrics_summary`.`servername` like 'f02dn%') group by `cmudata`.`test_metrics_summary`.`test_id`,`cmudata`.`test_metrics_summary`.`interval_time` orderby
`cmudata`.`test_metrics_summary`.`test_id`,`cmudata`.`test_metrics_summary`.`interval_time
test_metrics_summary : Retrieves the aggregate level data forboth the nodes and
ordered by test_id & interval id

14. HP Restricted
CMU Performance Monitoring tool - Prototype
Intern Name:
Goutam
Adwant
Final Output:
Data stored in MySQL Db. is saved as excel/csv and graphs/charts can be generated.

15. HP Restricted
CMU Performance Monitoring tool - Prototype
Intern Name:
Goutam
Adwant
DEMO