O slideshow foi denunciado.
Utilizamos seu perfil e dados de atividades no LinkedIn para personalizar e exibir anúncios mais relevantes. Altere suas preferências de anúncios quando desejar.

Productivity measurement of agile teams (IWSM 2015)

451 visualizações

Publicada em

Nowadays, as the software industry is slowly becoming more mature, software measurement and performance measurement are becoming increasingly important. Organizations need to know their productivity and competitiveness in software development projects for various reasons. In many software development contracts, targets are set for the suppliers to reach. These targets are based on software metrics like productivity, speed of delivery and software quality. In order to check if the targets are reached, it is necessary to measure the functional size of the software product that is delivered and also the functional size of the software development project that is carried out, as there is usually a difference between these two sizes. To be able to use functional size in contracts, it must be measured in an objective, repeatable, verifiable and therefore defensible way. That being the case, the industry’s best practice is to use an ISO/IEC standard for functional size measurement, e.g. Nesma, COSMIC or IFPUG function points. However, these methods only measure the functional user requirements from the total software requirements to be delivered. In activities like project estimation and productivity measurement, the influence of the non-functional requirements is expressed in the Project Delivery Rate (PDR) which is expressed in effort hours per function point. If more than the average amount of non-functional requirements need to be realized in a project (or more severe non-functional requirements), the PDR used should also be higher. In the industry it is customary to set productivity targets based on an average (or calibrated) influence of non-functional requirements and this works quite fine in traditional software projects. In software development projects that are executed in an agile way, this is not always the case. When working agile, there are forces that influence the traditional way of performance measurement significantly, resulting in a number of serious issues. In this paper these issues are explained and a method to overcome these issues is proposed.

Publicada em: Negócios
  • Seja o primeiro a comentar

Productivity measurement of agile teams (IWSM 2015)

  1. 1. Performance measurement of agile teams
  2. 2. | Performance measurement of agile teams continuous development of a product – performance measured by sprint Cracow Poland, October 7 2015 2Performance measurement of agile teams Harold van Heeringen, METRI Theo Prins, Sogeti Nederland Edwin van Gorp, Sogeti Nederland
  3. 3. | Sizing, Estimating & Control  Sogeti department responsible for:  Functional size measurement: Nesma, IFPUG and COSMIC;  Estimating new projects, releases and contracts;  Project control based on software metrics;  Performance measurement;  Benchmarking;  Performance reporting.  This presentation is about one engagement where the SEC department was asked to help solving the issues with regard to the productivity of a Sogeti agile team in a government project…. measured per sprint! Performance measurement of agile teams 3
  4. 4. | Background  RfP of Dutch government agency in 2014  Agile development of a product, taken over from another supplier;  Performance measurement of the team per sprint;  Based on ISO/IEC norm, in this case Nesma function points;  Realistic targets set, based on Sogeti historical data, external benchmark data and historical data of previous supplier!  Sogeti offered to measure performance based on Project Delivery Rate (PDR), measured in project function points delivered per sprint;  Politics: Customer needs to prove the new supplier performs better!  Customer selects Sogeti because of proved historical performance and specialized department Sizing, Estimating & Control. 4Performance measurement of agile teams
  5. 5. | The problem  The application turned out to be very complex. Even the OTAP environment was very complex to handle. The product backlog consisted of many non-functional backlog items;  Functional size measurement is a powerful way to objectively size progress when it comes to functionality added, changed, or deleted, but not for measuring non-functional sprint backlog items processed by the team;  The customer (product owner) decides which functional and non- functional backlog items are put on the sprint backlog;  Traditional data and performance measurement methods turned out to be ineffective when measuring performance on sprint level when more than average non-functional backlog items are put on the sprint backlog. 5Performance measurement of agile teams
  6. 6. | Two types of agile projects 1. Development of a set of specific requirements, prioritized on a backlog, realized in a specific duration by a specific team in a specific amount of effort hours and cost. At one point, the project, and the product, is finished. (in traditional terms: new development or release). 2. Continued development (evolving) an existing application. No definite end goal when the project or product is finished. A year divided into X sprints of Y weeks and a fixed team working to deliver the sprint backlog items. This only ends when the organization decides maintenance is no longer needed. (in traditional terms: maintenance). This industry paper investigates Type number 2. 6Performance measurement of agile teams
  7. 7. | First supplier  Big international system integrator;  Application developed from scratch;  About 15 sprints;  Sprints with fluctuating functional sizes in Nesma FP;  High complexity;  Average PDR about 17 hours/FP.  Almost all backlog items of functional nature!  However, the customer was not happy and decided to select a new supplier. 7Performance measurement of agile teams
  8. 8. | After transition  From Q3 2014 onwards, Sogeti took over;  Sprints of 3 weeks;  Sogeti team, product owner supplied by customer;  Product backlog contains many non-functional items:  Scrum team works hard, but hardly delivers function points;  PDR (h/FP) relatively high, not in line with target PDR;  Customer contract manager blames Sogeti for not being productive;  Contract under pressure, media attention, pressure and politic issue;  Sogeti department Sizing, Estimating & Control asked to analyze the performance and to propose improvements. Performance measurement of agile teams 8
  9. 9. | Function points  Important to understand, using a functional size measurement method (an ISO/IEC standard) means measuring the size of the functional user requirements that are implemented in the software;  Non-functional requirements are not measured at all!  More non-functional work in a sprint means that less functionality is realized, and therefore a higher PDR (hours/FP)/lower productivity.  In project estimation and benchmarking, the influence of NFR is accounted for by the historical data used or the parametric model used, or the peer group that is constructed based on projects with similar characteristics. Performance measurement of agile teams 9
  10. 10. | Story points  Usual way to estimate effort in agile teams;  Team members assign story points to each backlog item, reflecting the amount of work needed to realize the item;  Subjective, not repeatable, not verifiable and not defensible, but mainstream practice in agile teams because of ease of use;  Story point-based metrics can not be compared with any measurements or metrics outside the team;  Story point measurement is not a standard, but SP do take into account the effort spent on non-functional backlog items. Performance measurement of agile teams 10
  11. 11. | Non functional backlog items are important 11Performance measurement of agile teams Sprint X FP SP Effort hours Backlog item 1 4 4 90 Backlog item 2 0 6 120 Backlog item 3 0 2 45 Backlog item 4 5 3 65 Backlog item 5 4 3 80 Total 13 18 400 PDR = 400/13 = 30,8 Hours/FP Ratio F/NF SP backlog items: 10/8
  12. 12. | Sprint performance example Performance measurement of agile teams 12 Sprint 1 2 3 4 5 6 7 8 Effort 345 389 367 412 365 375 390 401 Size (FP) 15 5 16 3 25 0 36 32 Sprint 1 2 3 4 5 6 7 8 PDR (h/FP) 23,0 77,8 22,9 137,3 14,6 n/a 10,8 12,5 0,0 20,0 40,0 60,0 80,0 100,0 120,0 140,0 160,0 3 4 5 6 7 8 9 10 PDR(h/FP) Target PDR (h/FP)
  13. 13. | Issue  Team spends a lot of time on non-functional backlog items;  PDR is not good enough to reach target;  Stakeholders don’t understand this ‘technical issue’ and only see metrics on the dashboard  PDR significant worse than expected;  But… customer product owner decided on the product backlog items to put on the sprint backlog!  So, disappointing PDR mainly caused by the number of functional and non-functional backlog items put in the sprint by the product owner (= the customer!).  Sogeti SEC wishes to address this issue and to come up with a proposal for a more accurate performance measurement method. 13Performance measurement of agile teams
  14. 14. | SEC proposal Agile Normalized Size (ANS) Functional size that could have been realized if the product owner only had put functional backlog items in the sprint backlog. Based on this ANS, a PDR (hour/FP) can be determined that can be compared to the PDR’s in the databases with historical data. 14Performance measurement of agile teams
  15. 15. | Method 1. Measure the functional size of the realized functional backlog items with a standard method (Nesma/IFPUG FPA, COSMIC, …); 2. Determine whether the realized backlog items are functional or non- functional; 3. Determine the number of story points of the functional backlog items realized in the sprint; 4. Determine the total number of story points realized in the sprint; 5. Determine the agile normalized size: (functional size / # functional story points) * total # story points 15Performance measurement of agile teams
  16. 16. | The example extended 16Performance measurement of agile teams Sprint X FP SP Hours Backlog item 1 4 4 90 Backlog item 2 0 6 120 Backlog item 3 0 2 45 Backlog item 4 5 3 65 Backlog item 5 4 3 80 Total 13 18 400 Agile normalized size = (13 / 10) * 18 = 23,4 nFP PDR = 400/23,4 = 17,1 hours/nFP Functional size: 13 FP Functional SP: 10 SP Total SP: 18 SP Regular PDR: 400/13 = 30,7 hours/FP
  17. 17. | The effect in multiple sprints Sprint Size (FP) Functional SP Non-functional SP Total SP ANS (nFP) 16 20 32 12 44 27,5 17 25 28 16 44 39,3 18 18 24 20 44 33,0 19 29 35 4 39 32,3 20 4 6 36 42 28,0 21 15 16 24 40 37,5 17Performance measurement of agile teams
  18. 18. | Advantages / disadvantages Advantages  Reduced influence of non-functional backlog items;  The use of an ISO/IEC FSM standard – ability to benchmark. Disadvantages  Depending on accurate story point assignment (subjective);  Possible for the team to tweak the performance figures;  As the product owner is present, this risk is considered to be small;  Impossible to measure ANS when the functional size delivered is 0. Performance measurement of agile teams 18
  19. 19. | Productivity measurement Sprint Size (FP) ANS (nFP) Hours Hours/FP Hours/nFP 16 20 27,5 500 25,0 18,2 17 25 39,3 480 19,2 12,2 18 18 33,0 530 29,4 16,1 19 29 32,3 468 16,1 14,5 20 4 28,0 534 133,5 19,1 21 15 37,5 522 34,8 13,9 19Performance measurement of agile teams
  20. 20. | The effect in multiple sprints 20Performance measurement of agile teams
  21. 21. | The example Sprint Size (FP) Functional SP Non funct. SP Story Points ANS (nFP) 16 20 32 12 44 27,5 17 25 28 16 44 39,3 18 18 24 20 44 33,0 19 29 35 4 39 32,3 20 4 6 36 42 28,0 21 15 16 24 40 37,5 22 0 0 41 41 n.t.b. 23 18 24 20 44 33,0 21Performance measurement of agile teams
  22. 22. | Sprint 22: no productivity measurement Sprint Size (FP) ANS (nFP) Hours Hours/FP Hours/nFP 16 20 27,5 500 25,0 18,2 17 25 39,3 480 19,2 12,2 18 18 33,0 530 29,4 16,1 19 29 32,3 468 16,1 14,5 20 4 28,0 534 133,5 19,1 21 15 37,5 522 34,8 13,9 22 0 N/A 512 N/A N/A 23 18 33,0 508 28,2 15,4 22Performance measurement of agile teams
  23. 23. | Issue: completely non-functional sprints  In sprint 22, zero function points were delivered;  Size in FP is 0, ANS impossible to determine (dividing by zero);  Impossible to determine productivity.  Solution: progressive approach. 23Performance measurement of agile teams
  24. 24. | Progressive approach  Size measurement and productivity measurement not per sprint, but until the last sprint;  Does not focus on sprint, but on overall performance (∑1-n functional size / ∑1-n functional story points) * ∑1-n total story points 24Performance measurement of agile teams
  25. 25. | Progressive approach Sprint Size (FP) ANS (nFP) Hours Hours (cumulative) ANS Progressive Hours (cum) / nFP (prog) 16 20 27,5 500 500 27,5 18,2 17 25 39,3 480 980 66,0 14,8 18 18 33,0 530 1.510 99,0 15,3 19 29 32,3 468 1.978 132,2 15,0 20 4 28,0 534 2.512 163,6 15,4 21 15 37,5 522 3.034 199,2 15,2 22 0 N/A 512 3.546 231,4 15,3 23 18 33,0 508 4.054 264,3 15,3 25Performance measurement of agile teams
  26. 26. | Difference between the methods Performance measurement of agile teams 26
  27. 27. | Starting points Documentation After each sprint the functional documentation should be made up-to- date and it must be clear:  Which functionality was added in the sprint;  Which functionality was changed in the sprint and in which way;  Which functionality was deleted in the sprint;  This should be part of the definition of done. Effort administration  The effort hours need to be booked in the effort administration in such a way that it is possible to clearly identify the effort hours in scope and out of scope of the performance measurement. Performance measurement of agile teams 27
  28. 28. | Conclusions and recommendations  The productivity of an agile team in a contract can be measured and benchmarked while taking into account the effect of non-functional requirements;  The customer now understands that non-functional backlog items have impact on the PDR when using only Nesma/IFPUG function points in agile projects when measuring on a sprint level. Customer is able to explain that internally and politically. Pressure is less now, because targets are met.  The method can help other organizations as well! Performance measurement of agile teams 28

×