The Technical Debt Trap - NDC Oslo 2014

1. The  Technical Debt Trap Doc Norton doc@groupon.com @DocOnDev

2. Doc Norton! Global Director of Engineering Culture! doc@groupon.com | @DocOnDev Husband Father

3. Grandfather!

4. What is Technical Debt?

5. Shipping ﬁrst time code is like going into debt. OOPSLA ‘92 Ward Cunningham

6. Shipping ﬁrst time code is like going into debt. A little debt speeds development so long as it is paid back promptly with a rewrite. OOPSLA ‘92 Ward Cunningham

7. The danger occurs when the debt is not repaid. Every minute spent on not-quite-right code counts as interest on that debt. OOPSLA ‘92 Ward Cunningham

8. The danger occurs when the debt is not repaid. Every minute spent on not-quite-right code counts as interest on that debt. OOPSLA ‘92 Ward Cunningham

9. Technical Debt is Good

10. WTF? Technical Debt is Good

11. Technical Debt is Good Strategic Design Decision ❖ Allow for Rapid Delivery! ❖ To Elicit Quick Feedback! ❖ And Correct Design

12. Technical Debt is Good Indication of Learning ❖ Now know what you need! ❖ Implementation doesn’t match!

13. Technical Debt is a Metaphor Here Be Danger

14. Metaphor’s Rock We Reason by Analogy Building on a weak foundation Puts pressure on our design Can’t keep running at this pace It’s raining men (hallelujah)

15. Metaphorphosis When Metaphors Go Wrong SHORT-TERM Long-Term INADVERTENT RECKLESS DEBT IN THE THIRD QUADRANTPrudent Intentional Credit Card FRAUDULENT AUTO LOAN Student Loan HOME LOAN Loan Shark Return on Investment PYRAMID SCHEME VOLUNTARY PRAGMATIC LEVERAGE HIGH INTEREST OPERATIONAL

16. Metaphorphosis When Metaphors Go Wrong “QUICK AND DIRTY” MARTIN FOWLER HTTP://WWW.MARTINFOWLER.COM/BLIKI/TECHNICALDEBT.HTML “SLOPPY” DAVID LARIBEE HTTP://MSDN.MICROSOFT.COM/EN-US/MAGAZINE/EE819135.ASPX “JUST HACK IT IN” STEVE MCCONNELL HTTP://BLOGS.CONSTRUX.COM/BLOGS/STEVEMCC/ARCHIVE/2007/11/01/TECHNICAL-DEBT-2.ASPX “CUT A LOT OF CORNERS” JAMES SHORE HTTP://JAMESSHORE.COM/BLOG/CARDMEETING/VOLUNTARY-TECHNICAL-DEBT.HTML

17. The Technical Debt Quadrant http://martinfowler.com/bliki/TechnicalDebtQuadrant.html

18. [Many] have explained the debt metaphor and confused it with the idea that you could write code poorly with the intention of doing a good job later. YouTube ‘09 Ummm… No. http://www.youtube.com/watch?v=pqeJFYwnkjE&feature=player_embedded

19. confused the debt metaphor with the idea that you could write code poorly

20. The ability to pay back debt [...] depends upon you writing code that is clean enough to be able to refactor as you come to understand your problem. YouTube ‘09 Clean Code is Required http://www.youtube.com/watch?v=pqeJFYwnkjE&feature=player_embedded

21. The ability to pay back debt [...] depends upon you writing code that is clean enough to be able to refactor as you come to understand your problem. YouTube ‘09 Clean Code is Required http://www.youtube.com/watch?v=pqeJFYwnkjE&feature=player_embedded

22. Twitter ‘09 Dirty Code is a Bad Idea Dirty code is to technical debt as the pawn broker is to ﬁnancial debt.! http://twitter.com/WardCunningham/status/3742903303 Don’t think you are ever going to get your code back.

23. Is it Technical Debt? Ask yourself… ❖ Is the code clean?! ❖ Is the code tested?! ❖ Is there a learning objective or event?! ❖ Is there a plan for payback?! ❖ Is the business truly informed?

24. Is it Technical Debt? If you say no to even one… ❖ Is the code clean?! ❖ Is the code tested?! ❖ Is there a learning objective or event?! ❖ Is there a plan for payback?! ❖ Is the business truly informed? ... then you don’t have Technical Debt

25. You have a mess Mess (noun) ❖ Disorderly accumulation, heap, or jumble! ❖ A state of embarrassing confusion! ❖ An unpleasant or difﬁcult situation

26. You have cruft Cruft (noun) ❖ An unpleasant substance! ❖ The result of shoddy construction! ❖ Redundant, old or improperly written code

27. Chill. It’s just semantics, man. Just Semantics? Technical Debt is Good

28. Chill. It’s just semantics, man. Just Semantics? Technical Debt is GoodQuick and Dirty is Technical DebtQuick and Dirty is Good

29. Chill. It’s just semantics, man. Just Semantics? Quick and Dirty is GoodQuick and Dirty is Good

30. The Technical Debt Quadrant http://martinfowler.com/bliki/TechnicalDebtQuadrant.html

31. The Technical Debt Quadrant http://martinfowler.com/bliki/TechnicalDebtQuadrant.html “Let’s deploy and gather more information.”

32. Technical Debt in Other Fields

35. Technical Debt in Other Fields RECKLESS AND DELIBERATE RECKLESS AND DELIBERATE RECKLESS AND INADVERTENT

36. The Technical Debt Quadrant http://martinfowler.com/bliki/TechnicalDebtQuadrant.html IRRESPONSIBLE INCOMPETENT “Let’s deploy and gather more information.” TECHNICAL DEBT

37. Cruft or Debt? DataSet aDs, qDs;! aDs = _dbConnector.UpdateAgentList();! qDs = _dbConnector.GetQueueList();! foreach (DataTable aTable in aDs.Tables) {! foreach (DataRow aRow in aTable.Rows) {! foreach (DataColumn aColumn in aTable.Columns) {! DataSet asDs = _dbConnector.GetAgentSkills(aRow[aColumn].ToString());//AgentId! foreach (DataTable asTable in asDs.Tables) {! foreach (DataRow asRow in asTable.Rows) {! foreach (DataColumn asColumn in asTable.Columns) {! foreach (DataTable qTable in qDs.Tables) {! foreach (DataRow qRow in qTable.Rows) {! foreach (DataColumn qColumn in qTable.Columns) {! DataSet sqDs = _dbConnector.GetSkillsForQueue(qRow[qColumn].ToString());! foreach (DataTable sqTable in sqDs.Tables) {! foreach (DataRow sqRow in sqTable.Rows) {! foreach (DataColumn sqColumn in sqTable.Columns) {! foreach (string skill in sqRow[sqColumn].ToString().Split(paramDelimStr)) {! if (skill == asRow[asColumn].ToString()) {! try {! _dbConnector.SetAgentQueueSkill(aRow[aColumn].ToString(),! qRow[qColumn].ToString(),! skill);! }! catch { continue; }! }! }! }! }! }! }! }! }! }! }! }! }! }! } http://thedailywtf.com/Series/2010/3/CodeSOD.aspx

38. Cruft or Debt? DataSet aDs, qDs;! aDs = _dbConnector.UpdateAgentList();! qDs = _dbConnector.GetQueueList();! foreach (DataTable aTable in aDs.Tables) {! foreach (DataRow aRow in aTable.Rows) {! foreach (DataColumn aColumn in aTable.Columns) {! DataSet asDs = _dbConnector.GetAgentSkills(aRow[aColumn].ToString());//AgentId! foreach (DataTable asTable in asDs.Tables) {! foreach (DataRow asRow in asTable.Rows) {! foreach (DataColumn asColumn in asTable.Columns) {! foreach (DataTable qTable in qDs.Tables) {! foreach (DataRow qRow in qTable.Rows) {! foreach (DataColumn qColumn in qTable.Columns) {! DataSet sqDs = _dbConnector.GetSkillsForQueue(qRow[qColumn].ToString());! foreach (DataTable sqTable in sqDs.Tables) {! foreach (DataRow sqRow in sqTable.Rows) {! foreach (DataColumn sqColumn in sqTable.Columns) {! foreach (string skill in sqRow[sqColumn].ToString().Split(paramDelimStr)) {! if (skill == asRow[asColumn].ToString()) {! try {! _dbConnector.SetAgentQueueSkill(aRow[aColumn].ToString(),! qRow[qColumn].ToString(),! skill);! }! catch { continue; }! }! } } } } } } } } } } } } } http://thedailywtf.com/Series/2010/3/CodeSOD.aspx

39. Cruft or Debt? DataSet aDs, qDs, asDs, sqDs;! aDs = _dbConnector.UpdateAgentList();! qDs = _dbConnector.GetQueueList();! ! foreach (DataRow aRow in aDS.Tables[0].Rows) {! String agentID = aRow[“AgentId”].ToString();! asDs = _dbConnector.GetAgentSkills(agentID);! foreach (DataRow asRow in asDs.Tables[0].Rows) {! String agentSkill = asRow[“Skill”].ToString();! foreach (DataRow qRow in qDs.Tables[0].Rows) {! queueName = qRow[“QueueName”].ToString();! sqDs = _dbConnector.GetSkillsForQueue(queueName);! foreach (DataRow sqRow in sqDs.Tables[0].Rows) {! foreach (string skill in sqRow[“Skills”].ToString().Split(paramDelimStr)) {! if (skill == agentSkill) {! try { _dbConnector.SetAgentQueueSkill(agentID, queueName, skill); }! catch { continue; }! }! }! }! }! }! } http://thedailywtf.com/Series/2010/3/CodeSOD.aspx

40. Cruft or Debt? AgentList agents = new AgentList(_dbConnector.UpdateAgentList());! QueueList queues = new QueueList(_dbConnector.GetQueueList());! ! foreach (Agent agent in agents) {! foreach (Skill agentSkill in agent.skills) {! foreach (Queue queue in queues) {! foreach (Skill queueSkill in queue.skills.Where(x => x == agentSkill)) {! try {_dbConnector.SetAgentQueueSkill(agent.agentID, queue.name, agentSkill); }! catch { continue; }! }! }! }! } http://thedailywtf.com/Series/2010/3/CodeSOD.aspx

41. Cruft or Debt? if ((customer.state == “AL” && customer.type ==! CustomerType.GENERAL_AGENT && customer.revenue > 100000)! || (customer.type == CustomerType.RETRO_AGENT &&! (customer.state == “WI” || customer.state == “IL”))! || (customer.type == CustomerType.FED_MANAGEMENT &&! customer.revenue > 150000)) { ... }

42. Cruft or Debt? // If customer is Federally Regulated! if ((customer.state == “AL” && customer.type ==! CustomerType.GENERAL_AGENT && customer.revenue > 100000)! || (customer.type == CustomerType.RETRO_AGENT &&! (customer.state == “WI” || customer.state == “IL”))! || (customer.type == CustomerType.FED_MANAGEMENT &&! customer.revenue > 150000)) { ... }

43. Cruft or Debt? if (customer.isFederallyRegulated()) { ... }

44. Cruft or Debt? double getSpeed() {! switch (_type) {! case EUROPEAN:! return getBaseSpeed();! case AFRICAN:! return getBaseSpeed() - getLoadFactor() * _numberOfCoconuts;! case NORWEGIAN_BLUE:! return (_isNailed) ? 0 : getBaseSpeed(_voltage);! }! throw new RuntimeException ("Should be unreachable");! }! http://www.refactoring.com/catalog/replaceConditionalWithPolymorphism.html

45. Cruft or Debt? class Swallow ...! double getSpeed() { return getBaseSpeed(); }! end class! ! class EuropeanSwallow ...! end class! ! class AfricanSwallow ...! double getSpeed() { return super.getSpeed - coconutLoad(); }! double coconutLoad() { return getLoadFactor() * _numberOfCoconuts; }! end class! ! class NorwegianSwallow ...! double getSpeed() { return (_isNailed) ? 0 : getBaseSpeed(_voltage); }! end class

46. Cruft is a bad decision Every Time ❖ You are a professional developer! ❖ You’re going to create unintentional cruft! ❖ You have to clean up the existing cruft

47. The Trap Cruft begets Cruft ❖ Precedent for speed over quality! ❖ Expectation of increased velocity! ❖ Cruft slows you down! ❖ Must write more cruft to keep up! ❖ Ask permission to do your job correctly

48. Avoid The Trap Thresholds set false expectations http://blog.castsoftware.com/wp-content/uploads/2011/04/Technical-Debt-Software-Quality1.jpg

49. Avoid The Trap Incremental ﬁxes fail http://www.jacoozi.com/blog/wp-content/uploads/2007/01/refactoring_coc_big.jpg

50. http://www.jacoozi.com/blog/wp-content/uploads/2007/01/refactoring_coc_big.jpg

51. Avoid The Trap Clean Constantly ❖ Never make an intentional mess! ❖ Monitor your “Technical Debt”! ❖ Follow the Boy Scout Rule! ❖ Remember quality is your responsibility! ❖ NEVER ask permission to do your job correctly

52. Monitoring Cruft/Debt A few key metrics ❖ Code Coverage! ❖ Code Complexity! ❖ Coupling! ❖ Maintainability

53. Monitoring Cruft/Debt Code Coverage ❖ Code exercised by automated tests! ❖ Monitor types of tests separately! ❖ Don’t set a coverage target! ❖ 100% coverage is a smell! ❖ Monitor Trends, Not Points

54. Monitoring Cruft/Debt Complexity ❖ Count of logical branches in code! ❖ Direct relationship with bugs! ❖ Typically high in concentrated areas! ❖ Reduce conditionals! ❖ Monitor Trends, Not Points

55. Monitoring Cruft/Debt Coupling ❖ Interconnectedness of systems! ❖ Highly coupled is difﬁcult to change! ❖ Afferent (toward) and Efferent (away)! ❖ Dependency Injection! ❖ Monitor Trends, Not Points

56. Monitoring Cruft/Debt Make stories for cruft ❖ Velocity! ❖ Scalability! ❖ Availability! ❖ Maintainability

57. Review Technical Debt ❖ A strategic design decision! ❖ Requires business to be informed! ❖ Includes a pay-back plan Cruft ❖ Happens! ❖ Needs to be monitored and cleaned! ❖ Is NOT Technical Debt

58. Review Technical Debt ❖ A strategic design decision! ❖ Requires business to be informed! ❖ Includes a pay-back plan Cruft ❖ Happens! ❖ Needs to be monitored and cleaned! ❖ Is NOT Technical Debt NEVER ask perm ission to do your job correctly

59. The  Technical Debt Trap Doc Norton! doc@groupon.com! @DocOnDev Comments / Questions?

60. The  Technical Debt Trap Doc Norton! doc@groupon.com! @DocOnDev Thank You!