On Inherent Complexity of Computation, by Attila Szegedi
•
0 likes•1,291 views
The system you just recently deployed is likely an application processing some data, likely relying on some configuration, maybe using some plugins, certainly relying on some libraries, using services of an operating system running on some physical hardware. The previous sentence names 7 categories into which we compartmentalise various parts of a computation process that’s in the end going on in a physical world. Where do you draw the line of functionality between categories? From what vantage points do these distinctions become blurry? Finally, how does it all interact with the actual physical world in which the computation takes place? (What is the necessary physical minimum required to perform a computation, anyway?) Let’s make a journey from your AOP-assembled, plugin-injected, YAML-configured, JIT compiled, Hotspot-executed, Linux-on-x86 hosted Java application server talking JSON-over-HTTP-over-TCP-over-IP-over-Ethernet all the way down to electrons. And then back. Recorded at GeekOut 2013.
Recommended
Recommended
More Related Content
Similar to On Inherent Complexity of Computation, by Attila Szegedi
Similar to On Inherent Complexity of Computation, by Attila Szegedi (20)
More from ZeroTurnaround
More from ZeroTurnaround (20)
Recently uploaded
Recently uploaded (20)
On Inherent Complexity of Computation, by Attila Szegedi
- 1. On Inherent Complexity of Computation Attila Szegedi @asz
- 2. THE FOLLOWING IS INTENDED TO OUTLINE OUR GENERAL PRODUCT DIRECTION. FOR INFORMATION PURPOSES ONLY,AND MAY NOT BE INCORPORATED INTO ANY CONTRACT. IT IS NOT A COMMITMENT TO DELIVER ANY MATERIAL, CODE, OR FUNCTIONALITY,AND SHOULD NOT BE RELIED UPON IN MAKING PURCHASING DECISION.THE DEVELOPMENT, RELEASE,AND TIMING OF ANY FEATURES OR FUNCTIONALITY DESCRIBED FOR ORACLE'S PRODUCTS REMAINS AT THE SOLE DISCRETION OF ORACLE.
- 4. I won’t tell you anything humankind doesn’t already know for 40 years.
- 5. I won’t tell you anything humankind doesn’t already know for 40 years.
- 6. I won’t tell you anything humankind doesn’t already know for 40 years. This is not a Lisp talk.
- 7. I won’t tell you anything humankind doesn’t already know for 40 years. This is not a Lisp talk. No, seriously.
- 8. How complex is your system?
- 9. Is it this? Hardware Operating system Application Configuration Persistent data Runtime state
- 10. Is it maybe more like this? Hardware Hypervisor Operating system Virtual machine Application server Application Plugins Configuration Persistent data Runtime state
- 11. Hardware Hypervisor Operating system Virtual machine Application server Application Plugins Configuration Persistent data Runtime state Electricity Electric impulses on a wire Heat
- 12. The electricity to heat part is easy.
- 14. GET / HTTP/1.1 HTTP/1.1 301 Moved Permanently Content-Length: 0 Date: Thu, 02 May 2013 09:13:23 GMT Server: Apache/2.2.17 (Ubuntu) X-Powered-By: PHP/5.3.5-1ubuntu7.11 X-Pingback: http://geekout.ee/xmlrpc.php Location: http://geekout.ee/ Cache-Control: max-age=300 Expires: Thu, 02 May 2013 09:18:23 GMT Vary: Accept-Encoding Connection: close Content-Type: text/html; charset=UTF-8
- 15. GET / HTTP/1.1 HTTP/1.1 301 Moved Permanently Content-Length: 0 Date: Thu, 02 May 2013 09:13:23 GMT Server: Apache/2.2.17 (Ubuntu) X-Powered-By: PHP/5.3.5-1ubuntu7.11 X-Pingback: http://geekout.ee/xmlrpc.php Location: http://geekout.ee/ Cache-Control: max-age=300 Expires: Thu, 02 May 2013 09:18:23 GMT Vary: Accept-Encoding Connection: close Content-Type: text/html; charset=UTF-8 Hardware Hypervisor Operating system Virtual machine Application server Application Plugins Configuration Persistent data Runtime state
- 16. Hardware Hypervisor Operating system Virtual machine Application server Application Plugins Configuration Persistent data Runtime state Complexity Abstraction
- 17. How many layers are there? Are they inherent? Are they real?
- 18. Hardware Hypervisor Operating system Virtual machine Application server Application Plugins Configuration Persistent data Runtime state
- 19. Hardware Operating system Virtual machine Application A virtual machine takes bytecode and transforms it into something resembling a native application on the OS/hardware.
- 20. Hardware Virtual machine Application Then there are some attempts to even eliminate the operating system.
- 21. Hardware And some to go all the way down to
- 22. BitCoin craze MHash/J Generic CPU 0.1-1.14 GPU 2.4 FPGA 25 ASIC 160 https://en.bitcoin.it/wiki/Mining_hardware_comparison
- 23. BitCoin craze MHash/J Generic CPU 0.1-1.14 GPU 2.4 FPGA 25 ASIC 160 https://en.bitcoin.it/wiki/Mining_hardware_comparison
- 24. These are actual products
- 25. Hardware Hypervisor Operating system Virtual machine Application server Application Plugins Configuration Persistent data Runtime state
- 26. Application Configuration In languages where code is first- class data, you don’t have to distinguish between the two.
- 27. Application = Configuration • In Lua, recommended data serialization format is Lua source code. • You can just eval() JSON in JavaScript. • Lisp.‘nuff said. • Standard disclaimer about having your program execute unverified external code.
- 28. No, seriously, what is computation?
- 29. Computation is a transition from a more likely to a less likely state. (Decrease in physical entropy; increase in information entropy.)
- 30. Wait what? • In information theory, entropy is the measure of uncertainty of state. • In physics, it’s the amount of missing information to precisely describe the state of the system. • The two usages of the term are therefore exactly opposite. • Way to troll each other, fellow scientists!
- 31. This is very likely This is highly unlikely
- 32. This is very likely •An empty database •An uninitialized disk •RAM when power is turned on •Quiet network
- 33. This is very likely This is highly unlikely
- 34. This is highly unlikely •A populated database •A populated disk •RAM after substantial activity •Active network
- 35. Humans are part of the computation • Having an ongoing record of people’s thoughts and events persisted in a social network database is less likely than not having them. • Just think about the combined effort of creating Twitter/Facebook/etc databases • both people building and operating it, and • people pouring data into it.
- 36. Humans are part of the computation
- 37. Humans are part of the computation • Thinking of boundaries of identities in real world can be quite fascinating.
- 38. Humans are part of the computation • Thinking of boundaries of identities in real world can be quite fascinating. • Where does a computer end?
- 39. Humans are part of the computation • Thinking of boundaries of identities in real world can be quite fascinating. • Where does a computer end? • Where does a person end?
- 40. Humans are part of the computation • Thinking of boundaries of identities in real world can be quite fascinating. • Where does a computer end? • Where does a person end? • Can we influence another person in such a way that part of us actually lives in them?
- 41. Humans are part of the computation
- 42. Humans are part of the computation • Where does a computer end?
- 43. Humans are part of the computation • Where does a computer end? • Where does a person end?
- 44. Humans are part of the computation • Where does a computer end? • Where does a person end? • Can we influence another person in such a way that part of us actually lives in them?
- 45. Humans are part of the computation
- 46. Humans are part of the computation • Where does a computer end?
- 47. Humans are part of the computation • Where does a computer end? • Where does a person end?
- 48. Humans are part of the computation • Where does a computer end? • Where does a person end? • Can we influence another person in such a way that part of us actually lives in them?
- 49. Humans are part of the computation
- 50. Arrow of time • Transitioning from more likely to less likely gives us an arrow of time. • We naturally presume that more likely states precede less likely ones. • Opposite cases happen, and they are associated with idea of destruction.
- 51. Is unzip not a computation then? • It creates more likely (decompressed) from less likely (compressed) data! • It is still a computation, though.
- 52. Is unzip not a computation then? • You started with not having both the compressed and decompressed data, and you end up with both. • That’s less likely than just having the compressed data.
- 53. Arrow of time revisited • Future is a quantum superposition of all possible successive quantum states. • Past, too, is a quantum superposition of all possible preceding quantum states. • See Wheeler’s Delayed Choice Experiment.
- 54. This seemed like a good time for this slide
- 55. Thermodynamics in the way • Any physical process that dissipates heat will be irreversible. • It’s possible to go: • fast, hot and irreversible, or • slow, cool, and reversible. • (And any shade of gray in between.)
- 56. Thermodynamics in the way • e x t = const • e := energy dissipation in form of heat • t := time to operate an electric circuit • Going slow conserves energy
- 57. Why does the arrow of time matter anyway?
- 58. Here’s why
- 59. Here’s why A B A NAND B
- 60. Here’s why A B A NAND B Direction
- 61. Reversible gate A B A NAND B Direction reversing bit
- 62. Reversible gate A B A NAND B Direction reversing bit
- 63. Meandering. Not caring which way it flows.
- 66. How fast can we go?
- 67. How fast can we go? • I couldn’t figure out hard data for this…
- 68. How fast can we go? • I couldn’t figure out hard data for this… • Theoretical upper limit: divide highest power density with lowest energy for a physical state transition; you get “operations/m3/s”.
- 69. How fast can we go? • I couldn’t figure out hard data for this… • Theoretical upper limit: divide highest power density with lowest energy for a physical state transition; you get “operations/m3/s”. • Problem with:
- 70. How fast can we go? • I couldn’t figure out hard data for this… • Theoretical upper limit: divide highest power density with lowest energy for a physical state transition; you get “operations/m3/s”. • Problem with: • relativistic effects
- 71. How fast can we go? • I couldn’t figure out hard data for this… • Theoretical upper limit: divide highest power density with lowest energy for a physical state transition; you get “operations/m3/s”. • Problem with: • relativistic effects • controlling those amounts of energy output
- 72. Where are we? • We could go much hotter and faster. • Insert science fiction of atomic particle machines that can run few hours of subjective human brain simulation in few femtoseconds before heat-disintegrating. • We could also go much slower and cooler • Ain’t nobody got time for that!
- 74. Digital computation • Binary inputs go in, binary output goes out • Not all of the input precedes the output (think streams). • Later input can be shaped by earlier output (client resubmits a cookie received from server)
- 75. Digital computation • A particular computation (or “program”) is mapping of all possible inputs to all possible outputs.
- 78. “” -> “10001111000101101010010000010111110000010010001010100101010010100” “0” -> “0100010101100101110110110101101” “1” -> “010001010110100000010001010111110010101000011100101” “01” -> “111100010110101001000001011111000001001000101010010101001010” ... • We don’t write programs this way. Right? • We recognize patterns and map classes of inputs to classes of outputs. • It’s basically a compressed representation. • If the number of patterns is not finite, you’re describing a random process.
- 79. Kolmogorov complexity • Defined for strings. • The length of a program that produces the given string. • The programming language doesn’t matter • ‘cause you can just prefix your program with an interpreter written in another language, for a constant difference. • Can also apply to infinite sequences, such as our “enumerate all programs”
- 80. Inherent complexity of a computation
- 81. Inherent complexity of a computation
- 82. Inherent complexity of a computation • Inherent complexity of the computation: the length of the most compact form that can encode its full input-output mapping.
- 83. Inherent complexity of a computation • Inherent complexity of the computation: the length of the most compact form that can encode its full input-output mapping. • Incidentally, a string is “random” if it’s shorter than any program that can generate it.
- 85. • Oh, wait, Kolmogorov complexity is a non- computable function.
- 86. • Oh, wait, Kolmogorov complexity is a non- computable function.
- 87. • Oh, wait, Kolmogorov complexity is a non- computable function. • Consequence: no matter how good your optimizing compiler is, you can never prove that there’s not an even better one.
- 88. • Oh, wait, Kolmogorov complexity is a non- computable function. • Consequence: no matter how good your optimizing compiler is, you can never prove that there’s not an even better one. • a.k.a “Full Employment Theorem” for compiler writers.
- 91. • Sometimes, part of the decoder is external.
- 92. • Sometimes, part of the decoder is external. • It can be in your head.
- 93. • Sometimes, part of the decoder is external. • It can be in your head. • Again, human is part of the computation.
- 94. Complex look is sometimes a simple pattern
- 95. Complex look is sometimes a simple pattern
- 96. Complex look is sometimes a simple pattern SzegediButterfly1 { init: z = #pixel loop: float x = real(z) float y = imag(z) z = sqr(y) - sqrt(abs(x)) + (sqr(x) - sqrt(abs(y))) * 1i + #pixel bailout: |z| <= @Bailout default: param Bailout caption = "Bailout Value" default = 127.0 endparam }
- 97. Complex look is sometimes a simple pattern
- 98. Complex look is sometimes a simple pattern
- 99. Complex look is sometimes a simple pattern SzegediButterfly2 { init: z = #pixel loop: float x = real(z) float y = imag(z) z = sqr(x) - sqrt(abs(y)) + (sqr(y) - sqrt(abs(x))) * 1i + #pixel bailout: |z| <= @Bailout default: param Bailout caption = "Bailout Value" default = 127.0 endparam }
- 100. Even if you could find the most compact form… • … you probably wouldn’t want to use it. • It would be even less humanly maintainable than flipping bits in a zipped source file. • Ever seen 256-byte 8-bit CPU demos?
- 101. You can always reduce complexity • You always have excess complexity in your system. • It’s worth thinking of how to reduce it. • Pattern recognition (refactoring). • Favor environments that don’t force you into artificial layer boundaries. Code is data, data is code. • Patterns can be more overarching without artificial boundaries.
- 102. Artificial boundaries hobble you • Favor environments that don’t force you into artificial layer boundaries. Code is data, data is code. • Patterns can be more overarching without artificial boundaries. • Java program loads an XML file containing a flow graph and some embedded JavaScript? • You have a problem.
- 103. Take aways • Be sensible with the effort and the result. • Don’t overshoot into compact-but- unreadable territory. • Hitting on the minimally complex representation is as likely as inventing the perpetuum mobile. (Thermodynamics metaphor again.)
- 106. Image Credits Snowy branches: http://www.flickr.com/photos/aeioux/2398264997/ XKCD comic: http://xkcd.com/297/ Complex gizmo: http://www.flickr.com/photos/michaelheiss/2871996129/ Electric heater: http://www.flickr.com/photos/jocelynb/426268348/ Layer cake: http://www.flickr.com/photos/julessilver/3259734572 Abacus: http://www.flickr.com/photos/skidder/37675092 Airplanes: http://www.flickr.com/photos/good_day/198611998/ Meandering: http://www.flickr.com/photos/31856336@N03/7831522814/ Headphones: http://www.flickr.com/photos/doremigirl/8362322435 Kolmogorov Lecture: http://en.wikipedia.org/wiki/File:Kolm_complexity_lect.jpg All used images are Creative Commons licensed, and used according to their terms of license.