Authors: Carol V. Alexandru¹, José J. Merchante², Sebastiano Panichella¹,³ Sebastian Proksch¹, Harald C. Gall¹, Gregorio Robles² ¹Software Evolution and Architecture Lab, University of Zurich, Switzerland {alexandru,proksch,gall}@ifi.uzh.ch ²Grupo de Sistemas y Comunicaciones, Universidad Rey Juan Carlos, Spain jj.merchante@alumnos.urjc.es, grex@gsyc.urjc.es ³Service Prototype Lab, Zurich University of Applied Sciences, Switzerland panc@zhaw.ch, spanichella@gmail.com ABSTRACT: Developers discuss software architecture and concrete source code implementations on a regular basis, be it on question answering sites, online chats, mailing lists or face to face. In many cases, there is more than one way of solving a programming task. Which way is best may be decided based on case-specific circumstances and constraints, but also based on convention. Having strong conventions, and a common vocabulary to express them, simplifies communication and strengthens common understanding of software development problems and their solutions. While many programming ecosystems have a common vocabulary, Python’s relationship to conventions and common language is a particularly pronounced. The “Zen of Python”, a famous set of high-level coding conventions authored by Tim Peters, states “There should be one, and preferably only one, obvious way to do it”. This ‘one way to do it’ is often referred to as the ‘Pythonic’ way: the ideal solution to a particular problem. Few other programming languages have coined a unique term to label the quality of craftsmanship gone into a software artifact. In this paper, we explore how Python developers understand the term ‘Pythonic’ by means of structured interviews, build a catalogue of ‘pythonic idioms’ gathered from literature, and conjecture on the effects of having a language-specific term for quality code, considering the potential it could hold for other programming languages and ecosystems. We find that while the term means different things to novice versus experienced Python developers, it.

1. On the Usage of Pythonic Idioms
Carol V. Alexandru¹, José J. Merchante², Sebastiano Panichella¹,
³
Sebastian Proksch¹, Harald C. Gall¹, Gregorio Robles²
¹Software Evolution and Architecture Lab, University of Zurich, Switzerland
{alexandru,proksch,gall}@ifi.uzh.ch
²Grupo de Sistemas y Comunicaciones, Universidad Rey Juan Carlos, Spain
jj.merchante@alumnos.urjc.es, grex@gsyc.urjc.es
³Service Prototype Lab, Zurich University of Applied Sciences, Switzerland
panc@zhaw.ch
Onward! 2018
Boston Park Plaza Hotel
Boston, Massachusetts, United States
07 November 2018

3. Harald C.
Gall
Sebastiano
Panichella
Carol V.
Alexandru
Sebastian
Proksch
Gregorio
Robles
José J.
Merchante

4. 4
Things to know about Python

5. 5
Things to know about Python
●
Created by Guido van Rossum

6. 6
Things to know about Python
●
Created by Guido van Rossum
– He is (was) the “Benevolent Dictator for Life”
– Makes the final decisions when necessary

7. 7
Things to know about Python
●
Created by Guido van Rossum
– He is (was) the “Benevolent Dictator for Life”
– Makes the final decisions when necessary
●
Strong principles
– The “Zen of Python” >>> import this

8. 8
Things to know about Python

9. 9
Things to know about Python
●
Created by Guido van Rossum
– He is (was) the “Benevolent Dictator for Life”
– Makes the final decisions when necessary
●
Strong principles
– The “Zen of Python” >>> import this
– Python Enhancement Proposals (PEPs)

10. 10
Things to know about Python
●
Created by Guido van Rossum
– He is (was) the “Benevolent Dictator for Life”
– Makes the final decisions when necessary
●
Strong principles
– The “Zen of Python” >>> import this
– Python Enhancement Proposals (PEPs)
●
Widespread adoption across many fields

11. 11
Pythonic appears to be 'a thing'

12. 12
Pythonic appears to be 'a thing'
“It would be
more pythonic
to...”

13. 13
A simple question

14. 14
Top Answer claims to be “pythonic”...

15. 15
...but 100s of people seem sceptical

16. 16
“Pythonic” because of the style guide (PEP 8)

17. 17
“Pythonic” because of performance

18. 18
Idioms and signaling

19. 19
Idioms and signaling

20. 20
And there are exceptions

21. 21
Creative use of the word “Pythonic”

22. 22
Is there a definition for “Pythonic”?

23. 23
Is there a definition for “Pythonic”?

24. 24
Is there a definition for “Pythonic”?

25. 25
Is there a definition for “Pythonic”?

26. 26
Pythonic vs. Non-pythonic

27. 27
Pythonic vs. Non-pythonic

28. 28
Pythonic vs. Non-pythonic

29. 29
Is there a definition for “Pythonic”?
So it's “Using Python-specific syntax and concepts”, right?
But what do developers believe?

30. 30
Let's ask a few developers
Interviews done
●
in Person
●
at a Python conference
●
in Spain
●
using open questions

31. 31
Let's ask a few developers
Interviews done
●
in Person
●
at a Python conference
●
in Spain
●
using open questions

32. 32
What does Pythonic mean?

33. 33
What does Pythonic mean?
“makes code easier to
understand and maintain”
“elegant and readable code”
“boosts readability and
performance”

34. 34
What does Pythonic mean?
“makes code easier to
understand and maintain”
“using features provided
by the language or
standard library”
“elegant and readable code”
“boosts readability and
performance”

35. 35
What does Pythonic mean?
“makes code easier to
understand and maintain”
“using features provided
by the language or
standard library”
“elegant and readable code”
“boosts readability and
performance”
“simply the most
accepted way of writing
python”

36. 36
Using Python idioms != Pythonic

37. 37
Using Python idioms != Pythonic
→ Using idioms != Pythonic code
→ Using idioms != always more readable

38. 38
Novice vs. Pro
Novice:
– Better style
– Fewer lines of code
Pro:
– Using built-in functionality
– Efficient execution

39. 39
Novice vs. Pro
Novice:
– Better style
– Fewer lines of code
– Using idioms
Pro:
– Using built-in functionality
– Efficient execution
– Writing elegant code

40. 40
Novice vs. Pro
Novice:
– Better style
– Fewer lines of code
– Using idioms
– Simpler interpretation
Pro:
– Using built-in functionality
– Efficient execution
– Writing elegant code
– Less concrete interpretation

41. 41
Learning 'Pythonic'?

42. 42
Learning 'Pythonic'?
“StackOverflow shows you multiple points of
view of people, and you always learn.”

43. 43
Learning 'Pythonic'?
“StackOverflow shows you multiple points of
view of people, and you always learn.”
“reading code in repositories of other projects”
“saw them in documentation”

44. 44
Learning 'Pythonic'?
“StackOverflow shows you multiple points of
view of people, and you always learn.”
“reading code in repositories of other projects”
“from colleagues
during code review”
“saw them in documentation”

45. 45
Learning 'Pythonic'?
“StackOverflow shows you multiple points of
view of people, and you always learn.”
“Becoming a pythonic programmer takes time”
“My code became more
pythonic year after year”
“reading code in repositories of other projects”
“from colleagues
during code review”
“saw them in documentation”

46. 46
Learning 'Pythonic'?
“StackOverflow shows you multiple points of
view of people, and you always learn.”
“Becoming a pythonic programmer takes time”
“My code became more
pythonic year after year”
“reading code in repositories of other projects”
“from colleagues
during code review”
“saw them in documentation”
→ Pythonic not taught in books or lectures
→ Seems to creep in with experience

47. 47
Do you care? Do your peers care?

48. 48
Do you care? Do your peers care?
“Pythonic code is positively
viewed but not required”

49. 49
Do you care? Do your peers care?
“Pythonic code is positively
viewed but not required”
“If you're a novice Python programmer,
better focus on general programming skills”

50. 50
Do you care? Do your peers care?
“Pythonic idioms can at least
be used to measure a
developer’s knowledge.”
“Pythonic code is positively
viewed but not required”
“If you're a novice Python programmer,
better focus on general programming skills”

51. 51
Do you care? Do your peers care?
“Pythonic idioms can at least
be used to measure a
developer’s knowledge.”
“Pythonic code is positively
viewed but not required”
“If you're a novice Python programmer,
better focus on general programming skills”
“If I learn a new idiom I add it to my toolbox and then when I touch
something, I modify it and leave it better, but it’s not an obsession.

52. 52
Do you care? Do your peers care?
“Pythonic idioms can at least
be used to measure a
developer’s knowledge.”
“Pythonic code is positively
viewed but not required”
“If you're a novice Python programmer,
better focus on general programming skills”
“If I learn a new idiom I add it to my toolbox and then when I touch
something, I modify it and leave it better, but it’s not an obsession.
→ Pythonic important, but not formally
→ Pythonic signals expertise and garners respect

53. 53
A catalogue of Pythonic Idioms

54. 54
A catalogue of Pythonic Idioms
●
Published online at http://pythonic.libresoft.info/catalogue
– including examples, references and benchmarks

55. 55
A catalogue of Pythonic Idioms
●
Published online at http://pythonic.libresoft.info/catalogue
– including examples, references and benchmarks
●
Compiled from
– Several books (on learning and applying Python)
– Online statements by influential and renowned Python developers

56. 56
A catalogue of Pythonic Idioms
●
Published online at http://pythonic.libresoft.info/catalogue
– including examples, references and benchmarks
●
Compiled from
– Several books (on learning and applying Python)
– Online statements by influential and renowned Python developers
●
Classified into “performance” and “readability”
– performance measured using benchmarks

57. 57
A catalogue of Pythonic Idioms

58. 58
Empirical study

59. 59
Empirical study setup
●
Most recent commit in 1000 Python projects from GitHub
– >1mb, sorted by stars, cleaned for books etc., no forks, not archived
– 178,735 files, 38,505,577 lines of code

60. 60
Empirical study setup
●
Most recent commit in 1000 Python projects from GitHub
– >1mb, sorted by stars, cleaned for books etc., no forks, not archived
– 178,735 files, 38,505,577 lines of code
●
Analyzed using LISA (http://t.uzh.ch/Fk)
– AST-based detection of all idioms

61. 61
Empirical study setup
●
Most recent commit in 1000 Python projects from GitHub
– >1mb, sorted by stars, cleaned for books etc., no forks, not archived
– 178,735 files, 38,505,577 lines of code
●
Analyzed using LISA (http://t.uzh.ch/Fk)
– AST-based detection of all idioms
●
Idiom occurences in #projects and total occurence count

62. 62
Empirical study - a quick look
Idiom # projects (out of 1000) # of occurences
List comprehension 866 75,466
Generator expressions 709 33,038
Dict comprehension 146 796

63. 63
Empirical study - a quick look
Idiom # projects (out of 1000) # of occurences
List comprehension 866 75,466
Generator expressions 709 33,038
Dict comprehension 146 796
Simple magic methods
Intermediate magic methods
Advanced magic methods

64. 64
Empirical study - a quick look
Idiom # projects (out of 1000) # of occurences
List comprehension 866 75,466
Generator expressions 709 33,038
Dict comprehension 146 796
Simple magic methods
Intermediate magic methods
Advanced magic methods
__nonzero__(self)
Defines behavior for when bool() is called on an instance of your class.
Should return True or False, depending on whether you would want to
consider the instance to be True or False.
Magic methods

65. 65
Empirical study - a quick look
Idiom # projects (out of 1000) # of occurences
List comprehension 866 75,466
Generator expressions 709 33,038
Dict comprehension 146 796
Simple magic methods 759 78,376
Intermediate magic methods 417 13,255
Advanced magic methods 190 2,613

66. 66
Empirical study - a quick look
Idiom # projects (out of 1000) # of occurences
List comprehension 866 75,466
Generator expressions 709 33,038
Dict comprehension 146 796
Simple magic methods 759 78,376
Intermediate magic methods 417 13,255
Advanced magic methods 190 2,613
→ More in-depth research is needed
→ Detecting anti-idioms is difficult

67. 67
What we learned so far
●
“Pythonic” is important - somehow

68. 68
What we learned so far
●
“Pythonic” is important - somehow
●
“Pythonic” encompasses more than just programming idioms

69. 69
What we learned so far
●
“Pythonic” is important - somehow
●
“Pythonic” encompasses more than just programming idioms
●
Using “Pythonic idioms”...
– makes you appear more knowledgeable
– alone does not necessarily make your code better or more pythonic

70. 70
What we learned so far
●
“Pythonic” is important - somehow
●
“Pythonic” encompasses more than just programming idioms
●
Using “Pythonic idioms”...
– makes you appear more knowledgeable
– alone does not necessarily make your code better or more pythonic
●
“Pythonic” does not always mean “more readable” for everyone

71. 71
What we learned so far
●
“Pythonic” is important - somehow
●
“Pythonic” encompasses more than just programming idioms
●
Using “Pythonic idioms”...
– makes you appear more knowledgeable
– alone does not necessarily make your code better or more pythonic
●
“Pythonic” does not always mean “more readable” for everyone
●
“Pythonic” is not learned systematically

72. 72
Questions remain...
●
Is there something special in the culture of Python?
– The Zen of Python / “one way to do it”
– Why are there no words like “Rubyist”, “Javanese” or “C#y”

73. 73
Questions remain...
●
Is there something special in the culture of Python?
– The Zen of Python / “one way to do it”
– Why are there no words like “Rubyist”, “Javanese” or “C#y”
●
If “Pythonic” is not just syntax and idioms, what is it?

74. 74
Questions remain...
●
Is there something special in the culture of Python?
– The Zen of Python / “one way to do it”
– Why are there no words like “Rubyist”, “Javanese” or “C#y”
●
If “Pythonic” is not just syntax and idioms, what is it?
●
Does “Pythonic” correlate with code quality?

75. 75
Questions remain...
●
Is there something special in the culture of Python?
– The Zen of Python / “one way to do it”
– Why are there no words like “Rubyist”, “Javanese” or “C#y”
●
If “Pythonic” is not just syntax and idioms, what is it?
●
Does “Pythonic” correlate with code quality?
●
Is “Pythonic” just a posh mark of pride serving to create a two-tier
society within the Python community?

76. Carol V. Alexandru¹, José J. Merchante², Sebastiano Panichella¹,
³
Sebastian Proksch¹, Harald C. Gall¹, Gregorio Robles²
Read the paper: http://t.uzh.ch/S7
Get the slides: http://t.uzh.ch/Sb
Browse the catalogue: http://pythonic.libresoft.info/catalogue
On the Usage of Pythonic Idioms