Python workshop #1 at UGA

Getting started
How to design programs
Python concepts & party tricks

Python Workshop 2012
The essentials: getting started, program design, modules & I/O

Eric Talevich

Institute of Bioinformatics, University of Georgia

Nov. 8, 2012

Eric Talevich Python Workshop 2012

Getting started

1 Getting started

2 How to design programs
Data structures
Algorithms
Interfaces

3 Python concepts & party tricks
File-like objects
Iteration
The csv module


Getting started

Getting started: Interpreter

Your interactive interpreter is one of:
python on the command line
IDLE — Integrated DeveLopment Environment for Python 1

We’ll use IDLE.

Run this in the interpreter:

import this

1
Included in the standard Python installation on Windows and Mac

Getting started

Getting started: Scripts

1 Write this in a new, blank text ﬁle:
print "Hello, Athens!"


Getting started


2 Save it in plain-text format with the name hi.py in the same
directory as your interpreter session is running. Run it:
python hi.py


Getting started


2 Save it in plain-text format with the name hi.py in the same
directory as your interpreter session is running. Run it:
python hi.py

A CREDIT Add a line at the top of hi.py to tell Unix this is a Python
script:
#!/usr/bin/env python
Make hi.py executable (from the terminal shell):
chmod +x hi.py
Now you don’t need the .py extension:
mv hi.py hi
./hi

Getting started

Getting started: Documentation

In the interpreter: help(function-or-module )
On the command line: pydoc function-or-module
In IDLE: <F1> (launches docs.python.org)
Also, type(obj ), dir(obj ), vars(obj ) and auto-completion
(tab or Ctrl+Space) are pretty handy.

Try it:
>>> help(str)
>>> s = ’Hello’
>>> type(s)
>>> dir(s)
>>> help(s.capitalize)
>>> help(help)


Getting started

Example: string methods

Find the string (str) methods to. . .
Convert gOoFy-cAsE words to title case, upper case or
lower case
Remove certain characters from either end
Align (justify) within a ﬁxed width, padded by spaces


Getting started Data structures
How to design programs Algorithms
Python concepts & party tricks Interfaces


Data structure: Organizes a collection of data units
Algorithm: Deﬁnes operations for a speciﬁc task
Interface: I/O, how a program interacts with the world

Theorem
Program = Data Structures + Algorithms + Interfaces



Data structures
Organize a program’s information



Units of data: Atoms

Class int ﬂoat bool — str
Literal 1, -2 .1, 2.9e8 True, False None ’abc’

Try it:
>>> num = 0.000000002
>>> num
>>> type(num)

Features:
Create from a data literal or by calling the class name
(constructor)
Always immutable — assigning another value replaces the
original object



Collections of data

Class Literal Description
list [] Ordered array of arbitrary objects
dict {} Unordered map of ﬁxed to arbitrary objects
set Unordered collection of ﬁxed objects



Mutable vs. immutable objects

Some mutable objects have immutable counterparts:
list vs. tuple
set vs. frozenset

>>> x = (1, 2, 3)
>>> x[0] = 7

>>> y = list(x)
>>> y[0] = 7
>>> print y



Variables are names, not values
Mutable collections can be shared:
>>> x = [1, 2, 3]
>>> y = x
>>> y[0] = 4
>>> x is y
>>> print ’x =’, x, ’y =’, y

Reassigning to an immutable object replaces (changes the id of)
the original:
>>> x = (1, 2, 3)
>>> y = x
>>> y = (4, 2, 3)
>>> x is y
>>> print ’x =’, x, ’y =’, y


a fairly short
Break



Algorithms
Specify operations



Control ﬂow

Branching The if statement chooses between code paths:
x = -2
if x >= 0:
print x, "is positive"
else:
print x, "was negative"
x = -x



Control ﬂow

Branching The if statement chooses between code paths:
x = -2
if x >= 0:
else:
print x, "was negative"
x = -x

Iteration The for statement visits each element sequentially:
for x in [-2, -1, 0, 1, 2]:
if x >= 0:



# Define a function
# −−−−−−−−−−−−−−−−−
# T h i s e x a m p l e shows how t o d e f i n e a s i m p l e f u n c t i o n .
# The f u n c t i o n d e f i n i t i o n s t a r t s w i t h keyword ’ d e f ’ ,
# t h e n t h e name o f t h e f u n c t i o n and t h e a r g u m e n t s .
# The ’ d o c s t r i n g ’ i n q u o t e s i s o p t i o n a l , b u t n i c e .
# The r e s t o f t h e code , i n d e n t e d , i s t h e f u n c t i o n body .
#
# ( L i n e s t h a t b e g i n w i t h a h a s h (#) a r e comments . )

def n i c e a b s ( x ) :
””” R e t u r n t h e a b s o l u t e v a l u e o f a number . ”””
i f x < 0:
r e t u r n −x
else :
return x



Algorithms

Deﬁnition: A precise set of instructions for accomplishing a task.

We’ll treat an algorithm as one or more functions that operate on
some input data to produce some output according to a
speciﬁcation.

Useful fact: The structure of a function will resemble the
structure of the data it operates on.



The design recipe: a general method
Steps, in order, for developing a program from scratch: 2

Contract: Name the function; specify the types (i.e. atoms and
data structures) of its input data and its output.
abs: number → positive number

2
http://htdp.org/2003-09-26/Book/curriculum-Z-H-1.html



Purpose: Description of what the program does, in terms of
inputs and output — suﬃcient for a function
declaration and docstring.

2



Example: Function call with arguments, and expected result.
abs(-1) → 1
abs(1) → 1

2



Example: Function call with arguments, and expected result.
abs(-1) → 1
abs(1) → 1
Deﬁnition: The code!
Tests: Convert the example to actual code — by running
the program on it.
2


Example: Counting words in text

Given a sample text, count the frequencies of each word.

Name: wordcount
Input: A string of text.
Output: A dictionary, with string keys and integers values.





Name: wordcount
Purpose: Create a dictionary associating each unique word in
the text with the number of times it appears.





Name: wordcount
Purpose: Create a dictionary associating each unique word in
the text with the number of times it appears.
Example: wordcount("yes no yes no maybe")
→ {’yes’: 2, ’no’: 2, ’maybe’: 1}



def w o r d c o u n t ( t e x t ) :
””” Count t h e o c c u r e n c e s o f e a c h word i n t e x t .

I n p u t : s t r i n g o f w h i t e s p a c e −d e l i m i t e d words
Output : d i c t o f s t r i n g s and i n t e g e r s ( >0)
Example :
>>> w o r d c o u n t ( ” y e s no y e s no maybe ” )
{ ’ maybe ’ : 1 , ’ y e s ’ : 2 , ’ no ’ : 2}
”””
w o r d c o u n t s = {}
f o r word i n t e x t . s p l i t ( ) :
i f word not i n w o r d c o u n t s :
# New word ; s t a r t c o u n t i n g from 0
w o r d c o u n t s [ word ] = 0
w o r d c o u n t s [ word ] += 1
return word counts



Snack Time



Interfaces
Interact with the world



Modules

Think of a module as a bundle of features. When you import a
module, you load an additional set of features that your program
can use.

>>> import this
>>> type(this)
>>> help(this)
>>> dir(this)

A module may also be called a “package” or “library”, or possibly
“API”. (It’s nuanced.)
Python has many built-in modules. Together, these are called
the “standard library”.



Built-in I/O functions

You may already know:

print Write a line to the command-line interface.
>>> print ’Eureka!’
Eureka!



Built-in I/O functions

You may already know:

print Write a line to the command-line interface.
>>> print ’Eureka!’
Eureka!

raw input Read a line from the command-line interface.
>>> name = raw input(’Name: ’)
Name: Jonas
>>> print ’My name is’, name
My name is Jonas



The sys module

print and raw input are wrappers for standard output and
standard input:

import sys

sys.stdout File handle for writing to the program’s output.
>>> sys.stdout.write(’Eureka!n’)
Eureka!



The sys module

print and raw input are wrappers for standard output and
standard input:

import sys

sys.stdout File handle for writing to the program’s output.
>>> sys.stdout.write(’Eureka!n’)
Eureka!
sys.stdin File handle for reading from the program’s input.
>>> sys.stdout.write(’Name: ’);
... name = sys.stdin.readline()
Name: Jonas
>>> sys.stdout.write(’My name is ’ + name)
My name is Jonas



Program arguments

Command-line arguments are stored in sys.argv:
cat hi.py
print ’Hello, Athens!’



Program arguments

cat hi.py
echo ’Some text’ > example.txt



Program arguments

cat hi.py
echo ’Some text’ > example.txt
python hi.py example.txt
>>> import sys
>>> print sys.argv
[’hi.py’, ’example.txt’]



More types of interfaces

Pipelines (sys.stdin/stdout/stderr)
API — calling functions from other programs & libraries
IPC — inter-process communication (shm, subprocess)
GUI widgets
Web forms & Javascript events

(Don’t worry about the details here. Just keep this stuﬀ separate
from the main algorithm if you encounter it.)


Getting started File-like objects
How to design programs Iteration
Python concepts & party tricks The csv module

What is a ﬁle?

Consider:
A local ﬁle larger than available RAM
Reading data over an internet connection

Problem: Data can’t be viewed all at once
Solution: Retrieve bytes on demand



File handles

A handle points to a location in a byte stream (e.g. the ﬁrst byte
of the ﬁle to be read/written).

>>> myfile = open(’example.txt’)
>>> print myfile.read()
>>> myfile.seek(0)
>>> print myfile.readlines()
>>> myfile.close()



File-like objects

Text (ASCII or Unicode) ﬁle on disk
Binary ﬁle on disk
Standard input, output, error
Devices, pretty much everything else on Unix
Network connection
Memory map (inter-process communication)
Any Python object supporting the right methods (e.g.
StringIO)



Features of Python ﬁle objects

Modes:
infile = open(myfile, ’r’)

r, w — read, write
b — binary, to keep Windows from clobbering CR/LF bytes
U — Unicode

Iteration:
for line in infile:
print line.split()



another short
Break



Iteration

Iterable objects “know” how they can be looped over.
Python has functions that operate on each item in an iterable:

“for” loop: for x in iterable : ...
zip: for left , right in zip ( iter1 , iter2 ): ...
enumerate: for index , x in enumerate( iterable ): ...
List comprehension: [x ∗ x for x in range(20) if x % 2 == 0]



Example: Counting words in a file

Count the words in a file, and print a table sorted by
word frequency.
Let’s expand the wordcount function into a complete script.
Separate algorithms from interfaces
Don’t do any I/O inside wordcount.
Independent output formatting
Use a separate function to print the dictionary wordcount
produces.
Flexible file input
Take a file name as a program argument, or read from
standard input.



Enhancing wordcount

1 Read from an open ﬁle handle, not a string
Don’t worry about opening or closing it here:
def count words(infile): ...



Enhancing wordcount

2 Remove punctuation, ignore case when counting words
str.strip and str.lower will work — but could make ’’



Enhancing wordcount

2 Remove punctuation, ignore case when counting words
str.strip and str.lower will work — but could make ’’
3 Save a line of code with collections.defaultdict
Write what we want more directly:
>>> counts = collections.defaultdict(int)
>>> counts[’new’]
0



import c o l l e c t i o n s

def c o u n t w o r d s ( i n f i l e ) :
””” Count o c c u r r e n c e s o f e a c h word i n i n f i l e .
I n p u t : F i l e h a n d l e open f o r r e a d i n g
Output : d i c t o f { s t r i n g : i n t e g e r }
Example : ( i n f i l e c o n t a i n s ” y e s no y e s no maybe ” )
>>> c o u n t w o r d s ( i n f i l e )
{ ’ maybe ’ : 1 , ’ y e s ’ : 2 , ’ no ’ : 2}
”””
w counts = c o l l e c t i o n s . d e f a u l t d i c t ( i n t )
for l i n e in i n f i l e :
f o r word i n l i n e . s p l i t ( ) :
# I g n o r e c a s e and a d j a c e n t p u n c t u a t i o n
word = word . s t r i p ( ’ ,.;:?! -()" ’ ’ ) . l o w e r ( )
w c o u n t s [ word ] += 1
return w counts



Output formatting

Given the dictionary produced by count words . . .
1 Sort entries by counts, in descending order
Use the key argument to select values for comparison:
>>> pairs = [(’one’,1), (’three’,3), (’two’,2)]
>>> select second = lambda x: x[1]
>>> pairs.sort(key=select second); print pairs
[(’one’, 1), (’two’, 2), (’three’, 3)]



Output formatting

Given the dictionary produced by count words . . .
1 Sort entries by counts, in descending order
Use the key argument to select values for comparison:
>>> pairs = [(’one’,1), (’three’,3), (’two’,2)]
>>> select second = lambda x: x[1]
>>> pairs.sort(key=select second); print pairs
[(’one’, 1), (’two’, 2), (’three’, 3)]
2 Justify the printed words in a ﬁxed-with column
Use the length of the longest word as the column width.
>>> words = ’thanks for all the fish’.split()’
>>> max(len(word) for word in words)
6



def p r i n t d i c t i o n a r y ( d c t ) :
””” P r i n t t h e k e y s and v a l u e s i n d c t .

S o r t s i t e m s by d e c r e a s i n g v a l u e ; l i n e s up c o l u m n s .

I n p u t : d i c t w i t h s t r i n g k e y s and any−t y p e v a l u e s
”””
# Width n e e d e d f o r d i s p l a y i n g t h e k e y column
k e y w i d t h = max ( l e n ( k e y ) f o r k e y i n d c t )
k v p a i r s = dct . items ()
# S o r t by t h e s e c o n d i t e m i n t h e p a i r , d e c r e a s i n g
k v p a i r s . s o r t ( k e y=lambda kv : kv [ 1 ] , r e v e r s e=True )
f o r key , v a l u e i n k v p a i r s :
# A l i g n both columns a g a i n s t a s t r i p o f t a b s
p r i n t k e y . r j u s t ( k e y w i d t h ) , ’t’ , v a l u e



File I/O and scaﬀolding

Let’s put it all together.
1 Document usage at the top of the script
Several ways to view this: pydoc, help, doc




2 Check the given program arguments
If none: read from standard input
If one: open the ﬁle and read it
Otherwise: print a helpful error message




2 Check the given program arguments
If none: read from standard input
If one: open the ﬁle and read it
Otherwise: print a helpful error message
3 Exit with proper Unix return codes
Use sys.exit: 0 means OK, otherwise error
$ python -c ’import sys; sys.exit(1)’ && echo OK
$ python -c ’import sys; sys.exit()’ && echo OK
OK



#! / u s r / b i n / env p y t h o n

””” Count t h e words i n a t e x t f i l e o r str eam , and
p r i n t a t a b l e o f word c o u n t s s o r t e d by f r e q u e n c y .

Usage :
w o r d c o u n t . py [ f i l e n a m e ]
”””

import s y s
import c o l l e c t i o n s

def c o u n t w o r d s ( i n f i l e ) :
...

def p r i n t d i c t i o n a r y ( d c t ) :
...



if name == ’__main__ ’ :
i f l e n ( s y s . a r g v ) == 1 :
# Text s t r e a m i n p u t , e . g . from a p i p e
i n f i l e = sys . stdin
e l i f l e n ( s y s . a r g v ) == 2 :
# Read t e x t from t h e g i v e n f i l e
i n f i l e = open ( s y s . a r g v [ 1 ] , ’r’ )
else :
# Too many a r g u m e n t s ! P r i n t u s a g e & q u i t
sys . exit ( doc )

# Now , do e v e r y t h i n g
word counts = count words ( i n f i l e )
p r i n t d i c t i o n a r y ( word counts )



CSV: Comma Separated Values

Python can read and write spreadsheet files in CSV format. 3

(It’s handy if you don’t have a database set up.)

KEY one three five
two 2 6 10
four 4 12 20
six 6 18 30

Let’s convert this to a dictionary-of-dictionaries in Python — the
outer dictionary is keyed by the row labels (from the KEY column),
and the inner dictionary is the row data, keyed by column label.

3
Conversions seem to be easier in OpenOffice/LibreOffice Calc than in Excel.


# Save t h e p r e v i o u s t a b l e a s ’ m t a b l e . c s v ’
# Here , we c o n v e r t i t t o a d i c t −of −d i c t s
import c s v
i n f i l e = open ( ’mtable .csv ’ )
csvreader = csv . DictReader ( i n f i l e )
t a b l e = {}
fo r rowdata in c s v r e a d e r :
k e y = r o w d a t a . pop ( ’KEY ’ )
t a b l e [ key ] = rowdata

# P r e t t y −p r i n t t a b l e w i t h n i c e l y a l i g n e d rows
from p p r i n t import p p r i n t
pprint ( table )
# { ’ four ’: { ’ f i v e ’: ’20 ’ , ’ one ’ : ’ 4 ’ , ’ t h r e e ’ : ’ 1 2 ’ } ,
# ’ s i x ’: { ’ f i v e ’: ’30 ’ , ’ one ’ : ’ 6 ’ , ’ t h r e e ’ : ’ 1 8 ’ } ,
# ’ two ’ : { ’ f i v e ’ : ’ 1 0 ’ , ’ one ’ : ’ 2 ’ , ’ t h r e e ’ : ’ 6 ’ } }

p r i n t t a b l e [ ’four ’ ] [ ’five ’ ] # ’20 ’



Thanks
’Preciate it.
Gracias



Further reading

The oﬃcial Python tutorial is quite good:
http://docs.python.org/tutorial/index.html

This is the book I recommend for learning Python:
http://learnpythonthehardway.org/

This presentation on Slideshare, and the example script on GitHub:
http://www.slideshare.net/etalevich/
python-workshop-1-uga-bioinformatics
https://gist.github.com/661869


Python workshop #1 at UGA

Recomendados

Recomendados

Mais conteúdo relacionado

Mais procurados

Mais procurados (20)

Destaque

Destaque (6)

Semelhante a Python workshop #1 at UGA

Semelhante a Python workshop #1 at UGA (20)

Último

Último (20)

Python workshop #1 at UGA