It introduces how to make Python as fast as C by Cython at Taipei.py [1], COSCUP 2014 [2], Tainan.py [3], Taipei.py 2015 [4], Kaohsiung Python Meetup [5]. [1]: http://www.meetup.com/Taipei-py/events/169077442/ [2]: http://coscup.org/2014 [3]: http://www.meetup.com/Tainan-py-Python-Tainan-User-Group/events/219957183/ [4]: http://www.meetup.com/Taipei-py/events/222038313/ [5]: http://www.meetup.com/Kaohsiung-Python-Meetup/events/229978032/

1. Cython —
Making Python as Fast as C
Mosky

2. Mosky
➤ Python Charmer at Pinkoi
➤ has spoken at 10+ conferences
➤ TEDxNTUST 2015
➤ PyCons in TW/JP/SG/HK
➤ etc.
➤ has taught Python for 100+ hours
➤ has serval Python packages
➤ MoSQL, Clime, etc.
➤ http://mosky.tw/
2

3. Outline
1. Introduction
2. Setup
3. Foundation
4. Practicing
5. Tips
6. Uncovered Topics
7. Is Cython the best solution?
3

4. Introduction

5. Cython
➤ Cython is a source-to-source compiler (aka. transcompiler).
➤ Cython is a superset of Python.
➤ Provides optional static type declarations.
➤ Makes writing C extensions for Python easier.
➤ Makes Python program faster by pre-compiling and
static type.
➤ Sometimes faster by orders of magnitude
5

6. .pyx .c
Cython
C compiler
import.so .py
Python

7. Setup

8. Install C Compiler
➤ Mac:
➤ xcode-select --install
➤ Ubuntu / Debian:
➤ sudo apt-get install build-essential
➤ Other:
➤ http://docs.cython.org/src/quickstart/install.html
8

9. Install Cython
➤ Recommend to use PIP:
➤ sudo pip install cython
➤ Other:
➤ http://docs.cython.org/src/quickstart/install.html
9

10. The setup.py
from distutils.core import setup
from Cython.Build import cythonize
setup(
name = 'cython-lab',
ext_modules = cythonize('*.pyx'),
)
10

11. The hello_cython.pyx
print 'Hello Cython!'
11

12. Build
➤ Into package folder for development:
➤ python setup.py build_ext --inplace
➤ Into system for production:
➤ python setup.py install
➤ If clang: error: unknown argument: '-mno-fused-madd',
➤ export CFLAGS=-Qunused-arguments
12

13. Foundation

14. Define Static Types
cdef int i, j, k
cdef float f, g[42], *h
14

15. cdef struct Grail:
int age
float volume
cdef union Food:
char* spam
float* eggs
cdef enum CheeseType:
cheddar, edam,
camembert
cdef enum CheeseState:
hard = 1
soft = 2
runny = 3
15

16. ctypedef unsigned long ULong
ctypedef int* IntPtr
16

17. cdef struct Point:
int x
int y
# either `struct` or `ctypedef` is not need
cdef Point p
17

18. cdef:
struct Point:
int x
int y
Point p
18

19. Define Function
def say_hello(name='World'):
return 'Hello, %s!' % name
cdef say_hello(name='World'):
return 'Hello, %s!' % name
19

20. cdef say_hello(object name='World'):
return 'Hello, %s!' % name
cdef say_hello(char* name='World'):
return 'Hello, %s!' % name
20

21. cdef int add(int a, int b):
return a+b
cpdef say_hello(char* name='World'):
return 'Hello, %s!' % name
21

22. .pxd Exposes cdef Func
# mylib.pxd
cdef say_hello(char* name=?)
# another.pyx
from mylib cimport say_hello
22

23. Using C Lib
from libc.math cimport sin
# or
cdef extern from "math.h":
double sin(double x)
23

24. Function Visibility
24
Same
File
Other
.pyx
.py
Func in .h/.c
Visible
directly
Visible
via cdef extern
Invisible
cdef
Visible
via .pxd &
cimport
cpdef
Visible
via import
def

25. Binding During
25
Same
File
Other
.pyx
.py
Func in .h/.c
compile-time
compile-time x
cdef
cpdef
run-time
def

26. Type Conversions
26
C From PY To PY
[unsigned]
char/short int
int/long
int
long
unsigned
int/long
long
[unsigned]
long long

27. C From PY To PY
float/double
int/long/float float
long double
char* str/bytes
struct dict

28. Practicing

29. Suggestions
➤ LIB_NAME.pyx
➤ has an execute_self_tests function
➤ test_LIB_NAME.py
➤ call the execute_self_tests function
29

30. Overflow
➤ Static types may also overflow in Cython silently.
➤ Try to make an overflow!
➤ Hint:
➤ http://j.mp/test_overflow_in_c_c
➤ Ans:
➤ http://j.mp/overflow_in_pyx_pyx
30

31. Functions
➤ Write three functions
defined in
def, cdef, and cpdef.
➤ Try to call them in
➤ the same file,
➤ another .pyx file,
➤ and a .py file.
➤ Hints:
➤ Refer to the table,
“Function Visibility”.
➤ http://j.mp/
lib_in_pyx_pyx
➤ Ans:
➤ http://j.mp/
use_lib_in_pyx_pyx
➤ http://j.mp/
test_lib_in_pyx_py
31

32. Using C Function
➤ Try to use the functions in C.
➤ Playing with fork, the system call, may be fun.
➤ Hint:
➤ http://j.mp/test_fork_c
➤ Ans:
➤ http://j.mp/fork_in_pyx_pyx
32

33. Tips

34. cython -a
➤ cython -a NAME.pyx
➤ open NAME.html
➤ Lines are colored according to
the level of “typedness” –
white lines translates to pure
C without any Python API
calls.
34

35. PYXIMPORT
import pyximport; pyximport.install()
import my_pyx_lib # compile .pyx into .so
# or
pyximport.install(pyimport=True)
import my_py_lib # compile .py into .so
35

36. Uncovered Topics

37. Uncovered Topics
➤ Differences between C and Cython expressions
➤ http://docs.cython.org/src/userguide/
language_basics.html#differences-between-c-and-cython-
expressions
➤ Propagating Exceptions in cdef
➤ http://docs.cython.org/src/userguide/
language_basics.html#error-return-values
37

38. ➤ Extension Type — cdef class
➤ http://docs.cython.org/src/userguide/
extension_types.html
➤ Generic programming using Cython's Template
➤ http://docs.cython.org/src/userguide/fusedtypes.html
➤ Conditional Compilation
➤ http://docs.cython.org/src/userguide/
language_basics.html#conditional-compilation
38

39. ➤ Profiling
➤ http://docs.cython.org/src/tutorial/profiling_tutorial.html
➤ Parallelism (No GIL + OpenMP)
➤ http://docs.cython.org/src/userguide/parallelism.html
➤ Using C++ in Cython
➤ http://docs.cython.org/src/userguide/
wrapping_CPlusPlus.html
39

40. Is Cython
the best solution?

41. PYTHON
COMMUNITY
IS
DORAEMON!

42. Other Solutions
➤ Boost.Python — exposes C++ to Python
➤ Numba — compiles annotated code into LLVM by JIT
compiler
➤ PyPy — speeds up existent code by JIT compiler
➤ NumPy or Blaze — provides efficient array
➤ SciPy — provides fast scientific computing
42

43. Cool Down
➤ Algorithm still does matter in any case.
➤ Profile your program.
➤ Consider the portability — you are writing C program!
➤ Consider the improvement is enough or not.
➤ Then pick the most suitable tools.
43

44. Ending

45. Ending
➤ cdef
➤ static types
➤ functions
➤ extern for C functions
➤ .pxd exposes cdef functions
➤ Tips
➤ mosky.tw
➤ Any question?
45