5. Threads: Fluffy the three-headed dog
• Each thread has same parent process
• Threads share memory
• Good for I/O Bound tasks (Scraping websites)
• Poor for CPU Bound Tasks (Geoprocessing)
• At the mercy of Global Interpreter Lock (GIL)
• Problems can affect other threads
Flickr - jegcarpio's photostream
7. Processes: Hounds of Resurrection
• Each process is a separate python instance
• Memory space is NOT shared
• Easily distributed across all cores
• If one process dies, others are not affected
• Communication via IPC
Flickr - Blacksmith's photostream
11. Multiprocessing Pitfalls
• All worker arguments must be picklable
• Schema locks (File Geodatabase…)
• Script tools must be run “out-of-process”
• Worker startup cost
12. In Closing:
• Jason Pardy’s Multiprocessing post
– blogs.esri.com/esri/arcgis/2011/08/29/multiproce
ssing (link)
• Follow Me:
– @brendancol
– bcdcspatial.blogspot.com (link)
• Follow Blue Raster:
– @blueraster
– blueraster.com/blog (link)
Flickr - jystewart's photostream