Belief Propagation Algorithm using CUDA

1. Disparity-Map Generation using GPUs Yan Xu Tutor: Hui Chen School of Information Science and Engineering Aug. 1, 2009

2. Tsukuba Right Image Tsukuba Left Image Ground Truth

3. Disparity-Map in Stereo Vision Parallel Programming Programming on GPUs Belief Propagation BP on CUDA Experiment Results Conclusions and Future Works Over View

4. Disparity-Map Generation Disparity-Map Stereo Match Rectification Calibration

5. Local Algorithm Belief Propagation Graph Cut Dynamic Programming Disparity-Map Generation

6. Ground Truth Tsukuba Left Image Tsukuba Right Image Disparity Image by BP (F. Felzenszwalb) Disparity Image by DP Disparity Image by GC (Kolmogorov)

7. Parallel Programming Serial Programming Parallel Programming

8. Parallel Programming Traditionally, software has been written for serial computation: • To be run on a single computer having a single Central Processing Unit (CPU). • A problem is broken into a discrete series of instructions. • Instructions are executed one after another. • Only one instruction may execute at any moment in time. •Serial Programming Parallel Programming

9. Parallel Programming In the simplest sense, parallel computing is the simultaneous use of multiple compute resources to solve a computational problem: • To be run using multiple CPUs. • A problem is broken into discrete parts that can be solved concurrently. • Each part is further broken down to a series of instructions. • Instructions from each part execute simultaneously on different CPUs. Serial Programming • Parallel Programming

10. Serial Parallel

11. Programming on GPUs CPU (Host) GPU (Device)

12. (Device) Grid Block (0, 0) Block (1, 0) Shared Memory Shared Memory Registers Registers Registers Registers Thread (0, 0) Thread (1, 0) Thread (0, 0) Thread (1, 0) Local Memory Local Memory Local Memory Local Memory Host Global Memory Constant Memory Texture Memory Grid 1 Block (0, 0) Block (1, 0) Block (2, 0) Block (0, 1) Block (1, 1) Block (2, 1) Grid 2 Block (1, 1) Thread (0, 1) Thread (1, 1) Thread (2, 1) Thread (3, 1) Thread (4, 1) Thread (0, 2) Thread (1, 2) Thread (2, 2) Thread (3, 2) Thread (4, 2) Thread (0, 0) Thread (1, 0) Thread (2, 0) Thread (3, 0) Thread (4, 0) Programming on GPUs Host Device Kernel 1 Kernel 2

13. Programming on GPUs Main() { //Allocate memory on GPU float *Md; cudaMalloc((void**)&Md, size); //Copy data from CPU to GPU cudaMemcpy(Md, M, size, cudaMemcpyHostToDevice); //Call GPU kernel function kernel<<<dimGrid, dimBlock>>> (arguments); //Copy data from GPU back to CPU CopyFromDeviceMatrix(M, Md); //Free device matrices FreeDeviceMatrix(Md); }

14. Programming on GPUs • CUDA (Compute Unified Device Architecture) is a computing architecture developed by nVIDIA to use graphic processing unit as a general purpose parallel processor. nVIDIAGeFroce 8800

15. Belief Propagation Algorithm mlabels s sites data costs + discontinuity costs

16. Belief Propagation Algorithm

17. Belief Propagation on CUDA 1. Allocate GPU global memory 2. Load original images (left and right) to GPU global memory 3. (If real-world image) Pre-process images with Sobel / Residual 4. Calculate data cost 5. Calculate the data (Gaussian) pyramid 6. Message passing using created pyramid 7. Compute disparity map from messages and data-cost 8. Retrieve disparity map to local (host) memory

18. Experiment Results video

19. Experiment Results Original

20. Experiment Results Sobel

21. Experiment Results video Residual

22. Conclusions and Future Works • Improve Belief Propagation (faster and better) • Implement other stereo algorithms in parallel (such as DP, GC…) • Apply the algorithm to stereo images captured by Truck

23. Thank you for your attention ! Questions ?