We propose a Vision-Based Measurement (VBM) system and evaluate how different algorithms impact the results. The proposed system joins hardware and software to image the welding plates using a single CMOS camera, run computer vision algorithms and control the welding equipment. A complete prototype, using a commercial linear welding robot is presented. Authors: Cristiano R. Steffens, Bruno Q. Leonardo, Sidnei Carlos S. Filho, Valquiria Hüttner, Vagner S. Rosa, Silvia Silva C. Botelho 11°International Conference on Computer Vision Theory and Applications - VISAPP 2016

1. Welding Groove Mapping:
Image Acquisition and Processing on Shiny
Surfaces
Presenter : Jônata T. Carvalho
Authors: Cristiano R. Steffens, Bruno Q. Leonardo, Sidnei Carlos S. Filho, Valquiria Hüttner,
Vagner S. Rosa, Silvia Silva C. Botelho
Federal University of Rio Grande – FURG
Computational Sciences Center – C3
11°International Conference on Computer Vision Theory and Applications - VISAPP 2016

2. Motivation
• Manual process affects the quality of the weld
o Rework
o Material waste
o Weak and breakable final product
o Reproducibility and regularity
• The human side
o Welding is unhealthy – ergonomy, heat and fumes
o Laborious and repetitive task

3. Prior approaches for welding process automation
• We can highlight three main approaches:
o A combination of structured illumination laser and camera,
as used in Kawahara (1983), Drews et al. (1986), Liu (2010),
Zhang et al. (2014) and De Xu (2004);
o A touch sensor based approach as in Kim and Na (2000);
o Techniques where the arc current feedback is explored, as
in Dilthey and Gollnick (1998) and Halmøy (1999);

4. Typical Setup of a Linear Welding System
Figure 1 – Typical linear welding robot installation

5. The BUG-O MDS Welding Robot
• Robust Modular Robot
o Rails and Carriages
o Linear Weaver
o Pendulum Weaver
• Can be used on a large variety of surfaces
• Able to make different welding seams
• Weldor adjusts the linear rail and the parameters in runtime

6. The BUG-O MDS Welding Robot
Figure 2 – Bug-o MDS welding robot
Source : BUG-O Systems

7. Proposed Vision-based Measurement System
Figure 8 – High-level architecture of a vision-based measurement system

8. Proposed Vision-based Measurement System
Figure 3 – Image acquisition setup Figure 4 – Welding groove properties

9. Machine Vision System
• Groove measurement and modeling
• Custom LED lighting system
• Terasic D5M Camera + Altera DE0-Nano FPGA
• Full control of the camera registers
• HW & SW integration
• USB and Bluetooth communication
• Embedded image processing

10. Machine Vision System
Figure 7 – Altera DE0-Nano FPGA.
Source: Altera
Figure 6 – Terasic D5M
CMOS Camera
Source: AlteraFigure 5 – Illumination

11. Debevec’s HDR image composition
Figura 9 – HDR Input images

12. Processing in the VBM System
Figura 10 – Single exposure Figura 11 – HDR composed
Figura 12 – Line segment detection Figura 13– Final groove modeling

13. Machine Vision System (SW)
• Contrast enhancement
o Multi-exposure composition (Debevec)
o Normalization and Histogram Equalization
• Noise reduction
o Gaussian, Mean, Median
• Edge and line detection
o Canny + Hough, Fast LSD, Edlines, PPHT
• Heuristics
• Pixel to metric unit conversion

14. Controll System Implementation (HW)
Figure 14 – Overview of the digital control system

15. Results of the Measurement System (Best-Case)
Gap B - Plate Bottom Gap A - Plate Top
Mean Error Std. Dev. Mean Error Std. Dev.
0.143mm 0.084mm 0.780mm 0.157mm
Figure 15 – Measured/position Figure 16 – Repeatability
Table 1 – Gaussian filtering + LSD by Von Gioi (2012)

16. Method Comparisson – Gap A
Figure 17 – Error and Std. Deviation in millimeters for Gap A (smaller is better)

17. Method Comparisson – Gap B
Figure 18 – Error and Std. Deviation in millimeters for Gap B (smaller is better)

18. Conclusion
• End-to-end embedded welding system prototype
• Integrates different techniques to perform dimensional measurement of thick
steel plate bevel groove
• Computer Vision applied to reflective surfaces, without the need of structured
light, polarized lenses or complex optical arrangements
• Extracted dimensions can be mapped in settings for the robotic welding
equipment

19. Video (https://youtu.be/-fONDmtlnpw)

20. Future Work
• Explore lighting options, noise suppression algorithms and image
composition techniques to improve the system
• Bilateral and L0 gradient minimization filtering (not trivial to implement)
• Compare Debevec’s multi-exposure composition to other approaches that
minimize the computational cost and are hardware-friendly
• Online application - mapping while welding
• Deep learning based image restoration
• Produce a general purpose welding workcell

21. Welding Groove Mapping:
Image Acquisition and Processing on Shiny
Surfaces
Federal University of Rio Grande – FURG
Computational Sciences Center – C3
cristianosteffens@furg.br