Vision-based sensing of the welding process : a survey
---
paper_title: Vision Based GTA Weld Pool Sensing and Control Using Neurofuzzy Logic

paper_content:

A steerable scooter including a standing platform supported on a rollable rear support wheel, a steerable front support wheel, and adjustable seat assembly. The scooter includes a support platform positioned below the top surface of the support wheels. In addition, the scooter includes an adjustable seat that has a top surface positionable two or more inches below the top surface of the support wheels.
---
paper_title: A sensing torch for on-line monitoring of the gas tungsten arc welding process of steel pipes

paper_content:

Non-intrusive and real-time monitoring techniques are increasingly required by manufacturing industry in order to detect flaws in arc welding processes. In this work the development of an optical inspection system, for monitoring the manual gas tungsten arc welding (GTAW) process of steel pipes, is described. The arc plasma visible emission produced during the process was acquired and spectroscopically analysed. Measuring the intensities of selected argon emission lines allowed real time calculation and recording of the axial electron temperature of the plasma. Experimental results showed that the temperature signal varies greatly in the case of instabilities of the weld pool that cause weld defects. A suitable algorithm, based on a statistical analysis of the signal, was developed in order to real time flag defective joints. It is shown that several weld defects such as porosity, dropout, lack of fusion, solid inclusions and craters were successfully detected in a production environment. The performances of the optical sensor were compared with the results of state-of-the-art post-weld controls such as x-rays and penetrating dyes, showing good agreement and thus demonstrating the validity of this quality monitoring system.
---
paper_title: Determination of axial thermal plasma temperatures without Abel inversion

paper_content:

This paper presents an approximate formula for the determination of local axis temperatures of an inhomogeneous, axisymmetric, optically thin, local thermodynamic equilibrium plasma column without using the Abel inversion technique. The proposed method is straightforward and is based on recording the spatially integrated radiances of spectral lines. The formula is useful for high gradients thermal plasma columns of the type found in DC electric arcs or plasma jets of gaussian shaped temperature profile. The formulation permits a simple experimental arrangement for rapid monitoring or control of parameters in industrial plasma devices or for the determination of emissivities and line transition probabilities. We also show that the method can be applied to spatially unstable arc discharges, where traditional techniques are difficult and cumbersome to apply.
---
paper_title: Spectroscopic studies of plasma during cw laser materials interaction

paper_content:

The role of plasma during cw laser materials interaction was studied using emission spectroscopy. The experiments were carried out in two stages: first, with a pure argon plasma; and second, with an aluminum target in an argon atmosphere. The electron‐temperature distribution and electron density were determined. The experimental results were used to estimate laser attenuation and refraction as the beam propagated through the plasma column. Thus, transmitted energy and energy distribution on the target surface were determined. This was expected to be a more realistic input for transport phenomena models based on absorbed energy.
---
paper_title: Spectroscopic Study Of Laser-Induced Plasma In The Welding Process Of Steel And Aluminium

paper_content:

Results of spectral diagnostics of the plasma produced by intense laser radiation during the welding process of steel and aluminium with CO2 lasers are presented. The experiments were carried out in an intensity range I = 106 ÷ 107 W/cm and with He, N2, Co2, Ar as shielding gases. The electron plasma temperature T and density n were measured under various processing and shielding gas conditions.It is shown that the time-averaged plasma temperature correlates with the welding results such as e.g. a welding depth obtained by varation of the welding speed and process gas. In this way it is demonstrated that spectral characteristics of the plasma can be used in the monitoring of the plasma and in the optimalization of the welding process. Furthermore, it is shown that spectroscopic results help to explain the influence of the process gas. Examples of cooling, heating and shielding effects caused by process gases are shown and discussed in view of the efficiency of the welding process.
---
paper_title: Experimental study of laser-induced plasma in welding conditions with continuous CO2 laser

paper_content:

Laser‐induced plasmas obtained during a welding process have been studied. Spectroscopic diagnostics and an integrating sphere collecting the reflected CO2 light are the principal diagnostics used in order to determine the spatial variations of the microscopic parameters such as electron density and temperature, and the energy absorption during this process. For several experimental processing conditions of shielding gases, the main perturbing effects such as absorption and refraction of the CO2 laser radiation are quantified. Several possibilities for reducing these perturbing effects are then discussed.
---
paper_title: Spectroscopic characterization of laser-induced plasma created during welding with a pulsed Nd:YAG laser

paper_content:

A spectroscopic study of a laser-induced plume created during the welding of stainless steel and other materials (iron and chromium) has been carried out. A pulsed Nd:YAG laser of 1000 W average power is used. The evolutions of the electron temperature and electron density have been studied for several welding parameters. We use working powers from 300 to 900 W and pulse durations between 1.5 and 5 ms. The influence of shielding gases like nitrogen and argon has been taken into account. Temperature and density calculations are based on the observation of the relative intensities and shapes of the emission peaks. We assume that the plasma is in local thermal equilibrium. The temperature is calculated with the Boltzmann plot method and the density with the Stark broadening of an iron line. The electron temperatures vary in the range of 4500–7100 K, electron density between 3×1022 and 6.5×1022 m−3. The absorption of the laser beam in the plasma is calculated using the Inverse Bremsstrahlung theory.A spectroscopic study of a laser-induced plume created during the welding of stainless steel and other materials (iron and chromium) has been carried out. A pulsed Nd:YAG laser of 1000 W average power is used. The evolutions of the electron temperature and electron density have been studied for several welding parameters. We use working powers from 300 to 900 W and pulse durations between 1.5 and 5 ms. The influence of shielding gases like nitrogen and argon has been taken into account. Temperature and density calculations are based on the observation of the relative intensities and shapes of the emission peaks. We assume that the plasma is in local thermal equilibrium. The temperature is calculated with the Boltzmann plot method and the density with the Stark broadening of an iron line. The electron temperatures vary in the range of 4500–7100 K, electron density between 3×1022 and 6.5×1022 m−3. The absorption of the laser beam in the plasma is calculated using the Inverse Bremsstrahlung theory.
---
paper_title: Weld pool edge detection for automated control of welding

paper_content:

A vision system that determines the edges of the weld pool in sequences of gas-tungsten-arc welding images acquired by a coaxial viewing system is described. The vision system uses a transformation that maps the edge of a weld pool into a vertical line. The weld pool edge is detected in the transform domain by using a directional filter, which retains only intensity changes of interest, and a one-dimensional edge detector. The edge of the weld pool, in the physical domain, is determined using the inverse transformation. The transformation uses parameters that are updated when processing a sequence of images, and are initially determined by analyzing the first image frame in the physical domain. >
---
paper_title: An image analysis system for coaxially viewed weld scenes

paper_content:

We present a complete, working system for analyzing coaxially viewed robotic weld scenes. The analysis is cast in the form of a consistent labeling problem in which the objects are small image regions from a regular tesselation and the possible labels are base metal, electrode, gas cup, filler wire, weld bead, and weld pool. Local domain knowledge and measurements on the image function are used to produce an initial set of labeling probabilities. These are then adjusted by probabilistic relaxation using global domain knowledge to arrive at a final consistent labeling, which constitutes the image analysis. The primary goal of this analysis is a sufficiently accurate description of the size and shape of the weld pool to allow quality monitoring of the welding process. This represents a significant departure from the primary objectives of prior work in this area, most of which has focused on the seam tracking problem. Discussions with welding engineers indicate that they are anxious to acquire whatever information may be available. Thus, this system serves the secondary goal of providing some idea of what might be possible, given both relatively modest, and therefore affordable, resources and a real time performance requirement. In constructing and demonstrating a complete system, we provide useful insight into the engineering of such systems for practical applications, addressing attribute extraction, feature selection, and statistical region classification. A novel, efficient near-optimal feature selection algorithm which we callratchet search is also presented. Finally, we discuss how such a system, which is quite robust, could be embedded into robotic welding systems to provide important weld quality analysis for process control.
---
paper_title: Sensing and Control of Weld Pool Geometry for Automated GTA Welding

paper_content:

Weld pool geometry is a crucial factor in determining welding quality, especially in the case of sheet welding. Its feedback control should be a fundamental requirement for automated welding. However, the real-time precise measurement of pool geometry is a difficult procedure. It has been shown that vision sensing is a promising approach for monitoring the weld pool geometry. Quality images that can be processed in real-time to detect the pool geometry are acquired by using a high shutter speed camera assisted with nitrogen laser as an illumination source. However, during practical welding, impurities or oxides existing on the pool surface complicate image processing. The image features are analyzed and utilized for effectively processing the image. It is shown that the proposed algorithm can always detect the pool boundary with sufficient accuracy in less than 100 ms. Based on this measuring technique, a robust adaptive system has been developed to control the pool area. Experiments show that the proposed control system can overcome the influence caused by various disturbances.
---
paper_title: Vision-based measurement of weld pool geometry in constant-current gas tungsten arc welding

paper_content:

AbstractThe common commercial charge coupled device (CCD) camera is combined with a composite light filter to form a vision sensing system of low cost. It can capture the whole image of a weld pool clearly during constant-current gas tungsten arc welding (GTAW). Based on the image-processing algorithm, the edges of the weld pool under different welding conditions can be extracted. Calibration is made to obtain the weld pool geometry in real size. The measured results are useful for developing a welding process control system and verifying the mathematical models of weld pool behaviours.
---
paper_title: Computation of 3D weld pool surface from the slope field and point tracking of laser beams

paper_content:

A novel computation method of the 3D weld pool surface from specular reflection of laser beams is presented in this paper. The mathematical model for the three-dimensional surface measurement technique has been developed. The structured pattern of a laser is projected on the weld pool in a molten state and the reflection observed using a CCD camera. These reflected patterns can be tracked using image processing techniques including optical flow and moving point tracking. A simulation model has been developed implementing these techniques. The movement of the reflected laser during the welding process is indicative of the change in state of the weld pool. The surface slope field is calculated from the law of reflection and is used to compute the 3D surface of the weld pool. The measurement technique is tested on objects with a priori knowledge of geometry having a specular surface to test the effectiveness of the measurement technique.
---
paper_title: Real-Time Image Processing for Monitoring of Free Weld Pool Surface

paper_content:

The arc weld pool is always deformed by plasma jet. In a previous study, a novel sensing mechanism was proposed to sense the free weld pool surface. The specular reflection of pulsed laser stripes from the mirror-like pool surface was captured by a CCD camera. The distorted laser stripes clearly depicted the 3D shape of the free pool surface. To monitor and control the welding process, the on-line acquisition of the reflection pattern is required. In this work, the captured image is analyzed to identify the torch and electrode. The weld pool edges are then detected. Because of the interference of the torch and electrode, the acquired pool boundary may be incomplete. To acquire the complete pool boundary, models have been fitted using the edge points. Finally, the stripes reflected from the weld pool are detected. Currently, the reflection pattern and pool boundary are being related to the weld penetration and used to control the weld penetration.
---
paper_title: A novel stereo camera system by a biprism

paper_content:

We propose a novel and practical stereo camera system that uses only one camera and a biprism placed in front of the camera. The equivalent of a stereo pair of images is formed as the left and right halves of a single charge coupled device (CCD) image using a biprism. The system is therefore cheap and extremely easy to calibrate since it requires only one CCD camera. An additional advantage of the geometrical setup is that corresponding features lie on the same scanline automatically. The single camera and biprism have led to a simple stereo system for which correspondence is very easy and accurate for nearby objects in a small field of view. Since we use only a single lens, calibration of the system is greatly simplified. Given the parameters in the biprism-stereo camera system, we can reconstruct the three-dimensional structure using only the disparity between the corresponding points.
---
paper_title: Computation of 3D weld pool surface from the slope field and point tracking of laser beams

paper_content:

A novel computation method of the 3D weld pool surface from specular reflection of laser beams is presented in this paper. The mathematical model for the three-dimensional surface measurement technique has been developed. The structured pattern of a laser is projected on the weld pool in a molten state and the reflection observed using a CCD camera. These reflected patterns can be tracked using image processing techniques including optical flow and moving point tracking. A simulation model has been developed implementing these techniques. The movement of the reflected laser during the welding process is indicative of the change in state of the weld pool. The surface slope field is calculated from the law of reflection and is used to compute the 3D surface of the weld pool. The measurement technique is tested on objects with a priori knowledge of geometry having a specular surface to test the effectiveness of the measurement technique.
---
paper_title: A novel stereo camera system by a biprism

paper_content:

We propose a novel and practical stereo camera system that uses only one camera and a biprism placed in front of the camera. The equivalent of a stereo pair of images is formed as the left and right halves of a single charge coupled device (CCD) image using a biprism. The system is therefore cheap and extremely easy to calibrate since it requires only one CCD camera. An additional advantage of the geometrical setup is that corresponding features lie on the same scanline automatically. The single camera and biprism have led to a simple stereo system for which correspondence is very easy and accurate for nearby objects in a small field of view. Since we use only a single lens, calibration of the system is greatly simplified. Given the parameters in the biprism-stereo camera system, we can reconstruct the three-dimensional structure using only the disparity between the corresponding points.
---
paper_title: Mathematical formulation and simulation of specular reflection based measurement system for gas tungsten arc weld pool surface

paper_content:

Weld pool surface can change dynamically during welding and is indicative of information critical to controlling the process. Research has picked up in the field of observing the weld pool surface to understand the dynamics of the welding process. This paper will help visualize and understand the physics involved in observing the weld pool surface. A study of laser properties, weld pool and camera optics was incorporated in developing a model to describe the mechanism of observing the weld pool surface from specular reflection. This observation method projects a laser beam on the pool surface through an optical grid with a frosted glass attached. The corresponding specular reflection is calculated, which is derived based on the reflection law. The reflected laser beams are then captured by the camera to form the image. The model can be used to predict the outcome of experiments with grids placed in front of the laser and to determine the position where the camera should be placed to acquire the best image. Preliminary results showed that the camera should be placed with the weld pool along the optical axis, and the aperture should be as large as possible to allow as many rays into the camera as possible. The model can be used to find the optimal location of the laser and camera for materials of different thickness, by moving the electrode higher in the simulation, and adjusting the laser and camera location accordingly. The paper will give some insight into problems that might be encountered in observing the weld pool, and suggest the set-up of the laser and camera for obtaining the best image.
---
paper_title: The variational approach to shape from shading

paper_content:

Abstract We develop a systematic approach to the discovery of parallel iterative schemes for solving the shape-from-shading problem on a grid. A standard procedure for finding such schemes is outlined, and subsequently used to derive several new ones. The shape-from-shading problem is known to be mathematically equivalent to a nonlinear first-order partial differential equation in surface elevation. To avoid the problems inherent in methods used to solve such equations, we follow previous work in reformulating the problem as one of finding a surface orientation field that minimizes the integral of the brightness error. The calculus of variations is then employed to derive the appropriate Euler equations on which iterative schemes can be based. The problem of minimizing the integral of the brightness error term is ill posed, since it has an infinite number of solutions in terms of surface orientation fields. A previous method used a regularization technique to overcome this difficulty. An extra term was added to the integral to obtain an approximation to a solution that was as smooth as possible. We point out here that surface orientation has to obey an integrability constraint if it is to correspond to an underlying smooth surface. Regularization methods do not guarantee that the surface orientation recovered satisfies this constraint. Consequently, we attempt to develop a method that enforces integrability, but fail to find a convergent iterative scheme based on the resulting Euler equations. We show, however, that such a scheme can be derived if, instead of strictly enforcing the constraint, a penalty term derived from the constraint is adopted. This new scheme, while it can be expressed simply and elegantly using the surface gradient, unfortunately cannot deal with constraints imposed by occluding boundaries. These constraints are crucial if ambiguities in the solution of the shape-from-shading problem are to be avoided. Differrent schemes result if one uses different parameters to describe surface orientation. We derive two new schemes, using unit surface normals, that facilitate the incorporation of the occluding boundary information. These schemes, while more complex, have several advantages over previous ones.
---
paper_title: Technique for simultaneous real-time measurements of weld pool surface geometry and arc force

paper_content:

This paper describes a new technique for simultaneous real-time measurement of weld pool surface geometry and arc force. The weld pool surface profile is measured using real-time radiography by vertical x-ray transmission through the weld. The information is received at a rate of 30 frames per second in the form of two-dimensional images which are digitized and processed in real time by computer. The surface topography is determined from the images using the experimentally found relation between image brightness and material thickness with an accuracy of 0.2 mm (0.008 in.)
---
paper_title: A study of arc force, pool depression and weld penetration during gas tungsten arc welding

paper_content:

Weld pool depression, arc force, weld penetration, and their interrelations have been studied as a function of welding current. Pool depression and welding arc force have been measured simultaneously using a recently developed technique. The authors found quadratic dependence of arc force on current, confirming similar findings in previous studies. Pool depression is essentially zero below a threshold level of current (200 A in this experiment) and then increases quadratically with current. A perfectly linear relation between arc force and pool depression was found in the current range from 200 to 350 A, with pool depression onset at about 0.35 g force (0.34 [center dot] 10[sup [minus]2]N). The total surface tension and gravitational forces were calculated, from the measured surface topography, and found to be about five times that required to balance the arc force at 300 A. Thus electromagnetic and hydrodynamic forces must be taken into account to explain the measured levels of pool depression. The relation between weld penetration and pool depression for different welding currents has been established. Three distinct regimes of weld penetration on weld current were found.
---

Title: Vision-based Sensing of the Welding Process: A Survey

Section 1: Introduction
Description 1: Introduce the importance and applications of vision-based sensors, particularly in the welding process.

Section 2: Optical sensors
Description 2: Discuss the use of optical sensors to study oscillations and spectral analysis of the weld pool, including detailed techniques and setups.

Section 3: 2D analysis of the weld pool
Description 3: Present various techniques for real-time observation and measurement of the 2D shape of the weld pool, including laser strobe systems, edge detection, and co-axial viewing methods.

Section 4: 3D analysis of the weld pool
Description 4: Explore methods for measuring the 3D shape of the weld pool surface, including structured light, stereovision methods, and slope field computations.

Section 5: Conclusion
Description 5: Summarize the survey, highlighting the effectiveness, limitations, and application contexts of different vision-based sensing techniques.