Christian Demant • Bernd Streicher-Abel • Peter Waszkewitz

Industrial Image rrocessing

Springer-Verlag Berlin Heidelberg GmbH

Christian Demant • Bernd Streicher-Abel Peter Waszkewitz

Industrial Image Processing Visual Quality Control in Manufacturing

With 230 Figures

Springer

C H R I S T I A N D E M A N T

Silcherweg 26

71686 Remseck

Germany

B E R N D S T R E I C H E R - A B E L

Am Burgholz 23

71686 Remseck

Germany

P E T E R W A S Z K E W I T Z

Geigeräckerstr. 26

71336 Waiblingen

Germany

Translation:

M I C H A E L A STRICK

Geigeräckerstr. 26

71336 Waiblingen

Germany

GARY S C H M I D T

8105 Piers Drive

60517 Woodridge, IL

USA

ISBN 978-3-642-63642-4 ISBN 978-3-642-58550-0 (eBook) DOI 10.1007/978-3-642-58550-0

Library of Congress Cataloging-in-Publication Data

Industrial Image Processing [Medienkombination]: Visual Quality Control in Manufacturing / Christian Demant; Bernd Streicher-Abel; Peter Waszkewitz. Translated from the Germany by Michaela Strick; Gary Schmidt. - Berlin; Heidelberg; New York; Barcelona; Hong Kong; London; Milano; Paris; Singapore; Tokyo: Springer Buch. 1999

This work is subject to copyright. All rights are reserved, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilm or in other ways, and storage in data banks. Duplication of this publication or parts thereof is permitted only under the provisions of the German Copyright Law of September 9,1965, in its current version, and permission for use must always be obtained from Springer-Verlag. Violations are liable for prosecution act under German Copyright Law.

© Springer-Verlag Berlin Heidelberg 1999

The use of general descriptive names, registered names, trademarks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use.

Typesetting: Camera-ready by authors Cover-Design: MEDIO GmbH, Berlin Printed on acid-free paper SPIN: 10742337 62/3020-5 4 3 2 1 0

Preface

Digital image processing has become a key technology in the area of manu­facturing and quality control. Increasing quality demands require inspection of every single part, which in turn will lead to a much more widespread use of automatic visual inspection systems in the near future. Furthermore, the documentation requirements of ISO 9000 and similar quality control standards can only be met by fully automated, networked inspection systems.

On the other hand, despite a multitude of successful applications, digital image processing has not yet established itself as an accepted element of man­ufacturing technology. This holds true for the industrial practice as well as for the training of engineers. Digital image processing is still widely regarded as some kind of secret lore, mastered only by a small number of expensive -- experts. This impression of incomprehensibility frequently leads to the accusation of unreliability. The manufacturers of digital image processing systems in the industry are not least responsible for this state of affairs, due to their policy of giving the customer as little information as possible about the methods and technology used to inspect his products.

This book has been written with one goal in mind: to lift this veil as much as possible in the course of a few hundred pages. It is based on years of practical experience on the part of the editors in developing and putting into operation visual inspection systems in the manufacturing industry. We have tried to use a different approach than most books about digital image processing. Instead of introducing isolated methods in a mathematically systematic sequence we present applications taken with few exceptions from the industrial practice. These image processing problems then motivate the presentation of the applied algorithms, which focuses less on theoretical con­siderations than on the practical applicability of algorithms and how to make them work together in a consistently designed system. The mathematical foundations will not be neglected, of course, but they will also not be the main point of attention.

We hope that this approach will give students and practitioners alike an impression of the capabilities of digital image processing for the purposes of industrial quality control. We also hope that it will create an understanding for the prerequisites and methodology of its application.

6

No book of this kind would ever be written without support from a multitude of sources. First of all we want to name the persons and companies who participated directly in the creation of this book by supplying test samples and information material:

- Dr.-Ing. Sabine Plischki and Dipl.-Ing. (FH) Peter Hirschel, Behr GmbH & Co., Stuttgart, Germany;

- Dipl.-Ing. (BA) Martin Gartner and Dipl.-Ing. (BA) Gatz Eberle, Robert Bosch GmbH, Stuttgart, Germany;

- Dipl.-Ing. Claus Larcher and Dipl.-Phys. Hardy Burkle, Daimler-Benz AG, Stuttgart, Germany;

- Braun AG, Melsungen, Germany; - Chugai Boyeki (Deutschland GmbH), Dusseldorf, Germany; - Dolan-Jenner Europe B.V., Uden, The Netherlands; - PENTAX GmbH, Hamburg, Germany; - PULNiX Europe Ltd., Alzenau, Germany; - SONY Deutschland GmbH, Cologne, Germany;

Furthermore, we want to thank all the people who supported us in the years past and have been in one way or another involved in the evolution of NEUROCHECK@. Without their work and commitment the present book would not have been possible:

- Dipl.-Inf. Marcellus Buchheit, WIBU-SYSTEMS AG, Germany; - Patent attorney Dipl.-Ing. Christoph Sturm, Wiesbaden, Germany; - Dipl.-Ing. (FH) Walter Happold, Robert Bosch GmbH, development center

Schwieberdingen, Germany; - Mr. Dieter Ohngemach, WM-TEC GmbH, Waldachtal, Germany; - Dipl.-Ing. (BA) Winfried Klass, Data Translation GmbH, Bietigheim-

Bissingen, Germany; - Dipl.-Ing. Dirk Zinnacker; - Dr. Eberhard Rahm, Fulbright & Jaworski L.L.P., New York, USA; - Mr. Nigel Doe, B. Sc. (Hons) and Mr. Earl Yardley, B. Sc. (Hons), Data

Translation Ltd., Basingstoke, UK; - Mr. Lau Ludvigsen, PC Instruments, Lynge, Denmark; - Mr. Rob Pelle, Data@Vision B. V., Vlaardingen, The Netherlands; - Mr. Grant MacMeans, RDP Corporation, Dayton, Ohio, USA; - Ms. Wendy Hunter.

7

Finally we would like to express our special thanks to Springer Verlag for this opportunity to present our "vision" to an international audience, in particular Dr. Merkle and Ms. Grunewald-Heller for their commitment and patience, and, last but in no way least, to our translators, Ms. Michaela Strick and Mr. Gary Schmidt.

Stuttgart, Summer 1999 Christian Demant Bernd Streicher-Abel

Peter Waszkewitz

Table of Contents

1. Introduction.............................................. 15 1.1 Why write another book about image processing? .......... 15 1.2 Possibilities and limitations. . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 17 1.3 Types of inspection tasks. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 19 1.4 Structure of image processing systems. . . . . . . . . . . . . . . . . . . .. 20

1.4.1 Hardware setup. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 20 1.4.2 Signal flow in the process environment . . . . . . . . . . . . .. 23 1.4.3 Signal flow within an image processing system ....... 26

1.5 Solution approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 27 1.6 Introductory example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 29

1.6.1 Character recognition. . . . . . . . . . . . . . . . . . . . . . . . . . . .. 30 1.6.2 Thread depth. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 33 1.6.3 Presence verification. . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 35

1. 7 From here . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 36

2. Overview: Image Preprocessing. . . . .. . . . . . . . . . . . . . . . . . . . .. 39 2.1 Gray scale transformations .............................. 40

2.1.1 Look-up tables. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 40 2.1.2 Linear gray level scaling. . . . . . . . . . . . . . . . . . . . . . . . . .. 42 2.1.3 Contrast enhancement. . . . . . . . . . . . . . . . . . . . . . . . . . .. 43 2.1.4 Histogram equalization. . . . . . . . . . . . . . . . . . . . . . . . . . .. 44 2.1.5 Local contrast enhancement ....................... 45

2.2 Image arithmetic. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 47 2.2.1 Image addition and averaging. . . . . . . . . . . . . . . . . . . . .. 47 2.2.2 Image subtraction. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 48 2.2.3 Minimum and maximum of two images. . . . . . . . . . . . .. 50 2.2.4 Shading correction. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 51

2.3 Linear filters. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 52 2.3.1 Local operations and neighborhoods. . . . . . . . . . . . . . .. 52 2.3.2 Principle of linear filters. . . . . . . . . . . . . . . . . . . . . . . . . .. 53 2.3.3 Smoothing filter. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 56 2.3.4 Edge filters. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 61

2.4 Median filter. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 66 2.5 Morphological filters. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 67

10 Table of Contents

2.6 Other non-linear filters. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 70 2.7 Global operations ...................................... 71 2.8 Key terms. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 72

3. Positioning............................................... 75 3.1 Position of an individual object .......................... 75

3.1.1 Positioning using the entire object. . . . . . . . . . . . . . . . .. 76 3.1.2 Positioning using an edge. . . . . . . . . . . . . . . . . . . . . . . . .. 78

3.2 Orientation of an individual object ....................... 81 3.2.1 Orientation computation using principal axis. . . . . . . .. 81 3.2.2 Distance-versus-angle signature .................... 84

3.3 Robot positioning ............. . . . . . . . . . . . . . . . . . . . . . . . .. 86 3.3.1 Setting of tasks .................................. 86 3.3.2 Image processing components. . . . . . . . . . . . . . . . . . . . .. 87 3.3.3 Position determination on one object ............... 88 3.3.4 Orientation of an object configuration. . . . . . . . . . . . . .. 89 3.3.5 Comments concerning position adjustment. . . . . . . . . .. 90

3.4 Key terms. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 92

4. Overview: Segmentation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 95 4.1 Regions of interest. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 95

4.1.1 Regions and objects .............................. 95 4.2 Thresholding........................................... 96

4.2.1 Thresholds...................................... 97 4.2.2 Threshold determination from histogram analysis. . . .. 98 4.2.3 Gray level histograms. . . . . . . . . . . . . . . . . . . . . . . . . . . .. 99 4.2.4 Generalizations of thresholding ..................... 102

4.3 Contour tracing ........................................ 104 4.3.1 Pixel connectedness ............................... 104 4.3.2 Generating object contours ........................ 106 4.3.3 Contour representation ............................ 107

4.4 Edge based methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 108 4.4.1 Edge probing in industrial image scenes ............. 108 4.4.2 Edge detection with subpixel accuracy .............. 109

4.5 Template matching ..................................... 111 4.5.1 Basic operation .................................. 112 4.5.2 Optimizing template matching ..................... 115 4.5.3 Comments on template matching ................... 119

4.6 Key terms ............................................. 120

5. Mark Identification ....................................... 125 5.1 Bar code identification .................................. 125

5.1.1 Principle of gray-level-based bar code identification ... 126 5.1.2 Bar code symbologies ............................. 127 5.1.3 Examples of industrial bar code identification ........ 129

5.2

5.3

Table of Contents 11

5.1.4 Further information Character recognition .................................. . 5.2.1 Laser-etched characters on an IC .................. . 5.2.2 Basic configuration of the character recognition ..... . 5.2.3 Fundamental structure of a classifier application ..... . 5.2.4 Position adjustment on the IC ................ _ ... . 5.2.5 Improving character quality ......... _ ............ . 5.2.6 Optimization in operation ........................ . Recognition of pin-marked digits on metal ................ . 5.3.1 Illumination .................................... .

132 132 132 133 136 141 1-1;:1 1i8 149 149

5.3.2 Preprocessing .................................... ISO 5.3.3 Segmentation and classification ...... _ ............. 150

5.4 Block codes on rolls of film .............................. 152 5.5 Print quality inspection ................................. 156

5.5.1 Procedure ....................................... 158 5.5.2 Print quality inspection in individual regions ......... 159 5.5.3 Print quality inspection with automatic subdivision ... 160

5.6 Key terms ............................................. 161

6. Overview: Classification .................................. 165 6.1 What is classification? .................................. 165 6.2 Classification as function approximation ................... 167

6.2.1 Machine learning ................................. 167 6.2.2 Statistical foundations ............................ 169 6.2.3 Constructing classifiers ............................ 170

6.3 Instance-based classifiers ................................ 172 6.3.1 Nearest neighbor classifier ......... _ ............... 172 6.3.2 RCE networks ................................... 174 6.3.3 Radial basis functions ............ _ ................ 175 6.3.4 Vector quantization ..................... _ ......... 176 6.3.5 Template matching ............................... 177 6.3.6 Remarks on instance-based classifiers ............... 177

6.4 Function-based classifiers ................................ 178 6.4.1 Polynomial classifier . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 178 6.4.2 Multilayer perceptron-type neural networks .......... 179 6.4.3 Representation of other classifiers as neural networks . 182

6.5 Remarks on the application of neural networks ............. 183 6.5.1 Composition of the training set .................... 183 6.5.2 Feature scaling ................................... 183 6.5.3 Rejection ............................... __ ....... 184 6.5.4 Arbitrariness ...................... ___ ........... 185

6.6 Key terms ............................................. 186

12 Table of Contents

7. Dimensional Checking . ................................... 191 7.1 Gauging tasks .......................................... 191 7.2 Simple gauging ......................................... 192

7.2.1 Center point distance ............................. 193 7.2.2 Contour distances ................................ 196 7.2.3 Angle measurements .............................. 200

7.3 Shape checking on a punched part ........................ 201 7.3.1 Inspection task ................................... 201 7.3.2 Modeling contours by lines ........................ 202 7.3.3 Measuring the contour angle ....................... 205

7.4 Angle gauging on toothed belt ........................... 205 7.4.1 Illumination setup ................................ 206 7.4.2 Edge creation .................................... 208

7.5 Shape checking on injection-molded part .................. 209 7.5.1 Computing radii ................................. 209 7.5.2 Remarks on model circle computation ............... 211

7.6 High accuracy gauging on thread flange ................... 213 7.6.1 Illumination and image capture .................... 213 7.6.2 Subpixel-accurate gauging of the thread depth ....... 214

7.7 Calibration ............................................ 215 7.7.1 Calibration mode ................................. 217 7.7.2 Inspection-related calibration ...................... 217

7.8 Key terms ............................................. 218

8. Overview: Image Acquisition and Illumination . ........... 221 8.1 Solid-state sensors ...................................... 221

8.1.1 CCD sensor operation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 222 8.1.2 Properties of CCD sensors ......................... 224 8.1.3 Image degradation ................................ 226

8.2 Standard video cameras ................................. 228 8.2.1 Basic structure ................................... 228 8.2.2 The video standard ............................... 230 8.2.3 Sampling of the line signal ......................... 232 8.2.4 Extensions of the video standard ................... 235 8.2.5 Image quality .................................... 236

8.3 Other camera types ..................................... 238 8.3.1 Progressive scan cameras .......................... 238 8.3.2 Asynchronous cameras ............................ 238 8.3.3 Digital cameras .................................. 239 8.3.4 Line-scan cameras ................................ 240 8.3.5 Additional camera properties ...................... 242

8.4 Transmission to the computer ............................ 243 8.4.1 Basic operation of a frame grabber ................. 244 8.4.2 Frame grabbers for standard video cameras .......... 246 8.4.3 Frame grabbers for other camera types .............. 246

Table of Contents 13

8.4.4 Direct digital transmission ......................... 248 8.5 Optical foundations ..................................... 249

8.5.1 F-number ....................................... 249 8.5.2 Thin lens imaging equation ........................ 251 8.5.3 Depth of field .................................... 255 8.5.4 Typical imaging situations ......................... 259 8.5.5 Aberrations ...................................... 260 8.5.6 Lens selection .................................... 262 8.5.7 Special optical devices ............................ 264

8.6 Illumination technology ................................. 265 8.6.1 Light sources .................................... 266 8.6.2 Front lighting .................................... 267 8.6.3 Back Lighting .................................... 270

8.7 Key terms ............................................. 272

9. Presence Verification . .................................... 279 9.1 Simple presence verification .............................. 279

9.1.1 Part geometry ................................... 280 9.1.2 Illumination ..................................... 281 9.1.3 Position adjustment .............................. 282 9.1.4 Segmentation .................................... 284 9.1.5 Evaluation ...................................... 285 9.1.6 Segmentation with template matching .............. 286

9.2 Simple gauging for assembly verification ................... 288 9.2.1 Illumination ..................................... 288 9.2.2 Inspection criteria ................................ 289 9.2.3 Object creation and measurement computation ....... 291 9.2.4 Position adjustment .............................. 292

9.3 Presence verification using classifiers ...................... 293 9.3.1 Illumination ..................................... 293 9.3.2 Inspection of the caulking ......................... 297 9.3.3 Type verification of the flange ...................... 302

9.4 Contrast-free presence verification ........................ 306 9.5 Key terms ............................................. 308

10. Overview: Object Features . ............................... 311 10.1 Basic geometrical features ............................... 311

10.1.1 Enclosing rectangle ............................... 311 10.1.2 Area and perimeter ............................... 312 10.1.3 Center of gravity ................................. :U5 10.1.4 Axes and radii ................................... 316

10.2 Shape-descriptors ....................................... 317 10.2.1 Curvature ....................................... 317 10.2.2 Fiber features .................................... 320 10.2.3 Euler's number ................................... 321

14 Table of Contents

10.2.4 Moments and Fourier descriptors ................... 321 10.3 Gray level features ...................................... 322

10.3.1 First-order statistics .............................. 322 10.3.2 Texture features .................................. 323

10.4 Key terms ............................................. 324

11. Outlook: Visual Inspection Projects . ..................... 327

A. Mathematical Notes ...................................... 331 A.1 Backpropagation training ................................ 331

A.I.1 Neural networks - concept and history .............. 331 A.I.2 Fundamentals .................................... 332 A.I.3 Backpropagation ................................. 333

A.2 Computation of the depth of field ........................ 336 A.2.1 Limit distances ................................... 336 A.2.2 Depth of field at infinite distance ................... 339 A.2.3 Dependence of the depth of field on the focal length .. 340

B. The Companion CD ...................................... 343

References. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 345

Index ......................................................... 348