Chapter 1 Introduction

Contents:

1.1 Overview image processing

1.1.1 Terminology

Here is a comparison between the human eye and the brain system:

During image processing, an apparatus is used to make a digital image using not only visible light, but also other electromagnetic wavelengths such as infrared light or radar waves. Also sound, ultrasound for example, or radiation in the form of nuclear particles can be used. This acquisition of an image is named an image forming process.
Using computer programs the images can be improved or certain characteristical features such as edges of an object can be made better visible. After manipulation and projection onto a monitor, paper, or film, the image may be more suitable for interpretation by a human. This process is called picture or image processing.

If the image is manipulated into some measurements, for example the number of objects in the image or the size, then we call the process image analysis. If the image is transposed into an explicit description on a high level, for example this is John with his leg in a cast, then we speak of image interpretation. Image analysis and image interpretation are sometimes also called computer vision.
However, because there is no standard naming for these denominations, they are sometimes interleaved. Sometimes the term machine vision is used, especially for the processing of images of man-made objects as in manufacturing processes. Recognizing or classifying something into a certain category, class or idea is called pattern recognition. The input data for this process can come from different sources, such as (parts of) images, measurements from images, other measurements or by giving grades to make a classification.

Computer Graphics is the term used to describe the process with which an explicit description on a high level (e.g. John on a bicycle) or a low level (e.g. a few measurement points) is used to create an informative image, for example a realistic image of John on a bicycle or a graph of the Stock Exchange rates over the previous year. One can use this to visualize the result of the image manipulation process. In both computer graphics and image processing the same type of data structures and algorithms are used for the explicit description of an image.

1.1.2 Biological and computer processing

Visible light, reflected through a three dimensional object or made by such, finds its way through the eye lens onto the retina. Light sensitive cells revert the visible light into electrical signals, which can in be turn processed by the brain. Pimarily this is a subconscious activity, for example, we are very able and capable of recognizing faces, but we have very poor skills for describing that face to another person so that someone else can recognize that face. Foreknowledge and reasoning are extremely important for interpreting such an explicit description.

Biological visual hardware has developed and specialized itself over the last few million years. The organization and functionality of it is hardly known, except in extreme detail (the behavior of cells in laboratory animals) or generality (behavior of people during visual experiments).

Some characteristics of the human eye [see also fig. 2.1, 2.2]:

Visual Effects (see also illusionworks for multiple visual effects)

Some characteristics of images for computer processing:

Image processing is facing a difficult problem: to reinvent the most fundamental but unknown methods of the specialized, parallel and partially analog biological visual system, and implement its functionality using the aid of a totally different digital hardware.

1.1.3 Digital image processing

Some fundamental steps of image processing:

The are special methods to implement the fundamental steps mentioned previously:

Many challenges in this field are caused by:

A large number of other fields of expertise must be referenced:

Image interpretation; a computer will never yield the results that humans (and animals) are used to and consider to be normal. However, one can expect benefits:

Image processing is used for:

At the department of Informatics for Technical Applications we are doing research with satellite images for agricultural applications; including projects in cooperation with the EU Joint Research Center in Ispra. We are working on the segmentation of these multispectral images and on the recognition of crops using neural networks. We also concentrate on pixels which receive radiation from more than one crop type, which happens regularly with the 30 by 30 meter ground resolution which was used. Out of these mixed pixels we try to determine the percentage of each crop type. We also work in cooperation with the medical faculty and the Radboud hospital with the processing of medical images.

1.2 Lectures and practical work

During the lectures we will give an overview of the entire field of image processing, with the exception of the area of knowledge-based image recognition and close relations with the field of Artificial Intelligence. Computer vision hasn't been around for long and hardly has a theoretical basis laid out. Because of this, the lecture may seem more like a collection of methods and algorithms. Knowledge of these is necessary to resolve practical manipulation problems; and of course to be able to work on the theoretical basis of this field. That is why I will concentrate on the mathematical morphology and the methods based on the processing of special binary images, because this aspect does have a good theoretical basis.

During research in image manipulation and the use of it for resolving practical problems, one often relies on existing systems. This is composed out of many, methods and algorithms implemented as programs and a (graphical) user interface, on which after setting program parameters, you can execute on your own image. It is often possible to add your own program, potentially making use of particular functions in a library.

For the practical work we will make use of the free XITE system, which runs on the color SUN workstations or on a big SUN machine with a display on an Xterminal. The system can also be installed on a Linux PC, NT or Windows machine. Every group of 2 students will use this to do three assignments. The first assignment is to get comfortable with XITE and some elementary operations. Making use of an example, the second assignment is to implement an elementary operation. The last assignment is for each group to design and implement an algorithm as described in a scientific publication. This must further be developed (and improved) and described in detail. I will only plan a class or use the scheduled room during the extra course hour (werkcollege) if I or some students find it necessary.

During the lectures we will work in the breadth and with the practical work you will work into the depth of the field of image manipulation. The lecture will be concluded with an open-book exam, the grades of the practical work will be percentage wise weighed and averaged into one practical grade. The exam and the practical grade each count for half the resulting grade, if each is at least a 5.

The literature we will use is : Digital Image Processing,  Rafael C. Gonzalez and Richard E. Woods, Second Edition, Prentice Hall 2002  ISBN 0-201-18075-8. The material that will be examined is that which is dealt with during the lectures; the book gives an extensive explanation of most of the discussed methods and algorithms. All references in the text of these WWW pages are to this book, unless otherwise specified.

Over the last years I have posted my sheets onto web pages, revised them and added more text and pictures. I want to make the lectures more interactive. Over the last few years, my experience and that of students has taught me that the described individual algorithms and methods are not too complex to study independently. What is less suitable for independent studying is laying the link with the practical applications (with which I have more than 30 years of experience) and the relations with other algorithms and methods (this is necessary because otherwise the literature will become and stay a lose collection of diverse methods and algorithms). During the lectures I will concentrate on that as well as answering questions about the previously studied material. It is required that you read through the described methods and algorithms in advance.

1.3 Efficient implementation

In the field of image manipulation, efficient algorithms and efficient implementation of them is important because an image is composed of many pixels on which the algorithm must work. An execution time of minutes, hours or even days is absolutely no exception. Therefore I advise to try to first optimize an algorithm, often by storing calculations instead of repeating them, before you start on the implementation of it. Look at the expected execution time. A simple example follows:

/* slow.c */
#define SIZE 4096
float imin[SIZE][SIZE], imout[SIZE][SIZE];

int main(int argc, char **argv)
{
  int i, j, k, l;
/* initialize image */
  for(i=0; i < SIZE; i++) for(j=0; j < SIZE; j++) imin[j][i] = (i+j) % 256;
/* average each pixel with its neighbours */
  for(i=0; i < SIZE; i++) {
    for(j=0; j < SIZE; j++) {
      if( i == 0 || j == 0 ) imout[j][i] = imin[j][i];
      else if( i == SIZE-1 || j == SIZE-1 ) imout[j][i] = imin[j][i];
      else {
        imout[j][i] = 0;
        for(k=-1; k < 2 ; k++) for(l=-1; l < 2; l++) imout[j][i] += imin[j+l][i+k];
        imout[j][i] /= 9 ;
      }
    }
  }
}

Compiling with "gcc -O3 -o slow slow.c"  on my SUN machine and then in "/usr/bin/time slow" gives a (real) clocktime of 69.1 s. After rewriting the code the optimized version takes 14.9 s, a factor of 4.6 quicker. The faster code is a little bit longer but still well legible.

When SIZE is set to 6144 then the execution times become 1112.8 and 324.2 s, a factor 3.4 faster. When the SIZE is set to 2048, the times are 17.6 and 2.4 s, here the factor is 7.3.

Updated on 16 january 2003 by Theo Schouten.