LPR System

License Plate Recognition System

Abstract

License plate recognition (LPR) system plays an important role in numerous applications, such as parking accounting systems, traffic law enforcement, road monitoring, expressway toll system, electronic-police system, and security systems. This paper proposes an experimental license plate recognition system for Indian-style license plates. The system uses mathematical morphological operations and a projection searching algorithm for the extraction of license plates. The recognition result is obtained from a character recognition based on signature (or skeleton) matching method. The proposed work in this paper is the first attempt towards Indian-style license plates.

Keywords:

Plate Recognition; Morphological Operation; Character Segmentation; Electronic Toll System; Image Processing; License Plate Recognition

1. Introduction

License plate recognition (LPR) system plays an important role in numerous applications, such as parking accounting systems [1], traffic law enforcement [2], road monitoring, expressway toll system, electronic-police system, and security systems. There are a number of techniques used so far for the recognition of license plates such as vector quantization, template matching and neural network etc [3, 4]. In this paper, an experimental system for the recognition of license plates in India, which is based on morphological operations to locate the plate and signature matching method to recognize the characters, is presented.

The rest of this paper is organized as follows: section 2 discusses the system architecture and the steps involved in the recognition process; section 3 explains each step in details; section 4 shows the experimental results and has a conclusion regarding the system.

2. System Architecture

The typical components of the LPR system as in Figure 1, includes video cameras, image grabber card, computer and the corresponding license plate recognition software [5]. The proposed system presented is designed to recognize license plates from the front and rear of the vehicle. In this system, a high resolution camera captures the image of a vehicle. The captured image is passed through a license plate extractor, which in turn gives its output to the segmentation stage. The segmented characters are then input to the recognition stage for the final result.

Figure 1: LPR System

The typical license plate is composed of both letters and digits, as in Table 1. There are two kinds of plate according to the number of rows. The license plate is divided into four parts by three dots (or space).

One RowTwo RowsTable 1: Typical Indian License Plates

As shown in Table 1, the number of characters in a plate can be varied from 8 to 10. Besides the number of rows or the number of characters, a license plate can also be classified into two kinds according to its background colour and the colour of characters. The typical license plate in India has a white background and black characters, but for some tax-free cars (or taxi's), the colour of the characters is black with yellow background.

According to the previous discussion, the proposed license plate recognition system should be compatible to all these different kinds of plates in India.

The proposed system can be divided into four phases: Image acquisition, Candidate region extraction, Character segmentation and Character recognition.

Image Acquisition: It is very important part in license plate recognition system; it is the prerequisite for follow-up processing. The image quality affects directly the next processing and recognition effect. In literature [6], Qingxiang Liu proposed to obtain clearer images, needs to consider many influence of factors on image quality, mainly includes: the camera and image card selection, the video camera position demarcation, the vehicle speed, the distance between vehicles comes in or goes out, the weather conditions on camera exposure, captured images and so on.

• 
  
  Candidate Region Extraction: The proposed system uses mathematical morphological algorithms and matching of vertical edges to find the candidate regions that may be contain characters in the input image, and find out the most likely candidate as the location of a license plate. This approach involves five steps: morphological operations, vertical edge detection, size and shape filtering, vertical edge matching and finding the B/W (Black to White) pixels ratio.
• 
  
  Character or Glyph Segmentation: According to the value of the vertical and horizontal projections, extracted license plate can be divided into separated characters.
• 
  
  Character Recognition: The final stage is an O.C.R (Optical Character Recognition), here skeleton matching method is used to recognize the characters in a plate. Before that pruning and normalization is done, this is different from the traditional methods.

 The flowchart of the proposed system is shown in Flowchart 1.

  Flowchart 1: The Proposed System

In section 3, the detailed implementation of each step is presented.

3. System implementation

3.1 Image Acquisition

This is the first step in the LPR system. There are basically following three ways of acquiring an image.

• 
  
  Using a conventional analog camera and a scanner.
• 
  
  Using a digital camera.
• 
  
  Using a video camera and a frame grabber (capture card) to select a frame.

In the proposed system a high resolution video camera and a frame grabber is used to acquire an image as shown in Figure 2(a). The acquired image is first converted into gray scaled image, Figure 2(b). Conversion to gray scale facilitates the extraction of the license plate.

Figure 2(a): Color Image

Figure 2(b): Gray Image

3.2 License Plate Extraction

License plate or Candidate region extraction is the key step in an LPR system, which influences, the accuracy of the system significantly. As discussed in section 2, the Indian-style license plates are varied in number of rows, the number of characters, and the color scheme. The plate localization algorithm used in this paper should be workable in all those situations. In the adopted approach, extraction of license plate can be set up using five steps, which is explained in the following subsections.

3.2.1 Mathematical Morphology:

This is a part of digital image processing which is concerned with image filtering and geometric analysis by using structuring elements (SE). Dilation and Erosion are two fundamental operations in the algebra of mathematical morphology. These two operations can be applied in different combinations in order to obtain more sophisticated operations.

Dilation is a morphological transformation that combines two sets by using vector addition of set SEs, and Erosion can be obtained by dilating the complement of the black pixels and the taking the complement of the resulting point set.

Let 'B' denote an SE which is a 3x3 square. The dilation of the image 'A' using the SE 'B' result in another binary image 'z' which is defined as Eqn (1), and illustrated in Figure 3(a).

    ..… Eqn (1)

    Figure 3(a): Illustration of Dilation

Similarly, the erosion of the image 'A' by SE 'B' is the binary image 'z' defined as Eqn (2), and illustrated in Figure 3(b).

….. Eqn (2)

Figure 3(b): Illustration of Erosion

Usually dilation and erosion are combined into two other operations [7]: Opening and Closing.

Opening:

it is obtained by first eroding the image with an SE and dilating the result using the same SE. Opening an image smoothes the contours, breaks narrow isthmuses, and eliminates small islands. And it is defined as

….. Eqn (3)

Closing:

it is obtained by first dilating the image with an SE and then eroding the result using the same SE. Closing an image smoothes the contours, fuses narrow breaks and long thin gulfs, eliminates small holes and fills gaps in contours. And it is defined as

….. Eqn (4)

It is known that a license plate is composed of several characters, which are in high contrast to their background, so an opening operation can be used to blur the character in the license plate. The opening operation with an SE 'B' removes all details smaller than 'B' from an image, therefore, the location of the plate would be blurred after the opening.

3.2.2 Vertical Edge Detection:

For the transformed gray-scaled image, its corresponding vertical edges are
detected using sobel or prewitt or canny edge detectors. The proposed system uses canny edge detector because the threshold used is dynamic and it shows better results. Figure 3(c) shows an image with vertical edges. It is observed that most of the vehicles usually have more horizontal lines than vertical lines [8]. To reduce the complexity of algorithm, the vertical edges are detected.

As the fact that spatial variation in the character regions is much more frequent than in the plate, the character region can be found by the contour of vertical and horizontal projection. The dual-direction projection is calculated, as Figure 3(d). By computing the horizontal projection of the whole image, the region containing many white dots will be shown as continuing high values. For enclosing the proper location of the license plate in a vertical direction, a threshold is applied. Then the same calculation is applied vertically. The high concentration area in both projections is accepted as the candidate region which may contain the license plate. After this, easily four corners of the license plate can be located.

3.2.3 Size-and-Shape Filtering:

Filtering is basically used to remove objects that do not satisfy some specific features. In the proposed approach, seed-filling algorithm by smith [9] is used to filter the undesirable objects. After filtering, the result is shown in Figure 3(e). The process of filtering starts by first detecting all the regions. A group of white pixels is called a region if they are eight-connected pixels, i.e., all eight pixel around a particular pixel under consideration are called eight-connected pixels of the pixel under consideration.

Once all the pixels of a region are marked, the region is checked for its size and shape. Through filtering, the target is to select the regions that can serve as possible license plate boundaries and discard the others by tilling it with black pixels.

3.2.4 Vertical Edge Matching:

In this phase, the width to height ratio of license plate is used to match the vertical edges for finding the regions where there is a high probability of a license plate. The ratio of width to height of Indian license plate is about 4:1. After filtering step, the image in Figure 3(f) is extracted having several vertical regions. But, only two of the regions could be the possible boundary of the license plate. The task of vertical edge matching is to find out the correct pair of regions that include the license plate. To achieve this task, the width-to-height ratio of the rectangular area between two vertical regions is compared with the standard ratio of a license plate. The standard ratio is taken, ranges from 3.75:1 to 4.25:1. Figure 3(g) shows the extracted license plate for the matched vertical edges.

3.2.5 B/W Ratio and License Plate Extraction:

This phase is considered when more than one pair of probable license plate regions is obtained. In this phase, black to white pixels ratio of every pair of regions is calculated with respect to the Figure 3(g). If the ratio is within a specified threshold, it is taken as a probable license plate. The threshold is selected based on tests performed on a number of plates. If a pair is probable license plate then there will be more white pixels than a region pair which doesn't, since the license plate region pair contains characters which are represented by white pixel.

3.3 Character Segmentation

After the localization of the license plate, a vertical projection is performed to segment the characters. As shown in Figure 4(a), the projection contour would present high positive value among those characters and very low positive values among the intervals. The distance between each other low value area can be regarded as the width of a character.

Figure 4(a): Projection of License Plate

Figure 4(b) shows the case that, found corner points may not correspond to a rectangle then, characters are not exactly in a horizontal direction and the gaps in the projection contour are not wide enough to segment each character properly. The area described by the four corner points are transformed into a rectangular area by means of a bilinear transformation and converted to fixed dimensions of 200x50 pixels by means of bilinear interpolation. This is illustrated in figure 4(c). This figure shows that the bilinear transformation can correct for most of the perspective distortion, but it cannot correct the distortion that is caused by bent plates.

Figure 4(b): Projection of License Plate

Figure 4(c): Projection of License Plate

3.4 Character Recognition

This is the last step in the LPR system. There are many recognition methods for individual characters, such as neural network method [2, 5], template matching methods [7, 10] and skeleton matching method [7].

Because, the computation load of neural network algorithm is big and the computation speed is slow, it usually uses for the recognition of small character set. Template match algorithms are dealing with static or lower vehicle speed, moreover calculation is quite complex, the performance of real-time processing and manipulation of concurrent event is not very good, and brings about hidden danger for system stability. Skeleton matching is the direct character recognition method. If the characters are not belong to the limited character set, or if the quality of the pre-treatment image is low, then the recognition rate of this method is poor. Skeleton matching method compares the skeleton of the character to be recognition and the template character skeleton. It has advantages of the computation load small, the template not easy to confuse in single template, and as can be in a certain range within the framework adjustment make it more adaptive, taking into high recognition rate in multi template.

In this proposal, skeleton (or signature) matching method is used. This phase is divided into two stages:

• 
  
  Pruning and Normalization of Individual characters
• 
  
  Recognition using Skeleton Matching

3.4.1 Pruning and Normalization:

In this step, first the extracted characters are pruned to remove unwanted area and refined to fit them into a block containing no white spaces on all the four sides. Figure 5(a) shows the template of the extracted character 'O' with respect to the above refinement. Now each character image is normalized to a size of 40x40. Figure 5(b) shows the character 'O' pruned and normalized to 40x40. Normalization is done using window to view port transformation. This mapping is used to map every pixel of the original image to the corresponding pixel in the normalized image. The image obtained is invariant to scaling. This is one of the steps different from the traditional methods. It brings down the adverse influence caused by inaccurate character location and size.

Figure 5(a): Extracted Character 'O'

Figure 5(b): Pruned and Normalized to 40X40

3.4.2 Recognition using Skeleton Matching:

After each character is normalized, thinning and skeleton matching method is used for character recognition and a post-process would be used to improve the recognition rate. As the Indian–style license plate is composed of both letters and digits, it is difficult to get a high recognition rate between these two sets of characters, especially in the case of 'B' and '8', 'O' and '0'. 'G' and '6', etc., which are somewhat similar.

According to the characteristics of the license plate, the character recognition process can be divided into two sections. A set of letter patterns is used to recognize the first two characters and then another set of digit patterns is used to recognize the last four characters. With this prior knowledge, 60% of the ambiguity between characters will be solved.

However, there are also some characters which are very similar to each other. To tackle these problems, a post-process which uses some local features is applied to make the final decision on some characters, such as: digit '0' and the letter 'O', digit '8' and letter 'B'. etc.,. After the post-processing, some confused recognized characters can be corrected.

4. Experimental Result and Conclusions

A database containing 200 Indian-style license plate images has been used to test the proposed system. Some experimental results are shown in Table 2. Experiments show that the system has a good performance on Indian-style license plate recognition. The proposed system can extract the license plate from a image accurately in different colour scheme and different character layouts.

However, there is still some further research to be done to ensure improvement improved. For example, the system cannot work well in some situations, such as certain conditions when illumination is not very good, and with license plates which are badly positioned.

References

[1] Garibotto. G. Castello. P. etc, "Speed-vision: Speed measurement by license plate reading and tracking", Proceeding of Intelligent Transportation Systems, 2001, pp. 585-590, Aug. 2001.

[2] Maged M. M. Fahmy, "Automatic number-plate recognition: neural network approach" Proceedings of Vehicle Navigation and Information Systems Conference, pp.99-101, Sept. 1994.

[3] M., Yu and Y.D. Kin, "An approach to Korean license plate recognition based on vertical edge matching", IEEE International Conference on System, Man, and Cybernetics, vol. 4, pp.2975-2980,2000.

[4] S.L. Chang, L.S. Chen, Y.C. Chung and S.W. Chen, "Automatic license plate recognition", Intelligent Transportation Systems, IEEE Transactions, vol 5, no.1, pp. 42-53, 2004 Mar.

[5] Ling Liu, Xinghui Zhang, "Study on Intelligent License Plate Recognition System", Chinese Journal of Scientific Instrument, 2004, 25(S1): 565-566, 577.

[6] Qingxiang Liu, Tianfa Jiang, "Technque of Obtaining Image in the Intelligent Vehicle Plate Reorganization System", Journal of Wuhan University of Technology (Transportation Science & Engineering), 2003, 27(1): 127-130.

[7] Yohimori, S., Mitsukura, Y., Fukumi, M., Akamatsu, N.; Pedrycz, N., "License plate detection system by using threshold function and improved template matching method", Fuzzy Information, 2004. Processing NAFIPS '04. IEEE Annual Meeting of the Volume 1, 27-30 June 2004 Page(s):357 - 362 Vol.1

[8] M., Yu, and Y.D. Kim, "An approach to Korean license plate recognition based on vertical edge matching", IEEE International Conference on Systems, Man, and' Cybernetics, vol. 4, pp. 2975-2980, 2000.

[9] A.R., Smith, Tint Fill, Cornpurer Graphics, vol. 13,
nu. 2, pp. 276-283, 1979.