331 / 1971-01-01 00:00:00

An Electronic Image Stabilization Based On The Davinci Platform

DaVinci platform；electronic image stabilization；Levenberg-Marguardt algorithm

Abstract:In order to improve the video instability of the camera and the chattering problem,An electronic technology about image stabilization is proposed,which is based on the DM6467 development platform of DaVinci.This paper describes the basic principle of electronic image stabilization system.Then we use fast robust feature point algorithm to complete the match of feature points of video image,and use space random sample consensus algorithm to eliminate false matched feature point;finally we use Levenberg-Marguardt algorithm to calculate the image motion estimation vector,strategy of motion compensation for video information the reference frame update,which realizes high precision digital image stabilization.
introduction
Video signal of the camera on mobile platform contain both active movements, also random motion of the video surveillance platform (such as translation, rotation and other sports). This random motion make the video output unstable, and the observers will have visual fatigue that generate misjudgment or have omission of judge.This paper provides an overview of related resources of DM6467 processor,introduced the DM6467 software development environment.Electronic image stabilization algorithm implementation process,,using LM algorithm to estimate motion vector, and the use of a simple reference frame update strategy for motion compensation.
Hardware’s summary of the platform
Benefiting from the DSP and RISC technology,the TMS320DM6467 dual-core architecture includes an ARM926EJ-S ARM core and a high-performance C64x + DSP core.

Fig. 1 DaVinci technology system structure
System encode the output analog signal of the CCD camera by the TVP5147,and the data would be displayed on the LCD after processing.For DaVinci platform,In the DSP-side,DSP/BIOS would implement the audio/video algorithms,It use Codec Engine and Codec Server to manage data exchange between ARM and DSP. Electronic image stabilization system mainly consists the following three parts:DVEVM evaluation kit;CCD camera;LCD Monitor:
The theory of image stabilization
SURF feature extraction:In electronic image stabilization method, to calculate an affine model by matching feature points parameters,we could obtain affine transformation between two images. Fast robust feature points (SURF) algorithm is the basis of the scale invariant feature transform (SIFT) algorithms to simplify or similar the Hessian matrix determinant .

(1)

Formula (1)shows that, can be obtained to use iterative computation. Assuming for the pixels of the original image, the improved method to calculate the integral image shows as follows:

(2)

In it,，and make ，.After improvement, integrating images only need to check the address calls and read and write operations for ine variable vector in one space of an image, thereby reducing the space occupied by the data, reducing the number to call or check the site and improving the computing speed and reduce the computation time.
gauging of salting point based Fast-Hessian:Since the Hessian matrix has a good detection result, and the maximum value detected Blob similar determinant of the Hessian matrix of points. Determinant can also be used to select the scale. Given point in the image , Hessian matrix is defined as the following with scales:

(3)

In the formula,is the convolution of a Gaussian second order derivative and the image I at point x, and have the same meaning. SURF algorithm uses second order approximation of the Gaussian filter to the box deflector, in Figure 2, deal With or in this approximation.

Fig. 2 two order differential module based gauss and its simplicity
The method have the similar with the convolution using a Gaussian approximation method,the filter template is is approximate with a Gaussian function when in Fig. 3,,we can get approximately matrix,and its determinant is

(4)

Where, w is a weight factor to compensate Hessian determinant. Here the weighting factors are applied is in order to simplify the calculation of the influence of approximation.based ,it Can be further calculated by weighting factor close to Hessian values,we can see w of 0.9,which is 2-norm.Determinant of the Hessian matrix approximation response reflects the blob in the image of x locations. In order to obtain a scale blob response image, the need for all pixels in the image traversal. Using different scales can be multi-scale image pyramid blob response through this image pyramid, you can search for extreme points blob responses.
Motion detection of LM:Match detected image features and characteristics of the image sequence is the key to obtain a motion vector between frames.we can use the detected feature points to estimate the motion between frames of the Image sequence .The transformation of two frames can noted as the follow formula:

(5)

In which,is an orthogonal matrix, represents the angle of rotation between two frames.、represents the translation image pixels between two frames respectively,The method the using the least squares method (i.e.,to minimize the sum of squared residuals) to solve inter-frame motion estimation parameters are the most commonly used method.

(6)

Since the translation and rotational transformation,there be linear relationship between and :

(7)

In which, is the Jacobi matrix of transformation matrix consider with ,so:

(8)

Because in computer vision,most problems need to clearly describe the nonlinear least-squares linear relationship between the measured values and the unknown value, and for this project the two frames of the image rotation and translation calculations, can be formalized as:


(9)

In it,represents a given set of matching feature points(,).transform plane parameters. Jacques motion parameters based on the approach to the prediction error and the direction of specific strength matrix proportional.Then wen can calculate .

(10)

Updates the parameter vector.Additional damping parameter is used to ensure that the system take on the energy of a "down" steps,it is a necessary part of LM algorithm. In this issue, its initial value is set to 0.1.
Software Arithmetic

Figure 3 the software arithmetic
Using SURF algorithm to complete video image matching inter-relationship between the use of space - a random sample consistency (SCRAMSAC) algorithm to eliminate the mismatching feature points. Finally, the LM algorithm to obtain accurate motion estimation vector image (GMV). Obtained by reference frame motion compensation vector update strategy.
Result of experiment and analysis
Result of SURF feature point extraction:This experiment is carried out on the PC based Visual Studio 2010 and OpenCv2.4.4.In this experiment, it use an example of the opencv library to extract feature point,the operating results are showed in Figure 4.

（a） effect drawing of rotate 20° （b） effect drawing of rotate -20°
Fig. 4 original drawing and the matching drawing
The experiment’s result of image moving estimation
Tables 1 Result of rotating parameter
Standard parameters Calculating parameters error
angle Horizontal shift Vertical shift angle Horizontal Vertical angle Horizontal Vertical
5 10 10 4.674 9.528 10.14 6.5 4.7 1.4
-10 -10 5.082 -9.99 -9.67 1.6 0.1 3.3
10 10 10 9.560 9.970 10.21 4.4 0.3 2.1
-10 -10 9.622 10.108 -9.97 3.8 1.1 0.1
-5 10 10 -4.560 10.309 9.975 8.8 3.1 0.2
-10 -10 -4.978 -9.722 9.847 0.4 2.8 1.5
-10 10 10 -9.334 10.112 9.486 6.7 1.1 5.1
-10 -10 -9.984 -9.650 10.21 0.2 3.5 2.1
0 10 10 0.142 10.041 10.22 - 0.4 2.2
-10 -10 0.008 -9.925 -9.74 - 0.8 2.6
In this study, photoshop have done the image rotation and translation,After the effect of stabilization after the system shown in Fig. 5 .

(a)The tenth figure (b)the thirtieth figure (e)The 50th figure
Fig. 5 stable video of output
The figure is in the 720×576 resolution, 13fps case can stably output the video stream. After the algorithm of image feature points with the registration test image to rotational translation of motion estimation, and video quality evaluation tests with respect to the final reference frame, the description of the design of the final system is capable of image stabilization 720 × 576 Real-time video stream output stability under the resolution, 13fps requirements.
References:
[1]li huijuan.research and achievement about electronic image stabilization based on DSP.Xi’an:xi’an university of science and technology learned journal.2009
[2]Zhou Tong.electronic image stabilization based on modified canny edge detection.Computer application research.2010.2,27(02):506-508.
[3]K.Madsen,H.B.Nielsen,O.Tingleff.Methods for Non-linear Least Squares Problems[M]. Informatics and Mathemtical Modelling Technical Unversity of Denmark ,2004.4.
[4]erbert Bay,Andreas ESSTinne,Tuytelaars.Speeded Up Robust Features[J]. Computer Vision and Image Understanding,2006.
[5]Torsten Sattler,Bastian Leibe,Leif Kobbelt.SCRAMSAC: Improving RANSAC’s Efficiency with a Spatial Consistency Filter[J]. 2009 IEEE 12th International Conference on Computer Vision.
[6]K.Madsen,H.B.Nielsen,O.Tingleff.Methods for Non-linear Least Squares Problems(2nd Edition)[M].Informatics and Mathemtical Modelling Technical Unversity of Denmark , April 2004.
[7]Torsten Sattler,Bastian Leibe,Leif Kobbelt.SCRAMSAC: Improving RANSAC’s Efficiency with a Spatial Consistency Filter[J]. 2009 IEEE 12th International Conference on Computer Vision.
Contacts: Name: LANG Zhanpo Mobile phone: 18092855636
Address:Xi'an University of Science And Technology, Xi'an,Shaanxi province
Receiver:LANG Zhanpo Postcode: 710054