Some samples of my recent work

Camera Motion& Focal-Length Estimation form Six Pixels

Investigators: Hongdong Li

Published on ECCV’2006

Theoretically, it is well-known that one can estimate camera motion and some intrinsic parameters from less than eight points of two views. One example is the 6-point 2-view that estimates an unknown but constant focal length as well as the camera motion from two views. However in practice, there are very few practical algorithms available so far. Before our result the only existing algorithm is a method based on Groebner Basis technique which is quite involved and not easy to use for non-expert user.

In this research we present a simple and practical solution to the 6-point 2-view problem. Based on the hidden-variable technique we have derived a 15th degree polynomial in the unknown focal-length. During this course, a simple and constructive algorithm is established. To make use of multiple redundant measurements and then select the best solution, we propose a kernel-voting scheme. The proposed algorithm has been tested on both synthetic data and real images. Satisfactory results are obtained for both cases. For reference purpose we have included our Matlab implementation in the paper, which is very concise and consists of 20 lines of code only. The result of this paper will make a small but useful module in many computer vision systems.

Moreover, we further show that the above “hidden variable” idea is not a sole trick, rather it is a generally applicable technique which is valuable to many other vision problems too. For example, the five-point problem, and the absolute orientation problem, etc.

An Algebraic Technique for Camera Lens Distortion-Removal

Investigators: Hongdong Li and Richard Hartley

Published on ICCV’05 OmniVis workshop

Lens distortion is a very common and significant problem of daily-use cameras. Especially when the vision task is to measure, track or reconstruct 3D scene, a small lens distortion may cause a large deviation from the correct value. Although this problem was widely studied by photogrammetrists, striving for extreme accuracy, it has been largely overlooked in the extensive literature of computer vision during the past decade or so, and most existing algorithms are not fully satisfactory.

This project proposes a self-calibration method for automatically correcting lens distortion from point correspondences of two views. The camera is observing either a planar scene or a general 3D scene. For each case, based on the two-view invariants we have derived a system of algebraic equations which relate the invariants to the distortion parameters to be found. We then propose a non-iterative procedure to solve the equations system, and a kernel-voting scheme to select the best root. Being a non-iterative approach, our method overcomes many problems with the conventional iterative approach. It also largely decouples the estimation of the distortion from the estimation of other camera parameters, therefore delivers more reliable results. Experiments on both synthetic data and real images have provided satisfactory results.

Novel View Synthesis Using Inverse Tensor Transfer

Investigators: Hongdong Li and Richard Hartley

Published on Signal Processing: Image Comm (Elsevier).

This project studies a new transfer-based novel view synthesis method. This method does not need a pre-computed dense depth map, therefore overcomes most common problems associated with the dense correspondence algorithms, yet still produces very photo-realistic novel images. The power of the method comes from the introducing and using of a novel inverse tensor transfer technique, which offers a simple mechanism to exploit both photometric constraints and geometric constraints across multiple input images. Our method works equally well for both calibrated images and un-calibrated images. Experiments on real sequences show promising results.

Matching 3D Objects by Conformal Representations

Investigators: Hongdong Li and Richard Hartley

Published on ACCV’06

Shape-based 3D object matching is a central task of a 3D recognition system. Traditional approaches often use either local registration technique (e.g., ICP) or statistical matching technique (e.g., PCA) to fulfil this task. The former technique relies on a sufficiently-close initial estimation, while the second captures only a rough shape orientation, therefore their performances are not satisfactory.

In this research, we have investigated a new mathematical tool of Discrete Conformal Mapping and its application in genus-zero 3D shape matching. The basic procedure is as follows. An input genus-zero 3D shape is first mapped to a sphere by a simple planar graph embedding algorithm. The above initial embedding is then refined to be a spherical conformal mapping. Then apply Spherical Harmonic analysis on this sphere, followed by a invariant-normalization procedure. The resultant invariant-coefficients are then used as the representation of the shape. Such representation has captured the original shape geometry completely and faithfully. This enables user use this representation to rigorously analyse the shape, and perform various shape-related tasks. As an example, we demonstrate the performance of 3D shape matching using our method.

Invariants for Discrete Structures – An Extension of Haar Integrals over Transformation Groups to Dirac Delta Functions

Investigators: Hans Burkhardt, Hongdong LI

Published on DAGM-2004

Due to the increasing interest in 3D models in various applications there is a growing need to support e.g. the automatic search or the classification in such databases. As the description of 3D objects is not canonical it is attractive to use invariants for their representation.

We recently published a methodology to calculate invariants for continuous 3D objects defined in the real domain R3 by integrating over the group of Euclidean motion with monomials of a local neighbourhood of voxels as kernel functions and we applied it successfully for the classification of scanned pollen in 3D. In this paper we are going to extend this idea to derive invariants from discrete structures, like polygons or 3D-meshes by summing over monomials of discrete features of local support. This novel result for a space-invariant description of discrete structures can be derived by extending Haar integrals over the Euclidean transformation group to Dirac delta functions.

Feature Matching and Pose Estimation Using Newton Iteration

Hongdong Li and Richard Hartley

Published on ICAIP, ICASSP’04.

Feature matching and pose estimation are two crucial tasks in computer vision. The widely adopted scheme is first find the correct matches then estimate the transformation parameters. Unfortunately, such simple scheme does not work well sometimes, because these two tasks of matching and estimation are mutually interlocked.

This research provides a new method that is able to estimate the transformation and find the correct matches simultaneously. The above interlock is disentangled by an alternating Newton iteration method. We formulate the problem as a nearest-matrix problem, and provide a different numerical technique. Experiments on both synthetic and real images gave good results. Fast global convergence was obtained without the need of good initial guess.

Virtual Stereoscopic Video Generation (ARC Linkage Project)

Enjoy impressive 3D effects on your PC without wearing laborious special eyeglasses? The aim of this project is to provide an easier and cheaper way to convert a standard 2D video/image into true holographic-like 3D video/images, and display it on an eyeglasses-free stereo monitor. We are going to use some state-of-the-art computer vision technologies to perform such conversion automatically, efficiently and in real-time (frame-rate), and the generated stereo video will then be displaying on a lenticular 3D monitor.

Intelligent Video Surveillance

(NSFC China Project )

Multi-body motion vision understanding with application to traffic scenes surveillance,  National Natural Science Foundation China (NSFC),  Grant number: 60105003, 2002-2004, I was the Chief Investigator, the Project Leader.  

Multi-view 3D scene/object reconstruction

　

Vision Navigation for Autonomous Land Vehicle

China State Key R&D Project, 1995-2000, 2001-2005, I was then one of the Principle Investigators, prior to that a student researcher.  Here is one of our autonomous land vehicles.

　

　

Compressed domain image contents based retrieval 

Natural Science Foundation of Zhejiang Province, Grant number: 600025, 2001-2003, I was the Project Leader.

　Syntactic-Semantic Chinese character recognition

Supported by Natural Science Foundation of China (NSFC) and China 863-Hi-tek program Grant,  I was a Research Student, this is part of my Master thesis.

Vehicle License Plate Recognition System

Province Key R&D program, I am one of the Principal Investigators.