Raindrop Detection and Removal from Long Range Trajectories
Oral presentation in ACCV 2014
Abstract: Raindrops adhered to a windscreen or window glass can significantly degrade the visibility of a scene. Detecting and removing raindrops will, therefore, benefit many computer vision applications, particularly outdoor surveillance systems and intelligent vehicle systems. In this paper, a method that automatically detects and removes adherent raindrops is introduced. The core idea is to exploit the local spatio-temporal derivatives of raindrops. First, it detects raindrops based on the motion and the intensity temporal derivatives of the input video. Second, relying on an analysis that some areas of a raindrop completely occludes the scene, yet the remaining areas occludes only partially, the method removes the two types of areas separately. For partially occluding areas, it restores them by retrieving as much as possible information of the scene, namely, by solving a blending function on the detected partially occluding areas using the temporal intensity change. For completely occluding areas, it recovers them by using a video completion technique. Experimental results using various real videos show the ef-fectiveness of the proposed method.
Thick and thin raindrops : Unlike some existing methods, in this work, we introduce a novel detection method applicable for both thick and thin raindrops as well as raindrops of any size, shape, glare, and level of blurring. We call a raindrop thick when we cannot see the objects behind it, and thin, when it is sufficiently blurred, but still allows us to partially see the objects behind it.
Raindrop imagery model : Unlike opaque objects, raindrops can look different in different environments due to the focus of the camera on the environment.
Spatio-temporal space and dense trajectories: Given a video sequence, we can form a 3D
Dense trajectories on rainy scenes : In our findings, with
regard to o cclusions, a tra jectory can encounter the following events: (A) it is
o ccluded by a solid non-raindrop ob ject and drifted; (B) it is o c clude d by a thick
raindrop and drifted; (C) it is o ccluded by a thin raindrop and drifted; and (D)
it is o cc luded by a thin raindrop but not drifted.
Motion inconsitency : When a point is covered by a thin raindrop, it has two correspondences in other frames: the raindrop and the covered object. The causes incorrect tracking for optical flow that assumes only one correspondence.
Appearance along trajectories, appearace inconsitency and sharpness: all four events might
generate appearance changes, particularly for events A, B and C. We calculate
the appearance consistency for node using the difference of SIFT features. Similarly, we can evaluate the patch sharpness along the trajectory.
Label raindrops using MRF: The detection of raindrops can be described as a binary labeling problem, where
for given a frame, the labels are raindrop and non-raindrop. Similarly, the mixture level can be described as a multiple labeling problem. The labeling can be
done in the framework of Markov random fields (MRFs).
Raindrop Removal Results
Motion field completion results