Time-of-Flight Sensor Fusion

Save favourite 17 May May 2017

Capturing scenery depth information is a fundamental task in computer vision. Applications in manufacturing, security and surveillance, product quality control, robotic navigation, and three-dimensional (3D) media entertainment rely on accurate depth data. Dedicated range sensors, such as the Time-of-Flight camera (ToF), can simplify the scene depth capture process and overcome short-comings of traditional approaches.

On this page we provide selected evaluation sequences for our proposed Weighted Optimization Approach to Time-of-Flight Super Resolution, accompanying our publication [1]. Furthermore we provide the original ToF data and necessary data for the MPEG View Synthesis Reference Software” (VSRS) [2], so fellow researchers can relate to our work. The full evaluation of our approach will be available as soon as it is published [1].

 

Creative Commons License 

Disclaimer: The data presented on this page is licensed under a Creative Commons License.

 

Data is freely available for research and academia. We request you to cite the paper Schwarz, Sjöström, Olsson, "A Weighted Optimization Approach to Time-of-Flight Super Resolution "  [1]. Thank you!

 

Online self-evaluation

In these videos we present the different HRCs for SRC2 and 4 included in our subjective evaluation in [1].

 

Video 1: Depth map upscaling and view synthesis results.
Video 2: Subjective evaluation in stereoscopic 3D.

 

 

References

[1] S. Schwarz, M. Sjöström, and R. Olsson, “A Weighted Optimization Approach to Time-of-Flight Super Resolution,” Image Processing, IEEE Transactions on , vol.23, no.1, pp.214-225,2014, doi: 10.1109/TIP.2013.2287613t.

 

[2] “Report on experimental framework for 3D video coding,” ISO/IEC JTC1/SC29/WG11 MPEG2010/N11631, Oct. 2010, Guangzhou, China.

 

 

 

Test material download

The test sequences SRC1 (Video conferencing, little depth transition) and SRC3 (Video conferencing, large depth transition (fast)) are currently not publicly available.

 SRC2

SRC2

 SRC4

SRC4
Sceneario: Video conferencing, large depth transition (slow).
250 frames
Sceneario: Global camera movement on a still scene.
250 frames

 

 


View synthesis results for subjective stereo evaluation

Left and right view synthesis based on different depth maps (HRC1-4, plus simple interpolation). Resolution 1280 x 720, loss-less h.264 compression, AVI-format.

 

SRC2

(zip, 627 MB)

MD5: e8752b4c414424cbd980eeef34ce0907

SRC4

(zip, 668 MB)

MD5: 260ca02950ae2ad33d5f9fd6ddfd1f38


 

Original ToF data

Original readings from Fotonic B-70 ToF camera, including active brightness signal. Resolution 160 x 120 pixel, 16Bit, PGM-format.

 

SRC2

(zip, 22.7 MB)

MD5: 1b55b784252874beb4c78c4e62e245c6

SRC4

(zip, 21.5 MB)

MD5: 15244f8c2fcadcb2c98e2b79d32e519f


 

Texture & depth for VSRS

Original texture from video camera and upscaled depth sequences (HRC1-4, plus simple interpolation). Resolution 1280 x 720, YUV format.

 

 SRC2

(zip, 314 MB)

MD5: 709dedb92aef2218e66a3c297464df74

SRC4

(zip, 288 MB)

MD5: 2c026487fd43eb17e1345e673996796e

 

VSRS metadata (zip, 1.5kB)