Most cameras are inherently designed to mimic what is seen by the human eye: they have three channels of RGB and can achieve up to around 30 frames per second (FPS).
However, some cameras are designed to capture other modalities: some may have the ability to capture spectra from near UV to near IR rather than RGB, polarimetry, different times of light travel, etc. Such modalities are as yet unknown, but they can also collect robust data of the scene they are capturing.
This book will focus on the emerging computer vision techniques known as computational imaging. These include capturing, processing and analyzing such modalities for various applications of scene understanding.
Table of Contents
Introduction xiii
Takuya FUNATOMI and Takahiro OKABE
Part 1 Transient Imaging and Processing 1
Chapter 1 Transient Imaging 3
Adrian JARABO
1.1 Introduction 3
1.2.Mathematical formulation 5
1.2.1 Analysis of transient light transport propagation 7
1.2.2 Sparsity of the impulse response function T (x, t) 8
1.3.Capturinglight in flight 9
1.3.1 Single-photon avalanche diodes (SPAD) 11
1.4.Applications 14
1.4.1.Range imaging 14
1.4.2.Material estimation and classification 14
1.4.3 Light transport decomposition 15
1.5.Non-line-of-sight imaging 15
1.5.1.Backprojection 17
1.5.2 Confocal NLOS and the light-cone transform 17
1.5.3 Surface-based methods 18
1.5.4.Virtualwaves and phasor fields 19
1.5.5.Discussion 21
1.6.Conclusion 22
1.7.References 22
Chapter 2 Transient Convolutional Imaging 29
Felix HEIDE
2.1 Introduction 29
2.2.Time-of-flight imaging 30
2.2.1.Correlationimage sensors 32
2.2.2 Convolutional ToF depth imaging 32
2.2.3 Multi-path interference 34
2.3.Transient convolutional imaging 35
2.3.1 Global convolutional transport 37
2.3.2 Transient imaging using correlation image sensors 37
2.3.3 Spatio-temporal modulation 40
2.4.Transient imaging in scattering media 41
2.5.Present and future directions 43
2.6.References 43
Chapter 3 Time-of-Flight and Transient Rendering 45
Adithya Kumar PEDIREDLA
3.1 Introduction 45
3.2.Mathematicalmodeling 46
3.2.1 Mathematical modeling for time-of-flight cameras 47
3.3.Howto render time-of-flight cameras? 50
3.3.1 Challenges and solutions in time-of-flight rendering 51
3.4.Open-source implementations 56
3.5.Applicationsof transient rendering 57
3.6.Future directions 61
3.7.References 62
Part 2 Spectral Imaging and Processing 69
Chapter 4 Hyperspectral Imaging 71
Nathan HAGEN
4.1 Introduction 71
4.2.2D(raster scanning) architectures 75
4.2.1.Czerny-Turnergratingspectrometers 76
4.2.2 Transmission grating/prism spectrometers 78
4.2.3.Coded aperture spectrometers 79
4.2.4.Echelle spectrometers 80
4.3.1Dscanningarchitectures 81
4.3.1.Dispersive spectrometers 82
4.3.2 Interferometric methods 83
4.3.3 Interferometric filter methods 83
4.3.4 Polarization-based filter methods 86
4.3.5 Active illumination methods 88
4.4.Snapshot architectures 88
4.4.1.Bowen-Walravenimage slicer 89
4.4.2 Image slicing and imagemapping 90
4.4.3 Integral field spectrometry with coherent fiber bundles (IFS-F) 93
4.4.4 Integral field spectroscopy with lens let arrays (IFS-L) 94
4.4.5 Filter array camera (FAC) 94
4.4.6 Computed tomography imaging spectrometry (CTIS) 96
4.4.7 Coded aperture snapshot spectral imager (CASSI) 97
4.5.Comparisonof snapshot techniques 98
4.5.1.The disadvantages of snapshot 100
4.6.Conclusion 101
4.7.References 102
Chapter 5 Spectral Modeling and Separation of Reflective-Fluorescent Scenes 109
Ying FU, Antony LAM, Imari SATO, Takahiro OKABE, and Yoichi SATO
5.1 Introduction 109
5.2.RelatedWork 111
5.3.Separationof reflection and fluorescence 113
5.3.1.Reflection and fluorescence models 113
5.3.2 Separation using high-frequency illumination 114
5.3.3 Discussion on the illumination frequency 116
5.3.4.Error analysis 118
5.4.Estimating the absorption spectra 119
5.5.Experiment results and analysis 122
5.5.1.Experimental setup 122
5.5.2 Quantitative evaluation of recovered spectra 122
5.5.3.Visual separation and relighting results 126
5.5.4 Separation by using high-frequency filters 130
5.5.5 Ambient illumination 134
5.6.Limitations and conclusion 137
5.7.References 137
Chapter 6 Shape from Water 141
Yuta ASANO, Yinqiang ZHANG, Ko NISHINO, and Imari SATO
6.1 Introduction 141
6.2.Relatedworks 143
6.3.Light absorption in water 145
6.4 Bispectral light absorption for depth recovery 146
6.4.1.Bispectral depth imaging 146
6.4.2 Depth accuracy and surface reflectance 147
6.5.Practical shape from water 148
6.5.1 Non-collinear/perpendicular light-camera configuration 148
6.5.2 Perspective camera with a point source 150
6.5.3.Non-idealnarrow-bandfilters 151
6.6 Co-axial bispectral imaging system and experiment results 151
6.6.1.Systemconfigurationand calibration 151
6.6.2 Depth and shape accuracy 152
6.6.3 Complex static and dynamic objects 154
6.7 Trispectral light absorption for depth recovery 155
6.7.1.Trispectraldepthimaging 156
6.7.2 Evaluation on the reflectance spectra database 157
6.8.Discussions 157
6.9.Conclusion 158
6.10.References 158
Chapter 7 Far Infrared Light Transport Decomposition and Its Application for Thermal Photometric Stereo 161
Kenichiro TANAKA
7.1 Introduction 161
7.1.1.Contributions 162
7.2.Relatedwork 163
7.2.1 Light transport decomposition 163
7.2.2.Computational thermal imaging 164
7.2.3.Photometricstereo 165
7.3.Far infrared light transport 165
7.4 Decomposition and application 171
7.4.1 Far infrared light transport decomposition 171
7.4.2 Separating the ambient component 172
7.4.3.Separatingreflectionand radiation 172
7.4.4 Separating diffuse and global radiations 172
7.4.5.Other options 173
7.4.6 Thermal photometric stereo 173
7.5.Experiments 174
7.5.1 Decomposition result 175
7.5.2.Surfacenormal estimation 177
7.6.Conclusion 179
7.7.References 180
Chapter 8 Synthetic Wavelength Imaging: Utilizing Spectral Correlations for High-Precision Time-of-Flight Sensing 187
Florian WILLOMITZER
8.1 Introduction 187
8.2.Syntheticwavelengthimaging 189
8.3.Synthetic wavelength interferometry 193
8.4 Synthetic wavelength holography 197
8.4.1 Imaging around corners with synthetic wavelength holography 199
8.4.2 Imaging through scattering media with synthetic wavelength holography 200
8.4.3 Discussion and comparison with the state of the art 203
8.5 Fundamental performance limits of synthetic wavelength imaging 205
8.6.Conclusionand future directions 210
8.7.Acknowledgment 210
8.8.References 211
Part 3 Polarimetric Imaging and Processing 219
Chapter 9 Polarization-Based Shape Estimation 221
Daisuke MIYAZAKI
9.1 Fundamental theory of polarization 221
9.2 Reflection component separation 225
9.3.Phase angle of polarization 226
9.4 Surface normal estimation from the phase angle 228
9.5.Degree of polarization 233
9.6 Surface normal estimation from the degree of polarization 236
9.7.Stokes vector 236
9.8 Surface normal estimation from the Stokes vector 237
9.9.References 239
Chapter 10 Shape from Polarization and Shading 241
Thanh-Trung NGO, Hajime NAGAHARA, and Rin-ichiro TANIGUCHI
10.1 Introduction 241
10.2.Relatedworks 243
10.2.1.Shadingand polarization fusion 243
10.2.2 Shape estimation under uncalibrated light sources 244
10.3 Problem setting and assumptions 245
10.4.Shadingstereoscopic constraint 246
10.5.Polarizationstereoscopic constraint 248
10.6.Normal estimation with two constraints 249
10.6.1 Algorithm 1: Recovering individual surface points 250
10.6.2 Algorithm 2: Recovering shape and light directions 251
10.7.Experiments 252
10.7.1 Simulation experiments with weights for two constraints 253
10.7.2.Real-world experiments 254
10.8.Conclusionand future works 263
10.9.References 263
Chapter 11 Polarization Imaging in the Wild Beyond the Unpolarized World Assumption 269
Jérémy Maxime RIVIERE
11.1 Introduction 269
11.2.Mueller calculus 271
11.3.Polarizingfilters 273
11.3.1.Linear polarizers 273
11.3.2.Reflectors 274
11.4.Polarizationimaging 275
11.5 Image formation model 277
11.5.1 Partially linearly polarized incident illumination 277
11.5.2 Unpolarized incident illumination 279
11.5.3.Discussion 280
11.6.Polarization imaging reflectometry in the wild 282
11.7.DigitalSingle-Lens Reflex (DSLR) setup 283
11.7.1 Data acquisition 283
11.7.2.Calibration 285
11.7.3.Polarizationprocessingpipeline 285
11.8.Reflectance recovery 287
11.8.1.Surface normal estimation 287
11.8.2.Diffuse albedo estimation 288
11.8.3 Specular component estimation 288
11.9.Results and analysis 291
11.9.1.Results 291
11.9.2.Discussion and error analysis 293
11.10.References 296
Chapter 12 Multispectral Polarization Filter Array 299
Kazuma SHINODA
12.1 Introduction 299
12.2 Multispectral polarization filter array with a photonic crystal 302
12.3 Generalization of imaging and demosaicking with multispectral polarization filter arrays 306
12.4.Demonstration 311
12.5.Conclusion 313
12.6.References 313
List of Authors 317
Index 319
