Computed tomography (CT) aims to recover the volumetric three dimensional (3D) structure of heterogeneous objects. Traditionally, CT refers to a medical imaging modality. There the object, radiation source and detector array are fully controlled. This results in a linear image formation model. In contrast, we seek CT of natural objects, acquired outdoors, in an uncontrolled environment. We focus on underwater plankton and cloud droplets, which strongly affect climate. Cloud imaging is a highly complex and recursive model, involving solar radiation multiple scattering which makes CT non-linear in the unknowns. Moreover, cloud CT requires multi-view images acquired by satellites, but clouds evolve while the satellites overfly. Hence, we derive spatiotemporal CT of time-varying clouds. Moreover, we accelerate scattering-CT analysis by several orders of magnitude: We develop a deep neural network for variable imaging projection cloud tomography (VIP-CT). VIP-CT is agnostic by construction to the cameras’ positions, being flexible in imaging geometry. Then, we extend VIP-CT to estimate per 3D cloud location a function:. Consequently, 3D recovery includes various statistics, such as the most probable result and uncertainty. Also, we show the recovery uncertainty effect on precipitation and renewable energy forecasts. To improve out-of-distribution inference, we incorporate a novel self-supervised learning through differential rendering. The ability to do CT in variable geometries is further generalized for underwater plankton. There, each specimen imaged only once, at random unknown pose and scale. Using this image ensemble, we achieve plankton 3D tomography and estimate population statistics. To counter errors due to plankton non-rigid deformations, we weigh the data by an advanced statistical model, developed for Cryo-EM.
Roi Ronen is a Ph.D. student under the supervision of Prof. Yoav Schechner and a researcher at Amazon Web Service (AWS) – AI labs. His research explores the interface between machine learning, physics-based rendering, 3D tomography and computational photography. His paper won the JOSA A Emerging Researcher Best Paper Prize for 2021.
Roi also received the Jewish National Fund Climate scholarship.
Personal web page
