An illustrated tour of
Differentiable
Photon Transport
Go from zero to actual differentiable photon transport in CUDA/Warp without losing your sanity to physics, your livelihood to GPU rental and your will to live to maths.
The time-dependent Boltzmann radiative transfer equation.One over c times the time derivative of radiance, plus its directional spatial derivative, equals extinction loss plus a source and incoming scattering. This monoenergetic form assumes elastic scattering and a phase function normalised over solid angle. A cone beam passes through the simplified pelvic conditioning-mask geometry from the synthetic CT example. Recorded Poisson photon counts build an unframed phosphor image of a separate isolated-pelvis phantom. Individual impacts fade with display persistence. Visible flights select recorded impacts. The view moves around the fixed acquisition. This primary-only illustration is not a calibrated clinical fluoroscopy model.
Volume I
Foundations
We introduce differentiable photon transport, and try to do so in the most painless but educational way possible: by constructing, step by step, a differentiable C-arm.
- 01IntroductionDefines the questions and boundaries of differentiable photon transport.Defines the questions and boundaries of differentiable photon transport.Defines the questions and boundaries of differentiable photon transport, beginning with a C-arm and an image, then asking which quantities we want to recover and what a useful derivative would tell us about them.
- 02TransmissionEstablishes the transmission model used throughout the volume.Establishes the transmission model used throughout the volume.Establishes the transmission model used throughout the volume, following attenuation along a single path and keeping the parameters, units and assumptions visible as that physical account becomes an executable model.
- 03Frames and motionFixes the coordinate frames and motion conventions for later chapters.Fixes the coordinate frames and motion conventions for later chapters.Fixes the coordinate frames and motion conventions for later chapters, giving the source, detector and object a shared account of where everything is so that a change in pose means the same thing in the geometry and the code.
- 04Volumes and raysConnects volumetric representations with line-integral operators.Connects volumetric representations with line-integral operators.Connects volumetric representations with line-integral operators, tracing a ray through a sampled attenuation field and examining how interpolation, boundaries and numerical choices shape the projection we ask the GPU to compute.
- 05Projection gradientsDevelops derivatives of the projection operation.Develops derivatives of the projection operation.Develops derivatives of the projection operation, asking how an image changes when its inputs move and how independent numerical checks can distinguish a useful gradient from one that merely looks plausible.
- 06Pose recoveryStudies pose recovery with differentiable image formation.Studies pose recovery with differentiable image formation.Studies pose recovery with differentiable image formation, working backwards from an image to geometry while considering the choice of image comparison, the initial estimate and the ambiguities that a single view cannot resolve.
- 07Acquisition mismatchExamines acquisition changes that can confound geometric inference.Examines acquisition changes that can confound geometric inference.Examines acquisition changes that can confound geometric inference, looking at why two X-rays may differ even when nothing has moved and what a recovery method needs to account for before it starts adjusting the object's pose.
- 08Spectra and detectorsExtends the model to spectra and detector response.Extends the model to spectra and detector response.Extends the model to spectra and detector response, moving beyond a single photon energy to consider how the source spectrum and the detector's sensitivity affect the predicted image and the parameters we hope to infer from it.
- 09Monte Carlo transportIntroduces scattering and stochastic transport estimators.Introduces scattering and stochastic transport estimators.Introduces scattering and stochastic transport estimators, following photons beyond the straight path from source to detector and considering how sampling choices, variance and computational cost enter the task of estimating an image.
- 10Transport gradientsTreats derivatives through the full transport process.Treats derivatives through the full transport process.Treats derivatives through the full transport process, considering what changes when paths are sampled, which dependencies must be included and how to assess correctness without mistaking sampling noise for a modelling or implementation error.
- 112D–3D registrationAligns a known volume with measured radiographs.Aligns a known volume with measured radiographs.Aligns a known volume with measured radiographs, extending pose recovery to calibrated acquisitions, multiple views and independent evaluation when the anatomy and image formation do not agree perfectly.
- 12Volume reconstructionRecovers a three-dimensional attenuation field from projections.Recovers a three-dimensional attenuation field from projections.Recovers a three-dimensional attenuation field from projections, developing volume optimisation and regularisation before examining sparse views, limited angles and material decomposition with spectral measurements.
- 13Acquisition designChooses measurements that improve registration or reconstruction.Chooses measurements that improve registration or reconstruction.Chooses measurements that improve registration or reconstruction, using task-specific uncertainty and error objectives to select viewpoints and allocate exposure within geometric and acquisition constraints.
- AX-ray physicsCollects the X-ray concepts required by the main text.Collects the X-ray concepts required by the main text.Collects the X-ray concepts required by the main text, keeping physical quantities, imaging terminology and useful conventions within reach for readers who know their way around code but have no reason to know their way around a radiology department.