Sparse and transferable three-dimensional dynamic vascular reconstruction for instantaneous diagnosis

Overview

The 3D geometry is crucial when a clinician grades a stenosis, measures a bifurcation angle, or steers a guidewire past a side branch. However, X-ray angiography only provides sparse 2D projections of the deforming vessels, each from a different angle at a different moment. In practice, clinicians have to infer the 3D geometry from these 2D projections, which depends on expertise. Additionally, the 3D geometry and its mapping to the 2D image are indispensable to the development of robotic endovascular systems.

The fundamental challenge arises from the temporal variability. More specifically, cardiorespiratory motion deforms the vessels throughout the acquisition. And X-ray angiography is performed one view at a time, which means that views acquired at different times do not correspond to the same 3D geometry, breaking the multi-view consistency. This variability, combined with the constraints of the imaging protocol, results in a severely ill-posed inverse problem: reconstructing a slender, temporally evolving 3D geometry from sparse and temporally misaligned 2D projections. In this problem setting, it is infeasible to exploit multi-view consistency or to obtain 3D ground truths for training on real-world data, while a synthetically trained model must carefully handle the sim-to-real gap.

In this context, we propose an automated dynamic cardiovascular reconstruction method called AutoCAR, which includes the following key designs.

1. Sparse backward projection. It leverages the sparsity of vascular structures by treating cardiac vessels as one-dimensional manifolds in 3D space, and ensures detail preservation by restricting computation and storage to those manifolds.
2. Pose domain adaptation. It utilizes the viewing preferences of human experts in XA imaging, and extends the domain adaptation technique from the image domain to the pose domain to ensure the generalizability of a synthetically trained model in real-world scenarios.
3. Vascular graph optimization. It leverages the curve-network characteristics of vessels for robust vascular topology and temporal coherence.

A stenosis in one view is consistent with many 3D shapes, and they disagree about the one quantity being graded — cross-sectional area. A second view does not hand the answer over either. Two questions have to be settled by eye first: where is the same site in view 2, and what 3D shape do the two views agree on?

A start and a target in one view leave the 3D topology open: several routes project to the same picture, and they call for different moves — up rather than forward. Resolving which one is what the second view is for, and it is the same correspondence problem as above, asked about topology instead of severity.

Each row is one viewing pose. The protocol records a video from a fixed viewpoint and only then moves the C-arm to the next — it is not one continuous sweep, so a frame in one view and a frame in another are different instants. ECG gating narrows that mismatch without removing it. Capturing views within a sixth of a cardiac cycle would need the C-arm to rotate at 1080 deg/s and record at 900 fps, against 60 fps on current systems. Extended Data Fig. 3.

Demos

The reconstructed centreline deforms in step with the two X-ray views it was recovered from, and 2D–2D and 2D–3D correspondences are colour coded.The same colours are baked into the reprojected stills and clips, so a vessel segment keeps its colour across the 3D view and both 2D panels. The 3D centreline, instead of surfaces, is shown because of the browser's limitations on dynamic surface meshes. As a compensation, static surface meshes at a single time step are shown underneath. Please find more cases (dynamic centreline), more cases (static surface), and comparison with existing works, and failure cases.