A Real-Time Stitching Method for Dual-View Street-View Images Based on a TensorRT Hybrid Inference Architecture
A Real-Time Stitching Method for Dual-View Street-View Images Based on a TensorRT Hybrid Inference Architecture
Abstract—Deep image stitching networks deliver strong alignment quality, yet their inference latency keeps them below the frame rate that dual-view street-view applications demand. This paper reports a segmented hybrid inference architecture built on TensorRT. The partition criterion is the numerical sensitivity of each sub-graph: compute-bound but numerically stable modules are handed to TensorRT, while sensitive modules stay in PyTorch. Using the UDIS2 unsupervised deep image stitching algorithm as its base, the architecture re-cuts the two-stage pipeline of the Warp geometric alignment network and the Composition fusion network into a compute-bound segment and a sensitivity-critical segment that run on different backends. The ResNet-50 backbone and the correlation computation module are exported as a TensorRT FP16 engine, which lets the half-precision units of the GPU Tensor Cores carry the bulk of the arithmetic. Direct linear transformation solving, homography warping and thin-plate spline interpolation remain in the PyTorch FP32 environment, so that matrix inversion never runs under a low-precision floating-point representation. The Composition fusion network is deployed as a separate TensorRT FP16 engine. On an NVIDIA RTX 4060 Laptop GPU the end-to-end latency falls from 213.9 ms for the original PyTorch pipeline to 39.7 ms, giving a frame rate of 25.2 FPS and a speedup of about 5.4. Fidelity at the Composition output reaches 51.3 dB peak signal-to-noise ratio, and a pixel-level audit of the final panorama places 99.88 percent of the canvas within one grey level of the gold-standard render. Speed and image quality are therefore obtained together rather than traded against each other.
Keywords-Deep Learning; Image Stitching; Tensorrt Acceleration; Mixed-Precision Inference; UDIS2; Real-Time Vision System


