面向机器人手术模拟器的神经辐射场软组织动态三维重建  

Dynamic Three-Dimensional Reconstruction of Soft Tissue in NeuralRadiation Field for Robotic Surgery Simulators

作　　者：陈琪 秦芝宝 蔡晓誉 李世杰 王梓俊 石俊生[1,2] 邰永航 Chen Qi;Qin Zhibao;Cai Xiaoyu;Li Shijie;Wang Zijun;Shi Junsheng;Tai Yonghang(School of Physics and Electronic Information,Yunnan Normal University,Kunming 650500,Yunnan,China;Yunnan Key Laboratory of Optoelectronic Information Technology,Kunming 650500,Yunnan,China)

机构地区：[1]云南师范大学物理与电子信息学院,云南昆明650500 [2]云南省光电信息技术重点实验室,云南昆明650500

出　　处：《光学学报》2024年第7期271-283,共13页Acta Optica Sinica

基　　金：国家自然科学基金(62365017,62062070,62005235)、云南省优秀青年基金(202301AW070001)。

摘　　要：构建了一种基于自监督的框架,该框架从单目立体内窥镜视频中提取多视图图像,利用图像中的底层三维(3D)信息构建对象的几何约束,实现软组织结构的准确重建。基于分割任意场景模型对内窥镜下的动态手术器械、静态腹腔场景及可形变软组织结构进行分割解耦。该框架利用简单的神经网络多层感知机来表示动态神经辐射场(NeRF)中运动手术器械和形变软组织结构,基于偏斜熵损失对手术场景中的手术器械、腔体场景和软组织结构进行正确分离。在通过使用单目立体内窥镜捕获机器人手术模拟器场景的数据集上,将所提方法的结果与其他方法进行定量定性比较。结果表明本文方法在处理腹腔体场景、软组织结构重建、手术器械的分割解耦,以及来自多视点的3D信息和运动对象的图像分割等方面显著优于当前的方法。Objective Reconstructing soft tissue structures based on the endoscope position with robotic surgery simulators plays an important role in robotic surgery simulator training.Traditional soft tissue structure reconstruction is mainly achieved through surface reconstruction algorithms using medical imaging data sets such as computed tomography and magnetic resonance imaging.These methods fail to reconstruct the color information of soft tissue models and are not suitable for complex surgical scenes.Therefore,we proposed a method based on neural radiation fields,combined it with classic volume rendering to segment robotic surgery simulator scenes from videos with deformable soft tissue captured by a monocular stereoscopic endoscope,and performed three-dimensional reconstruction of biological soft tissue structures to restore soft tissue.By using segmented arbitrary scene model(SASM)for segmentation modeling of time-varying objects and time-invariant objects in videos,specific dynamic occlusions in surgical scenes can be removed.Methods Inspired by recent advances in neural radiation fields,we first constructed a self-supervision-based framework that extracted multi-view images from monocular stereoscopic endoscopic videos and used the underlying 3D information in the images to construct geometric constraints of objects,so as to accurately reconstruct soft tissue structures.Then,the SASM was used to segment and decouple the dynamic surgical instruments,static abdominal scenes,and deformable soft tissue structures under the endoscope.In addition,this framework used a simple neural network multilayer perceptron(MLP)to represent moving surgical instruments and deformed soft tissue structures in dynamic neural radiation fields and proposed skew entropy loss to correctly predict surgical instruments,cavity scenes,and soft tissue structures in surgical scenes.Results and Discussions We employ MLP to represent robotic surgery simulator scenes in the neural radiation field to accommodate the inherent geometric complexity and

关 键 词：视觉光学 神经辐射场 软组织三维重建 分割任意场景模型 分割解耦 

分 类 号：O436[机械工程—光学工程]