Assisted phase and step annotation for surgical videos - Archive ouverte HAL Access content directly
Journal Articles International Journal of Computer Assisted Radiology and Surgery Year : 2020

Assisted phase and step annotation for surgical videos


Purpose - Annotation of surgical videos is a time-consuming task which requires specific knowledge. In this paper, we present and evaluate a deep learning-based method that includes pre-annotation of the phases and steps in surgical videos and user assistance in the annotation process. Methods - We propose a classification function that automatically detects errors and infers temporal coherence in predictions made by a convolutional neural network. First, we trained three different architectures of neural networks to assess the method on two surgical procedures: cholecystectomy and cataract surgery. The proposed method was then implemented in an annotation software to test its ability to assist surgical video annotation. A user study was conducted to validate our approach, in which participants had to annotate the phases and the steps of a cataract surgery video. The annotation and the completion time were recorded. Results - The participants who used the assistance system were 7% more accurate on the step annotation and 10 min faster than the participants who used the manual system. The results of the questionnaire showed that the assistance system did not disturb the participants and did not complicate the task. Conclusion - The annotation process is a difficult and time-consuming task essential to train deep learning algorithms. In this publication, we propose a method to assist the annotation of surgical workflows which was validated through a user study. The proposed assistance system significantly improved annotation duration and accuracy.
Not file

Dates and versions

hal-02498462 , version 1 (04-03-2020)



Gurvan Lecuyer, Martin Ragot, Nicolas Martin, Laurent Launay, Pierre Jannin. Assisted phase and step annotation for surgical videos. International Journal of Computer Assisted Radiology and Surgery, 2020, 15 (4), pp.673-680. ⟨10.1007/s11548-019-02108-8⟩. ⟨hal-02498462⟩
63 View
0 Download



Gmail Facebook Twitter LinkedIn More