N-Sea and University of Strathclyde Collaborate for Practical Implementation of Deep Learning for Subsea Video Annotation

View all news from:
View directory entry for:

19 December 2017

Integrated subsea service provider, N-Sea, and the University of Strathclyde’s Institute of Sensors Signals and Communications at the Department of Electronic and Electrical Engineering, have developed a deep learning video annotation model for automatic eventing of subsea survey video.

The project is supported by The Data Lab Innovation Centre with the collaboration by combining the state-of-the-art inspection expertise of N-Sea and the innovative data analytics research at the University of Strathclyde.

The algorithm developed by the project annotates video frames from three general-purpose ROV cameras, automatically and in real-time. Whilst time consuming, the task of identifying events in the video footage is relatively straight forward for an expert operator. However, results are frequently affected by sand agitation, sea-life or vegetation. The approach taken by the project team utilises ensembles that combine the output of three classifiers operating on the three video streams, port, starboard and centre independently.

David Murray, Survey and Inspection Data Centre Manager at N-Sea, said: "Recently a number of automatic video annotation approaches have been announced. However, these have been demonstrated in clear waters, using bespoke and vendor specific camera systems that mitigate motion blur and poor image quality, through strobed lighting and high shutter speeds.

"Although these technological advancements in the equipment are beneficial, the vast majority of work class ROVs are still equipped with standard cameras operating in murky waters.”

To address this, the project has developed a 24-layer convolutional neural network to identify features in the video frames. It currently supports a number of events such as burial, exposure and field joints with high classification accuracy on still images. Combining predictions on a number of consecutive frames boosts further the network performance.

Commenting on the benefits this project will bring to the subsea sector, Dr Christos Tachtatzis, Lecturer and Chancellor's Fellow at the University of Strathclyde, said: "Prior state-of-the-art approaches demand high picture clarity and high visibility to operate effectively. We have purposely trained and validated the model using video footage typically acquired by ROVs, making the model applicable to the wider subsea survey community.”

The partners are currently in the process of establishing a follow-on project to increase the Technology Readiness Level of the model and permit the easy adoption by the inspection industry.

Gillian Docherty, The Data Lab CEO, added, "The Data Lab seeks to empower Scottish businesses and individuals to harness the full potential of data. Collaborative innovation projects are a key component of our work, and we were more than happy to partner with N-Sea and the University of Strathclyde to achieve their goal.

"Through the use of data science, we have worked together to develop an algorithm that will be the catalyst for a step-change in how the industry approaches sub-sea inspection in a challenging environment, a fantastic result. Scotland is certainly seizing the data opportunity and it is collaborations such as this which bolster our international reputation for excellence in data science skills and innovation.”

N-Sea is known for its innovative work as an independent offshore subsea contractor, specialising in IMR services for the oil and gas, renewable and telecom/utility industries, as well as for civil contracting communities. N-Sea provides near shore, offshore and survey services to major operators and service companies alike.