Cookies help us deliver the best experience on our website. By using our website, you agree to our use of cookies Dismiss

One billion points streamed in augmented reality

Visualisation of a massive point cloud model (1.1+ Billion points) merged with BWTS CAD design data. It is streamed to a HoloLens using ISAR SDK Visualisation of a massive point cloud model (1.1+ Billion points) merged with BWTS CAD design data. It is streamed to a HoloLens using ISAR SDK

ShipReality and Holo-Light have showcased the first holographic visualisation of ultra large ship 3D laser scans merged with complex CAD design data, using the most advanced XR streaming technology available.

Ships regularly undergo large-scale retrofits, but shipowners rarely have design data in digital form at hand. To design ship modifications, an engineering accuracy of the as-built ship geometry is required, which means each vessel must be 3D laser-scanned. ShipReality, a company specialised in AR/VR ship design automation and remote ops, synthesises these large ship laser scans with its CAD software to design directly in 3D, resulting in merged models of CAD in the as-built ship geometry point clouds.

“We want to speed up and optimise retrofit designs for 60,000 ships that require greenhouse gas emissions reduction, energy conversions & ballast water treatment system (BWTS) retrofits in the coming years”, said Georgios Bourtzos, CEO and co-founder of ShipReality. “A major challenge we faced designing directly in large point clouds was visualising entire vessels layered with resulting 3D designs for immersive design reviews on mobile XR devices like Oculus Quest 2 and HoloLens 2.”

Exploring the use of point clouds in AR

Point clouds are precise models of real environments based on 3D laser or photogrammetry scanning. Objects and space are represented in the form of “points”. Millions of such points combined formulate a point cloud scan. The scan is then imported into a 3D modeling platform with the purpose of creating an as-built model. Common CAD software used for ship designs, although incorporating 3D laser scans, still rely on 2D projections and screens to visualise and design in 3D. This often results in incompatibilities with the existing ship geometry, which are only realised during installation, creating substantial delays and high additional costs.

Visualisation in 3D is a key issue to address these problems. However, visualising large point clouds requires substantial CPU and graphics power. The performance requirements are simply too high for rendering it locally on a mobile XR device. It would cause an extremely low frame rate and even software crashes. Dealing with large datasets, ShipReality had to find a solution that could surpass the limited memory, CPU and GPU resources of mobile devices.

XR streaming solution that supports point cloud

XR streaming technology outsources the rendering process to a local server or the cloud. But not every solution on the market supports ultra large data or point clouds. We integrated the ISAR SDK into our solution to stream large 3D laser scans merged with CAD retrofit designs to a HoloLens 2,” said Mr Bourtzos. ISAR (Interactive Streaming for Augmented Reality) is a unique remote rendering software component that allows to stream entire augmented and virtual reality applications in real time. “The simplicity and ease of integration of the software development kit worked seamlessly with our large models.”

Integrating ISAR into their engine and software, ShipReality was able to visualise a massive model containing more than one billion points. Layered on top was also BWTS CAD design data created by the company’s ShipMR-design software with additional five million polygons. To compare, a mobile XR device could only render about one and half million polygons locally. As the remote rendering server ShipReality used a moderate gaming laptop and the local WiFi, broadcasting on 2.4GHz band. More performance and bandwidth enable even greater visualisations.

Next level immersive experiences

“ISAR has amazing potential for AR/VR visualisation of massive digital twins and real-time monitoring of projected complex 3D designs merged with as-built environments in shipping and other industries,” said Mr Bourtzos. ShipReality is now able to:

  • visualise ship models that are only suited for high-performance processing
  • capture large assets 1:1 and integrate complex 3D CAD designs/data
  • visualise detailed models for spatial analytics in augmented reality

“ISAR can save us a lot of time and resources because we can directly use point clouds in mobile AR: some pre-processing steps can be avoided.”

Data availability, accuracy, density, and size of 3D point clouds are also forecast to vastly increase within the next years. “To realise the full potential of immersive point cloud experiences, streaming will play a major role,” concluded Georgios Bourtzos.

Find out more here 

Related items

  • MOL upgrades AR navigation system

    Mitsui O.S.K. Lines (MOL) has announced an upgrade to its Augmented Reality (AR) navigation system to further support vessel navigation and improve safety at sea. 

  • Augmented Reality supports Subsea 7 operations

    Using an augmented reality HOLO Lens, Jumbo has completed its first of two campaigns to support Subsea 7’s EPIC projects for Total in Angola: CLOV MPP and Zinia Phase 2.

  • ClassNK unfolds Digital Grand Design for sustainability

    ClassNK recently unveiled its Digital Grand Design to express the pivotal role it expects digital technology to play not only in making the industry more efficient but also in meeting United Nations’ Sustainable Development Goals (SDGs) for a ‘better and more sustainable future for all’ by 2030. Last month, the Society followed through with the first significant step on this 10-year journey, after launching ‘Innovation Endorsement’ as a new certification service for pioneering digital solutions.

    Endorsing innovation

    Digital Grand Design outlines ClassNK’s vision for the blue economy of the future, establishing a roadmap for achieving that vision based on three fundamental policies: improving safety and efficiency by providing advanced classification services, diversifying classification services and expanding their scope, and supporting the realisation of social innovation.

    Aligning with these principles, Innovation Endorsement has been devised to facilitate and encourage promising innovations that utilise data from ships to enhance safety at sea, help preserve the marine environment and promote sustainable development.

    Changing tides

    During its 120-year history, ClassNK’s commitment to ensuring maritime safety and protecting the marine environment has focused on working with traditional key stakeholders such as shipbuilders, equipment manufacturers and vessel operators.

    However, the digital revolution sees the arrival of new players leveraging the data collected on board vessels, as well as fresh collaborations with existing stakeholders to create new value for shipping companies and others in the maritime supply chain. By enabling rapid and accurate performance monitoring and benchmarking, data sharing will dovetail with the emergence of new business models resulting from a shift to more sustainable practices as set out in the UN’s SDGs.

    Some outcomes are already clear. Firstly, data sharing has implications not only for shipyards and shipping lines but for the cargo owners and forwarders relishing the opportunity for greater transparency and accountability. It will also offer a platform for new stakeholders such as digital forwarders and system integrators.

    For ClassNK, the aim is to adjust, in order to serve the needs of a greater variety of players. The resulting response to emerging digitalisation involves a variety of areas like data assurance; technology assurance; anticipating regulatory hurdles; and evaluation.

    Data assurance

    Using data to make better decisions is the fundamental premise of digitalisation. Data collection is already standard practice, with an increasing number of operators investing in upgraded connectivity to send information ashore, either to technical departments, to equipment manufacturers, or to applications providers.

    To succeed, data-collecting systems must be reliable and the data itself needs to meet certain standards – and this relies on an impartial arbiter of data quality. As an independent third party with extensive engineering expertise, ClassNK is ideally positioned to fulfill this role.

    Technology assurance

    The maritime industry’s conservatism when faced with new technologies is especially apparent when it comes to digital innovations, which tend to develop at a fast pace. With new functionalities becoming available in quick succession and a shortage of early adopters, there is little scope for vessel operators to learn from the experience of others – the sector’s preferred approach to technology adoption.

    Here, too, ClassNK can exploit its impartial status by collaborating with providers of innovative technologies to assess and vouch for the quality and function of their solutions. This would increase confidence among shipping companies who may otherwise be wary of introducing digital innovations and help the sector more generally to overcome any misgivings.

    Anticipating regulatory hurdles

    Whenever new technology or approaches are proposed it is imperative that they are verified as fit-for-purpose – especially when safety is at stake. Difficulties arise when a novel solution falls outside existing regulatory frameworks or requires an altogether new testing regime, as this may significantly extend the approval process.

    To accelerate the implementation of new technology, ClassNK is leveraging in-house expertise and links with standards organisations and maritime authorities to establish evaluation methods and contribute to the development of regulations ahead of time.

    ClassNK digital design 2 sep 30 Copy


    Lastly, there is a need for accurate evaluation founded on engineering principles. To date, ClassNK has inspected and issued approvals for vessels based on its rules for steel ships and related instruments. However, a wider focus is necessary to cope with the assessment needs of new business models and unorthodox collaborations between a broader array of stakeholders.

    ClassNK sees considerable potential in applying the substantial technical knowhow it has accumulated for quantitatively assessing ships and their systems to meet these emerging needs.

    Charting a new course

    To exploit the opportunities described above and respond to the demands that the digital revolution imposes on the maritime industry, ClassNK plans to work closely with the maritime players developing and pioneering the use of new solutions. Establishing a framework for utilising the knowledge gained from this collaboration will accelerate the wider deployment of these technologies across the industry.

    Digital Grand Design therefore marks the beginning of the next phase in ClassNK’s continual evolution, as it steers the industry into uncharted territory shaped by digitalisation and mounting pressure to minimise environmental impact.

    In the digital era, ClassNK is also committed to ensuring that the rules that govern shipping do not obstruct innovation unnecessarily and, where benefits are clear, that they are adjusted to support it.

    This article was originally published on VPO Global.

  • Digitalisation in the maritime industry

    The Digital evolution is shaping the Maritime future in every aspect and all its segments, taking advantage of modern ships design and new technologies, writes Giampiero Soncini, CEO of IB Marine Division.

  • MOL VLCCs to use AR navigation systems

    Mitsui O.S.K. Lines (MOL) intends to install augmented reality (AR) navigation systems onboard 21 MOL Group operated very large crude oil carriers (VLCCs).

Joomla SEF URLs by Artio


Register or Login to view even more of our content. Basic registration is free.

Register now

Digital Ship magazine provides the latest information about maritime satellite communications technology, software systems, navigation technology, computer networks, data management and TMSA. It is published ten times a year.


Digital Ship Ltd
Digital Ship - Digital Energy Journal
39-41 North Road
N7 9DP
United Kingdom

Copyright © 2020 Digital Ship Ltd. All rights reserved           Cookie Policy         Privacy Policy