Engineering news

IBM augmented reality system lets engineers look through walls

Joseph Flaig

IBM master inventor Kevin Brown with the Hololens headset
IBM master inventor Kevin Brown with the Hololens headset

Turning my head to the left, two large white pipes come into view.

They are hidden behind a thick, opaque wall with a mounted telephone, but I can see their vertical path up from the floor, joining others before turning and crossing through the ceiling of the room.

I am looking at the pipes in augmented reality (AR), as part of a complex system involving everything from Internet of Things (IoT) sensors, 3D modelling, and even artificial intelligence (AI). Its inventors at IBM believe it could revolutionise training and efficiency in engineering and offer “end-to-end” support for building construction and maintenance.

The AR program runs on a Microsoft Hololens headset at IBM’s lab near Winchester. It is hugely customisable and could offer engineers many different types of information and feedback, said “master inventor” Kevin Brown.

"Highly accurate 3D models"

In the demonstration, the virtual pipes are seen running from a physical pump in the room. Although the pipes are not actually there at the testing lab, if commercially used the program would use highly-accurate 3D models of everything from a building’s architecture to electricity and plumbing.

The water flow rate hovers to the left of the pump, changing in real-time as it starts leaking. The information comes from a Raspberry Pi, which sends the information via IBM’s Watson “cogntitive computing” system. Users can see the pipes, showing where to work or make holes, and look at manuals for the pump.

The real-time data and instructions could ensure engineers have the right tools and training for a job, said Brown. “The mechanic fixing the car in your garage: would you rather the mechanic have access to the manual, or do it from memory? You would rather have some access to the data to fix the thing, rather than just letting them get on with it.”

Senior engineers could advise trainees using the headset “like they are standing next to you,” said Brown, and companies could even implement IBM’s Watson AI system. The program could be “fed” manuals for all the devices used by a company, letting users ask questions and receive vocal feedback.

"It all adds efficiency"

The system could be used by engineers across all industries and job roles, said Brown. “You face the same kinds of problems quite commonly across industries. With the IoT you can start managing more efficiently in real time. You can… be more efficient when allocating engineers.

“If you see roughly what is going on you can make sure they have the correct materials with them so they don’t have to make repeat visits. If they are wearing something like the Hololens they can get instructions so they don’t have to make multiple visits… it all adds efficiency.”

As the IoT becomes more widespread, technology like AR headsets will also be more widely used, said Brown. He said the sets could replace mobile phones as companies look beyond specialist use and focus more on everyday applications.

“I am sure in my lifetime we will probably be wearing glasses that augment our lives, rather than, say, mobile devices,” he said. “You start imagining walking into a building and getting directions to the first desk available from your personal preferences. All of these things start to become a reality.”

As well as augmented reality, some companies are designing virtual reality programs to assist engineers. Manchester-based Clicks and Links has created highly detailed models of a stretch of Crossrail tunnel under Liverpool Street in London, allowing engineers to “visit” from their offices.

“The main advantage of this is the ability to carry out preparatory work for an actual site visit, which can be costly, time consuming, potentially dangerous, and likely rushed,” said CEO Vin Sumner to PE. “Our platform grants engineers the ability to access the site virtually as many times as they like, take photographs, get a closer look, leave annotations for colleagues, or even access virtually with colleagues in a multi-user session.”


Professional Engineering magazine

Professional Engineering app

  • Industry features and content
  • Engineering and Institution news
  • News and features exclusive to app users

Download our Professional Engineering app

Professional Engineering newsletter

A weekly round-up of the most popular and topical stories featured on our website, so you won't miss anything

Subscribe to Professional Engineering newsletter

Opt into your industry sector newsletter

Related articles