By Ritchie Duncan
Augmentic Visualisations Ltd, United Kingdom
Cite as: Duncan, R. (2015), "The Potential Contribution of Augmented and Virtual Reality to the Oil and Gas Industry", International Journal of Management and Applied Research, Vol. 2, No. 3, pp. 112-120. https://doi.org/10.18646/2056.23.15-011 | Download PDF | Cited by
Abstract
This paper aims to evaluate the potential uses of Augmented Reality (AR) and Virtual Reality (VR) technologies for the key engineering activities in the oil and gas industry. Whilst these technologies are based on the same background framework, they both have a wide variety of uses, from creating realistic simulations of real world environments to recognising three-dimensional objects and overlaying a range of digital information.It then discusses the application of these technologies in addressing such long-standing industry challenges as effective training in preparing technical staff for work in harsh conditions in offshore environments.
1. The future…..today
The oil and gas industry is facing an unprecedented efficiency shortage with the added impact of accessing, finding and extracting the oil and gas becoming ever more technically difficult due to dwindling supplies. The result is an industry in which constantly evolving technology is being introduced to work alongside technology that is sometimes quite dated. This is represented by increasingly complex training and development challenges as well as an ever more difficult situation for the operators, technicians and engineers in meeting the maintenance and servicing challenges of the modern oil and gas industry.
One of the solutions advocated by Augmentic Visualisations (www.augmentic.co.uk) is the innovative use of 3D visualisations and computer games technology at relatively low cost. Imagine looking at a piece of unknown equipment and it being automatically recognised, with layers of information delivered in front of your eyes. Imagine a world with no more need for tech manuals and the updates that go with them, where a maintenance or installation engineer could receive training “on the job” with maximum efficiency still being delivered.
Imagine technology that has the ability to deliver immersive training, where the user interacts with the virtual world through the use of an intuitive control system. The replication of multiple real world scenarios with this technology will provide unique training situations for employees, where each experience differs from the last and provides the ability to easily evaluate the outcome.
That world is here today.
2. The difference between AR and VR
AR and VR are very similar technologies, but have one distinct difference. AR is a live view of a physical real world environment, whose elements are augmented with computer generated sensory input such as sound, animated graphics, or GPS led data. In comparison, VR immerses the user in an enclosed environment by replacing the real world with a simulated one (Slobounov et al., 2015).
There was a time that AR and VR headsets were the realms of science fiction. Recent technological advancements have allowed this to become reality. Various hardware solutions are now available for general release and are making an impact in the marketplace, where they are being used for a wide variety of purposes. High quality professional VR headsets such as the Oculus Rift and the HTC Vive are now becoming a widespread innovation in the visualisation and training fields.
Professional AR headsets are currently in development which will allow companies to revolutionise the way they manage their assets, and will be available early in the first quarter of2016.
AR lends itself perfectly to distributing complex information in a real world environment, whereas VR is an eye opening experience that lends itself perfectly to simulation where, instead of looking at a flat screen, the user is surrounded by a 3D computer generated environment. The user is able to move around in the virtual world and see it from different angles. Technology also now exists that allows the user to physically reach into it, grab it and reshape it.
3. Game Based Technology and What Lies Ahead
Game designer and Director of Game Research at the Institute For The Future, Jane McGonigal, argues that games can change the world for the better in her book “Reality is Broken: Why Games Make Us Better and How They Change the World” . Using theories from positive psychology, cognitive science, sociology, and philosophy, McGonigal (2012) connects how game playing can make us happier and more productive. Contrary to the popular misconception that playing games is an alternative to working, McGonigal describes game playing as hard work in which we voluntarily take on unnecessary obstacles. She uses a quote from Brian Sutton-Smith, a psychologist of play, as a starting point for what games can do. As Brian Sutton-Smith states: "The opposite of play isn't work. It's depression" (Sutton-Smith, 2001: 198).
Video games based technology has indeed become a powerful medium that the average person can embrace easily with a minimum amount of training. The interaction between man and machine has been highly developed in the last 40 years of mainstream gaming and technology. More recently, video game controllers have moved away from kinesthetic correspondence (e.g. keyboard and joystick interfaces) toward a more sophisticated control (Lee, 2011). The hand-held system of Nintendo Wii, for instance, introduced motion sensors which allowed players to control the motions of avatars within a game through their physical movement. In recent years, motion sensor systems such as Leap Motion and Microsoft Kinect, enable a totally hands-free interaction with the digital world.
In a similar vein, smartphones such as the iPhone rely on graphic displays, touch sensitive screens, and artificial intelligence applications (Feller, 2012). The advancement of mobile communications has seen a recent move to wearable technology. Apple has recently released their iWatch, which links to the users’ smart device to allow for a more innovative way of communicating (Frammer, 2015). In 2013 Google released AR eyewear with “Project Glass”, which was a type of wearable technology with an optical head-mounted display (Ackerman, 2013). It was developed with the mission of creating a computer system that can appear everywhere and anywhere, allowing a basic hands-free interaction and displaying information in a smartphone-like hands-free format, where wearers communicate with the internet by voice commands rather than typing.
While many AR prototype projects, such as Google Glass, were specifically focused on the consumer market, DAQRI have created the Smart Helmet (Kim, 2014). This fully independent, wearable technology features a highly innovative design with the industrial workplace in mind. DAQRI have concentrated their efforts on creating a truly usable ‘4D’ experience which features their industry leading sensor package, which has been merged with an intuitive, immersive, simple to use experience. Due to the infinite number of potential applications that are possible with technology such as this, the DAQRI system allows registered developers to produce bespoke software products. The ability to integrate with almost any external management system makes the Smart Helmet one of the most innovative production improvement tools of the 21st century with respect to training and asset management technology.
This system enables you to make work instructions intuitive by removing outdated paperwork procedures and provides a live feed direct to the user, capable of visualizing complex systems. The adaptable framework allows you to author flexible content that meets the needs and experience level of each member of theteam. The added value of the augmented visuals makes complex concepts and systems easier to understand and facilitate faster, more in depth learning. Company-wide updates to maintenance publications will be a thing of the past with technology such as this. WIFI connectivity creates the ability to update every user at the touch of a button, giving users access to the most up to date information every time the headset is turned on.
Today, VR and AR are poised to change the way we interact with the digital world. The ability of next generation AR and VR devices to track the skeletal movements of hands is already possible. This feature will allow people to take control of computers in ways that have never been possible until now. In the same way that the introduction of computers over 50 years ago was groundbreaking, its impacts are truly unknown. With new low-cost, high-quality VR headsets available to the general market at around the $350 mark, it could be argued that there will there be VR in every house, classroom, and office within the next 10 years. The recent acquisition of Oculus for $2bn by Facebook (Bercovici, 2014) suggests that this technology will be the must have add-on for any home system. However,Oculus themselves admit that they do not just see their technology being primarily used for gamesand aim to create fully immersive event coverage, where the user may pick their “place” at an offshore oil rig anywhere in the North Sea, while sittingat home in front their computer.
4. The many uses for Virtual and Augmented training
The oil and gas industry is prone to a high injury and fatality rate (Blackley, 2014). While real world experience is the preferred option, VR offers a way of reducing the cost of training while drastically reducing safety concerns. The ability for unskilled personnel to learn the skills in an off-site environment, without interfering with production or endangering very expensive machinery, is a valuable tool. While there are many benefits to digital off-site training, there are six main areas that can be improved by implementing this technology.
- Time and cost reduction
- Unlimited access to expensive and/or unavailable equipment
- Experience of hazardous areas or conditions
- No need for local training. Standard practice learning over a wide geographical area
- Reduce the need for specialists to carry out specific operations
- Integrating existing computer investments
The saying “A picture says a thousand words” is a very apt one for the AR and VR industry. AR gives the user the ability to see things in the real world that are invisible to the naked eye, whereas VR gives the ability to transport the user to a totally new world. This technology allows the user to gain a clear view and a better understanding of the process in hand, as well as any technical issues that may occur (Ginsburg, 2013).
With companies such as Petrofac delivering in excess of 250,000 man days training a year, it appears that the oil and gas industry requires a highly skilled and trained workforce. Lifelike training exercises, such as the “helicopter ditch” and “high jump” are an invaluable tool in providing a simulated experience as close to the real thing as possible. It can be argued that these are “one-off” experiences that the trainee will hopefully never have to come across but will remember clearly and be able to perform the drill when under extreme pressure. But what of the situations that occur daily that affect overall productivity? Would it not be more advantageous to train staff before they risk production targets with insufficient knowledge of the correct operational procedures? Of course, employment in the North Sea oil and gas industry alone is not limited to working offshore, and training must be carried out at all levels of the company.
While every care is taken to make the oil and gas industry as safe and as well trained as possible, it is by nature a hazardous field to operate in. Accidents still occur and are often fatal, so the questions must be asked. With calls from companies (such as Deloitte) for North Sea oil production to double in the next 10 years to maximise its potential, are the training methods that are employed currently robust enough for a fast growing industry and how will the industry cope with the massive influx of inexperienced staff?
Effective training is a cornerstone of preparing for employment in an offshore environment. It is a fairly easy correlation that the quality, consistency and frequency of training will significantly impact workplace readiness, which may then potentially affect other actors within the situation. The importance of such training is widely recognised but barriers such as time, cost and safety, limit the extent to which workers are exposed to hazards and be brought up to established standards.
Current “Virtual” training offers a unique realistic quality that is not generally present in classroom-based or web-based training. The immersive nature of this technology allows the user to transport themselves into the training environment and grasp the situation at a much deeper level, whereas AR may be classed as an informational addition to real life.
Digital solutions have the ability to aid with immersiveness and realism.
With the addition of interactivity, a wide variety of scenarios in trainingcan also be used to improve safety performance. The ability to reconstruct accidents / scenarios using video game technology allows for the replication of dangerous real-world scenarios in a totally safe environment. The added ability to replay the interactions of the user allows for a powerful tool in the assessment of trainee performance.
5. AR Adding to the Real World
Current mobile technology is fairly limited as far as serious AR applications may be applied. The ability to track two dimensional images or GPS based data to provide the overlay is a novel tool for marketers but is of limited use in the oil and gas industry. New technology, such as DAQRI’s Smart Helmet and META Spaceglasses, now have the ability to track three dimensional, real life objects to provide the overlay. This has lent itself extremely well to ideas such as “paperless manuals”, which show the maintenance of equipment in a clear, step by step manner. The ability to unobtrusively record employee activity whilst in both normal and hazardous operations will also be a valuable tool in improvements in aspects of health and safety with the added ability to replay the events after the fact to learn from the experience.
Hardware such as this has been created from the ground up with industrial applications in mind. Using technology such as a Virtual Inertial Information system allows the hardware to map the surrounding environment. The multi-user facility gives the ability to share information and create an entire model with the combined data to create a three-dimensional reconstruction of the facility. These ideas are currently unavailable on mainstream mobile technology.
Academic studies have proved that using three-dimensional training methods such as these have vastly improved training design and effectiveness (Gottschalk et al., 2015). With the beginner user in mind it has the ability to drastically reduce the number of errors the first time an employee performs a particular operation.
Modern hardware provides the ability to integrate seamlessly with external systems. This makes it possible to develop an asset management system that can integrate with existing computer investments. This system will use GPS data to ‘understand’ its surroundings and detect exactly what the user is looking at. Linkage with pre-existing control systems now enables users to adjust variables in their environment, such as pressure, temperatures and flow rates of equipment from the shop floor.
6. The Advantages of “Virtual” Training
Interactive disaster training can be tailored to specific users as well as companies. These designs may be based on their resources and hazard vulnerability analysis and can be developed for instructional task-focused training, in which the program responds to user inputs and provides instant feedback, such as performance of the task in hand. In addition, a “Virtual” exercise can also allow for the testing of action plans in order to assess their effectiveness and also identify gaps in knowledge and areas for improvement. Digital applications can also facilitate consistent training standards over geographical and organisational divides. Applications can provide a consistent synchronous localisation, the language and culture, and cross organisational training for both technical and natural occurrences in the workplace. Ideas such as this reduce the need for classroom based “Death by Powerpoint” and lightly loaded infographics that supply limited information. All these ideas may be readily embedded within the digital environment and made accessible in novel ways for trainees.
Virtual and Augmented-based training holds major advantages over conventional forms. From an environmental perspective, VR-based applications, created using programs such as Unreal Engine 4, not only have the ability to incorporate lifelike scenarios with realistic graphics but programmable Artificial Intelligence also allows reaction to user input and provides instant feedback.
Unlike a classroom environment, digital training applications also allow trainees to work at their own pace, thus providing the opportunity to fully grasp concepts through asynchronous learning. Participants may now fully interact with a virtual environment. Components such as control rooms, equipment, transportation options and other ideas such as disaster response may all be incorporated into a single solution. From a cost perspective, software-based disaster training has significant advantages over traditional methods. In an era of ever-tighter fiscal constraints, the available resources and funding to support oil &gas training has become increasingly restricted, highlighting the need for effective, cost-conscious solutions. VR-based disaster training offers a practical alternative that incorporates graphic realism and immersiveness at a fraction of the cost of real life exercise. This has meant that training scenarios can be practiced more frequently under different, varying conditions to either challenge responders or to establish better understanding of factors that may lead to alternate outcomes.
It could be argued that the greatest benefit of AR / VR training scenarios and exercise play is that it can be stored digitally. This allows an evaluation to be carried out more effectively with the ability to view individual elements of the exercise. The ability of these platforms to support data and video capture of time and critical action elements is an invaluable tool for the analysis of the individual and the training provider itself. This can be used to more accurately gather lessons learned and develop corrective actions necessary for the after-action review process.
7. Conclusion
The emergence of Virtual and Augmented-based technologies can now be applied to an infinite number of uses in the oil and gas industry. These methods offer significant potential advantages over other traditional forms of training and productivity improvement methods. Technology such as this is currently gaining increasing acceptance in forward-thinking industries as a valuable tool for process improvement. This can be seen to be due to the immersive and participatory nature of the experience.
The ability to create bespoke virtual training at a relatively low cost offers an alternative way of disseminating complicated or routine information. This unique realistic functionality that is not generally present in classroom or web based forms, provides considerable cost and effectiveness advantages over real life exercises. High tech solutions now allow the integration of hardware such as DAQRI’s Smart Helmet to allow for an “on the floor” control room. The ability to adjust systems “virtually”, while users “live in the field” will ultimately be a time- and cost-saving feature in developing training for the current and future needs of the oil and gas industry.
8. References
- Ackerman, E. (2013), “Google gets in your face: Google Glass offers a slightly augmented version of reality”, IEEE Spectrum, Vol. 50, No. 1, pp. 26-29.
- Bercovici, J. (2014), “Move Over, Glass: With Oculus Acquisition, Facebook Out-Googles Google”, Forbes, available from: https://www.forbes.com/sites/jeffbercovici/2014/03/25/move-over-glass-with-oculus-acquisition-facebook-out-googles-google/ (Accessed on 7 April 2014).
- Blackley, D. J., Retzer, K. D., Hubler, W. G., Hill, R. D. and Laney, A. S. (2014), “Injury rates on new and old technology oil and gas rigs operated by the largest United States onshore drilling contractor”, American Journal Of Industrial Medicine, Vol. 57, No. 10, pp. 1188-1192. https://doi.org//10.1002/ajim.22356
- Fellet, M. (2012), “Touch and go”, New Scientist, Vol. 214, No. 2868, pp. 40-43. https://doi.org//10.1016/S0262-4079(12)61502-4
- Frammer, D. (2015), “2015 is the year of the Apple Watch”, Quartz, available from: https://qz.com/319703/2015-is-the-year-of-the-apple-watch/ (Accessed on 7 April 2015).
- Ginsburg, B. (2013), “FuelFX Brings Revolutionary AR Mobile Applications to Energy and High-Tech Industries for Training and Marketing”, Business Wire, available from: https://www.businesswire.com/news/home/20130716005991/en/FuelFX-Brings-Revolutionary-Augmented-Reality-Mobile-Applications (accessed on 7 April 2014).
- Gottschalk, M., Yoon, S., Rhee, J., Park, D. and Mitchell, P. (2015), “Surgical training using three-dimensional simulation in placement of cervical lateral mass screws: A blinded randomized control trial”, Spine Journal, Vol. 15, No. 1, pp. 168-175. https://doi.org//10.1016/j.spinee.2014.08.444
- Kim, M. (2014), “DAQRI Unveils 4D Smart Helmet for Industrial Workforce”, Business Wire, available from: https://www.businesswire.com/news/home/20140908006433/en/DAQRI-Unveils-4D-Smart-Helmet-Industrial-Workforce (Accessed on 7 April 2014).
- Lee, C., Liu, C., Chen, Y. and Chen, Y. (2011), “Painting in the air with Wii Remote”, Expert Systems With Applications, Vol. 38, No. 12, pp. 14668-14678. https://doi.org//10.1016/j.eswa.2011.05.016
- McGonigal, J. (2012), Reality is Broken: Why Games Make Us Better and How They Change the World, London: Vintage.
- Slobounov, S. M., Ray, W., Johnson, B., Slobounov, E. and Newell, K. M. (2015), “Modulation of cortical activity in 2D versus 3D VR environments: An EEG study”, International Journal Of Psychophysiology, Vol. 95, No. 3, pp. 254-260. https://doi.org//10.1016/j.ijpsycho.2014.11.003
- Sutton-smith, B. (2001), The Ambiguity of Play, USA: Harvard University Press.
