Crédits photos : DALL·E
XR, or Extended Reality, is expected to become the fourth computing platform revolution after PCs, the Internet, and mobile phones. Enabling the implementation of disruptive use cases that other technologies simply cannot, it is already beginning to impact society across the board, changing the way people consume and interact with information. But many are wondering if XR has the potential to scale. The more money that flows into the sector, the more tools that are created to help new people enter the fold. Even if you have been sleeping under the proverbial “rock”, you must be aware of low-code applications where anyone can pick up and make basic applications after a little self-teaching. Tools like Unity, ZapWorks, and Blippar offer a range of ways for developers to get started, with a wealth of materials and courses to kick-start the process. With that ease, comes light barriers of entry for anyone to access, hence technology democratization.
From video games to metaverse
When we talk about metaverses, or more generally “digitally rendered worlds”, we are referring to a network of virtual and three-dimensional spaces that allow you to immerse yourself in a computer-generated world. Through a synthetic alter ego, an avatar, it allows social interaction by overcoming the physicality of the human body. Social interaction is easier because the obstacles of distance are overcome by physics present in the real world and the limits of a simple social app. The metaverse, often considered the evolution of the internet, is also a potential source of significant earnings and for this reason, there are more and more companies interested in transforming the idea into a real thing, easily accessible and sellable.
The most common attitude in professional organizations operating outside the immersive technology perimeter thinks of metaverses as something relegated to video games or technology enthusiasts, tech geeks, and perhaps the entertainment industry. We have already passed that point, as some industrial software starts to make an appearance in the space, and the line between “industry” and frivolous cyberspaces of entertainment becomes more and more blurred.
Just to be clear, it is a shared view that a metaverse is not only a digital creation, but it in widest meaning is any reality created by humans to fulfill that human drive to create and to shape reality according to one’s own needs and expectations. Not all metaverses are accessible through a headset or other contraptions, some use blockchain technology, and some use both, we are far from convergence at this stage.
What is a systems engineer?
An engineer is a person that has a degree from an engineering school. If you believe that, I will endeavor to expand your views somewhat. It may seem pointless to define what the authors mean by engineering, but it is essential to have a moment of alignment on this key point. An engineer is whoever will use ingenuity to create something, or in our specific domain, someone who would use technology to create value, to add to the substance of the world, and benefit mankind. Similarly, systems engineering is a way of thinking, not a badge to obtain by passing an exam. Everyone asking themselves the right questions is a systems engineer.
More precisely a systems engineer can manage complexity, harnessing many tools, some specific to the systems engineering discipline, some borrowed from other domains like project management, risk management, stakeholder mapping, functional modeling, and many more. What is a system? Everything is a system or is part of a system, from the device you are using to read this article, to the system of government we chose to the solar system where our planet spins through the cosmos.
In this article, we shall also be talking about the system lifecycle, as the human lifecycle include pregnancy, infancy, toddler years, childhood, puberty, adolescence, adulthood, middle age, and senior years; the technological system will have the planning stage, feasibility or analysis stage, design and prototyping stage, development stage, testing stage, implementation and integration, operations and maintenance stage, disposal stage.
A short history of XR, a systems engineer perspective
It could be argued that to incorporate technology into your product and to avail of its full potential, an in-depth knowledge of the subject would be absolutely essential. The counterargument to that is that a systems engineer, oftentimes, plays the part of an integrator of a “system”, or the producer of an immersive experience. Therefore, the reader will be dispensed with how the term “metaverse” was first mentioned in Neal Stephenson’s 1992 science fiction novel Snow Crash, where humans, as programmable avatars, interact with each other and software agents, in a three-dimensional virtual space or that one of the first consumer VR devices was the Nintendo VirtualBoy™, a simple gimmick with some promise.
Yes, some amusing anecdotes can be drawn from personal experience. I am one of the last millennials, so my first experience of immersion, and I was one of the lucky ones, were some bulky machines installed at my town’s amusement park as a temporary attraction. The system was absolutely cumbersome and involved carrying a small TV on your face, resolution was extremely poor and experiences were limited to a few video games. Yet it was enough to spark the imagination and to leave the doors open for further connection to the technology later in life, the nostalgia factor if you will.
Now moving to 2015, after the failure of the ambitious Google Glass, some early Oculus devices hit the labs of engineering firms where they came in contact with some curious enterprising employees like me, eager to push the technology to beat the competition. Some fortunate engineers have had the opportunity to try the Microsoft HoloLens, out of reach for most people, and from that point on the words “scaling” and “industrialization” came up as a real possibility. It was the onset of the 4th industrial revolution. Together with quantum computing, 3D printing, the Internet of Things, artificial intelligence, and many others, early VR took its place on the shelf of promising technologies for the decade to come.
Managing complexity: the art of modeling
Modeling is nothing else than a way to manage the complexity of the system at hand, it is an abstraction or simplification of the system. There are many types of models, abstract models, analytical models, and physical models. For enthusiasts of XR, the first model that comes to mind is probably 3D models or digital assets used in the construction of immersive experiences. Extended reality can play a big role in developing a truly model-centric approach to engineering. At this point a couple of definitions are in order:
- Digital Engineering: An integrated digital approach that uses authoritative sources of systems’ data and models as a continuum across disciplines to support life cycle activities from concept through disposal. (DAU Glossary) (Defense Acquisition Guidebook)
- Model-Based Systems Engineering (MBSE): Execution of Discipline (Systems Engineering) using digital model principles for system-level modeling and simulation of physical and operational behavior throughout the system lifecycle (INCOSE)
The core application of the MBSE approach implies the use of a single system model that serves as the “source of truth” for all embedded system analysis tools. A system model can therefore interact with a virtual reality environment to analyze system performance. Systems engineers can run interactive, immersive simulations of scenarios described in system models, and feed results from virtual reality environments back into system models. By utilizing the VR environment, the analysis of a system can be performed more efficiently by involving the customer in the product life cycle from an early stage. Customers can step into a VR environment to see what the system design looks like and interact with it. Based on the results of the interaction, system engineers can change product/service specifications and designs.
XR4MBSE and MBSE4XR methods: what are the differences?
The terminology XR4MBSE and MBSE4XR is an extrapolation of the popular reference, among systems engineers, to AI4SE and SE4AI, used for artificial intelligence. This refers to the pairing of two disciplines, where the first is users in support of the latter and vice versa. In this case, those acronyms are intended to convey the following.
XR4MBSE, is the use of extended reality technology to enhance (model-based) systems engineering processes, for instance, the virtual validation of a system, namely proving that the system does work, even before its construction (e.g. that is how airplanes are certified). In the illustration below, some examples of areas where XR technology could be beneficial to the creation and operation of different systems, are presented.
MBSE4XR, the use of MBSE to enhance XR technology. This aspect is a bit more obvious since the use of MBSE to develop XR-heavy systems is similar to the development of any other system. Yet the use of “systems” thinking to enhance the design of immersive experiences and immersion hardware, while not unique, is definitely beneficial to the reduction of costly errors that are expensive to fix later in the life of the system. This is the quintessential and most basic benefit of systems engineering.
For instance, in the illustration, you can see a proposal of the lifecycle of an XR system, for instance, and “Augmented reality ski goggles”. The need for such as system will have to be carefully analyzed so that a proper meaningful user story and business case can be captured, even before the proposal is made to prospective investors.
Systems engineering: clarification and definition
At this point, it is worthwhile to understand whether the definition of a model, its simulation, and virtual representation saves us time and costs during the product life cycle. Therefore, MBSE is an emerging approach in the field of SE and can be described as the formalized application of modeling principles, methods, languages, and tools to the entire life cycle of complex, interdisciplinary socio-technical systems. The simplified definition of MBSE, provided by Mellor as “… is simply the concept that we can build a model of a system that we can transform into a real system” helps us carry MBSE into the virtual reality where we need to manage complex systems or systems of systems.
The essence of MBSE is based on the application of models that can be applied to a system or even to systems that already exist in reality, in which boundaries have not yet been demarcated that relate both to technical aspects, such as interfaces, integration, and testing and to managerial aspects, such as governance. Therefore, systems engineering does not just focus on the system itself (the “ski goggles”), but also on the boundaries and interactions between seemingly independent and evolving systems.
From novelty to essential techno brick
Companies that successfully implement XR have a clear business problem they are trying to solve. This statement seems obvious but it is surprising how often companies and individuals, even experienced innovation managers, become excited by new hardware and software and will happily purchase to test and play. Sadly though these trials come without a real plan or lifecycle for the integration of these devices in the company “system”.
It’s critical that careful consideration and analysis of problems and potential solutions take place before purchasing XR solutions. To help understand the types of problems that can be solved, there are a number of well-established, industry-understood, use cases that have been delivered by various companies across the globe.
A word of warning, real-world business problems do not fit into clear-cut use cases. In reality, it is likely that various use case elements will need to be combined to solve a solution that’s unique to your company. However, having an understanding of the opportunities and problems that can be solved, helps companies to focus their projects, create clear features and requirements and ensure that the implementation of XR is successful. The use of principles of system thinking and model-based methodologies will go a long way toward solving “real” business problems. This will provide tangible benefits and return on investment that is crucial to ensuring future funding and support for your XR project.
