The critical importance of audio for training simulations

Introduction
Let me start by asking you a question. When you are driving your car, what is one of the key indicators that something is wrong, that it may need a tune-up, or worse still, that it is about to break down? There is a reason why hearing is our critical danger and warning sense. It is always on (even when we are asleep) and it works in a 360-degree sphere, so we can hear things we cannot see. Also, our brain, through what we hear, recognises patterns very effectively so we notice a change in sound that may indicate a problem.

Yet even with how effective our sense of hearing is I have been surprised on many occasions by the number of training simulators, both military and industrial that overlook the importance of sound in their trainers. I have spoken to users of some of the most advanced (and expensive) military flight sims and been told over and over that the sound was poor quality, not very useful and not designed in a way that aids the user.

Virtual Reality
VR is rapidly proving to be an effective format for providing training for vehicle and machinery users. Being able to place a user INSIDE a virtual vehicle and have them operate it in a virtual world allows for realistic training and the virtual nature of the space ensures safety.
The audio capabilities for VR surpass almost any other format of media, and we are still discovering just how effective VR audio is at providing useful feedback and enhancing the realism of VR environments. We can completely surround the user in audio that responds to their actions and can be used to indicate events in the virtual world.
Moreover, well implemented VR audio can make a massive impact on the value of a trainer far in excess of its cost.  Need to simulate a critical machinery failure? This can be done almost entirely with sound, it will indicate an issue the user cannot even see and does not require expensive graphic representation in many cases.

So, lets have a look at a couple of scenarios for simulators and how well-designed audio could support them.
 
Industrial
For an industrial training simulator designed to train heavy machinery operation, the audio feedback provides information confirming nominal operation. Essentially if everything sounds good, then everything is likely working correctly. For equipment used for digging, audio feedback can inform the user of the types of materials being processed. A drill of any kind, from your cordless home drill right up to a machine rig will alter its sound as it operates. Strain or stress will alter the sound it makes and cutting through harder materials will sound different. In this way you can simulate not only, operational issues, but potentially reflect exactly what is being drilled based on the sound it makes.

For training in critical failure scenarios audio is probably more important than other aspects of the sim. Engine or equipment issues, heavy vehicle performance and emergency situations are almost always indicated through the sounds you hear before any other indicator. A Hydro station turbine performs under constant stress and shutdown of this type of equipment is time consuming and costly. Capturing the sound of this equipment when working optimally allows immediate and easy reference at any time. Is your equipment working properly? Listen to how it currently sounds, then compare it to the reference recording for optimal operation.

The “all around us” nature of sound should never be underestimated. Operating many large industrial machines has the constant inherent issue that the operator cannot see much of their environment or even most of the machine they are using. But if something goes wrong behind them they will often hear it, even if they do not see it. Many pieces of industrial equipment are huge and very noisy, as a consequence the operators must use hearing protection. But even with hearing protection the vibrations of potential issues can carry through to the operator. Many training simulators cannot provide haptic feedback (vibrations and movement) but again sound can provide this vibration information using low frequency sound output.
 
Military
When working on a VR infantry training simulator I was tasked with providing the sounds that matched the firearms being used by the troops using the simulator. So, I was told the priority was for the gun in the hand of the user to sound realistic. This was a good start to creating a useful trainer. However, when I asked about the sounds to use for the virtual AI opponents I was told just to use anything, the sound of an AK47 would be fine. I found this strange. My time in basic training many years ago reinforced to me that one of the most important steps in any engagement was to ascertain the biggest current threat, i.e., who was trying to kill you and how they were doing it.

If you talk to any combat soldier you will find that many of them learn very quickly to recognise the sounds of various enemy weapons. This can mean the difference between life and death in the seconds you have to respond in a combat situation. What you hear is likely the first warning you have of a possible threat, spending time looking around to judge the situation may often get you killed, so your hearing at that point in time is critical.
-Direction of enemy threat
-Nature of threat: single weapon being fired or group or weapons
-Type of weapon being fired: Is it automatic weapon fire, or a high calibre rifle? The difference between these two tells you the potential accuracy and penetration of the incoming fire and may well determine where you should seek cover.
-Distance: After a while you learn that the quality of the sound can inform you of how far away the sound is coming from
-Reflections: Echoes can provide even further information on where the threat is coming from as a sound will echo more when it travels through a built-up area such as building walls.
Even within military vehicle simulators the audio can indicate systems failure, potential enemy threats, types of threat, direction of threat etc. Knowing that here is a potential imminent engine failure in your fighter plane because you hear a change in its sounds a second before the warning indicator lights up might just save your life.
 
Vehicles
Engine performance, road surface, weather conditions, status of your load (freight moving around, or loose ropes and straps) are all extremely important to simulate during training to ensure operators gain experience dealing with dangerous situations. All of these examples are also scenarios where audio can inform the user of exactly what the issue is and often the how bad the situation is.
The main reason we train anyone in operating various vehicles is to ensure they can do so safely, for their own well being and for the safety of others. Providing situations to test a user’s response to potentially dangerous situations allows assessors to judge the operator’s suitability and equally important allows the user themselves to gain confidence in their ability to handle a range of scenarios and know that can respond in the best way.
 
Summary
Creating training simulators for military and industrial uses can be very expensive and it is important that these products are effective. Lives are quite literally dependant on people being trained properly. Good quality audio is important for the reasons stated in this article, but more importantly any amount spent on adding and improving audio feedback to a simulator will have a more significant impact on the effectiveness of the training than money spent on other aspects of the product.
You could simulate all aspects of training safety with audio alone and know that the users understood exactly what the issue was and would never miss the problem. This is not the case with information to our other senses. As a developer you could add a huge range of situations to your training simulator utilising only audio functionality which is often faster to implement than other aspects of development.

Utilising effective audio design for training is cost effective, impactful and most importantly improves the effectiveness of the all safety training across all industries.