 Introduction Let me start by asking you a question. When you are driving your car, what is one of the key indicators that something is wrong, that it may need a tune-up, or worse still, that it is about to break down? There is a reason why hearing is our critical danger and warning sense. It is always on (even when we are asleep) and it works in a 360-degree sphere, so we can hear things we cannot see. Also, our brain, through what we hear, recognises patterns very effectively so we notice a change in sound that may indicate a problem. Yet even with how effective our sense of hearing is I have been surprised on many occasions by the number of training simulators, both military and industrial that overlook the importance of sound in their trainers. I have spoken to users of some of the most advanced (and expensive) military flight sims and been told over and over that the sound was poor quality, not very useful and not designed in a way that aids the user. Virtual Reality VR is rapidly proving to be an effective format for providing training for vehicle and machinery users. Being able to place a user INSIDE a virtual vehicle and have them operate it in a virtual world allows for realistic training and the virtual nature of the space ensures safety. The audio capabilities for VR surpass almost any other format of media, and we are still discovering just how effective VR audio is at providing useful feedback and enhancing the realism of VR environments. We can completely surround the user in audio that responds to their actions and can be used to indicate events in the virtual world. Moreover, well implemented VR audio can make a massive impact on the value of a trainer far in excess of its cost. Need to simulate a critical machinery failure? This can be done almost entirely with sound, it will indicate an issue the user cannot even see and does not require expensive graphic representation in many cases. So, lets have a look at a couple of scenarios for simulators and how well-designed audio could support them. Industrial For an industrial training simulator designed to train heavy machinery operation, the audio feedback provides information confirming nominal operation. Essentially if everything sounds good, then everything is likely working correctly. For equipment used for digging, audio feedback can inform the user of the types of materials being processed. A drill of any kind, from your cordless home drill right up to a machine rig will alter its sound as it operates. Strain or stress will alter the sound it makes and cutting through harder materials will sound different. In this way you can simulate not only, operational issues, but potentially reflect exactly what is being drilled based on the sound it makes. For training in critical failure scenarios audio is probably more important than other aspects of the sim. Engine or equipment issues, heavy vehicle performance and emergency situations are almost always indicated through the sounds you hear before any other indicator. A Hydro station turbine performs under constant stress and shutdown of this type of equipment is time consuming and costly. Capturing the sound of this equipment when working optimally allows immediate and easy reference at any time. Is your equipment working properly? Listen to how it currently sounds, then compare it to the reference recording for optimal operation. The “all around us” nature of sound should never be underestimated. Operating many large industrial machines has the constant inherent issue that the operator cannot see much of their environment or even most of the machine they are using. But if something goes wrong behind them they will often hear it, even if they do not see it. Many pieces of industrial equipment are huge and very noisy, as a consequence the operators must use hearing protection. But even with hearing protection the vibrations of potential issues can carry through to the operator. Many training simulators cannot provide haptic feedback (vibrations and movement) but again sound can provide this vibration information using low frequency sound output. Military When working on a VR infantry training simulator I was tasked with providing the sounds that matched the firearms being used by the troops using the simulator. So, I was told the priority was for the gun in the hand of the user to sound realistic. This was a good start to creating a useful trainer. However, when I asked about the sounds to use for the virtual AI opponents I was told just to use anything, the sound of an AK47 would be fine. I found this strange. My time in basic training many years ago reinforced to me that one of the most important steps in any engagement was to ascertain the biggest current threat, i.e., who was trying to kill you and how they were doing it. If you talk to any combat soldier you will find that many of them learn very quickly to recognise the sounds of various enemy weapons. This can mean the difference between life and death in the seconds you have to respond in a combat situation. What you hear is likely the first warning you have of a possible threat, spending time looking around to judge the situation may often get you killed, so your hearing at that point in time is critical. -Direction of enemy threat -Nature of threat: single weapon being fired or group or weapons -Type of weapon being fired: Is it automatic weapon fire, or a high calibre rifle? The difference between these two tells you the potential accuracy and penetration of the incoming fire and may well determine where you should seek cover. -Distance: After a while you learn that the quality of the sound can inform you of how far away the sound is coming from -Reflections: Echoes can provide even further information on where the threat is coming from as a sound will echo more when it travels through a built-up area such as building walls. Even within military vehicle simulators the audio can indicate systems failure, potential enemy threats, types of threat, direction of threat etc. Knowing that here is a potential imminent engine failure in your fighter plane because you hear a change in its sounds a second before the warning indicator lights up might just save your life. Vehicles Engine performance, road surface, weather conditions, status of your load (freight moving around, or loose ropes and straps) are all extremely important to simulate during training to ensure operators gain experience dealing with dangerous situations. All of these examples are also scenarios where audio can inform the user of exactly what the issue is and often the how bad the situation is. The main reason we train anyone in operating various vehicles is to ensure they can do so safely, for their own well being and for the safety of others. Providing situations to test a user’s response to potentially dangerous situations allows assessors to judge the operator’s suitability and equally important allows the user themselves to gain confidence in their ability to handle a range of scenarios and know that can respond in the best way. Summary Creating training simulators for military and industrial uses can be very expensive and it is important that these products are effective. Lives are quite literally dependant on people being trained properly. Good quality audio is important for the reasons stated in this article, but more importantly any amount spent on adding and improving audio feedback to a simulator will have a more significant impact on the effectiveness of the training than money spent on other aspects of the product. You could simulate all aspects of training safety with audio alone and know that the users understood exactly what the issue was and would never miss the problem. This is not the case with information to our other senses. As a developer you could add a huge range of situations to your training simulator utilising only audio functionality which is often faster to implement than other aspects of development. Utilising effective audio design for training is cost effective, impactful and most importantly improves the effectiveness of the all safety training across all industries.
2 Comments
 Introduction
Any new platform of technology needs a reason for people to adopt it. There is seldom a single incredible app or game that tips everyone over the edge, but more a series of products that have mass market appeal and are noticeable enough that people will decide to take the plunge into the new device. Last Week Sony announced that the PlayStation VR had reached 3 million sales. Not bad, but still a small percentage of the overall 80 million PS4 owners. The growth of VR is strong, and AR is going to be hot on its tail, the question is, where are our killer apps for these platforms? Two important Factors There are two factors in my mind that are affecting this situation. The first is Time. It takes time to create AAA games and applications, in some cases many years. This technology is still new enough that developers first need time to understand the format and then time to create high quality content. Maybe the next 12 months will see the release of a few of these longer development titles. The second factor is the reason for this article and that is Design. I have spent several years working and “playing” in the VR and AR spaces, but I have noticed a worrying trend specifically among the larger developers. Too many of them are trying to shoehorn existing genres, and in some cases existing content onto the VR platform often in the hope of making a quick dollar. I understand that there is a risk with creating any new projects and that porting existing content may be a way of testing the waters and learning some of the lessons, but this trend is a symptom of a wider problem and I think it is inhibiting these new formats rather than helping them. VR is very Different VR is a very different platform and a different experience to traditional game formats and we need to embrace that if we want to succeed in this area. Traditional “run and gun” style games simply do not work and neither do many other popular game genres. Playing sport in VR, controlling a strategy game and even traditional role-playing formats all need, not just a major overhaul in design, but an entirely new approach from the ground up. 360 VR cinema is the same. 100 years of cinematography and editing techniques are almost entirely redundant when we create content where the viewer can be looking anywhere at any time. Once we place our audience into the actual narrative environment for both VR film and VR games we have to approach the entire process differently. There are some incredible examples of what we can achieve with these formats. Acclaimed director Alejandro González Iñárritu’s Carne y Arena 360 video experience continues to sell out tickets almost as fast as it puts them on sale. This is an experience that has been carefully crafted to utilise the potential of the technology. And there are various examples of games and applications that have taken clever approaches to these new formats. Stop thinking about limitations I think the issue is we are still talking about what VR CANNOT do, or how we must adopt existing content to fit the LIMITATIONS of the format. It feels like we have been given the ability to fly and we are discussing the difficulties with swimming. One advantage of porting old formats to the new platforms is to figure out what works and what does not, in this way it is a legitimate exercise. But we have about 50 examples of FPS games not working on VR, so I think it might be time to move on. It is obviously not easy to simply design a killer app, but anyone who experiences VR or 360 content will understand pretty quickly which aspects of the experience work well and which do not. I have at least a dozen designs for VR content floating around inside my head that simply came from analysing the experiences I enjoyed and comparing them with those I did not. Rapid movement and combat mechanics defined many of our traditional game styles on consoles and computers. Mobile platforms introduced the concept of quickly swiping our fingers or guiding objects with multiple taps. We adapted our control methods to better suit the device capabilities. VR places us into an environment that completely surrounds us and yet so many experiences only utilise a tiny portion of that space. If rapid movement causes nausea in VR then don’t use it. If controlling 100 units in a strategic battle is impossible in VR then perhaps the user should be placed onto the battle field to command 5 characters instead of floating over it. God’s view games are obviously not the best use of VR…or are they? Creating a killer app is not going to be easy, and nor should it be. It should be the culmination of an understanding of how the tech can be utilised effectively with a design whose surface simplicity belies its underlying clever creation. What we can do If movement in VR is difficult, then maybe the audience should be static, OR the user could be carried or moved around. (Think wheelchair or sitting on the shoulder of a giant) Movement is not the problem, the audience controlling that movement currently is. Putting the audience on “rails” allows the developer to exactly tune the experience to avoid issues We have a 360 sphere to inhabit in VR, please use it, both for movement potential as well as drawing the audiences attention to anywhere around them. (Look up AND look down) Experiences in vehicles work well as the audience is seated and controls movement of the vehicle. This also can reduce nausea as the fixed points of a cockpit can counter the nausea. We can use arm gestures and movement combined with button input, combined with upper body movement. This is a massive amount of input data to work with. Spatial audio is incredibly powerful at both supporting narrative and providing data to the audience, use it! Scale is incredible in VR. When I can stand right in front of a Death Stalker or look out the window and see the USS Enterprise at what I perceive as 1:1 scale then every previous gaming experience I have ever had pales in comparison. VR s incredible in this manner. Use this to your advantage. VR isolates us from the real world. A Headset and Headphones blocks out real world sensory input. We are placed into a virtual world like diving into water. Embrace that ability, surround your audiences with both visual and auditory information. Feed their senses and they will love being inside the world you created. Conclusion All of our previous formats of media have positioned the audience at a window where they could peer into another world and view the tiny portion of that world visible from the window. VR and 360 cinema allows all of us to step through the window and enter those worlds. By contrast AR allows us to reach through the window and pull objects and characters through into our world. We cannot hope to achieve this level of immersion if we persist with our current designs. Pushing my face up against the window is not the same as letting me step through it.  During my time at Magic Leap my task was to design, create, think-about and research how audio would function on an AR wearable device and what the audience would experience. From that list above the most important item was “think-about”. All of the new reality formats differ in many ways from traditional media, but AR wearable devices introduce a whole new series of challenges and none more difficult than presenting our audiences with what they will hear through an AR wearable device. I want to start by offering my definition of AR vs VR. In VR we place our audience into a virtual world. We surround them with all the necessary aspects of that world to convince them that they have indeed been transported to a realistic and dynamic environment. We override their senses with the content that we have created in an attempt to coerce the audience that this virtual world actually exists. In some ways this is a reasonably simple task. A VR headset and headphones mostly obscure the audience’s sight and sound, so we are depriving them of real world input and the human brain will quite quickly latch on to the virtual information and process it as valid. (At least to some extent.) In AR the audience’s world still exists, they still receive sensory information for what they see and hear and instead we attempt to insert virtual objects into the real world for the audience to experience. This is a much tougher proposition for our brains to accept. At all times we have the comparison of the real world right there beside the created virtual objects. We must blend how the virtual objects look and sound into the space the audience is in. Visually this means an object needs to occlude and be occluded by objects in the real world, it should cast shadows and it should both affect and be affected by the lighting of the world space. From an audio point of view all the same factors apply. A virtual object should emit sound as though it was in our world, and this is incredibly hard, in fact right now with the existing technology it is TOO hard. There are too many factors and too many calculations of how sound behaves for us to replicate it perfectly on any device. There is also the very real challenge that we still do not 100% understand exactly how humans hear and perceive sound under all circumstances. We can create the most realistic looking object and place it into a real-world space with perfect lighting and occlusion, but if the sound of that object is not convincing to the audience then the entire illusion will collapse. So, the question becomes, how do we create convincing audio for AR? The first step is to understand the limitations of devices such as Magic Leap One, Hololens and other wearable AR devices. Remember limitations are not always problems to solve, they can be opportunities to utilize. Things that are difficult for AR audio.
I am not going to explain each of these aspects in this article, you will just need to trust me for now that these things are all pretty hard to achieve. (I may go through them in more details in the future) The nature of AR means that the sound created by the device needs to blend with the real world, so the device dos not obscure what we can hear in the real world. Our AR sound needs to blend with the sounds in the audience’s space. This alone creates a huge range of possibilities. Is the audience sitting quietly in their lounge room, or are they travelling on a noisy train at peak hour? Remember because the device allows real world sound in to the audience’s ears, it can also bleed sound out so that others around the user may hear it as well. So a virtual avatar speaking to the audience may be speaking too softly, or perhaps you could utilize the devices inbuilt microphones to detect the ambient noise level of the audience’s space and adjust the level of the avatar’ speech. Here is where we can analyses the creative opportunities of that single function and see how we can create an engaging experience for the audience.  Let us imagine that you are enjoying a virtual storyteller app on your AR device. A virtual avatar stands in front of you (or perhaps sits next to you if the device detects an empty seat). The avatar describes a mythical tale to you as you journey home on the train. This app has been cleverly designed to account for a range of environmental situations. Initially the avatar speaks to you in a normal voice as it tells its tale. But as the journey continues the clever design of the developers is demonstrated.
As more people get on board your train the ambient noise level increases, it becomes harder for you to hear the story. You ask the avatar to speak up. The application detects the ambient noise levels and increases the intensity of the avatar’s voice. I do not mean the volume, I mean the actual intensity. When this app was created, the developers recorded the vocals at various intensity levels. If you ask me to speak more loudly it is not just an increase in volume, but the entire way I speak will change. This application reflects that change by dynamically switching the dialogue to a more forceful speaking voice, so the words carry to the listener over increased background noise. As still more people get on the train it once again gets harder to hear the avatar’s story. If you ask the avatar to speak even louder you risk annoying the other passengers as the volume will become loud enough for others to hear. This time you request the avatar to come closer to make it easier to hear. Visually you see the avatar lean forward to speak in your ear. The sound changes to a stage whisper. The avatar is now speaking “closer” to your ear, it is a more direct sound and as such does not need the same level of intensity as the pervious speech. Essentially the volume is slightly louder to the listener, but also the frequency content has shifted to represented a sound source closer to your ear. Finally as you near home the train has emptied of most of the passengers and you ask the avatar to speak more softly. The avatar leans back (sits in the now empty chair next to you) and resumes the normal default speaking manner. There are various reasons why this “works” as an AR application. The audio consists only of human speech. That means it occupies a nice safe frequency range that the device can produce easily. Also, human hearing’s primary function is to input speech, it is familiar to us and so our brains help us to understand what we are hearing. There is only a single object producing sound and we are so familiar with how humans work that we expect the sound to come from the avatars mouth, we expect that sound to be clearer and “louder” when the avatar moves closer to our ears. So, our perceptions are doing a lot to support this virtual illusion. When the technology cannot provide 100% of the functionality we need, we can help the situation by utilizing human perceptions and this works best if we align with normal expectations. The more you break away from expectations the more you need the technology to support. A mouse is small, quiet and has a high-pitched squeak, an elephant is large loud and has a low-pitched roar. If we adhere to this the human brain will do much of the work for us. There really are limitless opportunities for amazing and engaging applications that can be created for AR. The critical point is that they may not be the sorts of experiences we are used to enjoying. The lack of low frequency support means we cannot easily create giant rumbly Hollywood explosions, giant robots, and earth-shaking events. So perhaps we shouldn’t try to. The various other limitations on vision, sound and user input mean we need to unlearn a few old habits and adopt a few new ones. What is very apparent is that we can no longer have a single person in a bubble design a game or an app and then expect the various disciplines to just make it happen. We need experts across all aspects of production to collaborate carefully to consider what is possible and how to make the most of that, rather than waste time trying to achieve production elements that are currently very hard or impossible to create. As a freelancer I have the advantage of working on a great and wide range of projects and design styles with many different teams. This provides me with an insight you often don’t get with teams that focus on the same type of content over and over. I think AR is a time for many of us to embrace concepts and ideas that may be different and even scary to us as we dig into what really is possible with a new format that has so much potential to amaze.  Definition from Wikipedia
Impostor syndrome (also known as impostor phenomenon, impostorism, fraud syndrome or the impostor experience) is a psychological pattern in which an individual doubts their accomplishments and has a persistent internalized fear of being exposed as a "fraud". Introduction I learnt about Impostor Syndrome (IS) several years ago. After having a colleague explain it to me and doing a bit of research I presented a series of conference presentations and panels on the subject in Melbourne, LA and San Francisco. One of the key things I discovered in my research is that it is very wide spread, it effects almost every industry, it is common in both men and women and it effects some of the most famous and accomplished people on the planet. Essentially, Impostor Syndrome can and does effects people from all walks of life. Like other issues that effect people it also has differing degrees of severity from person to person and can even have differing degrees of effect on the individual from day to day. It is important to understand the potential impact that Impostor Syndrome can have on an individual, but it is also important to understand that perhaps not all the consequences of IS have to be negative. The best way to deal with anything that effects how we function as a human is to understand the limitations that might be in play, but also work out if there is anything you can turn to your advantage. No one would choose to be blind, but many that have found themselves blind have discovered a new focus on their other senses can allow them to experience the world in unique and valuable ways. This obviously does not mitigate the impact of the blindness, but it allows the individual to focus on a life still rich with experience. Humans can be incredibly adaptive. The Bad People who do not experience Impostor Syndrome may not fully understand the effect it can have on others. The common assumption is that someone experiencing IS just feels unworthy or that their work is not good enough. Obviously, an unpleasant feeling, but does it really effect an individual’s life. The answer to that is yes and often in very significant ways. If you believe the work you do and the things you have accomplished are far less valuable than they actually are then you are less likely to apply for a job that you might be very qualified to do, you are less likely to ask for a raise that you may very much deserve and in general you can hinder your own career growth. Very few of us have someone else to champion our cause in our careers, we have to seek out promotion and pay rise, we have to sell our skills and ourselves when we apply for a job with a new company. These days often promotion only comes from bouncing from company to company every few years. I think we all know someone who lacks confidence in themselves or will convince themselves that they would not get the job they just applied for even though they would be amazing at the job itself. The unfair reality of modern business is that we not only need to be good at a job, we also need to be good at the interview skills that get us that job. The not so Bad One aspect of impostor syndrome that I have heard about in my industry is that it can keep the ego in check. Often too much in check, but this still has an up-side. I work in the entertainment industry and this is an industry that does have some folk who are very “confident” in their own skills and abilities. There is a fine line between confidence and arrogance and as much as downplaying your skills might undermine a job interview, being overly arrogant will often have the same result. This example is a little bit “silver-lining” the experience of impostor syndrome, but it is still important to understand that not every aspect of it is negative. And some aspects can be positive in certain circumstances. The Good I likely suffered from Impostor Syndrome long before I even knew it existed. I could not say how long it may have affected me, and I am also aware that certain cultural expectations can influence the extent to which IS plays out in each individual. Australians, for instance, are far more reserved in how they describe themselves compared to Americans, and many European and Asian cultures completely avoid the individual singing their own praises. So it can be tricky to know if “shyness” is from IS or cultural conditioning. Regardless of your background or personal exposure to IS I have realised there is one significant aspect of IS that for me has become quite positive. It makes me work my butt off! As a consequence of not feeling quite “good enough” or feeling “not as talented” as my colleagues, I have spent most of my life trying to gain ground on those that I perceive as being ahead of me talent wise. As a result, I have pushed myself constantly to improve my skills, broaden my skill-set and basically compensate for my perceived short-comings. I honestly wonder if I would have been so driven if I did not feel that level of unworthiness. I have ended up as somewhat of a Jack of all trades within my specific field. I think that is a result of feeling a need to learn more to make myself more valuable. Ambition is not a negative trait, it is a force that motivate us to work hard and even excel at what we do. It is not always about money. As I have matured I have discovered a desire to accomplish things for their own sake and less because fame and fortune may follow. That is certainly a consequence of getting older and wiser, but I also think the drive that came from Impostor Syndrome may have contributed. As I worked to broaden my skills to “compensate” I also discovered more aspects of my career that I enjoy. A broader skill set has allowed me to work on a far wider range of projects than I ever might have and from this I have discovered the joy of working on various things simply for the sake of it. So what might have started as an attempt to be “worthy” has resulted in significant satisfaction in what I am capable of doing. We cannot affect how we feel, but maybe we can affect how we will respond Great care needs to be taken when speaking about issues that effect how people function. You would not tell someone with two broken legs to just “walk it off”. But in the same way that time spent in a wheelchair while legs heal may have the benefit of increasing upper body strength there can be “silver linings” to mental and emotional issues. With all things it is how we respond to the circumstances in which we find ourselves. The benchmark is not comparing ourselves with others, but to ourselves. First I didn’t not know about impostor syndrome, then I understood that it affected me. Later I worried that it I might be handicapping myself, then I accepted that I will be who I will be. But now I realise I am able to take things that one step further and find benefit in a trait that many might see as purely negative. Impostor Syndrome is a very real thing and at times it can cripple you with self-doubt. Over the years I have composed live orchestral scores, written books, presented at conferences around the globe, been asked to join teams of incredibly talented people and worked for some of the biggest companies in the world. And yet I can still find myself hesitant to apply for a job because my internal voices will try to convince me I am unqualified. If I do end up feeling proud of something I have accomplished that can often be pushed down by those same inner voices for fear of seeming arrogant. A quick google search will provide a list of incredibly talented people from many and varied industries who also identify with effects of IS. But like most conditions both mental and physical it is how we respond to the effects that can influence the ultimate results. I do not consider Impostor Syndrome to be a positive. It is not pleasant to always feel your work lacks value, and the lack of confidence can have significant effects on someone’s career. But finding any benefit from such an effect can go a great way to helping the individual cope with the various negative issues. There are various quotes that say “We cannot control how we feel, we can only control how we respond to those feelings” It may not be what you think it is
I think one of the greatest issues with the word "success" is that the modern definition of it has been wildly distorted to a point where it is practically unobtainable. Modern media shows us success in the form of AAA Hollywood actors, or chart topping musicians, or billionaire tech company founders. In reality these people are aberrations. Anyone who has ever studied statistics can tell you that the first thing you do with a bunch of figures is, pretty much, ignore the very extreme examples at both ends. So out of a planet of 7 billion people a few hundred who are massively successful in their creative field are largely irrelevant to what normal people can achieve. And yet we set them as the bar we are supposed to reach. "Aim high" is a reasonable and useful thought process when you start a creative career. Ambition can be a really good thing if it motivates you to work hard and helps you focus on a goal. But aiming to achieve a level that is essentially impossible means you may never be happy with the things you do achieve. We can get trapped, forever looking to the top of the success ladder. I find myself often discounting achievements I should have celebrated because I was too busy looking to the next thing. It is really important to stop, look at the things we accomplish and acknowledge their significance. Is story becoming more important than gameplay? Observations on modern game design, casual players and the future of game development. I'm no longer hardcore I have recently been extremely busy and as a result I do not get to spend as much time playing games these days as I would like to and as I did in the past. This means that I am now officially what marketing departments refer to as a casual gamer, although I don’t seem to fall into the usual demographic of family with kids playing tennis on the Wii. The first thing this suggests to me is that there are probably several levels of “casual gamers”. As more of my generation get older and get busier we can no longer spend the many hours we used to invest into a game, but we have not all become what marketing people usually refer to by the casual gamer label. Sure we don’t have a lot of time to play games, but unlike many causal gamers this does not mean we want to play family orientated titles with a simple control interface. We still want the hardcore story driven immersive experiences we have played in the past, we just want them in more bite size chunks, and this is where I think there is a market that is at risk of being overlooked. I have played several games in the last year, some which I enjoyed greatly and a few too many that I felt let down by. Because of my limited time I tend to choose very carefully what games I want to invest my time in. I love good stories, and immersive environments with both nice graphics and of course good sound. There have been plenty of titles that seem to fit this bill in the last few years, but as the casual gamer I have also discovered what I see as some serious issues with game design as well as some really wonderful gaming experiences.  The Force Unleashed by Lucas Arts The Force Unleashed was an excellent game spoilt by what should have been the highlight of the game. As a Force user, the ability to rip a Star Destroyer out of the sky should make any fan scream with joy. Instead a clunky and confusing control interface made this the worst part of the entire game, and the point at which I gave up and shelved the game for good. Please don’t set me up for a fail Recently I have played three games that I think have a game-flow issue; Mass Effect, The Force Unleashed and most disappointingly Batman Arkham Asylum. The issue with all of these games was that they were so good, and so enjoyable to play, right up to where I came to a point I could not progress past, and that’s what made them frustrating. In all three there was a certain element of game-play, or a mission or a fight I was unable to overcome. Now there is the obvious response of “Well just toughen the heck up and get it done! Play that game till you conquer it” but this is part of the issue. As someone who is time poor, I choose how I spend my leisure time fairly carefully, and I want my game time to be about good story and an interactive experience, not being punished because I no longer have the reflexes of a twelve year old ninja, and more importantly the free time of that same twelve year old ninja. If I find myself with a spare hour to sit down and plug through a level or two, I don’t want to spend that time repeating the same boss fight or level I got stuck on last time, it’s not much fun and it certainly isn’t relaxing. Of the above titles, Batman was certainly the best because each time you failed in a task it would reset and offer a hint as to how to progress, and generally it was a far more forgiving game. This I thought was excellent as it allowed me to experience the story which was why I was playing the game in the first place, and this is my main point. I think we have reached a point where modern games with stories need to allow the player to get through the story as this has become the focal point for many players. The excellent stories and enjoyable interactive experiences are why many of us are playing these games.  Rocksteady Studio's Arkham Asylum Batman Arkham Asylum is so well designed and implemented that it was a real pleasure to play. Its forgiving nature and good control system make you feel like you are controlling a competant character. Although there is a "roadblock" section that I find frustrating and unnessesary, the rest of the game has been so well made that I will persist and spend the time to continue as I think the remainder of the game will be worth the effort. Sadly this is not a statement I can make for all games. Batman was a surprise to me; I did not particularly want to play a Batman game as I see the character as a heroic figure, not a game avatar that bumps into walls, falls down holes and has to spend his time looking for golden keys. The game however is very little of these things. Batman is indeed heroic, if you as the player fall down a hole there is usually an escape button and his movement is generally fluid and effective; he feels competent. This is where I think games need to go. I WANT to play a hero, I WANT to feel like I am guiding a competent character through a dangerous world, I really don’t want to feel like I am controlling a bumbling fool simply because I did not press the jump button on the exact pixel between two platforms. I am perfectly capable of being clumsy in my real life, I don’t need my game characters to follow suit. I am certainly not asking or expecting games in overall to be made easier, but considering the focus of many modern games is a story line, I would like to know that I can see this story through to the end. If for whatever reason I find there is a point I cannot proceed past, then the game should allow me to either progress anyway, tell me exactly what to do, or lower its difficulty level so far that I cannot help but get past the obstacle. It is extremely difficult for a developer to predict every possible way in which a player may approach their game, or the potential issues they may have with it, and this is not referring to any bug or design flaw issues, it is just looking at the vast difference between how people think. Because of this it seems to make sense to have an escape key that allows the player to always progress.  Mass Effect by Bioware For me, Mass Effect is probably the most dissapointing as it was such a good game. To place a boss fight directly after an unskippable cutscene of a few minute long is capital crime number in game design and should NEVER happen. It's bad enough that you might have to fight a certain boss over and over. To place it directly after a cutscene so you can't save, and then make the cutscene unskippable, sorry you deserve to rot in gaming hell with only copies of ET on the Atari 2600 to keep you company. The better you make them, the pickier your audience will be I consider Mass Effect to be the perfect example of why this is relevant. I had waited for quite some time before I purchased an Xbox360, when I finally did I was really looking forward to Mass effect. I had always enjoyed Bioware games and this looked very promising. I was really happy to find it lived up to my expectations. The game-play and graphics were excellent, and the general feel of the game was good, but the story and acting was where I really believe the game shined, it was excellent, and made playing the game a real pleasure. That is until I got stuck on an annoying boss fight and couldn’t progress. I was not far into the story either. As a result, Mass Effect 2 which has just been released is not even on my consider list, as I see no point in buying the sequel to a game I couldn’t finish. Now again, whether I should have been able to get through the game or not is not the point, if I had issues with it then it’s almost certain other people did as well. Five or ten years ago this situation was less relevant. Games with good stories have been around for a while, but in my opinion we are only just reaching the point where games are getting the budget to make really good use of the story elements. “A” list voice actors, well written scripts and story elements and current generation graphics and animation have resulted in a generation of games that are viewable for their own sake. My wife and I often take it in turns to play through the games we purchase. Often I won’t have time to play a game at all, but I still enjoy many current games as a spectator while I do other things and this is primarily because they have good stories. But when it's right it's oh so good 2K's Bioshock 2 I don't have a paternal bone in my body, so any game that can make me protective of a young child and send me into a rage if she is threatened has really pressed the right buttons as far as story and atmosphere. Bioshock 2 has so far been one of the most enjoyable games i have played in a long while. At the time of writing this article I am several hours into Bioshock 2 and it is incredibly enjoyable. The graphics and sound are excellent, but I find it is the story elements that I am enjoying most. The little snippets of people's diaries presented as recorded logs, and the dialogue of the main characters add so much to the game. I am fairly confident I am unlikely to come across a show stopping issue as I played through Bioshock 1 completely and had no issues; in fact many people complained that Bioshock 1 was too easy. Bioshock 2 allows you to switch off the ability to simply respawn from the resurrection chambers and with this one simple adjustment they can cater for both a player who wants a challenge and a player who wishes to be able to complete the game for the sake of story like myself. I think this is an excellent compromise and something that should become standard for all games.
The points I raise are more important for a game with a heavy story element than a more simple action game. If the fun is in action and destruction then I can understand that there is less need to progress to the end of the game, but for story games, please give us all a chance to see the fantastic work you have put into the entire game rather than risk your audience giving up half way through frustration. Boingy Boingy Boingy Why I hate repetition and what can be done about it I suspect it is the time I have spent in the video game industry that is the main cause of my hatred of repetition in audio. PCs and video game consoles have always had limitations. The amount of storage space on a disk, available memory, seek and read times of disk media (CD, DVD and Blueray) are all examples of some of the many issues developers need to deal with when producing games. Because of the limited resources and the need to share these resources amongst all aspects of the game, usually it is necessary to design a project that can cope with resource limitations. For game audio this often meant less variation than might have been desirable.Ever since I started working on games I have been aware of their limitations and that sometimes some repetition is unavoidable, however far too often I have played games that do not seem to be pushing the resources of the relevant console and yet still have overly repetitive sound design. One of the worst instances I encountered was a PS2 game that literally only had two footstep sounds, and the developers had also implemented them too loudly, so the entire game was accompanied by this clip, clop, clip, clop, clip, clop of the main character. For me it made the game unplayable after only a few hours. I am happy to admit I am pretty fussy where game audio is concerned, but it really spoilt what was otherwise a fun game. So why am I so distressed by a little repetition? Think of any sound that might repeat itself, and then think carefully about that sound. A dripping tap, bouncing a ball and footsteps are all good examples of sounds that repeat cyclically, but if you really listen and analyse the sounds themselves you realise that they are not exact replicas of the same sound. Bounce a ball sometime and see if you can get exactly the same sound twice in a row, let alone for longer. Each bounce is subtly different, the downward pressure you apply to the ball, the angle on which it might land and the surface on which the ball bounces will all alter the sound, they may not alter it much, but it doesn’t take much variation at all to break up a repetitive cycle. In fact it’s the super subtle differences that really make the difference. If you isolate a single tic toc from the sound of a clock, loop it and listen back to it, it does sound like a clock, but there is something about it that just doesn’t sound quite right.One of the issues with modern video games and CG films is that they are becoming very close to being indistinguishable from real life; close, but not yet completely there, and that’s one of the problems. The closer you get to making something that reflects reality the more the subtle differences stand out. So with audio, a bouncing ball or ticking clock sound that repeats is pretty close to how a ball or clock may really sound, but the subtle difference tells the listener that what they are listening to is not the real thing. In this case and to help create many other types of repetitive sounds, subtle variations can go a long way to convince a listener they are listening to the real thing. To use the ball analogy lets assume we are creating a basketball game. One of the main audio elements we require is the sound of the ball being bounced. With sound files time equals memory, at 44.1 KHz, 16 bit mono, a ten second long sound is 860Kb, this is almost a megabyte of memory for only ten seconds of sound. So we would not want to record long samples of someone bouncing a ball to add to the game as we would quickly run out of memory. Alternatively if we use a recording of a single bounce it will result in very obvious repetition. If we use 5 or 6 different recordings of a single bounce they can be used as a pool of sounds to draw from. Randomly selecting a different recording to play for each bounce event will instantly add more variety than using a single bounce. Adding a very slight variation of pitch each time a ball bounce is played will produce more variation and represent slightly more forceful or less forceful bounces. Finally adding a very slight volume variation to each bounce will add even more variety. In the end these three simple steps can turn a selection of 5 sound files into a sound event that can play endlessly without ever sounding like it is repeating. This effect has been achieved using a small number of files that will not use much memory, this is particularly important for smaller consoles like a Nintendo DS or for iphone apps. Repeated bounce A single bounce sound file, looped to repeat four time No variation in pitch, volume or time. Non repeated bounce Five unique bounce sound files played in sequence Each bounce is different in sound and pitch, volume and time spacing. When creating slightly more complex sounds there are more ways in which random elements can be added. Footstep sounds are created from the sound of the particular shoe impacting on the ground and the reaction of any materials that are stepped on. So a boot walking on short grass would only have a basic impact sound, but walking on gravel or leaves would produce the sound of the leaves being crushed or scattered, or the stones moving. Because there are essentially two layers the sound event can be created by combing two sound pools. The first would contain a selection of boot impact sounds to randomly draw from, and the second a selection of leaf sounds. If we have five sounds in each pool the number of combinations suddenly increases dramatically. This simple solution can effectively remove repetition almost entirely. I want to return to why I dislike repetition, and look at it from a different angle. As I mentioned, initially it was from frustration at being presented games that appeared to have poor or lazy sound design. As I looked into solutions to this issue so I would not fall into the same trap I realised that the randomisation of sound might even improve the overall feel of a game. Several years ago I read an article where some film makers were previewing their latest project to a test audience. They showed the audience two versions of the film and asked for their reactions. The vast majority of those surveyed said the second version was much better, they claimed the special effects were better and the pace and editing was improved. The amazing thing was that visually both versions were identical. The only difference was the music and sound effects used. Now obviously this can be seen as a good example of how important sound is for supporting visuals, but more than that it demonstrates that sound can even alter how someone perceives visuals. It has been well documented that altering a musical score can seriously change the feel of a film, adding ominous music to a scene can make even the most angelic character seem disturbing and playing the theme to Benny Hill has the disturbing effect of making the most horrific footage feel like a comedy routine. Music affects human’s emotional state to a level we still do not fully understand, so it is not surprising that it can have this kind of influence when accompanying a movie clip, but what about sound effects? Recently I was asked to write an article about using FMOD Designer for sound event creation. I have been using FMOD for many years as it allows me to create sounds with a large level of variation. For the purposes of the article I wanted to include a film clip of an explosion. Most people respond to visuals, so adding a film clip to watch helps convey the idea more easily. I chose a clip from a friend I have worked with of a mine exploding and added an appropriate sound effect. The effect was generated in FMOD as it would be in a game situation. I had set up the event to randomize the sound each time it played, so the explosion would always sound different. I recorded a series of this sound so I could choose which version I wanted to add to the clip. When it came to editing the clip I found lots of versions of the sound that worked really well. I added one to the clip and uploaded it for the article. I realised that this single clip did not demonstrate FMOD’s ability to randomise the sound each time it was triggered, so I made a second clip where I edited the explosion to repeat 4 times. I added a different version of the sound to each explosion. This clearly demonstrated what FMOD was capable of, but when I previewed the clip I realised it also demonstrated something else. The sounds were different enough that they even give the impression that you are viewing four different explosions. Part of this is the way I edited the clip, but most of it is a result of the sound effects. For me this really highlighted the fact that sound and visuals are more closely linked than we give credit for and that one can often influence how the other is perceived. For a long time I have worked towards reducing or removing repetition from game projects wherever possible, and I always try to produce high quality audio environments, but the example of the multiple explosions showed me how audio could go even further and be used to not just highlight visuals, but even alter how they are perceived. I would be interested to work on a game project where a level didn’t change visually but went through a series of transformations for the audio environment and see just how far you could alter people's perceptions without changing the visuals. Games are fantastic things to work on creatively because you can create worlds where almost anything is possible. Maybe you could even make a game where lots of repetition was entertaining, but I am doubtful.
|
AuthorStephan Schütze has been recording sounds for over twenty years. This journal logs his thoughts and experiences Categories
All
Archives
April 2019
|
RSS Feed
