Let me start by asking you a question. When you are driving your car, what is one of the key indicators that something is wrong, that it may need a tune-up, or worse still, that it is about to break down? There is a reason why hearing is our critical danger and warning sense. It is always on (even when we are asleep) and it works in a 360-degree sphere, so we can hear things we cannot see. Also, our brain, through what we hear, recognises patterns very effectively so we notice a change in sound that may indicate a problem.
Yet even with how effective our sense of hearing is I have been surprised on many occasions by the number of training simulators, both military and industrial that overlook the importance of sound in their trainers. I have spoken to users of some of the most advanced (and expensive) military flight sims and been told over and over that the sound was poor quality, not very useful and not designed in a way that aids the user.
VR is rapidly proving to be an effective format for providing training for vehicle and machinery users. Being able to place a user INSIDE a virtual vehicle and have them operate it in a virtual world allows for realistic training and the virtual nature of the space ensures safety.
The audio capabilities for VR surpass almost any other format of media, and we are still discovering just how effective VR audio is at providing useful feedback and enhancing the realism of VR environments. We can completely surround the user in audio that responds to their actions and can be used to indicate events in the virtual world.
Moreover, well implemented VR audio can make a massive impact on the value of a trainer far in excess of its cost. Need to simulate a critical machinery failure? This can be done almost entirely with sound, it will indicate an issue the user cannot even see and does not require expensive graphic representation in many cases.
So, lets have a look at a couple of scenarios for simulators and how well-designed audio could support them.
For an industrial training simulator designed to train heavy machinery operation, the audio feedback provides information confirming nominal operation. Essentially if everything sounds good, then everything is likely working correctly. For equipment used for digging, audio feedback can inform the user of the types of materials being processed. A drill of any kind, from your cordless home drill right up to a machine rig will alter its sound as it operates. Strain or stress will alter the sound it makes and cutting through harder materials will sound different. In this way you can simulate not only, operational issues, but potentially reflect exactly what is being drilled based on the sound it makes.
For training in critical failure scenarios audio is probably more important than other aspects of the sim. Engine or equipment issues, heavy vehicle performance and emergency situations are almost always indicated through the sounds you hear before any other indicator. A Hydro station turbine performs under constant stress and shutdown of this type of equipment is time consuming and costly. Capturing the sound of this equipment when working optimally allows immediate and easy reference at any time. Is your equipment working properly? Listen to how it currently sounds, then compare it to the reference recording for optimal operation.
The “all around us” nature of sound should never be underestimated. Operating many large industrial machines has the constant inherent issue that the operator cannot see much of their environment or even most of the machine they are using. But if something goes wrong behind them they will often hear it, even if they do not see it. Many pieces of industrial equipment are huge and very noisy, as a consequence the operators must use hearing protection. But even with hearing protection the vibrations of potential issues can carry through to the operator. Many training simulators cannot provide haptic feedback (vibrations and movement) but again sound can provide this vibration information using low frequency sound output.
When working on a VR infantry training simulator I was tasked with providing the sounds that matched the firearms being used by the troops using the simulator. So, I was told the priority was for the gun in the hand of the user to sound realistic. This was a good start to creating a useful trainer. However, when I asked about the sounds to use for the virtual AI opponents I was told just to use anything, the sound of an AK47 would be fine. I found this strange. My time in basic training many years ago reinforced to me that one of the most important steps in any engagement was to ascertain the biggest current threat, i.e., who was trying to kill you and how they were doing it.
If you talk to any combat soldier you will find that many of them learn very quickly to recognise the sounds of various enemy weapons. This can mean the difference between life and death in the seconds you have to respond in a combat situation. What you hear is likely the first warning you have of a possible threat, spending time looking around to judge the situation may often get you killed, so your hearing at that point in time is critical.
-Direction of enemy threat
-Nature of threat: single weapon being fired or group or weapons
-Type of weapon being fired: Is it automatic weapon fire, or a high calibre rifle? The difference between these two tells you the potential accuracy and penetration of the incoming fire and may well determine where you should seek cover.
-Distance: After a while you learn that the quality of the sound can inform you of how far away the sound is coming from
-Reflections: Echoes can provide even further information on where the threat is coming from as a sound will echo more when it travels through a built-up area such as building walls.
Even within military vehicle simulators the audio can indicate systems failure, potential enemy threats, types of threat, direction of threat etc. Knowing that here is a potential imminent engine failure in your fighter plane because you hear a change in its sounds a second before the warning indicator lights up might just save your life.
Engine performance, road surface, weather conditions, status of your load (freight moving around, or loose ropes and straps) are all extremely important to simulate during training to ensure operators gain experience dealing with dangerous situations. All of these examples are also scenarios where audio can inform the user of exactly what the issue is and often the how bad the situation is.
The main reason we train anyone in operating various vehicles is to ensure they can do so safely, for their own well being and for the safety of others. Providing situations to test a user’s response to potentially dangerous situations allows assessors to judge the operator’s suitability and equally important allows the user themselves to gain confidence in their ability to handle a range of scenarios and know that can respond in the best way.
Creating training simulators for military and industrial uses can be very expensive and it is important that these products are effective. Lives are quite literally dependant on people being trained properly. Good quality audio is important for the reasons stated in this article, but more importantly any amount spent on adding and improving audio feedback to a simulator will have a more significant impact on the effectiveness of the training than money spent on other aspects of the product.
You could simulate all aspects of training safety with audio alone and know that the users understood exactly what the issue was and would never miss the problem. This is not the case with information to our other senses. As a developer you could add a huge range of situations to your training simulator utilising only audio functionality which is often faster to implement than other aspects of development.
Utilising effective audio design for training is cost effective, impactful and most importantly improves the effectiveness of the all safety training across all industries.
Any new platform of technology needs a reason for people to adopt it. There is seldom a single incredible app or game that tips everyone over the edge, but more a series of products that have mass market appeal and are noticeable enough that people will decide to take the plunge into the new device.
Last Week Sony announced that the PlayStation VR had reached 3 million sales. Not bad, but still a small percentage of the overall 80 million PS4 owners. The growth of VR is strong, and AR is going to be hot on its tail, the question is, where are our killer apps for these platforms?
Two important Factors
There are two factors in my mind that are affecting this situation. The first is Time. It takes time to create AAA games and applications, in some cases many years. This technology is still new enough that developers first need time to understand the format and then time to create high quality content. Maybe the next 12 months will see the release of a few of these longer development titles.
The second factor is the reason for this article and that is Design.
I have spent several years working and “playing” in the VR and AR spaces, but I have noticed a worrying trend specifically among the larger developers. Too many of them are trying to shoehorn existing genres, and in some cases existing content onto the VR platform often in the hope of making a quick dollar.
I understand that there is a risk with creating any new projects and that porting existing content may be a way of testing the waters and learning some of the lessons, but this trend is a symptom of a wider problem and I think it is inhibiting these new formats rather than helping them.
VR is very Different
VR is a very different platform and a different experience to traditional game formats and we need to embrace that if we want to succeed in this area. Traditional “run and gun” style games simply do not work and neither do many other popular game genres. Playing sport in VR, controlling a strategy game and even traditional role-playing formats all need, not just a major overhaul in design, but an entirely new approach from the ground up.
360 VR cinema is the same. 100 years of cinematography and editing techniques are almost entirely redundant when we create content where the viewer can be looking anywhere at any time. Once we place our audience into the actual narrative environment for both VR film and VR games we have to approach the entire process differently.
There are some incredible examples of what we can achieve with these formats. Acclaimed director Alejandro González Iñárritu’s Carne y Arena 360 video experience continues to sell out tickets almost as fast as it puts them on sale. This is an experience that has been carefully crafted to utilise the potential of the technology. And there are various examples of games and applications that have taken clever approaches to these new formats.
Stop thinking about limitations
I think the issue is we are still talking about what VR CANNOT do, or how we must adopt existing content to fit the LIMITATIONS of the format. It feels like we have been given the ability to fly and we are discussing the difficulties with swimming.
One advantage of porting old formats to the new platforms is to figure out what works and what does not, in this way it is a legitimate exercise. But we have about 50 examples of FPS games not working on VR, so I think it might be time to move on. It is obviously not easy to simply design a killer app, but anyone who experiences VR or 360 content will understand pretty quickly which aspects of the experience work well and which do not. I have at least a dozen designs for VR content floating around inside my head that simply came from analysing the experiences I enjoyed and comparing them with those I did not.
Rapid movement and combat mechanics defined many of our traditional game styles on consoles and computers. Mobile platforms introduced the concept of quickly swiping our fingers or guiding objects with multiple taps. We adapted our control methods to better suit the device capabilities. VR places us into an environment that completely surrounds us and yet so many experiences only utilise a tiny portion of that space.
If rapid movement causes nausea in VR then don’t use it. If controlling 100 units in a strategic battle is impossible in VR then perhaps the user should be placed onto the battle field to command 5 characters instead of floating over it. God’s view games are obviously not the best use of VR…or are they? Creating a killer app is not going to be easy, and nor should it be. It should be the culmination of an understanding of how the tech can be utilised effectively with a design whose surface simplicity belies its underlying clever creation.
What we can do
If movement in VR is difficult, then maybe the audience should be static, OR the user could be carried or moved around. (Think wheelchair or sitting on the shoulder of a giant)
Movement is not the problem, the audience controlling that movement currently is. Putting the audience on “rails” allows the developer to exactly tune the experience to avoid issues
We have a 360 sphere to inhabit in VR, please use it, both for movement potential as well as drawing the audiences attention to anywhere around them. (Look up AND look down)
Experiences in vehicles work well as the audience is seated and controls movement of the vehicle. This also can reduce nausea as the fixed points of a cockpit can counter the nausea.
We can use arm gestures and movement combined with button input, combined with upper body movement. This is a massive amount of input data to work with.
Spatial audio is incredibly powerful at both supporting narrative and providing data to the audience, use it!
Scale is incredible in VR. When I can stand right in front of a Death Stalker or look out the window and see the USS Enterprise at what I perceive as 1:1 scale then every previous gaming experience I have ever had pales in comparison. VR s incredible in this manner. Use this to your advantage.
VR isolates us from the real world. A Headset and Headphones blocks out real world sensory input. We are placed into a virtual world like diving into water. Embrace that ability, surround your audiences with both visual and auditory information. Feed their senses and they will love being inside the world you created.
All of our previous formats of media have positioned the audience at a window where they could peer into another world and view the tiny portion of that world visible from the window. VR and 360 cinema allows all of us to step through the window and enter those worlds. By contrast AR allows us to reach through the window and pull objects and characters through into our world. We cannot hope to achieve this level of immersion if we persist with our current designs. Pushing my face up against the window is not the same as letting me step through it.
Developing content for Magic Leap One and AR Wearables Limitations can be opportunities
During my time at Magic Leap my task was to design, create, think-about and research how audio would function on an AR wearable device and what the audience would experience. From that list above the most important item was “think-about”.
All of the new reality formats differ in many ways from traditional media, but AR wearable devices introduce a whole new series of challenges and none more difficult than presenting our audiences with what they will hear through an AR wearable device.
I want to start by offering my definition of AR vs VR.
In VR we place our audience into a virtual world. We surround them with all the necessary aspects of that world to convince them that they have indeed been transported to a realistic and dynamic environment. We override their senses with the content that we have created in an attempt to coerce the audience that this virtual world actually exists.
In some ways this is a reasonably simple task. A VR headset and headphones mostly obscure the audience’s sight and sound, so we are depriving them of real world input and the human brain will quite quickly latch on to the virtual information and process it as valid. (At least to some extent.)
In AR the audience’s world still exists, they still receive sensory information for what they see and hear and instead we attempt to insert virtual objects into the real world for the audience to experience. This is a much tougher proposition for our brains to accept. At all times we have the comparison of the real world right there beside the created virtual objects.
We must blend how the virtual objects look and sound into the space the audience is in. Visually this means an object needs to occlude and be occluded by objects in the real world, it should cast shadows and it should both affect and be affected by the lighting of the world space.
From an audio point of view all the same factors apply. A virtual object should emit sound as though it was in our world, and this is incredibly hard, in fact right now with the existing technology it is TOO hard. There are too many factors and too many calculations of how sound behaves for us to replicate it perfectly on any device. There is also the very real challenge that we still do not 100% understand exactly how humans hear and perceive sound under all circumstances.
We can create the most realistic looking object and place it into a real-world space with perfect lighting and occlusion, but if the sound of that object is not convincing to the audience then the entire illusion will collapse. So, the question becomes, how do we create convincing audio for AR? The first step is to understand the limitations of devices such as Magic Leap One, Hololens and other wearable AR devices. Remember limitations are not always problems to solve, they can be opportunities to utilize.
Things that are difficult for AR audio.
I am not going to explain each of these aspects in this article, you will just need to trust me for now that these things are all pretty hard to achieve. (I may go through them in more details in the future)
The nature of AR means that the sound created by the device needs to blend with the real world, so the device dos not obscure what we can hear in the real world. Our AR sound needs to blend with the sounds in the audience’s space. This alone creates a huge range of possibilities. Is the audience sitting quietly in their lounge room, or are they travelling on a noisy train at peak hour? Remember because the device allows real world sound in to the audience’s ears, it can also bleed sound out so that others around the user may hear it as well.
So a virtual avatar speaking to the audience may be speaking too softly, or perhaps you could utilize the devices inbuilt microphones to detect the ambient noise level of the audience’s space and adjust the level of the avatar’ speech. Here is where we can analyses the creative opportunities of that single function and see how we can create an engaging experience for the audience.
Let us imagine that you are enjoying a virtual storyteller app on your AR device. A virtual avatar stands in front of you (or perhaps sits next to you if the device detects an empty seat). The avatar describes a mythical tale to you as you journey home on the train. This app has been cleverly designed to account for a range of environmental situations. Initially the avatar speaks to you in a normal voice as it tells its tale. But as the journey continues the clever design of the developers is demonstrated.
As more people get on board your train the ambient noise level increases, it becomes harder for you to hear the story. You ask the avatar to speak up. The application detects the ambient noise levels and increases the intensity of the avatar’s voice. I do not mean the volume, I mean the actual intensity. When this app was created, the developers recorded the vocals at various intensity levels. If you ask me to speak more loudly it is not just an increase in volume, but the entire way I speak will change. This application reflects that change by dynamically switching the dialogue to a more forceful speaking voice, so the words carry to the listener over increased background noise.
As still more people get on the train it once again gets harder to hear the avatar’s story. If you ask the avatar to speak even louder you risk annoying the other passengers as the volume will become loud enough for others to hear. This time you request the avatar to come closer to make it easier to hear. Visually you see the avatar lean forward to speak in your ear. The sound changes to a stage whisper. The avatar is now speaking “closer” to your ear, it is a more direct sound and as such does not need the same level of intensity as the pervious speech. Essentially the volume is slightly louder to the listener, but also the frequency content has shifted to represented a sound source closer to your ear.
Finally as you near home the train has emptied of most of the passengers and you ask the avatar to speak more softly. The avatar leans back (sits in the now empty chair next to you) and resumes the normal default speaking manner.
There are various reasons why this “works” as an AR application.
The audio consists only of human speech. That means it occupies a nice safe frequency range that the device can produce easily. Also, human hearing’s primary function is to input speech, it is familiar to us and so our brains help us to understand what we are hearing. There is only a single object producing sound and we are so familiar with how humans work that we expect the sound to come from the avatars mouth, we expect that sound to be clearer and “louder” when the avatar moves closer to our ears. So, our perceptions are doing a lot to support this virtual illusion.
When the technology cannot provide 100% of the functionality we need, we can help the situation by utilizing human perceptions and this works best if we align with normal expectations. The more you break away from expectations the more you need the technology to support. A mouse is small, quiet and has a high-pitched squeak, an elephant is large loud and has a low-pitched roar. If we adhere to this the human brain will do much of the work for us.
There really are limitless opportunities for amazing and engaging applications that can be created for AR. The critical point is that they may not be the sorts of experiences we are used to enjoying. The lack of low frequency support means we cannot easily create giant rumbly Hollywood explosions, giant robots, and earth-shaking events. So perhaps we shouldn’t try to. The various other limitations on vision, sound and user input mean we need to unlearn a few old habits and adopt a few new ones. What is very apparent is that we can no longer have a single person in a bubble design a game or an app and then expect the various disciplines to just make it happen. We need experts across all aspects of production to collaborate carefully to consider what is possible and how to make the most of that, rather than waste time trying to achieve production elements that are currently very hard or impossible to create.
As a freelancer I have the advantage of working on a great and wide range of projects and design styles with many different teams. This provides me with an insight you often don’t get with teams that focus on the same type of content over and over. I think AR is a time for many of us to embrace concepts and ideas that may be different and even scary to us as we dig into what really is possible with a new format that has so much potential to amaze.
Definition from Wikipedia
Impostor syndrome (also known as impostor phenomenon, impostorism, fraud syndrome or the impostor experience) is a psychological pattern in which an individual doubts their accomplishments and has a persistent internalized fear of being exposed as a "fraud".
I learnt about Impostor Syndrome (IS) several years ago. After having a colleague explain it to me and doing a bit of research I presented a series of conference presentations and panels on the subject in Melbourne, LA and San Francisco. One of the key things I discovered in my research is that it is very wide spread, it effects almost every industry, it is common in both men and women and it effects some of the most famous and accomplished people on the planet. Essentially, Impostor Syndrome can and does effects people from all walks of life.
Like other issues that effect people it also has differing degrees of severity from person to person and can even have differing degrees of effect on the individual from day to day. It is important to understand the potential impact that Impostor Syndrome can have on an individual, but it is also important to understand that perhaps not all the consequences of IS have to be negative. The best way to deal with anything that effects how we function as a human is to understand the limitations that might be in play, but also work out if there is anything you can turn to your advantage.
No one would choose to be blind, but many that have found themselves blind have discovered a new focus on their other senses can allow them to experience the world in unique and valuable ways. This obviously does not mitigate the impact of the blindness, but it allows the individual to focus on a life still rich with experience. Humans can be incredibly adaptive.
People who do not experience Impostor Syndrome may not fully understand the effect it can have on others. The common assumption is that someone experiencing IS just feels unworthy or that their work is not good enough. Obviously, an unpleasant feeling, but does it really effect an individual’s life. The answer to that is yes and often in very significant ways.
If you believe the work you do and the things you have accomplished are far less valuable than they actually are then you are less likely to apply for a job that you might be very qualified to do, you are less likely to ask for a raise that you may very much deserve and in general you can hinder your own career growth. Very few of us have someone else to champion our cause in our careers, we have to seek out promotion and pay rise, we have to sell our skills and ourselves when we apply for a job with a new company. These days often promotion only comes from bouncing from company to company every few years.
I think we all know someone who lacks confidence in themselves or will convince themselves that they would not get the job they just applied for even though they would be amazing at the job itself. The unfair reality of modern business is that we not only need to be good at a job, we also need to be good at the interview skills that get us that job.
The not so Bad
One aspect of impostor syndrome that I have heard about in my industry is that it can keep the ego in check. Often too much in check, but this still has an up-side. I work in the entertainment industry and this is an industry that does have some folk who are very “confident” in their own skills and abilities. There is a fine line between confidence and arrogance and as much as downplaying your skills might undermine a job interview, being overly arrogant will often have the same result. This example is a little bit “silver-lining” the experience of impostor syndrome, but it is still important to understand that not every aspect of it is negative. And some aspects can be positive in certain circumstances.
I likely suffered from Impostor Syndrome long before I even knew it existed. I could not say how long it may have affected me, and I am also aware that certain cultural expectations can influence the extent to which IS plays out in each individual. Australians, for instance, are far more reserved in how they describe themselves compared to Americans, and many European and Asian cultures completely avoid the individual singing their own praises. So it can be tricky to know if “shyness” is from IS or cultural conditioning. Regardless of your background or personal exposure to IS I have realised there is one significant aspect of IS that for me has become quite positive. It makes me work my butt off!
As a consequence of not feeling quite “good enough” or feeling “not as talented” as my colleagues, I have spent most of my life trying to gain ground on those that I perceive as being ahead of me talent wise. As a result, I have pushed myself constantly to improve my skills, broaden my skill-set and basically compensate for my perceived short-comings. I honestly wonder if I would have been so driven if I did not feel that level of unworthiness.
I have ended up as somewhat of a Jack of all trades within my specific field. I think that is a result of feeling a need to learn more to make myself more valuable. Ambition is not a negative trait, it is a force that motivate us to work hard and even excel at what we do. It is not always about money. As I have matured I have discovered a desire to accomplish things for their own sake and less because fame and fortune may follow. That is certainly a consequence of getting older and wiser, but I also think the drive that came from Impostor Syndrome may have contributed.
As I worked to broaden my skills to “compensate” I also discovered more aspects of my career that I enjoy. A broader skill set has allowed me to work on a far wider range of projects than I ever might have and from this I have discovered the joy of working on various things simply for the sake of it. So what might have started as an attempt to be “worthy” has resulted in significant satisfaction in what I am capable of doing.
We cannot affect how we feel, but maybe we can affect how we will respond
Great care needs to be taken when speaking about issues that effect how people function. You would not tell someone with two broken legs to just “walk it off”. But in the same way that time spent in a wheelchair while legs heal may have the benefit of increasing upper body strength there can be “silver linings” to mental and emotional issues. With all things it is how we respond to the circumstances in which we find ourselves. The benchmark is not comparing ourselves with others, but to ourselves.
First I didn’t not know about impostor syndrome, then I understood that it affected me. Later I worried that it I might be handicapping myself, then I accepted that I will be who I will be. But now I realise I am able to take things that one step further and find benefit in a trait that many might see as purely negative.
Impostor Syndrome is a very real thing and at times it can cripple you with self-doubt. Over the years I have composed live orchestral scores, written books, presented at conferences around the globe, been asked to join teams of incredibly talented people and worked for some of the biggest companies in the world. And yet I can still find myself hesitant to apply for a job because my internal voices will try to convince me I am unqualified. If I do end up feeling proud of something I have accomplished that can often be pushed down by those same inner voices for fear of seeming arrogant.
A quick google search will provide a list of incredibly talented people from many and varied industries who also identify with effects of IS. But like most conditions both mental and physical it is how we respond to the effects that can influence the ultimate results.
I do not consider Impostor Syndrome to be a positive. It is not pleasant to always feel your work lacks value, and the lack of confidence can have significant effects on someone’s career. But finding any benefit from such an effect can go a great way to helping the individual cope with the various negative issues. There are various quotes that say “We cannot control how we feel, we can only control how we respond to those feelings”
Overcoming a fear of rejection
Getting work is an essential part of any person's career, whether you are an industry veteran working as a full time employee or a newcomer looking for freelance jobs. We all have to sell our skills to get the jobs we want to do.
Part of that process has always been contacting potential employers or clients and presenting yourself as a good prospect for the work they need done. The down side of this process for most people is the realisation that the threat of rejection looms over every application.
It is a very real condition for most of us to find rejection uncomfortable. Years of experience in an industry does little to make a "no thanks" letter more fun to receive. The consequence of this is that many of us are reluctant to even try.
So, here is how I overcame my fear of no.
Many years back I needed to make more efficient use of my time as a freelancer. I needed to organise my schedule better. I set up a simple kind of game with myself.
Each Monday I would write myself a To Do List on my tablet for the week. I would include everything I thought I needed to get done. But I had a trick. Each time I completed a task I would write next to it DONE.
Now this may seem super obvious, but it was the emotional response I would receive by typing in those four letters...ALWAYS in capitals. It quickly became Pavlovian in nature. I would want to add more tasks to my week because I got huge satisfaction from completing things all for the simple reward of typing in done.
I realised recently I could apply this to job and client applications.
In this case I knew I needed something with more impact. Those dreaded rejections were a huge blow to ego and self image, so I decided to bribe myself. I changed the value of a NO from something potential hurtful and ego-bruising to something with actual value. One Dollar to be exact.
I created a spreadsheet for my new networking push, I had done spreadsheets like this before, name, company, date contacted etc. But this time I added a new column; the NO column. If I had no reply within 2 weeks or received a direct rejection to a request I would place a cross in the no column for that contact. Each NO was worth a buck, but I had to collect them in batches of 50 to be able to redeem them.
The Day I started this I sent out 50 contact requests. That was more requests in one morning than the previous 12 months combined. I was actually excited to send them out, frankly I was looking forward to getting back those NOs, if I got enough that was 50 bucks I could spend on a hobby interest.
Everything we do is based on our own self worth and how we motivate ourselves can be critically important in our success. I am probably one of the worst people I know for dealing with things like rejection letters, and yet such a simple ploy, such a small deceit has flipped my brain into not caring about potential rejection letters and in fact seeking them out.
This does not mean you approach unlikely connections or apply for things with anything less than your absolute best, but it takes a massive amount of the sting out of the tail of any NO you might be on the receiving end of.
try it, and I sincerely hope it helps.
It may not be what you think it is
I think one of the greatest issues with the word "success" is that the modern definition of it has been wildly distorted to a point where it is practically unobtainable.
Modern media shows us success in the form of AAA Hollywood actors, or chart topping musicians, or billionaire tech company founders. In reality these people are aberrations.
Anyone who has ever studied statistics can tell you that the first thing you do with a bunch of figures is, pretty much, ignore the very extreme examples at both ends. So out of a planet of 7 billion people a few hundred who are massively successful in their creative field are largely irrelevant to what normal people can achieve. And yet we set them as the bar we are supposed to reach.
"Aim high" is a reasonable and useful thought process when you start a creative career. Ambition can be a really good thing if it motivates you to work hard and helps you focus on a goal. But aiming to achieve a level that is essentially impossible means you may never be happy with the things you do achieve.
We can get trapped, forever looking to the top of the success ladder. I find myself often discounting achievements I should have celebrated because I was too busy looking to the next thing. It is really important to stop, look at the things we accomplish and acknowledge their significance.
Be Inspired by Real Life
I felt the need to write this article after what can only be described as a single perfect day. The recent long weekend for the Queen’s Birthday public holiday in Victoria, Australia, was a great chance to get out of the studio and spend a few days away from email, computers and sound recording in the Victorian Alpine region. Any time spent away from work has its benefits, but when you get three days of perfect weather, in a beautiful environment, with your favourite person in the world (my wife), a simple break from work can become so much more, and I think it is something that we as creative people should allow ourselves to experience for both the benefit of our health and the benefit of our work.
There was a blog entry recently by an Australian Indy game developer; Andrew Goulding of Brawsome games.
Andrew’s point was that an essential part of being a good developer is to maintain your physical and mental health and this is very true. I think as people who generally work in a creative industry, we need to go further than that and it took a wonderful day out in the country for me to realise the value of such an experience.
Many are used to slogging away in our studios or offices, rushing around to meetings and generally working hard to meet deadlines and improve our skills. These are all necessary parts of a creative industry, but I think too many of us are guilty of using our busy lives as excuses to forget what makes us creative in the first place. Creativity is a resource, and like many resources it needs to be replenished or we risk running on habit rather than being inspired to create at our fullest potential.
Years ago my French horn teacher told me that you had to live life, experience as much as possible simply to allow yourself those experiences. Go out, he said; get drunk, fall in love, fall out of love, enjoy the sun, dance badly, dress worse. He believed that if you had never experienced heart break how could you ever convey it to others? His words stuck with me and I was fortunate enough to understand his words and place significant value on them. What I realised this past weekend while I was away in the beautiful sunshine was that I had allowed real life and work to prevent me from experiencing as much as I could.
We did a variety of things over the course of the long weekend, and it seems strange to analyse a holiday, but some of it is worth discussing. On the Saturday we had a magnificent breakfast, sitting outside in cold crisp 4 degrees Celsius. The food was great, the sun on our backs kept the chill of the air at bay and the view of the mountain from our table was fantastic. We then drove up to the mountain and went for a short hike to a lookout.
My wife and I spent three years living in Japan, we hiked up mountains almost every week in all kinds of weather and yet standing at the lookout last weekend, looking at the snow on distant peaks it was like those memories belonged to someone else. The reality of looking with my own eyes at such a wonderful view was not only inspiring in itself but it brought back fully all the inspiration of the mountains of Japan. This was much like tasting a favourite food after many years and remembering why you liked it so much and wondering why you had waited such a long time to experience it again.
The next significant activity was a decision to hire a couple of bikes and ride to the next town. Victoria has created some fantastic bike tracks along old disused rail lines. This means the tracks generally follow a fairly flat path and are away from the busy roads. The track between the two towns we were taking was just over thirty kilometres each way and wound along through a valley between mountains following a river. Yes it was as gorgeous as it sounds, even more so actually. The bike I had chosen was different to my usual bike and took a bit more effort to ride. This meant that by the time we had returned and done over sixty kilometres my legs were getting really sore, in fact my thigh muscles were in agony, but even this felt like an appropriate part of the journey. The ride itself was nice; the speed of cycling allowed us to really absorb all the surrounding countryside, but more than that we had the time to feel the temperature shift as the day progressed, we could smell the fallen autumn leaves as we rode over sections of track blanketed with their colour and the water in the river was crisp and clean, straight off the surrounding mountains.
I have purposefully tried to use colourful and evocative language in describing these events. This is how we often see experiences portrayed in the projects we might be working on, but when was the last time you allowed yourself to really experience things in such a way. The histories of many writers, poets and artists speak of their times spent in remote locations, or far away countries where they have time and solitude to write, but also where they can immerse themselves in experiences worth writing about.
I won’t go into details of the final day of our break; in some ways I may be worried that committing it to paper may lessen it, and in all honesty there was not that much that was really significant. It was, however a day so full of the simple pleasures of good food, good environment and great company that I felt myself practically delirious in my state of contentment. It left me feeling so refreshed and revitalised that not only did returning to work not bother me, but that I felt I was ready and well prepared to conquer the remainder of the year and here I think is an important detail. My time in Japan was so full of inspirational experiences that I think it was an essential part of me starting my own company. I was filled with such amazing creative thoughts that I was driven to make as many of them happen as I could. This recent weekend has refired that motivation within me.
The thing that drives us to create, that motivates us to work hard and to keep working comes from within, but I think it draws its energy from the real world. I have seen great artworks and read amazing stories and played awesome games and all these things have encouraged me, but getting out and listening to live music, climbing a real mountain, or sitting in a theatre listening to live actors has something that we can’t package on the internet and that sometimes we just need to get out and away from our work and allow the real world to reinspire us.
Article on game narrative
Is story becoming more important than gameplay?
Observations on modern game design, casual players and the future of game development.
I'm no longer hardcore
I have recently been extremely busy and as a result I do not get to spend as much time playing games these days as I would like to and as I did in the past. This means that I am now officially what marketing departments refer to as a casual gamer, although I don’t seem to fall into the usual demographic of family with kids playing tennis on the Wii. The first thing this suggests to me is that there are probably several levels of “casual gamers”.
As more of my generation get older and get busier we can no longer spend the many hours we used to invest into a game, but we have not all become what marketing people usually refer to by the casual gamer label. Sure we don’t have a lot of time to play games, but unlike many causal gamers this does not mean we want to play family orientated titles with a simple control interface. We still want the hardcore story driven immersive experiences we have played in the past, we just want them in more bite size chunks, and this is where I think there is a market that is at risk of being overlooked.
I have played several games in the last year, some which I enjoyed greatly and a few too many that I felt let down by. Because of my limited time I tend to choose very carefully what games I want to invest my time in. I love good stories, and immersive environments with both nice graphics and of course good sound. There have been plenty of titles that seem to fit this bill in the last few years, but as the casual gamer I have also discovered what I see as some serious issues with game design as well as some really wonderful gaming experiences.
The Force Unleashed by Lucas Arts
The Force Unleashed was an excellent game spoilt by what should have been the highlight of the game. As a Force user, the ability to rip a Star Destroyer out of the sky should make any fan scream with joy. Instead a clunky and confusing control interface made this the worst part of the entire game, and the point at which I gave up and shelved the game for good.
Please don’t set me up for a fail
Recently I have played three games that I think have a game-flow issue; Mass Effect, The Force Unleashed and most disappointingly Batman Arkham Asylum. The issue with all of these games was that they were so good, and so enjoyable to play, right up to where I came to a point I could not progress past, and that’s what made them frustrating. In all three there was a certain element of game-play, or a mission or a fight I was unable to overcome. Now there is the obvious response of “Well just toughen the heck up and get it done! Play that game till you conquer it” but this is part of the issue. As someone who is time poor, I choose how I spend my leisure time fairly carefully, and I want my game time to be about good story and an interactive experience, not being punished because I no longer have the reflexes of a twelve year old ninja, and more importantly the free time of that same twelve year old ninja. If I find myself with a spare hour to sit down and plug through a level or two, I don’t want to spend that time repeating the same boss fight or level I got stuck on last time, it’s not much fun and it certainly isn’t relaxing.
Of the above titles, Batman was certainly the best because each time you failed in a task it would reset and offer a hint as to how to progress, and generally it was a far more forgiving game. This I thought was excellent as it allowed me to experience the story which was why I was playing the game in the first place, and this is my main point. I think we have reached a point where modern games with stories need to allow the player to get through the story as this has become the focal point for many players. The excellent stories and enjoyable interactive experiences are why many of us are playing these games.
Rocksteady Studio's Arkham Asylum
Batman Arkham Asylum is so well designed and implemented that it was a real pleasure to play. Its forgiving nature and good control system make you feel like you are controlling a competant character. Although there is a "roadblock" section that I find frustrating and unnessesary, the rest of the game has been so well made that I will persist and spend the time to continue as I think the remainder of the game will be worth the effort. Sadly this is not a statement I can make for all games.
Batman was a surprise to me; I did not particularly want to play a Batman game as I see the character as a heroic figure, not a game avatar that bumps into walls, falls down holes and has to spend his time looking for golden keys. The game however is very little of these things. Batman is indeed heroic, if you as the player fall down a hole there is usually an escape button and his movement is generally fluid and effective; he feels competent. This is where I think games need to go. I WANT to play a hero, I WANT to feel like I am guiding a competent character through a dangerous world, I really don’t want to feel like I am controlling a bumbling fool simply because I did not press the jump button on the exact pixel between two platforms. I am perfectly capable of being clumsy in my real life, I don’t need my game characters to follow suit.
I am certainly not asking or expecting games in overall to be made easier, but considering the focus of many modern games is a story line, I would like to know that I can see this story through to the end. If for whatever reason I find there is a point I cannot proceed past, then the game should allow me to either progress anyway, tell me exactly what to do, or lower its difficulty level so far that I cannot help but get past the obstacle. It is extremely difficult for a developer to predict every possible way in which a player may approach their game, or the potential issues they may have with it, and this is not referring to any bug or design flaw issues, it is just looking at the vast difference between how people think. Because of this it seems to make sense to have an escape key that allows the player to always progress.
Mass Effect by Bioware
For me, Mass Effect is probably the most dissapointing as it was such a good game. To place a boss fight directly after an unskippable cutscene of a few minute long is capital crime number in game design and should NEVER happen. It's bad enough that you might have to fight a certain boss over and over. To place it directly after a cutscene so you can't save, and then make the cutscene unskippable, sorry you deserve to rot in gaming hell with only copies of ET on the Atari 2600 to keep you company.
The better you make them, the pickier your audience will be
I consider Mass Effect to be the perfect example of why this is relevant. I had waited for quite some time before I purchased an Xbox360, when I finally did I was really looking forward to Mass effect. I had always enjoyed Bioware games and this looked very promising. I was really happy to find it lived up to my expectations. The game-play and graphics were excellent, and the general feel of the game was good, but the story and acting was where I really believe the game shined, it was excellent, and made playing the game a real pleasure. That is until I got stuck on an annoying boss fight and couldn’t progress. I was not far into the story either. As a result, Mass Effect 2 which has just been released is not even on my consider list, as I see no point in buying the sequel to a game I couldn’t finish. Now again, whether I should have been able to get through the game or not is not the point, if I had issues with it then it’s almost certain other people did as well.
Five or ten years ago this situation was less relevant. Games with good stories have been around for a while, but in my opinion we are only just reaching the point where games are getting the budget to make really good use of the story elements. “A” list voice actors, well written scripts and story elements and current generation graphics and animation have resulted in a generation of games that are viewable for their own sake. My wife and I often take it in turns to play through the games we purchase. Often I won’t have time to play a game at all, but I still enjoy many current games as a spectator while I do other things and this is primarily because they have good stories.
But when it's right it's oh so good
2K's Bioshock 2
I don't have a paternal bone in my body, so any game that can make me protective of a young child and send me into a rage if she is threatened has really pressed the right buttons as far as story and atmosphere.
Bioshock 2 has so far been one of the most enjoyable games i have played in a long while.
At the time of writing this article I am several hours into Bioshock 2 and it is incredibly enjoyable. The graphics and sound are excellent, but I find it is the story elements that I am enjoying most. The little snippets of people's diaries presented as recorded logs, and the dialogue of the main characters add so much to the game. I am fairly confident I am unlikely to come across a show stopping issue as I played through Bioshock 1 completely and had no issues; in fact many people complained that Bioshock 1 was too easy. Bioshock 2 allows you to switch off the ability to simply respawn from the resurrection chambers and with this one simple adjustment they can cater for both a player who wants a challenge and a player who wishes to be able to complete the game for the sake of story like myself. I think this is an excellent compromise and something that should become standard for all games.
The points I raise are more important for a game with a heavy story element than a more simple action game. If the fun is in action and destruction then I can understand that there is less need to progress to the end of the game, but for story games, please give us all a chance to see the fantastic work you have put into the entire game rather than risk your audience giving up half way through frustration.
There are numerous ways to set up equipment when recording either in a studio or out on location, how you set up your gear is as important as choosing the right microphone and knowing how to edit your sounds afterward. Personally I find that how I setup my gear is a constantly changing part of the job. Experience counts for a lot and often you won't know if something is going to work until you've tried it. New gear will often need new solutions, and recording material you have never encountered before will almost always challenge your current approaches to setting up for a recording session.
I don't believe there are any perfect solutions that work for every situation, I try to approach any session with an open mind and see if I can learn something from it. These are some of the solutions I have used and how I found working with them.
Using Multiple Microphones
Before I go into the various setups it might be worth explaining why I would want to use more than one microphone to record something. There are a couple of reasons why this might be desirable.
Firstly different microphones are made for different purposes and they all have an area where they will be more suitable. The sensitivity and pickup pattern will both influence what a microphone is able to record, while its design and construction will influence the quality of sound it will capture. Words such as warm, crisp, clear and full are all used to describe some of the sound characteristics of different microphones, often it is just a personal preference but sometimes there is a very clear choice of which mic is better for a particular situation. Sometimes it is worth using multiple microphones to try and capture all of these sound characteristics, this allows you to mix the various recordings together later and create a final sound that is warm, crisp and clear. The more material you capture, the more options you have to work with.
The second aspect is physical ability to capture certain sounds. If I place 5 different microphones at the muzzle of a canon when it is fired it is very likely that some if not all of them will not be able to capture a clean recording. Microphones have limitations to how loud a sound they can capture as well as how fast they respond. A gunshot is a very loud very fast impulse so a suitable mic is needed to capture those elements. The after shock and echo however have different qualities so a different mic might work better. This is less about what might capture better sound characteristics and more just capturing the sound at all.
Multiple mics on a single sound source
Finally there things that are just so complex that you need more than one microphone to capture all the sounds. A car is the best example of this. A Hollywood sound team would use at least half a dozen mics to record a car driving. usually you would place at lest one microphone near the exhaust, one int he engine bay, one or more inside the car and then have several others being carried and pointed at the car from a distance. In extreme cases you might then add another to capture the tyres on the road, one near the suspension and several in the engine bay to distinguish between piston noise, fan noise, and pre exhaust output. Each of these microphones might be a different type to bets suit its purpose.
Boom Pole Setup Japan
The main reason why I adopted this setup in Japan was because it was the gear I had with me at the time. I had previously used a Rycote windshield and shock mount system, but that belonged to the studio I was working for so it stayed with them when I moved to Japan. Initially I had just the mic a basic plastic clip and the boom pole which I had purchased in Australia. A trip to Tokyo allowed me to buy a simple but very cheap suspension system and a fluffy cover to keep out the wind. Now at least I could do some decent outdoor recordings.
The way this setup is designed it allows for the boom pole to be extended with the mic on the end. This would allow me to get the mic closer to birds in a tree, or lower the mic down a hole or maneuver it under a vehicle. It greatly increases where I can position the mic. The suspension system stops vibrations from traveling to the mic and corrupting the sounds it records.
The Zoom H4 has been mounted on the boom pole by attaching the head piece of an old broken camera tripod. The thread mount on the Zoom is designed to screw directly into a camera tripod, so customizing the head pieces to fit on the pole allows for the zoom to be easily screwed on and off. This means my entire recording setup is mounted on pole and so is light, easily portable and makes it much easier to get the mic into unusual locations. I would usually have headphones attached to the Zoom H4, but once I have set levels I can use the setup with nothing else attached to the pole but my hands. I have recorded thousands of sounds with this setup over a 2 year period and it is a very flexible system. Most of the other setups I use are adaptations of this first concept.
Boom Pole Setup
The current version of the boom pole setup is essentially the same. The basic elastic suspension system has been replaced with a Rode Blimp system. This works much better at isolating the microphone from vibrations and suppressing the effect of wind noise. Wind is probably the biggest issue with outdoor recording so having an effective measure of countering it is well worth the time effort and money to find.
The H4N has a screw thread hole in its body case so it doesn't not need a separate mounting plate like the H4 did, it just attaches directly to the tripod head piece mounted on the pole.
This system is slightly heavier than the setup I used in Japan because the H4N is heavier than the H4 and the Rode Blimp is slightly heavier than the simple shock system. But overall this extra weight is very slight and the advantages of the new setup are quite considerable.
The lessons I learnt adapting the boom pole setup to be practical and easy to use made the tripod mount a natural progression of the idea. Essentially I like the idea of having self contained gear setups without cables getting in the way or getting damaged. The main recurring issue I had with the boom pole was the inability to use both hands when I was recording. On occasion I would prop up the boom pole, or tie it to something so i could work hands free, but these were always less than ideal solutions.
I needed two things to make this setup work. The first was easy, I went to the hardware shop, took the H4 Mounting plate with me and found a screw that was the right size to screw into the plate. The second thing I needed was a little harder to find. I wanted a screw adapter that would go between the thread size of a camera tripod to the larger thread size used for microphone clips and stands. Thankfully my local pro audio shop has a large range of various adapters and after 30 minutes of looking through catalogues we found what I needed. I just had to wait a few days for them to order it in.
The end result is a screw mounted on one of the tripod legs, this allows the H4 or H4N to be mounted onto the leg of the tripod. The screw adapter on top allows me to mount the Rode Blip (or any microphone clip in fact) to the top of the tripod. This creates a totally independant hands free recording unit which I use very regularly. It also very conveniently attaches to the tripod carry straps on the back of my gear bag, another advantage of buying a bag designed for camera equipment. If I was going to do a large scale project that required a large number of microphones I think I would just replicate this setup over and over rather than have dozens of cables running everywhere.
Single Hand Setup
This setup came about from a desire to be more mobile and have the option of adding an extra microphone. The boom pole was a very useful setup, but it relied on the pole itself to hold all the components together. This became an issue when I was in a situation where the pole itself was not only not necessary, but actually got in the way. My visit to HMAS Castlemaine was a good example of this. A typical warship does not have large open spaces and nice wide doors that make wielding a big pole practical, so I needed to work without the pole.
I had experimented with making various hand grips with thread on both ends so I could attach mics, it generally was more trouble than it was worth. Eventually I used the Rode blimp and screwed a short piece of threaded rod into it that allowed me to add a second mic to the bottom of it. This would allow me to hold the blimp as usual but have an extra mic attached. It was occasionally useful, but became less useful after I lost my Shure Beta 58A as that was my main secondary mic. (It was great for recording animals)
The system now is quite simple. I attach the Zoom H4N to the Blimp by wrapping Velcro straps around them, this allows it to be removed quickly and easily and does no damage to either piece of equipment. It allows me to utilize all four input channels of the H4N, 2 leading from the Blimp with one of my shotguns inside and the two built in mics on the H4N. The Redhead wind shield blocks the wind on the H4N mics and means I have a stereo backup of everything I record. When using a single microphone into a stereo device like the H4 or H4N I set the left and right channels to different levels. By setting one channel at a lower input level it covers me if there is a sudden loud sound. Anything on the main channel that peaks is usually captured on the lower channel without peak distortion.
The Blimp setup is a suspension mount and wind jammer, it is one of the most important pieces of equipment I use, but perhaps its worth explaining why and showing how it works. Essentially it is designed to isolate a microphone from vibrations and wind noise. These two factors can be responsible for making recordings completely unusable and so it is worth the effort to mitigate them as much as possible.
Close up of suspension system
The initial part of the system is a pair of clips that you attach to the microphone you wish to use. There are various brands of blimp system available and each will come with a series of different sized clips to allow you to use different sizes and shapes of microphone. The most common mics used in blimps systems are hyper cardioid or "shotgun" mics, but it is possible to use any microphone that will fit into the clips. The clips themselves have small hooks that are attached to special rubber bands. These bands are then attached to the suspension mount. When in use the rubber bands isolate the microphone from vibrations caused by user handling or the environment it is being used in.
Mic mounted in suspension system
The microphone can be used in the basic setup and is often appropriate for inside close micing. This setup allows the user to handle the mic by holding the pistol grip or mounting it on a boom pole or tripod. Using it in this way allows the mic to be very close to the sound source without the wind cage getting int he way. At this stage the microphone has no protection from wind and so this setup could only be used out doors if it was a very still day. This setup is also slightly lighter than when the cage and fluffy are added.
Wind cage added to system
The wind cage is designed to slide onto the pistol grip and lock in place. The two half sphere ends can both be unscrewed to allow for easier access to the microphone inside. Once the cage is correctly positioned the ends screw on and it is ready to use. The cage is designed to protect the microphone from low level wind and prevent sounds from popping the microphone diaphragm in a similar way a foam cover does. The cage also offers some protection against rain and moisture getting to the mic. It will not protect a mic against a lot of water but it will generally keep light rain off. The disadvantage is that the cage can be a slight barrier if you need to get the microphone very close to a sound source.
Wind jammer 'Fluffy' added to system
The wind jammer is essentially a large fluffy sock that slips over the cage. Strong wind is countered because the wind moves the fluffy fibers rather than blowing against the microphone's diaphragm. Because the fibers are so soft they make no no noise when they are moved by the wind. There is still a limit to how much wind can be managed but generally I find this system can cope with very strong winds.
As I mentioned in the section about using multiple mics, vehicles are often so complex that they require a completely different approach to record. Capturing engine sounds can be difficult because they are often very loud and finding secure, safe places to mount microphones on a moving vehicle can be a challenge.
While there may be some locations on a vehicle that are generally the same like the exhaust, the engine and the interior it is rare to be able to use the same setup each time. I have found that the higher quality finish and aerodynamic designs of sports cars often mean there are few places to attach microphones to and if you are dealing with an expensive car you are going to be far more conscious of marking or damaging the car in any way. For this reason the best car to test on is your own, get used to the process and experiment with the best positions for mic placement. Things will change when you work with different vehicles, but the basic principles you pick up from your own tests will be helpful.
Article on repetition
Boingy Boingy Boingy
Why I hate repetition and what can be done about it
I suspect it is the time I have spent in the video game industry that is the main cause of my hatred of repetition in audio. PCs and video game consoles have always had limitations. The amount of storage space on a disk, available memory, seek and read times of disk media (CD, DVD and Blueray) are all examples of some of the many issues developers need to deal with when producing games. Because of the limited resources and the need to share these resources amongst all aspects of the game, usually it is necessary to design a project that can cope with resource limitations. For game audio this often meant less variation than might have been desirable.Ever since I started working on games I have been aware of their limitations and that sometimes some repetition is unavoidable, however far too often I have played games that do not seem to be pushing the resources of the relevant console and yet still have overly repetitive sound design. One of the worst instances I encountered was a PS2 game that literally only had two footstep sounds, and the developers had also implemented them too loudly, so the entire game was accompanied by this clip, clop, clip, clop, clip, clop of the main character. For me it made the game unplayable after only a few hours. I am happy to admit I am pretty fussy where game audio is concerned, but it really spoilt what was otherwise a fun game.
So why am I so distressed by a little repetition? Think of any sound that might repeat itself, and then think carefully about that sound. A dripping tap, bouncing a ball and footsteps are all good examples of sounds that repeat cyclically, but if you really listen and analyse the sounds themselves you realise that they are not exact replicas of the same sound. Bounce a ball sometime and see if you can get exactly the same sound twice in a row, let alone for longer. Each bounce is subtly different, the downward pressure you apply to the ball, the angle on which it might land and the surface on which the ball bounces will all alter the sound, they may not alter it much, but it doesn’t take much variation at all to break up a repetitive cycle. In fact it’s the super subtle differences that really make the difference. If you isolate a single tic toc from the sound of a clock, loop it and listen back to it, it does sound like a clock, but there is something about it that just doesn’t sound quite right.One of the issues with modern video games and CG films is that they are becoming very close to being indistinguishable from real life; close, but not yet completely there, and that’s one of the problems. The closer you get to making something that reflects reality the more the subtle differences stand out. So with audio, a bouncing ball or ticking clock sound that repeats is pretty close to how a ball or clock may really sound, but the subtle difference tells the listener that what they are listening to is not the real thing. In this case and to help create many other types of repetitive sounds, subtle variations can go a long way to convince a listener they are listening to the real thing.
To use the ball analogy lets assume we are creating a basketball game. One of the main audio elements we require is the sound of the ball being bounced. With sound files time equals memory, at 44.1 KHz, 16 bit mono, a ten second long sound is 860Kb, this is almost a megabyte of memory for only ten seconds of sound. So we would not want to record long samples of someone bouncing a ball to add to the game as we would quickly run out of memory. Alternatively if we use a recording of a single bounce it will result in very obvious repetition.
If we use 5 or 6 different recordings of a single bounce they can be used as a pool of sounds to draw from. Randomly selecting a different recording to play for each bounce event will instantly add more variety than using a single bounce. Adding a very slight variation of pitch each time a ball bounce is played will produce more variation and represent slightly more forceful or less forceful bounces. Finally adding a very slight volume variation to each bounce will add even more variety. In the end these three simple steps can turn a selection of 5 sound files into a sound event that can play endlessly without ever sounding like it is repeating. This effect has been achieved using a small number of files that will not use much memory, this is particularly important for smaller consoles like a Nintendo DS or for iphone apps.
A single bounce sound file, looped to repeat four time
No variation in pitch, volume or time.
Non repeated bounce
Five unique bounce sound files played in sequence
Each bounce is different in sound and pitch, volume and time spacing.
When creating slightly more complex sounds there are more ways in which random elements can be added. Footstep sounds are created from the sound of the particular shoe impacting on the ground and the reaction of any materials that are stepped on. So a boot walking on short grass would only have a basic impact sound, but walking on gravel or leaves would produce the sound of the leaves being crushed or scattered, or the stones moving. Because there are essentially two layers the sound event can be created by combing two sound pools. The first would contain a selection of boot impact sounds to randomly draw from, and the second a selection of leaf sounds. If we have five sounds in each pool the number of combinations suddenly increases dramatically. This simple solution can effectively remove repetition almost entirely.
I want to return to why I dislike repetition, and look at it from a different angle. As I mentioned, initially it was from frustration at being presented games that appeared to have poor or lazy sound design. As I looked into solutions to this issue so I would not fall into the same trap I realised that the randomisation of sound might even improve the overall feel of a game. Several years ago I read an article where some film makers were previewing their latest project to a test audience. They showed the audience two versions of the film and asked for their reactions. The vast majority of those surveyed said the second version was much better, they claimed the special effects were better and the pace and editing was improved. The amazing thing was that visually both versions were identical. The only difference was the music and sound effects used. Now obviously this can be seen as a good example of how important sound is for supporting visuals, but more than that it demonstrates that sound can even alter how someone perceives visuals. It has been well documented that altering a musical score can seriously change the feel of a film, adding ominous music to a scene can make even the most angelic character seem disturbing and playing the theme to Benny Hill has the disturbing effect of making the most horrific footage feel like a comedy routine. Music affects human’s emotional state to a level we still do not fully understand, so it is not surprising that it can have this kind of influence when accompanying a movie clip, but what about sound effects?
Recently I was asked to write an article about using FMOD Designer for sound event creation. I have been using FMOD for many years as it allows me to create sounds with a large level of variation. For the purposes of the article I wanted to include a film clip of an explosion. Most people respond to visuals, so adding a film clip to watch helps convey the idea more easily. I chose a clip from a friend I have worked with of a mine exploding and added an appropriate sound effect. The effect was generated in FMOD as it would be in a game situation. I had set up the event to randomize the sound each time it played, so the explosion would always sound different. I recorded a series of this sound so I could choose which version I wanted to add to the clip. When it came to editing the clip I found lots of versions of the sound that worked really well. I added one to the clip and uploaded it for the article.
I realised that this single clip did not demonstrate FMOD’s ability to randomise the sound each time it was triggered, so I made a second clip where I edited the explosion to repeat 4 times. I added a different version of the sound to each explosion. This clearly demonstrated what FMOD was capable of, but when I previewed the clip I realised it also demonstrated something else. The sounds were different enough that they even give the impression that you are viewing four different explosions. Part of this is the way I edited the clip, but most of it is a result of the sound effects. For me this really highlighted the fact that sound and visuals are more closely linked than we give credit for and that one can often influence how the other is perceived.
For a long time I have worked towards reducing or removing repetition from game projects wherever possible, and I always try to produce high quality audio environments, but the example of the multiple explosions showed me how audio could go even further and be used to not just highlight visuals, but even alter how they are perceived. I would be interested to work on a game project where a level didn’t change visually but went through a series of transformations for the audio environment and see just how far you could alter people's perceptions without changing the visuals. Games are fantastic things to work on creatively because you can create worlds where almost anything is possible. Maybe you could even make a game where lots of repetition was entertaining, but I am doubtful.
Stephan Schütze has been recording sounds for over twenty years. This journal logs his thoughts and experiences