To be honest, when we first started making Ghost of Tsushima, I was probably most excited about the possibility of traveling to Japan to capture ambience. We sent two teams over to Tsushima and throughout Japan but I didn’t get to go. It kinda made sense. These were more sight-seeing tours for environment artists, character modelers, and some of the other creative leads to get a sense what Japan was like. If I was going to go I needed isolation, not driving around in a van with 10 other people.
On the first trip, I gave Billy Harper, our character and animation lead one of my recorders and a brief tutorial on how to use it in hopes he could maybe get away and record some interesting stuff. I wasn’t expecting anything useful but Billy surprised me. He got some usable recordings of birds including some black kites, songbirds, and of course, the ubiquitous jungle crows. The team went to Tsushima and attended the Komodohama festival which commemorates the Mongol invasion landing on Tsushima in 1274. Our team was invited to the junior high school on Tsushima and Billy and our cinematics team recorded several instruments including various taiko drums and a horagai, a conch shell horn that was used by samurai to communicate in battle.
Being part of Sony, we have a great team at Japan Studio, so we reached out to them and ask if they would be willing to record some ambience for us and they did a great job. The team there drove out near Mount Fuji to capture all sorts of birds in the wild as well as some rivers and wind. They also went to the Kachoen Wild Bird Refuge and got tons of species isolated. One of the sound designers, Ito-san went on vacation to visit family over in Akita prefecture on the Northeastern side of Honshu, the main island of Japan, and got some fantastic recordings of songbirds at dawn and during the say in some rice paddies.
All of these recordings were great and totally usable, but we were still missing a few specific species of animals we knew were going to be in the game, so I decided fairly last minute to spend my Christmas and New Years traveling around Japan on a whirlwind trip chasing after more specific species. My partner, Bonnie, joined me as travel partner and documentarian. I had planned a 10 day trip for us. We would land in Tokyo one evening, then fly out to Hokkaido the following morning, record the tancho (the red-crowned cranes), travel to Lake Kasharro to record the Whooper Swans, then back to Honshu. We’d spend New Years in Tokyo, then head up to Nikko, followed by a trip into Nagano prefecture to try and record the snow monkeys.
The entire trip was absolutely incredible. Japan is insanely beautiful, the people exceptionally friendly, even when there’s a language barrier, and the food was always fresh and amazing. The entire trip was a highlight, but also included some fun surprises. When we were in Nikko, we checked out the temples as you do, but then hiked up a mountain near the Kirifuri Ice Arena. I set up my recorder to capture some ambience and walk a little ways away. A short while later we were startled by what sounded like a piercing alarm chip. It happened again and then once more. Bonnie and I looked at each other and mouthed “What the fuck was that?” Suddenly we saw two huge sika deer run by in the valley below us. The continue to chirp their alert calls for the next 10 minutes. Those recordings feature prominently in Ghost of Tsushima.
There were some other cool moments that didn’t make it into the game as well. In Kushiro on Hokkaido, the Kushiro River was half frozen with large sheets of ice flowing down the river and crashing into each other. One night I threw my hydrophone into the river, hoping to get some interesting underwater sounds of ice creaking, crunching and colliding, but there was nothing quite so interesting.
After compiling all this great content from so many people, I made a movie to share with the team as part of one of our team meetings. My attempt to evoke David Attenborough is pathetic, but the video was a fun glimpse into just how global the effort was to capture realistic and accurate ambience for the game.
The natural world we built for Ghost of Tsushima is one of the sonic highlights for me. The world is lush and full of so many different species of birds, insects, amphibians and mammals, many of which were recorded in Japan and most of which are native to Japan. It really creates this wondrous natural beauty for the rest of the game to sit on top of.
We had our friends at Japan Studio in Tokyo record a lot of ambience for us and I was fortunate enough to travel around Japan for a couple weeks and capture all sorts of wildlife. The ambience was one of the more time consuming systems in the game, but it was also one of the first to get solidified. We had good variety of wildlife species and the world felt rich, but as I explain in the video below, getting towards the end of the project they decided to add a few new specific species. We didn’t have time or resources to fly halfway around the world and try to track these specific birds down, nor could I find a library with them.
Fortunately, the Slack field recording channel was doing a bird crowdsource around this time, and while discussing our recordings, Alex Barnhart asked if anyone had tried pitching down real bird recordings, performing them, and then pitching them back up to see if you could replicate birdsong in that way. We all thought it sounded fun and cool and it was very shortly after this moment, that they decided to add 3 new species to the game, so I took this knowledge and went for it.
I went onto xeno-canto.org which is a fantastic website for information and birdsong and found some decent quality samples that I could use as reference. From there I tried pitching them down to various ranges from an octave to four octaves and everywhere in between, trying to find that sweet spot where it was in my vocal range and performable. After performing, and looking and sounding appropriately goofy, I was pretty surprised and pleased when playbacking the results. Once I put them in game, I knew this was going to work!
They even added a couple more species (a cormorant and a Eurasian sparrowhawk) that I was able to perform adequately. The one place I failed was when trying to augment our existing Black-naped oriole recordings. This is a really important bird in the game, as he guides you to your various objectives. I had captured one while recording in Sri Lanka, but my recordings were limited, so I hoped to bolster our content with some faked versions, but when playing a fake bird against a real bird, it becomes VERY evident which one is fake, so I abandoned it.
The results were so surprisingly decent, we played a game at work where the team listened to bird samples and tried to identify whether they were a real bird or me, and it was hard. Even I got half of them wrong! So here’s a video that goes into more of these details and also shows the process in action:
One of the first decisions made on Ghost of Tsushima was that our main means of fast locomotion in the game would be horses. It made sense since that was what was used by both Mongols and Japanese warriors in the 13th century. There would be no grinding power lines or zipping through air vents or beaming from one radar dish to another. So of course we knew the horses needed to sound great.
Surprisingly I think the only other game I’ve worked on that almost had a horse was a Shrek game that got canceled. In that game in a very on-the-nose homage to Monty Python, there were knight enemies that rode around on hobby horses. For the knights footsteps I recorded coconuts and the results were as hilarious as intended. So for fun, when we first got the horse working in game, I re-recorded the same coconut shells and, while still hilarious, it didn’t quite fit tonally.
We began to research some places to record horses. The most important thing was we needed shoeless horses. While there was conflicting information about whether or not Mongols shoed their horses, we could find no information demonstrating the Japanese shoed their horses with metal, so we opted to keep things simpler and go for natural footsteps. After calling some places that fit the bill and driving around to check out their properties, assess noise levels, etc., we decided upon the Northwest Natural Horsemanship Center out in Fall City, WA. The owner Jim Hutchins, was keen to work with us and seemed genuinely interested in our work.
Unfortunately between the time we agreed to record there and the date of the recording, my sound designer had left for another position, so I was on my own for the session. One fewer person to operate microphones posed some problems, so I got creative: I followed the horse with a boom mic and strapped a recorder and a pair of omni electret condenser mics (a Roland CS-10EM) to the saddle which we taped down (and taped all buckles on the saddle).
Horses are amazing animals and when the horse first saw my furry blimp, it was not happy. We got lots of great nervous vocalizations which ended up as the final assets in the game as it got used to the presence of the blimp. From there Jim guided the horse around on various surfaces in their property (grass, tall grass, dirt, mud, wood, gravel, stone/concrete, and asphalt) at trot, canter and occasionally gallop speeds and I chased after them with my mic. We got A LOT of horse grazing because when a horse is hungry and has the lay of the land, they eat when they feel like it. These assets too were eventually massaged by Erik Buensuceso and made it into the game. In fact, the amount of grazing sounds we got and the frequency with which the horses naturally ate grass was the impetus for us adding the grazing animations into the game!
To get bigger whinnies and neighs, Jim put two of the horses together who were really good friends, and then separated them. Once one was led far enough away, the other would bellow out a very loud call for their friend. I inadvertently recorded a horse fart at one point, but was too far away so unfortunately it was not usable. Lastly, we went into their gear barn and recorded a bunch of sounds of various bridles and saddles, again taping down any metal parts since we wanted to avoid jangling components in the sound design.
Once I chopped up the assets and integrated them into the game I made this delightfully silly video to show at a company meeting:
But we were still a long way from being done. The horse had to sound great because it was used SO much in the game and we wanted to really push on the detail of everything about the horse from its footsteps to its saddle and bridle sounds to its vocalizations. I think when all was said and done the assets for the horse were probably touched by almost every sound designer on the project. I can think of at least 5 of us that did some work on the horse, tuning and improving and iterating to make them sound great.
On the mix side, Josh came up with a great idea we used on both hero and horse foley. We created an RTPC that tracked how long the player was running or the horse was galloping, which we used to subtle mix down the non-footstep sounds after 10-15 seconds. This served the purpose of helping reduce ear fatigue and allow other sounds to cut through the mix on long traversals. With this attention to detail we got some really nice dynamic behavior in the sounds in game.
The dogs in Ghost of Tsushima are fun to talk about because, like so much of the soundscape of the game, we started with a lot of research and then recording them was a blast. Let’s just ignore the fact that Jin kills so many dogs in his journey and rewind a few years when we first started talking about having dogs in the game. The impetus for adding dogs was to have a unique enemy type that was also more adept at hunting Jin than the Mongols. Like everything else in the game, we did a bunch of research and found that Mongols did indeed use dogs and that from Genghis Khan on, their dogs were ancestors of the breed now known as the Tibetan Mastiff (which technically is not a mastiff, but I’m not here to get into a discussion about dog breeds). So the art team began working on models of Tibetan mastiffs, which meant we needed to find some sounds for them! We started looking online and these dogs looked and sounded fierce, but also fairly unique. So I started looking to see if there were any Tibetan Mastiff breeders in the area and I struck gold: there were actually a handful within a few hour drive from Seattle!
We ended up connecting with Debbie Parsons of Dreamcatcher Mastiffs. She mentioned they had several Tibetan Mastiffs and we were welcome to come record them. So Josh and I packed up our mics and recorders and headed out to Graham, WA, just an hour south of Seattle on the way to Mount Rainier. As soon as we showed up, we were greeted by 3 VERY large, scary and seemlingly angry dogs. We started recording immediately and got some terrific barks and growls (unfortunately interspersed with chain link fence rattle as they were pretty excited). Debbie was super helpful and started taking most of the dogs inside and allowing us to record each one individually. We got a good range of growls, snarls and barks. Debbie even ended up putting on a hooded sweatshirt and acting like a prowler to get them riled up. After getting all these great barks, we went into the property to record more and these dogs transformed into the cuddliest, sweetest bears (seriously they looked like bears in dog suits).
We were able to get panting and breathing and even chewing on bones as they had settled down and got used to us. From there Josh took the recordings, cleaned them up and got them in game. Rob Castro did a final polish pass towards the end of the project to make them even more terrifying. As you can hear from this video we created for a company meeting, the were already pretty scary before the sonic magic was applied.
We knew as soon as we began working on Ghost of Tsushima that swordplay would be a major part of the game and that the swords had to sound dangerous. We had been working on some close quarters sword-based combat prototypes and in moving to the world of feudal Japan, we wanted to approach combat from a grounded perspective but also really make the sound of the swords slicing through enemies feel razor like. Lots of games and films go for a really meaty, thuddy impact to make things feel powerful. We wanted to go against this trend and make our swords SOUND sharp.
To do this, we really focused our impact sounds on elements of slicing rather than hard impacts. But of course it took a lot of experimentation and a lot of recording. Josh Lord (our senior sound designer) and I amassed a collection of various blades and other sharp objects and applied them to myriad fruits and vegetables. There were some surprising results: digging into celery with the tip of a knife sounding like cutting through bone; a razor blade through an onion sounded like a gut being opened; a serrated bread knife created interesting zippered slicing tears.
Once we got all this source material together, we started experimenting with integration techniques, adapting a technique utilizing sound states in Wwise tied to our animations to trigger different impacts based on animation data. Towards the end of the project, we passed off our combat sounds to Mike Niederquell from Sony for some final polish and for him to work the magic he does. We ended up abandoning our state-based multi-impact approach and Mike made bespoke content from our recordings for the various impact types as the animations weren’t so excessive, and the state rigging introduced unnecessary complexity.
Here’s a short video of some of the props that sacrificed themselves for the greater combat good as well as how they ended up sounding in game:
Of all the powers in inFamous Second Son, Video powers may have been the most esoteric. I mean smoke at least has an analog in fire (and we used some fire elements in both the visual and sound design), but video? You think video, you may think laser, but we already had a neon power (which was even sometimes referred to as laser). So how the hell did we get something sounding as unique as our video powers without treading on the other power sets?
Part of the answer is interestingly with how the power set itself was initially conveyed to the team. Video power was actually called “TV power” internally for most of production. Heaven’s Hellfire, the video game that Eugene, the video power conduit, is obsessed with was initially a TV show. We realized after many months that it made more sense to make it into a video game instead and that would open up more avenues for us to play around with in the gameplay (such as the mildly retro boss battle).
But we still had “TV powers” stuck in our brain and when Andy and I began brainstorming about how to make sounds that were powerful and unique and “TV like” we started thinking about televisions. We stalked thrift stores around town hoping we’d come across some old 1970s vacuum or cathode tube televisions to take apart and record. We failed there, but Andy eventually came across a couple old CRT TV/VCR combos. Double obsolete points! We brought these into the studio and proceeded to record all kinds of sounds with an array of microphones from shotguns to contact mics to crappy telephone microphones which did an amazing job of capturing bizarre electromagnetic interference around the power supply, and other surfaces. We recorded all possible permutations of power on and power off sounds and even got the VCR mechanisms to give us some very bizarre whines and hums. We also did some recordings of the Sucker Punch MAME arcade cabinet which has a very old CRT monitor in it with tons of wires exposed, as well as a shortwave radio I’ve had for years, but never really needed for a video game sound before.
We recorded all of these sounds at 192kHz and the frequency content of the recordings on the CRT monitors at the higher frequencies was pretty astounding. While some of them we had to remove the >20kHz content to save our ears and speakers, Andy also did some pitch shifting to play around with some of these normally inaudible sounds and they became part of the video power palette.
A few words on the telephone microphones we used: they are cheap and really neat for recording electromagnetic interference. Although Radio Shack may be dead and gone now, you can still get them online. It’s pretty neat the wide array of sounds you can get from one of them by waving near essentially any power source from a monitor to a computer, plugs, etc. Basically any electronic device will give you some interesting content. For a lot of the TV powers, Andy took various EMF sounds and morphed them together using Zynaptiq’s Morph plugin.
So, similar to our other power sets, below is a video showing some of our field recording as well as the final in-game sounds. What’s different here is that the video powers were finalized later in the project and we were so focused on finishing the game, that we did not make a fancy, fun video for the team. So, it may not be as fun as the previous videos, but still shows what we recorded and how it ended up sounding.
It’s hard to believe that inFamous Second Son is a year old already! I’ve been completely lagging on finishing up these posts about the powers design for the game, so let me use this opportunity to make good and present the first of the final 2 parts of this series. I will hopefully get around to posting my presentation on the Systems Design for the game soon as well so those who haven’t heard/seen it can have the information available to them. Anyway, on to the magic and mystery of concrete!
For those who haven’t played or seen inFamous Second Son you play a guy who gets superpowers battling an authoritarian government agency called the DUP whose soldiers are all imbued with concrete superpowers by their leader Dana Augustine (as normally happens with government agencies).
The biggest challenge for us with concrete was how to make it sound unique. It’s just rocks and stone right? We’ve all heard countless variations on rock sounds in everything from impacts to destruction and rubble/debris sounds. We needed to figure out ways to make our sounds stand out as unique, while also conveying the power of the enemies in the game who used concrete.
The powers ran the gamut from concrete grenades to spawning concrete shields to launching off spires of concrete and forming a concrete balcony on walls. In short there was tons of concrete objects being created and broken in the world. Not only did we need these to sound unique and “powered” but they also had to sound completely distinct from all the “normal” concrete in the world you could destroy or collide other objects with. It was a huge challenge, but one that Andy Martin was definitely up for.
The place to start, naturally, was by buying a bunch of concrete. I looked into the process of concrete, which is usually just a mixture of water, an aggregate like sand or gravel, and Portland cement (named after a type of stone used in the UK, not the sleepy hamlet of the Pacific Northwest of the US). While the thought of mixing up my own concrete sounded appealing to my construction worker wannabe side, we weren’t in a position in the project where we had limitless time to experiment. So we did the next best thing: went to Home Depot. Andy and I both made trips to the hardware store and bought all kinds of concrete and stone, from paver stones (which were often too resonant) to clay bricks, cinder blocks, and more. They were demolishing a building across the street from my house and I noticed some particularly large chunks of both asphalt and concrete sitting on the other side of the fence. I waited until nightfall, donned my ninja costume (really just a bathrobe with a scarf tied around my head) and absconded with the almost-final resources we would need to make our concrete powers come to life.
From here, Andy began to run wild and experiment with all kinds of torture he could enact on our various pieces of concrete. From scraping everything against the slabs from metal disks to binder clips to resonating a jews harp against them to, yes, crushing, beating and destroying, he created an elaborate and unique palette of concrete sounds. As a few of the characters in the game developed, their powers also evolved. Some characters now had “beams” of concrete they would shoot out to shield allies while another burrowed underground like Bugs Bunny on his way to Albuquerque, and another sat atop a giant swirling tornado of concrete chunks. We needed something unique here and I devised a way to record a constantly moving collection of some of the concrete chunks we had broken (and wrote up a blog post about it here).
Andy’s wizardry both in recording these sounds and in shaping them in ProTools and Wwise into the layers of concrete powers was top notch as always and now it was time to show the team what we’d been doing (and that our jobs are more fun than theirs). Below is another Sonic Equation of sorts which we showed at a company meeting demonstrating some of recording techniques used to make the concrete powers of Second Son:
Thanks again for reading. I hope to get a write-up of the video powers (which naturally entailed a lot of fun creative recording and manipulation) done next week in time for the proper anniversary of Second Son’s release. Stay tuned!
In contrast to past inFAMOUS games, Second Son was a tricky beast in that our power sets were pretty abstract. Electricity can really hurt someone, but smoke, neon, and video? This was definitely one of the many challenges we faced with the sound design of the powers. For neon, we took a pretty direct approach and then got creative with our source materials.
We struggled early on with making neon sound “neon” and not “laser.” There was some confusion during development in which power we were making as those two words were often interchangeable. (Fetch even refers to herself at one point in the game as “Laser Girl”). Making her sounds laser-y was ok, but at the same time I didn’t want to tread on the hallowed ground of Ben Burtt. I actually cursed his name a few times during production because Andy had made some beautiful sounds that unfortunately sounded too Star-Wars-laser-gun. Andy had a REALLY long spring (originally an induction coil for an industrial kiln we got from a local glass maker named Chris Daly) attached to the ceiling of his office. Whenever he would accidentally hit it, I would hear the telltale “pew pew” in my office next door.
The first element we captured which really felt “neon” was an actual neon tube. We have a couple Sly Cooper neon signs in the office, so I took my Barcus Berry contact mic and attached it to one and got some really nice neon hum. For more variety we captured a bunch of fluorescent lights as well, both via contact mics and using a Sennheiser MKH 8060 to capture various flickering sounds of turning them on and off. I have a very old fluorescent fixture in my house that created some amazing sounds which we ended up using for neon power sources powering down. And Delsin’s neon drain was composed of several tracks of neon hum processed through Izotope Iris with various frequencies cutout and some filter sweeps.
For the rest of Delsin’s powers, Andy got REALLY creative. As you’ll see in the video below no sounds were off limits and we used a broad range of varied sounds to create the final neon palette. Andy used Zynaptiq’s Morph plugin extensively to do some interesting blends of EMF interference and various hits on the aforementioned induction coil. Other tricks up our sleeve included an old signal generator I have which emits square and sine wave sweeps and some very cool power on and off sounds and a crazy electric shocking device from the 50s which would shoot small arcs of electricity at anything you put near it.
Once we got our power set close to completion, it was time for another milestone meeting and thus time for another movie to show off our work. The response from the team from our previous movie, the smoke “Sonic Equation,” was so overwhelmingly positive, I felt compelled to do another. Sure the equation doesn’t EXACTLY equate to the sounds as they are in the game, but it at least shows off part of our design methodology as well as the fun we’re still having.
Next time, we’ll discuss the enemy concrete powers and show some of the abuse we wrought upon varied chunks of concrete!
You know how sometimes you have lofty plans to do a project and then months later you think, “What the hell happened? I still haven’t gotten to that thing I meant to do months ago!” Well that’s pretty much where I’m at. I’ve been meaning to write a few short posts about some of the sound design we did on inFAMOUS Second Son for quite some time, and I’m FINALLY getting around to it. I hope this to be the first in a series of posts with an entertaining movie or two showing off some of the sounds we captured to make our various sfx in Second Son and how those ended up sounding in-game. Since the powers are the biggest sonic show piece of the game, I figured we’d start there.
A lot has already been written about smoke power, but since it’s the first power you gain in the game, I’ll touch briefly on it one more time in part just to show you the movie below.
But before that here’s something which may be of interest that has never been seen or heard outside of Sucker Punch. The first thing we ever did in regards to powers on Second Son was to concept some ideas of what these powers may sound like. We had NO idea what they were going to look like (and as you can see from the video we were even concepting powers that never made the cut into the game). This was merely an exercise to start playing with sound and seeing what kinds of things were resonating with us in regards to these potential power sets. A lot of what we started with helped inform our extensive recording sessions to capture elements to mold and bend to our will. Other concepts we tried here didn’t work and were abandoned. For example, I thought it would be cool if the player’s footsteps had a sweetener applied to them based on your current power set. In the end it felt too heavy handed so we cut it. We played around with the notion of USTV feeds making their way into the video powers sounds (similar to Andy’s Neil Armstrong clip in the concept for what was then called TV powers), but that also just didn’t work in any meaningful way. None of the sounds you hear in this concept made it into the game, but it at least gave Andy, our senior sound designer, and myself a jumping off point to explore from.
Smoke was the first power we worked on, but it was also one of the most challenging: how do you make something as amorphous as smoke sound powerful? Furthermore, how do you make it sound like smoke, and by that I mean NOT like fire. These were the challenges before us. I noticed some steam pouring out from a grate in the ground one day and thought that could be interesting. But it made no sound! We experimented with other air releases from helium tanks and compressed air, but none of it fit the bill. I pretty quickly gravitated towards charcoal. I don’t mean those neatly-formed imitation charcoal briquettes either. I’m talking real burned chunks of wood. I knew from ample barbecuing experience that they made really interesting crackling sounds when burned and also they had a resonance to them when moving around which was kinda unique. After buying a couple bags of charcoal and a small grill I set to work doing most imaginable things to these chunks of burnt wood: moving them around, bouncing them off each other, crushing them, burning them, lighting them on fire and then dousing them, etc. It was a great start. Many other elements ended up playing into the final sounds: surprisingly blowing air through a plastic tube became a very important element in Delsin’s smoke dash and various movements of sand also played a role in both quicker smoke attacks and Delsin’s navigation abilities. Below is a video showing off some of these elements as they were recorded and as they sounded in the game. One quick word on this video: it was originally shown as part of a company meeting. Every milestone during production, each team would show a short video highlighting their work over the past several weeks. We liked to show the team not only how much fun sound design is, but how much fun we have doing it. Enjoy and stay tuned next week for an exposé on Neon powers!
While I’m proud of so much of the audio design in inFamous Second Son, one feature stands out as a testament to never letting go of a good idea. It was a concept, not new or necessarily innovative, that began incubating around 7 years ago. It wasn’t until 2013 that I was able to make the idea work in a title. I thought it’d be fun to trace that feature from its nascent stages through to its full fledged life. To do so, we have to go all the way back in time to a year we called 2007. Ah 2007! There was a palpable hum in the air. The iPhone was introduced by a little upstart company called Apple, Microsoft excitedly released their newest blockbuster (*cough*) Operating System, Vista, and the Nintendo Wii had captured people’s hearts, minds, wrists, and pocketbooks.
I was working at Shaba Games, where we had just finished up the DLC/Gold Edition of Marvel Ultimate Alliance and were looking for a new project. Like many others, we were captivated by the Wii and began working on a concept for a downhill skateboarding game for the platform. Shaba’s other sound designer, Lorien Ferris, and myself began brainstorming ways we could introduce interesting audio to what would ostensibly be a multiplayer racing game. Obviously the skateboard sounds would reign supreme and we came up with an idea of emitters tied to occluder objects such as buildings which would play a quick whoosh as you passed them (an idea I would later harvest for the mobile title, SummitX Snowboarding). Another idea we had was to have music emanating from buildings as you skated by. You’d be going fast and could never go back uphill, so they could be short loops, and once we applied some doppler it would sound awesome!
Unfortunately, while the Wii as a piece of hardware was popular for a slew of years, the software didn’t seem to sell as well, so the project was scrapped before we got very far. BUT after multiple other false starts we were finally given something wholly different and rather exciting: Spider-Man, and what would eventually become Web of Shadows. The goal was straightforward: create a new, unique open world Spider-Man game using the engine from the recently released Spider-man 3. Once again Lorien and I dove into brainstorming cool new features we could implement on the audio front to push the superhero qualities of Spider-man and the real life interactivity of the city. Early on, our storefront music concept was revived. I even added some various loops to embed into some stores simulating dance and jazz clubs and restaurants. Unfortunately we ran into some design problems early on: the storefronts we had in the game didn’t really match the music, they were destructible but we didn’t have a signal to turn off the music when the store was destroyed, and truthfully it just didn’t sound super-convincing to have the sound of filtered talking and clinking dishes and glasses of a restaurant while you’re right outside fighting. You think there’d be screams and hushed whispers. Basically with a tight schedule and a skeleton crew, our storefront music plans would have to wait for another day…
…which came just a year and a half later. We were working on a new superhero title and, with so much of the infrastructure in place now, we spent some time focusing on how to make storefronts believable. We created a multi-stage approach: idle, which would be the default and would play a basic ambient loop. For example some cheesy Italian music emanating from a restaurant. If a fight broke out in the vicinity we would enter a threatened state which would trigger an appropriate one-shot sound effect of screams and maybe instruments falling, dishes breaking, etc. and the music would cease. During high-tension moments (using the same tension meter as our interactive music system) the stores would be silent. Once tension went back down to low, we would slowly ramp up the idle state again until another fight broke out. Perfect plan! Unfortunately the studio ended up shifting gears and we moved from superhero games to music games. The storefront music would lay dormant again…
Fast forward to early 2012. I had just joined Sucker Punch and we were in pre-production on inFamous Second Son. Being back in an open world title, I pretty quickly started to think about my beloved storefront music concept again. Everyone I pitched it to from our creative director to our music team down at Sony HQ loved the concept. So now it was time to design it. The first step was just to get looping sounds emitting from a point in space and figuring out proper attenuation and processing for them. Next it was time to get into the real nitty gritty. I had several challenges to tackle:
A world inside a world
inFamous Second Son takes place in present-day/slightly-future Seattle. It’s not real Seattle, it’s our take on the city, but we still wanted it to be a unique, diverse, funky place, just like real Seattle. We did not want it to be full of grunge music (and that is a story for another day!). I began talking with the environment team to get a sense of the variety of storefronts we would have, and some of what they created helped influence my ideas. Early on, we got an Irish Pub in the game. At which point I thought, man that’d be cool to have it play Irish music during the day and then become a punk club at night. Just like in real life! Then I started to take it further: what if we had traditional Irish music in the earlier times of day, changing to more upbeat, raucous Irish music in the evening and THEN a punk club at night. I was on to something. As we fleshed out the stores my list of music grew and grew. I wanted jazz and Chinese and J-Pop, club, top 40, and why not mariachi music in the Mexican restaurant and thai music, new age music for the yoga studio, and hell even Russian music to put into apartments where the Akulan gangs live? Sure they’re musical stereotypes, but they’re serving the purpose of a low ambient bed, they were never meant to be featured sounds. The result will be filling the city with greater perceived life. I also wanted to reach out to some local bands and get them featured in the game as well. I wanted a lot. So how the hell were we gonna get all this music?
APM to the rescue!
As anyone who’s worked with Sony can attest to, they have some of the most amazingly talented, brilliant people working in their music department. We were very fortunate to have a few of them working closely with us throughout the project. Beyond the game score, we started discussing this source music idea and they carved out some of their budget for a blanket license from APM for stock music. Matt Levine worked directly with APM who would put together playlists for various genres of music we were interested in. He would then send me the lists, which I would review, make notes and approve or ask for more. In the end, we had over 100 tracks in the game spread out over 8 times of day. On the local band side, having been in a band and played with some acts up here I reached out to some friends’ bands and also KEXP, the local college station, and got a list of some potential candidates, several of which made it into the final game. We also started talking to Sir Mix-A-Lot and he really wanted to get some tracks in, too. Now that we had music, we just had to get it playing in game.
Rock Against the Man
As I mentioned, I had earlier rigged up a test playing source music in a test world pretty quickly to help figure out volume, attenuation, and processing. From there, it was on to the challenging part: figuring out how to make it gel in-game. In inFamous Second Son, you play as Delsin Rowe, a rebelistic youth with super powers battling against an evil authoritarian police force, the D.U.P. (Department of Unified Protection, think of the TSA with guns, armor and superpowers). Delsin can clear the DUP out of each district of Seattle as part of the systemic, non-mission open world gameplay. The main theme here is freedom vs. security. The DUP keeps people secure, but Delsin gives them the freedom to do as they wish. To help reinforce this thematically we decided that when the DUP controls a district we’d only hear DUP music. We started with stoic, patriotic sounding cues, but steered the direction more towards syrupy, happy music that provides a wonderfully stark juxtaposition to the menace of the DUP. Once Delsin begins to drive the DUP out of a district, we stop the DUP music from playing in that area and instead let the storefronts come to life with their own individuality. We had a programmer working on the district status rigging, so I asked him to give me a callback signal for when the district status changed. I was then able to use this to determine what district the player was in and whether DUP music should be playing there (it emits from DUP speakers and closed off DUP storefronts), or whether the other storefronts should be allowed to rock in the slowly-becoming-free world. I didn’t feel my initial idea from way back about multiple states work work in this instance. The music acted more as personality for the district than simulating people inside, so I didn’t pursue any kind of multi-state reactive environment. Maybe next time!
At the same time, I wanted some semblance of reactivity and also wanted to ensure the source music didn’t clash with the game score. So I tied the volume of the source music to our tension rtpc (Real-time Parameter Control in Wwise) which is also used for controlling the music intensity. When the player got caught up in combat, the music would fade out, when the combat abated, the source music would slowly ramp back up in volume. As if the owners of the shops were peeking through their windows, and once they saw the DUP dispatched, they cranked up the tunes again. So everything was working great, but now I had dozens and dozens of songs across ten or so genres, how was I going to make it all fit in a shippable state?
Making it fit
Beyond the goal and using source music to bring more life to our fictitious Seattle, I also wanted breadth and variation within the music so you wouldn’t hear the same cue EVERY time you passed a storefront. With a blanket license from APM plus around 20 local musician tracks the content was near limitless. Our soundbank budget, unfortunately, was not. However, every time we change the time of day in the game, we do a load to bring in our new skybox and other time-specific content. In fact, I was already loading all of my ambient sounds with these time of day loads. I devised a scheme to load certain music which could play at any time of day in our core ambient bank, which is always loaded. This ended up being the DUP music and our local acts. For the rest of the store fronts, I would load in 3-5 cues per TOD per genre. This way we have some variation during each time of day, as well as completely new tracks for most storefronts for each time of day change. For the local music, we had all 20 tracks in a random playlist emanating from Sonic Boom Records (a real Seattle record store), Sir Mix-A-Lot played from some of our neon-drainable low-rider hatchbacks (we HAD to have My Hooptie for that!), and the aforementioned Irish punk club featured 3 bands each rotating through a set of 5 songs each. You could theoretically stand by the Irish pub at night and enjoy a whole night of music (if it wasn’t so much fun to run around and use your powers instead!)
My budget for the TOD banks was 7mb, of which I used 2-3mb for source music at VERY low bitrates. We processed them heavily with severe low pass filters and reverb, so we really didn’t need a lot of high end, and the lower encoded bitrates (24kbps OGG) aided in making the tracks sound like they were coming out of crappy speakers inside the storefronts. Most of the cues were edited to around 60-90 seconds since most people wouldn’t really be standing around listening to the music, and we wanted more quantity of tracks than longer songs for this reason.
Here’s a video showing off just a few of the myriad storefronts we added music to. If you have a copy of Second Son, I highly suggest pushing the DUP out of some districts and running around to see how the source music aids in filling in the world without stepping on the score or any critical gameplay. It’s a subtle effect no one would likely notice, but subtlety is often the key to effective sound design.
