No five year long project would be complete without at least a few goofy bloopers. There were so many more that we just didn’t capture, but I hope this brings a smile to your face. For me, it is just a reminder of how lucky we are to do what we do day to day and that we’re able to be creative and have fun for a living is one of the treasures of my life. I feel exceptionally fortunate every day even when (or maybe especially because) I do stupid stuff like some of what you’ll see here:
Category: Sound Design
-
The Sound Design of Ghost of Tsushima: Crafting the Ambience
To be honest, when we first started making Ghost of Tsushima, I was probably most excited about the possibility of traveling to Japan to capture ambience. We sent two teams over to Tsushima and throughout Japan but I didn’t get to go. It kinda made sense. These were more sight-seeing tours for environment artists, character modelers, and some of the other creative leads to get a sense what Japan was like. If I was going to go I needed isolation, not driving around in a van with 10 other people.
On the first trip, I gave Billy Harper, our character and animation lead one of my recorders and a brief tutorial on how to use it in hopes he could maybe get away and record some interesting stuff. I wasn’t expecting anything useful but Billy surprised me. He got some usable recordings of birds including some black kites, songbirds, and of course, the ubiquitous jungle crows. The team went to Tsushima and attended the Komodohama festival which commemorates the Mongol invasion landing on Tsushima in 1274. Our team was invited to the junior high school on Tsushima and Billy and our cinematics team recorded several instruments including various taiko drums and a horagai, a conch shell horn that was used by samurai to communicate in battle.
Being part of Sony, we have a great team at Japan Studio, so we reached out to them and ask if they would be willing to record some ambience for us and they did a great job. The team there drove out near Mount Fuji to capture all sorts of birds in the wild as well as some rivers and wind. They also went to the Kachoen Wild Bird Refuge and got tons of species isolated. One of the sound designers, Ito-san went on vacation to visit family over in Akita prefecture on the Northeastern side of Honshu, the main island of Japan, and got some fantastic recordings of songbirds at dawn and during the say in some rice paddies.
All of these recordings were great and totally usable, but we were still missing a few specific species of animals we knew were going to be in the game, so I decided fairly last minute to spend my Christmas and New Years traveling around Japan on a whirlwind trip chasing after more specific species. My partner, Bonnie, joined me as travel partner and documentarian. I had planned a 10 day trip for us. We would land in Tokyo one evening, then fly out to Hokkaido the following morning, record the tancho (the red-crowned cranes), travel to Lake Kasharro to record the Whooper Swans, then back to Honshu. We’d spend New Years in Tokyo, then head up to Nikko, followed by a trip into Nagano prefecture to try and record the snow monkeys.
The entire trip was absolutely incredible. Japan is insanely beautiful, the people exceptionally friendly, even when there’s a language barrier, and the food was always fresh and amazing. The entire trip was a highlight, but also included some fun surprises. When we were in Nikko, we checked out the temples as you do, but then hiked up a mountain near the Kirifuri Ice Arena. I set up my recorder to capture some ambience and walk a little ways away. A short while later we were startled by what sounded like a piercing alarm chip. It happened again and then once more. Bonnie and I looked at each other and mouthed “What the fuck was that?” Suddenly we saw two huge sika deer run by in the valley below us. The continue to chirp their alert calls for the next 10 minutes. Those recordings feature prominently in Ghost of Tsushima.
There were some other cool moments that didn’t make it into the game as well. In Kushiro on Hokkaido, the Kushiro River was half frozen with large sheets of ice flowing down the river and crashing into each other. One night I threw my hydrophone into the river, hoping to get some interesting underwater sounds of ice creaking, crunching and colliding, but there was nothing quite so interesting.
After compiling all this great content from so many people, I made a movie to share with the team as part of one of our team meetings. My attempt to evoke David Attenborough is pathetic, but the video was a fun glimpse into just how global the effort was to capture realistic and accurate ambience for the game.
-
The Sound Design of Ghost of Tsushima: Fake Birds
The natural world we built for Ghost of Tsushima is one of the sonic highlights for me. The world is lush and full of so many different species of birds, insects, amphibians and mammals, many of which were recorded in Japan and most of which are native to Japan. It really creates this wondrous natural beauty for the rest of the game to sit on top of.
We had our friends at Japan Studio in Tokyo record a lot of ambience for us and I was fortunate enough to travel around Japan for a couple weeks and capture all sorts of wildlife. The ambience was one of the more time consuming systems in the game, but it was also one of the first to get solidified. We had good variety of wildlife species and the world felt rich, but as I explain in the video below, getting towards the end of the project they decided to add a few new specific species. We didn’t have time or resources to fly halfway around the world and try to track these specific birds down, nor could I find a library with them.
Fortunately, the Slack field recording channel was doing a bird crowdsource around this time, and while discussing our recordings, Alex Barnhart asked if anyone had tried pitching down real bird recordings, performing them, and then pitching them back up to see if you could replicate birdsong in that way. We all thought it sounded fun and cool and it was very shortly after this moment, that they decided to add 3 new species to the game, so I took this knowledge and went for it.
I went onto xeno-canto.org which is a fantastic website for information and birdsong and found some decent quality samples that I could use as reference. From there I tried pitching them down to various ranges from an octave to four octaves and everywhere in between, trying to find that sweet spot where it was in my vocal range and performable. After performing, and looking and sounding appropriately goofy, I was pretty surprised and pleased when playbacking the results. Once I put them in game, I knew this was going to work!
They even added a couple more species (a cormorant and a Eurasian sparrowhawk) that I was able to perform adequately. The one place I failed was when trying to augment our existing Black-naped oriole recordings. This is a really important bird in the game, as he guides you to your various objectives. I had captured one while recording in Sri Lanka, but my recordings were limited, so I hoped to bolster our content with some faked versions, but when playing a fake bird against a real bird, it becomes VERY evident which one is fake, so I abandoned it.
The results were so surprisingly decent, we played a game at work where the team listened to bird samples and tried to identify whether they were a real bird or me, and it was hard. Even I got half of them wrong! So here’s a video that goes into more of these details and also shows the process in action:
-
The Sound Design of Ghost of Tsushima: Horses
One of the first decisions made on Ghost of Tsushima was that our main means of fast locomotion in the game would be horses. It made sense since that was what was used by both Mongols and Japanese warriors in the 13th century. There would be no grinding power lines or zipping through air vents or beaming from one radar dish to another. So of course we knew the horses needed to sound great.
Surprisingly I think the only other game I’ve worked on that almost had a horse was a Shrek game that got canceled. In that game in a very on-the-nose homage to Monty Python, there were knight enemies that rode around on hobby horses. For the knights footsteps I recorded coconuts and the results were as hilarious as intended. So for fun, when we first got the horse working in game, I re-recorded the same coconut shells and, while still hilarious, it didn’t quite fit tonally.
We began to research some places to record horses. The most important thing was we needed shoeless horses. While there was conflicting information about whether or not Mongols shoed their horses, we could find no information demonstrating the Japanese shoed their horses with metal, so we opted to keep things simpler and go for natural footsteps. After calling some places that fit the bill and driving around to check out their properties, assess noise levels, etc., we decided upon the Northwest Natural Horsemanship Center out in Fall City, WA. The owner Jim Hutchins, was keen to work with us and seemed genuinely interested in our work.
Unfortunately between the time we agreed to record there and the date of the recording, my sound designer had left for another position, so I was on my own for the session. One fewer person to operate microphones posed some problems, so I got creative: I followed the horse with a boom mic and strapped a recorder and a pair of omni electret condenser mics (a Roland CS-10EM) to the saddle which we taped down (and taped all buckles on the saddle).
Horses are amazing animals and when the horse first saw my furry blimp, it was not happy. We got lots of great nervous vocalizations which ended up as the final assets in the game as it got used to the presence of the blimp. From there Jim guided the horse around on various surfaces in their property (grass, tall grass, dirt, mud, wood, gravel, stone/concrete, and asphalt) at trot, canter and occasionally gallop speeds and I chased after them with my mic. We got A LOT of horse grazing because when a horse is hungry and has the lay of the land, they eat when they feel like it. These assets too were eventually massaged by Erik Buensuceso and made it into the game. In fact, the amount of grazing sounds we got and the frequency with which the horses naturally ate grass was the impetus for us adding the grazing animations into the game!
To get bigger whinnies and neighs, Jim put two of the horses together who were really good friends, and then separated them. Once one was led far enough away, the other would bellow out a very loud call for their friend. I inadvertently recorded a horse fart at one point, but was too far away so unfortunately it was not usable. Lastly, we went into their gear barn and recorded a bunch of sounds of various bridles and saddles, again taping down any metal parts since we wanted to avoid jangling components in the sound design.
Once I chopped up the assets and integrated them into the game I made this delightfully silly video to show at a company meeting:
But we were still a long way from being done. The horse had to sound great because it was used SO much in the game and we wanted to really push on the detail of everything about the horse from its footsteps to its saddle and bridle sounds to its vocalizations. I think when all was said and done the assets for the horse were probably touched by almost every sound designer on the project. I can think of at least 5 of us that did some work on the horse, tuning and improving and iterating to make them sound great.
On the mix side, Josh came up with a great idea we used on both hero and horse foley. We created an RTPC that tracked how long the player was running or the horse was galloping, which we used to subtle mix down the non-footstep sounds after 10-15 seconds. This served the purpose of helping reduce ear fatigue and allow other sounds to cut through the mix on long traversals. With this attention to detail we got some really nice dynamic behavior in the sounds in game.
-
The Sound Design of Ghost of Tsushima: Dogs
The dogs in Ghost of Tsushima are fun to talk about because, like so much of the soundscape of the game, we started with a lot of research and then recording them was a blast. Let’s just ignore the fact that Jin kills so many dogs in his journey and rewind a few years when we first started talking about having dogs in the game. The impetus for adding dogs was to have a unique enemy type that was also more adept at hunting Jin than the Mongols. Like everything else in the game, we did a bunch of research and found that Mongols did indeed use dogs and that from Genghis Khan on, their dogs were ancestors of the breed now known as the Tibetan Mastiff (which technically is not a mastiff, but I’m not here to get into a discussion about dog breeds). So the art team began working on models of Tibetan mastiffs, which meant we needed to find some sounds for them! We started looking online and these dogs looked and sounded fierce, but also fairly unique. So I started looking to see if there were any Tibetan Mastiff breeders in the area and I struck gold: there were actually a handful within a few hour drive from Seattle!
We ended up connecting with Debbie Parsons of Dreamcatcher Mastiffs. She mentioned they had several Tibetan Mastiffs and we were welcome to come record them. So Josh and I packed up our mics and recorders and headed out to Graham, WA, just an hour south of Seattle on the way to Mount Rainier. As soon as we showed up, we were greeted by 3 VERY large, scary and seemlingly angry dogs. We started recording immediately and got some terrific barks and growls (unfortunately interspersed with chain link fence rattle as they were pretty excited). Debbie was super helpful and started taking most of the dogs inside and allowing us to record each one individually. We got a good range of growls, snarls and barks. Debbie even ended up putting on a hooded sweatshirt and acting like a prowler to get them riled up. After getting all these great barks, we went into the property to record more and these dogs transformed into the cuddliest, sweetest bears (seriously they looked like bears in dog suits).
We were able to get panting and breathing and even chewing on bones as they had settled down and got used to us. From there Josh took the recordings, cleaned them up and got them in game. Rob Castro did a final polish pass towards the end of the project to make them even more terrifying. As you can hear from this video we created for a company meeting, the were already pretty scary before the sonic magic was applied.
-
The Sound Design of Ghost of Tsushima: Combat Impacts
We knew as soon as we began working on Ghost of Tsushima that swordplay would be a major part of the game and that the swords had to sound dangerous. We had been working on some close quarters sword-based combat prototypes and in moving to the world of feudal Japan, we wanted to approach combat from a grounded perspective but also really make the sound of the swords slicing through enemies feel razor like. Lots of games and films go for a really meaty, thuddy impact to make things feel powerful. We wanted to go against this trend and make our swords SOUND sharp.
To do this, we really focused our impact sounds on elements of slicing rather than hard impacts. But of course it took a lot of experimentation and a lot of recording. Josh Lord (our senior sound designer) and I amassed a collection of various blades and other sharp objects and applied them to myriad fruits and vegetables. There were some surprising results: digging into celery with the tip of a knife sounding like cutting through bone; a razor blade through an onion sounded like a gut being opened; a serrated bread knife created interesting zippered slicing tears.
Once we got all this source material together, we started experimenting with integration techniques, adapting a technique utilizing sound states in Wwise tied to our animations to trigger different impacts based on animation data. Towards the end of the project, we passed off our combat sounds to Mike Niederquell from Sony for some final polish and for him to work the magic he does. We ended up abandoning our state-based multi-impact approach and Mike made bespoke content from our recordings for the various impact types as the animations weren’t so excessive, and the state rigging introduced unnecessary complexity.
Here’s a short video of some of the props that sacrificed themselves for the greater combat good as well as how they ended up sounding in game:
-
The Sound Design of Ghost of Tsushima: the Mongol Warhorn
A sound we knew we needed early on was a war horn for the Mongols. It had to be iconic and immediately recognizable so that a player could hear it nearby or in the distance and know “uh oh, there are Mongols nearby, and they may already know about me.”
In researching both samurai and Mongols we found many musical instruments used by Samurai to both communicate on the battlefield and attempt to instill fear in their opponents. We were fortunate enough to record some of these instruments including the horagai, a conch shell horn, at Komodahama during their yearly festival commemorating the first Mongol invasion of 1274. In all the research we did on the Mongols, however, I couldn’t find much in the way of music used for war purposes. Most of their music practices were either for celebrations such as marriage, played at imperial court, or just to pass the time. The Mongols were a nomadic people and while their main instruments were stringed instruments like the morin khuur they did also have horns, including some that were made of either wood or animal horn.
With all that knowledge, but a lack of access to traditional instruments, we opted to take some liberty in crafting a sound that we could tie to the Mongols, and would sound unique from the samurai horagai. In their native lands (and by extension the lands they conquered throughout China and the Korean peninsula) the Mongols used several domestic animals for food and drink. The horse was the most important delivering them everything from transportation to alcohol (airag, also known as kumis, is fermented horse milk), but they did also use yaks and cattle. Using this as a jumping off point, I found some steer horns and purchased one, hoping I could teach myself to blow through it and that it would make a sound unique from the conch shell. Fortunately it didn’t take long for the gasping breaths and pushed air to turn tonal and we recorded a bunch of variations on the war horns. We experimented with different calls to signify different enemy states, but in the end we opted to use it for a simpler enemy alert.
We processed the recordings to provide greater reflection and distance modeling the further away you go, and so we could fire off closer or more distant variants depending on what the in-game action called for. Here’s a short clip of recording some of those horns and then what they sounded like in-game:
-
Wwise Tour 2020: Sonic Storytelling
I finally updated my website’s backend after being on PHP v5.6 (current version is something like 7.4) for far too long. Good news is now you can access the site via https for better security. Now that that is done I hope to post some articles soon about some of the specific sound design and recording we did on Ghost of Tsushima.
For now, here’s a talk we did in December about Sonic Storytelling and exploring some of the ways we use audio to tell our stories. It was a lot of fun to share some of the world with the audio community:
-
The Sound Design of inFamous Second Son: Video Powers
Of all the powers in inFamous Second Son, Video powers may have been the most esoteric. I mean smoke at least has an analog in fire (and we used some fire elements in both the visual and sound design), but video? You think video, you may think laser, but we already had a neon power (which was even sometimes referred to as laser). So how the hell did we get something sounding as unique as our video powers without treading on the other power sets?
Part of the answer is interestingly with how the power set itself was initially conveyed to the team. Video power was actually called “TV power” internally for most of production. Heaven’s Hellfire, the video game that Eugene, the video power conduit, is obsessed with was initially a TV show. We realized after many months that it made more sense to make it into a video game instead and that would open up more avenues for us to play around with in the gameplay (such as the mildly retro boss battle).
But we still had “TV powers” stuck in our brain and when Andy and I began brainstorming about how to make sounds that were powerful and unique and “TV like” we started thinking about televisions. We stalked thrift stores around town hoping we’d come across some old 1970s vacuum or cathode tube televisions to take apart and record. We failed there, but Andy eventually came across a couple old CRT TV/VCR combos. Double obsolete points! We brought these into the studio and proceeded to record all kinds of sounds with an array of microphones from shotguns to contact mics to crappy telephone microphones which did an amazing job of capturing bizarre electromagnetic interference around the power supply, and other surfaces. We recorded all possible permutations of power on and power off sounds and even got the VCR mechanisms to give us some very bizarre whines and hums. We also did some recordings of the Sucker Punch MAME arcade cabinet which has a very old CRT monitor in it with tons of wires exposed, as well as a shortwave radio I’ve had for years, but never really needed for a video game sound before.
We recorded all of these sounds at 192kHz and the frequency content of the recordings on the CRT monitors at the higher frequencies was pretty astounding. While some of them we had to remove the >20kHz content to save our ears and speakers, Andy also did some pitch shifting to play around with some of these normally inaudible sounds and they became part of the video power palette.
A few words on the telephone microphones we used: they are cheap and really neat for recording electromagnetic interference. Although Radio Shack may be dead and gone now, you can still get them online. It’s pretty neat the wide array of sounds you can get from one of them by waving near essentially any power source from a monitor to a computer, plugs, etc. Basically any electronic device will give you some interesting content. For a lot of the TV powers, Andy took various EMF sounds and morphed them together using Zynaptiq’s Morph plugin.
So, similar to our other power sets, below is a video showing some of our field recording as well as the final in-game sounds. What’s different here is that the video powers were finalized later in the project and we were so focused on finishing the game, that we did not make a fancy, fun video for the team. So, it may not be as fun as the previous videos, but still shows what we recorded and how it ended up sounding.
-
The Sound Design of inFamous Second Son: Concrete Powers
It’s hard to believe that inFamous Second Son is a year old already! I’ve been completely lagging on finishing up these posts about the powers design for the game, so let me use this opportunity to make good and present the first of the final 2 parts of this series. I will hopefully get around to posting my presentation on the Systems Design for the game soon as well so those who haven’t heard/seen it can have the information available to them. Anyway, on to the magic and mystery of concrete!
For those who haven’t played or seen inFamous Second Son you play a guy who gets superpowers battling an authoritarian government agency called the DUP whose soldiers are all imbued with concrete superpowers by their leader Dana Augustine (as normally happens with government agencies).
The biggest challenge for us with concrete was how to make it sound unique. It’s just rocks and stone right? We’ve all heard countless variations on rock sounds in everything from impacts to destruction and rubble/debris sounds. We needed to figure out ways to make our sounds stand out as unique, while also conveying the power of the enemies in the game who used concrete.
The powers ran the gamut from concrete grenades to spawning concrete shields to launching off spires of concrete and forming a concrete balcony on walls. In short there was tons of concrete objects being created and broken in the world. Not only did we need these to sound unique and “powered” but they also had to sound completely distinct from all the “normal” concrete in the world you could destroy or collide other objects with. It was a huge challenge, but one that Andy Martin was definitely up for.
The place to start, naturally, was by buying a bunch of concrete. I looked into the process of concrete, which is usually just a mixture of water, an aggregate like sand or gravel, and Portland cement (named after a type of stone used in the UK, not the sleepy hamlet of the Pacific Northwest of the US). While the thought of mixing up my own concrete sounded appealing to my construction worker wannabe side, we weren’t in a position in the project where we had limitless time to experiment. So we did the next best thing: went to Home Depot. Andy and I both made trips to the hardware store and bought all kinds of concrete and stone, from paver stones (which were often too resonant) to clay bricks, cinder blocks, and more. They were demolishing a building across the street from my house and I noticed some particularly large chunks of both asphalt and concrete sitting on the other side of the fence. I waited until nightfall, donned my ninja costume (really just a bathrobe with a scarf tied around my head) and absconded with the almost-final resources we would need to make our concrete powers come to life.
From here, Andy began to run wild and experiment with all kinds of torture he could enact on our various pieces of concrete. From scraping everything against the slabs from metal disks to binder clips to resonating a jews harp against them to, yes, crushing, beating and destroying, he created an elaborate and unique palette of concrete sounds. As a few of the characters in the game developed, their powers also evolved. Some characters now had “beams” of concrete they would shoot out to shield allies while another burrowed underground like Bugs Bunny on his way to Albuquerque, and another sat atop a giant swirling tornado of concrete chunks. We needed something unique here and I devised a way to record a constantly moving collection of some of the concrete chunks we had broken (and wrote up a blog post about it here).
Andy’s wizardry both in recording these sounds and in shaping them in ProTools and Wwise into the layers of concrete powers was top notch as always and now it was time to show the team what we’d been doing (and that our jobs are more fun than theirs). Below is another Sonic Equation of sorts which we showed at a company meeting demonstrating some of recording techniques used to make the concrete powers of Second Son:
Thanks again for reading. I hope to get a write-up of the video powers (which naturally entailed a lot of fun creative recording and manipulation) done next week in time for the proper anniversary of Second Son’s release. Stay tuned!
