I was asked to join Lacuna Observatory, an artist collective a while back and we just had our first show exploring the spaces between dreams and the waking state. While I showcased a couple sculptures I created for the show, Dreamcrusher I and Dreamcrusher II, my main contributions were naturally on the audio side.
I had created a phone number where people could call in and leave recordings of their dreams. I then put the dreams onto a small AdaFruit sound board and installed that into an old rotary phone. And thus, the dream phone was born. Pick it up, and hear people’s dreams. It sat on an oilcloth with the text of the dreams laid out so that the dreams ran into each other, forming new images, much like they do.
The first piece I did for the show, which is most relevant to audio, is called Vision through Sound. It examines the way our senses intermingle to create a perception of reality. Each scene begins with a soundscape, giving your brain time and space to create an image of what it hears. Text prompts appear on screen which will alter your brain’s perception in some way. Next, an image appears to cement or shift that reality. Finally, everything dissolves away, leaving just the audio to drift from your memory.
I recommend you give it a listen with headphones as several of the recordings were captured binaurally. Curious to hear how people perceive it and how they feel about the confluence of audio, word and image.
It was an exciting week for me and all of us at Sucker Punch, as we finally announced our new project. Ghost of Yotei to the world! I look forward to doing some deep dives into the new systems and sounds that we ‘ve built for this game. I’m so excited to share more soon!
I’ve long been a fan of Sam Hughes’ Sound Architect podcast. He always asks great questions and the results are informative discussions with very talented audio people. I was pleasantly surprised when he reached out asking if I’d like to chat with him about Ghost of Tsushima. It took a lot longer than either of us would have liked to set things up, but once we did, we had a really great chat. We went over the normal one hour time for his interviews and easily could have spoken for another hour or three. I hate the sound of my own voice and my penchant for interjections of “um,” “like,” and “you know,” but hopefully you’ll find it informative too.
More recently, I was asked by Alistair Hirst over at Dolby to talk with him about Ghost, so I got Adam Lidbetter, our supervising sound designer at Playstation, Andrew Buresh, our senior music editor, and Apoorva Bansal, who did most of the audio programming on Ghost to sit down and talk about the audio of Ghost and some of the fun challenges we dealt with. Here’s the result there:
Last one here, but worth a mention. I was contacted by Hamidreza Nikoofar, an Iranian sound designer, composer and podcaster. He has a podcast called Wassup Conversations and has hosted an impressive array of guests. We had a good chat about my career and Ghost and I highly recommend checking out Hamid’s other episodes.
[editor’s note: This was originally published here on June 9, 2021]. It’s hard to believe Ghost of Tsushima is almost a year old already! It came out on July 17, 2020. We started planning for the final mix in February of that same year. Do you remember February of 2020? The only people, at least in the U.S., that wore masks were superheroes, vaccines were for kids and traveling, and people felt totally comfortable crammed into enclosed spaces. With that mindset, we started planning. I would fly down to Sony San Mateo, other Sony sound folks would fly up from San Diego and Los Angeles and collectively we would spend a month packed into a studio mixing the game. I knew a month wouldn’t be enough to mix the game, but it’s all I could get. The thought of me being away from the office for that long was already daunting. The initial plan was for only 3 weeks, but I negotiated to 4. Because we weren’t going to have enough time, my strategy was to focus on what most players would experience: start with the core systemic gameplay, followed by the Golden Path missions, our main narrative, the special Legendary Item missions, and finish up with as many of the buddy chain Silver missions as we could get to. It was a solid plan.
As things got worse with the coronavirus, our mix plans became a bit more fluid. Maybe I could fly a small charter airline down to the Bay Area. Maybe we’d need to limit the number of people who came. Then the shelter-in-place orders came. Because the audio team needed special facilities to do our jobs we were initially given special consideration during this time. I kept working in the office with a skeleton crew of others, and there were a couple people at Sony doing the same. As the situation worsened, I was forced to move home, and our plans continued to change. We started floating every idea possible. Mixing at home was not ideal since I didn’t have a decent, quiet room. Would I need to seal myself off at work and mix alone in my non-calibrated studio? By early April things had solidified into what would promise to be the most unique mix experience I’d ever been a part of. I would drive down from Seattle to San Mateo (a 14 hour drive normally without traffic, yet only a 12 hour drive at peak pandemic time). I would stay in a hotel right across from the Sony campus. There would be one other member from each discipline joining me: Adam Lidbetter from sound design, Kyle Richards from dialogue, and Nick Mastroianni from the music team. Adam, Kyle, and myself would be situated a minimum of 10 feet apart from each other in Studio A, and Nick would be in the live room playing the game. It was not ideal, but it was a plan.
And surprisingly it worked exceptionally well. Part of the reason for that was that we asked the other folks who were supposed to be at the mix to play the game, and to play ahead of us. I maintained a google doc of mix notes, and every day as people played through they would add to the mix notes. The reason we had them play ahead of us was so we could address their notes as we made it through the game in the studio.
The unsung heroes of the entire pandemic were our IT department. They and a few programmers figured out how to get the entire studio and our proprietary toolchain working remotely in a matter of days. It was remarkable. Using this tech, I was able to download package builds from a devkit at the hotel I brought with me everyday and play them each night while I was in my hotel room ordering takeout and wearing a combination of N95 masks, bandannas and boxer shorts as facemasks (how far we’ve come).
About halfway through the mix we found out that our ship date was getting pushed slightly which gave us an extra week for mixing, which was a godsend. Somehow we managed to cover everything we’d planned in the initial 4 weeks, but that extra week gave us more time to polish and cover more of the game. We were so fortunate and that time really paid off.
I shot a ton of video while I was down there (though often not at the most opportune times), because I knew this was going to be a weird experience and such a strange mix process. The fact that it worked, and worked so well, is a testament to the entire team. The support I had from Sucker Punch and Sony was incredible and I still am in a daze over how lucky I’ve been to work with such a phenomenal team. So here’s a video showing some of the “highlights” of the mix. It’s long. It’s often boring. It’s sometimes funny. But it shows what the process of the mix really looked like, and I get honest about a lot of my personal and professional concerns, learnings and shortcomings. I don’t know that I would recommend watching it because it really is over a half hour of watching me talk with underwear covering my face or clicking a mouse in Wwise, but hopefully there will be some nuggets of interest to people getting a glimpse behind the curtain of a strange mix in a strange time.
No five year long project would be complete without at least a few goofy bloopers. There were so many more that we just didn’t capture, but I hope this brings a smile to your face. For me, it is just a reminder of how lucky we are to do what we do day to day and that we’re able to be creative and have fun for a living is one of the treasures of my life. I feel exceptionally fortunate every day even when (or maybe especially because) I do stupid stuff like some of what you’ll see here:
To be honest, when we first started making Ghost of Tsushima, I was probably most excited about the possibility of traveling to Japan to capture ambience. We sent two teams over to Tsushima and throughout Japan but I didn’t get to go. It kinda made sense. These were more sight-seeing tours for environment artists, character modelers, and some of the other creative leads to get a sense what Japan was like. If I was going to go I needed isolation, not driving around in a van with 10 other people.
On the first trip, I gave Billy Harper, our character and animation lead one of my recorders and a brief tutorial on how to use it in hopes he could maybe get away and record some interesting stuff. I wasn’t expecting anything useful but Billy surprised me. He got some usable recordings of birds including some black kites, songbirds, and of course, the ubiquitous jungle crows. The team went to Tsushima and attended the Komodohama festival which commemorates the Mongol invasion landing on Tsushima in 1274. Our team was invited to the junior high school on Tsushima and Billy and our cinematics team recorded several instruments including various taiko drums and a horagai, a conch shell horn that was used by samurai to communicate in battle.
Being part of Sony, we have a great team at Japan Studio, so we reached out to them and ask if they would be willing to record some ambience for us and they did a great job. The team there drove out near Mount Fuji to capture all sorts of birds in the wild as well as some rivers and wind. They also went to the Kachoen Wild Bird Refuge and got tons of species isolated. One of the sound designers, Ito-san went on vacation to visit family over in Akita prefecture on the Northeastern side of Honshu, the main island of Japan, and got some fantastic recordings of songbirds at dawn and during the say in some rice paddies.
All of these recordings were great and totally usable, but we were still missing a few specific species of animals we knew were going to be in the game, so I decided fairly last minute to spend my Christmas and New Years traveling around Japan on a whirlwind trip chasing after more specific species. My partner, Bonnie, joined me as travel partner and documentarian. I had planned a 10 day trip for us. We would land in Tokyo one evening, then fly out to Hokkaido the following morning, record the tancho (the red-crowned cranes), travel to Lake Kasharro to record the Whooper Swans, then back to Honshu. We’d spend New Years in Tokyo, then head up to Nikko, followed by a trip into Nagano prefecture to try and record the snow monkeys.
The entire trip was absolutely incredible. Japan is insanely beautiful, the people exceptionally friendly, even when there’s a language barrier, and the food was always fresh and amazing. The entire trip was a highlight, but also included some fun surprises. When we were in Nikko, we checked out the temples as you do, but then hiked up a mountain near the Kirifuri Ice Arena. I set up my recorder to capture some ambience and walk a little ways away. A short while later we were startled by what sounded like a piercing alarm chip. It happened again and then once more. Bonnie and I looked at each other and mouthed “What the fuck was that?” Suddenly we saw two huge sika deer run by in the valley below us. The continue to chirp their alert calls for the next 10 minutes. Those recordings feature prominently in Ghost of Tsushima.
There were some other cool moments that didn’t make it into the game as well. In Kushiro on Hokkaido, the Kushiro River was half frozen with large sheets of ice flowing down the river and crashing into each other. One night I threw my hydrophone into the river, hoping to get some interesting underwater sounds of ice creaking, crunching and colliding, but there was nothing quite so interesting.
After compiling all this great content from so many people, I made a movie to share with the team as part of one of our team meetings. My attempt to evoke David Attenborough is pathetic, but the video was a fun glimpse into just how global the effort was to capture realistic and accurate ambience for the game.
The natural world we built for Ghost of Tsushima is one of the sonic highlights for me. The world is lush and full of so many different species of birds, insects, amphibians and mammals, many of which were recorded in Japan and most of which are native to Japan. It really creates this wondrous natural beauty for the rest of the game to sit on top of.
We had our friends at Japan Studio in Tokyo record a lot of ambience for us and I was fortunate enough to travel around Japan for a couple weeks and capture all sorts of wildlife. The ambience was one of the more time consuming systems in the game, but it was also one of the first to get solidified. We had good variety of wildlife species and the world felt rich, but as I explain in the video below, getting towards the end of the project they decided to add a few new specific species. We didn’t have time or resources to fly halfway around the world and try to track these specific birds down, nor could I find a library with them.
Fortunately, the Slack field recording channel was doing a bird crowdsource around this time, and while discussing our recordings, Alex Barnhart asked if anyone had tried pitching down real bird recordings, performing them, and then pitching them back up to see if you could replicate birdsong in that way. We all thought it sounded fun and cool and it was very shortly after this moment, that they decided to add 3 new species to the game, so I took this knowledge and went for it.
I went onto xeno-canto.org which is a fantastic website for information and birdsong and found some decent quality samples that I could use as reference. From there I tried pitching them down to various ranges from an octave to four octaves and everywhere in between, trying to find that sweet spot where it was in my vocal range and performable. After performing, and looking and sounding appropriately goofy, I was pretty surprised and pleased when playbacking the results. Once I put them in game, I knew this was going to work!
They even added a couple more species (a cormorant and a Eurasian sparrowhawk) that I was able to perform adequately. The one place I failed was when trying to augment our existing Black-naped oriole recordings. This is a really important bird in the game, as he guides you to your various objectives. I had captured one while recording in Sri Lanka, but my recordings were limited, so I hoped to bolster our content with some faked versions, but when playing a fake bird against a real bird, it becomes VERY evident which one is fake, so I abandoned it.
The results were so surprisingly decent, we played a game at work where the team listened to bird samples and tried to identify whether they were a real bird or me, and it was hard. Even I got half of them wrong! So here’s a video that goes into more of these details and also shows the process in action:
One of the first decisions made on Ghost of Tsushima was that our main means of fast locomotion in the game would be horses. It made sense since that was what was used by both Mongols and Japanese warriors in the 13th century. There would be no grinding power lines or zipping through air vents or beaming from one radar dish to another. So of course we knew the horses needed to sound great.
Surprisingly I think the only other game I’ve worked on that almost had a horse was a Shrek game that got canceled. In that game in a very on-the-nose homage to Monty Python, there were knight enemies that rode around on hobby horses. For the knights footsteps I recorded coconuts and the results were as hilarious as intended. So for fun, when we first got the horse working in game, I re-recorded the same coconut shells and, while still hilarious, it didn’t quite fit tonally.
We began to research some places to record horses. The most important thing was we needed shoeless horses. While there was conflicting information about whether or not Mongols shoed their horses, we could find no information demonstrating the Japanese shoed their horses with metal, so we opted to keep things simpler and go for natural footsteps. After calling some places that fit the bill and driving around to check out their properties, assess noise levels, etc., we decided upon the Northwest Natural Horsemanship Center out in Fall City, WA. The owner Jim Hutchins, was keen to work with us and seemed genuinely interested in our work.
Unfortunately between the time we agreed to record there and the date of the recording, my sound designer had left for another position, so I was on my own for the session. One fewer person to operate microphones posed some problems, so I got creative: I followed the horse with a boom mic and strapped a recorder and a pair of omni electret condenser mics (a Roland CS-10EM) to the saddle which we taped down (and taped all buckles on the saddle).
Horses are amazing animals and when the horse first saw my furry blimp, it was not happy. We got lots of great nervous vocalizations which ended up as the final assets in the game as it got used to the presence of the blimp. From there Jim guided the horse around on various surfaces in their property (grass, tall grass, dirt, mud, wood, gravel, stone/concrete, and asphalt) at trot, canter and occasionally gallop speeds and I chased after them with my mic. We got A LOT of horse grazing because when a horse is hungry and has the lay of the land, they eat when they feel like it. These assets too were eventually massaged by Erik Buensuceso and made it into the game. In fact, the amount of grazing sounds we got and the frequency with which the horses naturally ate grass was the impetus for us adding the grazing animations into the game!
To get bigger whinnies and neighs, Jim put two of the horses together who were really good friends, and then separated them. Once one was led far enough away, the other would bellow out a very loud call for their friend. I inadvertently recorded a horse fart at one point, but was too far away so unfortunately it was not usable. Lastly, we went into their gear barn and recorded a bunch of sounds of various bridles and saddles, again taping down any metal parts since we wanted to avoid jangling components in the sound design.
Once I chopped up the assets and integrated them into the game I made this delightfully silly video to show at a company meeting:
But we were still a long way from being done. The horse had to sound great because it was used SO much in the game and we wanted to really push on the detail of everything about the horse from its footsteps to its saddle and bridle sounds to its vocalizations. I think when all was said and done the assets for the horse were probably touched by almost every sound designer on the project. I can think of at least 5 of us that did some work on the horse, tuning and improving and iterating to make them sound great.
On the mix side, Josh came up with a great idea we used on both hero and horse foley. We created an RTPC that tracked how long the player was running or the horse was galloping, which we used to subtle mix down the non-footstep sounds after 10-15 seconds. This served the purpose of helping reduce ear fatigue and allow other sounds to cut through the mix on long traversals. With this attention to detail we got some really nice dynamic behavior in the sounds in game.
The dogs in Ghost of Tsushima are fun to talk about because, like so much of the soundscape of the game, we started with a lot of research and then recording them was a blast. Let’s just ignore the fact that Jin kills so many dogs in his journey and rewind a few years when we first started talking about having dogs in the game. The impetus for adding dogs was to have a unique enemy type that was also more adept at hunting Jin than the Mongols. Like everything else in the game, we did a bunch of research and found that Mongols did indeed use dogs and that from Genghis Khan on, their dogs were ancestors of the breed now known as the Tibetan Mastiff (which technically is not a mastiff, but I’m not here to get into a discussion about dog breeds). So the art team began working on models of Tibetan mastiffs, which meant we needed to find some sounds for them! We started looking online and these dogs looked and sounded fierce, but also fairly unique. So I started looking to see if there were any Tibetan Mastiff breeders in the area and I struck gold: there were actually a handful within a few hour drive from Seattle!
We ended up connecting with Debbie Parsons of Dreamcatcher Mastiffs. She mentioned they had several Tibetan Mastiffs and we were welcome to come record them. So Josh and I packed up our mics and recorders and headed out to Graham, WA, just an hour south of Seattle on the way to Mount Rainier. As soon as we showed up, we were greeted by 3 VERY large, scary and seemlingly angry dogs. We started recording immediately and got some terrific barks and growls (unfortunately interspersed with chain link fence rattle as they were pretty excited). Debbie was super helpful and started taking most of the dogs inside and allowing us to record each one individually. We got a good range of growls, snarls and barks. Debbie even ended up putting on a hooded sweatshirt and acting like a prowler to get them riled up. After getting all these great barks, we went into the property to record more and these dogs transformed into the cuddliest, sweetest bears (seriously they looked like bears in dog suits).
We were able to get panting and breathing and even chewing on bones as they had settled down and got used to us. From there Josh took the recordings, cleaned them up and got them in game. Rob Castro did a final polish pass towards the end of the project to make them even more terrifying. As you can hear from this video we created for a company meeting, the were already pretty scary before the sonic magic was applied.
We knew as soon as we began working on Ghost of Tsushima that swordplay would be a major part of the game and that the swords had to sound dangerous. We had been working on some close quarters sword-based combat prototypes and in moving to the world of feudal Japan, we wanted to approach combat from a grounded perspective but also really make the sound of the swords slicing through enemies feel razor like. Lots of games and films go for a really meaty, thuddy impact to make things feel powerful. We wanted to go against this trend and make our swords SOUND sharp.
To do this, we really focused our impact sounds on elements of slicing rather than hard impacts. But of course it took a lot of experimentation and a lot of recording. Josh Lord (our senior sound designer) and I amassed a collection of various blades and other sharp objects and applied them to myriad fruits and vegetables. There were some surprising results: digging into celery with the tip of a knife sounding like cutting through bone; a razor blade through an onion sounded like a gut being opened; a serrated bread knife created interesting zippered slicing tears.
Once we got all this source material together, we started experimenting with integration techniques, adapting a technique utilizing sound states in Wwise tied to our animations to trigger different impacts based on animation data. Towards the end of the project, we passed off our combat sounds to Mike Niederquell from Sony for some final polish and for him to work the magic he does. We ended up abandoning our state-based multi-impact approach and Mike made bespoke content from our recordings for the various impact types as the animations weren’t so excessive, and the state rigging introduced unnecessary complexity.
Here’s a short video of some of the props that sacrificed themselves for the greater combat good as well as how they ended up sounding in game:
