diff --git a/site/content/entry/accurate-audio-emulation.md b/site/content/entry/accurate-audio-emulation.md index d6cd1c7..f339f41 100644 --- a/site/content/entry/accurate-audio-emulation.md +++ b/site/content/entry/accurate-audio-emulation.md @@ -17,7 +17,7 @@ You've been asking for it for years now, and we've been listening, we promise! -Thanks to the hard work of one of our very talented developers, some of Citra's longest standing issues are finally fixed! Special thanks to all those who are supporting these efforts on [Patreon](https://www.patreon.com/citraemu). These donations are given directly to support the hard working developers such as [wwylele](https://github.com/wwylele/) who spent almost an entire year of his spare time on the feature in this blog post! We love working on this project, and have a whole lot more to talk about in the coming weeks! +Thanks to the hard work of one of our very talented developers, some of Citra's longest standing issues are finally fixed! Special thanks to all those who are supporting these efforts on [Patreon](https://www.patreon.com/citraemu). These donations are given directly to support the hardworking developers such as [wwylele](https://github.com/wwylele/) who spent almost an entire year of his spare time on the feature in this blog post! We love working on this project, and have a whole lot more to talk about in the coming weeks! It's been a long time in development, but we are finally pleased to announce that many of the oldest bugs in Citra are now fixed in the latest Canary build thanks to the tireless efforts of [wwylele](https://github.com/wwylele/). Among the titles that had issues with Citra's HLE audio emulation, one stands out as the number one most requested game of all time: Pokémon X and Y. @@ -39,7 +39,7 @@ In the Audio tab of the Configuration menu, there is a new option for Emulation. * Why is it so slow on every game? -**We still recommend using HLE (Fast) Audio emulation until further speed improvements are added to LLE (Accurate) emulation! Today we are pleased to announce that many bugs are fixed, but also we are aware that LLE (Accurate) audio makes every game run at around 5FPS or less!** +**We still recommend using HLE (Fast) Audio emulation until further speed improvements are added to LLE (Accurate) emulation! Today we are pleased to announce that many bugs are fixed, but we are also aware that LLE (Accurate) audio makes every game run at around 5FPS or less!** As you'll see in the rest of the article, this feature has been in development by a single developer for almost a whole year now. During this time, the focus was on accurate emulation, but now that it's released, we can and will put effort into optimizing it. @@ -80,14 +80,14 @@ But there's a very different way to handle audio, and this is hinted about at th ## Debugging Pokémon X Told Us That It's Broken but not WHY Before looking in depth at what LLE audio emulation is all about, a quick diversion into the debugging effort that went into Pokémon X / Y is in order. -Looking at the [Citra YouTube channel](https://www.youtube.com/channel/UC_dcdgzuapBtAY4ol3x-90Q), one will find several videos talking about the games progress! +Looking at the [Citra YouTube channel](https://www.youtube.com/channel/UC_dcdgzuapBtAY4ol3x-90Q), one will find several videos talking about the games' progress! At the time, we were just as excited as everyone else to see how well the games were advancing, but at some point, they stopped getting better. After CRO support landed to get the games running (thanks to both [Subv](https://github.com/Subv/) and [wwylele](https://github.com/wwylele/)), several fixes to the 3DS GPU emulation, followed by geometry shader support (thanks to the hard work of [neobrain](https://github.com/neobrain/), [JayFoxRox](https://github.com/JayFoxRox/), [ds84182](https://github.com/ds84182), and once again [wwylele](https://github.com/wwylele/)), we really hoped that the games would finally start working! -As everyone knows, they still didn't work! +But as everyone knows, they still didn't work! -We kept feeding more and more time into features that made the emulation so much better, yet this one very popular, and very stubborn game would not work. +We kept feeding more and more time into features that made the emulation so much better, yet this one very popular and very stubborn game would not work. [Subv](https://github.com/Subv/) spent many long hours reverse engineering the game, and found that at the core of the game lay a state machine that drives the game engine. -The game would transition from state to state, and mysteriously whenever the game softlocked, it simply wasn't moving onto the next state. +The game would transition from state to state, and, mysteriously, whenever the game softlocked, it simply wasn't moving onto the next state. As cool as it is to learn what causes the softlock, it doesn't answer the big question of *why* the game doesn't transition to the next state like it should. After spending more time than anyone could have asked, eventually he burned out and moved on to develop other amazing features for Citra such as multiplayer network support, leaving us without any more clues to why the game freezes. @@ -102,17 +102,17 @@ AAC (or [Advanced Audio Coding](https://en.wikipedia.org/wiki/Advanced_Audio_Cod While we appreciate the detective work, there was one glaring problem with deciding that we just needed to add AAC audio support. How *does* one add AAC audio support to Citra's HLE audio? -The answer was talked about earlier, in order to add a new feature in HLE audio, one needs to reverse engineer the audio code that the games use, and then find out exactly how the audio code processes AAC audio; where the data is passed through, where each decoding option is stored, and all the way down what every bit of data written to the audio pipe semantically means. -To make matters worse, there's no guarentee that this will fix any other games with similar symptoms. +The answer was talked about earlier: in order to add a new feature in HLE audio, one needs to reverse engineer the audio code that the games use, and then find out exactly how the audio code processes AAC audio; where the data is passed through, where each decoding option is stored, and all the way down to what every bit of data written to the audio pipe semantically means. +To make matters worse, there's no guarantee that this will fix any other games with similar symptoms. After all, a game can upload any code it wants to the audio chip, meaning even if they both used AAC audio, they could potentially use different decoding options, causing the HLE AAC decoder to work for one game and not the other. And worst of all, it's possible that everyone is wrong, and X/Y are freezing because of a completely unrelated issue! Faced with this dilemma, [wwylele](https://github.com/wwylele) designed a test to show that audio is likely the cause of the softlock. -He knew from the get go that the HLE audio code was based off reverse engineering the most common audio code that games use, the home menu, and decided to make a custom rom hack for Pokémon X that replaced it's audio code with the home menu audio. +He knew from the get go that the HLE audio code was based off reverse engineering the most common audio code that games use, the home menu, and decided to make a custom rom hack for Pokémon X that replaced its audio code with the home menu audio. Upon launching the game on a 3DS, everything seemed fine at first, but soon enough the familiar lack of background music kicked in, and Pokémon X started to behave exactly like it does in Citra. One short level up later, and the game froze, just like in Citra! -We double checked the results by recreating this on different copies of Pokémon X and different 3dses, and it all went the exact same way. -Audio issues is very likely the cause, but what then should be the fix? +We double-checked the results by recreating this on different copies of Pokémon X and different 3DSes, and it all went the exact same way. +Audio issues were very likely the cause, but what then should be the fix? {{< figure src="/images/entry/accurate-audio-emulation/dp.png" title="Detective Pikachu cutscenes no longer skip parts and now run at the proper speed" @@ -120,14 +120,14 @@ Audio issues is very likely the cause, but what then should be the fix? ## The Long Road to LLE Audio Emulation -The methods for fixing any audio issues boils down to two potential solutions. +The methods for fixing any audio issues boil down to two potential solutions. Either take a good amount of time to research, reverse engineer, and recreate the audio code for Pokémon X/Y in HLE, or build out a program that can read the original binary audio code in Pokémon X/Y and emulate the actual audio chip, known as LLE. Both of the options have advantages and disadvantages. -HLE means it'll be faster overall to get working, but also will likely take a lot more time and effort to fix bugs in other games with slightly different audio code; whereas LLE means that potentially every audio code in every game will *just work*, but also will take a lot longer to write, involve even more detailed technical research, and scariest of all, will probably end up running much slower. +HLE means it'll be faster overall to get working, but also will likely take a lot more time and effort to fix bugs in other games with slightly different audio code; whereas with LLE, potentially all audio code in every game will *just work*, but it also will take a lot longer to write, involve even more detailed technical research, and scariest of all, will probably end up running much slower. Weighing the risks and rewards, [wwylele](https://github.com/wwylele) ended up choosing to build out a full LLE audio emulator, what's now known as Teakra. -The first commit on Teakra started in late January 2018, but design and research phase started before this. -This same audio chip, the TeakLite, was a part of the DSi system as well, albeit only used by a very small number of DSiWare. +The first commit on Teakra started in late January 2018, but the design and research phase started before this. +The audio chip in the 3DS, the TeakLite, was a part of the DSi system as well, albeit only used by a very small number of DSiWare titles. In spite of its limited use on the DSi, [GBATek](http://problemkaputt.de/gbatek.htm#dsixpertteakdsp) is loaded with valuable information about the processor, put together by the reverse engineering efforts of Martin Korth and many other contributors. There are others who have worked hard to write documentation, which [wwylele](https://github.com/wwylele) referenced often throughout development. @@ -151,24 +151,24 @@ It's daunting to know that you'll gain a deep understanding of the architecture ## Wait, How Long Until This Starts Playing Sound? Several months after starting, the interpreter was coming along nicely, and it was time to start hooking things together. -As a quick explaination about what interpreter is, the interpreter is responsible for running the actual machine code of the audio code, and emulating each of the instructions on the computer that it's running on. -This marked another very exciting yet very scary point in time; it's been about 5 months of work on audio emulation, and all of it in complete silence! -To understand why, first one must understand that audio hardware is rather complicated, and it's more than just the running machine code. +As a quick explanation about what the interpreter is, the interpreter is responsible for running the actual machine code of the audio code, and emulating each of the instructions on the computer that it's running on. +This marked another very exciting yet very scary point in time; it had been about 5 months of work on audio emulation, and all of it in complete silence! +To understand why, first one must understand that audio hardware is rather complicated, and it's more than just running the machine code. Anything beyond simple audio output is typically generated through specialized hardware known as a [Digital Signal Processor](https://en.wikipedia.org/wiki/Digital_signal_processor), or DSP for short. -This custom built hardware is very efficient at transforming and processing audio samples while using less power, enabling the game developers to produce high quality audio without severely impacting battery life. +This custom built hardware is very efficient at transforming and processing audio samples while using less power than the CPU would, enabling the game developers to produce high quality audio without severely impacting battery life. In order to start seeing results, [wwylele](https://github.com/wwylele) needed to emulate the ways that the 3DS communicates with the DSP chip, such as [Memory Mapped IO](https://en.wikipedia.org/wiki/Memory-mapped_I/O), [hardware interrupts](https://en.wikipedia.org/wiki/Interrupt), and [Direct Memory Access](https://en.wikipedia.org/wiki/Direct_memory_access). -The first two are a piece of cake to recreate when compared to DMA. -There are straightfoward test cases one can write and run on a real 3DS to verify how the 3DS and TeakLite DSP respond and recreate this in Teakra, and after a month of work, [wwylele](https://github.com/wwylele) got them both functional. +The first two are a piece of cake to recreate compared to DMA. +There are straightforward test cases one can write and run on a real 3DS to verify how the 3DS and TeakLite DSP communicate and recreate this in Teakra, and after a month of work, [wwylele](https://github.com/wwylele) got them both functional. But DMA poses a unique challenge, as DMA enables the DSP to do direct memory to memory transfers, independent of what the CPU is doing, making it really hard to figure out how it works! -To add to the trouble, DMA is the least documented feature on [GBATek](http://problemkaputt.de/gbatek.htm#dsixpertteakdsp), meaning so there is nothing one can study to get an understanding of how the TeakLite does DMA transfers. +To add to the trouble, DMA is the least documented feature on [GBATek](http://problemkaputt.de/gbatek.htm#dsixpertteakdsp), meaning there is nothing one can study to get an understanding of how the TeakLite does DMA transfers. -Maybe, just maybe, DMA won't be needed to produce audio output! +Maybe, just maybe, DMA wouldn't be needed to produce audio output! Around August 2018, [wwylele](https://github.com/wwylele) hooked everything together except for DMA with this hope in mind and... Nothing. -Even worse, the games exhibited the same behavior from before HLE audio was added to Citra, implying that the games weren't detecting that anything was working at all. +Even worse, the games exhibited the same behavior as before HLE audio was added to Citra, implying that the games weren't detecting that anything was working at all. After spending a few weeks fiddling around with the code, and trying to rule out that it wasn't playing audio because of some bug in the existing code, [wwylele](https://github.com/wwylele) reaffirmed what he always knew. When it comes to writing an LLE DSP emulator, it's all or nothing. Either all of your code works, or absolutely nothing happens. @@ -180,17 +180,17 @@ Time to work on DMA.