One of the best-selling things in terms of technology is so-called three-dimensional sound, which relies on the positioning of the sounds produced in the scene according to the position of the viewer or player, but always taking into account a system of physical speakers. .
The idea of Ray Tracing applied to audio is very simple, using the ray tracing algorithm so as not to know the trajectory of light photons but of all sound waves in an environment and the way they reach the viewer. All this in a much more precise and less limited way than the speaker systems used in the cinema.
Audio ray tracing: sound waves instead of photons
Ray Tracing Audio follows the same principles as its graphic variant, but instead of representing the travel of photons in a scene, the same algorithm and principle is used to represent the travel of sound waves in an environment.
So we are talking about representing how sound travels in three dimensional space and we can use the same model as ray tracing which represents the travel of light in three dimensional space but with sound.
- As with light we can assign energy to sound, the difference with light is that energy in this case is lost with distance.
- We can affect the ray generation shader (reflections, shadows) to generate audio rays as well.
- As with graphic rays, audio rays can bounce and behave in one way or another depending on the nature of the objects with which they interact.
The last point is important, because when a sound wave leaves one medium to enter another, for example when a sound bounces off a wall, the sound wave changes. Part of it will attempt to pass through the medium while another will be bounced off by the object and this process will vary depending on the type of material it is interacting with.
Not all sound waves are important
What interests us in the case of Audio Ray Tracing is only what the player can hear in the scene, any ray which, because it is far away or does not reach the player’s area of influence, would be rejected.
There’s no point in moving around in the midst of a sonic cacophony that would drive the player completely crazy and completely destroy the immersion in the game.
The best way is for the GPU to simply calculate the path from the original source to the player’s position in the scene. Unlike its graphic variant, we would not speak of generating new waves in the scene by a bounce, but rather its distribution which would depend on its energy, which would result in the intensity with which we hear the said sound, if the intensity it is too low so it is thrown away.
In ray tracing via graphics, we have to take into account the entire trajectory of the photon and how it “paints” the scene it passes through by combining its colors with existing ones, in the case of ray tracing via audio , just take into account the end of the sound path and whether it affects the player or not.
This implies that when the audio beam passes through a part of the stage far from the player, the audio would not be generated but would continue and only the sound that would occur in an area close to the player would be processed.
GPU for Audio Ray Tracing
First of all, we are going to need a GPU to calculate the path of the sound waves, so its use completely affects the performance of the graphics card, because we are using the specialized hardware of it to calculate the intersection. rays with the objects to create a new type of ray, but with the difference that it would not manipulate the color values of an object or a group of objects but rather it would manipulate the audio of the Stage.
The problem is that this would mean that audio tracks would have to be processed through shader programs, meaning that the audio tracks they would have to work with would be extremely small and therefore would have to lose detail and resolution if executed from there. ‘calculation units. because these have very few kilobytes.
Audio tracks are loaded as textures, keep in mind that a GPU’s caching system doesn’t load a huge texture, but if you are loading an audio track you will need at least some hardware. built-in fast enough to decompress audio tracks. on the fly and could therefore be another additional component.
3D helmet and immersion
In the market, we have seen the recent appearance of 3D headphones whose particularity is that they have integrated three-dimensional positioning elements that allow the listener to be placed in the middle of a virtual stage.
This type of headphones will benefit a lot from Ray Tracing Audio, for example in a future game we can turn our heads, see how our character turns it slightly, with this the position of the camera also turns and capture the sounds according to the new one. position of objects according to the camera. The idea of the gameplay will be that to have a simulated 7.1 in which the “cinema experience” is achieved is completed, but we are talking about each of the sounds of the game being represented not by a limited virtual speaker system, because each one from them will have its sound source represented for greater precision and immersion in the game.
The changes required in the current hardware to be able to take advantage of Ray Tracing Audio are really nil and its requirements are much lower than those for graphics rendering. As for the video game industry, it will cost nothing to apply them, only if, it can be an excuse for headphone manufacturers to create and sell new models.
Table of Contents
