What’s New?
All new virtual studios created will now have loss concealment enabled by default. Loss concealment is a new buffering technique that uses artificial intelligence to improve audio for less-than-perfect Internet connections.
You can enable loss concealment on existing studios by selecting it as the “Queue Strategy” under the “Settings” tab.
Challenges with Audio over the Internet
Sending audio over the Internet involves breaking it up into tiny packages, where each package contains a few microseconds of signal. Unfortunately, the Internet isn’t perfect. Those packages can get delayed or even lost, and this happens far more often than most people realize.
Traditionally, online collaboration tools like Zoom, Meets or Teams try to adapt to this using two techniques:
-
Using buffers, which is like putting each arriving package into a storage room. The room allows packages to arrive at different times, while enabling them to be removed at a constant rate to produce the sound you hear. The larger the room, the more tolerance it has for lateness.
-
Using retries, which is kind of like calling someone up on the phone and asking them to resend a package because you never received it. This helps avoid gaps in audio when packages are lost.
Low-Latency Audio
Unfortunately, both of these common techniques are the enemy of live collaboration. They both introduce significant delays between the time you send audio and another person being able to hear it. This delay is often referred to as latency, and the latency for most online collaboration platforms range from 1/10 to 1/2 of a second.
That delay may be bearable for a simple conversation between two people. You may not even notice it when you talk with someone on Zoom or over your cell phone. But live music collaboration has a far lower tolerance for latency, in the range of 1/100 to 1/50 of second (about 20 times shorter). More than that and you will have a really hard time maintaining the tempo even for the slowest of songs.
Loss Concealment
JackTrip takes a very different approach. Some amount of buffering is still required, but it tries to use the smallest buffers possible to minimize latency. And it never tries to recover from lost audio packages. If it did, the latency would be so high that there would be no point. This approach can sometimes introduce gaps, where packages are missing in your audio stream.
What to do with these gaps has been a longstanding question. Sometimes audio platforms will fill them in with the last signal received. Othertimes, they may fill in the gaps with silence. Both produce unnatural experiences including clicks, pops and other oddities.
Loss concealment is a new approach that leverages artificial intelligence techniques. JackTrip keeps track of the history of the audio packages it receives from someone, and uses that history to predict what was likely contained in any packages that have been lost or delayed. Rather than filling in gaps with silence or old packages, it uses those predictions.
This technique isn’t perfect and cannot correct for all scenarios; however, we have found that it works remarkably well for most. In many cases, you can no longer hear the glitches that previously represented network issues. In others, you may hear something but it is less likely to impact your abililty to understand them or play along.
In addition to studios, loss concealment can be easily enabled using the “Advanced” settings for any bridge devices. While the latest desktop app includes support for loss concealment, we are still working out some bugs with that. This is only the beginning and we plan to continue improving upon it over time.