LiveVoice shares how they use AnyCable

Simultaneous interpretation and guided tours powered by AnyCable Pro

In early 2024, the co-founder of LiveVoice.io, Sebastian Poell, reached out with some feedback for AnyCable:

LiveVoice is completely running on AnyCable - and it really rocks ;) I have to say, seeing the CPUs on an AWS c6g.2xlarge instance at only 6% utilization for 10K listeners is pretty crazy (in a very positive way) ^^ Aside from performance, using AnyCable fixed every issue we ever had with ActionCable (broken connection detection, race conditions, ...).

We got excited and asked Sebastian for a few more details about the use case. But could we have imagined that Sebastian would write not just some sentences, but a full story of LiveVoice and the role of AnyCable in it?!

Without further ado, Sebastian’s story:

About us

LiveVoice is an ultra-low-latency audio streaming SaaS platform, used mostly for simultaneous interpretation, guided tours, silent stages and audio description. What we do is basically transmitting audio and video from one point of the Earth to another with as little delay as possible (~0.2s), coupled with a friendly UI and tons of other features.

With LiveVoice, you can set up events and channels and invite your speakers and your audience. The speakers and the audience can then use our mobile apps or the website to simply join the event and speak / listen to your channels. You can imagine it a bit like Skype or Zoom, but instead of many people talking to each other, only one person speaks and thousands of people listen to them.

Our setup

Not only do we have to transmit audio/video data very fast to a lot of people, we also have to do a ton of signaling. For example, if a speaker presses the "mute" button, we need to inform the whole audience to display the muted state. Ok, that is simple. But what about allowing one speaker to switch between different channels? Or allowing a speaker to take over a stream from another active speaker? How do we handle connection losses and reconnects? When it comes to live events, being reliable in those scenarios becomes more challenging quickly.

Our backend is built with Rails and we "grew up" using Action Cable. For listeners and speakers, we provide a web interface as well as native apps for Android and iOS. Regarding the apps, building them natively turned out to be the only viable option for us. Hence, we had to build our own native Action/AnyCable libraries in Kotlin and Swift. Right now, there are about 20 inbound message types and 35 outbound message types. To always guarantee compatibility with old clients, we introduced versioning for cable channels (and messages) on the server.

Challenge 1: Broken Connection Detection

Whenever a speaker loses the network connection, we have to inform the audience and the other speakers that the speaker went offline as fast as possible. Just think of a guided tour in a city, where people walk around, having an unstable internet connection. For us, the (almost) instant, server-side detection of those broken connections is of utmost priority.

Since Rails does not support this kind of server-side broken connection detection as of now (although there are promising PRs), a couple of years ago, we introduced our own version of a "pong". Every time a client received a "ping" from the server it responded with a "pong" back to the server. We also implemented a timeout using periodic timers and patched Rails to accept a "pong" format, similar to "ping". Using this workaround, we could set speakers who timed-out to offline within a couple of seconds.

As we wanted to migrate from ActionCable to AnyCable, we tried out different approaches to migrate or circumvent that pong feature, but got stuck. Evil Martians to the rescue! We reached out and you (editor’s note: Martians) implemented that pong feature directly into AnyCable server. Exactly what we needed; even better because terminating the command in Go without the need of RPC is super performant.

Since we were already using the AnyCable JS client library before even migrating our backend to AnyCable, we did not even had to implement the client side "pong" responses in the web client, we just had to enable them.

Challenge 2: Tidy up!

I think, one very common pitfall when using cables is that code often looks like it should happen synchronously, but in fact happens asynchronously. For example:

stream_from("my_stream")
ActionCable.server.broadcast("my_stream", { hello: :world })
stop_stream_from("my_stream")

Does the client receive the message? Well, it’s random, depending on when the broadcast is done (note that the behaviour of Action Cable and AnyCable regarding when streams are subscribed differ). Take another example:

ActionCable.server.broadcast("my_stream", { hello: :world_1 })
ActionCable.server.broadcast("my_stream", { hello: :world_2 })

Both messages will be received by the clients without the guarantee of any particular order. AnyCable helps us here by allowing us to send broadcasts in batches making sure broadcasted messages are always sent in order, which increases reliability and reduces the need for workarounds.

Also, we occasionally found ourselves in the situation that the current user who performed an action needed to get another response from the server than all the other clients. For example, when a speaker switches the channel, we need to send a lot of information to this speaker but all other speakers should only get a very brief message that a switch happened. AnyCable's broadcast-to-other feature came in super handy here. Now we can send different responses to the current user and to all other users:

ActionCable.server.broadcast("my_stream", { type: :another_speaker_switched }, to_others: true)
transmit({ type: :special_response_for_current_speaker })

This allowed us to tidy up our responses, send fewer messages in general (which is also easier to debug and to test) and simplified our client-side code bases.

Challenge 3: Performance

When running a streaming service, load testing is one important part of what lets us sleep well at night. Thus, we implemented our own CLI load-testing tool, based on our iOS client libraries (in Swift). This is especially useful, because with our own libraries, we can test realistic-behaving LiveVoice clients without the need to implement them in tools like k6 or Selenium. Our simulated listeners join an event, press start/stop once in a while and are streaming real audio. When testing AnyCable, we found that 10K listeners only consumed approx. 6% of the server’s CPU —which was pretty much unheard-of for us 😁

Stack: AWS c6g.2xlarge instance (ARM64), Debian 12, Ruby 3.3 + YJIT, AnyCable 1.4 (via HTTP RPC, pongs enabled), Jemalloc 5.3, Rails 7.1, Puma 6.4, Redis 7, Nginx 1.22. Note, that this server does not handle the actual audio/video streaming or the database, but handles all the web traffic and API requests, as well as all websocket connections.

Outlook

We are super happy with migrating to AnyCable and we'll definitely continue to dive into all the features (and there are a lot!). Especially relevant to our use case are features like optimized messaging formats or multi-regional scaling. An exciting future ahead 🫡 and cheers to you, Evil Martians!