# Audio Subsystem

For usage related information, see audio feature.

## Implementations

| Component | Link | |-------------------|------------------------------------------------------------------------------------------------------| | client | xpra.client.mixins.audio | | client connection | xpra.server.source.audio | | server | xpra.server.mixins.audio |

## Common

xpra.audio contains the components used for capturing and playing back audio streams using GStreamer.
In order to avoid interfering with the performance of the main thread, all audio processing is done in a separate process.
For historical reasons, this is done using a subprocess wrapper rather than the builtin multiprocessing module.

### Pulseaudio

xpra.audio.pulseaudio is often used for playback on Linux systems.
This is also the prefered backend for audio capture in server sessions. The xpra server will usually start a pulseaudio instance hidden away in a per-session user prefix so that multiple sessions can forward audio streams independently.

## Capabilities

The client and server should expose the following capabilities in their `hello` packet with the `audio` prefix:

| Capability | Type | Purpose | |------------|-----------------|----------------------------------------------------| | `decoders` | List of strings | The audio formats that can be received and decoded | | `encoders` | List of strings | The audio formats that can be encoded and sent | | `send` | boolean | If sending audio is enabled | | `receive` | boolean | If receiving audio is enabled |

The lists of `decoders` and `encoders` contain strings such as: `mp3`, `opus+ogg`, `vorbis`...
You can run xpra.audio.gstreamer_util to see which encoders and decoders are available on the system.

## Network Packets

This protocol is identical in both directions. Audio being forwarded from the server to the client (aka "speaker forwarding") uses the same packets as audio coming from the client to the server (aka "microphone forwarding").

| Packet Type | Arguments | Purpose | Information | |----------------------|----------------------------------------------------------------------------------------------|----------------------------|----------------------------------------------------------------------------------------------------------------------------------| | `sound-data` | `codec` : string
`data` : bytes
`attributes` : dictionary | Audio stream data | The initial and final packets may omit the `data` argument and should set the `start-of-stream` / `end-of-stream` attributes | | `sound-control` | `subcommand` : string
(ie: `start`, `stop`, `sync`, `new-sequence`)
`argument` : Any | Send a request to the peer |