Codecs
The linphone-sdk supports a large variety of audio and video codecs. Some are provided by the software, others by the platform (hardware codecs, typically for video ones).
Audio codecs
The below codecs are supported in linphone-sdk, for all platforms. These are software-provided codecs.
| Codec name | Built-in by default | Enabled by default | Remarks |
|---|---|---|---|
| opus | yes | yes | State-of-art high quality audio codec with configurable bitrate, up to 48khz |
| pcmu | yes | yes | Legacy ITU-G.711 |
| pcma | yes | yes | Legacy ITU-G.711 |
| speex | yes | no | |
| G729 | no, use -DENABLE_G729=ON | no | For proprietary-mode licencees, a licence extension is required. |
| gsm | yes | no | Legacy GSM codec. |
| G722 | yes | no | Low complexity wideband codec from ITU-G.722. |
| AMR | no, use -DENABLE_AMR=ON | no | 8khz Adaptive Multi-Rate codec. |
| AMR-WB | no, use -DENABLE_AMRWB=ON | no | 16 khz AMR variant |
| BV16 | no, use -DENABLE_BV16=ON | no | Low complexity wideband codec. |
| Codec2 | no, use -DENABLE_CODEC2 | no | Very low bitrate codec |
When linphone-sdk has support for a codec, the codec enablement is controlled dynamically from PayloadType.enable() . For example, activating a given codec may be done using:
if (!payloadType.enabled) payloadType.enabled = true
The default preference order of audio codecs is as follows:
- opus/48000
- speex/16000
- speex/8000
- pcmu/8000
- pcma/8000
The codec order may be altered by retrieving and setting the audio payload type list using Core.getAudioPayloadTypes() and Core.setAudioPayloadTypes().
Video codecs
Availability and enablement
The following video codecs are supported, depending on target platform.
| Codec name | MacOS/iOS availability | Windows | GNU/Linux | Android | Enabled by default | Remarks |
|---|---|---|---|---|---|---|
| AV1 | yes | yes - except UWP target | yes | yes | yes | Software implementation from Dav1d and Aom |
| H265 | yes - hardware accelerated from VideoToolbox framework | no | no | yes - hardware accelerated from MediaCodec API | yes | |
| H264 | yes - hardware accelerated from VideoToolbox framework | yes, see notes about OpenH264 below | yes, see notes about OpenH264 below | yes - hardware accelerated from MediaCodec API | yes | |
| VP8 | yes | yes | yes | yes | yes | Software implementation from libvpx. |
Preference order policy
As of today, AV1 and H265 are the most advanced codecs in terms of quality and compression.
However, the hardware acceleration is the most important factor to take into account. Indeed, a software implemented codec will reach CPU consumption limits above a certain video resolution and framerate. In addition, a software codec drains battery faster, and causes the device to heat due to intense CPU power required.
For all these reasons, it is always preferable to use hardware accelerated codecs over software ones. They are able to achieve greater resolution and frame rate, in all situations, with better fluidity.
Liblinphone thus implements a specific SDP offer/answer policy in order to favour the use of hardware accelerated codecs in video calls.
Typically, if an offerer proposes AV1, H264, VP8 in this preference order, but the answerer has hardware acceleration for H264, but not for AV1, H264 will be selected for the call. See Core.setVideoCodecPriorityPolicy() for more information about this feature.
Manual configuration of video codec preference order is possible using Core.setVideoPayloadTypes().
About OpenH264
H264 is provided on Windows and GNU/Linux platforms thanks to the open-source Cisco's OpenH264 implementation. It is not included by default in linphone-sdk because to patents constraints.
The following linphone-sdk compilation options must be give:
- -DENABLE_OPENH264=ON in order to compile openH264 support (msopenh264 plugin)
- -DENABLE_EMBEDDED_OPENH264=ON in order to include the openh264 library inside the generated SDK binaries: in this case, as a distributor of the app you become responsible for acquiring a patent licence from MPEG-LA.
- use -DENABLE_EMBEDDED_OPENH264=OFF in order to NOT include the openh264 library inside the SDK binaries. In that case, the app can download at run-time the openH264 library binary from Cisco, in order to benefit from a free-of-charge patent licence.
Despite it is technically possible, there is no interest in using OpenH264 on platforms where the hardware accelerated H264 codec is supported (Android, iOS, Mac OS).