Tech
Once a stream reaches the Media Server, it can be transcoded: decoded and encoded again with another codec, resolution or bitrate. It doesn't matter how it was published — WebRTC, RTMP, RTSP, MPEG-TS over SRT, or any other supported protocol.
Transcoding happens entirely on the server side. The Media Server decodes the incoming stream and encodes its picture and sound again, as the viewer or your backend asks: H.264, H.265 or VP8 video, Opus, AAC, G.711 or G.722 audio, another size, frame rate or bitrate.
The transcoded stream is like any other on the server: it can be played over WebRTC, HLS, RTSP or RTMP, recorded, mixed and pushed to YouTube or another server.
Transcoding takes server CPU, or a GPU: with an NVIDIA card the server encodes and decodes video with NVENC and NVDEC.
When a viewer plays a stream as it was published, the Media Server passes it on untouched. It turns transcoding on by itself when the viewer asks for something the stream does not have: another codec, such as VP8 for a stream published in H.264, a resolution or a bitrate.
A resolution or a bitrate set by the player always means transcoding, even when it matches the published stream: a browser can change its resolution while it publishes, and the server keeps the picture to the size the viewer asked for.
For the first viewer to ask for this picture, the server starts a transcoding group, keyed by what was asked for: Group = {H.264, 640, 360}. Every later viewer who asks for 640×360 in H.264 joins the same group and reuses its encoder, so a hundred viewers of one size cost the server one encoding, not a hundred.
Now let's see how to ask for a transcoded stream from each SDK.
In every SDK the viewer plays the stream by its name, as usual, and sets the picture it wants in the constraints of the stream it plays.
In a browser, the Web SDK plays the stream with the size and bitrate the viewer wants:
session
.createStream({
name: 'alice',
display: remoteVideo,
constraints: {
audio: true,
video: { width: 640, height: 360, bitrate: 500 },
},
})
.play();The bitrate is in kbps. stripCodecs leaves a codec out of the negotiation: with H264 left out, the viewer gets VP8.
On Android the size and the bitrate go into the video constraints of the stream to play:
StreamOptions options = new StreamOptions("alice");
VideoConstraints video = new VideoConstraints();
video.setResolution(640, 360);
video.setBitrate(500);
options.getConstraints().setVideoConstraints(video);
Stream stream = session.createStream(options);
stream.play();The server transcodes alice to 640×360 and 500 kbps for this viewer only; the others get the stream as it was published.
On iOS the size and the bitrate go into the stream’s constraints too:
let video = FPWCSApi2VideoConstraints()
video.minWidth = 640
video.maxWidth = 640
video.minHeight = 360
video.maxHeight = 360
video.bitrate = 500
let constraints = FPWCSApi2MediaConstraints()
constraints.audio = FPWCSApi2AudioConstraints()
constraints.video = video
let options = FPWCSApi2StreamOptions()
options.name = "alice"
options.display = remoteView
options.constraints = constraints
let stream = try session.createStream(options)
try stream.play()The same request as in the browser: alice at 640×360 and 500 kbps.
Your backend can transcode any stream published on the Media Server through the REST API, whatever protocol it came in with, and give the result a name of its own: one stream for every viewer who needs it, made once.
The backend starts a transcoder with /transcoder2/startup, giving it an identifier like transcoder2://alice-360p, the name of the stream to transcode and the name of the transcoded stream, with the encoder settings.
A 360p VP8 copy of a published stream, and the end of it:
const api = 'https://media.example.com:8444/rest-api/v3';
const post = (method, body) =>
fetch(`${api}/transcoder2/${method}`, {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify(body),
});
const uri = 'transcoder2://alice-360p';
await post('startup', {
uri,
remoteStreamName: 'alice',
localStreamName: 'alice-360p',
encoder: { videoCodec: 'VP8', width: 640, height: 360 },
});
// when it is no longer needed
await post('terminate', { uri });Without encoder the stream is encoded again with its own parameters. startup answers 200 OK when the transcoder is up within a few seconds, or 202 Accepted with a job to check later.
Since the transcoder draws every frame anew, it can put a watermark over it: a PNG picture, from the server or by URL, a line of text, or both at once. /transcoder2/set_watermark puts it over a running transcoder’s stream, and moves, changes or removes it on the fly, without stopping the stream.
All of it happens while the stream plays: the picture comes up, moves to a corner, gets a caption under it and goes back, and every viewer of the transcoded stream sees each change at once. The source stream stays as it was published, so its own viewers get no watermark at all.
With the post helper from above:
A logo and a caption over the transcoded stream, as soon as it runs:
await post('set_watermark', {
uri: 'transcoder2://alice-360p',
watermark: 'https://example.com/logo.png',
text: 'LIVE',
fontSize: 40,
textAlign: 'TOP_LEFT',
});A file name alone is looked up in the server’s conf folder; a full path or an http(s) URL works too.
The logo to the top left corner, at least 20 pixels from the edges:
await post('set_watermark', {
uri: 'transcoder2://alice-360p',
watermark: '/opt/media/logo.png',
x: 10,
y: 10,
marginTop: 20,
marginLeft: 20,
});A margin larger than its coordinate wins. An empty watermark takes the picture off.
A new caption at the bottom, on a half-transparent background:
await post('set_watermark', {
uri: 'transcoder2://alice-360p',
text: 'Back in 5 minutes',
fontSize: 32,
textAlign: 'BOTTOM_CENTER',
textColour: '#FFFFFF',
textBackgroundColour: '#000000',
textBackgroundOpacity: 50,
});A transcoder is started with /transcoder2/startup, and its encoder says what the transcoded stream is: the video codec, its size, frame rate, key frames and bitrate, the audio codec, sample rate, channels and bitrate, and which encoder does the work.
The encoder is FF or OpenH264 on the CPU, OpenH264 in several threads for 720p and above, or NVENC on an NVIDIA GPU, which takes the load off the CPU.
Let’s take a closer look at the available controls.
Every control above is a parameter of the encoder in /transcoder2/startup. With the post helper from the REST API example:
The video and the audio of the transcoded stream, all of it set:
await post('startup', {
uri: 'transcoder2://alice-360p',
remoteStreamName: 'alice',
localStreamName: 'alice-360p',
encoder: {
videoCodec: 'H264',
width: 640,
height: 360,
fps: 30,
keyFrameInterval: 60,
bitrate: 500,
type: 'NVENC',
audioCodec: 'opus',
audioRate: 48000,
audioChannels: 2,
audioBitrate: 64000,
},
});The video bitrate is in kbps, the audio bitrate in bps. Audio settings for a stream with no audio, or video settings for one with no video, get 400 Bad request.
No setup, no installation. Publish your camera, set the width, height or bitrate to play it at, and press Transcode: the Media Server transcodes the stream for you, and the player shows the resolution it gets.
The stream could not be played.
LLcast will deploy a dedicated Media Server just for you, ready to transcode your streams. Publish over WebRTC, RTMP, SRT or RTSP, and give every viewer the codec, size and bitrate they need, on request or over the REST API, on the CPU or the GPU.
