Building a Self-Hosted Low-Latency Live Video Streaming Platform with SRT and FFmpeg
Quick Summary (TL;DR)
- SRT handles contribution over unreliable networks with ARQ-based error correction, achieving 100–200 ms of added latency.
- FFmpeg transcodes the incoming SRT feed into adaptive bitrate renditions and packages them for distribution.
- Distribution choice dominates latency: WebRTC lands under 500 ms, LL-HLS around 2–3 s, standard HLS 6–10 s.
- Tune FFmpeg with
-tune zerolatency, small GOPs (-g 30), no B-frames, and hardware encoders (NVENC, QSV, VAAPI).- Budget for redundancy (SRT bonding), AES encryption, and Prometheus-based monitoring before you go live.
Introduction: The Need for Low-Latency Self-Hosted Streaming
Commercial streaming platforms are convenient, but they impose a tax: per-minute billing, opaque latency, and zero control over your pipeline. For interactive use cases — live auctions, remote production, betting, esports commentary, or internal broadcast — that tax is unacceptable. You need deterministic latency, full observability, and the freedom to tune every buffer.
Self-hosting a low-latency stack is now genuinely practical. The two workhorses are SRT (Secure Reliable Transport) for contribution and FFmpeg for transcoding. SRT solves the “last mile” problem of getting a clean feed from a camera or encoder over the public internet; FFmpeg turns that feed into the renditions your viewers actually consume. This handbook walks through architecture, configuration, tuning, benchmarking, and the production edge cases that will bite you at 2 a.m.
Architecture Overview: SRT Contribution and FFmpeg Transcoding
A production pipeline has four logical stages: capture → contribution → processing → distribution. SRT owns contribution; FFmpeg owns processing; a packager and origin own distribution.
flowchart LR
A[Camera / OBS<br/>SRT Caller] -->|SRT over UDP<br/>AES + ARQ| B[SRT Listener<br/>srt-live-transmit]
B -->|MPEG-TS / UDP| C[FFmpeg<br/>Transcoder]
C --> D[Renditions<br/>1080p/720p/480p]
D --> E[Packager<br/>HLS / DASH / WebRTC]
E --> F[Origin + CDN]
F --> G[Viewers]
C -.stats.-> H[Prometheus / Grafana]
B -.stats.-> H
SRT Protocol: Reliable Low-Latency Transport
SRT is built on UDT and adds a control mechanism, timestamp-based packet delivery, and Automatic Repeat reQuest (ARQ). Unlike TCP, which stalls the whole stream on a lost packet, SRT selectively retransmits only what’s missing, using a configurable latency buffer to absorb retransmission round-trips. The sender keeps packets in a buffer for the duration of that latency window; if an ACK/NAK round-trip completes in time, the packet is recovered transparently.
Key parameters you’ll tune constantly:
| Parameter | Purpose | Typical Value |
|---|---|---|
latency | Receiver buffer for retransmits | 120–200 ms |
rcvlatency / peerlatency | Asymmetric tuning | = latency |
passphrase | AES encryption | 16–32 chars |
pbkeylen | AES key length | 16 (128-bit) |
maxbw | Bandwidth cap | 1.3× stream bitrate |
mode | caller / listener / rendezvous | caller (encoder) |
The rule of thumb: set latency to at least 2–3× your network RTT. Too low and you drop packets; too high and you’ve thrown away your latency advantage.
FFmpeg as the Transcoding Engine
FFmpeg ingests the SRT stream, decodes it, and re-encodes into multiple renditions. It’s the most battle-tested media Swiss Army knife in existence, and with --enable-libsrt it speaks SRT natively. The critical insight is that FFmpeg’s default settings are optimized for quality and throughput, not latency — you must explicitly fight those defaults.
Distribution: HLS, DASH, or WebRTC?
This decision, more than anything else, determines your end-to-end latency.
| Protocol | Typical Latency | Scale | Complexity | Best For |
|---|---|---|---|---|
| WebRTC | 200–500 ms | Moderate | High | Truly interactive |
| LL-HLS | 2–4 s | Excellent | Medium | Large audiences |
| LL-DASH | 2–4 s | Excellent | Medium | Open standards |
| Standard HLS | 6–10 s | Excellent | Low | VOD-like live |
Most self-hosters land on LL-HLS for scale with WebRTC as an optional low-latency tier for premium viewers.
Setting Up the SRT Listener and FFmpeg Pipeline
Installing SRT and FFmpeg with SRT Support
On Ubuntu, build from source for full control:
# SRT library
git clone https://github.com/Haivision/srt.git
cd srt && ./configure && make -j$(nproc) && sudo make install
# FFmpeg with SRT
git clone https://github.com/FFmpeg/FFmpeg.git
cd FFmpeg
./configure --enable-libsrt --enable-gpl --enable-nonfree \
--enable-libx264 --enable-libx265 --enable-nvenc
make -j$(nproc) && sudo make install
Verify with ffmpeg -protocols | grep srt. Prebuilt static binaries (BtbN, John Van Sickle) also ship SRT support if you’d rather not compile.
Configuring the SRT Listener
The simplest listener is srt-live-transmit, which relays SRT to a local UDP/TCP endpoint:
srt-live-transmit \
"srt://:1234?mode=listener&latency=200&passphrase=MySecret123&pbkeylen=16" \
"udp://127.0.0.1:5000?mode=caller"
Alternatively, let FFmpeg listen directly:
ffmpeg -i "srt://:1234?mode=listener&latency=200" ...
The srt-live-transmit approach is preferred in production: it isolates transport from transcoding, so a transcoder crash doesn’t drop the contribution feed.
FFmpeg Transcoding Command with SRT Input
Here’s a production-shaped command producing three renditions:
ffmpeg -fflags nobuffer -flags low_delay \
-i "srt://127.0.0.1:5000?mode=caller&latency=200" \
-filter_complex "[0:v]split=3[v1][v2][v3]; \
[v1]scale=1920:1080[v1out]; \
[v2]scale=1280:720[v2out]; \
[v3]scale=854:480[v3out]" \
-map "[v1out]" -c:v libx264 -preset veryfast -tune zerolatency \
-b:v 4500k -g 30 -bf 0 -f flv rtmp://origin/1080p \
-map "[v2out]" -c:v libx264 -preset veryfast -tune zerolatency \
-b:v 2500k -g 30 -bf 0 -f flv rtmp://origin/720p \
-map "[v3out]" -c:v libx264 -preset veryfast -tune zerolatency \
-b:v 1000k -g 30 -bf 0 -f flv rtmp://origin/480p \
-map 0:a -c:a aac -b:a 128k -f flv rtmp://origin/audio
Note -fflags nobuffer and -flags low_delay at the input — these prevent FFmpeg’s demuxer from accumulating frames before starting.
Optimizing FFmpeg for Low-Latency Transcoding
Encoder Presets and Tune Options
-preset trades CPU for compression efficiency. For live, use veryfast to superfast — anything slower adds latency you can’t recover. The -tune zerolatency flag is non-negotiable: it disables lookahead, frame reordering, and B-frame buffering.
GOP and Buffer Settings
The GOP size (-g) determines how often keyframes appear. Smaller GOPs mean faster channel-join and lower latency but higher bitrate. For low-latency live, -g 30 (≈1 second at 30 fps) is a good balance. Always pair with -bf 0 (no B-frames) and consider -x264-params "rc-lookahead=0:sync-lookahead=0".
Hardware Acceleration
Software x264 is CPU-hungry. Hardware encoders dramatically reduce latency and CPU load:
| Encoder | Flag | Latency | Quality vs x264 |
|---|---|---|---|
| NVENC | -c:v h264_nvenc | Lowest | ~90% |
| Quick Sync | -c:v h264_qsv | Very low | ~85% |
| VAAPI | -c:v h264_vaapi | Low | ~85% |
| x264 | -c:v libx264 | Moderate | Baseline |
Example NVENC line:
-c:v h264_nvenc -preset p1 -tune ll -rc cbr -b:v 4500k -g 30 -bf 0
-tune ll (low-latency) and -preset p1 are the NVENC equivalents of zerolatency and ultrafast.
Benchmarking and Performance Tuning
Measuring End-to-End Latency
The gold standard is a glass-to-glass test: point a camera at a millisecond timer, stream it through your pipeline, and photograph the playback screen alongside the source. The delta is your true latency. For component-level measurement, timestamp frames at each hop and log deltas.
sequenceDiagram
participant Cam as Camera
participant SRT as SRT Listener
participant FF as FFmpeg
participant CDN as Origin/CDN
participant View as Viewer
Cam->>SRT: t0 capture
SRT->>FF: t1 relay (+~150ms)
FF->>CDN: t2 transcode (+~400ms)
CDN->>View: t3 package (+~2000ms for LL-HLS)
Note over Cam,View: Glass-to-glass = t3 - t0
CPU and GPU Utilization
Monitor with nvidia-smi dmon (GPU) and pidstat -p <ffmpeg_pid> 1 (CPU). A healthy 1080p NVENC transcode uses 5–15% of a modern GPU. If CPU hits 100%, you’re dropping frames — check ffmpeg stderr for frame= ... fps= ... speed=; anything below 1.0x means you can’t keep up in real time.
Network Considerations
SRT’s maxbw should be set to ~1.3× your stream bitrate to leave headroom for retransmits. Monitor RTT and packet loss with srt-live-transmit’s stats output. On congested links, SRT’s congestion control (default live mode) will back off gracefully — but if loss exceeds ~5%, increase latency.
Production Edge Cases and Troubleshooting
Handling Packet Loss and Jitter
If you see freezing or macroblocking, your SRT latency is too low for the RTT. Bump latency by 50 ms increments and re-test. Persistent loss on a specific link suggests you should enable SRT bonding (see below) or switch to a redundant path.
Scaling with Multiple Streams
A single FFmpeg process per stream is the simplest model but wastes resources. For 10+ streams, run one FFmpeg per input but share GPU encoders, and use a process supervisor (systemd, supervisord) with restart policies. Consider segmenting transcoding across machines: SRT listeners on edge nodes, transcoders on GPU workers, packagers on origin.
Monitoring and Logging
FFmpeg’s stderr is your friend — pipe it to a log aggregator and alert on speed= dropping below 1.0. For SRT, expose stats via the SRT API and scrape them into Prometheus. Key metrics: pktRcvLoss, pktRetrans, msRTT, mbpsBandwidth.
flowchart TD
A[Stream Degraded?] --> B{Speed < 1.0x?}
B -->|Yes| C[CPU/GPU bottleneck<br/>switch to HW encoder]
B -->|No| D{Packet loss > 2%?}
D -->|Yes| E[Increase SRT latency<br/>or enable bonding]
D -->|No| F{Latency too high?}
F -->|Yes| G[Reduce GOP, check packager]
F -->|No| H[Investigate client-side]
Advanced Topics: Redundancy, Security, and CDN Integration
SRT Redundancy and Bonding
SRT supports bonding multiple network paths (e.g., fiber + LTE) via the group feature. If one path degrades, packets flow over the other transparently. Configure with srt-live-transmit group mode or use the SRT_GROUP API. This is the single most effective resilience upgrade for remote production.
Securing SRT with AES Encryption
Always set a passphrase on both ends. SRT negotiates AES-128/192/256 based on pbkeylen. For public internet contribution, AES-256 (pbkeylen=32) is the safe default. Note that SRT encryption protects transport but not the payload at rest — pair with TLS for the distribution leg.
Integrating with a CDN for Scale
Your origin (nginx-rtmp, MediaMTX, or a custom packager) serves LL-HLS segments to a CDN. Configure short segment durations (1–2 s) and partial segments for LL-HLS. Point your CDN at the origin, set appropriate cache TTLs, and use origin shielding to protect against thundering herds during popular events.
Conclusion and Best Practices
Building a self-hosted low-latency platform with SRT and FFmpeg is a solved problem — but only if you respect the details. The checklist:
- Set SRT latency to 2–3× RTT — never guess.
- Always use
-tune zerolatency, small GOPs, and no B-frames. - Prefer hardware encoders for both latency and CPU headroom.
- Choose your distribution protocol deliberately — it dominates end-to-end latency.
- Monitor everything: FFmpeg
speed=, SRT loss/retrans/RTT, GPU utilization. - Plan for failure: SRT bonding, process supervision, and redundant origins.
Get these right and you’ll have a pipeline that’s faster, cheaper, and far more observable than any commercial alternative.
FAQ
What is the typical end-to-end latency with SRT and FFmpeg? With proper tuning, sub-2-second latency from contribution to playback is achievable. SRT adds 100–200 ms for error correction; FFmpeg transcoding can add under 500 ms. Distribution dominates: WebRTC reaches sub-500 ms, LL-HLS around 2–3 s, standard HLS 6–10 s.
Can I use SRT without FFmpeg? Yes — SRT works standalone for point-to-point streaming. But FFmpeg is essential for transcoding to adaptive bitrate and packaging for multi-viewer distribution. SRT handles transport; FFmpeg handles processing.
How do I install FFmpeg with SRT support?
Compile with --enable-libsrt after installing the SRT library, or use prebuilt static builds from BtbN or John Van Sickle that include SRT. Verify with ffmpeg -protocols | grep srt.
What are the best SRT latency settings for live streaming?
Set latency to at least 2–3× your network RTT. For typical internet connections, 120–200 ms works well. Adjust via the URL parameter: srt://:1234?latency=200.
How can I reduce FFmpeg transcoding latency?
Use ultrafast/superfast presets, -tune zerolatency, GOP size around 30, -bf 0, hardware acceleration (NVENC/QSV/VAAPI), and -fflags nobuffer. Avoid unnecessary filters.
Is SRT secure for public internet streaming?
SRT supports AES-128/192/256 encryption via passphrase and pbkeylen. For extra defense, run SRT over a VPN or secure tunnel — encryption protects transport, not payload at rest.