# Adaptive transcoding

Adaptive transcoding is a **Fregata-exclusive** feature — it does not exist in upstream
Frigate. It serves recordings and live camera streams as an HTTP Live Streaming (HLS)
adaptive bitrate ladder: the player automatically selects the highest-quality rung the
viewer's connection can sustain without buffering.

**The primary use case is remote and mobile viewing.** A 4K or even a typical 1080p
high-bitrate recording stream is completely unwatchable on a weak LTE or 5G signal — the
stream simply won't load at all, or buffers every few seconds. With adaptive transcoding,
Fregata automatically drops to a lower quality, which loads cleanly even on
one bar of cellular, and steps back up to full quality once the connection improves.

All transcoding runs entirely on the Apple Silicon **dedicated Media Engine** via
VideoToolbox — **zero CPU**. Everything is off by default and must be enabled in your config file.

:::note[Adaptive Transcoding is conservative by design]
By default, adaptive transcoding will start new streams at the lowest configured quality
and move up from there, if bandwidth allows. You can override this behavior with `starting_rung_height`.

Adaptive transcoding is built for fast loading and smooth transitions between qualities while on
weak mobile, or other low bandwidth, connections. Our goal is fast starts and minimal stuttering or 
buffering. This means you may notice a lower quality than you'd expect as we make extra sure you
have plenty of bandwidth before raising the quality.
:::

## Quick Start

Add the following to config.yml and restart Fregata

```yaml
adaptive_transcoding:
  enabled: true
  live: true    # also enable live stream transcoding
  lan_networks:    # adaptive transcoding (recorded clips) will be disabled on clients with IPs in any of these local LAN ranges
    - "127.0.0.0/8"
    - "10.0.0.0/8"
    - "172.16.0.0/12"
    - "192.168.0.0/16"
  rungs:
    - height: 1080
      bitrate: 4000    # bitrate in kbps
    - height: 720
      bitrate: 1500
    - height: 360
      bitrate: 600
   starting_rung_height: 720    # start at 720p, not the lowest rung
```

## How it works for recorded clips

When adaptive transcoding is enabled:

1. **ffprobe** inspects the camera's stream once at startup and caches the result — dimensions,
   codec, bitrate.
2. The **ABR engine** builds an HLS segment cache on the RAM disk for each active rung.
3. The **top rung** is always served as original quality if bandwidth allows — free,
   no encode cycles at all.
4. **Lower rungs** are transcoded on the Media Engine: VideoToolbox hardware decode
   and encode.
5. **By default** streams of recordings always start on the lowest rung. This can be
   overridden by `starting_rung_height`.
7. The **hls.js player** in the web UI reads the HLS master playlist and switches rungs
   in real time as it measures throughput.
8. When adaptive transcoding is active, you will see the currently playing quality as a small
   bubble in the lower left corner of the video stream. If you do not see this, original quality
   is being played. 

![Fregata recordings player showing the quality-rung bubble in the lower-left corner of the video while adaptive transcoding is active.](/screenshots/adaptive-transcoding-recordings.png)

## LAN bypass (recorded clips only)

Clients whose IP falls within `lan_networks` (default: all private, loopback, and
link-local ranges) receive the **original source stream**, bypassing the ABR ladder
entirely. On a local network you have more than enough bandwidth; transcoding would 
not add any benefit.

Set `lan_networks: []` to disable the bypass and force the ABR ladder for all clients.

:::caution[VPN and Tailscale users]
If you connect remotely through **WireGuard** or another VPN that assigns tunnel addresses
in the `10.0.0.0/8`, `172.16.0.0/12`, or `192.168.0.0/16` ranges, those addresses will
match the default `lan_networks` list and receive the passthrough stream — defeating
adaptive transcoding on that connection.

Remove the conflicting subnet from `lan_networks` to fix it:

```yaml
adaptive_transcoding:
  lan_networks:
    - "127.0.0.0/8"
    - "192.168.0.0/16"     # keep your actual LAN subnet
    # "10.0.0.0/8" removed — WireGuard tunnel is using 10.x.x.x
```

**Tailscale** uses the `100.64.0.0/10` CGNAT range by default, which is outside the
default `lan_networks` list — Tailscale users need no changes.
:::

## Live transcoding

Setting `live: true` under `adaptive_transcoding` also enables adaptive rungs for the
**live view**. 

The stream dropdown in the live view lists each configured rung as a selectable option
(e.g. "720p 1500 kbps (Fregata Transcode)"). For live viewing the stream does NOT currently
switch qualities automatically, the selection is manual. This is due to the fact that live
streams are, well, live. There is no buffer to determine if we are draining the buffer faster
than we can replensish it to signal that we need to drop the quality. 

Live transcoding is a **separate opt-in** from recordings ABR (`live: true`): live rungs are
created as go2rtc streams at startup. They consume no resources until played.
Requires the camera to be set up to be restreamed via go2rtc (default).

![Fregata live view showing the stream quality dropdown with adaptive transcoding rungs listed as selectable options.](/screenshots/adaptive-transcoding-live.png)

## Enabling adaptive transcoding

Minimal global AND per-camera config:

```yaml
# All cameras use transcoding for recordings by default
adaptive_transcoding:
  enabled: true
  live: true           # live view rungs (optional, separate opt-in)

cameras:
  indoor_cam:
    adaptive_transcoding:
      enabled: false       # this camera opts out
```

Or set per-camera:

```yaml
cameras:
  front_yard:
    adaptive_transcoding:
      enabled: true        # recordings ABR
      live: true           # live view rungs (optional, separate opt-in)
```

Restart Fregata after changing `config.yml`.

## Configuration reference

### Per-camera fields

These fields can be set at the top level (`adaptive_transcoding:`) to apply a default to
every camera, or per-camera (`cameras.<name>.adaptive_transcoding:`) to override.

| Field | Type | Default | Description |
|---|---|---|---|
| `enabled` | bool | `false` | Enable recordings ABR for this camera. |
| `live` | bool | `false` | Enable live-view rungs. Does not follow `enabled` — must be set explicitly. |
| `starting_rung_height` | int or null | `null` | Start playback on the highest configured rung whose height is ≤ this value. `null` starts at the lowest rung — safest for variable connections. |
| `rungs` | list | See below | The ABR ladder, highest quality first. Each entry: `height` (px), `bitrate` (kbps), optional `name`. The engine will never upscale, if you have a 1080p rung listed by have a single camera that maxes out at 720p, the 1080p rung will be disabled for that camera. |

**Default rung ladder:**

| Name | Height | Bitrate |
|---|---|---|
| high | 1080p | 4000 kbps |
| medium | 720p | 1500 kbps |
| low | 360p | 600 kbps |

Original quality is always served if bandwidth allows with no transcoding at all.

### Global-only resource fields

These fields are honored **only at the top-level `adaptive_transcoding:` block**, not
per-camera.

| Field | Type | Default | Description |
|---|---|---|---|
| `lan_networks` | list of CIDRs | Private/loopback/link-local | Clients in these ranges receive the original source stream. See [LAN bypass](#lan-bypass) above. |

### Full annotated example

```yaml
adaptive_transcoding:
  enabled: true
  live: true    # enable live stream transcoding
  lan_networks:    # adaptive transcoding (recorded clips) will be disabled all connections from IPs in any of these local LAN ranges
    - "127.0.0.0/8"
    - "10.0.0.0/8"
    - "172.16.0.0/12"
    - "192.168.0.0/16"
  rungs:
    - height: 1080
      bitrate: 4000
      name: high
    - height: 720
      bitrate: 1500
      name: medium
    - height: 360
      bitrate: 600
      name: low
   starting_rung_height: 720    # start at 720p, not the lowest rung

cameras:
  front_yard:
    adaptive_transcoding:
      enabled: true
      live: true
      starting_rung_height: 720    # start at 720p, not the lowest rung
  indoor_cam:
    adaptive_transcoding:
      enabled: true                # recording transcode only; live stays off
```

## Hardware requirements

Adaptive transcoding runs exclusively on the **Apple Silicon Media Engine** via
VideoToolbox. It is macOS-only and is not available in upstream Frigate-on-Docker. Any
M-series chip (M1 or later) supports it; no special hardware beyond Fregata's system
requirements is needed.


## Debugging

Set `FREGATA_ABR_DEBUG=1` (Tray → Settings → Environment Variables, then restart) to
enable verbose ABR diagnostics in the Frigate log. The player logs rung switches,
bandwidth estimates, and segment timing.

For recordings, when adaptive transcoding is active, you will see the current quality in a small
bubble in the bottom left corner of the video stream (see screenshots above). If you do not see 
this bubble the original quality video is being played. 

Live stream transcode streams must be selected manually from the stream dropdown for that camera
but they also show a bubble in the bottom left of the UI when a non-original
quality stream is being played.

To further verify adaptive transcoding is active, open the recordings view for an enabled camera
and watch the browser DevTools **Network** tab. You should see requests to:

```
/api/<camera>/start/<s>/end/<e>/adaptive/master.m3u8
/api/<camera>/start/<s>/end/<e>/adaptive/rung/<height>.m3u8
/api/<camera>/start/<s>/end/<e>/adaptive/rung/<height>/<seq>.ts
```

If you see `/vod/` requests instead, adaptive transcoding is not active for that camera —
check that `enabled: true` is set and that Fregata was restarted after the config change.