An ffmpeg relay between every camera and Cloudflare

Per-minute billing

Encoding and recording were billed per minute, and a camera fixed to a court streamed whether or not a session was in progress. A session started by QR code and ran for a set time. The stream had to start and stop with it.

MediaLive

• The client required AWS and suggested Kinesis. I chose MediaLive over Kinesis.
• Each court had one channel. The web request that started a session reconfigured it: stop, wait for idle, push encoder settings, wait again, start, wait for running.
• Each wait was allowed up to three minutes.
• A new input was created at every session start and the old one left in place. Idle inputs accumulated and kept billing.

Mux

• At handover I recommended two alternatives: Mux for compatibility, and Cloudflare Stream for the lowest total cost, with a workaround.
• MediaLive's cost later led the client to ask for Mux, and I made the switch.
• Mux handled ingest and adaptive bitrate and reported finished recordings by webhook.
• A cron cleared sessions whose recording stayed in the processing state.
• Recordings were deleted after a retention period to limit storage cost.

Cloudflare Stream and the relay

• Cloudflare Stream later replaced Mux at the client's request, because it priced delivery and storage lower.
• The workaround was an ffmpeg relay on the platform's own VPS. The handover recommendation had warned of potential high memory use and VPS cost at peak and in normal operation. The client went ahead on the condition that Mux would return if the server could not carry the load.
• Cameras pushed RTMP to an nginx on the platform's server. The backend started an ffmpeg relay to Cloudflare only while a session ran, and switched recording on with the relay and off after it.
• No webhook event covered this flow. Finished recordings were found by polling.
• Recordings moved to R2 storage after four days.

Faults at the relay

• A camera that dropped for a second could not reconnect, because the dead connection held its stream key for about two hours. A 30-second idle timeout in nginx fixed that in November.
• The same dropout killed the relay. A relay started for a camera that was not publishing exited with no error. Reconnect flags and an ffprobe check before launch fixed both the following March.

Relay state in one process

The relay's state was a map in one process's memory. The backend ran as one process.