Jitsi in Incus: Sysctl

Environment Overview

This setup runs with:

• Host: Debian Trixie
• Incus Container (Jitsi): Debian 12 (Bookworm)

These differences matter because sysctl capabilities depend on the host kernel version and Incus' container isolation model.

Introduction

Jitsi, particularly its Videobridge component, relies heavily on high‑performance UDP networking. Understanding how sysctl behaves inside Incus containers is essential for ensuring smooth media transmission and avoiding misleading warnings.
This documentation provides a clean and structured explanation of why certain sysctl warnings appear when running Jitsi inside an Incus (LXC) container, which sysctl keys are allowed, which must be applied on the host instead of the container, and how to correctly tune the system for optimal performance.

1. Why sysctl warnings occur in Incus

Incus containers operate under strict kernel isolation. Many sysctl keys are not namespaced, meaning they apply globally to the host and cannot be changed from inside a container.

Common error messages include:

sysctl: setting key "net.core.rmem_max": Invalid argument
sysctl: permission denied on key "kernel.pid_max"
sysctl: permission denied on key "fs.protected_fifos"
sysctl: permission denied on key "fs.protected_hardlinks"
sysctl: permission denied on key "fs.protected_regular"
sysctl: permission denied on key "fs.protected_symlinks"

1.1 Category explanation

• kernel.* → Host‑level only, cannot be changed in a container.
• fs.protected_* → Global security settings, fixed on the host.
• net.core.rmem_max / wmem_max → Allowed only when within host limits; otherwise rejected.
• Most net.ipv4.* → Allowed, depending on namespacing.

These warnings are expected and harmless. Jitsi will still function.

2. Identifying problematic sysctl entries

Run inside the container:

grep -R "pid_max\|protected_\|rmem_max\|wmem_max" /etc/sysctl.conf /etc/sysctl.d 2>/dev/null

Typical files containing incompatible entries:

• /etc/sysctl.d/10-jvb.conf
• /etc/sysctl.d/20-jitsi-videobridge.conf
• /etc/sysctl.d/99-custom.conf

3. Fixing the configuration inside the container

Commenting out entries rather than deleting them is recommended, as it preserves context for future audits, allows easy re‑enabling if the container setup changes, and prevents confusion when referencing upstream Jitsi tuning guides.
Open the affected file:

vim /etc/sysctl.d/10-jvb.conf

Comment out the host‑only keys:

# kernel.pid_max = 4194303
# fs.protected_fifos = 2
# fs.protected_hardlinks = 1
# fs.protected_regular = 2
# fs.protected_symlinks = 1
# net.core.rmem_max = 10485760
# net.core.wmem_max = 10485760

Apply changes:

sysctl --system

The warnings should now be gone or significantly reduced.

4. Recommended host‑level tuning for Jitsi

These host‑applied values significantly improve Jitsi Videobridge performance by increasing UDP buffer sizes, expanding kernel queuing capacity, reducing packet drops under load, and ensuring sufficient throughput for real‑time media transmission.
Performance‑critical network tuning must be done on the Incus host, not inside the container.

Create the following configuration on the host:

vim /etc/sysctl.d/20-jitsi-host.conf

Recommended values:

net.core.rmem_max = 10485760
net.core.wmem_max = 10485760
net.core.netdev_max_backlog = 100000
net.core.somaxconn = 4096
net.ipv4.ip_forward = 1
net.ipv4.tcp_rmem = 4096 87380 6291456
net.ipv4.tcp_wmem = 4096 16384 4194304

Activate:

sysctl --system

5. Sysctl keys allowed inside containers

Overview Table

Allowed (namespaced)

Allowed (namespaced)

• net.ipv4.conf.*
• net.ipv4.ip_local_port_range
• net.ipv4.tcp_* (partially)
• net.core.default_qdisc
• net.ipv4.tcp_congestion_control

Forbidden (host‑wide)

• kernel.*
• fs.protected_*
• fs.suid_dumpable
• vm.*
• Any global security or memory setting

Limited / rejected depending on host

• net.core.rmem_max
• net.core.wmem_max

6. Clean recommended setup for Jitsi in Incus

Inside the container

• Remove or comment out all keys not allowed in containers.
• Keep only namespaced networking keys.
• Avoid applying Jitsi’s bare‑metal sysctl template.

On the host

• Apply the full set of performance sysctls.
• Increase network buffers and queue sizes.
• Enable forwarding where required.

7. Summary

Running Jitsi inside an Incus container requires a split approach:

• Host → complete sysctl tuning
• Container → only namespaced networking settings

All warnings related to non‑namespaced keys are expected and not critical.

8. Troubleshooting checklist

5. Verify the container's effective network limits via:

incus exec <container> -- cat /proc/sys/net/core/rmem_max
incus exec <container> -- cat /proc/sys/net/core/wmem_max

1. Run sysctl --system inside the container and verify no host‑only keys remain.
2. Confirm the host sysctl file is active.
3. Check Jitsi Videobridge logs for UDP buffer complaints.
4. Confirm current values with:

cat /proc/sys/net/core/rmem_max
cat /proc/sys/net/core/wmem_max

9. Next Steps

If desired, further examples can be added:

• A host‑ready sysctl template
• A minimal Jitsi‑Incus deployment profile
• Additional security and network tuning guidelines