Pi 5 Trixie: firmware mailbox property requests all return status 0x00000000 (vcgencmd broken, no cpufreq)

[dankorotin]

Description

On Raspberry Pi 5 running Pi OS Trixie (Debian 13), the firmware mailbox property interface does not process any requests. All requests return status 0x00000000 ("request not processed") instead of 0x80000000 ("success"). This breaks:

• vcgencmd — all commands fail with ioctl_set_msg failed:-1
• raspberrypi-clk — probe fails with error -22 at boot, disabling CPU frequency scaling
• vc4-drm — "Couldn't stop firmware display driver: -22"

System Info

Pi: Raspberry Pi 5 Model B Rev 1.1 (16GB)
OS: Debian GNU/Linux 13 (trixie)
Kernel: 6.12.62+rpt-rpi-2712 #1 SMP PREEMPT Debian 1:6.12.62-1+rpt1
EEPROM: Wed  5 Nov 17:37:18 UTC 2025 (1762364238) — up to date
raspi-firmware: 1:1.20250915-1
raspi-utils-core: 20251120-1

Boot log evidence

All firmware property requests return status 0x00000000:

raspberrypi-firmware soc@107c000000:firmware: Request 0x00000001 returned status 0x00000000
raspberrypi-firmware soc@107c000000:firmware: Request 0x00000003 returned status 0x00000000
raspberrypi-firmware soc@107c000000:firmware: Request 0x00030046 returned status 0x00000000
raspberrypi-firmware soc@107c000000:firmware: Request 0x00010007 returned status 0x00000000

This causes the clock driver probe failure:

raspberrypi-clk soc@107c000000:firmware:clocks: probe with driver raspberrypi-clk failed with error -22
vc4-drm axi:gpu: [drm] Couldn't stop firmware display driver: -22

strace of vcgencmd

openat(AT_FDCWD, "/dev/vcio", O_RDONLY) = 3
ioctl(3, _IOC(_IOC_READ|_IOC_WRITE, 0x64, 0, 0x8), 0x7fffe7ae1770) = -1 EINVAL (Invalid argument)

Even sudo vcgencmd measure_temp and a custom C program opening /dev/vcio with O_RDWR produce the same EINVAL — the error comes from rpi_firmware_property_list() in the kernel because buf[1] != RPI_FIRMWARE_STATUS_SUCCESS.

What was ruled out

• Permissions: /dev/vcio is crw-rw---- root:video, user is in video group, sudo also fails
• EEPROM: Updated to latest (Nov 2025)
• Kernel: Upgraded from 6.12.47 to 6.12.62, no change
• Package conflict: raspi-utils-core 20251120-1 only, no libraspberrypi-bin
• ioctl encoding: Confirmed 0xC0086400 matches _IOWR(100, 0, char*) on aarch64

config.txt

Default config with one addition — Hailo AI HAT+ overlay:

dtoverlay=vc4-kms-v3d
max_framebuffers=2
disable_fw_kms_setup=1
arm_64bit=1
arm_boost=1

[all]
dtoverlay=hailo-ai,cma_size=256

Symptoms

• No /sys/devices/system/cpu/cpufreq/ directory (cpufreq governor not loaded)
• No get_throttled in sysfs
• Continuous firmware request spam in dmesg: Request 0x00030087 returned status 0x00000000 and Request 0x00038087 returned status 0x00000000 (~2 per ms)
• Temperature reporting works via sysfs (/sys/class/thermal/thermal_zone0/temp), hwmon, and Hailo Python API — only the firmware mailbox path is broken

Expected behavior

vcgencmd measure_temp should return the SoC temperature. raspberrypi-clk should probe successfully, enabling CPU frequency scaling.

[pelwell]

This is not a common problem - I've not heard of it before - so there must be something different about your setup. What is in your config.txt?

[dankorotin]

Here's my /boot/firmware/config.txt:

# For more options and information see
# http://rptl.io/configtxt
# Some settings may impact device functionality. See link above for details

# Uncomment some or all of these to enable the optional hardware interfaces
#dtparam=i2c_arm=on
#dtparam=i2s=on
#dtparam=spi=on

# Enable audio (loads snd_bcm2835)
dtparam=audio=on

# Additional overlays and parameters are documented
# /boot/firmware/overlays/README

# Automatically load overlays for detected cameras
camera_auto_detect=1

# Automatically load overlays for detected DSI displays
display_auto_detect=1

# Automatically load initramfs files, if found
auto_initramfs=1

# Enable DRM VC4 V3D driver
dtoverlay=vc4-kms-v3d
max_framebuffers=2

# Don't have the firmware create an initial video= setting in cmdline.txt.
# Use the kernel's default instead.
disable_fw_kms_setup=1

# Run in 64-bit mode
arm_64bit=1

# Disable compensation for displays with overscan
disable_overscan=1

# Run as fast as firmware / board allows
arm_boost=1

[cm4]
# Enable host mode on the 2711 built-in XHCI USB controller.
# This line should be removed if the legacy DWC2 controller is required
# (e.g. for USB device mode) or if USB support is not required.
otg_mode=1

[cm5]
dtoverlay=dwc2,dr_mode=host

[all]
dtoverlay=hailo-ai,cma_size=256

This is the stock Trixie config.txt — I haven't modified it beyond adding the dtoverlay=hailo-ai,cma_size=256 line for the Hailo AI HAT+. Could disable_fw_kms_setup=1 be related to the firmware not processing mailbox property requests?

[dankorotin]

One more detail that may be relevant: I'm running non-standard Hailo packages from the Hailo Developer Zone, not the ones in the Pi repo:

ii  hailort                5.2.0    arm64    (Pi repo has 4.23.0)
ii  hailort-pcie-driver    5.2.0    all      (Pi repo has 4.23.0)

The hailort-pcie-driver 5.2.0 doesn't use DKMS, so after the kernel upgrade to 6.12.62 I had to manually rebuild the hailo1x_pci module from /usr/src/hailort-pcie-driver/linux/pcie/. The Hailo device itself works fine (/dev/hailo0 present, inference runs correctly).

The dtoverlay=hailo-ai,cma_size=256 line in config.txt was added for the Hailo AI HAT+. Could either the overlay or the 5.2.0 driver be interfering with the firmware mailbox?

Happy to test with the overlay removed or with the stock 4.23.0 packages from the Pi repo if that would help narrow it down.

[pelwell]

I'm wondering if the CMA carve-out is causing problems, but start by just commenting out the whole hailo-ai dtoverlay line and rebooting. You could also see if the output of sudo vclog -m contains any interesting information.

[nullr0ute]

I'm seeing the same on Fedora with a pure upstream kernel.

Setting cma=0Mb on the command line makes the errors go away but the GPU needs CMA for things like gnome.

[dankorotin]

Can confirm CMA is the trigger — @nullr0ute's finding matches our setup. We haven't tested with the overlay removed, but we can't ship without it: the Raspberry Pi AI HAT+ 2 (Hailo-10H) requires cma=256M for its Speech2Text and larger model inference APIs. Without it, HEF loading fails with insufficient contiguous memory.

So this is a conflict between two official Pi products — the AI HAT+ 2 needs a large CMA carve-out, and that carve-out breaks the firmware mailbox. Both are sold on raspberrypi.com for Pi 5.

For our use case (headless robot, no GPU/display needed) we can live with sysfs alternatives for monitoring, but this will bite anyone running the AI HAT+ 2 on a desktop setup where both GPU and vcgencmd/cpufreq are expected to work.

[nullr0ute]

Does the mailbox need a reserved memory area that is possibly being overwritten by the CMA reservation?

[naushir]

Right now, we have only validated and packaged the Hailo v5.1.1 packages. We are in the process of getting v5.2.0 ready to release.

Having said that, could you point to the instructions used to manually install the v5.2.0 kernel driver? I'm not sure why the use of CMA is required.

[pelwell]

As another datapoint, I'm running an 8GB Pi 5 without a Hailo HAT. However, I've added cma-256 and cma-512 parameters to the vc4-kms-v3d overlay line with no obvious ill effects.

The firmware mailbox interface does not require CMA. However, it does need to be able to allocate very small amounts (normally just a single page at a time) of DMA-addressable RAM in the first 1GB, and we arrange this by creating a CMA region in that range - the default size is 64MB.

[pelwell]

What do the commands dmesg | grep firmware and sudo vclog -m report?

[nullr0ute]

kernel output with 64Mb CMA:

[    0.580468] raspberrypi-firmware soc@107c000000:firmware: Request 0x00000001 returned status 0x00000000
[    0.581319] raspberrypi-firmware soc@107c000000:firmware: Request 0x00030046 returned status 0x00000000
[    0.931610] raspberrypi-firmware soc@107c000000:firmware: Request 0x00010007 returned status 0x00000000
[    0.931617] raspberrypi-clk soc@107c000000:firmware:clocks: probe with driver raspberrypi-clk failed with error -22
[    0.932611] raspberrypi-firmware soc@107c000000:firmware: Request 0x00030030 returned status 0x00000000
[   17.376578] platform 1002000000.gpu: deferred probe pending: platform: supplier soc@107c000000:firmware:clocks not ready

vclog

005099.252: *** Restart logging
005113.889: Initial voltage 800000 temp 73001
005314.465: avs_2712: AVS pred 8898 889800 temp 72452
005318.067: vpred 889 mV +0
005326.713: FB framebuffer_swap 1
005346.227: Select resolution HDMI0/2 hotplug 0 max_mode 2
005350.279: Select resolution HDMI1/2 hotplug 0 max_mode 2
005357.991: dtb_file 'bcm2712-rpi-5-b.dtb'
005485.002: dtparam: audio=on
005493.485: Unknown dtparam 'audio' - ignored
005567.606: Failed to open command line file 'cmdline.txt'
005775.853: RPM 200, max RPM 287
005888.861: Kernel relocated to 0x80000
005893.625: Device tree loaded to 0x2efec600 (size 0x1399a)
005965.740: Starting OS 5965 ms
005971.260: 00000040: -> 00000480
005973.110: 00000030: -> 00100080
005977.822: 00000034: -> 00100080
005982.535: 00000038: -> 00100080
005987.248: 0000003c: -> 00100080

kernel output with 0Mb CMA:

[    0.581260] raspberrypi-firmware soc@107c000000:firmware: Attached to firmware from 2025-08-27T19:01:47

vclog:

005268.917: *** Restart logging
005283.553: Initial voltage 800000 temp 70254
005484.120: avs_2712: AVS pred 8889 888900 temp 71353
005487.721: vpred 888 mV +0
005496.361: FB framebuffer_swap 1
005515.874: Select resolution HDMI0/2 hotplug 0 max_mode 2
005519.927: Select resolution HDMI1/2 hotplug 0 max_mode 2
005527.637: dtb_file 'bcm2712-rpi-5-b.dtb'
005656.591: dtparam: audio=on
005665.076: Unknown dtparam 'audio' - ignored
005739.087: Failed to open command line file 'cmdline.txt'
005947.335: RPM 196, max RPM 287
006060.269: Kernel relocated to 0x80000
006065.031: Device tree loaded to 0x2efec600 (size 0x1399a)
006137.147: Starting OS 6137 ms
006142.667: 00000040: -> 00000480
006144.516: 00000030: -> 00100080
006149.229: 00000034: -> 00100080
006153.941: 00000038: -> 00100080
006158.654: 0000003c: -> 00100080
066266.440: initial_turbo of 60 deactivated

[pelwell]

Interesting - there are no smoking guns in there. Are there any kernel errors or warnings (dmesg -l err,warn)?

[nullr0ute]

Nothing particularly out of the ordinary in the context of this bug. Also RPi3/4/02W all work as expected with the same image.

# dmesg -l err,warn
[    0.000000] OF: reserved mem: invalid size property in 'linux,cma' node.
[    0.872823] brcm-pcie 1000110000.pcie: link down
[    0.880504] pci_bus 0002:01: supply vpcie3v3 not found, using dummy regulator
[    0.880548] pci_bus 0002:01: supply vpcie3v3aux not found, using dummy regulator
[    0.880560] pci_bus 0002:01: supply vpcie12v not found, using dummy regulator
[    3.592131] sdhci-brcmstb 1001100000.mmc: error -EINVAL: invalid resource (null)
[    3.679407] mmc0: cannot verify signal voltage switch
[    3.789281] mmc1: cannot verify signal voltage switch
[    3.857956] vc4_hdmi 107c701400.hdmi: 'dmas' DT property is missing or empty, no HDMI audio
[    3.863872] vc4_hdmi 107c706400.hdmi: 'dmas' DT property is missing or empty, no HDMI audio
[    7.240888] brcmfmac mmc1:0001:1: Direct firmware load for brcm/brcmfmac43455-sdio.raspberrypi,5-model-b.bin failed with error -2
[    7.267407] FAT-fs (mmcblk0p1): Volume was not properly unmounted. Some data may be corrupt. Please run fsck.
[   17.889078] v3d 1002000000.gpu: deferred probe timeout, ignoring dependency
[   17.889095] v3d 1002000000.gpu: probe with driver v3d failed with error -110

[pelwell]

Where is your CMA area being created (dmesg | grep CMA)?

[nullr0ute]

So I've dug into that a bit. I have the following lines in my config.txt:

# Raspberry Pi 5
[pi5]
# Enable DRM VC4 V3D driver with 256Mb CMA
dtoverlay=vc4-kms-v3d-pi5,cma-256
max_framebuffers=2

I also previously had dtoverlay=cma,cma-256 but in both cases I get the following error and end up with the Fedora default of 64Mb:

# dmesg | grep -i CMA
[    0.000000] OF: reserved mem: invalid size property in 'linux,cma' node.
[    0.000000] cma: Reserved 64 MiB at 0x00000000fc000000
[    0.005205] Memory: 7942704K/8384512K available (22848K kernel code, 6030K rwdata, 19484K rodata, 14720K init, 10298K bss, 368452K reserved, 65536K cma-reserved)

The vclog shows these two lines:

005563.706: Loaded overlay 'vc4-kms-v3d-pi5'
005566.533: dtparam: cma-256=true