The forgetful CPU (Linux on M4)

It turned out to be more difficult, since the M4 machines are the first generation of Apple Silicon to mandate SPTM (Secure Page Table Monitor), which provides hardening against vulnerabilities in the XNU kernel of macOS. In previous generations, the Linux bringup was largely based on MMIO traces captured using the m1n1 hypervisor, allowing the analysis of interactions between the original macOS drivers and the hardware. With SPTM, major changes to m1n1 are required to get macOS running under the hypervisor, and these changes certainly go beyond what I could come up with as a newbie in this space.
Still, it wasn’t entirely a lost cause. In parallel to the hypervisor work, I started my first attempts to boot Linux on the M4. This meant disabling strict boot security, installing m1n1 as a custom boot object via macOS recovery¹, and obtaining a serial console² to inspect the logs.

Locked registers

Initially, m1n1 was only able to start in BRINGUP mode and immediately crashed while attempting to initialize GXF³ otherwise. GXF functionality turned out to be disabled/locked in raw boot mode on these M4+ SoCs, so making its initialization conditional and skipping it on these machines was the correct thing to do. Besides the disabled GXF feature, there was also the RVBAR (Reset Vector Base Address Register): a place in memory (one per CPU core) that determines where the core starts executing when powered on. The m1n1 code writes its entrypoint address to the RVBAR for each core when booting the kernel or chainloading another m1n1. Writing to it also led to a crash on M4, but long story short, it already contained the correct value so this write needed to be skipped as well.

2024-12-01 17:37 <yuka> can confirm this state gives a working usb proxy, as in a "Generic m1n1 uartproxy v1.4.17-61-ga24ff77" turns up in lsusb, and I can open a shell :) 

At this point, a long time went by without me touching the Mac mini. At the Chaos Communication Congress at the end of 2025, I found some new motivation. Finally, I hacked together a very minimal device tree containing only the CPU cores and AIC interrupt controller and loaded the Linux kernel (using m1n1’s linux.py) with the earlycon parameter, but I did not see any output after “Vectoring to next stage”. Since I wasn’t getting any useful output from the kernel, I could only take wild guesses at what was going wrong, right? I decided to try the brute-force method: good old println-debugging. I took the debug_putc assembly routine from m1n1 and adjusted it to print a single ‘a’ character ⁴. I inserted this into the Linux kernel code very early in boot, and sure enough, I got an ‘a’ after “Vectoring to next stage”! Essentially, I bisected the Linux boot code and landed at the MMU init code (which is still very early, still in the assembly code in arch/arm64/kernel/head.S). Was the MMU initialization crashing the CPU somehow? Not quite: the UART is accessed using memory-mapped I/O. Once the MMU is enabled, all memory accesses are directed to virtual addresses, which are mapped to a corresponding physical addresses through the page-tables. While m1n1 creates mappings to expose the MMIO address space at identical virtual addresses, Linux does not do this, meaning we end up accessing unmapped space instead of the UART once the MMU is enabled. I modified the initial pagetables to add this 1:1 mapping for the MMIO space, and now my debug_putc worked much further into the boot process. Bisecting again, the print now worked up to somewhere in the interrupt controller initialization! I narrowed it down to a write to the implementation-specific CPU register SYS_IMP_APL_VM_TMR_FIQ_ENA_EL2, which triggered the new crash. After I commented out this write, the kernel booted to a shell. Let’s goooo! SYS_IMP_APL_VM_TMR_FIQ_ENA_EL2, which is related to virtualization, has since been unlocked in new iBoot versions, so commenting out the write is no longer necessary.

Now that I knew that Linux code was actually being executed, I looked once again into why I hadn’t gotten any printk output on the serial console earlier despite the earlycon boot parameter.

2026-01-23 12:35 <yuka> added earlycon=s5l,0x3ad200000 to my bootargs
2026-01-23 12:36 <yuka> and now I'm actually getting useful output before the crash is happening! 

Sure enough, the device tree was just missing stdout-path = "serial0", and after adding it, I got full register dumps and stack traces from Linux on early crashes.

For now, m1n1 didn’t start the secondary cores because smp_start_offset was missing (an offset hardcoded in m1n1; without it, smp_init is skipped). I tried the offset used for base M1 - M3 and was able to start the secondary cores. Once I attempted loading Linux again, I arrived at another mysterious crash.

Previous Apple Silicon CPUs already had known quirks regarding the WFI instruction. Depending on the state of the chicken bit (a bit in a register, that allows the vendor to disable some CPU optimization or feature, or to chicken out) called ARM64_REG_CYC_OVRD_ok2pwrdn_force_mask in the XNU OSS code, the WFI instruction causes the CPU registers x0-x31 to be zeroed on these previous generations. XNU saves these registers to the stack before the WFI and restores them afterwards. In M1-M3 SoCs, m1n1 disables this behavior so the CPU behaves like other arm64 processors. The Asahi kernel later specifically re-enables the behavior to allow the CPU cores to reach deeper sleep states and save power. This is also required to allow one core in the cluster to boost to higher clock speeds when all other cores are in this deeper WFI sleep state.

It seems this chicken bit is either locked or has been removed on M4, and the default behavior does not comply with the ARM64 specification (specifically: “If the system is configured such that the WFI instruction can be completed, then the WFI instruction must not cause a loss of architectural state.”⁵). In April 2026, I managed to boot Linux on the M4 with all cores enabled by replacing all WFI and WFIT (Wait For Interrupt with Timeout) instructions in my kernel with NOP (no-op).