OS hard-faults every ~21 minutes on 3.1.1, reproducible by running a manual system update

Platform: Device-side bug, not SDK. Firmware 057d750f-3.1.1-release.210171-gitlab-runner. Sideloading from macOS, but the SDK is not involved.

Expected vs actual

Expected: The periodic system update check either succeeds, or fails and the device carries on.

Actual: The check hard-faults the OS. The device shows error e0, reboots itself, and does it again on the next cycle. Sixteen crashes were logged in one evening.

When and where

Device sitting idle at the launcher, wifi connected, no game running. It crashes roughly every 21 minutes indefinitely.

The interval is a timer, not chance. Intervals between the twelve crashes that share one signature, in seconds:

1267  1270  1267  2974  1268  1267  1266  1268  1268  1284  1268

Setting aside the single 2974 s gap, where a different fault interrupted the cycle, that is 1266 s to 1284 s: an 18 second spread across eleven cycles, averaging 21 minutes 7 seconds.

This began around the time of the DDoS against Panic.com on 8/12, which I mention only because it may have changed what the update servers were returning.

There is no user-side mitigation. The settings offer no way to disable wifi, so the check cannot be prevented from running.

Steps to reproduce

  1. Connect the device to wifi.
  2. Trigger a manual system update check.
  3. The OS crashes immediately with error e0.

Or leave the device idle at the launcher on wifi and wait ~21 minutes.

Not caused by a sideloaded app

  • Fifteen of the sixteen crashes report Lua totalbytes=0, so no Lua VM was loaded and no game was running.
  • errorlog.txt has no entries for any of these crashes.
  • I also removed a sideloaded app's 24 frame animated card-highlighted from the device and left it at the launcher with that app selected. It crashed again 24 minutes later on the same cycle.

Fault detail

Twelve of the sixteen are the same crash. Identical faulting instruction, identical return address, identical heap size (1278496), and a faulting address that varies by only six bytes.

--- crash at 2026/08/13 23:36:53---

build:057d750f-3.1.1-release.210171-gitlab-runner

r0:90084bdf r1:733e30b0 r2:20919d2b r3: 80000000

r12:00000002 lr:2409434d pc:240535c8 psr: 81071e00

cfsr:00008200 hfsr:40000000 mmfar:733e30b0 bfar: 733e30b0

heap allocated: 1278496

Lua totalbytes=0 GCdebt=0 GCestimate=0 stacksize=0

Classification: CFSR 0x00008200 is BusFault PRECISERR + BFARVALID, a precise data bus fault. HFSR 0x40000000 is FORCED, so it escalated to HardFault with nothing containing it. IPSR is 0, so it faulted in thread mode rather than an interrupt.

Constant in all twelve: pc = 0x240535c8, lr = 0x2409434d (Thumb bit set, so the call site is 0x2409434c), r3 = 0x80000000.

r1 equals BFAR in all twelve, so it is the pointer being dereferenced. Four values, all even, two bytes apart:

0x733e30aa  0x733e30ac  0x733e30ae  0x733e30b0

r0 takes six values, all odd, in two clusters 122 bytes apart, stepping by two within each:

0x90084b5b  0x90084b5d  0x90084b5f    0x90084bd9  0x90084bdd  0x90084bdf

PSR flags across the four values seen:

psr        N Z C V  T  GE[3:0]  IT

a10f1e00 1 0 1 0 1 1111 0x07

810d1e00 1 0 0 0 1 1101 0x07

81071e00 1 0 0 0 1 0111 0x07

a1051e00 1 0 1 0 1 0101 0x07

The GE flags are set and differ on every crash. Those are written only by the SIMD byte and halfword instructions (UADD8, USUB8, SEL), which compare four bytes at once. The IT field is non-zero, so the faulting instruction sat inside an IF-THEN block.

Byte-parallel comparison inside a conditional, with a byte pointer advancing two at a time, is the shape of an optimised string or buffer scan.

The faulting address is not a valid address. 0x733e30ae as bytes is 73 3e 30 ae: the characters s, >, 0, and one non-printable byte. Across the whole log, every PC and LR is 0x24xxxxxx, RAM pointers are 0x20xxxxxx, and r0 is 0x90xxxxxx. The 0x73xxxxxx range appears only as the faulting pointer, 36 times, and nowhere else.

My reading, offered as interpretation rather than fact: something in the update path scans a text response, walks past the end of its buffer, and dereferences an address assembled from the response's own bytes.

Two other signatures in the same window

3 occurrences, also BusFault escalated to HardFault, at pc = 0x24061a58 with lr = 0x00000000 and heap 1.96–2.02 MB. Faulting addresses 0xb2445a90, 0xbc3b0f0c, 0xbc38891c. The zero link register suggests a corrupted return address.

1 occurrence, MemManage DACCVIOL (CFSR 0x00000082, not escalated) at pc = 0x240acf8c, MMFAR 0x77444146 (wDAF) with r1 = 0x77444141 (wDAA). This is the only one of the sixteen with a live Lua VM, so a game was running for that one alone.

What would help

pc = 0x240535c8 and lr = 0x2409434c should resolve directly against your build for 057d750f-3.1.1-release.210171 and identify both the scanning routine and its caller.

Separately from whatever the servers were returning: a bad pointer in this path escalates to a HardFault and takes the device down, rather than failing the update and continuing.

The device's full crashlog.txt follows, since attachments here are images only.

--- forced reset at 2023/08/25 00:55:03---

build:f5a85fc69f11-2.0.1-release.156292-buildbot

r0:00006702 r1:30006170 r2:00527bbe r3: 00000000

r12:a5a5a5a5 lr:2403142d pc:2403142e psr: 01000000

rcccsr:00000000

heap allocated: 137775782

Lua totalbytes=-339771589 GCdebt=1601741642 GCestimate=-1917407628 stacksize=563585758

--- crash at 2026/02/12 21:58:20---

build:83737252-3.0.3-release.198853-gitlab-runner

r0:00008001 r1:20002800 r2:38800088 r3: 00000000

r12:00000090 lr:24021437 pc:24021436 psr: 41000000

cfsr:00010000 hfsr:40000000 mmfar:00000000 bfar: 54472cfb

rcccsr:00000000

heap allocated: 674646150

Lua totalbytes=-339771589 GCdebt=1601708874 GCestimate=-1917407628 stacksize=563520202

--- crash at 2026/08/04 14:30:02---

build:057d750f-3.1.1-release.210171-gitlab-runner

r0:20010fe8 r1:2403ee5f r2:300002cc r3: 20010fe8

r12:ffffffff lr:2403ee5f pc:2406233a psr: 410f0000

cfsr:00000082 hfsr:00000000 mmfar:40021fdc bfar: 40021fdc

rcccsr:00000000

heap allocated: 2844160

Lua totalbytes=0 GCdebt=0 GCestimate=0 stacksize=0

--- forced reset at 2026/08/11 20:58:52---

build:057d750f-3.1.1-release.210171-gitlab-runner

r0:00000003 r1:ffffffff r2:e000ed00 r3: 000065fc

r12:00000000 lr:24006545 pc:24007d40 psr: 2100000f

rcccsr:00000000

heap allocated: 2844160

Lua totalbytes=0 GCdebt=0 GCestimate=0 stacksize=0

--- crash at 2026/08/13 20:21:52---

build:057d750f-3.1.1-release.210171-gitlab-runner

r0:20010ef8 r1:20010f1c r2:00000010 r3: 900a3818

r12:901d5d38 lr:00000000 pc:24061a58 psr: 810d0200

cfsr:00008200 hfsr:40000000 mmfar:b2445a90 bfar: b2445a90

rcccsr:00000000

heap allocated: 1965440

Lua totalbytes=0 GCdebt=0 GCestimate=0 stacksize=0

--- crash at 2026/08/13 21:01:39---

build:057d750f-3.1.1-release.210171-gitlab-runner

r0:20010ef8 r1:20010f1c r2:00000010 r3: 9009d4d8

r12:901df298 lr:00000000 pc:24061a58 psr: 81050200

cfsr:00008200 hfsr:40000000 mmfar:bc3b0f0c bfar: bc3b0f0c

rcccsr:00000000

heap allocated: 1962688

Lua totalbytes=0 GCdebt=0 GCestimate=0 stacksize=0

--- crash at 2026/08/13 21:22:47---

build:057d750f-3.1.1-release.210171-gitlab-runner

r0:90084b5d r1:733e30ae r2:20919d2b r3: 80000000

r12:240df730 lr:2409434d pc:240535c8 psr: a10f1e00

cfsr:00008200 hfsr:40000000 mmfar:733e30ae bfar: 733e30ae

rcccsr:00000000

heap allocated: 1278496

Lua totalbytes=0 GCdebt=0 GCestimate=0 stacksize=0

--- crash at 2026/08/13 21:43:54---

build:057d750f-3.1.1-release.210171-gitlab-runner

r0:90084b5b r1:733e30ac r2:20919d2f r3: 80000000

r12:00000002 lr:2409434d pc:240535c8 psr: 810d1e00

cfsr:00008200 hfsr:40000000 mmfar:733e30ac bfar: 733e30ac

rcccsr:00000000

heap allocated: 1278496

Lua totalbytes=0 GCdebt=0 GCestimate=0 stacksize=0

--- crash at 2026/08/13 22:05:04---

build:057d750f-3.1.1-release.210171-gitlab-runner

r0:90084b5f r1:733e30b0 r2:20919d2b r3: 80000000

r12:00000002 lr:2409434d pc:240535c8 psr: 81071e00

cfsr:00008200 hfsr:40000000 mmfar:733e30b0 bfar: 733e30b0

rcccsr:00000000

heap allocated: 1278496

Lua totalbytes=0 GCdebt=0 GCestimate=0 stacksize=0

--- crash at 2026/08/13 22:26:11---

build:057d750f-3.1.1-release.210171-gitlab-runner

r0:90084bd9 r1:733e30aa r2:20919d2f r3: 80000000

r12:240df730 lr:2409434d pc:240535c8 psr: a1051e00

cfsr:00008200 hfsr:40000000 mmfar:733e30aa bfar: 733e30aa

rcccsr:00000000

heap allocated: 1278496

Lua totalbytes=0 GCdebt=0 GCestimate=0 stacksize=0

--- crash at 2026/08/13 22:54:38---

build:057d750f-3.1.1-release.210171-gitlab-runner

r0:20010ef8 r1:20010f1c r2:00000010 r3: 90074ee8

r12:901ed4c8 lr:00000000 pc:24061a58 psr: 810f0200

cfsr:00008200 hfsr:40000000 mmfar:bc38891c bfar: bc38891c

rcccsr:00000000

heap allocated: 2020576

Lua totalbytes=0 GCdebt=0 GCestimate=0 stacksize=0

--- crash at 2026/08/13 23:15:45---

build:057d750f-3.1.1-release.210171-gitlab-runner

r0:90084bdf r1:733e30b0 r2:20919d2b r3: 80000000

r12:00000002 lr:2409434d pc:240535c8 psr: 81071e00

cfsr:00008200 hfsr:40000000 mmfar:733e30b0 bfar: 733e30b0

rcccsr:00000000

heap allocated: 1278496

Lua totalbytes=0 GCdebt=0 GCestimate=0 stacksize=0

--- crash at 2026/08/13 23:36:53---

build:057d750f-3.1.1-release.210171-gitlab-runner

r0:90084bdf r1:733e30b0 r2:20919d2b r3: 80000000

r12:00000002 lr:2409434d pc:240535c8 psr: 81071e00

cfsr:00008200 hfsr:40000000 mmfar:733e30b0 bfar: 733e30b0

rcccsr:00000000

heap allocated: 1278496

Lua totalbytes=0 GCdebt=0 GCestimate=0 stacksize=0

--- crash at 2026/08/13 23:58:00---

build:057d750f-3.1.1-release.210171-gitlab-runner

r0:90084bdd r1:733e30ae r2:20919d2b r3: 80000000

r12:240df730 lr:2409434d pc:240535c8 psr: a10f1e00

cfsr:00008200 hfsr:40000000 mmfar:733e30ae bfar: 733e30ae

rcccsr:00000000

heap allocated: 1278496

Lua totalbytes=0 GCdebt=0 GCestimate=0 stacksize=0

--- crash at 2026/08/14 00:19:06---

build:057d750f-3.1.1-release.210171-gitlab-runner

r0:90084bdd r1:733e30ae r2:20919d2b r3: 80000000

r12:240df730 lr:2409434d pc:240535c8 psr: a10f1e00

cfsr:00008200 hfsr:40000000 mmfar:733e30ae bfar: 733e30ae

rcccsr:00000000

heap allocated: 1278496

Lua totalbytes=0 GCdebt=0 GCestimate=0 stacksize=0

--- crash at 2026/08/14 00:40:14---

build:057d750f-3.1.1-release.210171-gitlab-runner

r0:90084bd9 r1:733e30aa r2:20919d2f r3: 80000000

r12:240df730 lr:2409434d pc:240535c8 psr: a1051e00

cfsr:00008200 hfsr:40000000 mmfar:733e30aa bfar: 733e30aa

rcccsr:00000000

heap allocated: 1278496

Lua totalbytes=0 GCdebt=0 GCestimate=0 stacksize=0

--- crash at 2026/08/14 01:01:22---

build:057d750f-3.1.1-release.210171-gitlab-runner

r0:90084bdd r1:733e30ae r2:20919d2b r3: 80000000

r12:240df730 lr:2409434d pc:240535c8 psr: a10f1e00

cfsr:00008200 hfsr:40000000 mmfar:733e30ae bfar: 733e30ae

rcccsr:00000000

heap allocated: 1278496

Lua totalbytes=0 GCdebt=0 GCestimate=0 stacksize=0

--- crash at 2026/08/14 01:22:46---

build:057d750f-3.1.1-release.210171-gitlab-runner

r0:90084b5d r1:733e30ae r2:20919d2b r3: 80000000

r12:240df730 lr:2409434d pc:240535c8 psr: a10f1e00

cfsr:00008200 hfsr:40000000 mmfar:733e30ae bfar: 733e30ae

rcccsr:00000000

heap allocated: 1278496

Lua totalbytes=0 GCdebt=0 GCestimate=0 stacksize=0

--- crash at 2026/08/14 01:43:54---

build:057d750f-3.1.1-release.210171-gitlab-runner

r0:90084b5b r1:733e30ac r2:20919d2f r3: 80000000

r12:00000002 lr:2409434d pc:240535c8 psr: 810d1e00

cfsr:00008200 hfsr:40000000 mmfar:733e30ac bfar: 733e30ac

rcccsr:00000000

heap allocated: 1278496

Lua totalbytes=0 GCdebt=0 GCestimate=0 stacksize=0

--- crash at 2026/08/14 01:56:33---

build:057d750f-3.1.1-release.210171-gitlab-runner

r0:00000000 r1:77444141 r2:80000000 r3: a0000000

r12:20009b80 lr:240acf3d pc:240acf8c psr: 810f0000

cfsr:00000082 hfsr:00000000 mmfar:77444146 bfar: 77444146

rcccsr:00000000

heap allocated: 1273472

Lua totalbytes=639794 GCdebt=-2000 GCestimate=633086 stacksize=42

Two other signatures in the same window

3 occurrences, also BusFault escalated to HardFault, at pc = 0x24061a58 with lr = 0x00000000 and heap 1.96–2.02 MB. Faulting addresses 0xb2445a90, 0xbc3b0f0c, 0xbc38891c. The zero link register suggests a corrupted return address.

1 occurrence, MemManage DACCVIOL (CFSR 0x00000082, not escalated) at pc = 0x240acf8c, MMFAR 0x77444146 (wDAF) with r1 = 0x77444141 (wDAA). This is the only one of the sixteen with a live Lua VM, so a game was running for that one alone.

What would help

pc = 0x240535c8 and lr = 0x2409434c should resolve directly against your build for 057d750f-3.1.1-release.210171 and identify both the scanning routine and its caller.

Separately from whatever the servers were returning: a bad pointer in this path escalates to a HardFault and takes the device down, rather than failing the update and continuing.

We have a fix in hand for this in the next OS release, but it should be resolved server-side now.