Lesson 59. Flash wear, endurance and write amplification

1. Today's topic

After crash consistency, calculate Flash lifetime. Physical program/erase operations consume endurance, not the size of the variable by itself. The main idea: writing 4 bytes once per second can become hundreds of millions of logical updates over 10 years.

2. Why this matters

Risky candidates:

mqtt_reconnect_count++
modem_reset_count++
uart_error_count++
phase_transition_count++
watchdog_warning_count++

If every increment is followed by nvs_commit(), endurance may be exhausted long before the intended service life.

3. Theory

Endurance budget

Estimate:

N_updates ≈ E * P * U

where:

E = erase cycles per page
P = number of wear-levelling pages
U = logical updates between erase operations on one page

ESP-IDF NVS

NVS stores append-only records. A new value is appended and the old one is invalidated. An NVS page is 4096 bytes, an entry is 32 bytes, and an integer up to 8 bytes occupies one entry. For small values, NVS significantly reduces erase frequency, but endurance is still finite.

The 10-year scale

1 commit/s       -> 315 576 000 commits / 10 years
1 commit/10s     -> 31 557 600
1 commit/min     -> 5 259 600
1 commit/hour    -> 87 660

Durability classes

D0 volatile: may be lost on reset
D1 approximate: the last few minutes may be lost
D2 checkpointed: the last N operations may be lost
D3 immediately durable: ACK only after commit

Counter batching

The runtime value changes often; the Flash checkpoint changes rarely:

commit if dirty >= 100 or age >= 60s

Coalescing

Combine several rapid configuration changes into one snapshot/commit.

Separate partitions

It is better to separate different write rates into different NVS partitions:

nvs_factory read-only
nvs_config rare durable writes
nvs_stats periodic checkpoints

4. Common mistakes

1. Assuming “only 4 bytes” means low wear.
2. Confusing program cycles with erase cycles.
3. Calling nvs_commit() after every counter update.
4. Storing high-frequency statistics beside critical configuration.
5. Using NVS as a binary logger.
6. Using a generic 100k-cycle average instead of the datasheet for the specific Flash.
7. Writing a huge blob snapshot when one field changes.
8. Using a minimal partition without space for GC.
9. Delaying the commit of a D3 operation already acknowledged to the backend.
10. Rewriting one STM32 Flash page as though it were EEPROM.
11. Not counting commits during soak/HIL testing.

5. Practical task

Create a table of persistent objects:

name
updates/day
allowed data loss
D-class
storage backend

Find all writes:

rg "nvs_set_|nvs_commit|esp_partition_write|wl_write"

Add storage diagnostics:

typedef struct {
    uint64_t nvs_set_calls;
    uint64_t nvs_commit_calls;
    uint64_t bytes_requested;
    uint64_t checkpoint_skips;
    uint64_t checkpoint_writes;
} storage_write_stats_t;

Refactor one counter, such as mqtt_reconnect_count: increment in RAM and commit if dirty >= 10 or age >= 60s. Check the acceptable loss after esp_restart().

6. What to try next

• Add nvs_get_stats() to the CLI.
• Compare entry consumption for 1000 uint32_t updates versus 1000 updates of a 1024-byte blob.
• For STM32, calculate the required endurance before choosing a Flash layout.