Systems Research and Architecture SRA
a SRA Theses Memory Reliability as an Allocatable Resource

Memory Reliability as an Allocatable Resource

Context

For a long time, 3–5-year hardware upgrade cycles were the norm in data centers. However, this might be changing due to several factors: As recent geopolitical developments increase the likelihood of supply chain attacks, the latest and greatest hardware might not always be available. In an extreme scenario Europe's data centers might need to extend the lifetime of existing hardware or rely on third-rate purchases from a restricted set of suppliers. Additionally, from an ecological standpoint, phasing out working hardware for small efficiency gains might not be sensible due to the large amounts of embodied carbon involved. Finally, diminishing incremental improvements make upgrading generally less appealing. Therefore, it might be desirable to use hardware for longer, even in a partially degraded state e.g, due to aging or low quality. For this, strategies for managing aging hardware need to be developed and systems must be re-designed to take advantage of partially degraded resources.

Problem

Operating systems treat memory resources uniformly with respect to reliability and resilience ignoring defects, degradation, and capability differences. While statistical models for memory degradation and techniques for error detection exist, operating systems fail to take full advantage of this information. Linux for example does try to offline bad page frames, but simply adds them to a blocklist and forgets them after reboot. Furthermore, current attempts of online error detection, such as memory scrubbing, rely on hardware ECC support, which might not be available in a resource constrained scenario. Finally, applications might not need perfectly reliable memory for all allocations, but have no way to communicate that fact to the system.

Goal

The goal of this thesis is to turn memory reliability into an allocatable resource and prototype this concept inside the Linux kernel. Most importantly, infrastructure needs to be created to separate chunks of physical memory into reliability classes, with support for promotion and demotion based on live reclassification. To expose this functionality to applications, an interface is needed to allocate memory of different classes. Additionally, mechanisms (e.g., event channels) for communicating changes, such as reclassification or detected poisoning of previously allocated memory, to applications need to be designed. Finally, the system must be evaluated using error injection (e.g., simulating ECC faults while injecting errors manually according to known failure classes). For this, suitable applications that allow for mixed reliability need to be identified and extended (e.g., video encoding, cache memory).

Challenges for this thesis include:

  1. Data structures to separate physical memory into chunks sorted by reliability.
  2. Kernel infrastructure to handle (re)classification of memory independently of the actual classfication mechanism.
  3. Allocation API that allows programs to (optionally) specify reliability requirements when requesting memory.
  4. Event channels to communicate memory errors or reclassification to relevant applications.
  5. Evaluation on real world applications using error injection to simulate known memory failure modes (e.g., single-cell, row, column).
  6. (Optional) methods for (re)classification such as online memory testing (e.g., test patterns, heuristics, intervals).
  7. (Optional) exploring different reliability granularities e.g., on process or application level instead of per allocation.
  8. (Optional) force actual memory failures via forcing DRAM to operate on unstable timings.

Topics: Linux Kernel, DDR, C, Fault Injection

References

Revisiting Memory Errors in Large-Scale Production Data Centers: Analysis and Modeling at Facebook
Jianmin Q. Wu, Andrea C. Arpaci-Dusseau, Remzi H. Arpaci-Dusseau, Shan LuIEEE/IFIP International Conference on Dependable Systems and Networks2015.
[] 
@inproceedings{wu:15:dsn,
author = {Wu, Jianmin Q. and Arpaci-Dusseau, Andrea C. and Arpaci-Dusseau, Remzi H. and Lu, Shan},
booktitle = {IEEE/IFIP International Conference on Dependable Systems and Networks},
series = {DSN},
title = {Revisiting Memory Errors in Large-Scale Production Data Centers: Analysis and Modeling at Facebook},
year = {2015},
}
Revisiting reliability in large-scale machine learning research clusters
Apostolos Kokolis, Michael Kuchnik, John Hoffman, Adithya Kumar, Parth Malani, Faye Ma, Zachary DeVito, Shubho Sengupta, Kalyan Saladi, Carole-Jean WuInternational Symposium on High Performance Computer Architecture2025.
[] 
@inproceedings{kokolis:25:hpca,
author = {Kokolis, Apostolos and Kuchnik, Michael and Hoffman, John and Kumar, Adithya and Malani, Parth and Ma, Faye and DeVito, Zachary and Sengupta, Shubho and Saladi, Kalyan and Wu, Carole-Jean},
booktitle = {International Symposium on High Performance Computer Architecture},
series = {HPCA},
title = {Revisiting reliability in large-scale machine learning research clusters},
year = {2025},
}
DDR5 DRAM Faults in the Field
Majed Valad Beigi, Yi Cao, Grace Tsai, Sudhanva Gurumurthi, Vilas SridharanIEEE/IFIP International Conference on Dependable Systems and Networks-Supplemental Volume2025.
[] 
@inproceedings{beigi:25:dsn,
author = {Beigi, Majed Valad and Cao, Yi and Tsai, Grace and Gurumurthi, Sudhanva and Sridharan, Vilas},
booktitle = {IEEE/IFIP International Conference on Dependable Systems and Networks-Supplemental Volume},
series = {DSN-S},
title = {DDR5 DRAM Faults in the Field},
year = {2025},
}