Analyzing NVMe SSD Characteristics
- Typ der Arbeit: Bachelor-/Masterarbeit
- Status der Arbeit: reserviert
- Projekte: ParPerOS
- Betreuer: Kenny Albes, Daniel Lohmann
[Generated with AI]
Context
In recent years SSDs have evolved into highly optimized hardware capable of delivering bandwidths >14 GB/s. Modern SSD controllers are based on full-fledged ARM cores that perform complex tasks such as advanced bookkeeping, caching, wear leveling and health monitoring. However, their inner workings and performance characteristics are largely hidden from application and driver developers. Aside from unspecific best-case performance metrics in the marketing materials, SSDs only offer scarce performance hints via their NVMe1 interface. This makes it increasingly harder to take full advantage of the growing bandwidth of modern SSDs. Even for a simple sequential read our initial measurements showed large differences (~1.5x) in throughput for varying command submission scenarios.
Problem
Modern SSDs promise very high theoretic throughput. In practice however, it is often unclear what performance can be extracted under which circumstances. There are large performance differences depending on the type of access, the submission queue depth and the internal state of the controller. Controllers may perform maintenance tasks at unspecified times in the background heavily impacting write throughput. Widely used benchmark tools either measure only naive access patterns2 or rely on the file system layer3 making them unfit for accurately capturing the costs of specific micro operations. Existing performance analysis was either done on old SATA drives 4 or focuses primarily on optimizing latencies 5.
Goal
The goal of this thesis is to explore SSD characteristics and their impact on I/O performance. Since the parameter space is virtually limitless, the first step is to identify a set of relevant parameters to investigate. This includes a thorough literature research on prior work as well as commonly used workloads and their I/O behavior. Following that suitable workload patterns must be designed to capture the specific behaviors. The SSD can be directly controlled via our existing low-level NVMe driver. Since writing such benchmarks by hand is tedious and error-prone, fitting abstractions (builders, code generator, executor) are needed. Once the infrastructure is in place, experiments need to be conducted on a range of different SSD models.
A basic set of parameters might include (BA):
- Sequential r/w
- Impact of using seq. hint
- Random r/w
- Queue parameters
- Batch size
- Thread count
- Queue depth
- Performance hints
- Page sizes (will most likely only impact CPU costs)
- Block size: 512B/4KiB
- Impact of block contents (e.g., reading zeroed/uninitialized blocks vs. random content)
- Differences between
write/write_zeroed - Writing deallocated blocks vs. writing allocated (zeroed) ones
- Block deallocation performance (zeroing with deallocate bit)
A more advanced analysis could look at (BA/MA):
- Measuring latencies
- Cached vs. uncached blocks
- Relationship to internal flash page size
- Distribution of accesses across queues
- Cross operation dependencies
- Atomic r/w sequences
- Sustained writes to same block
- Interplay of queue scheduling: different priorities
- Flush behavior
- Capture management tasks during sustained r/w loads
Schedule
- Getting started: Familiarize with the provided NVMe driver, set up a suitable development environment, and establish a functional test setup.
- Identify relevant parameters: Perform literature research and initial measurements.
- Design Experiments: Find access patterns to trigger the relevant behavior, perform initial measurements.
- Design fitting abstractions: Find a convenient way to define workload patterns and setup routines, automate benchmark execution, collect artifacts, …
- Benchmark campaign: Run the benchmarks. Depending on the parameter selection, this may take some time.
- Analyze results: Critically analyze the results and present them appropriately.
While the provided NVMe driver is written in C, the benchmarks and infrastructure may be implemented in a more modern language such as Rust or Python.
Topics: NVMe, Linux, Driver, C/C++/Rust
References
Developing a low-level NVMe driver for microbenchmarks
- Typ
- Bachelorarbeit
- Status
- abgeschlossen
- Supervisors
- Kenny Albes
Daniel Lohmann - Project
- ParPerOS
- Bearbeiter
- Ash Vollmer