NOVA – a file system designed to maximize performance on hybrid memory systems – was introduced in 2016 by Jian Xu and Steven Swanson of the University of California, San Diego. Essentially, NOVA adapts conventional log-structured file system techniques to exploit the fast, random access that Non-Volatile Memory (NVMs) deliver. NOVA maintains separate logs for each inode to improve concurrency and stores file data outside the log to minimize size and reduce garbage collection costs. Moreover, NOVA’s logs provide metadata, data and mmap atomicity. In short, NOVA focuses on simplicity and reliability by keeping complex metadata structures in DRAM to accelerate lookup operations.

NOVA and Intel’s Optane DC Persistent Memory Module

NOVA was recently cited in a paper published by the Non-Volatile Systems Laboratory, which operates under the auspices of the Computer Science & Engineering University of California, San Diego. The paper, penned by Joseph Izraelevitz and additional researchers (including Steven Swanson), explores the basic performance measurements of Intel’s Optane DC Persistent Memory Module.

According to the paper, Intel’s nonvolatile DIMM supports byte-granularity accesses with access times on the order of DRAM, while also providing data storage that survives power outages. As the researchers note, Optane DC memory occupies a tier in-between SSDs and DRAM. It boasts higher latency (346 ns) than DRAM, although lower latency than an SSD. As opposed to DRAM, its bandwidth is asymmetric with respect to access type. Max read bandwidth is 33.2 GB/s and scales with thread count, while its maximum write bandwidth is 8.9 GB/s and peaks at only four threads.

“Optane DC PMMs can be used as large memory devices with a DRAM cache to hide their lower bandwidth and higher latency. When used in this Memory (or cached) mode, Optane DC memory has little impact on applications with small memory footprints. Applications with larger memory footprints may experience some slow-down relative to DRAM, [although they] are now able to keep much more data in memory,” the researchers explain.

Image Credit: Non-Volatile Systems Laboratory, the Computer Science & Engineering University of California, San Diego (https://arxiv.org/pdf/1903.05714.pdf)

“In contrast, in App Direct (or uncached) mode, Optane DC PMMs can be used as a persistent storage device. When used under a file system, this configuration can result in significant performance gains, especially when the file system is optimized to use the load/store interface of the Optane DC PMM and the application uses many small, persistent writes. For instance, using the NOVA-relaxed NVMM file system, we can improve the performance of Kyoto Cabinet by almost 2×.”

Non-Volatile Memory: Raising the memory Ziggurat

The NOVA Non-Volatile Memory-based file system also serves as the basis for an Non-Volatile Memory optimization tool known as Ziggurat. Essentially, Ziggurat exploits the benefits of NVM through intelligent data placement during file writes and data migration. More specifically, Ziggurat includes two placement predictors that analyze the file write sequences and predicts whether the incoming writes are both large and stable and whether updates to the file are likely to be synchronous.

Image Credit: University of California, San Diego (https://arxiv.org/pdf/1904.11560.pdf)

Subsequently, Ziggurat routes the incoming writes to the most suitable tier based on the prediction – with writes to synchronously-updated files going to the Non-Volatile Memory tier to minimize the synchronization overhead. Small, random writes also go to the NVM tier to entirely avoid random writes to disk. The remaining large sequential writes to asynchronously-updated files go to disk.

According to Morteza Hoseinzadeh of the University of California, San Diego, Ziggurat seeks five primary design goals: Send writes to the most suitable tier, only migrate cold data in cold files, high NVM space utilization, migrate file data in groups and high scalability. In a nutshell, says Hoseinzadeh, Ziggurat bridges the gap between disk-based storage and NVM-based storage and provides high performance and large capacity to applications.

The Future of SCMs

Commenting on NOVA and Ziggurat, Steven Woo, Rambus fellow and distinguished inventor, tells Rambus Press that the work of Hoseinzadeh and Izraelevitz shows that the industry’s efforts to bring storage class memories (SCMs) to market have a lot of potential.

“Tiering is a logical way to integrate SCMs into memory and storage hierarchies. However, the newness of the technology means that the industry must continue doing research to determine how best to obtain the advantages that these memories have to offer. Both Ziggurat and NOVA illustrate the importance of re-thinking assumptions about file systems and the memory/storage hierarchy in order to get the most out of SCMs,” he concludes.

Rambus and the IBM Server Group (IBM SG) are currently researching hybrid memory systems. Targeting one of the industry’s key performance challenges, Rambus Labs and IBM SG are working to optimize DRAM and emerging memories to create a high-capacity memory subsystem that delivers comparable performance to DRAM.

In this context, Rambus will develop a flexible prototype hybrid memory platform using the OpenCAPI interface to demonstrate performance of multiple memory types in real-world server applications.

“The exploding volume of data and rapidly evolving workloads for Big Data applications are placing tremendous pressure on data center memory systems for increased performance and capacity,” said Laura Stark, senior vice president and general manager of the Emerging Solutions Division at Rambus. “This agreement with the IBM Server Group demonstrates our ongoing collaboration with the industry to accelerate the development and adoption of advanced memory solutions.”

As Stark notes, enabling a high-capacity and high-density memory subsystem close to the processor improves overall system performance and allows data centers to more fully utilize CPU resources.

“Rambus and IBM’s flexible and innovative hybrid memory system architectures will combine standard DRAM using other technologies such as Flash, enhanced Flash, Phase Change Memory (PCM), Resistive RAM (ReRAM) and Spin Torque Transfer Magnetic RAM (STT-MRAM) to create high capacity memories at lower cost per bit, with performance levels comparable to that of DRAM,” she added.

Rambus joins the OpenCAPI Consortium and OpenPOWER foundation

In addition to collaborating with the IBM STG group, Rambus is joining the OpenCAPI Consortium, an open development community based on Coherent Accelerator Processor Interface technology, along with the OpenPOWER Foundation, an open development community based on the POWER microprocessor architecture. Rambus will be using the unique attributes of the POWER9 systems’ OpenCAPI interfaces to build a hybrid memory and development subsystem prototype.

Rambus joins a growing roster of technology organizations that are contributing to the OpenCAPI Consortium and driving data center server innovation. Through the Consortium, members are working collaboratively to innovate on top of OpenCAPI, a high-performance coherent bus standard designed to help the technology industry work collaboratively to better meet growing demands for more advanced memory, accelerators, networking and storage technology. With the OpenCAPI specification, developers can more easily enable high performance accelerators such as FPGAs, GPUs, network and storage accelerators to perform functions that a server’s general-purpose CPU is not optimized to execute.

Interested in learning more? You can check out our hybrid memory page here rambus.com/hybrid-memory.