Digital Design & Computer Architecture - Lecture 23: Memory Hierarchy and Caches (Spring 2022)

Why It Matters

These technology trade-offs directly shape processor architecture, system performance, and storage design; emerging nonvolatile memories could upend conventional memory hierarchies and enable new system-level optimizations. Understanding the limits of each memory type is essential for architects and businesses planning for future compute and storage products.

Summary

The lecture reviewed fundamentals of memory organization and the design of memory hierarchies and caches, emphasizing why SRAM is used for on-chip caches while DRAM serves as main memory due to differing fabrication and capacitor requirements. It surveyed memory technologies — DRAM, SRAM, flash, phase-change (PCM) and emerging resistive memories — highlighting trade-offs such as latency versus density, non-volatility, and endurance constraints. The instructor noted how technologies like flash and PCM can disrupt traditional hierarchies by blurring the line between working memory and storage, and described the complexity of flash-based storage controllers and their internal memory hierarchies. Practical manufacturing and system-design considerations were stressed as key reasons for current architectural choices and potential future shifts.

Original Description

Digital Design and Computer Architecture, ETH Zürich, Spring 2022 (https://safari.ethz.ch/digitaltechnik/spring2022/)

Lecture 23: Memory Hierarchy and Caches

Lecturer: Professor Onur Mutlu (https://people.inf.ethz.ch/omutlu/)

Date: May 20, 2022

Lecture 23 Slides (pptx): https://safari.ethz.ch/digitaltechnik/spring2022/lib/exe/fetch.php?media=onur-digitaldesign_comparch-2022-lecture23-memory-hierarchy-and-caches-afterlecture.pptx

Lecture 23 Slides (pdf): https://safari.ethz.ch/digitaltechnik/spring2022/lib/exe/fetch.php?media=onur-digitaldesign_comparch-2022-lecture23-memory-hierarchy-and-caches-afterlecture.pdf

Recommended Reading:

====================

A Modern Primer on Processing in Memory

https://arxiv.org/abs/2012.03112

Intelligent Architectures for Intelligent Computing Systems

https://arxiv.org/abs/2012.12381

RowHammer: A Retrospective

https://people.inf.ethz.ch/omutlu/pub/RowHammer-Retrospective_ieee_tcad19.pdf

Fundamentally Understanding and Solving RowHammer

https://arxiv.org/abs/2211.07613

RECOMMENDED LECTURE VIDEOS & PLAYLISTS:

========================================

Computer Architecture Fall 2021 Lectures Playlist:

https://www.youtube.com/watch?v=4yfkM_5EFgo&list=PL5Q2soXY2Zi-Mnk1PxjEIG32HAGILkTOF

Digital Design and Computer Architecture Spring 2021 Livestream Lectures Playlist:

https://www.youtube.com/watch?v=LbC0EZY8yw4&list=PL5Q2soXY2Zi_uej3aY39YB5pfW4SJ7LlN

Featured Lectures:

https://www.youtube.com/watch?v=jVYCchBGNVc&list=PL5Q2soXY2Zi8VrmOTz44l2WupethSdh-M&index=1

Interview with Professor Onur Mutlu:

https://www.youtube.com/watch?v=8ffSEKZhmvo&list=PL5Q2soXY2Zi8VrmOTz44l2WupethSdh-M&index=9

The Story of RowHammer Lecture:

https://www.youtube.com/watch?v=sgd7PHQQ1AI&list=PL5Q2soXY2Zi8D_5MGV6EnXEJHnV2YFBJl&index=39

Accelerating Genome Analysis Lecture:

https://www.youtube.com/watch?v=r7sn41lH-4A&list=PL5Q2soXY2Zi8D_5MGV6EnXEJHnV2YFBJl&index=41

Memory-Centric Computing Systems Tutorial at IEDM 2021:

https://www.youtube.com/watch?v=H3sEaINPBOE&list=PL5Q2soXY2Zi8D_5MGV6EnXEJHnV2YFBJl&index=35

Intelligent Architectures for Intelligent Machines Lecture:

https://www.youtube.com/watch?v=GTieZPY4Wmc&list=PL5Q2soXY2Zi8D_5MGV6EnXEJHnV2YFBJl&index=38

Computer Architecture Fall 2020 Lectures Playlist:

https://www.youtube.com/watch?v=c3mPdZA-Fmc&list=PL5Q2soXY2Zi9xidyIgBxUz7xRPS-wisBN

Digital Design and Computer Architecture Spring 2020 Lectures Playlist:

https://www.youtube.com/watch?v=AJBmIaUneB0&list=PL5Q2soXY2Zi_FRrloMa2fUYWPGiZUBQo2

Public Lectures by Onur Mutlu, Playlist:

https://www.youtube.com/watch?v=kgiZlSOcGFM&list=PL5Q2soXY2Zi8D_5MGV6EnXEJHnV2YFBJl

Computer Architecture at Carnegie Mellon Spring 2015 Lectures Playlist:

https://www.youtube.com/watch?v=zLP_X4wyHbY&list=PL5PHm2jkkXmi5CxxI7b3JCL1TWybTDtKq

Rethinking Memory System Design Lecture @stanfordonline :

https://www.youtube.com/watch?v=F7xZLNMIY1E&list=PL5Q2soXY2Zi8D_5MGV6EnXEJHnV2YFBJl&index=4