Introduction to RCU

Introduction to RCU

The best introduction to RCU is myLinux Weekly News three-part series, with update:

1. What is RCU, Fundamentally? with Jonathan Walpole(bibtex).
2. What is RCU? Part 2: Usage(bibtex).
3. RCU part 3: the RCU API(bibtex).
4. The RCU API, 2010 Edition.

These expand on the older“What is RCU?” introduction.The Wikipedia articlealso has some good information, as does theACM Queuearticle.

There is also someresearchon the general family of algorithms of which RCU is a member(bibtex)and an annotated bibliography.Alexey Gotsman, Noam Rinetzky, and Hongseok Yang have produced aformalization of RCU based on separation logic.

How much is RCU used in the Linux kernel?

Implementing RCU

The following papers describe how to implement RCU, in roughlyincreasing order of accessibility:

1. Lockdep-RCU.
2. RCU: The BloatwatchEdition (optimized for uniprocessor operation)(bibtex).
3. SleepableRead-Copy Update (SRCU), revision ofLinux Weekly Newsarticle(bibtex).
4. The classic PDF revision ofPDCS'98 paper on DYNIX/ptx's RCU implementation(bibtex).
5. The February 2012IEEE TPDS paper(bibtex)is the best source of information on what RCU is, how toimplement it in userspace, and how it performs.The pre-publication accepted version of this paper may be foundhere (main paper) andhere (supplementary materials).Some of the material in this paper came fromMathieu Desnoyers's Ph.D.dissertation(bibtex).
6. Using Promela and Spinto verify parallel algorithms at Linux Weekly News(bibtex).Includes description of QRCU implementation.
7. My Ph.D. dissertationon RCU, which includes descriptions of a number of earlyimplementations(bibtex).
8. The design of preemptableread-copy update (Linux Weekly News article)(bibtex).Please be warned: this is a detailed design document of themost complex known RCU implementation. This implementation hassince been replaced by a faster, simpler, and more scalableimplementation, and an update of the documentation is pending.

There is an RCU to-do list that is updated sporadically.

Read-Copy Update (RCU) Papers

A more-complete list in reverse chronological order:

1. November 2013User-space RCU,Linux Weekly News, with Mathieu Desnoyers, Lai Jiangshan, andJosh Triplett.Subparts of this article are:URCU-protected hash tables,The URCU hash table API,URCU-protected queues and stacks,The URCU stack/queue API,User-space RCU: Atomic-operation and utility API,User-space RCU: Memory-barrier menagerie,The user-space RCU API,The RCU-protected list API,The RCU-barrier menagerie,
2. November 2013What Is RCU?,guest lecture to University of Cambridge(Prof. Peter Sewell).
3. October 2013Introduction to RCU Concepts: Liberal application of procrastinationfor accommodation of the laws of physics — for more than twodecades,LinuxCon Europe 2013 (part of Mathieu Desnoyers's Hands-OnTutorial on Scalability with Userspace RCU).
4. May 2013What Is RCU?,presented to TU Dresden Distributed OS class (Prof. Hermann Härtig).
5. May 2013What Is RCU?(video),presented to Indian Institute of Science (IISc) (Prof. K. Gopinath).
6. May 2013Structured Deferral: Synchronization via Procrastination,ACM Queue.
7. August 2012Real-Time Response on Multicore Systems:It Is Bigger Than You Think,Scaling Microconference, Linux Plumbers Conference.
8. May 2012What Is RCU?presented to TU Dresden Distributed OS class (Prof. Hermann Härtig).
9. February 2012Making RCU Safe For Battery-Powered Devicespresented to the Embedded Linux Conference.
10. February 2012User-Level Implementations of Read-Copy Update(bibtex) coveringwhat RCU is, how to implement it in userspace, and how it performs.The pre-publication accepted version of this paper may be foundhere (main paper) andhere (supplementary materials).
11. July 20113.0 and RCU: what went wrong.
12. December 2010The RCU API, 2010 Edition.
13. August 2010Scalable Concurrent Hash Tables via Relativistic Programming(bibtex).
14. February 2010Lockdep-RCUdescribing software-engineering enhancements to the Linux-kernelRCU implementations(bibtex).
15. January 2010Simplicity Through Optimization(presentation)(bibtex).
16. January 2009Using a Malicious User-LevelRCU to Torture RCU-Based Algorithms,at linux.conf.au(bibtex).Describes several user-level RCU implementations, and describeshow they can be used to validate kernel-level code using RCU.
17. November 2008HierarchicalRCU, in Linux Weekly News(bibtex).Describes a Linux-kernel RCU implementation designed to scaleto thousands of CPUs.
18. July 2008Introducing technology into the Linux kernel: a case studyin ACM SIGOPS Operating System Review, with Jon Walpole(updated to includeRCU changes through the 2.6.36 Linux kernel)(bibtex).
19. May 2008The read-copy-update mechanism for supporting real-timeapplications on shared-memory multiprocessor systems with Linuxin IBM Systems Journal,with Dinakar Guniguntala, Josh Triplett, and Jon Walpole(bibtex).
20. February 2008Introducing Technologyinto Linux(bibtex),or "Introducing your technology into Linux willrequire intoducing a LOT of Linux into your technology!!!"at the 2008 Linux Developer Symposium - China (revised).(Chinese translationof original.)
21. January 2008RCU part 3: the RCU API at Linux Weekly News(bibtex).
22. December 2007What is RCU? Part 2: Usage at Linux Weekly News(bibtex).
23. December 2007What is RCU, Fundamentally? at Linux Weekly News with Jonathan Walpole(bibtex).
24. December 2007Performance of Memory Reclamation for Lockless Synchronizationin the Journal of Parallel and Distributed Computing, withTom Hart, Angela Demke Brown, and Jonathan Walpole(bibtex).(Journal version of the IPDPS'06 paper.)
25. October 2007The design of preemptableread-copy update at Linux Weekly News(bibtex).
26. August 2007Using Promela and Spinto verify parallel algorithms at Linux Weekly News(bibtex).Includes proof of correctness for QRCU.
27. February 2007"Priority-Boosting RCU Read-Side Critical Sections",revision of earlierLinux Weekly Newsversion(bibtex).
28. October 2006"Sleepable Read-Copy Update", revision of earlierLinux Weekly Newsversion(bibtex).
29. July 2006"Extending RCU for Realtime and Embedded Workloads"with Dipankar Sarma, Ingo Molnar, and Suparna Bhattacharyaat OLS'2006(bibtex),and correspondingpresentation.
30. April 2006"Making Lockless Synchronization Fast: Performance Implicationsof Memory Reclamation", with Tom Hart and Angela Demke.IPDPS 2006 Best Paper.Paper(bibtex).Presentation.
31. July 2005Abstraction, Reality Checks,and RCU presented at University of Toronto's "Cider Seminar"series (abstract).
32. April 2005paper (revised) andpresentationdescribingLinux realtime and yet more modifications to RCU to enableeven more aggressive realtime response(bibtex).Presented at the 2005 linux.conf.au.
33. January 2005RCU Semantics: AFirst Attempt with Jon Walpole(bibtex).Technical report: engineering math meets RCU semantics.
34. December 2004James Morris'sRecent Developments in SELinux Kernel Performancepaper describes how RCU helped scalability of the SELinuxaudit vector cache (AVC).(I didn't have any involvement in creating this paper, butbelieve that it is well worth bringing to your attention.)
35. June 2004 paper describingmodifications to the Linux RCU implementation to make it safefor realtime use(bibtex).
36. May 2004dissertation andpresentation from Ph.D.defense(bibtex).Also some advicefor others who are embarking on a part-time Ph.D. program,and theannouncement.
37. January 2004 paper andpresentationfor RCU performance on different CPUsat linux.conf.au in Adelaide, Australia(bibtex).
38. January 2004Scaling dcache with RCU(bibtex).
39. October 2003Linux Journal introduction to RCU(bibtex).
40. PDF revision of FREENIX'03 paper(focusing on use of RCU in Linux's System-V IPC implementation)and correspondingpresentation(bibtex).
41. Enabling Autonomic Behavior in Systems Software With Hot Swapping(bibtex): describes how a RCU(AKA "generations") is used in K42 to enable hot-swapping ofimplementations of kernel algorithms.
42. PDF revision of OLS'02 paper(focusing on Linux-kernel infrastructure) and correspondingpresentation(bibtex).
43. PDF revision of OLS'01 paper(oriented to Linux kernel) and correspondingpresentation(bibtex).
44. PDF revision of RPE paper(more theoretical).
45. PDF revision of PDCS'98 paper(DYNIX/ptx)(bibtex).
46. Read-Copy Update:Using Execution History to Implement Low-Overhead Solutionsto Concurrency Problems.Introduction to read-copy update.
47. (slightly outdated)HTML version.

Linux RCU Work

The best summary of Linux RCU work is graphical, and may be foundhere.

Some selected RCU patches:

1. RCU wasaccepted into the Linux 2.5.43 kernel. Patches to RCU wereapplied to the 2.5.44 and 2.5.45 kernels. RCU was thus fullyfunctional in 2.5.45 and later Linux kernels, just in time forthe Halloween functionality freeze. ;-)
2. Patch to the System V IPC implementation using RCU wasaccepted into the Linux 2.5.46 kernel.
3. Patch providing a lock-free IPv4 route cache wasaccepted into the Linux 2.5.53 kernel.
4. Patch providing lock-free handler traversal for IPMI handling,added to theLinux 2.5.58 kernel.
5. Patch providing lock-free lookup of directory entries in thedcache subsystem, added to theLinux 2.5.62 kernel.
6. Patches to replace many uses of brlock with RCUin the2.5.69 kernel,with brlock being entirely eliminated in the2.5.70 kernel.
7. NMI handling for oprofile uses RCU in the2.5.73 kernel.
8. Fix ppc64 {pte,pmd}_free vs. hash_page race with RCU in the2.6.2 kernel.
9. Additional patches to the Linux kernel applyRCU to FD-set management, task-list traversal, and i_shared_semcontention reduction.
10. Yet more patches change RCU's API to conserve memory and stackspace.
11. Another patch tomonitor RCU grace period.
12. Another patch to applyRCU to fasync_lock,perhaps for the 2.7 timeframe.
13. Another set of patches apply modifications to the RCU infrastructureto make it safe for soft-realtime use(0/2,1/2,2/2).
14. The Reiser4filesystem uses RCU to defer freeing of jnodes.
15. An auditing patch uses RCU to guard the lists of auditingrules.
16. An SELinux scalability patchuses RCU to guard the audit vector cache, with 500x improvementin write() throughput on 32 CPUs, and about 50% improvement on2 CPUs.

K42 RCU Work

1. K42 is a researchOS at IBM that uses RCU pervasively as an existence lock.K42 developed RCU independently, as described in theGamsa paper.
2. K42 also uses RCU as a basis for hot-swapping:overviewand infrastructure, andimplementation details and results.