15-440, Spring 2014, Class 05, Jan 28, 2014
Notes by Srinivasan Seshan and David G. Andersen
- Waitlist update
- Piazza questions checkin and reminder
All code available in:
/afs/cs.cmu.edu/academic/class/15440-f12/code/class05
Managing Concurrency
Useful references
http://golang.org/doc/effective_go.html#concurrency
Coverage:
* Classical synchronization with locks & condition variables
* Using Go channels to control access to resources
* Using a client/server model to manage concurrent access to
shared resources (style encouraged by Go).
We're going to come back to issues of concurrency again and
again in this class: Ensuring correct operation when
multiple independent actors (threads, processes, computers, etc.)
are trying to use a set of shared resources at the same time.
This happens from settings ranging from a single-core CPU
to wide-area distributed systems involving millions of hosts.
Today we'll start with some of the basics with threads on a single
node, but don't be fooled - we'll be expanding upon these to
the multiple machine case in the future. The difference between
today and those future lectures is that today, we don't have
to worry as much about independent failures: If the computer
running a thread loses power, e.g., all of the threads will disappear.
[Instructor nootes: Convey desirable properties of mutual exclusion:
- Correctness - achieves mutex, does not deadlock, does not livelock
- Efficiency
- Fairness (often)
]
Review. In 213, you learned the basics of concurrency.
Classical model:
Have set of threads running within an address space. Some parts of
state are shared, some are private. In typical application, establish
set of conventions:
- What data will be shared between threads
- How will we control access to shared data
Latter is done via various synchronization mechanisms. In particular,
you learned about semaphores:
Integer variable x that can operated on with two operations:
x.P():
while (x == 0) wait;
x--
x.V():
x++
Both operations are done atomically, meaning that all steps take place
without any intervening operations.
Special case: Binary semaphore == Mutex:
x = 1: Unlocked. Resource is available
x = 0: Locked. Must wait to get to resource.
Common refer to operations P = "Lock" and V = "Unlock"
Let's look at following problem:
[Instructor note: Ask about semantics of Go channels.]
Want to create FIFO queue that supports thread-safe operations:
(Note analogy here to Go channels! But without capacity limit.)
b.Init()
Initialize values
b.Insert(x)
Insert item into queue
b.Remove()
Block until queue not empty (if necessary)
Return element at head of queue
b.Flush()
Clear queue
Assume that we already have a sequential implementation of a buffer.
Suppose that b represented by structure with fields:
sb: Sequential buffer implementation
mutex: Mutual exclusion lock
Clearly need to wrap with mutex:
b.Init():
b.sb = NewBuf()
b.mutex = 1
b.Insert(x):
b.mutex.lock()
b.sb.Insert(x)
b.mutex.unlock()
b.Remove():
b.mutex.lock()
x = b.sb.Remove() # Oops. What if sb is empty?
b.mutex.unlock()
return x
b.Flush():
b.mutex.lock()
b.sb.Flush()
b.mutex.unlock()
What's wrong with this code?
Answer: If call Remove when buffer is empty, will call sb.Remove(),
which is invalid.
Let's try to fix this.
Silly fix 1:
b.Remove()
retry:
b.mutex.lock()
if !(b.sb.len() > 0) {
b.mutex.unlock()
goto retry
}
...
What's wrong with this? This is a spin lock! Wastes resources,
and no guarantee that an Insert() will ever make progress.
Inefficient and potential LIVELOCK, though eventually, it might
get through. Not efficient. Not necessarily correct.
Bryant & O'Hallaron, Figure 12.25 (p. 968) use semaphore "items" that
counts number of items in buffer
b.Initialize():
b.sb = NewBuf()
b.mutex = 1
b.items = 0
b.Insert(x):
b.lock()
b.sb.Insert(x)
b.mutex.unlock()
b.items.V()
b.Remove():
b.items.P()
// This is the point of vulnerability.
// What if someone else flushes right here?
b.mutex.lock()
x = b.sb.Remove()
b.mutex.unlock()
return x
b.Flush():
b.mutex.lock()
b.sb.Flush()
b.items = 0
b.mutex.unlock()
What's wrong?
Answer: For just Insert & Remove, this would work fine. But Flush
messes things up. If flush occurs at point of vulnerability in
Remove, then again find self trying to remove from empty buffer.
Fixing race condition. How about this:
b.Remove():
b.mutex.lock()
// What if get here with an empty buffer? We're blocking
// any thread that could fill it.
b.items.P()
x = b.sb.Remove()
b.mutex.unlock()
return x
Answer: Avoids race, but prone to DEADLOCK: reach point where no one
is able to proceed.
In this case:
Remove when buffer is empty. Remove gets lock. Somewhere else, want
to Insert, but can't get past lock.
Find that it's really hard to fix. My attempts with using binary
semaphore to indicate whether or not buffer empty failed.
(See code in code/class05/syncbuf/lbuf)
Better approach: Use CONDITION VARIABLES.
Condition variables provide a synchronization point, where one thread
can suspend until activated by another.
Condition variable always associated with a mutex.
(Must have unique mutex for given cvar. One mutex can work with
multiple cvar's).
Assume cvar connected to mutex:
cvar.Wait():
Must be called after locking mutex.
Atomically: release mutex & suspend operation
When resume, lock mutex (but maybe not right away)
cvar.Signal():
If no thread suspended, then no-op
Wake up (at least) one suspended thread.
(Typically do within scope of mutex, but not required)
Code for buffer with condition variables:
b.Initialize():
b.sb = NewBuf()
b.mutex = 1
b.cvar = NewCond(b.mutex)
b.Insert(x):
b.lock()
b.sb.Insert(x)
b.cvar.Signal() # Optionally: Do only when previously empty
b.mutex.unlock()
// First Version
b.Remove():
b.mutex.lock()
if b.sb.Empty() {
b.cvar.Wait() // Note that lock is first released & then retaken
}
x = b.sb.Remove()
b.mutex.unlock()
return x
b.Flush():
b.mutex.lock()
b.sb.Flush()
b.mutex.unlock()
Remove isn't quite right. Here's the problem:
cvar.wait has 3 steps:
Atomically { Release lock; suspend operation }
...
Resume execution
// Point of vulnerability. Small chance that someone could flush here.
Get lock
// Correct Version
b.Remove():
b.mutex.lock()
// Code looks weird. But remember that are releasing and
// regaining lock each time around loop.
while b.sb.Empty() {
b.cvar.Wait() // Note that lock is first released & then retaken
}
x = b.sb.Remove()
b.mutex.unlock()
return x
What the loop makes happen:
Lock
if !sb.empty() goto ready
Unlock
wait for signal
Lock
if !sb.empty() goto ready
Unlock
wait for signal
Lock
. . .
ready: Can safely assume that have lock & that buffer nonempty
(Complete code in code/class05/syncbuf/cvbuf)
Using Go channels
Go promotes a different view of concurrency, where set up miniature
client/server structures within a single program. Use "channels" as
mechanism for:
- Passing information around
- Synchronizing goroutines
- Providing pointer to return location (like a "callback")
Basic idea:
Can make channel of any object type:
* Bounded FIFO queue
c := make(chan int, 17)
d := make(chan string, 0)
* Insertion: c <- 21
If channel already full, then wait for receiver.
Then put value at end
* Removal s := <- d
If channel empty, then wait for sender
Then get first value
Note that when channel has capacity 0, then insertion & removal are a
"rendezvous"
Variations:
Capacity = 0: Synchronized send & receive
Insert Remove
|
----->|----->
|
Capacity = 1: Token passed from sender to receiver
Insert Remove
+---+
----->| |----->
+---+
Capacity = n: Bounded FIFO
Insert Remove
+-----------------------+
----->| |----->
+-----------------------+
Example: Use as mutex
type Mutex struct {
mc chan int
}
// Create an unlocked mutex
func NewMutex() *Mutex {
m := &Mutex{make(chan int, 1)}
m.Unlock() # Initially, channel empty == locked
return m
}
func (m *Mutex) Lock() {
<- m.mc # Don't care about value
}
func (m *Mutex) Unlock() {
m.mc <- 1 # Stick in value 1.
}
How about using channel to implement concurrent buffer:
- Acts as FIFO
- Allows concurrent insertion & removal
Shortcomings:
Size bounded when initialized. Cannot implement bounded buffer
No way to test for emptiness. When read from channel, cannot put
back value at head positionNo way to flush
No way to examine first element ("Front" operation)
Basic point:
- Channels are very low level.
- Most applications require building more structure on top of
channels.
Method 1: Using channels for rendezvous:
(See code/class05/chanbuf/abuf/abuf.go)
Idea: Have goroutine for buffer that acts as traffic director:
- Receives request(s) on incoming channel(s)
- Selects one that may proceed
- Calling function does operation
- Tells director that it is done.
Find that need two request channels:
Operations that can proceed in any case
-
Operations that block if buffer is empty
Read Ops ---->|
Other Ops ---->| Director
|<---- Ack channel
When buffer empty, only accept requests from 1st channel.
Use Go operation "select" to choose between them when buffer nonempty.
Can share channel for Acking back to function.
Makes use of rendezvous property of channels
type Buf struct {
sb *bufi.Buf // Sequential buffer
ackchan chan int // Signals completion of operation
readchan chan int // Allows blocking when reading
opchan chan int // For nonblocking operations
}
func NewBuf() *Buf {
bp := new(Buf)
bp.sb = bufi.NewBuf()
bp.ackchan = make(chan int)
bp.readchan = make(chan int)
bp.opchan = make(chan int)
go bp.director()
return bp
}
// Go routine to respond to requests
func (bp *Buf) director() {
for {
if bp.sb.Empty() {
// Enable only nonblocking operations
bp.opchan <- 1
} else {
// Enable reads and other operations
select { # Will allow only one communication
case bp.readchan <- 1:
case bp.opchan <- 1:
}
}
<- bp.ackchan // Wait until operations completed
}
}
func (bp *Buf) startop() { <- bp.opchan }
func (bp *Buf) startread() { <- bp.readchan }
func (bp *Buf) finish() { bp.ackchan <- 1 }
func (bp *Buf) Insert(val interface{}) {
bp.startop()
bp.sb.Insert(val)
bp.finish()
}
func (bp *Buf) Remove() interface{} {
bp.startread()
v := bp.sb.Remove()
bp.finish()
return v
}
Even More Go-Like:
Use channels to implement client/server model.
Go routine that does all operations on buffer
Functions supply requests into channel
Request includes reply channel as "return address"
(See code/class05/srvbuf/sserver/sserver.go)
This is how to get an enumerated type in Go
const (
doinsert = iota
doremove
doflush
doempty
)
Message format. Use same message format for all operation
If operation does not require value, then use value nil.
Reply will be either buffer value, nil, or boolean
type request struct {
op int // What operation is requested
val interface{} // Optional value for operation
replyc chan interface{} // Channel to which to send response
}
Version 1: Maintain two request channels:
type Buf struct {
// Buffer has two request channels
opc chan *request // Nonblocking operations
readc chan *request // Operations that block for empty buffer
}
func NewBuf() *Buf {
bp := &Buf{make(chan *request), make(chan *request)}
go bp.runServer()
return bp
}
func (bp *Buf) runServer () {
// Create actual buffer
sb := bufi.NewBuf() // Note that this can be private to goroutine
for {
var r *request
if sb.Empty() {
r = <- bp.opc
} else {
select {
case r1 := <- bp.opc:
r = r1
case r2 := <- bp.readc:
r = r2
}
}
switch r.op {
case doinsert:
sb.Insert(r.val)
r.replyc <- nil
case doremove:
v,_ := sb.Remove()
r.replyc <- v
case doflush:
sb.Flush()
r.replyc <- nil
case doempty:
e := sb.Empty()
// Can send Boolean along channel
r.replyc <- e
}
}
}
func (bp *Buf) doop(op int, val interface{}) interface{} {
r := &request{op, val, make(chan interface{})}
bp.opc <- r
v := <- r.replyc ## Wait until operation completed
return v
}
func (bp *Buf) doread(op int, val interface{}) interface{} {
r := &request{op, val, make(chan interface{})}
bp.readc <- r
v := <- r.replyc ## Wait until operation completed
return v
}
Exported functions
func (bp *Buf) Insert(val interface{}) {
bp.doop(doinsert, val)
}
func (bp *Buf) Remove() interface{} {
return bp.doread(doremove, nil)
}
func (bp *Buf) Flush() {
bp.doop(doflush, nil)
}
Final implementation. Same idea, but rather than using separate
channels, create buffer of "deferred" requests. We just happen to
have a suitable buffer data structure available!
(See code/class05/chanbuf/abuf/abuf.go)
// Which operations require waiting when buffer is empty
This is the way to implement a set in Go.
var deferOnEmpty = map [int] bool { doremove : true }
Same ideas as before
type request struct {
op int // What operation is requested
val interface{} // Optional value for operation
replyc chan interface{} // Channel to which to send response
}
Only one channel to implement external interface
type Buf struct {
requestc chan *request // Request channel for buffer
}
func NewBuf() *Buf {
bp := &Buf{make(chan *request)}
go bp.runServer()
return bp
}
func (bp *Buf) runServer () {
// Buffer to hold data
sb := bufi.NewBuf()
// Buffer to hold deferred requests
db := bufi.NewBuf()
for {
var r request
// No need for select. We do our own scheduling!
if !sb.Empty() && !db.Empty() {
// Revisit deferred operation
b, _ := db.Remove()
r = b.(request)
} else {
r = <- bp.requestc
if sb.Empty() && deferOnEmpty[r.op] {
// Must defer this operation
db.Insert(r)
continue
}
}
switch r.op {
case doinsert:
sb.Insert(r.val)
r.replyc <- nil
case doremove:
v := sb.Remove()
r.replyc <- v
case doflush:
sb.Flush()
r.replyc <- nil
case doempty:
e := sb.Empty()
// Can send Boolean along channel
r.replyc <- e
case dofront:
v := sb.Front()
r.replyc <- v
}
}
}
func (bp *Buf) dorequest(op int, val interface{}) interface{} {
r := &request{op, val, make(chan interface{})}
bp.requestc <- r
v := <- r.replyc
return v
}
func (bp *Buf) Insert(val interface{}) {
bp.dorequest(doinsert, val)
}
func (bp *Buf) Remove() interface{} {
return bp.dorequest(doremove, nil)
}
func (bp *Buf) Empty() bool {
v := bp.dorequest(doempty, nil)
e := v.(bool)
return e
}
func (bp *Buf) Flush() {
bp.dorequest(doflush, nil)
}