GD vs SGD

##GD

small number of model updates

accurate

each epoch may be expensive

easy to parallelize

##SGD

Requires lots of model updates

Not as accurate, but often good enough

A log of progress in one pass for big data

Not trivial to parallelize