[torch]parallel

examples: imagenet
https://github.com/soumith/imagenet-multiGPU.torch

http://facebook.github.io/fbcunn/fbcunn/#fbcunn.fbcunn.DataParallel.dok
fbcunn.DataParallel

DataParallel splits the input along separate columns, that run the same models on distinct partitions of the input.

Pictorially

                    +--------+
    column 1        |        |         column 3
       +------------+  Input +-------------+
       |            |        |             |
       |            +----+---+             |
       |                 |                 |
       |                 |                 |
  +----+---+        +----+---+        +----+---+
  |        |        |        |        |        |
  | Linear |        | Linear |        | Linear |       row 1
  |        |        |        |        |        |
  +----+---+        +----+---+        +----+---+
       |                 |                 |
       |                 |                 |
  +----+---+        +----+---+        +----+---+
  |        |        |        |        |        |
  |  Tanh  |        |  Tanh  |        |  Tanh  |       row 2
  |        |        |        |        |        |
  +----+---+        +----+---+        +----+---+
       |                 |                 |
       |                 |                 |
       |                 |                 |
       |            +----+---+             |
       |            |        |             |
       +------------+ Output +-------------+
                    |        |
                    +--------+

Undocumented methods

fbcunn.DataParallel:name()
fbcunn.DataParallel:updateGradInput(_input, gradOutput)
fbcunn.DataParallel:accUpdateGradParameters(_input, _gradOutput, lr)

https://github.com/clementfarabet/lua—parallel