| ... | ... | @@ -7,7 +7,7 @@ Note that : |
|
|
|
- Transfer of gradients from worker to server is done through MPI_Reduce (SUM)
|
|
|
|
- Transfer of updated parameters from server to workers is done through MPI_Bcast
|
|
|
|
- The data sent for each of these operations are about 250 000 floats so about 1 Gb.
|
|
|
|
- Accroding to [MN5 overview](https://www.bsc.es/supportkc/docs/MareNostrum5/overview), the transfer speed of a node is about 1Gb/s
|
|
|
|
- According to [MN5 overview](https://www.bsc.es/supportkc/docs/MareNostrum5/overview), the transfer speed of a node is about 10Gb/s
|
|
|
|
- We are using one socket per rank
|
|
|
|
|
|
|
|
## Using 4 workers and 1 server
|
| ... | ... | |
