Skip to content
GitLab
Projects
Groups
Topics
Snippets
/
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Register
Sign in
Toggle navigation
Menu
GitLab is undergoing maintenance
You are on a read-only GitLab instance.
tchatela
llm.c - GPT2
Wiki
Distributed Model
Distributed Model
· Changes
Page history
Update Distributed Model
authored
Sep 20, 2024
by
tchatela
Show whitespace changes
Inline
Side-by-side
Distributed-Model.md
View page @
a2adf0f5
...
@@ -148,3 +148,10 @@ With B=16, T=1024, 1 rank/socket
...
@@ -148,3 +148,10 @@ With B=16, T=1024, 1 rank/socket
| 16 | 4 | 15084 ms | 1086 | 4.9 |
| 16 | 4 | 15084 ms | 1086 | 4.9 |
| 32 | 8 | 31649 ms | 1035 | 4.6 |
| 32 | 8 | 31649 ms | 1035 | 4.6 |
| 64 | 16 | 68551 | 956 | 4.3 |
| 64 | 16 | 68551 | 956 | 4.3 |
## Trace
Here we are using B08, T=1024, 4 ranks, 1 rank/socket
