Skip to content
GitLab
Projects Groups Topics Snippets
  • /
  • Help
    • Help
    • Support
    • Community forum
    • Submit feedback
  • Register
  • Sign in
  • L llm.c - GPT2
  • Project information
    • Project information
    • Activity
    • Labels
    • Members
  • Repository
    • Repository
    • Files
    • Commits
    • Branches
    • Tags
    • Contributor statistics
    • Graph
    • Compare revisions
  • Issues 4
    • Issues 4
    • List
    • Boards
    • Service Desk
    • Milestones
  • Merge requests 0
    • Merge requests 0
  • CI/CD
    • CI/CD
    • Pipelines
    • Jobs
    • Schedules
  • Deployments
    • Deployments
    • Environments
    • Releases
  • Packages and registries
    • Packages and registries
    • Package Registry
    • Container Registry
    • Terraform modules
  • Monitor
    • Monitor
    • Metrics
    • Incidents
  • Analytics
    • Analytics
    • Value stream
    • CI/CD
    • Repository
  • Wiki
    • Wiki
  • Snippets
    • Snippets
  • Activity
  • Graph
  • Create a new issue
  • Jobs
  • Commits
  • Issue Boards
Collapse sidebar
  • tchatela
  • llm.c - GPT2
  • Wiki
  • Runtime and performances

Runtime and performances · Changes

Page history
Update Runtime and performances authored Jun 28, 2024 by tchatela's avatar tchatela
Hide whitespace changes
Inline Side-by-side
Runtime-and-performances.md
View page @ baa9c26f
...@@ -46,11 +46,11 @@ The paraver trace shows us that nearly half of the threads are working simultane ...@@ -46,11 +46,11 @@ The paraver trace shows us that nearly half of the threads are working simultane
![legend](uploads/1fbc249856438649a150b86717133a6a/legend.png) ![legend](uploads/1fbc249856438649a150b86717133a6a/legend.png)
A test has been run using openmp and extrae, with 2 threads and 2 cpus on one single NUMA (**numactl -N 1 -m 1**), to create a paraver trace of the execution. On the trace, we can see that most of the runtime is passed in matmul_backward (in green and yellow as there are 2 #pragma omp in this layer). A test has been run using openmp and extrae, with 2 threads and 2 cpus on one single NUMA (**numactl -N 1 -m 1**), to create a paraver trace of the execution. On the trace, we can see that most of the runtime is passed in matmul_backward (in green and yellow as there are 2 #pragma omp in this layer).
## Comparisons with NUMA system specification ## Comparisons with NUMA system specification
Here below is a graph of the time taken per iteration for this same test: Here below is a graph of the time taken per iteration for the test in previous section:
![test-2-openmp-tinyshakespeare-mean](uploads/80003c0cba5f8759a46e445151d8fcd4/test-2-openmp-tinyshakespeare-mean.png) ![test-2-openmp-tinyshakespeare-mean](uploads/80003c0cba5f8759a46e445151d8fcd4/test-2-openmp-tinyshakespeare-mean.png)
......
Clone repository
  • Distributed Model
  • Fork Join Model
  • GPT2 Parallelization and porting
  • Metrics
  • Runtime and performances
  • Task Based Model
  • Various informations
  • _sidebar
  • Home