Skip to content
GitLab
Projects Groups Topics Snippets
  • /
  • Help
    • Help
    • Support
    • Community forum
    • Submit feedback
  • Register
  • Sign in
  • L llm.c - GPT2
  • Project information
    • Project information
    • Activity
    • Labels
    • Members
  • Repository
    • Repository
    • Files
    • Commits
    • Branches
    • Tags
    • Contributor statistics
    • Graph
    • Compare revisions
  • Issues 4
    • Issues 4
    • List
    • Boards
    • Service Desk
    • Milestones
  • Merge requests 0
    • Merge requests 0
  • CI/CD
    • CI/CD
    • Pipelines
    • Jobs
    • Schedules
  • Deployments
    • Deployments
    • Environments
    • Releases
  • Packages and registries
    • Packages and registries
    • Package Registry
    • Container Registry
    • Terraform modules
  • Monitor
    • Monitor
    • Metrics
    • Incidents
  • Analytics
    • Analytics
    • Value stream
    • CI/CD
    • Repository
  • Wiki
    • Wiki
  • Snippets
    • Snippets
  • Activity
  • Graph
  • Create a new issue
  • Jobs
  • Commits
  • Issue Boards
Collapse sidebar
  • tchatela
  • llm.c - GPT2
  • Wiki
  • Task Based Model

Task Based Model · Changes

Page history
Update Task Based Model authored Sep 20, 2024 by tchatela's avatar tchatela
Hide whitespace changes
Inline Side-by-side
Task-Based-Model.md
View page @ 588a3514
...@@ -46,8 +46,27 @@ We would like to leverage taskiter to remove the overhead of task creation. Howe ...@@ -46,8 +46,27 @@ We would like to leverage taskiter to remove the overhead of task creation. Howe
for (int token=0; token < T; token+=grainsize) { for (int token=0; token < T; token+=grainsize) {
#pragma oss task #pragma oss task
{ {
#pragma oss taskloop #pragma oss taskloop collapse(x)
for ( /* innerloop */) for ( /* innerloop(s) */) {
/* We can have some other loops here */
/* calculations */
}
} }
} }
``` ```
\ No newline at end of file
We will need to move to this kind of structure :
```
for (int token=0; token < T; token+=grainsize) {
for ( /* previous inner loops flatten in one loop */) {
# pragma oss task
/* calculations */
}
}
```
This way, we have transformed the outer loops and all collapsed inner loops into two loops, inside of it we launch a single task. This way we have removed the nested task by implementing the two task by hand.
NB : Currently, you can set the collapsed depth and grainsize in _ompss-2_wrappers.h_, however the grainsize is not as powerful as an ompss grainsize, as it applies only to the innermost flatten loop.
# Performances
Clone repository

GPT2 Parallelization and Porting

  • Model Description
  • Runtime and Performances
  • Fork Join Model
  • Task Based Model
  • Distributed Model
  • Various Informations