|
|
# Why going towards a task based model
|
|
|
|
|
|
We want to go from a for join model to a task based one.
|
|
|
![backward_transformer_block](uploads/41e0ed558170fe7d13bd7efbf848bb57/backward_transformer_block.png) |
|
|
\ No newline at end of file |
|
|
![backward_transformer_block](uploads/41e0ed558170fe7d13bd7efbf848bb57/backward_transformer_block.png)
|
|
|
![Forward_pass](uploads/4e68b0a08e2153e3e885cd805fd1f8e3/Forward_pass.png)
|
|
|
|
|
|
Basically, we would like to avoid the black regions (sequential portions) that we can see in these two traces. Thus, creating a task based model will allow us to overlap the computation of 2 distinct layers, avoiding any CPU inactivity.
|
|
|
|
|
|
# Analysis of the for join model current state
|
|
|
|
|
|
First, we need to analyze what are the parts of the model that can be overlapped. Therefore, we will inspect each layer and check for their data dependencies. Then, depending on these dependencies, we will create a data flow diagram that will show the different tasks and how each one depends on a previous one.
|
|
|
|
|
|
## Data dependency
|
|
|
|