... | ... | @@ -12,7 +12,7 @@ All the files contained in the benchmark should follow the following convention: |
|
|
- **N**: the number of elements
|
|
|
- **U**: a multiplier (M for millions, B for billions, etc.)
|
|
|
|
|
|
Example of a case *sphere* containing approximately 16 millions of elements: **sphere-16M**
|
|
|
Example of a case **[sphere]()** containing approximately 16 millions elements: **sphere-16M**
|
|
|
|
|
|
```sh
|
|
|
README.md
|
... | ... | @@ -30,4 +30,81 @@ sphere-16M-LNODS.mpio.bin |
|
|
sphere-16M-LTYPB.mpio.bin
|
|
|
sphere-16M.nsi.dat
|
|
|
sphere-16M.post.alyadat
|
|
|
``` |
|
|
\ No newline at end of file |
|
|
```
|
|
|
|
|
|
## README.md
|
|
|
|
|
|
```
|
|
|
Flow over a sphere
|
|
|
|
|
|
* Mesh: 16.7M elements, 2.9M nodes
|
|
|
* Modules: Nastin
|
|
|
* Physics: turbulent flow over a sphere
|
|
|
* Numerical model: Vreman turbulence model, convective term using the EMACS scheme
|
|
|
* Solution strategy: fractional step with Runge-Kutta of order 3
|
|
|
* Algebraic solvers: CG
|
|
|
```
|
|
|
|
|
|
## json
|
|
|
|
|
|
```json
|
|
|
{
|
|
|
"authors": [
|
|
|
"author1@mail.es",
|
|
|
"author2@mail.es"
|
|
|
],
|
|
|
"comparisons": [],
|
|
|
"description": "Turbulent flow over a sphere. Vreman turbulence model, convective term using the EMACS scheme. Solution strategy: fractional step with Runge-Kutta of order 3. Algebraic solvers: CG",
|
|
|
"executions": [
|
|
|
{
|
|
|
"mpi": 336,
|
|
|
"openmp": 0
|
|
|
}
|
|
|
],
|
|
|
"geometry": {
|
|
|
"elements": 16677400,
|
|
|
"nodes": 2876880
|
|
|
},
|
|
|
"modules": [
|
|
|
"nastin"
|
|
|
],
|
|
|
"name": "sphere-16M",
|
|
|
"postprocess": "none",
|
|
|
"timeout": "02:00:00"
|
|
|
}
|
|
|
```
|
|
|
`json` file follows almost the same [structure](Test-configuration) as for the testsuite.
|
|
|
|
|
|
Note that the field `comparisons` must be defined but empty since the benchmark does not perform any actual verification after the run.
|
|
|
|
|
|
Postprocess must also be defined as `"none"` because we do not want to perform any post-process.
|
|
|
|
|
|
A new field `geometry` is mandatory: it contains the fields `elements` (number of elements) and `nodes` (number of nodes), provided by the `.dom.dat` file.
|
|
|
|
|
|
## Case dimensions
|
|
|
|
|
|
The case must be able to run between 5 minutes and 1 hour. However, we set the timeout to 2 hours to anticipate any execution problem. The ideal MPI range is between 48 (1 MN node) and 768 (16 MN nodes) so it can fit with the debug queue.
|
|
|
The maximum MPI number is 2400 (50 MN nodes).
|
|
|
|
|
|
## Case configuration
|
|
|
|
|
|
Configure your case with the optimal number of MPI (launch several runs and select the run with the best performance) and the adequate number of iterations to not exceed one hour.
|
|
|
The reference build is the following:
|
|
|
|
|
|
```sh
|
|
|
#Edit those variables:
|
|
|
#Installation path:
|
|
|
export ALYA_CMAKE_INSTALL_DIR=/path/to/install/alya
|
|
|
#Execute these commands in the alya root directory
|
|
|
module purge
|
|
|
module load impi/2018.4
|
|
|
module load intel/2018.4
|
|
|
module load mkl/2018.4
|
|
|
rm -fr build
|
|
|
mkdir -p build
|
|
|
cd build
|
|
|
cmake -DCMAKE_BUILD_TYPE=Release -DINTEGER_SIZE=4 -DOPTIMIZATION_LEVEL=3 -DVECTOR_SIZE=16 -DWITH_MPI=ON -DWITH_NDIMEPAR=OFF -DCMAKE_Fortran_COMPILER=mpif90 -DCONF=none -DCUSTOM_Fortran_FLAGS="-fpp" -DCUSTOM_Fortran_FLAGS_ARCHITECTURE="-xCORE-AVX512 -mtune=skylake" -DCUSTOM_Fortran_FLAGS_CODE_COVERAGE="-prof-gen=srcpos -prof-dir /gpfs/projects/bsc21/alya_cicd/ts/gitlab/ts/alya-cc/cmake-mn-intel-2018.4-i4 -prof-src-root=/gpfs/projects/bsc21/alya_cicd/ts/gitlab/ts/alya" -DCUSTOM_Fortran_FLAGS_DEBUG="-ftrapuv -check all,noarg_temp_created -traceback -debug full -warn all,nodec,nointerfaces -fp-stack-check -ansi-alias -g" -DCUSTOM_Fortran_FLAGS_I8=-m64 -DCUSTOM_Fortran_FLAGS_IPO=-ipo -DCUSTOM_Fortran_FLAGS_O0=-O0 -DCUSTOM_Fortran_FLAGS_O1=-O1 -DCUSTOM_Fortran_FLAGS_O2=-O2 -DCUSTOM_Fortran_FLAGS_O3=-O3 -DCUSTOM_Fortran_FLAGS_STD2008=-std08 -DSHARED=OFF -DWITH_CODE_COVERAGE=ON -DWITH_LIBRARIES_ONLY=OFF -DWITH_METIS=auto -DWITH_STD2008=ON -DCMAKE_INSTALL_PREFIX=$ALYA_CMAKE_INSTALL_DIR -DWITH_MODULE_TURBUL=ON -DWITH_MODULE_SOLIDZ=ON -DWITH_MODULE_NASTIN=ON -DWITH_MODULE_ALEFOR=ON -DWITH_MODULE_MAGNET=ON -DWITH_MODULE_PARTIS=ON -DWITH_MODULE_LEVELS=ON -DWITH_MODULE_FAKEMO=ON -DWITH_MODULE_CHEMIC=ON -DWITH_MODULE_NASTAL=ON -DWITH_MODULE_TEMPER=ON -DWITH_MODULE_EXMEDI=ON ..
|
|
|
make -j4
|
|
|
ctest
|
|
|
make install
|
|
|
``` |