... | @@ -20,7 +20,7 @@ All examples showing the usage of below environment variables assume a system us |
... | @@ -20,7 +20,7 @@ All examples showing the usage of below environment variables assume a system us |
|
|
|
|
|
## Loading EAR Library
|
|
## Loading EAR Library
|
|
|
|
|
|
### EAR_LOADER_APPLICATION
|
|
### EAR\_LOADER\_APPLICATION
|
|
|
|
|
|
Rules the EAR Loader to load the EAR Library for a specific application that does not follow any of the current programming models (or maybe a sequential app) supported by EAR.
|
|
Rules the EAR Loader to load the EAR Library for a specific application that does not follow any of the current programming models (or maybe a sequential app) supported by EAR.
|
|
Your system must have installed the non-MPI version of the Library (ask your system administrator).
|
|
Your system must have installed the non-MPI version of the Library (ask your system administrator).
|
... | @@ -38,7 +38,7 @@ srun --ntasks 1 --job-name=my_job_name ./my_exec_file |
... | @@ -38,7 +38,7 @@ srun --ntasks 1 --job-name=my_job_name ./my_exec_file |
|
|
|
|
|
See the [Use cases](User-guide#use-cases) section to read more information about how to run jobs with EAR.
|
|
See the [Use cases](User-guide#use-cases) section to read more information about how to run jobs with EAR.
|
|
|
|
|
|
### EAR_LOAD_MPI_VERSION
|
|
### EAR\_LOAD\_MPI\_VERSION
|
|
|
|
|
|
Forces to load a specific MPI version of the EAR Library.
|
|
Forces to load a specific MPI version of the EAR Library.
|
|
This is needed, for example, when you want to load the EAR Library for Python + MPI applications, where the Loader is not able to detect the MPI implementation the application is going to use.
|
|
This is needed, for example, when you want to load the EAR Library for Python + MPI applications, where the Loader is not able to detect the MPI implementation the application is going to use.
|
... | @@ -68,7 +68,7 @@ See the [Use cases](User-guide#use-cases) section to read more information about |
... | @@ -68,7 +68,7 @@ See the [Use cases](User-guide#use-cases) section to read more information about |
|
|
|
|
|
## Report plug-ins
|
|
## Report plug-ins
|
|
|
|
|
|
### EAR_REPORT_ADD
|
|
### EAR\_REPORT\_ADD
|
|
|
|
|
|
Specify a report plug-in to be loaded. The value must be a shared object file, and it must be located at `$EAR_INSTALL_PATH/lib/plugins/report` or at the path from where the job was launched.
|
|
Specify a report plug-in to be loaded. The value must be a shared object file, and it must be located at `$EAR_INSTALL_PATH/lib/plugins/report` or at the path from where the job was launched.
|
|
Alternatively, you can provide the full path (absolute or relative) of the report plug-in.
|
|
Alternatively, you can provide the full path (absolute or relative) of the report plug-in.
|
... | @@ -83,7 +83,7 @@ srun -n 10 my_mpi_app |
... | @@ -83,7 +83,7 @@ srun -n 10 my_mpi_app |
|
|
|
|
|
## Verbosity
|
|
## Verbosity
|
|
|
|
|
|
### EARL_VERBOSE_PATH
|
|
### EARL\_VERBOSE\_PATH
|
|
|
|
|
|
Specify a path to create a file (one per node involved in a job) where to print messages from the EAR Library.
|
|
Specify a path to create a file (one per node involved in a job) where to print messages from the EAR Library.
|
|
This is useful when you run a job in multiple nodes, as EAR verbose information for each of them can result in lots of messages mixed at stderr (EAR messages default channel).
|
|
This is useful when you run a job in multiple nodes, as EAR verbose information for each of them can result in lots of messages mixed at stderr (EAR messages default channel).
|
... | @@ -113,7 +113,7 @@ After the above job example completion, in the same directory where the applicat |
... | @@ -113,7 +113,7 @@ After the above job example completion, in the same directory where the applicat |
|
|
|
|
|
## Frequency management
|
|
## Frequency management
|
|
|
|
|
|
### EAR_GPU_DEF_FREQ
|
|
### EAR\_GPU\_DEF\_FREQ
|
|
|
|
|
|
Set a GPU frequency (in kHz) to be fixed while your job is running.
|
|
Set a GPU frequency (in kHz) to be fixed while your job is running.
|
|
The same frequency is set for all GPUs used by the job.
|
|
The same frequency is set for all GPUs used by the job.
|
... | @@ -137,7 +137,7 @@ srun --cpu-bind=core --ear-policy=min_energy gmx_mpi mdrun \ |
... | @@ -137,7 +137,7 @@ srun --cpu-bind=core --ear-policy=min_energy gmx_mpi mdrun \ |
|
-s $GROMACS_INPUT -noconfout -ntomp 1
|
|
-s $GROMACS_INPUT -noconfout -ntomp 1
|
|
```
|
|
```
|
|
|
|
|
|
### EAR_JOB_EXCLUSIVE_MODE
|
|
### EAR\_JOB\_EXCLUSIVE\_MODE
|
|
|
|
|
|
Indicate whether the job will run in a node exclusively (non-zero value).
|
|
Indicate whether the job will run in a node exclusively (non-zero value).
|
|
EAR will reduce the CPU frequency of those cores not used by the job.
|
|
EAR will reduce the CPU frequency of those cores not used by the job.
|
... | @@ -163,7 +163,7 @@ On this page we will use the term *uncore* to refer both of them. |
... | @@ -163,7 +163,7 @@ On this page we will use the term *uncore* to refer both of them. |
|
Environment variables related to uncore control covers [policy specific settings](#ear_set_imcfreq) or
|
|
Environment variables related to uncore control covers [policy specific settings](#ear_set_imcfreq) or
|
|
the chance for a user to [fix it](#ear_max_imcfreq-and-ear_min_imcfreq) during an entire job.
|
|
the chance for a user to [fix it](#ear_max_imcfreq-and-ear_min_imcfreq) during an entire job.
|
|
|
|
|
|
#### EAR_SET_IMCFREQ
|
|
#### EAR\_SET\_IMCFREQ
|
|
|
|
|
|
Enables/disables EAR's [eUFS](Home#publications) feature.
|
|
Enables/disables EAR's [eUFS](Home#publications) feature.
|
|
Type `ear-info` to see whehter eUFS is enabled by default.
|
|
Type `ear-info` to see whehter eUFS is enabled by default.
|
... | @@ -185,7 +185,7 @@ export SLURM_EAR_POLICY_IMC_TH=0.035 |
... | @@ -185,7 +185,7 @@ export SLURM_EAR_POLICY_IMC_TH=0.035 |
|
srun [...] my_app
|
|
srun [...] my_app
|
|
```
|
|
```
|
|
|
|
|
|
#### EAR_MAX_IMCFREQ and EAR_MIN_IMCFREQ
|
|
#### EAR\_MAX\_IMCFREQ and EAR\_MIN\_IMCFREQ
|
|
|
|
|
|
Set the maximum and minimum values (in kHz) at which *uncore* frequency should be.
|
|
Set the maximum and minimum values (in kHz) at which *uncore* frequency should be.
|
|
Two variables were designed because Intel(R) architectures let to set a range of
|
|
Two variables were designed because Intel(R) architectures let to set a range of
|
... | @@ -216,7 +216,7 @@ Please, contact with [ear-support@bsc.es](mailto:ear-support@bsc.es) if you want |
... | @@ -216,7 +216,7 @@ Please, contact with [ear-support@bsc.es](mailto:ear-support@bsc.es) if you want |
|
|
|
|
|
> A correct CPU binding it's required to get the most benefit of this feature. Check the documentation of your application programming model/vendor/flavour or yur system batch scheduler.
|
|
> A correct CPU binding it's required to get the most benefit of this feature. Check the documentation of your application programming model/vendor/flavour or yur system batch scheduler.
|
|
|
|
|
|
#### EAR_LOAD_BALANCE
|
|
#### EAR\_LOAD\_BALANCE
|
|
|
|
|
|
Enables/Disables EAR's Load Balance strategy in energy policies.
|
|
Enables/Disables EAR's Load Balance strategy in energy policies.
|
|
Type `ear-info` to see whether this feature is enabled by default.
|
|
Type `ear-info` to see whether this feature is enabled by default.
|
... | @@ -263,7 +263,7 @@ If you enable [EARL verbosity](User-guide#ear-job-submission-flags) you will see |
... | @@ -263,7 +263,7 @@ If you enable [EARL verbosity](User-guide#ear-job-submission-flags) you will see |
|
the mapping of the CLOS set for each CPU in the node.
|
|
the mapping of the CLOS set for each CPU in the node.
|
|
Note that a `-1` value means that no change was done on the specific CPU.
|
|
Note that a `-1` value means that no change was done on the specific CPU.
|
|
|
|
|
|
#### EAR_PRIO_TASKS
|
|
#### EAR\_PRIO\_TASKS
|
|
|
|
|
|
A list that specifies the CLOS that CPUs assigned to tasks must be set.
|
|
A list that specifies the CLOS that CPUs assigned to tasks must be set.
|
|
This variable is useful because you can configure your application transparently
|
|
This variable is useful because you can configure your application transparently
|
... | @@ -341,7 +341,7 @@ PRIO3: MAX GHZ - 0.0 GHz (low) |
... | @@ -341,7 +341,7 @@ PRIO3: MAX GHZ - 0.0 GHz (low) |
|
[120,-1] [121,-1] [122,-1] [123,-1] [124,-1] [125,-1] [126,-1] [127,-1]
|
|
[120,-1] [121,-1] [122,-1] [123,-1] [124,-1] [125,-1] [126,-1] [127,-1]
|
|
```
|
|
```
|
|
|
|
|
|
#### EAR_PRIO_CPUS
|
|
#### EAR\_PRIO\_CPUS
|
|
|
|
|
|
A list of priorities that should have the same length as the number of CPUs your
|
|
A list of priorities that should have the same length as the number of CPUs your
|
|
job is using.
|
|
job is using.
|
... | @@ -355,6 +355,12 @@ But it becomes more flexible when the user has more control over the affinity se |
... | @@ -355,6 +355,12 @@ But it becomes more flexible when the user has more control over the affinity se |
|
to its application, because you can discriminate between different CPUs assigned to the same task.
|
|
to its application, because you can discriminate between different CPUs assigned to the same task.
|
|
Moreover, this is the only way to set different priorities over different threads in no-MPI applications.
|
|
Moreover, this is the only way to set different priorities over different threads in no-MPI applications.
|
|
|
|
|
|
|
|
### EAR\_MIN\_CPUFREQ
|
|
|
|
|
|
|
|
This variable can only be set by **authorized users**, and modifies the minimum CPU frequency the EAR Library can set.
|
|
|
|
The [EAR configuration](Configuration) file has a field called `cpu_max_pstate` which sets this limits on the tag it is configured.
|
|
|
|
Authorized users can modify this limit at submission time by using this environment to test, for example, the best value for the `ear.conf` field.
|
|
|
|
|
|
### Disabling EAR's affinity masks usage
|
|
### Disabling EAR's affinity masks usage
|
|
|
|
|
|
For both [Load Balancing](load-balancing) and [Intel(R) SST](#support-for-intel-r-speed-select-technology)
|
|
For both [Load Balancing](load-balancing) and [Intel(R) SST](#support-for-intel-r-speed-select-technology)
|
... | @@ -364,7 +370,7 @@ To avoid any unexpected problem, **we highly recommend you** to export `EAR_NO_A |
... | @@ -364,7 +370,7 @@ To avoid any unexpected problem, **we highly recommend you** to export `EAR_NO_A |
|
|
|
|
|
## Data gathering
|
|
## Data gathering
|
|
|
|
|
|
### EAR_GET_MPI_STATS
|
|
### EAR\_GET\_MPI\_STATS
|
|
|
|
|
|
Use this variable to generate two files at the end of the job execution that will contain global, per process MPI information.
|
|
Use this variable to generate two files at the end of the job execution that will contain global, per process MPI information.
|
|
You must specify the prefix (optionally with a path) of the filename. One file (*[path/]prefix.ear_mpi_stats.full_nodename.csv*) will contain a resume about MPI throughput (per-process), while the other one (*[path/]prefix.ear_mpi_calls_stats.full_nodename.csv*) will contain a more fine grained information about different MPI call types.
|
|
You must specify the prefix (optionally with a path) of the filename. One file (*[path/]prefix.ear_mpi_stats.full_nodename.csv*) will contain a resume about MPI throughput (per-process), while the other one (*[path/]prefix.ear_mpi_calls_stats.full_nodename.csv*) will contain a more fine grained information about different MPI call types.
|
... | | ... | |