Configuration.md

-Configuration requirements
--------------
+[[_TOC_]]
+
+## Configuration requirements
+
 The following requirements must be met for EAR to work properly:
-- EAR folders: EAR uses two paths for EAR configuration. 
-    - **EAR_TMP:** *tmp_ear_path* must be a private folder per compute node. It must have read/write permissions for normal users. Communication files are created here. It must be created by the admin.
+
+### EAR paths
+
+**EAR folders** EAR uses two paths for EAR configuration:
+- **EAR_TMP:** *tmp_ear_path* must be a private folder per compute node. It must have read/write permissions for normal users. Communication files are created here. It must be created by the admin.
 For instance: `mkdir /var/ear; chmod ugo +rwx /var/ear`
-    - **EAR_ETC:** *etc_ear_path* must be readable for normal users in all compute nodes. It can be a shared folder in "GPFS" (simple to manage) or replicated data because it has very few data and it is modified at a very low frequency (**ear.conf** and coefficients). Coefficients can be installed in a different path specified at configure time in **COEFFS** flag. Both ear.conf and coefficients must be readable in all the nodes (compute and “service” nodes).
-- Configure **ear.conf**: ear.conf is an ascii file setting default values and cluster descriptions. An ear.conf is automatically generated based on a **ear.conf.in** template. However, sysadmin must include installation details such as hostname details for EAR services, ports, default values, and list of nodes. For more details, check [EAR configuration file](#ear-configuration-file) below.
-- MySQL DB or PostgreSQL DB: EAR saves data in a MySQL/PostgreSQL DB server. EAR DB can be created using `edb_create` command  provided (MySQL/PostgreSQL server must be running and root access to the DB is needed).
-- Set EAR SLURM plugin
-    - EAR SLURM plugin must be set in /etc/slurm/plugstack.conf. EAR generates an example at *ear_etc_path/slurm/ear.plugstack.conf*. For more information see our [Plugin section](Configuration#slurm-spank-plugin-configuration-file) down below.
+- **EAR_ETC:** *etc_ear_path* must be readable for normal users in all compute nodes. It can be a shared folder in "GPFS" (simple to manage) or replicated data because it has very few data and it is modified at a very low frequency (**ear.conf** and coefficients). Coefficients can be installed in a different path specified at configure time with **COEFFS** flag. Both `ear.conf` and coefficients must be readable in all the nodes (compute and _"service"_ nodes).
+
+**ear.conf** `ear.conf` is an ascii file setting default values and cluster descriptions. An `ear.conf` is automatically generated based on a **ear.conf.in** template. However, the administrator must include installation details such as hostname details for EAR services, ports, default values, and the list of nodes. For more details, check [EAR configuration file](#ear-configuration-file) below.
+
+### DB creation and DB server
+
+MySQL or PostgreSQL database: EAR saves data in a MySQL/PostgreSQL DB server. EAR DB can be created using `edb_create` command  provided (MySQL/PostgreSQL server must be running and root access to the DB is needed).
+
+### EAR SLURM plug-in
 
-EAR configuration file
-----------------------
-**ear.conf** is a text file describing the EAR package behaviour in the cluster. It must be readable by all compute nodes and by nodes where commands are executed. Two ear.conf templates are generated with default values and will be installed as reference when executing `make etc.install` 
+EAR SLURM plug-in can be enabled by adding an additional line at the `/etc/slurm/plugstack.conf` file.
+You can copy from the `ear_etc_path/slurm/ear.plugstack.conf` file).
 
-Usually the first word in the configuration file expresses the component related with the option. Lines starting with `#` are comments. 
+Another way to enable it is to create the directory `/etc/slurm/plugstack.conf.d` and copy there the `ear_etc_path/slurm/ear.plugstack.conf` file. On that case, the content of `/etc/slurm/plugstack.conf`
+must be `include /etc/slurm/plugstack.conf.d/*`.
 
-A test for ear.conf file can be found in the path `src/test/functionals/ear_conf`.
+# EAR configuration file
 
-###### In-depth EAR configuration file options
+The **ear.conf** is a text file describing the EAR package behaviour in the cluster.
+It must be readable by all compute nodes and by nodes where commands are executed.
+Two `ear.conf` templates are generated with default values and will be installed as reference when executing `make etc.install`. 
 
-### Database configuration
+Usually the first word in the configuration file expresses the component related with the option. Lines starting with `#` are comments.
+A test for `ear.conf` file can be found in the path `src/test/functionals/ear_conf`.
+It is recommended to test it since the `ear.conf` parser is very sensible to errors in the `ear.conf` syntax, spaces, newlines, etc.
 
-```INI
-# The IP of the node where the MariaDB (MySQL) or PostgreSQL server process is running. Current version uses same names for both DB servers
+## Database configuration
+
+```
+# The IP of the node where the MariaDB (MySQL) or PostgreSQL server process is running. Current version uses same names for both DB servers.
 DBIp=172.30.2.101
 # Port in which the server accepts the connections.
 DBPort=3306
-# MariaDB user that the services will use. Needs INSERT/SELECT privileges. Used by EARDBD
+
+# MariaDB user that services will use. Needs INSERT/SELECT privileges. Used by the EARDBD.
 DBUser=eardbd_user
 # Password for the previous user. If left blank or commented it will assume the user has no password.
 DBPassw=eardbd_pass
@@ -35,9 +51,12 @@ DBPassw=eardbd_pass
 DBCommandsUser=ear_commands
 # Password for the previous user. If left blank or commented it will assume the user has no password.
 DBCommandsPassw=commandspass
+
 # Name of EAR's database in the server.
 DBDatabase=EAR
-# Maximum number of connections of the commands user to prevent server saturation/malicious actuation. Applies to DBCommandsUser
+
+# Maximum number of connections of the commands user to prevent server 
+# saturation/malicious actuation. Applies to DBCommandsUser.
 DBMaxConnections=20
 # The following specify the granularity of data reported to database.
 # Extended node information reported to database (added: temperature, avg_freq, DRAM and PCK energy in power monitoring).
@@ -48,189 +67,252 @@ DBReportSigDetail=1
 DBReportLoops=1
 ```
 
-### EARD configuration. EARD are executed in compute nodes
+## EARD configuration
 
-```INI
+```
 # The port where the EARD will be listening.
 NodeDaemonPort=50001
+
 # Frequency used by power monitoring service, in seconds.
 NodeDaemonPowermonFreq=60
 # Maximum supported frequency (1 means nominal, no turbo).
 NodeDaemonMaxPstate=1
 # Enable (1) or disable (0) the turbo frequency.
 NodeDaemonTurbo=0
+
 # Enables the use of the database.
 NodeUseDB=1
 # Inserts data to MySQL by sending that data to the EARDBD (1) or directly (0).
 NodeUseEARDBD=1
 # '1' means EAR is controlling frequencies at all times (targeted to production systems) and 0 means EAR will not change the frequencies when users are not using EAR library (targeted to benchmarking systems).
 NodeDaemonForceFrequencies=1
+
 # The verbosity level [0..4]
 NodeDaemonVerbose=1
-# When set to 1, the output is saved in '$EAR_TMP'/eard.log (common configuration) as a log file.Otherwsie, stderr is used.
+# When set to 1, the output is saved at '$EAR_TMP'/eard.log (common configuration) as a log file. Otherwsie, stderr is used.
 NodeUseLog=1
-# Minimum time between two energy readings for performance accuracy
-MinTimePerformanceAccuracy=10000000
+
+# Report plug-ins to be used by the EARD. Default= eardbd.so. 
+# Add extra plug-ins by separating with colons (e.g., eardbd.so:plugin1.so).
+EARDReportPlugins=eardbd.so
 ```
 
-### EARDBD configuration
+## EARDBD configuration
 
-```INI
-# Port where the EARDBD server is listening
+```
+# Port where the EARDBD server is listening.
 DBDaemonPortTCP=50002
-# Port where the EARDBD mirror is listening
+# Port where the EARDBD mirror is listening.
 DBDaemonPortSecTCP=50003
-# Port is used to synchronize the server and mirror
+# Port used to synchronize the server and mirror.
 DBDaemonSyncPort=50004
-# In seconds, interval of time of accumulating data to generate an energy aggregation
+
+# In seconds, interval of time of accumulating data to generate an energy aggregation.
 DBDaemonAggregationTime=60
-# In seconds, time between inserts of the buffered data
+# In seconds, time between inserts of the buffered data.
 DBDaemonInsertionTime=30
-# Memory allocated per process. This allocations is used for buffering the data sent to the database by EARD or other components. If there is a server and mirror in a node a double of that value will be allocated. It is expressed in MegaBytes.
+# Memory allocated per process. These allocations are used for buffering the data
+# sent to the database by EARD or other components. If there is a server and a
+# mirror in a node a double of that value will be allocated. It is expressed in MegaBytes.
 DBDaemonMemorySize=120
-# When set to 1, eardbd uses a '$EAR_TMP'/eardbd.log file as a log file
+
+# When set to 1, EARDBD uses a '$EAR_TMP'/eardbd.log file as a log file.
 DBDaemonUseLog=1
+
+# Report plug-ins to be used by the EARDBD. Default= mysql.so. 
+# Add extra plug-ins by separating with colons (e.g., mysql.so:plugin1.so).
+EARDBDReportPlugins=mysql.so
 ```
 
-### EARL configuration
+## EARL configuration
 
-```INI
-# Path where coefficients are installed, usually $EAR_ETC/ear/coeffs
+```
+# Path where coefficients are installed, usually $EAR_ETC/ear/coeffs.
 CoefficientsDir=/path/to/coeffs
-# Number of levels used by DynAIS algorithm.
+
+# NOTE: It is not recommended to change the following
+# attributes if you are not an expert user.
+# Number of levels used by DynAIS algorithm. 
 DynAISLevels=10
 # Windows size used by DynAIS, the higher the size the higher the overhead.
 DynAISWindowSize=200
-# Maximum time in seconds that EAR will wait until a signature is computed. After this value, if no signature is computed, EAR will go to periodic mode.
+# Maximum time (in seconds) that EAR will wait until a signature is computed. After this value, if no signature is computed, EAR will go to periodic mode.
 DynaisTimeout=15
 # Time in seconds to compute every application signature when the EAR goes to periodic mode.
 LibraryPeriod=10
 # Number of MPI calls whether EAR must go to periodic mode or not.
 CheckEARModeEvery=1000
+# EARL default report plug-ins
+EARLReportPlug-ins=eard.so
 ```
 
-### EARGM configuration
+## EARGM configuration
 
-```INI
-# The IP or hostname of the node where the EARGMD daemon is running.
-EARGMHost=hostname
-# Port where EARGMD will be listening.
-EARGMPort=50000
-# Use '1' or not '0' aggregated metrics to compute total energy.
+You can skip this section if EARGM is not used in your installation.
+
+```
+# Use aggregated periodic metrics or periodic power metrics.
+# Aggregated metrics are only available when EARDBD is running.
 EARGMUseAggregated=1
-# Period T1 and period T2 are specified in seconds. T1 must be less than T2. Global manager updates the information every T1 seconds and uses the energy/power in T2 period to estimate energy/power constraints
+# Period T1 and T2 are specified in seconds. T1 must be less than T2, e.g., 10min and 1 month.
 EARGMPeriodT1=90
 EARGMPeriodT2=259200
-# Units field, Can be '-' (Joules), 'K' KiloJoules or 'M' MegaJoules
+# '-' are Joules, 'K' KiloJoules and 'M' MegaJoules.
 EARGMUnits=K
-# This limit means the maximum energy allowed in 259200 seconds in 550000 KJoules
+
+# Energy limit applies to EARGMPeriodT2.
 EARGMEnergyLimit=550000
-#
-# Global manager modes. Two modes are supported '0' (manual) or '1' (automatic). Manual means Gobal Manager is only monitoring energy&power and reporting to the DB . Automatic means it takes actions to guarantee energy limits. 
+EARGMPort=50000
+
+# Two modes are supported '0=manual' and '1=automatic'.
+# manual means no actions are token, only monitoring.
 EARGMMode=0
-# A mail can be sent reporting the warning level (and the action taken in automatic mode). 'nomail' means no mail is sent. This option is independent of the node.
+# Email address to report the warning level (and the action taken in automatic mode).
 EARGMMail=nomail
 # Percentage of accumulated energy to start the warning DEFCON level L4, L3 and L2.
 EARGMWarningsPerc=85,90,95
-# Number of "grace" T1 periods before doing a new re-evaluation. After a warning, EARGM will wait T1xGlobalManagerGracePeriods seconds until it raises a new warning.
+# T1 "grace" periods between DEFCON before re-evaluate.
 EARGMGracePeriods=3
-# Verbose level
+# Verbosity
 EARGMVerbose=1
-# When set to 1, the output is saved in '$EAR_TMP'/eargmd.log (common configuration) as a log file.
+# When set to 1, the output is saved at 'TmpDir'/eargmd.log (common configuration) as a log file.
 EARGMUseLog=1
-# Format for action is: command_name energy_T1 energy_T2  energy_limit T2 T1  units "
-# This action is automatically executed at each warning level (only once per grace periods)
+# Format for action is: "command_name energy_T1 energy_T2 energy_limit T2 T1  units"
+# This action is automatically executed at each warning level (only once per grace periods).
 EARGMEnergyAction=no_action
-#### POWERCAP definition for EARGM: Powercap is still under development. Do not activate
-# 0 means no powercap
-EARGMPowerLimit=0
-```
 
-### Common configuration
+# Period at which the powercap thread is activated.
+EARGMPowerPeriod=120
+# 1 means automatic, 0 is only monitoring.
+EARGMPowerCapMode=1
+# Admins can specify to automatically execute a command in
+# EARGMPowerCapSuspendAction when total_power >= EARGMPowerLimit*EARGMPowerCapResumeLimit/100
+EARGMPowerCapSuspendLimit=90
+# Format for action is: command_name current_power current_limit total_idle_nodes total_idle_power 
+EARGMPowerCapSuspendAction=no_action
+# Admins can specify to automatically execute a command in EARGMPowerCapResumeAction
+# to undo EARGMPowerCapSuspendAction when total_power >= EARGMPowerLimit*EARGMPowerCapResumeLimit/100.
+# Note that this will only be executed if a suspend action was executed previously.
+EARGMPowerCapResumeLimit=40
+# Format for action is: command_name current_power current_limit total_idle_nodes total_idle_power 
+EARGMPowerCapResumeAction=no_action
+
+# EARGMs must be specified with a unique id, their node and the port that receives
+# remote connections. An EARGM can also act as meta-eargm if the meta field is filled,
+# and it will control the EARGMs whose ids are in said field. If two EARGMs are in the
+# same node, setting the EARGMID environment variable overrides the node field and
+# chooses the characteristics of the EARGM with the correspoding id. If energy is
+# set to 0, cluster_energy_cap will be disabled for that EARGM. Currently, only 1
+# cluster_energy_cap is supported.
+EARGMId=1 energy=1800 power=600 node=node1 port=50100 meta=1,2,3
+EARGMId=2 energy=0 power=500 node=node1 port=50101
+EARGMId=3 energy=0 power=500 node=node2 port=50100
+```
 
-```INI
+## Common configuration
 
+```
 # Default verbose level
 Verbose=0
-# Path used for communication files, shared memory, etc. It must be PRIVATE per compute node and with read/write permissions. $EAR_TMP
+# Path used for communication files, shared memory, etc. It must be PRIVATE per
+# compute node and with read/write permissions. $EAR_TMP
 TmpDir=/tmp/ear
 # Path where coefficients and configuration are stored. It must be readable in all compute nodes. $EAR_ETC
 EtcDir=/path/to/etc
 InstDir=/path/to/inst
-# Path where metrics are generated in text files when no database is installed. A suffix is included.
-DataBasePathName=/etc/ear/dbs/dbs.
-# Energy reading plugin (without the extension). Allows to use different system components to read the energy of the node. In this case, this plugin reads the energy of the system using Intel Node Manager.
-# look at /path/to/inst/lib/plugins/energy folder to see the list of installed energy plugins
-Energy_plugin=energy_nm.so
-# Power model plugin (without the extension). The power model plugin is used to predict the power and energy consumption of the next iteration of the executing application.
-Energy_model=avx512_model.so
+
+# Network extension: To be used in case the DC has more than one
+# network and a special extension needs to be used for global commands
+#NetworkExtension=
 ```
 
-### EAR-Authorized users/groups/accounts
+## EAR Authorized users/groups/accounts
 
-Authorized users that are allowed to change policies, thresholds and frequencies are supposed to be administrators. A list of users, Linux groups, and/or SLURM accounts can be provided to allow normal users to perform that actions. Only normal Authorized users can execute the learning phase.
+Authorized users that are allowed to change policies, thresholds and frequencies are supposed to be administrators.
+A list of users, Linux groups, and/or SLURM accounts can be provided to allow normal users to perform that actions. Only normal Authorized users can execute the learning phase.
 
-```INI
+```
 AuthorizedUsers=user1,user2
 AuthorizedAccounts=acc1,acc2,acc3
 AuthorizedGroups=xx,yy
 ```
-### Energy tags
+## Energy tags
 
-Energy tags are pre-defined configurations for some applications (EAR library is not loaded). This energy tags accept a user ids, groups and SLURM accounts of users allowed to use that tag.
+Energy tags are pre-defined configurations for some applications (EAR Library is not loaded).
+This energy tags accept a user ids, groups and SLURM accounts of users allowed to use that tag.
 
-```INI
+```
 # General energy tag
 EnergyTag=cpu-intensive pstate=1
 # Energy tag with limited users
 EnergyTag=memory-intensive pstate=4 users=user1,user2 groups=group1,group2 accounts=acc1,acc2
 ```
 
-
-
-### Tags
-Tags are used for architectural descriptions. Max. AVX frequencies are used in predictor models and are SKU-specific. At least a default tag is mandatory to be included for a cluster to work properly.
-
-The `min_power`, `max_power` and `error_power` are threshold values that determine if the metrics read might be invalid, and a warning message to syslog will be reported if the values are outside of said thresholds. `error_power` is a more extreme value that if a metric surpasses it, said metric will not be reported to database.
-
-A special energy plugin or energy model can be specified in a tag that will override the global values previously defined in all nodes that have this tag associated with them.
+## Tags
+Tags are used for architectural descriptions. Max. AVX frequencies are used in predictor models and are SKU-specific.
+At least a default tag is mandatory to be included for a cluster to properly work.
+
+The **min_power**, **max_power** and **error_power** are threshold values that determine if the metrics read might be invalid, and a warning message to syslog will be reported if the values are outside of said thresholds. The **error_power** field is a more extreme value that if a metric surpasses it, said metric will not be reported to the DataBase.
+
+A special energy plug-in or energy model can be specified in a tag that will override the global values previously defined in all nodes that have this tag associated with them.
+
+Powercap set to 0 means powercap is disabled and cannot be enabled at runtime.
+Powercap set to 1 means no limits on power consumption but a powercap can be set without stopping eard.
+List of accepted options:
+- max_avx512            (GHz)
+- max_avx2              (GHz)
+- max_power             (W)
+- min_power             (W)
+- error_power           (W)
+- coeffs                (filename)
+- powercap              (W)
+- powercap_plugin       (filename)
+- energy_plugin         (filename)
+- gpu\_powercap\_plugin (filename)
+- max_powercap          (W)
+- gpu\_def\_freq        (GHz)
+- cpu\_max\_pstate      (0..max_pstate)
+- imc\_max\_pstate      (0..max\_imc\_pstate)
+- energy\_model         (filename)
  
-```INI
-Tag=6148 default=yes max_avx512=2.2 max_avx2=2.6 max_power=500 min_power=50 error_power=600 coeffs=coeffs.default 
+```
+Tag=6148 default=yes max_avx512=2.2 max_avx2=2.6 max_power=500 powercap=1 max_powercap=600 gpu_def_freq=1.4 energy_model=avx512_model.so energy_plugin=energy_nm.so powercap_plugin=dvfs.so gpu_powercap_plugin=gpu.so min_power=50 error_power=600 coeffs=coeffs.default
 Tag=6126 max_avx512=2.3 max_avx2=2.9 ceffs=coeffs.6126.default max_power=600 error_power=700
 ```
 
-### Power policies plugins
+## Power policies plug-ins
 
-```INI
-#---------------------------------------------------------------------------------------------------
-## Power policies
-## ---------------------------------------------------------------------------------------------------
-#
-## policy names must be exactly file names for policies installeled in the system
+```
+# Policy names must be exactly file names for policies installeled in the system.
 DefaultPowerPolicy=monitoring
 Policy=monitoring Settings=0 DefaultFreq=2.4 Privileged=0
 Policy=min_time Settings=0.7 DefaultFreq=2.0 Privileged=0
 Policy=min_energy Settings=0.05 DefaultFreq=2.4 Privileged=1
 
-# For homogeneous systems, default frequencies can be easily specified using freqs, for heterogeneous systems it is preferred to use pstates 
+# For homogeneous systems, default frequencies can be easily specified using freqs.
+# For heterogeneous systems it is preferred to use pstates. 
 
-# Example with pstates (lower pstates corresponds with higher frequencies). Pstate=1 is nominal and 0 is turbo
+# Example with pstates (lower pstates corresponds with higher frequencies).
+# Pstate=1 is nominal and 0 is turbo
 #Policy=monitoring Settings=0 DefaultPstate=1 Privileged=0
 #Policy=min_time Settings=0.7 DefaultPstate=4 Privileged=0
 #Policy=min_energy Settings=0.05 DefaultPstate=1 Privileged=1
 
-
+# Tags can be also used with policies for specific configurations
+#Policy=monitoring Settings=0 DefaultFreq=2.6 Privileged=0 tag=6126
 ```
 
+## Island description
 
-### Island description
-
-This section is mandatory since it is used for cluster description. Normally nodes are grouped in islands that share the same hardware characteristics as well as its database managers (EARDBDS). Each entry describes part of an island, and every node must be in an island.
+This section is mandatory since it is used for cluster description. Normally nodes
+are grouped in islands that share the same hardware characteristics as well as its
+database managers (EARDBDS). Each entry describes part of an island, and every node must be in an island.
 
-There are two kinds of database daemons. One called 'server' and other one called 'mirror'. Both performs the metrics buffering process, but just one performs the insert. The mirror will do that insert in case the 'server' process crashes or the node fails.
+There are two kinds of database daemons. One called **server** and other one called **mirror**.
+Both perform the metrics buffering process, but just one performs the insert.
+The mirror will do that insert in case the 'server' process crashes or the node fails.
 
-It is recommended for all islands to have maintain server-mirror symmetry. For example, if the island I0 and I1 have the server N0 and the mirror N1, the next island would have to point the same N0 and N1 or point to new ones N2 and N3, not point to N1 as server and N0 as mirror.
+It is recommended for all islands to maintain server-mirror symmetry. For example, if the island I0 and I1 have the server N0 and the mirror N1, the next island would have to point the same N0 and N1 or point to new ones N2 and N3, not point to N1 as server and N0 as mirror.
 
 Multiple EARDBDs are supported in the same island, so more than one line per island is required, but the condition of symmetry have to be met.
 
@@ -238,20 +320,23 @@ It is recommended that for an island the server and the mirror to be running in
 
 A tag can be specified that will apply to all the nodes in that line. If no tag is defined, the default one will be used as hardware definition.
 
-```INI
-#
-# In the following example the nodes are clustered in two different islands, but the Island 1 have
-# two types of EARDBDs configurations. 
-#
+Finally, if an EARGM is being used to cap power, the EARGMID field is necessary in at least one line, and will specify what EARGM controls the nodes declared in that line. If no EARGMID is found in a line, the first one found will be used (ie, the previous line EARGMID).
 
-Island=0 DBIP=node1081 DBSECIP=node1082 Nodes=node10[01-80]
+
+```
+# In the following example the nodes are clustered in two different islands,
+# but the Island 1 have two types of EARDBDs configurations.
+
+Island=0 DBIP=node1081 DBSECIP=node1082 Nodes=node10[01-80] EARGMID=1
 
 # These nodes are in island0 using different DB connections and with a different architecture
+
 Island=0 DBIP=node1084 DBSECIP=node1085 Nodes=node11[01-80] DBSECIP=node1085 tag=6126
+
 # These nodes are in island0 and will use default values for DB connection (line 0 for island0) and default tag
+#These nodes will use the same EARGMID as the previous ones
 Island=0 Nodes=node12[01-80]
 
-
 # Will use default tag 
 Island=1 DBIP=node1181 DBSECIP=node1182 Nodes=node11[01-80]
 ```
@@ -270,9 +355,9 @@ Detailed island accepted values:
         - Island=1 Nodes=`node[1-3],node4`
 
 
-SLURM spank plugin configuration file
-------------------------
-SLURM loads the plugin through a file called `plugstack.conf`, which is composed by a list of a plugins. In the file `etc/slurm/ear.plugstack.conf`, there is an example entry with the paths already set to the plugin, temporal and configuration paths.
+# SLURM SPANK plug-in configuration file
+
+SLURM loads the plug-in through a file called `plugstack.conf`, which is composed by a list of a plug-ins. In the file `etc/slurm/ear.plugstack.conf`, there is an example entry with the paths already set to the plug-in, temporal and configuration paths.
 
 __Example__:
 ```
@@ -281,15 +366,15 @@ required ear_install_path/lib/earplug.so  prefix=ear_install_path sysconfdir=etc
 
 The argument `prefix` points to the EAR installation path and it is used to load the library using `LD_PRELOAD` mechanism. Also the `localstatedir` is used to contact with the EARD, which by default points the path you set during the `./configure` using `--localstatedir` or `EAR_TMP` arguments. Next to these fields, there is the field `earlib_default=off`, which means that by default EARL is not loaded. Finally there are `eargmd_host` and `eargmd_port` if you plan to connect with the EARGMD component (you can leave this empty).
 
-Also, there are two additional arguments. The first one, `nodes_allowed=` followed by a comma separated list of nodes, enables the plugin only in that nodes. The second, `nodes_excluded=`, also followed by a comma separated list of nodes, disables the plugin only in nodes in the list. These are arguments for very specific configurations that must be used with caution, if they are not used it is better that they are not written.
+Also, there are two additional arguments. The first one, `nodes_allowed=` followed by a comma separated list of nodes, enables the plug-in only in that nodes. The second, `nodes_excluded=`, also followed by a comma separated list of nodes, disables the plug-in only in nodes in the list. These are arguments for very specific configurations that must be used with caution, if they are not used it is better that they are not written.
 
 __Example__:
 ```
 required ear_install_path/lib/earplug.so  prefix=ear_install_path sysconfdir=etc_ear_path localstatedir=tmp_ear_path earlib_default=off nodes_excluded=node01,node02
 ```
 
-MySQL/PostgreSQL
------
+# MySQL/PostgreSQL
+
 **WARNING**: If any EAR component is running in the same machine as the MySQL server some connection problems might occur. This will not happen with PostgreSQL. To solve those issues, input into MySQL's CLI client the `CREATE USER` and `GRANT PRIVILEGES` queries from `edb_create -o` changing the portion `'user_name'@'%'` to `'user_name'@'localhost'` so that EAR's users have access to the server from the local machine.
 There are two ways to configure a database server for EAR's usage.
 - run `edb_create -r` located in `$EAR_INSTALLATION_PATH/sbin` from a node with root access to the MySQL server. This requires MySQL/PostgreSQL's section of ear.conf to be correctly written. For more info run `edb_create -h`.
@@ -297,7 +382,17 @@ There are two ways to configure a database server for EAR's usage.
 
 For more information about how each `ear.conf` flag changes the database creation, see our [Database section](EAR-Database).
 
+# MSR Safe
+
+MSR Safe is a kernel module that allows to read and write MSR without root permission. EAR opens MSR Safe files if the ordinary MSR files fail. MSR Safe requires a configuration file to allow read and write registers. You can find configuration files in `etc/msr_safe` for Intel Skylake and superior and AMD Zen and superior.
+
+You can pass these configuration files to MSR Safe kernel mode like this:
+```
+cat intel63 > /dev/cpu/msr_allowlist
+```
+
+You can find more information in the [official repository]( https://github.com/LLNL/msr-safe)
+
+# Next step
 
-Next step
----------
+Visit the [execution page](Starting services) to run EAR's different components.
\ No newline at end of file
-Visit the [execution page](Execution) to run EAR's different components.