Analyze-the-results.md

@@ -17,6 +17,8 @@ If you click on the build item, you will access the build page. For each build,
 
 ## Tests
 
+### Test main page
+
 If you click on the test item, you will access the test page. For each test, you'll see an item with its name, its authors, a brief description, the alya modules it requires, the post-process tool (`alya2pos`, `mpio2txt` or `none`), if the job has been relaunched one or several times because of a possible system failure (retry), and the global status of this job (showing the errors that happened during this test execution with each of the build and mpi/openmp combination).
 
 Here, the list of statuses:
@@ -39,33 +41,43 @@ Here, the list of statuses:
 
 A test is considered failed if it has not the OK status.
 
-As for the compilation, you can get more insight into the test results by clicking on the corresponding red cross.
-
-![Capture_d_écran_de_2019-07-23_17-45-01](uploads/3a53a3c63f1303ed767f3c607f67e313/Capture_d_écran_de_2019-07-23_17-45-01.png)
-_Tests page_
-
-There are two kinds of failed tests:
-
-### Hard Failure
-
-It is shown by a red cross. it means that:
-
-*   execution did not occur correctly,
-*   result files have not been generated,
-*   result files content do not match reference files,
-*   etc.
-
-In this case, click on the failed tests until you get more details about the issue.
-
-![Capture_d_écran_de_2019-07-23_17-42-38](uploads/a8d37c5e8d37639165e9ce3f5ad89320/Capture_d_écran_de_2019-07-23_17-42-38.png)
-_This page shows the error type. You can investigate by looking to the corresponding alya, alya2pos and mpio2txt outputs._
-
-### Tolerance Issue
-
-It is shown by an orange cross.
-
-This problem is related to the results generated by the execution of Alya. Their value is different from the reference result file values enough to consider that the behavior of Alya is incorrect.
-You have two options: tune the tolerance if you consider that it is too strict, or modify Alya such that the results are correct. See the https://gitlab.bsc.es/alya/alya-testsuite/wikis/Manage-the-tolerances page to know how to manage the tolerances.
-
-
-
+![Capture_d_écran_de_2020-03-30_18-10-45](uploads/de1f8fff9b41a53cb29ddf206019eae9/Capture_d_écran_de_2020-03-30_18-10-45.png)
+*Example of test main page with two failed tests*
+
+### Test page
+
+If you click on one test symbol, you will access its test page.
+You'll have details on
+*  The build
+*  MPI number
+*  OpenMP number
+*  Json file of the test
+*  Job content
+*  Main output (out/err)
+*  Alya output (out/err)
+*  Postprocess output (out/err)
+*  Job Id
+*  Time of each step (Init, Alya, Postprocess, Validation, Total)
+*  Number of retries
+*  Status
+
+![Capture_d_écran_de_2020-03-30_18-12-09](uploads/d5b6a993faf1eb5e6d42318b4d8f86cf/Capture_d_écran_de_2020-03-30_18-12-09.png)
+*Example of test page with failed test*
+
+### Detail
+
+Depending on the type of error, you may have access to more details.
+For each file is shown:
+*  The name
+*  The comparison method (`absolute`, `relative`, `power`, `diff`) defined in the json file
+*  The tolerance value defined in the json file
+*  The maximum absolute difference
+*  The maximum relative difference
+*  The maximum power difference
+*  The number of differences
+*  The total of compared elements
+*  The type of error
+*  The status: <img src="uploads/397a6076cb9ba95116a9a578f5a67397/ko.png" width="20"> = general error or <img src="uploads/fb4df6e048544dd43f1d2dd8c0d9f486/tol.png" width="20"> = tolerance error
+
+![Capture_d_écran_de_2020-03-30_18-15-48](uploads/ad932eb2119ad11d6fbb931deb03127d/Capture_d_écran_de_2020-03-30_18-15-48.png)
+*Example of detail page: in this example, base and result files of `coh_mshDiv_3d-BOSET-00000000.post.mpio.txt` and `coh_mshDiv_3d-BOCOD-00000000.post.mpio.txt` don't have the same number of lines and thus cannot be compared*