|
|
|
|
|
|
|
|
|
|
|
This figure shows an example of tolerance issues
|
|
|
|
This figure shows an example of tolerance issues. It means that some of the result files contain values that are too far from the reference files values.
|
|
|
|
|
|
|
|
#### Tune the tolerance
|
|
|
|
Generally, it means that the behavior of Alya is wrong and that you should investigate why results are different from what you expected. However, it could also be related to rounding issues generated by a specific compiler or architecture, which leads to small but acceptable differences compared to the reference values. In this case, you may want to adjust your test tolerance so the testsuite does not trigger a failure.
|
|
|
|
|
|
|
|
This operation can be done modifying the `.json` associated with the test (tests are located in the tests directory of the svn directory).
|
|
|
|
In the section `comparisons` of the `.json` file, you will find a list of files that have to be compared.
|
|
|
|
This operation can be done modifying the `.json` associated with the test (see https://gitlab.bsc.es/alya/alya-testsuite/wikis/Contribution-workflow to know how to proceed).
|
|
|
|
In the section `comparisons` of the `.json` file, you will find a list of files are compared.
|
|
|
|
For each file, there is a field "method" that is used to define how to compare the reference and the test result files.
|
|
|
|
|
|
|
|
##### "method": "absolute"
|
| ... | ... | |
