| ... | @@ -44,14 +44,12 @@ There are some steps to follow for achieving the integration of a your applicati |
... | @@ -44,14 +44,12 @@ There are some steps to follow for achieving the integration of a your applicati |
|
|
## How to bring a new tool?
|
|
## How to bring a new tool?
|
|
|
Essentially, VRE will need two elements, (1) your application or workflow wrapped within a **VRE RUNNER**, and (2) **metadata** annotating it (*i.e.* input files requirements, descriptions). The following steps describe how to achieve it.
|
|
Essentially, VRE will need two elements, (1) your application or workflow wrapped within a **VRE RUNNER**, and (2) **metadata** annotating it (*i.e.* input files requirements, descriptions). The following steps describe how to achieve it.
|
|
|
|
|
|
|
|
- [Step 1](#step-1-pre-define-your-tool-to-build-a-test-set-of-VRE-job-execution-files) Define your tool's input and output files. The information is used to build job execution files for testing the RUNNER.
|
|
- [Step 1](#step-1-pre-define-your-tool-to-build-a-test-set-of-VRE-job-execution-files) Define your tool's input and output files. Build job execution files for testing the RUNNER.
|
|
|
- [Step 2](#step-2-prepare-a-VRE-RUNNER-wrapping-your-application) Prepare a new VRE RUNNER wrapping your application.
|
|
- [Step 2](#step-2-prepare-a-VRE-RUNNER-wrapping-your-application) Prepare a new VRE RUNNER wrapping your application.
|
|
|
- [Step 3](#step-3-annotate-and-submit-the-new-VRE-tool) Annotate and submit the new VRE tool
|
|
- [Step 3](#step-3-annotate-and-submit-the-new-VRE-tool) Annotate and submit the new VRE tool
|
|
|
- [Step 4](#step-4) Submit the RUNNER code and the tool-specification file to VRE administrators, who will install and register the new tool.
|
|
- [Step 4](#step-4-test-and-debug-the-new-tool-from-the-VRE) Test and debug the new tool from the VRE user interface
|
|
|
- [Step 5](#step-5) Test and debug the new tool from the VRE user interface
|
|
- [Step 5](#step-5-prepare-a-web-page-to-display-a-summary-report-on-each-execution) (optional) Prepare a web page to display a summary report on each execution
|
|
|
- [Step 6](#step-6) (optional) Prepare a web page to display a summary report on each execution
|
|
- [Step 6](#step-6-provide-documentation-for-the-new-tool) Provide documentation for the new tool
|
|
|
- [Step 7](#step-7) Provide documentation for the new tool
|
|
|
|
|
|
|
|
|
|
<br/>
|
|
<br/>
|
|
|
|
|
|
|
|
#### STEP 1: Pre-define your tool to build a test set of VRE job execution files
|
|
#### STEP 1: Pre-define your tool to build a test set of VRE job execution files
|
| ... | @@ -101,6 +99,9 @@ For preparing the RUNNER, the easiest option is to **take as reference a `RUNNER |
... | @@ -101,6 +99,9 @@ For preparing the RUNNER, the easiest option is to **take as reference a `RUNNER |
|
|
**Step-by-step**
|
|
**Step-by-step**
|
|
|
|
|
|
|
|
1. **Fork or clone** the repository of the RUNNER template in your local development environment.
|
|
1. **Fork or clone** the repository of the RUNNER template in your local development environment.
|
|
|
|
| RUNNER template repository | |
|
|
|
|
| ------------- | ------------- |
|
|
|
|
| https://github.com/inab/vre_template_tool | [documentation](https://vre-template-tool.readthedocs.io/en/latest/reference/classes.html) |
|
|
|
|
|
|
|
|
2. (optional) Run the `hello_word` example. The RUNNER template is initially configured to "wrap" an application called `hello.py`. It demonstrates the overall flow of a VRE RUNNER.
|
|
2. (optional) Run the `hello_word` example. The RUNNER template is initially configured to "wrap" an application called `hello.py`. It demonstrates the overall flow of a VRE RUNNER.
|
|
|
- How to: https://github.com/inab/vre_template_tool#run-the-wrapper
|
|
- How to: https://github.com/inab/vre_template_tool#run-the-wrapper
|
| ... | @@ -110,11 +111,6 @@ For preparing the RUNNER, the easiest option is to **take as reference a `RUNNER |
... | @@ -110,11 +111,6 @@ For preparing the RUNNER, the easiest option is to **take as reference a `RUNNER |
|
|
|
|
|
|
|
4. **Implement the `run` method of the `VRE_Tool`** so that the function executes the application, module or pipeline to be integrated. The input file locations and argument values as defined in the job execution files are going to be the content of parameters received in the `run` method.
|
|
4. **Implement the `run` method of the `VRE_Tool`** so that the function executes the application, module or pipeline to be integrated. The input file locations and argument values as defined in the job execution files are going to be the content of parameters received in the `run` method.
|
|
|
|
|
|
|
|
| RUNNER template repository | |
|
|
|
|
|
| ------------- | ------------- |
|
|
|
|
|
| https://github.com/inab/vre_template_tool | [documentation](https://vre-template-tool.readthedocs.io/en/latest/reference/classes.html) |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
5. The RUNNER will be ready when the wrapped application is properly executed and the output files are generated at the location specified in `output_files[].file.path`. These paths are usually defined in `config.json` file.
|
|
5. The RUNNER will be ready when the wrapped application is properly executed and the output files are generated at the location specified in `output_files[].file.path`. These paths are usually defined in `config.json` file.
|
|
|
Alternatively, if the name and number of the output files cannot be known before the execution, you should extend the `VRE_Tool.run` method to **define the `file.path` attribute** into the `output_files` dictionary. The RUNNER will write down it into the out-files metadata file (*i.e.* `out_metadata.json`).
|
|
Alternatively, if the name and number of the output files cannot be known before the execution, you should extend the `VRE_Tool.run` method to **define the `file.path` attribute** into the `output_files` dictionary. The RUNNER will write down it into the out-files metadata file (*i.e.* `out_metadata.json`).
|
|
|
|
|
|
| ... | @@ -126,11 +122,11 @@ For preparing the RUNNER, the easiest option is to **take as reference a `RUNNER |
... | @@ -126,11 +122,11 @@ For preparing the RUNNER, the easiest option is to **take as reference a `RUNNER |
|
|
|
|
|
|
|
#### STEP 3: Annotate and submit the new VRE tool
|
|
#### STEP 3: Annotate and submit the new VRE tool
|
|
|
|
|
|
|
|
Once the RUNNER is successfully executing the application in your local development environment, it is time to ask for registering the new tool to the corresponding VRE server. To do so, some descriptive metadata on the new application is required, *i.e.*, tool descriptions and titles, ownership, references, keywords, etc.
|
|
Once the RUNNER is successfully executing the application in your local development environment, it is time to ask for **registering the new tool** to the corresponding VRE server. To do so, some descriptive metadata on the new application is required, *i.e.*, tool descriptions and titles, ownership, references, keywords, etc.
|
|
|
|
|
|
|
|
Again, two approaches are supported:
|
|
Again, two approaches are supported:
|
|
|
|
|
|
|
|
- Manual approach:
|
|
- **Manual approach**:
|
|
|
|
|
|
|
|
Generate the `tool specification file` taking as reference some examples to fully annotate the new tool
|
|
Generate the `tool specification file` taking as reference some examples to fully annotate the new tool
|
|
|
- JSON schemes:
|
|
- JSON schemes:
|
| ... | @@ -138,7 +134,32 @@ Again, two approaches are supported: |
... | @@ -138,7 +134,32 @@ Again, two approaches are supported: |
|
|
- Examples:
|
|
- Examples:
|
|
|
- dpfrep RUNNER (example of a R-based tool): [tool_specification.json](https://github.com/inab/vre_dpfrep_executor/tree/master/tool_specification)
|
|
- dpfrep RUNNER (example of a R-based tool): [tool_specification.json](https://github.com/inab/vre_dpfrep_executor/tree/master/tool_specification)
|
|
|
|
|
|
|
|
Save your tool specification file in your repository and send it all together to
|
|
Save your tool specification file in your repository and send it all together to VRE administrators. They will validate the data and register the tool the the VRE.
|
|
|
|
|
|
|
|
- **VRE web interface approach**:
|
|
|
|
|
|
|
|
Go to the tools' developer administration panel and fill in the missing information for the tool entry generated in *STEP 1* when preparing the job execution files.
|
|
|
|
- Where:
|
|
|
|
1. in the left navigation menu, Admin → My Tools → Development →. Find the row corresponding to your tool in preparation.
|
|
|
|
2. Fill in the two last columns:
|
|
|
|
- `bring us your code`: URL of the your RUNNER's repository created in *STEP 2*
|
|
|
|
- `Define Tool`: edit online the template JSON document. Find a title for your tool, a description, etc. All this information will be displayed to the user on the web application.
|
|
|
|
3. Send the `Submit` button. It will send an email to VRE administrators, who will validate the data and register the tool to the VRE.
|
|
|
|
|
|
|
|
After approval, the tool will be accessible on the web application in *test mode*, *i.e.*, only tool developers and administrators will be able to find and run the tool at the VRE.
|
|
|
|
|
|
|
|
<br/>
|
|
|
|
|
|
|
|
#### STEP 4: Test and debug the new tool from the VRE
|
|
|
|
|
|
|
|
<br/>
|
|
|
|
|
|
|
|
#### STEP 5: Prepare a custom report viewer for each execution
|
|
|
|
|
|
|
|
<br/>
|
|
|
|
|
|
|
|
#### STEP 6: Provide documentation for the new tool
|
|
|
|
|
|
|
|
- VRE web interface approach
|
|
- Tool logo: minimal resolution of 400px x 400px
|
|
|
Make sure that all output files are generated at the root of the working directory (defined at the argument with key `execution`. Read from `test/config.jso |
|
- Sample datasets
|
|
\ No newline at end of file |
|
- Help pages |
|
|
|
\ No newline at end of file |
