Commit 6489af67 authored by gsaxena's avatar gsaxena

Merge branch 'master' of https://gitlab.bsc.es/gsaxena/biofvm_x

The README file updated (1) Text improvement Gaurav (2) Markdown Arnau
parents 13c1e947 3f9367f0
This diff is collapsed.
# plot_microenvironment.py - plot results from
# Code adapted from plot_microenvironment.py, originally build to plot results from
# http://www.mathcancer.org/blog/biofvm-warmup-2d-continuum-simulation-of-tumor-growth/
import sys
import os.path
......
####################################
BioFVM_X is based on BioFVM v 1.1.6.
BioFVM-X (based on BioFVM v 1.1.6)
####################################
+ BioFVM (http://BioFVM.MathCancer.org) supports only on-node shared memory
......@@ -14,18 +14,22 @@ BioFVM_X is based on BioFVM v 1.1.6.
+ Most of the MPI-based parallel functions are made available in
BioFVM_parallel.cpp
+ A Makefile is given and a sample example in ./examples/tutorial1_BioFVM.cpp
=============================================================================
Running the example tutorial1_BioFVM.cpp in ./GS_BioFVM_X/GS_BioFVM/examples/
=============================================================================
+ A "general" Makefile is given in ./Gs_BioFVM_X/GS_BioFVM/ and a sample (parallelized) example in ./examples/tutorial1_BioFVM.cpp
is completely parallelized using MPI + OpenMP.
+ Procedure to run:
(1) git clone the repository
(2) Change to the top level directory i.e. $ cd GS_BioFVM_X/GS_BioFVM
(3) Make sure GCC 8.1 (or other version) and OpenMPI 3.1.1 (or other version)
are present (or the modules are loaded in a module environment)
(4) Execute $ make tutorial1
This results in an executable file "tutorial1" in the directory ./examples
(3) Make sure GCC 8.1 (or any other version) and OpenMPI 3.1.1 (or any other version)
are present (or the modules are loaded in a module environment if you have a module environment)
(4) Execute $ make tutorial1 from ./GS_BioFVM_X/GS_BioFVM/
This results in an executable file "tutorial1" in the directory ./GS_BioFVM_X/GS_BioFVM/examples
Example output:
Example output (of compilation):
bsc99102@login1:~/GS_BioFVM_X/GS_BioFVM> make tutorial1
mpic++ -march=native -O2 -fomit-frame-pointer -mfpmath=both -fopenmp -m64 -std=c++11 -g -c BioFVM_vector.cpp
......@@ -49,8 +53,7 @@ bsc99102@login1:~/GS_BioFVM_X/GS_BioFVM> ls -lrt ./examples | grep tutorial1
-rw-r--r-- 1 bsc99102 bsc99 34502 Feb 14 13:46 tutorial1_BioFVM.cpp
-rwxr-xr-x 1 bsc99102 bsc99 6991728 Apr 16 16:03 tutorial1
(5) For the SLURM scheduler, the following script can be used
(script_tuturial1.sh):
(5) For the SLURM scheduler, the following script can be used (script_tuturial1.sh):
============================================================================
#!/bin/bash
......@@ -71,34 +74,34 @@ mpiexec --map-by ppr:1:socket:pe=24 --report-bindings ./examples/tutorial1
This script :
(a) uses 1 single node
(b) spawns 2 MPI processes in that node (Since we have 2 sockets per node, we
(a) uses 1 single node (--nodes=1)
(b) spawns 2 MPI processes (--ntasks-per-node=2) in that node (Since we have 2 sockets per node, we
create 2 MPI processes i.e. 1 MPI process per socket).
(c) create 24 threads per MPI process (Since we have 24 cores in 1 socket,
(c) create 24 threads (--cpus-per-task=24) per MPI process (Since we have 24 cores in 1 socket,
we create 24 threads i.e. 1 thread per core).
IMPORTANT:
(a) Please make sure that the dimension of the Physical domain
(a) PLEASE MAKE SURE that the dimension of the Physical domain
(specified in ./examples/tutorial1_BioFVM.cpp)
is completely divisible by the Voxel dimensions (given by
'mesh_resolution' in ./examples/tutorial1_BioFVM.cpp).
(b) Please make sure that the number of Voxels in the X direction are perfectly
divisible by the total number of processes.
(b) PLEASE MAKE SURE that the number of Voxels in the X direction are PERFECTLY
divisible by the total number of MPI processes.
For the given example: N=1000, minX=0, maxX=N, mesh_resolution=10. This gives
total Voxels in X (or Y or Z) direction as (maxX-minX)/mesh_resolution =
total Voxels in X direction as = (maxX-minX)/mesh_resolution =
(1000 - 0)/10 = 100 Voxels. Now the total number of Voxels in the X-direction
must be perfectly divisible by total number of MPI processes (in this case P=2).
Thus, we have 100/2 = 50 Voxels per process.
must be PERFECTLY divisible by the total number of MPI processes (in this case
total MPI processes P=2). Thus, we have 100/2 = 50 Voxels per process in the x-direction.
(6) Submit the batch script using: $sbatch script_tutorial1.sh
(6) Submit the batch script to SLURM using: $sbatch script_tutorial1.sh
(7) After the simulation completes, two text files are produced. One is the
(7) After the simulation completes, two text files are (generally) produced. One is the
error file that contains the bindings of MPI processes and threads:
For example:
For example, the contents of the error file are:
========================================================================================================================================================
[s05r2b10:340036] MCW rank 0 bound to socket 0[core 0[hwt 0]], socket 0[core 1[hwt 0]], socket 0[core 2[hwt 0]], socket 0[core 3[hwt 0]], socket 0[core 4[hwt 0]], socket 0[core 5[hwt 0]], socket 0[core 6[hwt 0]], socket 0[core 7[hwt 0]], socket 0[core 8[hwt 0]], socket 0[core 9[hwt 0]], socket 0[core 10[hwt 0]], socket 0[core 11[hwt 0]], socket 0[core 12[hwt 0]], socket 0[core 13[hwt 0]], socket 0[core 14[hwt 0]], socket 0[core 15[hwt 0]], socket 0[core 16[hwt 0]], socket 0[core 17[hwt 0]], socket 0[core 18[hwt 0]], socket 0[core 19[hwt 0]], socket 0[core 20[hwt 0]], socket 0[core 21[hwt 0]], socket 0[core 22[hwt 0]], socket 0[core 23[hwt 0]]: [B/B/B/B/B/B/B/B/B/B/B/B/B/B/B/B/B/B/B/B/B/B/B/B][./././././././././././././././././././././././.]
......@@ -141,7 +144,7 @@ OPENMP DISPLAY ENVIRONMENT END
and the second file is that of the output.
For example:
For example, the contents of the output file are:
========================================================
TIME FOR RESIZING MICROENVIRONMENT = 1.0491
TIME FOR GENERATING GAUSSIAN PROFILE = 0.011721
......@@ -152,10 +155,13 @@ TIME FOR WRITING FINAL FILE = 0.170324
TOTAL PROGRAM EXECUTION TIME = 3.99776
========================================================
(7) The simulation can produce .mat files as outputs. We produce at least 2 .mat files,
one at the beginning and 1 at the end of the simulation. The
(7) The simulation can/will produce .mat files (MATLAB files) as outputs.
We produce at least 2 .mat files,
one at the beginning and the second at the end of the simulation. The
location where the files are produced can be controlled from the
./examples/tutorial1_BioFVM.cpp file.
./examples/tutorial1_BioFVM.cpp file (see below the relevant lines in ./example/tutorial1_BioFVM.cpp).
PLEASE SEE: If there are multiple calls to the function write_to_matlab(...), then ALL the calls must contain the path at which
the user wants to store/produce the .mat files.
For example:
......@@ -166,13 +172,14 @@ microenvironment.write_to_matlab( "/gpfs/projects/initial_concentration.mat", mp
...
=================================================================================================================
(8) This .mat file can be plotted using the Matlab software (or Octave -
as in our case). The .m help scripts for plotting can be found in ./GS_BioFVM_X/
GS_BioFVM/matlab. Any of the scripts can be tried; we plotted using
(8) This .mat file can be plotted using the MATLAB software (or Octave -
as in our case). The .m help scripts for plotting can be found in ./GS_BioFVM_X/GS_BioFVM/matlab.
Any of the scripts can be tried; we plotted using
(a) cross_section_colormap.m
(b) cross_section_surface.m
For example, inside (a) above, change file name from
For example, inside the MATLAB script (a) above, change file name from
===================================================
file_name='output_4.000000_0.000100_10.000000.mat';
===================================================
......@@ -180,16 +187,36 @@ to
===================================================
file_name='initial_concentration.mat';
===================================================
and then execute the script.
and then execute the script. To execute the .m script using Octave, see below.
In order to load Octave, we needed to load the following modules as well:
In order to load Octave, we needed to load the following modules (on Marenostrum MN4):
1) gcc/7.2.0 2) impi/2018.2 3) mkl/2018.2 4) suitesparse/5.2.0
5) octave/4.2.2
Save the script file above and run.
If everything goes well the 2-D colormap or the 3-D surface should appear !
If everything goes well the "2-D colormap" or the "3-D surface" should appear !
=============================================================================================
Running the tumor growth example : 3D_tumor_example_X.cpp in ./GS_BioFVM_X/GS_BioFVM/examples
=============================================================================================
Steps:
(1) From the ./GS_BioFVM_X/GS_BioFVM/ directory, execute $ make 3DtumorX
This should produce an executable named 3DtumorX in ./GS_BioFVM_X/GS_BioFVM/examples/ directory.
(2) A help script named script_3DtumorX.sh is given in ./GS_BioFVM_X/GS_BioFVM/ directory and to submit
to SLURM, execute $ sbatch script_3DtumorX.sh
As in the previous example, 2 MPI processes with 24 threads each are spawned to execute 3DtumorX.
(3) After the execution, there will be multiple .mat files created in ./GS_BioFVM_X/GS_BioFVM/
These .mat files can be plotted using the plotting_microenv_tumor.py Python script
in ./GS_BioFVM_X/GS_BioFVM/matlab directory.
The plotting_microenv_tumor.py can plot the microenvironment and we describe this below in detail.
(4) Running the plotting_microenv_tumor.py script:
####################################################
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment