Commit ca62139e authored by Hélène Arduin's avatar Hélène Arduin
Browse files

[Doc] update Tutorials pages

parent 443524eb
......@@ -7,13 +7,11 @@ All @h2 headers will automatically appear in a "Contents" section at the top of
SPACING
Spaces around headers are automatically created, no need to put additional spaces when creating a new section
To facilitate reading, manually add spaces around images (add @br@br before and @br after the image, except when the next element is a header) and code bits (add @br@br before and @br after the code bit).
Spaces around headers are automatically created, no need to put additional spaces when creating a new section, adding an img or a code snippet.
FORMATTING
To create a new line use @br
To create a new paragraph use @br@br
To create a new paragraph use @br
Bold with @b{word}
Italic with @i{word}
Create an unordered list with @ul
......
......@@ -43,12 +43,10 @@ Some of these tutorials are also detailed in the @aa("Tutorials", href := tutori
There are a number of NetLogo examples, to illustrate various OpenMOLE methods.
Two classic NetLogo models are used in these examples: the Fire model and the Ants model.
@br@br
@br
Not all examples are listed here, for a full list look at the GitHub @aa("repository", href := shared.link.repo.market) or in the GUI.
@br@br
@img(src := Resource.img.guiGuide.market.file, width := "95%")
......
......@@ -10,27 +10,18 @@
In this section you will find some tutorials to help you build your first experiments.
Feel free to suggest topics for new tutorials you would find helpful on the @aa("forum", href := shared.link.forum).
@br@br
You can even write your own tutorial and share it with the OpenMOLE community on the forum!
We'll make sure to integrate it with the next release.
@br@br
The available tutorials are the following:
@br@br
@b{Getting Started}
@ul
@h3{Getting Started}
@ul
@li
@a("A Step by Step Introduction to OpenMOLE", href := stepByStepIntro.file)
@li
@a("How to Execute an Exploration Task", href := exploreTuto.file)
@b{NetLogo Tutorials}
@ul
@h3{NetLogo Tutorials}
@ul
@li
@a("Simple Sensitivity Analysis of the Fire NetLogo Model", href := simpleSAFire.file)
@li
......
......@@ -10,7 +10,7 @@
In this tutorial, you will learn how to prepare and run an experiment, in order to explore a simple model.
The toy model is @i{Ants.nlogo}, since we already know how to import and run it from the @a("previous tutorial", href := stepByStepIntro.file).
@br@br
@br
The simplest numerical experiments that can be done with such a stochastic model are:
@ul
......@@ -34,10 +34,10 @@ In order to repeat easily a model execution and concatenate the results, OpenMOL
As the model has a random seed parameter named @code{mySeed}, it can be passed to the @code{Replication} task as an argument.
The task will then attribute seeds itself to each model run (necessary for exact reproducibility).
@br
@b{Warning:} The default value for @code{mySeed} must be removed from the model task, otherwise it will overwrite the seeds generated by the @code{Replication} task.
@br@br
@b{Warning:} The default value for @code{mySeed} must be removed from the model task, otherwise it will overwrite the seeds generated by the @code{Replication} task.
@hl.openmole("""
// Input values
......@@ -91,8 +91,6 @@ val replications = Replication(
replications hook (workDirectory / "replications_results.csv")
""")
@br
You can run this workflow by clicking on the Run button in the GUI.
......@@ -102,11 +100,7 @@ Once the execution is over, refresh your workspace by clicking on the second-to-
A new file will appear named @i{replications_results.csv}, this is the file that our workflow created, containing the replication results!
You can click on it.
@br@br
@img(src := Resource.img.tutorial.repliOutput.file, width := "50%")
@br
@img(src := Resource.img.tutorial.repliOutput.file)
You won't have the same numbers as above, since the random seed won't be the same.
However, you should have the same column names and 100 lines of results: one for each model execution.
......@@ -124,19 +118,21 @@ This is exactly the aim of sensitivity analysis and exploration methods which ar
Let's illustrate a simple experiment of a grid search of the parameter space.
Each input parameter will take a finite number of values within a defined interval.
@br
Rate parameters @code{diffusionRate} and @code{evaporationRate} are between 0 and 100, and let's say we consider ant populations of sizes comprised between 50 and 150.
With a step of 50 for each interval, the complete grid has a size of 27 (3x3x3) parameter sets to explore.
@br@br
@br
We use the @code{DirectSampling} method, which takes as arguments the sampling itself (@i{i.e.} the grid we just defined with the parameter values), and the model to evaluate.
Instead of running one instance of the model for each parameter set of our grid, which would not make much sense because of the model's stochasticity we discussed above, we will replicate each one 100 times with the @code{Replication} task.
This way, statistics can then be computed for each parameter set.
@br
The script is the following:
@br@br
The script is the following:
@hl.openmole("""
// Input values
......@@ -199,8 +195,6 @@ exploration hook (workDirectory / "grid_results.csv")
In the same way we did before, after refreshing your workspace you will find a new file named @i{grid_results.csv}, containing the results of our grid sampling.
You can then use the results to better understand how your model behaves.
@br@br
@img(src := Resource.img.tutorial.gridOutput.file, width := "75%")
......
......@@ -31,8 +31,6 @@ Such a script is called a @i{workflow} and should contain:
After @aa("downloading OpenMOLE", href := download.file + "#InstallOpenMOLE"), launch it by executing the @i{openmole} file in the installation directory with the @code{./openmole} command in a terminal.
Once OpenMOLE is up and running in your browser, you should see the GUI:
@br@br
@img(src := Resource.img.guiGuide.emptyGUI.file, `class` := "uiScreenshot", width := "100%")
......@@ -41,13 +39,13 @@ Once OpenMOLE is up and running in your browser, you should see the GUI:
To illustrate this tutorial, we will be using the Ants model from the NetLogo library.
The model is available @aa("here", href := Resource.script.antsNLogo.file).
@br
Once you've downloaded the model on your computer, you need to import it in OpenMOLE.
There are two ways to do that, either by using the "upload a file" button on the left series of icons to import the @i{.nlogo} file as is, or by using the "New project" button and selecting "Import your model" to open the model wizard.
The model wizard will automatically detect the language in which the file is written, and the input and output parameters of your model:
@br@br
@img(src := Resource.img.tutorial.modelImport.file, width := "100%")
......@@ -57,11 +55,8 @@ The model wizard will automatically detect the language in which the file is wri
@h3{Workflow}
Based on the automatically detected parameters and NetLogo commands, the model wizard creates an OpenMOLE script containing variables for the inputs and outputs, and a basic workflow to run your model.
If you didn't use the model wizard to import the Ants model, you will need to write a workflow to execute it, similar to the one generated by the wizard and containing all the basics.
@br
The @i{.oms} script generated for the Ants model is shown here:
@br@br
@hl.openmole("""
// Input values
val mySeed = Val[Int]
......@@ -112,8 +107,6 @@ antsTask hook display
To execute a workflow you just need to have the oms script open in the GUI and click on the "Run" button.
The execution panel will then open for you to monitor the execution, as shown below:
@br@br
@img(src := Resource.img.tutorial.modelExecution.file, width := "100%")
......@@ -121,8 +114,6 @@ The execution panel will then open for you to monitor the execution, as shown be
Once the model execution is over, and since we asked in the workflow to @code{display} the outputs, we can look at the output values @code{countFood}, @code{finalTicksFood1}, @code{finalTicksFood2}, and @code{finalTicksFood3} by clicking on the eye icon in the Executions panel.
@br@br
@img(src := Resource.img.tutorial.modelOutput.file, width := "100%")
......
......@@ -32,49 +32,60 @@
gEvaporationRate := 50
)"""
@p
This example presents how to explore a NetLogo model step by step with an Evolutionary/Genetic Algorithm (EA/GA) in
OpenMOLE.
@br
For more generic details regarding the exploitation of Genetic Algorithms using OpenMOLE, you can check the
@a("GA section of the methods documentation", href := DocumentationPages.explore.file + "#Geneticalgorithms")
This example presents how to explore a NetLogo model step by step with an Evolutionary/Genetic Algorithm (EA/GA) in OpenMOLE.
For more generic details regarding the use of Genetic Algorithms within OpenMOLE, you can check the @aa("GA section of the methods documentation", href := explore.file + "#Geneticalgorithms")
@h2{The ant model}
We demonstrate this tutorial using the ants foraging model present in the Netlogo library. This model was created by
Ury Wilensky. According to @a("NetLogo's website", href := shared.link.netlogoAnts),
this model is described as:
@br @i{In this project, a colony of ants forages for food. Though each ant follows a set of simple rules, the colony
In this tutorial we will be using the Ants foraging model, present in the Netlogo library.
This model was created by Ury Wilensky.
According to @aa("NetLogo's website", href := shared.link.netlogoAnts), this model is described as:
@br
@i{In this project, a colony of ants forages for food. Though each ant follows a set of simple rules, the colony
as a whole acts in a sophisticated way. When an ant finds a piece of food, it carries the food back to the nest,
dropping a chemical as it moves. When other ants “sniff” the chemical, they follow the chemical toward the food. As
more ants carry food to the nest, they reinforce the chemical trail.}
@p A visual representation of this model looks like that:
@br @img(src := Resource.img.example.ants.file)
@br
A visual representation of this model looks like:
@img(src := Resource.img.example.ants.file)
@br
In this tutorial we use a headless version (@aa("see NetLogo task documentation", href := DocumentationPages.netLogo.file)) of the model.
This modified version is available @aa("here", href := Resource.script.antsNLogo.file).
@p In this tutorial we use a headless version (@a("see NetLogo task documentation", href := DocumentationPages.netLogo.file)) of the
model. This modified version is available @aa("here", href := Resource.script.antsNLogo.file).
@h2{An optimisation problem}
This model manipulates three parameters:
@ul
@li{Population: number of Ants in the model,}
@li{Evaporation-rate: controls the evaporation rate of the chemical,}
@li{Diffusion-rate: controls the diffusion rate of the chemical.}
@li{population: number of Ants in the model,}
@li{evaporation-rate: controls the evaporation rate of the chemical,}
@li{diffusion-rate: controls the diffusion rate of the chemical.}
@br
@p Ants forage from three sources of food (see the number in the picture below). Each source is positioned at
different distances from the Ant colony.
@br @img(src := Resource.img.example.antNumbers.file)
Ants forage from three sources of food (see the number in the picture below).
Each source is positioned at different distances from the Ant colony.
@p It can be interesting to search the @b{best combination of the two parameters}
@hl("evaporation-rate", "plain") and @hl("diffusion-rate", "plain")
which minimises the eating time of each food source.
@img(src := Resource.img.example.antNumbers.file)
@p To build our fitness function, we modify the NetLogo Ants source code to store for each food source the first ticks
indicating that this food source is empty.
@br
@p @hl("""
It can be interesting to search for the @b{best combination of the two parameters} @code{evaporation-rate} and @code{diffusion-rate} which minimises the eating time of each food source.
To build our fitness function, we modify the NetLogo Ants source code to store, for each food source, the first ticks indicating that this food source is empty.
@hl("""
to compute-fitness
if ((sum [food] of patches with [food-source-number = 1] = 0) and (final-ticks-food1 = 0)) [
set final-ticks-food1 ticks ]
......@@ -84,26 +95,24 @@ to compute-fitness
set final-ticks-food3 ticks ]
end""", "plain")
@p At the end of each simulation we return the values for the three objectives (or criteria) :
At the end of each simulation we get the values of the three objectives (or criteria):
@ul
@li{The simulation ticks indicating that source 1 is empty,}
@li{The simulation ticks indicating that source 2 is empty,}
@li{The simulation ticks indicating that source 3 is empty.}
@p The combination of the three objectives indicates the quality of the parameters used to run the simulation.
This situation is a
@a("multi-objective optimisation", href := shared.link.multiobjectiveOptimization) problem. In
case there is a compromise between these 3 objectives, we will obtain a
@a("Pareto frontier", href := shared.link.paretoEfficency) at the end of the optimisation
The combination of the three objectives indicates the quality of the parameters used to run the simulation.
This situation is a @aa("multi-objective optimisation", href := shared.link.multiobjectiveOptimization) problem.
In case there is a compromise between these 3 objectives, we will get a @a("Pareto front", href := shared.link.paretoEfficency) at the end of the optimisation
process.
@h2{Run it in OpenMOLE}
When building a calibration or optimisation workflow, the first step is to make the model run in
OpenMOLE. This script simply plugs the NetLogo model and runs one single execution of the model with arbitrary
parameters. More details about the NetLogo5 task used in this script can be found in
@a("this section of the documentation", href := DocumentationPages.netLogo.file).
When building a calibration or optimisation workflow, the first step is to make the model run in OpenMOLE.
This script simply plugs the NetLogo model, and runs one single execution of the model with arbitrary parameters.
More details about the NetLogo5 task used in this script can be found in @aa("this section of the documentation", href := DocumentationPages.netLogo.file).
@hl.openmole(s"""
$model
......@@ -118,19 +127,23 @@ val model_execution = (ants on env hook displayHook)
model_execution
""", name = "ga netlogo model")
@p The result of this execution should look like:
@br @hl("{food1=746.0, food2=1000.0, food3=2109.0}", "plain")
The result of this execution should look like:
@hl("{food1=746.0, food2=1000.0, food3=2109.0}", "plain")
@h2{The optimisation algorithm}
We will try to find the parameter settings minimising these estimators.
This script describes how to use the NSGA2 multi-objective optimisation algorithm in OpenMOLE.
The result files are written to @hl("/tmp/ants", "plain").
Notice how the @hl("evaluation", "plain") parameter of the @hl("SteadyStateEvolution", "plain")
method, is the netlogo task i.e. running the model, which indeed provides an @b{evaluation} of the genome
(parameter settings) efficiency regarding the @hl("objective","plain").
The result files are written to @i{/tmp/ants}.
@br
@br @hl.openmole("""
Notice how the @code{evaluation} parameter of the @code{SteadyStateEvolution} method is the NetLogo task i.e. running the model, which indeed provides an @b{evaluation} of the genome (parameter settings) efficiency regarding the @hl("objective","plain").
@hl.openmole("""
// Define the inputs and their respective variation bounds.
// Define the objectives to minimize.
// Tell OpenMOLE that this model is stochastic and that it should generate a seed for each execution
......@@ -150,11 +163,16 @@ NSGA2Evolution(
) hook (workDirectory / "results")
""", header = model, name = "ga netlogo evolution")
@h2{Scale up}
If you use distributed computing, it might be a good idea to opt for an Island model. Islands are better suited to exploit distributed computing resources than classical generational genetic algorithms. See how the end of the script changes to implement islands in the workflow. Here we compute 2,000 islands in parallel, each running during 10 minutes on the European grid:
If you use distributed computing, it might be a good idea to opt for an Island model.
Islands are better suited to exploit distributed computing resources than classical generational genetic algorithms.
See how the end of the script changes to implement islands in the workflow.
Here we compute 2,000 islands in parallel, each running during 10 minutes on the European grid:
@br @hl.openmole("""
@hl.openmole("""
// Define the execution environment
val env = EGIEnvironment("vo.complex-systems.eu")
......@@ -168,4 +186,5 @@ NSGA2Evolution(
evaluation = ants,
termination = 10000000,
parallelism = 1000
) by Island(10 minutes) on env hook (workDirectory / "results")""", header = model, name = "ga netlogo islands")
) by Island(10 minutes) on env hook (workDirectory / "results")
""", header = model, name = "ga netlogo islands")
@import org.openmole.site.stylesheet._
@import org.openmole.site._
@import org.openmole.site.tools._
@import DocumentationPages._
@def variables = """
val mySeed = Val[Int]
val density = Val[Double]
val seed = Val[Int]
val burned = Val[Double]
"""
@h2
Sensitivity analysis
Typical Sensitivity analysis (in a simulation experiment context) is the study of how the variation of an input
affect the output(s) of a model. Basically it
@h2{Sensitivity analysis}
Typical sensitivity analysis (in a simulation experiment context) is the study of how the variations of an input affects the output(s) of a model.
@br
@img(src := Resource.img.method.sensitivityAnim.file, width := "80%")
@basicButton("Run", classIs(btn, btn_danger))(id := shared.sensitivity.button, stylesheet.svgRunButton(10))
@break
@h3{Prerequisites}
A plugged model in OpenMOLE (see Step 1 : Model)
A plugged model in OpenMOLE: see @aa("Step By Step Introduction", href := stepByStepIntro.file + "#ImportingatoymodelAnts")
@h2
Variation of one input
@h2{Variation of one input}
The most simple case to consider is to observe the effect of a single input variation on a single output.
This is achieved by using an @b{exploration task}, which will generate a sequence of input values, based on its given boundaries, and discretisation step.
This is achieved by using an @b{exploration task} , who will generate the sequence of values of an input, according to its
boundaries values and a discretisation step.
@br
@hl.openmole("""
@hl.openmole("""
val my_input = Val[Double]
val exploration =
ExplorationTask(
(my_input in (0.0 to 10.0 by 0.5))
)""", name = "variation of 1 input in sensitivity")
)""", name = "variation of 1 input in sensitivity")
@h2
Real world example
@h2{Real world example}
the @hl("Fire.nlogo", "plain") model is a simple, one-parameter, simulation model that simulates fire propagation.
This model features a threshold value in its unique parameter domain, below which fire fails to burn the majority
of the forest, and beyond which fire propagates and burn most of it.
the @code{Fire.nlogo} model is a simple, one-parameter, simulation model that simulates fire propagation.
This model features a threshold value in its unique parameter domain, below which fire fails to burn the majority of the forest, and beyond which fire propagates and burns most of it.
We will perform sensitivity analysis to make this change of regime appear.
The Fire model integration has been covered in the @a("NetLogo page of the Model section", href:= DocumentationPages.netLogo.file), so we take it from here.
@br
The Fire model integration has been covered in the @aa("NetLogo page of the Model section", href:= DocumentationPages.netLogo.file), so we take it from here.
The former script was already performing a sensitivity Analysis, by varying density from 20 to 80 by step of 10, with 10 replication for each (seed is taken 10 times).
The former script was already performing a sensitivity analysis, by varying @code{density} from 20 to 80 by step of 10, with 10 replications for each (@code{mySeed} has 10 different values for each value of @code{density}).
@br
In our case, the quantum of 10 percent is quite coarsed, so we make it 1 percent :
In our case, the quantum of 10 percent is quite coarsed, so we make it 1 percent:
@hl.openmole("""
val exploration =
......@@ -68,15 +63,14 @@ val exploration =
(density in (20.0 to 80.0 by 1.0)) x
(seed in (UniformDistribution[Int]() take 10))
)
""", header=variables)
@break
This is the resulting scatterplot of Number of burned trees according to density varying from 20% to 80% by 1% steps.
@br@br
""", header=variables)
This is the resulting scatterplot of the number of burned trees according to density varying from 20% to 80% by 1% steps.
@img(src := Resource.img.method.densityBurned.file, width := "80%")
@br
The change of regime clearly appears between 50% and 75% percent density, so we are going to take a closer look at this domain: we change the exploration task to have the density taken from 50% to 75% by step of 0.1%, still with 10 replications:
@br
@hl.openmole("""
val exploration =
ExplorationTask(
......@@ -85,52 +79,38 @@ The change of regime clearly appears between 50% and 75% percent density, so we
)
""", header=variables)
@break
This gives us the following scatter plot, where we added a summary curve (using the geom_smooth() function from @aa("ggplot2 R library", href:=shared.link.ggplot2)) which acts like a smoother, giving the statistical trend of the y axis values of points along the x axis.
This gives us the following scatter plot, where we added a summary curve (using the geom_smooth() function from @a("ggplot2 R library", href:=shared.link.ggplot2)) which acts like a smoother, giving the statistical trend of the Y-Axis values of points along the X-axis.
@img(src := org.openmole.site.Resource.img.method.densityBurnedZoom.file)
@br@br
@img(src := org.openmole.site.Resource.img.method.densityBurnedZoom.file, width := "80%")
The trend appears to be a sigmoïd shape on which we could find the mid point, the steepness of the curve and so on, to better characterize the phenomenon.
@br
The trend appears to be a sigmoïd shape on which we could find the mid point, the steepness of the curve and so on, to better characterize the phenomenon.
@break
Eventually, let's have a look at the stochasticity of the model. Looking at the plot, we see that there is a lot of variation possible, especially in the transition zone, around 60% density.
Let's make a final exploration Task to investigate the variation of model outputs for several replications of the same parameterization.
Eventually, let's have a look at the stochasticity of the model.
Looking at the plot, we see that there is a lot of variation possible, especially in the transition zone, around 60% density.
Let's make a final exploration task to investigate the variation of model outputs for several replications of the same parameterization.
We now take densities within [50;70] and take 100 replications for each:
@hl.openmole("""
val exploration =
ExplorationTask(
(density in (50.0 to 70.0 by 0.1)) x
(seed in (UniformDistribution[Int]() take 100))
)""", header= variables)
@br
We obtain the following results, displayed as boxplots every percent (but results are obtained for every 0.1% density) to emphasize on the variations.
)""", header= variables)
@br@br
We obtain the following results, displayed as boxplots every percent (but results are obtained for every 0.1% density) to emphasize the variations.
@img(src := Resource.img.method.densitySeedBox.file, width := "80%")
As we can see, stochasticity has a huge effect right in the middle of the transition of regime, reaching its maximum at 59%.
Another interesting thing to investigate would be the density from which the fire propagates to the end of the zone (right side of NetLogo World).
A slight modification of the model code would do the trick, hint: @code{any? patches with-max [pxcor] ...} would be a good starting point :-) but we let you do it on your own (any successful attempts would win the right to be displayed right here!).
@br
As we can see, stochasticity has an high effect right in the middle of the transition of regime, reaching maximum at 59%.
Another interesting thing to investigate what would be the density from which the fire propagates to the end of the zone (right side of NetLogo World). A slight modification of the model code would do the trick (hint:
@hl("any? patches with-max [pxcor] ... ","plain") would be a good starting point :-) but we let you do it on your own (any successful attempt would win the right to be displayed right here ! )
@p
For completeness sake, here is the R code snippet to produce plots from your results file (RStudio version 1.0.153, ggplot2 version 2.2.1). The result files correspond to the experiments above in the same order.
For completeness sake, here is the R code snippet to produce plots from your results file (RStudio version 1.0.153, ggplot2 version 2.2.1).
The result files correspond to the experiments above in the same order.
@hl("""
library(ggplot2)
......@@ -169,14 +149,6 @@ For completeness sake, here is the R code snippet to produce plots from your res
@h2{To go further}
In the @a("next tutorial", href := DocumentationPages.netLogoGA.file), more advanced methods such as calibration using genetic algorithms are shown still on a NetLogo model.
In the @a("next tutorial", href := DocumentationPages.netLogoGA.file), more advanced methods such as calibration using genetic algorithms are shown, still on a NetLogo model.
\ No newline at end of file
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment