Commit a813e643 authored by Sam Thiriot's avatar Sam Thiriot
Browse files

added test functions for NSGA2 with BK, ConstrEx, Schaffer N1 and N2

parent 871a8c02
......@@ -7,27 +7,25 @@ Genetic Algorithms such as NSGA2 are even less trivial.
Their implementation in the [mgo](https://github.com/openmole/mgo) used in [OpenMole](https://openmole.org/) was adapted so they are efficient for computation on clusters or grids.
This leaves a lot of uncertainty to the user on what really happens.
Why should you, as a user, trust this implementation?
The answer is simple: you should not.
Any implementation in scientific computing should be verified.
The answer is simple: you should not.
Any implementation in scientific computing should be verified to be trusted.
## Test Functions
Many functions named "Test Functions" were proposed over time to test and compare multi-objective optimization algorithmls.
Many functions named "Test Functions" were proposed over time to test and compare multi-objective optimization algorithms.
The [wikipedia page](https://en.wikipedia.org/wiki/Test_functions_for_optimization) lists several of them.
A test function is an optimization function which you can propose as the problem to optimize to your implementation.
Functions test:
A test function is an function you can use as the problem to optimize, for which the expected results are known.
Test functions might test:
- problems with multiple parameters and multiple objectives
- problems with constraints: a part of the space of parameter leads to computation errors as a result, so the algorithm should achieve to deal with it
- problems with parts of the space of solution that are tricky to detect, either because they are statistically unlikely to find, or because they are is a part of the space of solutions which is heavily constrained, etc.
- problems with constraints: a part of the space of solutions can not be computed, so the algorithm should achieve to deal with it
- problems with parts of the space of solution that are tricky to detect, either because they are statistically unlikely to find, or because they are in a part of the space of solutions which is heavily constrained, etc.
Test tunctions enable the testing of several aspects including:
The results of a test function are usually compared in both terms of:
- coverage of the Pareto front,
- speed of convergence
Once a test function was explored by an implementation of an algorithm, you should compare the Pareto front obtained withou your implementation with the one expected in literature.
## Test and Learn
......@@ -35,14 +33,22 @@ Test functions have another role: they constitute a simple example of optimizati
As a user, you can also learn to tune the parameters of the genetic algorithm as check when convergence
occurs.
## Test Workflows for OpenMole
We provide here a few examples of test functions which you can open with OpenMole.
For each workflow:
- Choose a test function.
- Run the workflow.
- Open the files with the solutions, check you understand them.
- Graph them and compare them with the literature (you have these results in the [wikipedia page](https://en.wikipedia.org/wiki/Test_functions_for_optimization)).
- ConstrEx
- CP1
- Schaffer N1
- Schaffer N2
To run a test
- Choose one of the workflows starting with "test function"
- Run the workflow
- Update the view on the left using the "refresh" button
- Download the graph of the last Pareto front, and compare it with the literature (you might use the [wikipedia page](https://en.wikipedia.org/wiki/Test_functions_for_optimization) )
You might then tune the parameters of the optimization algorithm and analyze the results, to understand how to better use the optimization method.
// about the local computer
val threads = 4
val envMultiThread = LocalEnvironment(threads, name="multithread")
\ No newline at end of file
//model inputs
val x = Val[Double]
val y = Val[Double]
//model outputs
val f1 = Val[Double]
val f2 = Val[Double]
val testFunctionCPT1 =
ScalaTask("""
val f1 = x;
val f2 = (1 + y) * Math.exp( -x / (1 + y) );
""") set (
inputs += x,
inputs += y,
outputs += f1,
outputs += f2
)
val evolutionCPT1 =
NSGA2Evolution(
genome = Seq(
x in (0.0, 10000.0),
y in (-10000.0, 1.0)
),
objective = Seq(f1, f2),
evaluation = testFunctionCPT1,
parallelism = 10,
termination = 1000
)
val savePopulation = SavePopulationHook(evolutionCPT1, workDirectory/"results/CPT1")
evolutionCPT1 hook savePopulation on LocalEnvironment(4)
val last_pareto = Val[File]
val directoryWithResults = Val[File]
val filesHaveHeaders = Val[Int]
val countInputs = Val[Int]
val taskPlotLastParetoFront = RTask("""
library(ggplot2)
colnames <- if (countInputs == 2) c("iteration", "x", "y", "f1", "f2") else c("iteration", "x", "f1", "f2")
coltypes <- if (countInputs == 2) c("integer", "numeric", "numeric", "numeric", "numeric") else c("integer", "numeric", "numeric", "numeric")
names(coltypes) <- colnames
directoryWithResultsName <- "mydirectory"
# ensure check the directory exists
if (!file.exists(directoryWithResultsName)) { stop(paste("ERROR: the directory", directoryWithResultsName, "does not exists!")) }
# get the most recent file (will be the last result)
allfiles <- file.info(list.files(directoryWithResultsName, full.names = T))
lastfilename <- rownames(allfiles)[which.max(allfiles$mtime)]
if (!file.exists(lastfilename)) { stop(paste("ERROR: no file found in", directoryWithResultsName) ) }
print(lastfilename)
# read the last file
pop <- read.csv(header = filesHaveHeaders>0, col.names=colnames, colClasses=coltypes, file=lastfilename)
print(paste("there are", nrow(pop), "points on the last Pareto front"))
# plot
g <- ggplot(pop, aes(x=f1,y=f2)) + geom_point()
ggsave(filename="/tmp/last_pareto.png", plot=g)
""",
install = Seq(
"fakeroot sed -i 's/deb.debian.org/linux.pleiade.edf.fr/g' /etc/apt/sources.list",
"fakeroot cat /etc/apt/sources.list",
// update the list of available packages
"fakeroot apt-get -o Acquire::http::proxy=false update ",
// required; attempts to update dbus to a newer version would require permissions we do not have
"DEBIAN_FRONTEND=noninteractive fakeroot apt-mark hold dbus",
"""echo "dbus hold" | fakeroot dpkg --set-selections""",
// install the libs required for the compilation of R packages
"DEBIAN_FRONTEND=noninteractive fakeroot apt-get -o Acquire::http::proxy=false install -y libssl-dev libcurl4-openssl-dev libudunits2-dev",
// install required R packages in their binary version (quicker, much stable!)
"DEBIAN_FRONTEND=noninteractive fakeroot apt-get -o Acquire::http::proxy=false install -y r-cran-ggplot2",
), //
libraries = Seq(), // "ggplot2","gganimate","plotly","GGally","htmlwidgets"
//resources = Seq(workDirectory / "multiobjective/results/ConstrEx")
) set (
inputFiles += (directoryWithResults, "mydirectory"),
outputFiles += ("/tmp/last_pareto.png", last_pareto),
inputs += filesHaveHeaders.mapped,
inputs += countInputs.mapped,
(inputs, outputs) += directoryWithResults,
filesHaveHeaders := 1,
countInputs := 2
)
import _file_.plottingFunctions._
import _file_.execEnvironment._
//model inputs
val x = Val[Double]
val y = Val[Double]
//model outputs
val f1 = Val[Double]
val f2 = Val[Double]
// about the current experiment
val relativePath = "results/BinhKorn"
// the test function
val testFunctionBinhKorn =
ScalaTask("""
val g1 = Math.pow(x - 5, 2) + Math.pow(y, 2)
val g2 = Math.pow(x - 8, 2) + Math.pow(y + 2, 3)
val constrained = g1 > 25 || g2 < 7.7
val f1 = if (constrained) Double.NaN else 4*Math.pow(x,2) + 4*Math.pow(y,2)
val f2 = if (constrained) Double.NaN else Math.pow(x-5,2) + Math.pow(y-5,2)
""") set (
inputs += x,
inputs += y,
outputs += f1,
outputs += f2
)
// the optimisation algorithm under test
val evolutionBinhKorn =
NSGA2Evolution(
genome = Seq(
x in (0.0, 5.0),
y in (0.0, 3.0)
),
objective = Seq(f1, f2),
evaluation = testFunctionBinhKorn,
parallelism = threads * 2,
termination = 1000
)
// compute evolution on the test Function
(evolutionBinhKorn on envMultiThread hook SavePopulationHook(evolutionBinhKorn, workDirectory/relativePath)
// then plot the last Pareto front
) -- (taskPlotLastParetoFront set ( directoryWithResults := workDirectory/relativePath) hook CopyFileHook(last_pareto, workDirectory/"last Pareto front BinhKorn.png" ) )
import _file_.plottingFunctions._
import _file_.execEnvironment._
//model inputs
val x = Val[Double]
val y = Val[Double]
......@@ -6,29 +9,39 @@ val y = Val[Double]
val f1 = Val[Double]
val f2 = Val[Double]
// about the current experiment
val relativePath = "results/ConstrEx"
// the test function
val testFunctionConstrEx =
ScalaTask("""
val f1 = x;
val f2 = (1 + y) / x;
val g1 = y + 9*x
val g2 = -y + 9*x
val constrained = g1 < 6 || g2 < 1
val f1 = if (constrained) Double.NaN else x;
val f2 = if (constrained) Double.NaN else (1 + y) / x;
""") set (
inputs += x,
inputs += y,
outputs += f1,
outputs += f2
)
// the optimisation algorithm under test
val evolutionConstrEx =
NSGA2Evolution(
genome = Seq(
x in (0.1, 1.0),
y in (0.0, 1.0)
y in (0.0, 5.0)
),
objective = Seq(f1, f2),
evaluation = testFunctionConstrEx,
parallelism = 10,
termination = 1000
parallelism = threads * 2,
termination = 5000
)
val savePopulation = SavePopulationHook(evolutionConstrEx, workDirectory/"results/ConstrEx")
evolutionConstrEx on LocalEnvironment(4) hook savePopulation
// compute evolution on the test Function
(evolutionConstrEx on envMultiThread hook SavePopulationHook(evolutionConstrEx, workDirectory/relativePath)
// then plot the last Pareto front
) -- (taskPlotLastParetoFront set ( directoryWithResults := workDirectory/relativePath) hook CopyFileHook(last_pareto, workDirectory/"last Pareto front ConstrEx.png" ) )
import _file_.plottingFunctions._
import _file_.execEnvironment._
//model inputs
val x = Val[Double]
......@@ -8,6 +11,10 @@ val f2 = Val[Double]
// explored space
val A = 1000.0 // up to 10^5
// about the current experiment
val relativePath = "results/SchafferN1"
// the test function
val testFunctionSchafferN1 =
ScalaTask("""
val f1 = Math.pow(x, 2);
......@@ -17,6 +24,8 @@ val testFunctionSchafferN1 =
outputs += f1,
outputs += f2
)
// the optimisation algorithm under test
val evolutionSchafferN1 =
NSGA2Evolution(
genome = Seq(
......@@ -24,9 +33,11 @@ val evolutionSchafferN1 =
),
objective = Seq(f1, f2),
evaluation = testFunctionSchafferN1,
parallelism = 10,
parallelism = threads * 2,
termination = 1000
)
val savePopulation = SavePopulationHook(evolutionSchafferN1, workDirectory/"results/SchafferN1")
evolutionSchafferN1 hook savePopulation on LocalEnvironment(4)
// compute evolution on the test Function
(evolutionSchafferN1 on envMultiThread hook SavePopulationHook(evolutionSchafferN1, workDirectory/relativePath)
// then plot the last Pareto front
) -- (taskPlotLastParetoFront set ( directoryWithResults := workDirectory/relativePath, countInputs := 1) hook CopyFileHook(last_pareto, workDirectory/"last Pareto front SchafferN1.png" ) )
import _file_.plottingFunctions._
import _file_.execEnvironment._
//model inputs
val x = Val[Double]
//model outputs
val f1 = Val[Double]
val f2 = Val[Double]
// about the current experiment
val relativePath = "results/SchafferN2"
// the test function
val testFunctionSchafferN2=
ScalaTask("""
val f1 = if (x <= 1) {
-x*1.0
} else if (x <= 3) {
x - 2.0
} else if (x <= 4) {
4.0 - x
} else {
x - 4.0
}
val f2 = Math.pow(x - 5, 2);
""") set (
inputs += x,
outputs += f1,
outputs += f2
)
// the optimisation algorithm under test
val evolutionSchafferN2 =
NSGA2Evolution(
genome = Seq(
x in (-5.0, 10.0)
),
objective = Seq(f1, f2),
evaluation = testFunctionSchafferN2,
parallelism = threads * 2,
termination = 1000
)
// compute evolution on the test Function
(evolutionSchafferN2 on envMultiThread hook SavePopulationHook(evolutionSchafferN2, workDirectory/relativePath)
// then plot the last Pareto front
) -- (taskPlotLastParetoFront set ( directoryWithResults := workDirectory/relativePath, countInputs := 1) hook CopyFileHook(last_pareto, workDirectory/"last Pareto front SchafferN2.png" ) )
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment