Commit c5ecef5d authored by Sam Thiriot's avatar Sam Thiriot
Browse files

first examples

parent 6f7e6505
# Test Functions for NSGA2
## Why Test?
Metaheuristics for optimization such as genetic algorithms leave a huge freedom to developers for their implementation.
Genetic Algorithms such as NSGA2 are even less trivial.
Their implementation in the [mgo](https://github.com/openmole/mgo) used in [OpenMole](https://openmole.org/) was adapted so they are efficient for computation on clusters or grids.
This leaves a lot of uncertainty to the user on what really happens.
Why should you, as a user, trust this implementation?
The answer is simple: you should not.
Any implementation in scientific computing should be verified to be trusted.
## Test Functions
Many functions named "Test Functions" were proposed over time to test and compare multi-objective optimization algorithms.
The [wikipedia page](https://en.wikipedia.org/wiki/Test_functions_for_optimization) lists several of them.
A test function is an function you can use as the problem to optimize, for which the expected results are known.
Test functions might test:
- problems with multiple parameters and multiple objectives
- problems with constraints: a part of the space of solutions can not be computed, so the algorithm should achieve to deal with it
- problems with parts of the space of solution that are tricky to detect, either because they are statistically unlikely to find, or because they are in a part of the space of solutions which is heavily constrained, etc.
The results of a test function are usually compared in both terms of:
- coverage of the Pareto front,
- speed of convergence
## Test and Learn
Test functions have another role: they constitute a simple example of optimization problem.
As a user, you can also learn to tune the parameters of the genetic algorithm as check when convergence
occurs.
## Test Workflows for OpenMole
We provide here a few examples of test functions which you can open with OpenMole.
- ConstrEx
- CP1
- Schaffer N1
- Schaffer N2
To run a test
- Choose one of the workflows starting with "test function"
- Run the workflow
- Update the view on the left using the "refresh" button
- Download the graph of the last Pareto front, and compare it with the literature (you might use the [wikipedia page](https://en.wikipedia.org/wiki/Test_functions_for_optimization) as a start)
You might then tune the parameters of the optimization algorithm and analyze the results, to understand how to better use the optimization method.
//model inputs
val x = Val[Double]
//model outputs
val f1 = Val[Double]
val f2 = Val[Double]
// about the current experiment
val relativePath = "results/SchafferN2"
// the test function
val testFunctionSchafferN2=
ScalaTask("""
val f1 = if (x <= 1) {
-x*1.0
} else if (x <= 3) {
x - 2.0
} else if (x <= 4) {
4.0 - x
} else {
x - 4.0
}
val f2 = Math.pow(x - 5, 2);
""") set (
inputs += x,
outputs += f1,
outputs += f2
)
// the optimisation algorithm under test
val evolutionSchafferN2 =
NSGA2Evolution(
genome = Seq(
x in (-5.0, 10.0)
),
objective = Seq(f1, f2),
evaluation = testFunctionSchafferN2,
parallelism = 8,
termination = 1000
)
val envMultiThread = LocalEnvironment(4, name="multithread")
// compute evolution on the test Function
(evolutionSchafferN2 on envMultiThread hook (workDirectory/relativePath, keepAll=false)
// then plot the last Pareto front
) -- (taskPlotLastParetoFront set ( directoryWithResults := workDirectory/relativePath, countInputs := 1) hook CopyFileHook(last_pareto, workDirectory/"last Pareto front SchafferN2.png" ) )
// this variable will transmit the path where the CSV files to graph will be found
val directoryWithResults = Val[File]
// variables used to parameter the graphing function
val filesHaveHeaders = Val[Int]
val countInputs = Val[Int]
val graphWidth = Val[Int]
val graphHeight = Val[Int]
val framerate = Val[Int]
// this variable will contain the file with the graphical rendering of the last PAreto front
val video = Val[File]
val taskPlotAsVideo = RTask("""
library(ggplot2)
library(gganimate)
colnames <- if (countInputs == 2) c("iteration", "x", "y", "f1", "f2") else c("iteration", "x", "f1", "f2")
coltypes <- if (countInputs == 2) c("integer", "numeric", "numeric", "numeric", "numeric") else c("integer", "numeric", "numeric", "numeric")
names(coltypes) <- colnames
directoryWithResultsName <- "mydirectory"
pop <- NULL
i <- 1
while (TRUE) {
# TODO check creation time of the file
filename <- paste(directoryWithResultsName,"/population",i,".csv", sep="");
if (!file.exists(filename)) {
break
}
#print(filename)
popraw <- read.csv(header = FALSE, col.names=colnames, colClasses=coltypes, file=filename)
#print(head(popraw))
pop <- if (is.null(pop)) popraw else rbind(pop, popraw)
i <- i + 1
}
# the ggplot
p <- ggplot(pop, aes(x=f1,y=f2)) + geom_point()
# render with gganimate
gganimation <- p + transition_states(iteration) + transition_time(iteration) #+ labs(title="iteration: {iteration}")
# ... first render individual PNG frames which are always of use
animate(gganimation,
renderer=file_renderer("/tmp/rendered", overwrite=T, prefix="iteration"),
height=graphHeight, width=graphWidth,
device='png')
# render as mp4
print("rendering as a video")
system(paste("ffmpeg -y -framerate", framerate, "-i /tmp/rendered/iteration%04d.png -c:v libx264 -r", framerate," /tmp/render.mp4"))
""",
install = Seq(
"fakeroot sed -i 's/deb.debian.org/linux.pleiade.edf.fr/g' /etc/apt/sources.list",
"fakeroot cat /etc/apt/sources.list",
// update the list of available packages
"fakeroot apt-get -o Acquire::http::proxy=false update ",
// required; attempts to update dbus to a newer version would require permissions we do not have
"DEBIAN_FRONTEND=noninteractive fakeroot apt-mark hold dbus",
"""echo "dbus hold" | fakeroot dpkg --set-selections""",
// install the libs required for the compilation of R packages
"DEBIAN_FRONTEND=noninteractive fakeroot apt-get -o Acquire::http::proxy=false install -y libssl-dev libcurl4-openssl-dev libudunits2-dev",
// install required R packages in their binary version (quicker, much stable!)
"DEBIAN_FRONTEND=noninteractive fakeroot apt-get -o Acquire::http::proxy=false install -y r-cran-ggplot2 r-cran-gganimate r-cran-ggally r-cran-plotly r-cran-zip",
// install external tools in the VM for rendering
"DEBIAN_FRONTEND=noninteractive fakeroot apt-get -o Acquire::http::proxy=false install -y ffmpeg",
), //
libraries = Seq() // "ggplot2","gganimate","plotly","GGally","htmlwidgets"
//resources = Seq(workDirectory / "multiobjective/results/ConstrEx")
) set (
inputFiles += (directoryWithResults, "mydirectory"),
outputFiles += ("/tmp/render.mp4", video),
inputs += filesHaveHeaders.mapped,
inputs += countInputs.mapped,
inputs += graphWidth.mapped,
inputs += graphHeight.mapped,
inputs += framerate.mapped,
filesHaveHeaders := 1,
countInputs := 2,
graphWidth := 600,
graphHeight := 600,
framerate := 5,
)
// import from the other file an example of optimization
import _file_.example_of_optimization._
// run the optimization
( evolutionSchafferN2 on envMultiThread hook (workDirectory/relativePath, keepAll=false)
) -- (
// then run the plotting function
taskPlotAsVideo set (
// ... which will read all the results from this file
directoryWithResults := workDirectory / relativePath,
// ... analyze them knowing there is only one input in the files
countInputs := 1
) hook CopyFileHook(video, workDirectory/"iterations_video.mp4" )
)
\ No newline at end of file
// this variable will transmit the path where the CSV files to graph will be found
val directoryWithResults = Val[File]
// variables used to parameter the graphing function
val filesHaveHeaders = Val[Int]
val countInputs = Val[Int]
// this variable will contain the file with the graphical rendering of the last PAreto front
val plotlyPage = Val[File]
val taskPlotAsVideo = RTask("""
library(ggplot2)
library(plotly)
library(htmlwidgets)
colnames <- if (countInputs == 2) c("iteration", "x", "y", "f1", "f2") else c("iteration", "x", "f1", "f2")
coltypes <- if (countInputs == 2) c("integer", "numeric", "numeric", "numeric", "numeric") else c("integer", "numeric", "numeric", "numeric")
names(coltypes) <- colnames
directoryWithResultsName <- "mydirectory"
pop <- NULL
i <- 1
while (TRUE) {
# TODO check creation time of the file
filename <- paste(directoryWithResultsName,"/population",i,".csv", sep="");
if (!file.exists(filename)) {
break
}
#print(filename)
popraw <- read.csv(header = FALSE, col.names=colnames, colClasses=coltypes, file=filename)
#print(head(popraw))
pop <- if (is.null(pop)) popraw else rbind(pop, popraw)
i <- i + 1
}
# the ggplot
pop$index <- 1:nrow(pop) # add an index to avoid animation of points which are not at all similar
p <- ggplot(pop, aes(x=f1,y=f2,l1=x)
) + geom_point(
alpha=0.7, colour = "#51A0D5", aes(frame=iteration,ids=index)
) + labs(
x = "f1",
y = "f2",
title = "Evolution of the Pareto front with NSGA2"
) + theme_classic()
# render as plotly
dir.create("/tmp/plotly")
fig <- ggplotly(p)
saveWidget(fig, selfcontained=F, file = "/tmp/plotly/evolution.html")
lf <- list.files("/tmp/plotly/", recursive=T, include.dirs=F)
print(lf)
# zip resulting files
library(zip)
setwd("/tmp/plotly/")
zip("/tmp/evolution.zip", lf)
print("done")
""",
install = Seq(
"fakeroot sed -i 's/deb.debian.org/linux.pleiade.edf.fr/g' /etc/apt/sources.list",
"fakeroot cat /etc/apt/sources.list",
// update the list of available packages
"fakeroot apt-get -o Acquire::http::proxy=false update ",
// required; attempts to update dbus to a newer version would require permissions we do not have
"DEBIAN_FRONTEND=noninteractive fakeroot apt-mark hold dbus",
"""echo "dbus hold" | fakeroot dpkg --set-selections""",
// install the libs required for the compilation of R packages
"DEBIAN_FRONTEND=noninteractive fakeroot apt-get -o Acquire::http::proxy=false install -y libssl-dev libcurl4-openssl-dev libudunits2-dev",
// install required R packages in their binary version (quicker, much stable!)
"DEBIAN_FRONTEND=noninteractive fakeroot apt-get -o Acquire::http::proxy=false install -y r-cran-ggplot2 r-cran-gganimate r-cran-ggally r-cran-plotly r-cran-zip",
// install required R packages in their binary version (quicker, much stable!)
"DEBIAN_FRONTEND=noninteractive fakeroot apt-get -o Acquire::http::proxy=false install -y ffmpeg",
), //
libraries = Seq() // "ggplot2","gganimate","plotly","GGally","htmlwidgets"
//resources = Seq(workDirectory / "multiobjective/results/ConstrEx")
) set (
inputFiles += (directoryWithResults, "mydirectory"),
outputFiles += ("/tmp/evolution.zip", plotlyPage),
inputs += filesHaveHeaders.mapped,
inputs += countInputs.mapped,
filesHaveHeaders := 1,
countInputs := 2
)
// import from the other file an example of optimization
import _file_.example_of_optimization._
// run the optimization
( evolutionSchafferN2 on envMultiThread hook (workDirectory/relativePath, keepAll=false)
) -- (
// then run the plotting function
taskPlotAsVideo set (
// ... which will read all the results from this file
directoryWithResults := workDirectory / relativePath,
// ... analyze them knowing there is only one input in the files
countInputs := 1
) hook CopyFileHook(plotlyPage, workDirectory/"evolutionPlotly.zip" )
)
\ No newline at end of file
// this variable will transmit the path where the CSV files to graph will be found
val directoryWithResults = Val[File]
// variables used to parameter the graphing function
val filesHaveHeaders = Val[Int]
val countInputs = Val[Int]
val graphWidth = Val[Int]
val graphHeight = Val[Int]
val graphFormat = Val[String]
// this variable will contain the file with the graphical rendering of the last PAreto front
val pngForIterations = Val[File]
val taskPlotEvereyIteration = RTask("""
library(ggplot2)
library(gganimate)
colnames <- if (countInputs == 2) c("iteration", "x", "y", "f1", "f2") else c("iteration", "x", "f1", "f2")
coltypes <- if (countInputs == 2) c("integer", "numeric", "numeric", "numeric", "numeric") else c("integer", "numeric", "numeric", "numeric")
names(coltypes) <- colnames
directoryWithResultsName <- "mydirectory"
pop <- NULL
i <- 1
while (TRUE) {
# TODO check creation time of the file
filename <- paste(directoryWithResultsName,"/population",i,".csv", sep="");
if (!file.exists(filename)) {
break
}
#print(filename)
popraw <- read.csv(header = FALSE, col.names=colnames, colClasses=coltypes, file=filename)
#print(head(popraw))
pop <- if (is.null(pop)) popraw else rbind(pop, popraw)
i <- i + 1
}
# the ggplot
p <- ggplot(pop, aes(x=f1,y=f2)) + geom_point()
# render with gganimate
gganimation <- p + transition_states(iteration) + transition_time(iteration)# + labs(title="iteration: {iteration}")
# ... first render individual PNG frames which are always of use
animate(gganimation,
renderer=file_renderer("/tmp/rendered", overwrite=T, prefix="iteration"),
height=graphHeight, width=graphWidth,
device=graphFormat)
""",
install = Seq(
"fakeroot sed -i 's/deb.debian.org/linux.pleiade.edf.fr/g' /etc/apt/sources.list",
"fakeroot cat /etc/apt/sources.list",
// update the list of available packages
"fakeroot apt-get -o Acquire::http::proxy=false update ",
// required; attempts to update dbus to a newer version would require permissions we do not have
"DEBIAN_FRONTEND=noninteractive fakeroot apt-mark hold dbus",
"""echo "dbus hold" | fakeroot dpkg --set-selections""",
// install the libs required for the compilation of R packages
"DEBIAN_FRONTEND=noninteractive fakeroot apt-get -o Acquire::http::proxy=false install -y libssl-dev libcurl4-openssl-dev libudunits2-dev",
// install required R packages in their binary version (quicker, much stable!)
"DEBIAN_FRONTEND=noninteractive fakeroot apt-get -o Acquire::http::proxy=false install -y r-cran-ggplot2 r-cran-gganimate r-cran-ggally r-cran-plotly r-cran-zip",
// install external tools in the VM for rendering
"DEBIAN_FRONTEND=noninteractive fakeroot apt-get -o Acquire::http::proxy=false install -y ffmpeg",
), //
libraries = Seq() // "ggplot2","gganimate","plotly","GGally","htmlwidgets"
//resources = Seq(workDirectory / "multiobjective/results/ConstrEx")
) set (
inputFiles += (directoryWithResults, "mydirectory"),
outputFiles += ("/tmp/rendered", pngForIterations),
inputs += filesHaveHeaders.mapped,
inputs += countInputs.mapped,
inputs += graphWidth.mapped,
inputs += graphHeight.mapped,
inputs += graphFormat.mapped,
filesHaveHeaders := 1,
countInputs := 2,
graphWidth := 600,
graphHeight := 600,
graphFormat := "png"
)
// import from the other file an example of optimization
import _file_.example_of_optimization._
// run the optimization
( evolutionSchafferN2 on envMultiThread hook (workDirectory/relativePath, keepAll=false)
) -- (
// then run the plotting function
taskPlotEvereyIteration set (
// ... which will read all the results from this file
directoryWithResults := workDirectory / relativePath,
// ... analyze them knowing there is only one input in the files
countInputs := 1
) hook CopyFileHook(pngForIterations, workDirectory/"iterations as graphs" )
)
\ No newline at end of file
// this variable will transmit the path where the CSV files to graph will be found
val directoryWithResults = Val[File]
// variables used to parameter the graphing function
val filesHaveHeaders = Val[Int]
val countInputs = Val[Int]
val graphWidth = Val[Int]
val graphHeight = Val[Int]
val graphFormat = Val[String]
// this variable will contain the file with the graphical rendering of the last PAreto front
val lastPareto = Val[File]
val taskPlotLastParetoFront = RTask("""
library(ggplot2)
colnames <- if (countInputs == 2) c("iteration", "x", "y", "f1", "f2") else c("iteration", "x", "f1", "f2")
coltypes <- if (countInputs == 2) c("integer", "numeric", "numeric", "numeric", "numeric") else c("integer", "numeric", "numeric", "numeric")
names(coltypes) <- colnames
directoryWithResultsName <- "mydirectory"
# ensure check the directory exists
if (!file.exists(directoryWithResultsName)) { stop(paste("ERROR: the directory", directoryWithResultsName, "does not exists!")) }
# get the most recent file (will be the last result)
allfiles <- file.info(list.files(directoryWithResultsName, full.names = T))
lastfilename <- rownames(allfiles)[which.max(allfiles$mtime)]
if (!file.exists(lastfilename)) { stop(paste("ERROR: no file found in", directoryWithResultsName) ) }
print(lastfilename)
# read the last file
pop <- read.csv(header = filesHaveHeaders>0, col.names=colnames, colClasses=coltypes, file=lastfilename)
print(paste("there are", nrow(pop), "points on the last Pareto front"))
# plot
g <- ggplot(pop, aes(x=f1,y=f2)) + geom_point()
dpi <- 72
ggsave(filename="/tmp/last_pareto",
device=graphFormat,
plot=g,
width=graphWidth/dpi, height=graphHeight/dpi)
""",
install = Seq(
"fakeroot sed -i 's/deb.debian.org/linux.pleiade.edf.fr/g' /etc/apt/sources.list",
"fakeroot cat /etc/apt/sources.list",
// update the list of available packages
"fakeroot apt-get -o Acquire::http::proxy=false update ",
// required; attempts to update dbus to a newer version would require permissions we do not have
"DEBIAN_FRONTEND=noninteractive fakeroot apt-mark hold dbus",
"""echo "dbus hold" | fakeroot dpkg --set-selections""",
// install the libs required for the compilation of R packages
"DEBIAN_FRONTEND=noninteractive fakeroot apt-get -o Acquire::http::proxy=false install -y libssl-dev libcurl4-openssl-dev libudunits2-dev",
// install required R packages in their binary version (quicker, much stable!)
"DEBIAN_FRONTEND=noninteractive fakeroot apt-get -o Acquire::http::proxy=false install -y r-cran-ggplot2 r-cran-gganimate r-cran-ggally r-cran-plotly r-cran-zip",
// install external tools in the VM for rendering
"DEBIAN_FRONTEND=noninteractive fakeroot apt-get -o Acquire::http::proxy=false install -y ffmpeg",
), //
libraries = Seq() // "ggplot2","gganimate","plotly","GGally","htmlwidgets"
//resources = Seq(workDirectory / "multiobjective/results/ConstrEx")
) set (
inputFiles += (directoryWithResults, "mydirectory"),
outputFiles += ("/tmp/last_pareto", lastPareto),
inputs += filesHaveHeaders.mapped,
inputs += countInputs.mapped,
inputs += graphWidth.mapped,
inputs += graphHeight.mapped,
inputs += graphFormat.mapped,
filesHaveHeaders := 1,
countInputs := 2,
// in pixels
graphWidth := 600,
graphHeight := 600,
graphFormat := "png"
)
// import from the other file an example of optimization
import _file_.example_of_optimization._
// run the optimization
( evolutionSchafferN2 on envMultiThread hook (workDirectory/relativePath, keepAll=false)
) -- (
// then run the plotting function
taskPlotLastParetoFront set (
// ... which will read all the results from this file
directoryWithResults := workDirectory / relativePath,
// ... analyze them knowing there is only one input in the files
countInputs := 1,
// ... parameters of the graph
graphWidth := 500,
graphHeight := 500
) hook CopyFileHook(lastPareto, workDirectory/"last Pareto front.png" )
)
\ No newline at end of file
// this variable will transmit the path where the CSV files to graph will be found
val directoryWithResults = Val[File]
// variables used to parameter the graphing function
val filesHaveHeaders = Val[Int]
val countInputs = Val[Int]
// this variable will contain the file with the graphical rendering of the last PAreto front
val pngForPairs = Val[File]
val taskPlotEvereyIteration = RTask("""
library(ggplot2)
library(GGally)
colnames <- if (countInputs == 2) c("iteration", "x", "y", "f1", "f2") else c("iteration", "x", "f1", "f2")
coltypes <- if (countInputs == 2) c("integer", "numeric", "numeric", "numeric", "numeric") else c("integer", "numeric", "numeric", "numeric")
names(coltypes) <- colnames
directoryWithResultsName <- "mydirectory"
pop <- NULL
i <- 1
while (TRUE) {
# TODO check creation time of the file
filename <- paste(directoryWithResultsName,"/population",i,".csv", sep="");
if (!file.exists(filename)) {
break
}
#print(filename)
popraw <- read.csv(header = FALSE, col.names=colnames, colClasses=coltypes, file=filename)
#print(head(popraw))
pop <- if (is.null(pop)) popraw else rbind(pop, popraw)
i <- i + 1
}
# the ggplot
n <- ncol(pop)
ggsave(filename="/tmp/pairs.png", ggpairs(pop), width=n*6, height=n*6, dpi = 150, units="cm", limitsize=F)
""",
install = Seq(
"fakeroot sed -i 's/deb.debian.org/linux.pleiade.edf.fr/g' /etc/apt/sources.list",
"fakeroot cat /etc/apt/sources.list",
// update the list of available packages
"fakeroot apt-get -o Acquire::http::proxy=false update ",
// required; attempts to update dbus to a newer version would require permissions we do not have
"DEBIAN_FRONTEND=noninteractive fakeroot apt-mark hold dbus",
"""echo "dbus hold" | fakeroot dpkg --set-selections""",
// install the libs required for the compilation of R packages
"DEBIAN_FRONTEND=noninteractive fakeroot apt-get -o Acquire::http::proxy=false install -y libssl-dev libcurl4-openssl-dev libudunits2-dev",
// install required R packages in their binary version (quicker, much stable!)
"DEBIAN_FRONTEND=noninteractive fakeroot apt-get -o Acquire::http::proxy=false install -y r-cran-ggplot2 r-cran-gganimate r-cran-ggally r-cran-plotly r-cran-zip",
// install external tools in the VM for rendering
"DEBIAN_FRONTEND=noninteractive fakeroot apt-get -o Acquire::http::proxy=false install -y ffmpeg",
), //
libraries = Seq() // "ggplot2","gganimate","plotly","GGally","htmlwidgets"
//resources = Seq(workDirectory / "multiobjective/results/ConstrEx")
) set (
inputFiles += (directoryWithResults, "mydirectory"),
outputFiles += ("/tmp/pairs.png", pngForPairs),
inputs += filesHaveHeaders.mapped,
inputs += countInputs.mapped,
filesHaveHeaders := 1,
countInputs := 2
)
// import from the other file an example of optimization
import _file_.example_of_optimization._
// run the optimization
( evolutionSchafferN2 on envMultiThread hook (workDirectory/relativePath, keepAll=false)
) -- (
// then run the plotting function
taskPlotEvereyIteration set (
// ... which will read all the results from this file
directoryWithResults := workDirectory / relativePath,
// ... analyze them knowing there is only one input in the files
countInputs := 1
) hook CopyFileHook(pngForPairs, workDirectory/"pairs.png" )
)
\ No newline at end of file
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment