Commit c5ecef5d by Sam Thiriot

### first examples

parent 6f7e6505
 # Test Functions for NSGA2 ## Why Test? Metaheuristics for optimization such as genetic algorithms leave a huge freedom to developers for their implementation. Genetic Algorithms such as NSGA2 are even less trivial. Their implementation in the [mgo](https://github.com/openmole/mgo) used in [OpenMole](https://openmole.org/) was adapted so they are efficient for computation on clusters or grids. This leaves a lot of uncertainty to the user on what really happens. Why should you, as a user, trust this implementation? The answer is simple: you should not. Any implementation in scientific computing should be verified to be trusted. ## Test Functions Many functions named "Test Functions" were proposed over time to test and compare multi-objective optimization algorithms. The [wikipedia page](https://en.wikipedia.org/wiki/Test_functions_for_optimization) lists several of them. A test function is an function you can use as the problem to optimize, for which the expected results are known. Test functions might test: - problems with multiple parameters and multiple objectives - problems with constraints: a part of the space of solutions can not be computed, so the algorithm should achieve to deal with it - problems with parts of the space of solution that are tricky to detect, either because they are statistically unlikely to find, or because they are in a part of the space of solutions which is heavily constrained, etc. The results of a test function are usually compared in both terms of: - coverage of the Pareto front, - speed of convergence ## Test and Learn Test functions have another role: they constitute a simple example of optimization problem. As a user, you can also learn to tune the parameters of the genetic algorithm as check when convergence occurs. ## Test Workflows for OpenMole We provide here a few examples of test functions which you can open with OpenMole. - ConstrEx - CP1 - Schaffer N1 - Schaffer N2 To run a test - Choose one of the workflows starting with "test function" - Run the workflow - Update the view on the left using the "refresh" button - Download the graph of the last Pareto front, and compare it with the literature (you might use the [wikipedia page](https://en.wikipedia.org/wiki/Test_functions_for_optimization) as a start) You might then tune the parameters of the optimization algorithm and analyze the results, to understand how to better use the optimization method.
 //model inputs val x = Val[Double] //model outputs val f1 = Val[Double] val f2 = Val[Double] // about the current experiment val relativePath = "results/SchafferN2" // the test function val testFunctionSchafferN2= ScalaTask(""" val f1 = if (x <= 1) { -x*1.0 } else if (x <= 3) { x - 2.0 } else if (x <= 4) { 4.0 - x } else { x - 4.0 } val f2 = Math.pow(x - 5, 2); """) set ( inputs += x, outputs += f1, outputs += f2 ) // the optimisation algorithm under test val evolutionSchafferN2 = NSGA2Evolution( genome = Seq( x in (-5.0, 10.0) ), objective = Seq(f1, f2), evaluation = testFunctionSchafferN2, parallelism = 8, termination = 1000 ) val envMultiThread = LocalEnvironment(4, name="multithread") // compute evolution on the test Function (evolutionSchafferN2 on envMultiThread hook (workDirectory/relativePath, keepAll=false) // then plot the last Pareto front ) -- (taskPlotLastParetoFront set ( directoryWithResults := workDirectory/relativePath, countInputs := 1) hook CopyFileHook(last_pareto, workDirectory/"last Pareto front SchafferN2.png" ) )
 // this variable will transmit the path where the CSV files to graph will be found val directoryWithResults = Val[File] // variables used to parameter the graphing function val filesHaveHeaders = Val[Int] val countInputs = Val[Int] val graphWidth = Val[Int] val graphHeight = Val[Int] val framerate = Val[Int] // this variable will contain the file with the graphical rendering of the last PAreto front val video = Val[File] val taskPlotAsVideo = RTask(""" library(ggplot2) library(gganimate) colnames <- if (countInputs == 2) c("iteration", "x", "y", "f1", "f2") else c("iteration", "x", "f1", "f2") coltypes <- if (countInputs == 2) c("integer", "numeric", "numeric", "numeric", "numeric") else c("integer", "numeric", "numeric", "numeric") names(coltypes) <- colnames directoryWithResultsName <- "mydirectory" pop <- NULL i <- 1 while (TRUE) { # TODO check creation time of the file filename <- paste(directoryWithResultsName,"/population",i,".csv", sep=""); if (!file.exists(filename)) { break } #print(filename) popraw <- read.csv(header = FALSE, col.names=colnames, colClasses=coltypes, file=filename) #print(head(popraw)) pop <- if (is.null(pop)) popraw else rbind(pop, popraw) i <- i + 1 } # the ggplot p <- ggplot(pop, aes(x=f1,y=f2)) + geom_point() # render with gganimate gganimation <- p + transition_states(iteration) + transition_time(iteration) #+ labs(title="iteration: {iteration}") # ... first render individual PNG frames which are always of use animate(gganimation, renderer=file_renderer("/tmp/rendered", overwrite=T, prefix="iteration"), height=graphHeight, width=graphWidth, device='png') # render as mp4 print("rendering as a video") system(paste("ffmpeg -y -framerate", framerate, "-i /tmp/rendered/iteration%04d.png -c:v libx264 -r", framerate," /tmp/render.mp4")) """, install = Seq( "fakeroot sed -i 's/deb.debian.org/linux.pleiade.edf.fr/g' /etc/apt/sources.list", "fakeroot cat /etc/apt/sources.list", // update the list of available packages "fakeroot apt-get -o Acquire::http::proxy=false update ", // required; attempts to update dbus to a newer version would require permissions we do not have "DEBIAN_FRONTEND=noninteractive fakeroot apt-mark hold dbus", """echo "dbus hold" | fakeroot dpkg --set-selections""", // install the libs required for the compilation of R packages "DEBIAN_FRONTEND=noninteractive fakeroot apt-get -o Acquire::http::proxy=false install -y libssl-dev libcurl4-openssl-dev libudunits2-dev", // install required R packages in their binary version (quicker, much stable!) "DEBIAN_FRONTEND=noninteractive fakeroot apt-get -o Acquire::http::proxy=false install -y r-cran-ggplot2 r-cran-gganimate r-cran-ggally r-cran-plotly r-cran-zip", // install external tools in the VM for rendering "DEBIAN_FRONTEND=noninteractive fakeroot apt-get -o Acquire::http::proxy=false install -y ffmpeg", ), // libraries = Seq() // "ggplot2","gganimate","plotly","GGally","htmlwidgets" //resources = Seq(workDirectory / "multiobjective/results/ConstrEx") ) set ( inputFiles += (directoryWithResults, "mydirectory"), outputFiles += ("/tmp/render.mp4", video), inputs += filesHaveHeaders.mapped, inputs += countInputs.mapped, inputs += graphWidth.mapped, inputs += graphHeight.mapped, inputs += framerate.mapped, filesHaveHeaders := 1, countInputs := 2, graphWidth := 600, graphHeight := 600, framerate := 5, ) // import from the other file an example of optimization import _file_.example_of_optimization._ // run the optimization ( evolutionSchafferN2 on envMultiThread hook (workDirectory/relativePath, keepAll=false) ) -- ( // then run the plotting function taskPlotAsVideo set ( // ... which will read all the results from this file directoryWithResults := workDirectory / relativePath, // ... analyze them knowing there is only one input in the files countInputs := 1 ) hook CopyFileHook(video, workDirectory/"iterations_video.mp4" ) ) \ No newline at end of file
 // this variable will transmit the path where the CSV files to graph will be found val directoryWithResults = Val[File] // variables used to parameter the graphing function val filesHaveHeaders = Val[Int] val countInputs = Val[Int] // this variable will contain the file with the graphical rendering of the last PAreto front val plotlyPage = Val[File] val taskPlotAsVideo = RTask(""" library(ggplot2) library(plotly) library(htmlwidgets) colnames <- if (countInputs == 2) c("iteration", "x", "y", "f1", "f2") else c("iteration", "x", "f1", "f2") coltypes <- if (countInputs == 2) c("integer", "numeric", "numeric", "numeric", "numeric") else c("integer", "numeric", "numeric", "numeric") names(coltypes) <- colnames directoryWithResultsName <- "mydirectory" pop <- NULL i <- 1 while (TRUE) { # TODO check creation time of the file filename <- paste(directoryWithResultsName,"/population",i,".csv", sep=""); if (!file.exists(filename)) { break } #print(filename) popraw <- read.csv(header = FALSE, col.names=colnames, colClasses=coltypes, file=filename) #print(head(popraw)) pop <- if (is.null(pop)) popraw else rbind(pop, popraw) i <- i + 1 } # the ggplot pop\$index <- 1:nrow(pop) # add an index to avoid animation of points which are not at all similar p <- ggplot(pop, aes(x=f1,y=f2,l1=x) ) + geom_point( alpha=0.7, colour = "#51A0D5", aes(frame=iteration,ids=index) ) + labs( x = "f1", y = "f2", title = "Evolution of the Pareto front with NSGA2" ) + theme_classic() # render as plotly dir.create("/tmp/plotly") fig <- ggplotly(p) saveWidget(fig, selfcontained=F, file = "/tmp/plotly/evolution.html") lf <- list.files("/tmp/plotly/", recursive=T, include.dirs=F) print(lf) # zip resulting files library(zip) setwd("/tmp/plotly/") zip("/tmp/evolution.zip", lf) print("done") """, install = Seq( "fakeroot sed -i 's/deb.debian.org/linux.pleiade.edf.fr/g' /etc/apt/sources.list", "fakeroot cat /etc/apt/sources.list", // update the list of available packages "fakeroot apt-get -o Acquire::http::proxy=false update ", // required; attempts to update dbus to a newer version would require permissions we do not have "DEBIAN_FRONTEND=noninteractive fakeroot apt-mark hold dbus", """echo "dbus hold" | fakeroot dpkg --set-selections""", // install the libs required for the compilation of R packages "DEBIAN_FRONTEND=noninteractive fakeroot apt-get -o Acquire::http::proxy=false install -y libssl-dev libcurl4-openssl-dev libudunits2-dev", // install required R packages in their binary version (quicker, much stable!) "DEBIAN_FRONTEND=noninteractive fakeroot apt-get -o Acquire::http::proxy=false install -y r-cran-ggplot2 r-cran-gganimate r-cran-ggally r-cran-plotly r-cran-zip", // install required R packages in their binary version (quicker, much stable!) "DEBIAN_FRONTEND=noninteractive fakeroot apt-get -o Acquire::http::proxy=false install -y ffmpeg", ), // libraries = Seq() // "ggplot2","gganimate","plotly","GGally","htmlwidgets" //resources = Seq(workDirectory / "multiobjective/results/ConstrEx") ) set ( inputFiles += (directoryWithResults, "mydirectory"), outputFiles += ("/tmp/evolution.zip", plotlyPage), inputs += filesHaveHeaders.mapped, inputs += countInputs.mapped, filesHaveHeaders := 1, countInputs := 2 ) // import from the other file an example of optimization import _file_.example_of_optimization._ // run the optimization ( evolutionSchafferN2 on envMultiThread hook (workDirectory/relativePath, keepAll=false) ) -- ( // then run the plotting function taskPlotAsVideo set ( // ... which will read all the results from this file directoryWithResults := workDirectory / relativePath, // ... analyze them knowing there is only one input in the files countInputs := 1 ) hook CopyFileHook(plotlyPage, workDirectory/"evolutionPlotly.zip" ) ) \ No newline at end of file
 // this variable will transmit the path where the CSV files to graph will be found val directoryWithResults = Val[File] // variables used to parameter the graphing function val filesHaveHeaders = Val[Int] val countInputs = Val[Int] val graphWidth = Val[Int] val graphHeight = Val[Int] val graphFormat = Val[String] // this variable will contain the file with the graphical rendering of the last PAreto front val pngForIterations = Val[File] val taskPlotEvereyIteration = RTask(""" library(ggplot2) library(gganimate) colnames <- if (countInputs == 2) c("iteration", "x", "y", "f1", "f2") else c("iteration", "x", "f1", "f2") coltypes <- if (countInputs == 2) c("integer", "numeric", "numeric", "numeric", "numeric") else c("integer", "numeric", "numeric", "numeric") names(coltypes) <- colnames directoryWithResultsName <- "mydirectory" pop <- NULL i <- 1 while (TRUE) { # TODO check creation time of the file filename <- paste(directoryWithResultsName,"/population",i,".csv", sep=""); if (!file.exists(filename)) { break } #print(filename) popraw <- read.csv(header = FALSE, col.names=colnames, colClasses=coltypes, file=filename) #print(head(popraw)) pop <- if (is.null(pop)) popraw else rbind(pop, popraw) i <- i + 1 } # the ggplot p <- ggplot(pop, aes(x=f1,y=f2)) + geom_point() # render with gganimate gganimation <- p + transition_states(iteration) + transition_time(iteration)# + labs(title="iteration: {iteration}") # ... first render individual PNG frames which are always of use animate(gganimation, renderer=file_renderer("/tmp/rendered", overwrite=T, prefix="iteration"), height=graphHeight, width=graphWidth, device=graphFormat) """, install = Seq( "fakeroot sed -i 's/deb.debian.org/linux.pleiade.edf.fr/g' /etc/apt/sources.list", "fakeroot cat /etc/apt/sources.list", // update the list of available packages "fakeroot apt-get -o Acquire::http::proxy=false update ", // required; attempts to update dbus to a newer version would require permissions we do not have "DEBIAN_FRONTEND=noninteractive fakeroot apt-mark hold dbus", """echo "dbus hold" | fakeroot dpkg --set-selections""", // install the libs required for the compilation of R packages "DEBIAN_FRONTEND=noninteractive fakeroot apt-get -o Acquire::http::proxy=false install -y libssl-dev libcurl4-openssl-dev libudunits2-dev", // install required R packages in their binary version (quicker, much stable!) "DEBIAN_FRONTEND=noninteractive fakeroot apt-get -o Acquire::http::proxy=false install -y r-cran-ggplot2 r-cran-gganimate r-cran-ggally r-cran-plotly r-cran-zip", // install external tools in the VM for rendering "DEBIAN_FRONTEND=noninteractive fakeroot apt-get -o Acquire::http::proxy=false install -y ffmpeg", ), // libraries = Seq() // "ggplot2","gganimate","plotly","GGally","htmlwidgets" //resources = Seq(workDirectory / "multiobjective/results/ConstrEx") ) set ( inputFiles += (directoryWithResults, "mydirectory"), outputFiles += ("/tmp/rendered", pngForIterations), inputs += filesHaveHeaders.mapped, inputs += countInputs.mapped, inputs += graphWidth.mapped, inputs += graphHeight.mapped, inputs += graphFormat.mapped, filesHaveHeaders := 1, countInputs := 2, graphWidth := 600, graphHeight := 600, graphFormat := "png" ) // import from the other file an example of optimization import _file_.example_of_optimization._ // run the optimization ( evolutionSchafferN2 on envMultiThread hook (workDirectory/relativePath, keepAll=false) ) -- ( // then run the plotting function taskPlotEvereyIteration set ( // ... which will read all the results from this file directoryWithResults := workDirectory / relativePath, // ... analyze them knowing there is only one input in the files countInputs := 1 ) hook CopyFileHook(pngForIterations, workDirectory/"iterations as graphs" ) ) \ No newline at end of file
 // this variable will transmit the path where the CSV files to graph will be found val directoryWithResults = Val[File] // variables used to parameter the graphing function val filesHaveHeaders = Val[Int] val countInputs = Val[Int] val graphWidth = Val[Int] val graphHeight = Val[Int] val graphFormat = Val[String] // this variable will contain the file with the graphical rendering of the last PAreto front val lastPareto = Val[File] val taskPlotLastParetoFront = RTask(""" library(ggplot2) colnames <- if (countInputs == 2) c("iteration", "x", "y", "f1", "f2") else c("iteration", "x", "f1", "f2") coltypes <- if (countInputs == 2) c("integer", "numeric", "numeric", "numeric", "numeric") else c("integer", "numeric", "numeric", "numeric") names(coltypes) <- colnames directoryWithResultsName <- "mydirectory" # ensure check the directory exists if (!file.exists(directoryWithResultsName)) { stop(paste("ERROR: the directory", directoryWithResultsName, "does not exists!")) } # get the most recent file (will be the last result) allfiles <- file.info(list.files(directoryWithResultsName, full.names = T)) lastfilename <- rownames(allfiles)[which.max(allfiles\$mtime)] if (!file.exists(lastfilename)) { stop(paste("ERROR: no file found in", directoryWithResultsName) ) } print(lastfilename) # read the last file pop <- read.csv(header = filesHaveHeaders>0, col.names=colnames, colClasses=coltypes, file=lastfilename) print(paste("there are", nrow(pop), "points on the last Pareto front")) # plot g <- ggplot(pop, aes(x=f1,y=f2)) + geom_point() dpi <- 72 ggsave(filename="/tmp/last_pareto", device=graphFormat, plot=g, width=graphWidth/dpi, height=graphHeight/dpi) """, install = Seq( "fakeroot sed -i 's/deb.debian.org/linux.pleiade.edf.fr/g' /etc/apt/sources.list", "fakeroot cat /etc/apt/sources.list", // update the list of available packages "fakeroot apt-get -o Acquire::http::proxy=false update ", // required; attempts to update dbus to a newer version would require permissions we do not have "DEBIAN_FRONTEND=noninteractive fakeroot apt-mark hold dbus", """echo "dbus hold" | fakeroot dpkg --set-selections""", // install the libs required for the compilation of R packages "DEBIAN_FRONTEND=noninteractive fakeroot apt-get -o Acquire::http::proxy=false install -y libssl-dev libcurl4-openssl-dev libudunits2-dev", // install required R packages in their binary version (quicker, much stable!) "DEBIAN_FRONTEND=noninteractive fakeroot apt-get -o Acquire::http::proxy=false install -y r-cran-ggplot2 r-cran-gganimate r-cran-ggally r-cran-plotly r-cran-zip", // install external tools in the VM for rendering "DEBIAN_FRONTEND=noninteractive fakeroot apt-get -o Acquire::http::proxy=false install -y ffmpeg", ), // libraries = Seq() // "ggplot2","gganimate","plotly","GGally","htmlwidgets" //resources = Seq(workDirectory / "multiobjective/results/ConstrEx") ) set ( inputFiles += (directoryWithResults, "mydirectory"), outputFiles += ("/tmp/last_pareto", lastPareto), inputs += filesHaveHeaders.mapped, inputs += countInputs.mapped, inputs += graphWidth.mapped, inputs += graphHeight.mapped, inputs += graphFormat.mapped, filesHaveHeaders := 1, countInputs := 2, // in pixels graphWidth := 600, graphHeight := 600, graphFormat := "png" ) // import from the other file an example of optimization import _file_.example_of_optimization._ // run the optimization ( evolutionSchafferN2 on envMultiThread hook (workDirectory/relativePath, keepAll=false) ) -- ( // then run the plotting function taskPlotLastParetoFront set ( // ... which will read all the results from this file directoryWithResults := workDirectory / relativePath, // ... analyze them knowing there is only one input in the files countInputs := 1, // ... parameters of the graph graphWidth := 500, graphHeight := 500 ) hook CopyFileHook(lastPareto, workDirectory/"last Pareto front.png" ) ) \ No newline at end of file
 // this variable will transmit the path where the CSV files to graph will be found val directoryWithResults = Val[File] // variables used to parameter the graphing function val filesHaveHeaders = Val[Int] val countInputs = Val[Int] // this variable will contain the file with the graphical rendering of the last PAreto front val pngForPairs = Val[File] val taskPlotEvereyIteration = RTask(""" library(ggplot2) library(GGally) colnames <- if (countInputs == 2) c("iteration", "x", "y", "f1", "f2") else c("iteration", "x", "f1", "f2") coltypes <- if (countInputs == 2) c("integer", "numeric", "numeric", "numeric", "numeric") else c("integer", "numeric", "numeric", "numeric") names(coltypes) <- colnames directoryWithResultsName <- "mydirectory" pop <- NULL i <- 1 while (TRUE) { # TODO check creation time of the file filename <- paste(directoryWithResultsName,"/population",i,".csv", sep=""); if (!file.exists(filename)) { break } #print(filename) popraw <- read.csv(header = FALSE, col.names=colnames, colClasses=coltypes, file=filename) #print(head(popraw)) pop <- if (is.null(pop)) popraw else rbind(pop, popraw) i <- i + 1 } # the ggplot n <- ncol(pop) ggsave(filename="/tmp/pairs.png", ggpairs(pop), width=n*6, height=n*6, dpi = 150, units="cm", limitsize=F) """, install = Seq( "fakeroot sed -i 's/deb.debian.org/linux.pleiade.edf.fr/g' /etc/apt/sources.list", "fakeroot cat /etc/apt/sources.list", // update the list of available packages "fakeroot apt-get -o Acquire::http::proxy=false update ", // required; attempts to update dbus to a newer version would require permissions we do not have "DEBIAN_FRONTEND=noninteractive fakeroot apt-mark hold dbus", """echo "dbus hold" | fakeroot dpkg --set-selections""", // install the libs required for the compilation of R packages "DEBIAN_FRONTEND=noninteractive fakeroot apt-get -o Acquire::http::proxy=false install -y libssl-dev libcurl4-openssl-dev libudunits2-dev", // install required R packages in their binary version (quicker, much stable!) "DEBIAN_FRONTEND=noninteractive fakeroot apt-get -o Acquire::http::proxy=false install -y r-cran-ggplot2 r-cran-gganimate r-cran-ggally r-cran-plotly r-cran-zip", // install external tools in the VM for rendering "DEBIAN_FRONTEND=noninteractive fakeroot apt-get -o Acquire::http::proxy=false install -y ffmpeg", ), // libraries = Seq() // "ggplot2","gganimate","plotly","GGally","htmlwidgets" //resources = Seq(workDirectory / "multiobjective/results/ConstrEx") ) set ( inputFiles += (directoryWithResults, "mydirectory"), outputFiles += ("/tmp/pairs.png", pngForPairs), inputs += filesHaveHeaders.mapped, inputs += countInputs.mapped, filesHaveHeaders := 1, countInputs := 2 ) // import from the other file an example of optimization import _file_.example_of_optimization._ // run the optimization ( evolutionSchafferN2 on envMultiThread hook (workDirectory/relativePath, keepAll=false) ) -- ( // then run the plotting function taskPlotEvereyIteration set ( // ... which will read all the results from this file directoryWithResults := workDirectory / relativePath, // ... analyze them knowing there is only one input in the files countInputs := 1 ) hook CopyFileHook(pngForPairs, workDirectory/"pairs.png" ) ) \ No newline at end of file
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!