Commit a50f1093 authored by Hélène Arduin's avatar Hélène Arduin
Browse files

[Doc] update Developers pages

parent 15e287f7
@import org.openmole.site.tools._
@import org.openmole.site._
@import DocumentationPages._
@aa("OpenMOLE GUI", href := DocumentationPages.gui.file) implements all the OpenMOLE features. However you may be interested in using OpenMOLE in interactive console mode.
To do so, use the @i("-c") argument in your console : @hl("""./openmole -c""", "plain")
The @aa("OpenMOLE GUI", href := gui.file) implements all the OpenMOLE features.
However you may be interested in using OpenMOLE in interactive console mode.
To do so, use the @code{-c} argument in your console: @code{./openmole -c}.
@br
The only difference between the script in the console mode and the ones from the editor concerns the way you launch the execution, you cancel it and you follow the execution progress. An console workflow is launched like this:
@br @hl.code("""val ex = exploration -< (model on env) start""")
The only difference between the scripts in the console mode and the ones from the editor is the way you launch their execution, you cancel it, and you follow their progress.
A console workflow is launched as follow:
@hl.code("""val ex = exploration -< (model on env) start""")
Using the @code{ex} and the @code{env} variables, you can follow the progress of the execution by using the commands: @code{print(ex)} and @code{print(env)}.
To cancel the execution, you should use @code{ex.cancel}.
@p Using the ex and the env variables you can follow the progress of the execution by using the commands: @hl("print(ex)", "plain") and @hl("print(env)", "plain"). To cancel the execution you should use: @hl("ex.cancel", "plain").
@h2{Authentications}
In console mode, you can define an authentication using a pair of login / password with the following command:
@br @hl.openmole("""
SSHAuthentication += LoginPassword("login", encrypted, "machine-name")""".stripMargin, header = """def encrypted = "" """)
In console mode, you can define an authentication using a pair of login/password with the following command:
@hl.openmole("""
SSHAuthentication += LoginPassword("login", encrypted, "machine-name")
""".stripMargin, header = """def encrypted = "" """)
Or, to authenticate with a private key:
@hl.openmole("""
SSHAuthentication += PrivateKey("/path/to/the/private/key", "login", encrypted, "machine-name")
""", header = """def encrypted = "" """)
@p Or to authenticate with a private key:
@br @hl.openmole("""
SSHAuthentication += PrivateKey("/path/to/the/private/key", "login", encrypted, "machine-name")""", header = """def encrypted = "" """)
It mentions the @i{encrypted} function.
This function will prompt for the password/passphrase of the private key, right after the call to the @code{Environment} builder, using @code{SSHAuthentication}.
@br
The last part of @code{SSHAuthentication}, @code{machine-name}, should match exactly the address of the machine in your execution environment.
OpenMOLE searches the matching SSH keys using an @b{exact match} on @code{login} and @code{machine-name} between the environment and the stored keys.
@p It mentions the @i{encrypted} function. This function will prompt for the password/passphrase of the private key right after the call to the builder of the @i{Environment} using this @hl.openmole("SSHAuthentication").
@p The last part of the @hl.openmole("SSHAuthentication"): "machine-name" should match exactly the address of the machine in your execution environment. OpenMOLE searches the matching ssh keys using an @b{exact match} on @i{login} and @i{machine-name} between the environment and the stored keys.
@h2{Run script}
In console mode, you have to copy-paste your whole workflow to run it.
The only difference between the script in the console mode and the ones from the editor concerns the way you launch the execution,
you cancel it and you follow the execution progress.
A console workflow is launched like this:
@hl.code("""
val ex = exploration -< (model on env) start""")
val ex = exploration -< (model on env) start
""")
Note that you need to invoke the @hl("start", "plain") on your workflow, in contrary to Editor mode.
Note that you need to invoke @code{start} on your workflow, contrary to Editor mode.
@break
Using the @i{ex} and the @i{env} variables you can follow the progress of the execution by using the commands:
@hl("print(ex)", "plain") and @hl("print(env)","plain").
@break
@br
To cancel the execution you should use: @hl("ex.cancel", "plain").
Using the @code{ex} and the @code{env} variables you can follow the progress of the execution by using the commands:
@code{print(ex)} and @code{print(env)}.
To cancel the execution you should use @code{ex.cancel}.
......@@ -3,13 +3,14 @@
@import org.openmole.site._
@import org.openmole.site.tools._
@import org.openmole.site.content.Environment._
@import DocumentationPages._
In this section you will find information on:
@ul
@li{How to use OpenMOLE in interactive @a("console mode", href := DocumentationPages.console.file)}
@li{How to develop your own @a("plugin", href := DocumentationPages.pluginDevelopment.file) to have full control over your code, and how to integrate it in OpenMOLE: }
@li{How to develop an OpenMOLE extension @a("extension", href := DocumentationPages.extensionAPI.file) }
@li{How to use the OpenMOLE @a("REST API", href := DocumentationPages.restAPI.file)}
@li{How to compile and modify documentation @a("documentation", href := DocumentationPages.documentationGen.file)}
@li{How to use OpenMOLE in interactive @a("console mode", href := console.file)}
@li{How to develop your own @a("plugin", href := pluginDevelopment.file) to have full control over your code, and how to integrate it in OpenMOLE}
@li{How to develop an OpenMOLE extension @a("extension", href := extensionAPI.file) }
@li{How to use the OpenMOLE @a("REST API", href := restAPI.file)}
@li{How to compile and modify the @a("documentation", href := documentationGen.file)}
@import org.openmole.site.tools._
@import org.openmole.site._
@import DocumentationPages._
@h2{Get ready}
You will need the following tools to design your plugin:
@ul
@li
The @aa("git", href := shared.link.git) software.
The @aa("git", href := shared.link.git) software
@li
@aa("SBT", href := shared.link.sbt), the Scala Building Tool.
@aa("SBT", href := shared.link.sbt), the Scala Building Tool
The first step is to clone the github repository for OpenMOLE :
......@@ -19,8 +21,8 @@ The first step is to clone the github repository for OpenMOLE :
git clone https://gitlab.openmole.org/openmole/openmole.git
""")
In order to generate documentation with scalaTex, openMole need some libraries.
Into your recent openmole cloned folder, call these three commands to compile libraries :
In order to generate documentation with scalaTex, openMOLE need some libraries.
Into your recently cloned openmole folder, call these three commands to compile libraries:
@hl.code("""
git lfs fetch
......@@ -28,9 +30,13 @@ git lfs fetch
(cd libraries && sbt publishLocal)
""")
@h2{Compiling documentation}
You're now ready to compile the actual documentation. Move into openmole (nested) folder and run sbt to generate website :
You're now ready to compile the actual documentation.
Move into the nested openmole folder and run sbt to generate the website:
@hl.code("""
cd openmole
sbt buildSite
......@@ -38,15 +44,17 @@ sbt buildSite
The generated site is visible by opening @code{openmole/openmole/bin/org.openmole.site/jvm/target/site/index.html} in your browser !
@h2{Adding new page}
Scalatex file are located into bin folder @code{bin/org.openmole.site/jvm/src/main/scalatex/openmole} :
@h2{Adding a new page}
Scalatex file are located into the bin folder @code{bin/org.openmole.site/jvm/src/main/scalatex/openmole}:
@hl.plain("""
cd openmole/openmole/bin/org.openmole.site/jvm/src/main/scalatex/openmole
""")
For this example, we try to add this current page "Documentation Generation" to @a("Developpers", href := "https://openmole.org/Developers.html").
For this example, we try to add this current page "Documentation Generation" to @a("Developers", href := developers.file).
Into your favorite IDE :
@ul
......@@ -57,18 +65,18 @@ For this example, we try to add this current page "Documentation Generation" to
lazy val compileDocumentation = DocumentationPage.fromScalatex(name = "Documentation generation", content = scalatex.documentation.developers.DocumentationGen)
""")
After that, we add @code{compileDocumentation} to @code{pageNode} that corresponding to developers entry on website :
After that, we add @code{compileDocumentation} to the @code{pageNode} corresponding to the "Developers" entry on the website:
@hl.code("""
def developersPages = pageNode(developers, Vector(console, pluginDevelopment, extensionAPI, restAPI, documentationGen))
""")
Now, leave the @code{Pages.scala} file and create a new scalatex file @code{Developers.scalatex} into @code{/openmole/community/documentation/developers/} folder.
Now, leave the @code{Pages.scala} file and create a new scalatex file @code{documentaionGen.scalatex} into @code{/openmole/community/documentation/developers/}.
We add a new link to list of pages using this code :
We add a new link to the list of pages using:
@hl.code("""
@li{How to compile and modify documentation @a("documentation", href := DocumentationPages.documentationGen.file)}
@li{How to compile and modify the @a("documentation", href := DocumentationPages.documentationGen.file)}
""")
@h2{Cheatsheet Scalatex}
......@@ -4,30 +4,32 @@
@import org.openmole.site.tools._
While OpenMOLE's core code is not intended to be directly accessible to most users, an easy-to-use transparent and flexible API has
been developed for simple development of user extensions to the core. This API allows the user to implement new tasks, methods, samplings.
While OpenMOLE's core code is not intended to be directly accessible to most users, an easy-to-use transparent and flexible API has been developed for simple development of user extensions to the core.
This API allows the user to implement new tasks, methods, and samplings.
@h2{Concepts of the API}
The primitives for the API are imported by importing contents of package @code{org.openmole.core.dsl.extension}.
@h2{Concepts of the API}
These primitive provide constructors for
The primitives for the API are imported through the contents of the package @code{org.openmole.core.dsl.extension}.
These primitive provide constructors for:
@ul
@li{tasks}
@li{samplings}
@li{hooks}
@h2{Task extensions}
To define a new task, use @code{Task(name: String)(process: FromContext => Context)}. What the task does is defined by the provided closure, which transforms a @code{FromContext} into a @code{Context}.
@h2{Task extensions}
To define a new task, use @code{Task(name: String)(process: FromContext => Context)}.
What the task does is defined by the provided closure, which transforms a @code{FromContext} into a @code{Context}.
You can add implicits in your @code{apply} method to get advanced services (mole services, network services, etc.).
Validation is provided with the @code{validate} method which transforms validation parameters into a sequence of throwables.
@br
For example
Validation is provided with the @code{validate} method which transforms validation parameters into a sequence of throwables.
For example:
@hl.code("""
object MyTask {
......@@ -46,10 +48,8 @@ For example
@h2{Sampling extensions}
To implement a sampling, the constructor @code{Sampling} takes a function transforming a @code{FromContext} into a sampling results,
which is an @code{Iterator[Iterable[Variable[_]]]}.
For example the following sampling assigns uniformally a sequence of doubles to some prototypes :
To implement a sampling, the constructor @code{Sampling} takes a function transforming a @code{FromContext} into a sampling result, which is an @code{Iterator[Iterable[Variable[_]]]}.
For example, the following sampling assigns uniformally a sequence of doubles to some prototypes :
@hl.code("""
object MySampling {
......@@ -62,28 +62,26 @@ For example the following sampling assigns uniformally a sequence of doubles to
@h2{Integrating your extension into the build}
OpenMOLE uses OSGI bundles to integrate its different components into a single application. This allows for example loading bundle in a dynamical way, what is done when adding user plugins through the GUI.
OpenMOLE uses OSGI bundles to integrate its different components into a single application.
This allows to load bundles dynamically for instance, which is done when adding user plugins through the GUI.
To integrate your module into the build process of OpenMOLE, several steps have thus to be performed:
@ul
@li{Add your module as an Osgi project into the main build file @b{build.sbt}. For example, in the case of a new sampling, the syntax would be similar to the sobol sampling: @hl.code("""
@li{Add your module as an OSGI project into the main build file @b{build.sbt}. For example, in the case of a new sampling, the syntax would be similar to the Sobol sampling: @hl.code("""
lazy val quasirandomSampling = OsgiProject(pluginDir, "org.openmole.plugin.sampling.quasirandom", imports = Seq("*")) dependsOn(exception, workflow, workspace, openmoleDSL) settings (
libraryDependencies += Libraries.math
) settings (pluginSettings: _*)
"""). Note that you need to specify the bundles you use in your code, and the external libraries dependencies.
You also need to add your module to the integrated ones, by adding the project to @code{allSamplings} for example.
If your library dependency requires specific resolvers, these can be added to @code{defaultSettings} or only locally to your project.
If your library dependency requires specific resolvers, these can be added to @code{defaultSettings}, or only locally to your project.
}
@li{Add your library dependencies to the @code{Libraries} object in the file @b{openmole/project/Libraries.scala}. For the code above, you would have to specify the @code{Libraries.math} value, which in this particular case is the apache common math library: @code{lazy val math = "org.openmole.library" %% "org-apache-commons-math" % "3.6.1"}. The library is defined as a bundle under the organisation org.openmole.library. The construction of this bundle is detailed in the next step.}
@li{Transform the required library dependencies into OSGI bundles. This step is done during the library publishing step of OpenMOLE compilation: you have thus to modify @code{libraries/build.sbt} by adding a new OSGI project, for example @hl.code("""
lazy val math = OsgiProject(dir, "org.apache.commons.math", exports = Seq("org.apache.commons.math3.*"), privatePackages = Seq("assets.*")) settings (
libraryDependencies += "org.apache.commons" % "commons-math3" % mathVersion, version := mathVersion) settings(settings: _*)
"""). At this stage, the exports statement is important since it is thanks to it that the classes will be visible to other OSGI bundles and more particularly your plugin.}
@li{A last step may be necessary in the case your dependencies have themselves dependencies which are already OSGI bundle.
@li{A last step may be necessary in case your dependencies also have dependencies which are already OSGI bundles.
In that case, a conflict will occur and they must be added to a bundle filter used during the final assembly of OpenMOLE.
This filter is defined as @code{bundleFilter} in main build. Add the names of the offending libraries in the @code{bundleFilter.exclude} function.
}
}
\ No newline at end of file
......@@ -5,7 +5,7 @@
OpenMOLE is a plugable platform.
OpenMOLE is a pluggable platform.
It means that you can easily write your own extension and plug it into OpenMOLE.
This tutorial explains how to write an OpenMOLE plugin using Scala and SBT.
OpenMOLE is based on the JVM so you can create OpenMOLE plugins using Scala or any other JVM based languages such as Java, Groovy, Clojure, Jython, etc.
......@@ -21,20 +21,13 @@ You will need the following tools to design your plugin:
@li
@aa("SBT", href := shared.link.sbt), the Scala Building Tool.
The first step is to clone the github repository for OpenMOLE plugins:
@br@br
@hl.code("""
git clone git://github.com/openmole/myopenmoleplugin.git
""")
@br
This repository contains a template to help you create OpenMOLE plugins easily.
The hello directory contains the source code of the plugin and the materials to build it:
@br@br
The @i{hello} directory contains the source code of the plugin and the materials to build it:
@hl.code("""
package myopenmoleplugin
......@@ -51,8 +44,6 @@ object Hello {
The file @b{build.sbt} contains the building instructions for SBT.
The most important part are the OSGi instructions:
@br@br
@hl.plain(s"""
enablePlugins(SbtOsgi)
......@@ -65,8 +56,6 @@ OsgiKeys.privatePackage := Seq("*")
OsgiKeys.requireCapability := ${tq}osgi.ee;filter:="(&(osgi.ee=JavaSE)(version=1.8))"${tq}
""")
@br
@ul
@li
@code{exportPackage} instruction makes the @code{myopenmoleplugin} package visible to OpenMOLE.
......@@ -75,15 +64,10 @@ OsgiKeys.requireCapability := ${tq}osgi.ee;filter:="(&(osgi.ee=JavaSE)(version=1
@li
@code{privatePackage} means that every package in the project, or in the dependencies, should be embedded except for the package starting by the "scala" word. The scala packages provided by OpenMOLE will be used by the plugin instead.
To build the plugin execute @hl.plain("sbt osgiBundle").
To build the plugin, execute @code{sbt osgiBundle}.
SBT will then construct the plugin in @code{target/scala-2.12/myopenmoleplugin_2.12-1.0.jar}.
@br
This JAR file contains the classes you have developed (*.class) along with the metadata relative to imports and exports in the @code{MANIFEST.INF} file:
@br@br
@hl.plain("""
META-INF/MANIFEST.MF
myopenmoleplugin/
......@@ -91,9 +75,7 @@ myopenmoleplugin/Hello$.class
myopenmoleplugin/Hello.class
""")
@br
You can check in the MANIFEST.MF that you namespace is exported.
You can check in the @i{MANIFEST.MF} that your namespace is exported.
......@@ -101,18 +83,12 @@ You can check in the MANIFEST.MF that you namespace is exported.
To enable your plugin in OpenMOLE, either use the plugin panel in the GUI, or use the option -p:
@br@br
@hl.plain("""
openmole -p target/scala-2.12/myopenmoleplugin_2.12-1.0.jar
""")
@br
You can now use the @code{Hello} object in your workflows:
@br@br
@hl.openmole("""
// Declare the variable
val i = Val[Int]
......
......@@ -2,23 +2,34 @@
@import org.openmole.site.tools._
@import org.openmole.site._
@h2{REST API}
Warning: The REST API of OpenMOLE is still experimental, it might be subject to some backward incompatible changes in the future.
@p OpenMOLE ships with a web server providing a REST API to @b{start} workflows, @b{manage} their execution and @b{retrieve} their output data. To start the OpenMOLE REST API, run the command: "openmole --rest --port 8080" from the console. If you need to launch it automatically in a deamon for instance you should also you the --password-file argument to provide the password for encryption of the preferences.
@br
OpenMOLE ships with a web server providing a REST API to @b{start} workflows, @b{manage} their execution, and @b{retrieve} their output data.
To start the OpenMOLE REST API, run the command @code{openmole --rest --port 8080} from the console.
If you need to launch it automatically in a deamon for instance, you should also give the --password-file argument to provide the password for encryption of the preferences.
@br
The web server can be accessed at the URL @i{http://localhost:8080}.
Replace @i{localhost} with the remote machine's hostname or IP address if the web server is not running on your local system.
@p The web server can be accessed at the URL http://localhost:8080. Replace @i{localhost} with the remote machine's hostname or IP address if the web server is not running on your local system.
@h2{API Reference}
The API of exposes the following routes to submit and manage executions:
The API exposes the following routes to submit and manage executions:
@ul
@li{@b{POST /job} - start a mole execution. It has the following parameters:
@ul
@li{@b{workDirectory} - a tar.gz archive containing the workDirectory for the script}
@li{@b{script} - the path (relative to the workDirectory) of script to execute, the last line should be a puzzle}
When successful, it return a structure containing:
@li{@b{script} - the path (relative to the workDirectory) of the script to execute, the last line should be a puzzle}
When successful, it returns a structure containing:
@ul
@li{@b{id} - the id of the execution}
When something has failed, it returns a structure containing:
......@@ -32,7 +43,7 @@ The API of exposes the following routes to submit and manage executions:
When successful, it returns a structure representing the state:
@ul
@li{@b{state} - the state of the execution, it can be running, finished or failed}
When running the other fields are:
When running, the other fields are:
@ul
@li{@b{ready, running, completed} - the number of jobs in each of these states in the execution}
@li{@b{environments} - that contains the state for each execution environment on the execution. This is a JSON structure containing, the name of the environment if it has been set (name), the number of jobs in submitted (submitted), running (running), done (done) and failed (failed) state, a list of errors that happened since the last state query (errors) with the message (message), the stack (stackTrace) and the error level (level).}
......@@ -80,7 +91,7 @@ The API of exposes the following routes to submit and manage executions:
@li{@b{GET /job/} - list execution ids on the server.}
The API of exposes the following routes to submit and manage the plugins:
The API exposes the following routes to submit and manage the plugins:
@ul
@li
@b{GET /plugin/} - list all user plugins loaded in OpenMOLE
......@@ -88,7 +99,7 @@ The API of exposes the following routes to submit and manage the plugins:
@li
@b{POST /plugin} - load one or several plugins in OpenMOLE. It has the following parameter:
@ul
@li{ @b{file} - an OpenMOLE plugin file. Repeat this parameter to submit several plugins at once. }
@li{ @b{file} - an OpenMOLE plugin file. Repeat this parameter to submit several plugins at once. }
When some errors occurs while loading some plugins it return a list containing the name of the plugin an the error that occurred while loading this plugins.
@li
@b{DELETE /plugin} - unload (and remove) one or several plugins in OpenMOLE. Depending plugin are unloaded as well. It has the following parameter:
......@@ -96,9 +107,12 @@ The API of exposes the following routes to submit and manage the plugins:
@li{ @b{name} - the name of an OpenMOLE plugin. Repeat this parameter to submit several plugins at once.}
It return a list with the name of the plugins which have ben unloaded.
@h2{Examples}
Launch the REST server:
@h3{Launch the REST server}
@plain("""
[reuillon:~] $ openmole --rest
Enter your OpenMOLE password (for preferences encryption): *******
......@@ -106,7 +120,8 @@ Jan 08, 2020 3:34:41 PM org.openmole.rest.server.RESTServer server$lzycompute
INFO: binding HTTP REST API to port 8080
""")
Prepare the work directory:
@h3{Prepare the work directory}
@plain(s"""
[reuillon:~] $$ cd /tmp/pi/
[reuillon:/tmp/pi] $$ ls
......@@ -146,7 +161,8 @@ Replication(
[reuillon:/tmp/pi] $$ tar -cvzf pi.tgz *
""")
Submit the job:
@h3{Submit the job}
@plain("""
[reuillon:/tmp/pi] $ curl -X POST -F 'script=Pi.oms' -F 'workDirectory=@/tmp/pi.tgz' http://localhost:8080/job
{
......@@ -181,7 +197,8 @@ curl: Saved to filename 'result.json.gz'
""")
List the plugins:
@h3{List the plugins}
@plain("""
[reuillon:~/myopenmoleplugin] $ curl -X GET http://localhost:8080/plugin/
[ {
......@@ -193,7 +210,8 @@ List the plugins:
} ]
""")
Load a plugin:
@h3{Load a plugin}
@plain("""
[reuillon:~/myopenmoleplugin] $ curl -X POST http://localhost:8080/plugin -F 'file=@./target/scala-2.12/myopenmoleplugin_2.12-1.0.jar'
""")
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment