Commit 15725ec3 authored by Jean-Baptiste Mouret's avatar Jean-Baptiste Mouret
Browse files

merge with master

parents 41a1df5e fabfdb20
......@@ -39,6 +39,9 @@ addons:
- libboost1.55-all-dev
- libeigen3-dev
- sudo sed -i -e 's/^Defaults\tsecure_path.*$//' /etc/sudoers
- if [ "$LIBCMAES" = "ON" ]; then 'ci/' ; fi
- if [ "$NLOPT" = "ON" ]; then 'ci/' ; fi
Using Limbo as an environment for experiments
.. _framework-guide:
Using Limbo as an environment for scientific experiments
The typical use case of Limbo for research in Bayesian Optimization is:
......@@ -9,8 +11,42 @@ The typical use case of Limbo for research in Bayesian Optimization is:
Limbo provides basics tools to make these steps easier. They are mostly additions to ``waf`` (see our :ref:`FAQ about waf <faq-waf>`). For users who are used to ROS, you can see these additions as our 'catkin for Bayesian optimization'.
**The use of these tools are optional**: you can use Limbo as header-only library in your own project.
What is a Limbo experiment?
Each time we want to investigate an idea (e.g. a particular function to optimize, a new kernel function, etc.), we create a new experiment in the directory ``exp``. For instance, we can have ``exp/test``. This directory should contain all the code that is specific to your experiment (.cpp files, but also .hpp, data files, etc.).
Experiments give you the following benefits:
- it keeps things organized (with code that is specific to a specific paper in a directory and generic code that is maintained by Limbo's team)
- Limbo provides a service to easily generate variants of an experiment (e.g. compare using kernel XX using kernel YY)
- experiments can be easily submitted to a cluster (``--oar=...``)
- experiments can be easily run multiple times locally (if you do not have access to a cluster), via ``--local`` or ``--loca-serial``
How to quickly create a new experiment?
To quickly create a new experiment, you can use ``./waf --create=your_name``. For instance ``./waf --create=test`` will create a new directory in exp/test with a ``wscript`` and a file called ``test.cpp``, based on a basic template.
The experiment can the be compiled using ``./waf --exp test``
If you want to customize the parameters, you can use the following options:
- ``--dim_in=DIM_IN``: Number of dimensions for the function to optimize [default: 1]
- ``--dim_out=DIM_OUT``: Number of dimensions for the function to optimize [default: 1]
- ``--bayes_opt_boptimizer_noise=BAYES_OPT_BOPTIMIZER_NOISE``: Acquisition noise of the function to optimize [default: 1e-6]
- ``--bayes_opt_bobase_stats_enabled``: Enable statistics [default: true]
- ``--init_randomsampling_samples=INIT_RANDOMSAMPLING_SAMPLES``: Number of samples used for the initialization [default: 10]
- ``--stop_maxiterations_iterations=STOP_MAXITERATIONS_ITERATIONS``: Number of iterations performed before stopping the optimization [default: 190
**These parameters can be changed later.** You will only need to open the generated cpp file and put the values you want.
How to add / compile your experiment?
If you do not want to use ``./waf --create``, you can do it yourself:
- add a directory called ``exp`` at the root the limbo tree
- add a directory for your experiment (e.g. ``my_experiment``)
.. _compilation-tutorial:
Download and Compilation
......@@ -29,7 +31,7 @@ Optional but highly recommended
* `NLOpt <>`_ with C++ binding: ::
./configure --with-cxx --enable-shared --without-python --without-matlab --without-octave
make install
sudo make install
......@@ -4,6 +4,7 @@ Tutorials
.. toctree::
:maxdepth: 2
Quick Start
Get limbo
To get **limbo**, simply clone the source code from with git, or download it
as a zip.
Install the dependencies:
For Ubuntu:
apt-get install libeigen3-dev libboost* libtbb*
For OSX with brew:
brew install eigen3
brew install boost
We highly recommend that you install NLOpt. Infortunately, the Ubuntu packages are missing the C++. You can get NLOpt here:
tar zxvf TODO
./configure --with-cxx --enable-shared --without-python --without-matlab --without-octave
sudo make install
For more options and troubleshootings, see the :ref:`Compilation tutorial <compilation-tutorial>`.
Compile Limbo
./waf configure
./waf build
Create a new experiment
./waf --create test
See the :ref:`Framework guide <framework-guide>`
Edit the Eval function to define the function that you want to optimized
$EDITOR exp/test.cpp
Build your experimental
./waf --exp test
Run your experimental
Analyze the results
......@@ -49,7 +49,7 @@ using namespace limbo;
struct Params {
struct bayes_opt_boptimizer : public defaults::bayes_opt_boptimizer {
BO_PARAM(double, noise, 0.0);
BO_PARAM(double, noise, 1e-10);
BO_PARAM(int, hp_period, 10);
struct bayes_opt_bobase {
......@@ -54,7 +54,7 @@ using namespace limbo;
struct Params {
// no noise
struct bayes_opt_boptimizer : public defaults::bayes_opt_boptimizer {
BO_PARAM(double, noise, 0.0);
BO_PARAM(double, noise, 1e-10);
// depending on which internal optimizer we use, we need to import different parameters
// please see the explanation in the documentation
#include <iostream>
// you can also include <limbo/limbo.hpp> but it will slow down the compilation
#include <limbo/bayes_opt/boptimizer.hpp>
using namespace limbo;
struct Params {
struct bayes_opt_boptimizer : public defaults::bayes_opt_boptimizer {
// depending on which internal optimizer we use, we need to import different parameters
struct opt_cmaes : public defaults::opt_cmaes {
#elif defined(USE_NLOPT)
struct opt_nloptnograd : public defaults::opt_nloptnograd {
struct opt_gridsearch : public defaults::opt_gridsearch {
struct bayes_opt_bobase : public defaults::bayes_opt_bobase {
struct kernel_exp : public defaults::kernel_exp {
struct init_randomsampling : public defaults::init_randomsampling {
struct stop_maxiterations : public defaults::stop_maxiterations {
// we use the default parameters for acqui_ucb
struct acqui_ucb : public defaults::acqui_ucb {
struct Eval {
// number of input dimension (x.size())
static constexpr size_t dim_in = @DIM_IN;
// number of dimenions of the result (res.size())
static constexpr size_t dim_out = @DIM_OUT;
// the function to be optimized
Eigen::VectorXd operator()(const Eigen::VectorXd& x) const
int main()
// we use the default acquisition function / model / stat / etc.
bayes_opt::BOptimizer<Params> boptimizer;
// run the evaluation
// the best sample found
std::cout << "Best sample: " << @CODE_BEST_SAMPLE << " - Best observation: " << @CODE_BEST_OBS << std::endl;
return 0;
#!/usr/bin/env python
def configure(conf):
def options(opt):
def build(bld):
bld(features='cxx cxxprogram',
includes='. ../../src',
......@@ -69,7 +69,7 @@ def check_libcmaes(conf):
libs_check = ['/usr/local/lib', '/usr/lib']
conf.start_msg('Checking for libcmaes includes')
conf.start_msg('Checking for libcmaes includes (optional)')
res = conf.find_file('libcmaes/cmaes.h', includes_check)
......@@ -60,9 +60,13 @@ except:
json_ok = False
print "WARNING simplejson not found some function may not work"
def options(opt):
opt.add_option('--qsub', type='string', help='json file to submit to torque', dest='qsub')
def add_create_options(opt):
opt.add_option('--dim_in', type='int', dest='dim_in', help='Number of dimensions for the function to optimize [default: 1]')
opt.add_option('--dim_out', type='int', dest='dim_out', help='Number of dimensions for the function to optimize [default: 1]')
opt.add_option('--bayes_opt_boptimizer_noise', type='float', dest='bayes_opt_boptimizer_noise', help='Acquisition noise of the function to optimize [default: 1e-6]')
opt.add_option('--bayes_opt_bobase_stats_enabled', action='store_true', dest='bayes_opt_bobase_stats_enabled', help='Enable statistics [default: true]')
opt.add_option('--init_randomsampling_samples', type='int', dest='init_randomsampling_samples', help='Number of samples used for the initialization [default: 10]')
opt.add_option('--stop_maxiterations_iterations', type='int', dest='stop_maxiterations_iterations', help='Number of iterations performed before stopping the optimization [default: 190]')
def create_variants(bld, source, uselib_local,
......@@ -90,6 +94,53 @@ def create_variants(bld, source, uselib_local,
def create_exp(name, opt):
if not os.path.exists('exp'):
os.mkdir('exp/' + name)
ws_tpl = ""
for line in open("waf_tools/exp_template.wscript"):
ws_tpl += line
ws_tpl = ws_tpl.replace('@NAME', name)
ws = open('exp/' + name + "/wscript", "w")
cpp_tpl = ""
for line in open("waf_tools/exp_template.cpp"):
cpp_tpl += line
cpp_params = {}
cpp_params['BAYES_OPT_BOPTIMIZER_NOISE'] = ' BO_PARAM(double, noise, ' + str(opt.bayes_opt_boptimizer_noise) + ');\n ' if opt.bayes_opt_boptimizer_noise and opt.bayes_opt_boptimizer_noise >= 0 else ''
cpp_params['BAYES_OPT_BOBASE_STATS_ENABLED'] = '' if opt.bayes_opt_bobase_stats_enabled else ' BO_PARAM(bool, stats_enabled, false);\n '
cpp_params['INIT_RANDOMSAMPLING_SAMPLES'] = ' BO_PARAM(int, samples, ' + str(opt.init_randomsampling_samples) + ');\n ' if opt.init_randomsampling_samples and opt.init_randomsampling_samples > 0 else ''
cpp_params['STOP_MAXITERATIONS_ITERATIONS'] = ' BO_PARAM(int, iterations, ' + str(opt.stop_maxiterations_iterations) + ');\n ' if opt.stop_maxiterations_iterations and opt.stop_maxiterations_iterations > 0 else ''
cpp_params['DIM_IN'] = str(opt.dim_in) if opt.dim_in and opt.dim_in > 1 else '1'
cpp_params['DIM_OUT'] = str(opt.dim_out) if opt.dim_out and opt.dim_out > 1 else '1'
if opt.dim_in and opt.dim_in > 1:
cpp_params['CODE_BEST_SAMPLE'] = 'boptimizer.best_sample().transpose()'
cpp_params['CODE_BEST_SAMPLE'] = 'boptimizer.best_sample()(0)'
if opt.dim_out and opt.dim_out > 1:
cpp_params['CODE_RES_INIT'] = 'Eigen::VectorXd res(' + str(opt.dim_in) + ')'
cpp_params['CODE_RES_RETURN'] = 'return res;'
cpp_params['CODE_BEST_OBS'] = 'boptimizer.best_observation().transpose()'
cpp_params['CODE_RES_INIT'] = 'double y = 0;'
cpp_params['CODE_RES_RETURN'] = '// return a 1-dimensional vector\n return tools::make_vector(y);'
cpp_params['CODE_BEST_OBS'] = 'boptimizer.best_observation()(0)'
for key, value in cpp_params.iteritems():
cpp_tpl = cpp_tpl.replace('@' + key, value)
cpp = open('exp/' + name + "/" + name + ".cpp", "w")
def summary(bld):
lst = getattr(bld, 'utest_results', [])
total = 0
......@@ -67,7 +67,7 @@ def check_mkl(conf):
includes_mkl = ['/usr/local/include', '/usr/include', '/opt/intel/mkl/include']
libpath_mkl = ['/usr/local/lib/', '/usr/lib', '/opt/intel/mkl/lib/intel64']
conf.start_msg('Checking Intel MKL includes')
conf.start_msg('Checking Intel MKL includes (optionnal)')
res = conf.find_file('mkl.h', includes_mkl)
......@@ -72,13 +72,13 @@ def check_nlopt(conf):
libs_check = [os.environ['RESIBOTS_DIR'] + '/lib'] + libs_check
conf.start_msg('Checking for NLOpt includes')
conf.start_msg('Checking for NLOpt C++ includes (optional)')
res = conf.find_file('nlopt.hpp', includes_check)
conf.end_msg('Not found', 'RED')
return 1
conf.start_msg('Checking for NLOpt libs')
conf.start_msg('Checking for NLOpt C++ libs (optional)')
found = False
for lib in ['', 'libnlopt_cxx.a', 'libnlopt_cxx.dylib']:
......@@ -67,7 +67,7 @@ def check_tbb(self, *k, **kw):
includes_tbb = ['/usr/local/include', '/usr/include', '/opt/intel/tbb/include']
libpath_tbb = ['/usr/local/lib/', '/usr/lib', '/opt/intel/tbb/lib']
self.start_msg('Checking Intel TBB includes')
self.start_msg('Checking Intel TBB includes (optional)')
self.find_file('tbb/parallel_for.h', includes_tbb)
......@@ -4,32 +4,32 @@
#| This project has received funding from the European Research Council (ERC) under
#| the European Union's Horizon 2020 research and innovation programme (grant
#| agreement No 637972) - see
#| Contributor(s):
#| - Jean-Baptiste Mouret (
#| - Antoine Cully (
#| - Kontantinos Chatzilygeroudis (
#| - Federico Allocati (
#| - Vaios Papaspyros (
#| This software is a computer library whose purpose is to optimize continuous,
#| black-box functions. It mainly implements Gaussian processes and Bayesian
#| optimization.
#| Main repository:
#| Documentation:
#| This software is governed by the CeCILL-C license under French law and
#| abiding by the rules of distribution of free software. You can use,
#| modify and/ or redistribute the software under the terms of the CeCILL-C
#| license as circulated by CEA, CNRS and INRIA at the following URL
#| "".
#| As a counterpart to the access to the source code and rights to copy,
#| modify and redistribute granted by the license, users are provided only
#| with a limited warranty and the software's author, the holder of the
#| economic rights, and the successive licensors have only limited
#| liability.
#| In this respect, the user's attention is drawn to the risks associated
#| with loading, using, modifying and/or developing or reproducing the
#| software by the user in light of its specific status of free software,
......@@ -40,10 +40,10 @@
#| requirements in conditions enabling the security of their systems and/or
#| data to be ensured and, more generally, to use and operate it in the
#| same conditions as regards security.
#| The fact that you are presently reading this means that you have had
#| knowledge of the CeCILL-C license and that you accept its terms.
import sys
sys.path.insert(0, './waf_tools')
......@@ -72,6 +72,8 @@ def options(opt):
opt.add_option('--create', type='string', help='create a new exp', dest='create_exp')
opt.add_option('--exp', type='string', help='exp(s) to build, separate by comma', dest='exp')
opt.add_option('--qsub', type='string', help='config file (json) to submit to torque', dest='qsub')
opt.add_option('--oar', type='string', help='config file (json) to submit to oar', dest='oar')
......@@ -127,13 +129,22 @@ def configure(conf):
all_flags = common_flags + opt_flags
conf.env['CXXFLAGS'] = conf.env['CXXFLAGS'] + all_flags.split(' ')
print conf.env['CXXFLAGS']
print 'CXXFLAGS:', conf.env['CXXFLAGS']
if conf.options.exp:
for i in conf.options.exp.split(','):
print 'configuring for exp: ' + i
conf.recurse('exp/' + i)
print ''
print 'WHAT TO DO NOW?'
print '---------------'
print '[users] To compile Limbo (inc. unit tests): ./waf build'
print '[users] Read the documentation (inc. tutorials) on'
print '[developpers] To compile the HTML documentation (this requires sphinx and the resibots theme): ./waf doc'
print '[developpers] To compile the benchmarks: ./waf build_benchmarks'
print '[developpers] To compile the extensive tests: ./waf build_extensive_tests'
def build(bld):
......@@ -195,6 +206,8 @@ def run_benchmark(ctx):
retcode =, shell=True, env=None)
def shutdown(ctx):
if ctx.options.create_exp:
limbo.create_exp(ctx.options.create_exp, ctx.options)
if ctx.options.qsub:
if ctx.options.oar:
......@@ -207,6 +220,10 @@ def shutdown(ctx):
def insert_license(ctx):
def build_docs(ctx):
s = "cd docs; make html"
retcode =, shell=True, env=None)
class BuildExtensiveTestsContext(BuildContext):
cmd = 'build_extensive_tests'
fun = 'build_extensive_tests'
......@@ -218,3 +235,7 @@ class BuildBenchmark(BuildContext):
class InsertLicense(BuildContext):
cmd = 'insert_license'
fun = 'insert_license'
class BuildDoc(BuildContext):
cmd = 'docs'
fun = 'build_docs'
Supports Markdown
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment