include the text from a different file

docs/benchmark_res_bo.inc

+This page presents benchmarks in which we compare the Bayesian optimization performance of LIMBO against BayesOpt (https://github.com/rmcantin/bayesopt , a state-of-the-art Bayesian Optimization library in C++). Each library is given 200 evaluations (10 random samples + 190 function evaluations) to find the optimum of the hidden function. We compare both the accuracy of the obtained solution (difference with the actual optimum solution) and the time (wall clock time) required by the library to run the optimization process. The results show that while the libraries generate solutions with similar accuracy (they are based on the same algorithm), LIMBO generates these solutions significantly faster than BayesOpt.
+
+In addition to comparing the performance of the libraries with their default parameter values (and evaluating LIMBO with the same parameters than BayesOpt, see variant: limbo/bench_bayes_def), we also evaluate the performance of multiple variants of LIMBO, including different acquisition functions (UCB or EI), different inner-optimizers (CMAES or DIRECT) and whether optimizing or not the hyper-parameters of the model. In all the these comparisons, we can observe that LIMBO is faster than BayesOpt, ebven when BayesOpt is not optimizing its hyper-parameters.
+
+
+Details
+-------
+- We compare to BayesOpt (https://github.com/rmcantin/bayesopt)
+- Accuracy: lower is better (difference with the optimum)
+- Wall time: lower is better
+- In each replicate, 10 random samples + 190 function evaluations
+ - see `src/benchmarks/limbo/bench.cpp` and `src/benchmarks/bayesopt/bench.cpp`
+
+

waf_tools/plot_bo_benchmarks.py

@@ -217,6 +217,12 @@ def plot(func_name, data, rst_file):
     rst_file.write(str(len(da_acc[0])) + " replicates \n\n")
     rst_file.write(".. figure:: fig_benchmarks/" + name + ".png\n\n")
 
+
+# dst file is already open
+def include(src_file, dst_file):
+    for i in open(src_file):
+        dst_file.write(i)
+    
 def plot_all():
     if not plot_ok:
         print_log('YELLOW', "No plot")
@@ -235,23 +241,7 @@ def plot_all():
     node = platform.node()
     rst_file.write("*" + date + "* -- " + node + " (" + str(multiprocessing.cpu_count()) + " cores)\n\n")
 
-    rst_file.write("- We compare to BayesOpt (https://github.com/rmcantin/bayesopt) \n")
-    rst_file.write("- Accuracy: lower is better (difference with the optimum)\n")
-    rst_file.write("- Wall time: lower is better\n\n")
-    rst_file.write("- In each replicate, 10 random samples + 190 function evaluations\n")
-    rst_file.write("- see `src/benchmarks/limbo/bench.cpp` and `src/benchmarks/bayesopt/bench.cpp`\n\n")
-
-    rst_file.write("Naming convention\n")
-    rst_file.write("------------------\n\n")
-
-    rst_file.write("- limbo_def: default Limbo parameters\n\n")
-    rst_file.write("- opt_cmaes: use CMA-ES (from libcmaes) to optimize the acquisition function\n")
-    rst_file.write("- opt_direct: use DIRECT (from NLopt) to optimize the acquisition function\n")
-    rst_file.write("- acq_ucb: use UCB for the acquisition function\n")
-    rst_file.write("- acq_ei: use EI for the acquisition function\n")
-    rst_file.write("- hp_opt: use hyper-parameter optimization\n")
-    rst_file.write("- bayesopt_def: same parameters as default parameters in BayesOpt\n")
-    
+    include("docs/benchmark_res_bo.inc", rst_file)
     
     
     print('loading data...')