========
BlackBox
========

The examples presented so far show how a model implemented
in Python or with Python bindings can be used. Here, we
present how we can use the BlackBox module of OptiLog_
to calibrate any model that can be executed using a CLI,
even models that are compiled and we do not have the source
code nor bindings.

For simplicity, we will consider that we have a (discrete)
SIR model available through:

.. code-block::

    $ ./sir ${instance} --beta ...
    MSE: XXXXX

The parameters that it accepts are:

- --beta: :math:`\beta` in the ODEs.
- --delta: :math:`\delta` in the ODEs.
- --n: the effective population.
- --initial-i: the initial population in the Infected compartment.
- --initial-r: the initial population in the Recovered compartment.

--------------------
Model implementation
--------------------

The first step is to define the BlackBox that will represent the
external program, by creating a subclass of :code:`SystemBlackBox`
in **model.py**, defining the parameters that the program
accepts in the config class attribute, and how to call the
program in the constructor.

.. code-block:: python

    ...
    from optilog.blackbox import *
    from optilog.running import ParsingInfo
    from optilog.tuning import *

    class ExternalSir(SystemBlackBox):
        config = {
            "n": Int(70_000, 500_000, default=70_000),
            "initial-i": Int(1, 1_000, default=40),
            "initial-r": Int(0, 1_000, default=4),
            "beta": Real(0.1, 1.0, default=0.7),
            "delta": Real(0.01, 1.0, default=0.1),
        }

        def __init__(self, *args, **kwargs):
            _model = (Path(__file__).parent / "sir").resolve()
            super().__init__(
                arguments=[_model, SystemBlackBox.Instance],
                *args,
                parsing_info=self.get_output_parser(),
                **kwargs,
            )
        ...

In the constructor, when calling the :code:`super().__init__` method
we have the `parsing_info` parameter set. This will instruct the
BlackBox to parse the output of the external program and create 
attributes after its execution. For this example, we define
a filter in the parser that will read the line containing the MSE of
the model, and set it as a *cost* attribute after execution.

.. code-block:: python

    class ExternalSir(SystemBlackBox):
        ...
        @staticmethod
        def get_output_parser():
            parser = ParsingInfo()
            parser.add_filter(
                "cost", r"MSE: ([+-]?(?:\d+(?:\.\d*)?|\.\d+)(?:[eE][+-]?\d+)?)",
                cast_to=float)
            return parser
        ...

Finally, we implement the function that will format the parameters
correctly for this program:

.. code-block:: python

    class ExternalSir(SystemBlackBox):
        ...
        def format_config(self, args):
            other = ""
            for k, v in self.configured.items():
                other += f" --{k} {v}"
            args = args + shlex.split(other)
            return args


Once we have the BlackBox defined, we could execute the model by using:

.. code-block:: python

    sir = ExternalSir()
    sir.run("myData.dat")
    print("The cost for the default parameters is", sir.cost)

To be calibrated, we can define the BlackBox instance as a parameter of
the :code:`model` function (which have the :code:`@ac` decorator). Instead
of defining all the parameters that can be configured in this BlackBox
another time, we set the BlackBox type as :code:`CfgObj(ExternalSir)`. The
:code:`model` function is as simple as:

.. code-block:: python

    ...
    @ac
    def model(datafile, sir: CfgObj(ExternalSir)):
        sir.run(datafile)
        return sir.cost

Finally, the entrypoint for this example does not need to load the dataset, so
it can pass it directly to the model as:

.. code-block:: python

    ...
    def entrypoint(data, seed):
        cost = model(data)
        print(f"Result: {cost}")

------------------------------------------
Preparing the model to be run using docker
------------------------------------------


-----------------
Example of output
-----------------
Once the calibration process finishes, the software will output
the best configuration found during the calibration.

The best parameters are listed for each configurable function.
Note that, as the :code:`sir` object was internally calibrated,
the parameters of this object are indicated as: :code:`sir>parameter`.
Also, the configuration JSON is reported as it can be used to inject
the parameters using OptiLog_.

.. code-block::

    [...]
    Tuning process finished.
    ========================
    Best parameters:
    - model(sir>n): 141718
    - model(sir>initial_i): 802
    - model(sir>initial_r): 847
    - model(sir>beta): 0.24870931153434647
    - model(sir>delta): 0.01311400823831338
    ========================
    Best Config JSON:
    {'_0_0__sir__n': 141718, '_1_0__sir__initial_i': 802, [...]}