Exposing C++ to Python#

This section contains fundamental concepts about pybind11, a library to expose C++ to Python, and more specific indications for users who want to expose tudat functionalities to tudatpy.

Note

In this context, the terms expose and bind (and derived words) will be treated as synonyms.

The reader should be familiar with the content of the Developer Environment page before moving on to the remainder of this guide.

Learning Objectives

  1. Be able to expose a simple function from C++ to Python.

  2. Be able to expose overloaded functions.

  3. Be able to expose classes, including overloaded constructors.

  4. Understand the different access policies on attributes and methods.

  5. Understand the type conversions required and introduced by specific pybind headers.

The contents of this guide are shown below:

Pybind11#

pybind11 is an open-source library that exposes C++ types in Python. Through this software, the user interfaces of tudat, written in C++, can be made available in tudatpy.

pybind11 has an extensive and well-written documentation accessible through the link reported above, which the reader can refer to at anytime. The main goal of this page is to help the reader gain familiarity with the nomenclature and functionalities offered by pybind11 that are specifically useful to expose tudat code to Python. pybind11 features that are not directly applicable to tudat will not be presented.

Note

The hierarchical structure of the binding code is explained in this section. It is noted that the actual compilation of the binding code is achieved by calling the kernel.cpp file; however, all the pybind functionalities that will be explained above are employed in the respective submodules.

Headers and preliminaries#

To write a C++ exposition file, the following header is needed:

#include <pybind11/pybind11.h>

However, additional headers may be needed, such as:

#include <pybind11/stl.h>  // to enable conversions from/to C++ standard library types

#include <pybind11/eigen.h>  // to enable conversions from/to Eigen library types

#include <pybind11/numpy.h>  // to enable conversions from/to Numpy library types

In addition, it is assumed that the following piece of code is present in each code snippet shown in this page:

namespace py = pybind11;

Exposing a function#

In this section, the procedure to expose a simple function through pybind11 will be explained. We will make use of an example taken from tudat.

Suppose that we want to expose to Python the following tudat function (taken from this file):

inline std::shared_ptr< SingleDependentVariableSaveSettings > machNumberDependentVariable(
     const std::string& associatedBody,
     const std::string& bodyWithAtmosphere )
{
 return std::make_shared< SingleDependentVariableSaveSettings >(
             mach_number_dependent_variable, associatedBody, bodyWithAtmosphere );
}

This function is used to save the Mach number dependent variable associated to a certain body. More specifically, it returns a smart pointer to a SingleDependentVariableSaveSettings object and takes as input two standard pointers to std::string (these refer the body whose Mach number should be saved and the body whose atmosphere should be used to compute the Mach number respectively). This is the code (available here) needed to expose the above function to Python:

PYBIND11_MODULE(example, m) {
    m.def("mach_number",
          &tp::machNumberDependentVariable,
          py::arg("body"),
          py::arg("central_body"));
}

The code reported above creates a Python module, called example (the creation of a module through the PYBIND11_MODULE() function is done in tudatpy only in the kernel.cpp file; most of the binding code is organized through submodules structured as explained in section pybind11 of this page). def() is the pybind function that creates binding code for a specific C++ function [1]. def() takes two mandatory arguments:

  1. a string (i.e., "mach_number"), representing the name of the exposed function in Python;

  2. a pointer to the C++ function that should be exposed (i.e., &tp::machNumberDependentVariable), where tp is an abbreviation for the tudat::propagators namespace.

There are also additional input arguments that can be passed to the pybind def() function. In the context of the example above, these are the keywords for the input arguments of the exposed function in Python, denoted by the syntax py::arg , which takes a string as input (i.e., "body" and "central_body"). py is a shortcut for the pybind11 namespace [2].

Note

There are many other optional input arguments to the def() function. For instance, a third positional argument after &tp::machNumberDependentVariable can be passed (of type std::string) to provide a short documentation to the function. However, this pybind functionality is not employed for tudat/tudatpy.

As a result, pybind11 will generate a Python function that can be used as follows:

It is also allowed to call the tudatpy function mach_number() through the keyword arguments as follows:

dep_var_to_save = example.mach_number(body="Spacecraft", central_body="Earth")

It is also possible to have default values for certain keyword arguments. Suppose, for instance, that we want to have "Earth" as default central body. This can be achieved through the following implementation [3]:

PYBIND11_MODULE(example, m) {
    m.def("mach_number",
          &tp::machNumberDependentVariable,
          py::arg("body"),
          py::arg("central_body") = "Earth");
}

The first issue that arises in the binding process is the conversion between variable types. C++ is a statically-typed language, while Python is dynamically-typed. However, the type conversion is still needed and in both directions. In other words, the user can pass a Python variable as input to an exposed function. The type of such variable will have to be converted to a C++ type before it is passed to the actual C++ function acting “behind the scenes”. The inverse process takes place for the output of a function. This is one of the reasons why pybind11 is necessary. Indeed, conversions between native types are dealt with automatically in pybind. For instance, a C++ std::map<> is converted into a Python dict and vice-versa. In our example, this automatic type conversion takes place between the input arguments, between the std::string in C++ and str in Python. A table reporting common conversions is reported below.

Python

C++

list

std::vector<>/std::deque<>/std::list<>/std::array<>

set

std::set<>/std::unordered_set<>

dict

std::map<>/std::unordered_map<>

However, non-native data types need to be known to pybind to be converted properly. This is the case of the output type of the machNumberDependentVariable() function, returning a pointer to an instance of the SingleDependentVariableSaveSettings class. If this class is not exposed to Python, the binding process will fail. This offers the opportunity to explain how to generate binding code for classes, which will be done in Exposing a class.

Templated functions#

When a function is templated (see for instance here) it is mandatory to specify the template argument when exposing it. Therefore, the exposition code must be duplicated for each variable type (shown below for double, example taken from here).

m.def("multi_arc",
    &tp::multiArcPropagatorSettings<double>,
    py::arg("single_arc_settings"),
    py::arg("transfer_state_to_next_arc") = false );

Overloading functions#

If a free function or a member function is overloaded (i.e., it bears the same name but it accepts different sets of input argument types), it is not possible to generate binding code in the traditional way explained in Exposing a function, because pybind will not know which version should be chosen to generate Python code. Suppose, for instance, that we want to expose the following overloaded function:

//! Function to create a set of acceleration models from a map of bodies and acceleration model types.
basic_astrodynamics::AccelerationMap createAccelerationModelsMap(
     const SystemOfBodies& bodies,
     const SelectedAccelerationMap& selectedAccelerationPerBody,
     const std::map< std::string, std::string >& centralBodies )

//! Function to create acceleration models from a map of bodies and acceleration model types.
basic_astrodynamics::AccelerationMap createAccelerationModelsMap(
     const SystemOfBodies& bodies,
     const SelectedAccelerationMap& selectedAccelerationPerBody,
     const std::vector< std::string >& propagatedBodies,
     const std::vector< std::string >& centralBodies )

Both overloads of the createAccelerationModelsMap() function accept the system of bodies and an acceleration map as first two input arguments. In addition, the function needs to know the central body of each propagated body. This information can be passed as a std::map (where each propagated body is associated to its own central body key-value pairs) or through two different std::vector objects, one containing the propagated bodies and the other containing the respective central bodies. The code to expose both overloads is reported below:

m.def("create_acceleration_models",// overload [1/2]
       py::overload_cast<const tss::SystemOfBodies &,
       const tss::SelectedAccelerationMap &,
       const std::vector<std::string> &,
       const std::vector<std::string> &>(
           &tss::createAccelerationModelsMap),
       py::arg("body_system"),
       py::arg("selected_acceleration_per_body"),
       py::arg("bodies_to_propagate"),
       py::arg("central_bodies"));

m.def("create_acceleration_models",// overload [2/2]
       py::overload_cast<const tss::SystemOfBodies &,
       const tss::SelectedAccelerationMap &,
       const std::map<std::string, std::string> &>(
           &tss::createAccelerationModelsMap),
       py::arg("body_system"),
       py::arg("selected_acceleration_per_body"),
       py::arg("central_bodies"));

The def() function is still used, where the first input argument is the function name in Python. The difference with respect to a non-overloaded function exposition (see Exposing a function) lies in the second input argument, where pybind’s templated py::overload_cast<> is used [8]. This pybind function casts overloaded functions to function pointers and its syntax is as follows:

  1. the types of input arguments of the original C++ function are passed as template arguments (e.g., const tss::SystemOfBodies &, etc…);

  2. a reference to the original C++ function are passed as regular input arguments (e.g., &tss::createAccelerationModelsMap, where tss is a shortcut for the tudat::simulation_setup namespace).

The optional arguments to def() do not change with respect to what was explained in Exposing a function.

Warning

In the (rare) case where a function is overloaded based on constness only, the pybind tag py::const_ must be added as second parameter to py::overload_cast<>.

Exposing a class#

As explained above, the SingleDependentVariableSaveSettings class should be exposed to Python as well. This class, available at this link, is defined as follows:

class SingleDependentVariableSaveSettings : public VariableSettings
{
public:
     SingleDependentVariableSaveSettings(
             const PropagationDependentVariables dependentVariableType,
             const std::string& associatedBody,
             const std::string& secondaryBody = "",
             const int componentIndex = -1 ):
         VariableSettings( dependentVariable ),
         dependentVariableType_( dependentVariableType ),
         associatedBody_( associatedBody ),
         secondaryBody_( secondaryBody ),
         componentIndex_( componentIndex ) { }

     // Attributes
     PropagationDependentVariables dependentVariableType_;
     std::string associatedBody_;
     std::string secondaryBody_;
     int componentIndex_;
 };

The class has a constructor and it is derived class, whose parent is the VariableSettings class. The code to expose it to Python, available through this link is as follows, where the exposition of the constructor was omitted for now:

py::class_<tp::SingleDependentVariableSaveSettings,
     std::shared_ptr<tp::SingleDependentVariableSaveSettings>,
     tp::VariableSettings>(m, "tp::SingleDependentVariableSaveSettings")

It makes use of pybind’s py::class_<> templated function [4]. In the template, there are three arguments, of which only the first one is mandatory:

  1. the first template argument declares the C++ class that should be exposed (i.e., tp::SingleDependentVariableSaveSettings);

  2. the second template argument declares the type of pointer that should be used by pybind to refer to instances of such class (i.e., std::shared_ptr<tp::SingleDependentVariableSaveSettings>). The default argument is a std::unique_ptr, but in tudat the common and consistently used pointer is a std::shared_ptr<> [5];

  3. the third template argument informs pybind that the class to be exposed is derived by the parent class tp::VariableSettings [6].

Todo

When does a parent class need to be exposed? In theory, tp::VariableSettings does not have to be exposed… According to GG, “only when the class is part of the signature of a different function” (see recording at 14m01s).

Warning

The third template argument is necessary to ensure automatic downcasting of pointers referring to polymorphic base classes. In other words, when a function returns a pointer to an instance of a derived class, pybind automatically knows to “downcast” the pointer to the type of the derived class only if the base class is polymorphic (a class is said polymorphic if it has at least one virtual function).

In addition, there are two input arguments to the py::class_ function:

  1. the name of the Python module to which the exposed class will belong to (i.e., m);

  2. the name of the exposed class in Python, provided as a std::string (i.e., "tp::SingleDependentVariableSaveSettings").

Exposing class constructors#

Once the class has been exposed, one can also expose its member functions (in C++) which will become methods (in Python). The first member function that will be exposed is the class constructor. This can be exposed through the following code:

py::class_<tp::SingleDependentVariableSaveSettings,
     std::shared_ptr<tp::SingleDependentVariableSaveSettings>,
     tp::VariableSettings>(m, "tp::SingleDependentVariableSaveSettings")
     .def(py::init<
          const tp::PropagationDependentVariables,
          const std::string &,
          const std::string &,
          const int>(),
          py::arg("dependent_variable_type"),
          py::arg("associated_body"),
          py::arg("secondary_body") = "",
          py::arg("component_idx") = -1);

The first three lines were explained above. To expose the class constructor, it is possible to use the pybind def() function, which is common to any function (whether it is a member of a class or not). In addition, the pybind py::init<> function is used to declare the definition of the constructor. This function takes the types of the input arguments to the constructor as template arguments (i.e., const tp::PropagationDependentVariable, const std::string &, etc…). The templated function py::init<> makes it easy to overload the class constructor: it is sufficient to define multiple .def(py::init<>), with different template arguments, to expose several versions of the constructor, whose correct version is selected according to the input arguments types passed to the constructor. An example, taken from this tudat class exposed through this code, is provided below. Overloading simple functions will be explained in section Overloading functions.

Example: overloading a class constructor Show/Hide

py::class_<
    tp::TranslationalStatePropagatorSettings<double>,
    std::shared_ptr<tp::TranslationalStatePropagatorSettings<double>>,
    tp::SingleArcPropagatorSettings<double>>(m, "TranslationalStatePropagatorSettings")
    .def(// ctor 1
         py::init<
         const std::vector<std::string> &,
         const tba::AccelerationMap &,
         const std::vector<std::string> &,
         const Eigen::Matrix<double, Eigen::Dynamic, 1> &,
         const std::shared_ptr<tp::PropagationTerminationSettings>,
         const tp::TranslationalPropagatorType,
         const std::shared_ptr<tp::DependentVariableSaveSettings>,
         const double>(),
         py::arg("central_bodies"),
         py::arg("acceleration_models"),
         py::arg("bodies_to_integrate"),
         py::arg("initial_states"),
         py::arg("termination_settings"),
         py::arg("propagator") = tp::TranslationalPropagatorType::cowell,
         py::arg("output_variables") = std::shared_ptr<tp::DependentVariableSaveSettings>(),
         py::arg("print_interval") = TUDAT_NAN)
    .def(// ctor 2
         py::init<const std::vector<std::string> &,
         const tss::SelectedAccelerationMap &,
         const std::vector<std::string> &,
         const Eigen::Matrix<double, Eigen::Dynamic, 1> &,
         const std::shared_ptr<tp::PropagationTerminationSettings>,
         const tp::TranslationalPropagatorType,
         const std::shared_ptr<tp::DependentVariableSaveSettings>,
         const double>(),
         py::arg("central_bodies"),
         py::arg("acceleration_settings"),
         py::arg("bodies_to_integrate"),
         py::arg("initial_states"),
         py::arg("termination_settings"),
         py::arg("propagator") = tp::cowell,
         py::arg("output_variables") = std::shared_ptr<tp::DependentVariableSaveSettings>(),
         py::arg("print_interval") = TUDAT_NAN)
    .def(// ctor 3
         py::init<const std::vector<std::string> &,
         const tba::AccelerationMap &,
         const std::vector<std::string> &,
         const Eigen::Matrix<double, Eigen::Dynamic, 1> &,
         const double,
         const tp::TranslationalPropagatorType,
         const std::shared_ptr<tp::DependentVariableSaveSettings>,
         const double>(),
         py::arg("central_bodies"),
         py::arg("acceleration_models"),
         py::arg("bodies_to_integrate"),
         py::arg("initial_states"),
         py::arg("termination_time"),
         py::arg("propagator") = tp::cowell,
         py::arg("output_variables") = std::shared_ptr<tp::DependentVariableSaveSettings>(),
         py::arg("print_interval") = TUDAT_NAN)
    .def(// ctor 4
         py::init<const std::vector<std::string> &,
         const tss::SelectedAccelerationMap &,
         const std::vector<std::string> &,
         const Eigen::Matrix<double, Eigen::Dynamic, 1> &,
         const double,
         const tp::TranslationalPropagatorType,
         const std::shared_ptr<tp::DependentVariableSaveSettings>,
         const double>(),
         py::arg("central_bodies"),
         py::arg("acceleration_settings"),
         py::arg("bodies_to_integrate"),
         py::arg("initial_states"),
         py::arg("termination_time"),
         py::arg("propagator") = tp::cowell,
         py::arg("output_variables") = std::shared_ptr<tp::DependentVariableSaveSettings>(),
         py::arg("print_interval") = TUDAT_NAN)

Warning

The template arguments must be always provided to py::init<>, even if the constructor is not overloaded.

The def function follows its standard behavior (explained above) even when it is used to expose a class constructor; in other words, it can take a number of optional arguments that specify the keyword corresponding to each input argument to the class constructor in Python (i.e., py::arg("dependent_variable_type"), etc…). In this example, the last two input arguments have default values.

Note

The set of parentheses after py::init<>(), needed to comply with the correct syntax, is empty. Optional arguments can be passed to create custom constructors in Python [7]. However, this pybind functionality is not used for tudat, therefore it will not be treated in this guide.

Exposing class attributes#

Class attributes in C++ vs. in Python#

There are a few differences between the Object-Oriented Programming (OOP) philosophy in C++ and Python. It is important to know these differences before proceeding to the next sections. The reader who is already aware of this information can skip this section.

One of the principles used in Object-Oriented Programming in C++ is data encapsulation. According to this principle, class attributes should be accessible only from within the class and not by the user dealing with an instance of that class. This is principle is (partly) enforced by C++: for instance, class attributes are by default private (i.e., accessible only from within the class and its methods, also called friends) [9]. This policy is useful mainly for security reasons (data protection), but also because interaction with the data contained within a class becomes only possible through its public methods; in other words, the user can interact with the class data through a dedicated user interface, without knowing or dealing with the class’s internal functioning directly. This strategy also ensures that any changes to the class’s internal structure will not affect the code that creates and uses instances of that class [10]. The most basic form of a user interface are accessors and mutators (hereafter referred to as getters and setters).

In Python, on the other hand, the possibility of keeping class attributes private is not provided. Among Python programmers, there is a widespread convention to use attribute names starting with an underscore (e.g., myclass._myattribute) to inform other developers and users that such attribute should not be called directly outside of the class. However, this is only a convention and the programming language does not enforce this behavior. For this reason, getters and setters are not as common in Python as they are in other OOP languages, such as C++ or Java. In addition, the dot notation in Python to access and mutate class attributes makes the code much more readable [11].

However, there may be cases where getters and setters are needed in Python classes as well. This is the case when code is exposed from another OOP language, such as C++, as it happens for tudat: it is obviously easier to maintain the same user interface, thus having keeping getters and setters in Python as well. In this case, it is recommended to create a class property. This solution has the advantage of having getters and setters, while at the same time benefitting from the dot notation [12].

These concepts will be partially re-explained and applied in Exposing public attributes (for attributes that are not private, thus do not have associated getters and setters) and Exposing private attributes (for attributes that are private, thus do have associated getters and setters, which can become properties in Python).

Exposing public attributes#

Analogously to the def() method of pybind’s py::class_, useful to expose member functions, pybind offers two other methods to expose public attributes of a class (for private attributes, see Exposing private attributes) [9]. def_readwrite() can be used to expose a non-constant attribute. For instance, let’s consider the following piece of code that exposes this class:

py::class_<ta::AerodynamicGuidance, ta::PyAerodynamicGuidance,
         std::shared_ptr< ta::AerodynamicGuidance > >(m, "AerodynamicGuidance")
         .def(py::init<>())
         .def("updateGuidance", &ta::AerodynamicGuidance::updateGuidance, py::arg("current_time") )
         .def_readwrite("angle_of_attack", &ta::PyAerodynamicGuidance::currentAngleOfAttack_)
         .def_readwrite("bank_angle", &ta::PyAerodynamicGuidance::currentBankAngle_)
         .def_readwrite("sideslip_angle", &ta::PyAerodynamicGuidance::currentAngleOfSideslip_);

The highlighted lines show the def_readwrite() function at work. It takes two arguments in the same way explained in Exposing a function:

  1. the name of the attribute of the exposed Python class, passed as a string;

  2. the attribute of the original C++ class, passed as a reference.

Similarly, the def_readonly() function can be used to expose const public class attributes. For instance, look at this example exposing this thrust direction class:

 py::class_<
      tss::ThrustDirectionGuidanceSettings,
      std::shared_ptr<tss::ThrustDirectionGuidanceSettings>>(m, "ThrustDirectionGuidanceSettings")
      .def(py::init<
           const tss::ThrustDirectionGuidanceTypes,
           const std::string>(),
           py::arg("thrust_direction_type"),
           py::arg("relative_body"))
      .def_readonly("thrust_direction_type", &tss::ThrustDirectionGuidanceSettings::thrustDirectionType_)
      .def_readonly("relative_body", &tss::ThrustDirectionGuidanceSettings::relativeBody_);

The highlighted lines use def_readonly() in the same way as for def_readwrite().

Note

In tudat, it was decided to have as few public attributes as possible. Therefore, in principle, a developer should not rely on def_readonly() and def_readwrite() too much, as classes should be designed so that attributes are generally private and interaction with those is possible through getters (and setters).

Exposing private attributes#

If class attributes are private, it is likely that they can be accessed (and, in some cases, modified) through getters and setters. pybind has specific methods from the py::class_ to deal with this situation, namely with def_property() and def_property_readonly() [13]. The latter is used for private attributes that have both getters and setters, while the former is used for private attributes that cannot be modified (i.e., they only have a getter). The following example, exposing a spherical harmonics class in tudat, illustrates the usage of both:

py::class_<tg::SphericalHarmonicsGravityField,
         std::shared_ptr<tg::SphericalHarmonicsGravityField >,
         tg::GravityFieldModel>(m, "SphericalHarmonicsGravityField")
         .def_property_readonly("reference_radius", &tg::SphericalHarmonicsGravityField::getReferenceRadius )
         .def_property_readonly("maximum_degree", &tg::SphericalHarmonicsGravityField::getDegreeOfExpansion )
         .def_property_readonly("maximum_order", &tg::SphericalHarmonicsGravityField::getOrderOfExpansion )
         .def_property("cosine_coefficients", &tg::SphericalHarmonicsGravityField::getCosineCoefficients,
                       &tg::SphericalHarmonicsGravityField::setCosineCoefficients)
         .def_property("sine_coefficients", &tg::SphericalHarmonicsGravityField::getSineCoefficients,
                       &tg::SphericalHarmonicsGravityField::setSineCoefficients);

The syntax is as follows:

  1. the first argument is, as usual, the name of the attribute of the exposed Python class, passed as a string;

  2. the second argument is the getter function of the original C++ class, passed as a reference;

  3. [only for def_property()] the third argument is the setter function of the original C++ class, passed as a reference.

As a result, in Python it will be possible to operate without getter and setters, simply accessing properties through the dot notation (see the Python documentation about the property decorator). As an example, in Python one could do:

# Create spherical harmonics object
spherical_harmonics_model = ...
# Retrieve sine coefficients
sin_coeff = spherical_harmonics_model.sine_coefficients
# Set sine coefficients
spherical_harmonics_model.sine_coefficients = sin_coeff
# Retrieve reference radius
r = spherical_harmonics.reference_radius
# Set reference radius
spherical_harmonics.reference_radius = r  # THIS WOULD THROW AN ERROR

Note

In the current status of tudatpy, def_property() is not always used, because in some cases the getter and setter functions are exposed individually through the traditional def() method. However, this behavior is discouraged when generating other binding code in the future. When getters (and setters) are available in C++, it is recommended to rely on def_property() or def_property_readonly().

Todo

@Dominic, @Geoffrey, do you confirm the note above?

Exposing class methods#

Other class methods that are not part of the categories explained above can be simply exposed with the same syntax used for free functions (see Exposing a function).

Exposing an enum#

Exposing enumerations types is relatively straightforward. Suppose we would like to expose the following enum, located in the tudat::propagators namespace:

//! Enum listing types of dynamics that can be numerically integrated
enum IntegratedStateType
{
     hybrid = 0,
     translational_state = 1,
     rotational_state = 2,
     body_mass_state = 3,
     custom_state = 4
};

This can be done through pybind’s py::enum_<> function as follows (original code):

py::enum_<tp::IntegratedStateType>(m, "StateType")
         .value("hybrid_type", tp::IntegratedStateType::hybrid)
         .value("translational_type", tp::IntegratedStateType::translational_state)
         .value("rotational_type", tp::IntegratedStateType::rotational_state)
         .value("mass_type", tp::IntegratedStateType::body_mass_state)
         .value("custom_type", tp::IntegratedStateType::custom_state)
         .export_values();

py::enum_<> takes the name of the original C++ enum as template argument and the name of the Python equivalent as second parameter (i.e., " StateType"), with the first one being the module m as usual. Each element of the enum can then be exposed using the value() function, that takes two parameters:

  1. the name of the element in Python;

  2. the name of the original C++ element to be exposed (where tp is, as usual, a shortcut for the tudat::propagators namespace).

The final function export_values() is needed to export the elements to the parent scope; without it, tudat::propagators::hybrid_type would be not be valid code [14].

Todo

to address: structure of the PYBIND11_MODULE (in kernel) and module/submodule definition. However, this overlaps with the content of this tudat developer guide. I propose to either redirect from here to there or transfer its content here.

References#