Writing Interfaces

Reconfigurable interfaces represent the point of contact between module developers and system integrators because they are the point of contact between modules and architectures in which they reside. By enforcing such an abstraction barrier, the hope is that we minimize and formalize the interdependencies of module development and architecture development.

In many systems, the module developers may not have to write any reconfigurable interfaces at all, as they may just be dictated and handed down from on high by the system integrators. In such setups all requests for new interfaces and interface changes need to go through the system integrators, and the module developers can remain blissfully ignorant of everything that goes on beyond their curtain of abstraction.

In most practical development environments the separation of jobs is not so extreme, as it puts an undue burden on the system integrators. In usual practice, module developers may develop interfaces for novel sensors that they are working with. They are the ones that know exactly what information is coming out of the sensor and how to best abstract it, not the system integrator. In addition, module developers also often develop "output" interfaces specific to their module, as they know what the relevant data they output is better than the system integrators. These interfaces are usually developed under the supervision and guidance of the system integrators, but often remain intellectually the "property" of a module developer. Once a module has started to be fully integrated into a system, then the system integrator may write new interface instances to adapt the module into the particular target architecture. At this point, changes in the interfaces need to be negotiated between the system integrator and the module developer, as the system integrator may have to write "adapators" to either fake or transform data for the module's new requirements and capabilities. While reconfigurable interfaces in practice do not solve all problems in the interactions between module developers and system integrators, they at least provide a known point of negotiation for changes.

This section presents the next step in understanding the ModUtils package: developing interfaces. We will build up the necessary knowledge using practical examples in the rough order we expect module and system developers to need as they work. The practical examples we give are not written in stone, they just represent the current best practices in designing and implementing interfaces discovered empirically in the course of developing various robotic systems. The source for all of the examples can be found in the directory ModUtils/doc/examples.

Directory Structure Advice

While the directory structure for your interface can be fairly arbitrary, we have some guidelines for what works best in our opinion.

First, you should create a directory with the same name as your interface. For example, see the directory examples/RoadSource for the definition of the RoadSource reconfigurable interface. You will see that RoadSource has a bunch of peer directories (and peer interfaces) including RoadDest, VehPoseSource, and VehPoseDest. We have grouped together all of the interfaces we will be using for examples in the examples directory. This is a common pattern in which a set of interface directories are grouped together within a common directory. If you project has the discipline of grouping all reconfigurable interfaces together, this common directory might be called Interfaces and have a single makefile which properly compiles all of the interfaces in order. Another common pattern is to have just related interfaces sharing a common parent directory. If we had taken this approach then RoadSource and RoadDest would have been grouped in one common "road processing" directory and VehPoseSource and VehPoseDest would have been grouped in one common "vehicle pose processing" directory.

Your reconfigurable interface directory (such as examples/RoadSource) will contain all of the header and source files for the definition and implementation of the interface. While keeping header files and source files separate has some advantages, we find that gathering all of the header and source for a reconfigurable interface eases use and maintainability of the interface, as with this approach there is only one place to go for the files fundamentally involved in the interface.

Making a Reconfigurable Interface

A reconfigurable interface is a suite of concrete instances which can be selected by a configuration string. Each reconfigurable interface class has a name, such as RoadDest for a class which outputs road information. Each configuration string has a "tag", such as logger for an interface which outputs the road information to a log file. The reconfigurable interface system looks for a "plug-in" to dynamically load into memory at run time with the form ClassName/tag.so , which for our example would be RoadDest/logger.so. As previously stated, the reconfigurable interface system looks in a variety of places for this plug-in, loads it into memory, and then invokes a specifically named creation function within the plug-in in order to generate the instance.

We use our standard makefile helper fragments to build up the makefiles used to build and install reconfigurable interface libraries. It may be useful to look at complete examples of Makefiles as well as the fragments embedded in the text. We provide makefile examples for the RoadDest, RoadSource, VehPoseDest, and VehPoseSource, reconfigurable interfaces. We will go through the RoadSource Makefile in detail.

Defining the interface class name

Just as in a module makefile, you start out by defining the SRC_MODULE variable.

SRC_MODULE = RoadSource

In this usage, this will not define the module name, but will define the basic interface class name. This basic interface class name will form both the basic library name (libRoadSource.so), the header installation site ( $INSTALL_DIR/include/RoadSource) , and where the interface instance plugins will reside ( $INSTALL_DIR/Interfaces/RoadSource/*.so ).

Including necessary makefile fragments

include ${UTILS_DIR}/include/maketools/MakeDefs

We first include the basic makefile definition fragment to set up the rest of the Makefile

include $(UTILS_DIR)/include/maketools/ipt.mk

Then, in this example, we include a helper fragment which defines variables and targets necessary to use the IPT communications library. Specifically, we will be using , which can be used as a target to build IPT if it is not built already and which contains the necessary libraries to link with IPT.

include ../RoadDest/RoadDest.mk

Also, in this example we will be including the .mk file produced by a locally defined interface, i.e., RoadDest. Since we will need to be linking with this library, the RoadDest.mk makefile fragment will supply the variable which lists the libraries necessary to link with the RoadDest interface class. Every interface library created using the makefile fragments will generate a ClassName.mk file which defines a ClassName_LIBS variable to assist linking with it.

Library objects

Next we define the list of objects that will be combined to form the basic interface class shared library:

OBJS = RoadPlayer.o

These objects will be linked together to form lib$(SRC_MODULE).so after a make or make all command and installed into $(INSTALL_DIR)/lib/lib$(SRC_MODULE).so. Every interface class must have a library. In most interface patterns, this is not a problem, as there is almost always a shared logger or player class that can go into the library, but there can be interfaces that have no such "shared" code. There must always be a library produced even for this degenerate case. Basically, when there is no shared code, just create an empty $(SRC_MODULE).cc file, and enter $(SRC_MODULE).o on the OBJS line.

Installation headers

Every interface class should install a header file name $(INSTALL_DIR)/include/$(SRC_MODULE)/$(SRC_MODULE).h. You have to enter at least that header file name on the HEADERS line, along with any other header files you feel should be installed at compilation for public use.

HEADERS = RoadSource.h RoadPlayer.h

Interface destination

You have to define the destination of the interfaces, i.e., the directory where the interface instance plug-ins will be installed:

INTERFACE_DEST=$(INTERFACE_DIR)/RoadSource

When you define this variable, the makefile system will automatically attempt to create the corresponding directory if it does not already exist when the user makes the interfaces. This variable usually takes the form of $(INTERFACE_DIR)/$(SRC_MODULE). We make you specify it explicitly to cover the possibility you want to install these interfaces somewhere slightly more non-standard ($(PROJECT_INTERFACE_DIR)/$(SRC_MODULE) for example). Regardless, the INTERFACE_DEST directory must have the interface class name as its terminal component.

Interface objects

Next you define the list of all object files that go into making all of the interface instances:

INTF_OBJS = FakeRoadSource.o PlayerRoadSource.o \
        ShmemRoadSource.o OutputRoadSource.o \
        LoggerRoadSource.o \
        ShmemPublishRoadSource.o

This list is used mainly for bookkeeping, so that the proper dependencies on header files are generated and the proper work is done in a clean-up. Thus, it is not fatal if the list is not accurate, but if it is inaccurate, the correct files might not get remade when relevant header files are changed, and the unnamed object files might remain after a make clean.

Interface targets

You must list all of the interface plug-in targets you intend on building in the EXTRA_BUILDS line:

EXTRA_BUILDS = \
        $(INTERFACE_DEST)/fake.so \
        $(INTERFACE_DEST)/player.so \
        $(INTERFACE_DEST)/shmem.so \
        $(INTERFACE_DEST)/out.so \
        $(INTERFACE_DEST)/logger.so \
        $(INTERFACE_DEST)/shmemPublish.so 

EXTRA_BUILDS is simply the list of targets to be built in addition to any built-in targets after a make invocation. If you leave an interface plugin out of this list, it will not get built.

Interface make rules

For each interface plug-in to be built, you need to define a make rule to say how to build it. We provide the MAKE_INTF macro to make this easier. It takes as input all of the dependencies, and links them together with the interface class library into the single interface plugin you specify as the rule target. For example, if you have a simple plug-in class such as FakeRoadSource which does not require any additional objects or libraries, you simply do,

$(INTERFACE_DEST)/fake.so: FakeRoadSource.o
        $(MAKE_INTF) 

You use the same rule if you have a class which relies only on functions available within the core class library, such as PlayerRoadSource which depends on the RoadPlayer class:

$(INTERFACE_DEST)/player.so: PlayerRoadSource.o
        $(MAKE_INTF) 

In most cases, the dependency line will include the object files which go into the plug-in (and which should be listed in the INTF_OBJS definition). You may also see the idiom where a particular support library target is included on the dependency line before the object files.

$(INTERFACE_DEST)/shmem.so: $(IPT_TARGET) ShmemRoadSource.o 
        $(MAKE_INTF) $(IPT_LIB)

In this case, if the IPT library is not built and installed, the $(IPT_TARGET) target will (through mechanisms buried in the ipt.mk makefile fragment) cause IPT to be built and installed. This library is then linked automatically into the plug-in via the MAKE_INTF macro, but we also include the $(IPT_LIB) entry on the MAKE_INTF line to indicate we are adding any additional IPT libraries to link in with the plug-in.

You need to include any libraries required by the plug-in not specified on the dependency line on the $(MAKE_INTF) line. For example, the OutputRoadSource depends on the RoadDest library, as it lets you transparently output roads to a destination while reading that are read in from another source.

$(INTERFACE_DEST)/out.so: OutputRoadSource.o
        $(MAKE_INTF) $(RoadDest_LIBS)

In this case we use the $(RoadDest_LIBS) library list which was in the RoadDest.mk makefile fragment to generate the needed libraries. If you neglect to specify the proper libraries, then while the compilation will succeed, the library will have a "missing symbol" error upon being loaded dynamically.

If you have a interface instance which is subclassed from another interface instance, you must include that superclass's plug-in explicitly on the $(MAKE_INTF) line, or the loading of the subclassed plug-in will fail with the missing symbols of the superclass upon being loaded dynamically at run-time. The rule should look like this,

$(INTERFACE_DEST)/logger.so: LoggerRoadSource.o
        $(MAKE_INTF) $(INTERFACE_DEST)/out.so 

Note that the logger plug-in is not exactly a subclass of OutputRoadSource: it is simply a parameterization of OutputRoadSource that specifically creates a LoggerRoadDest as the output destination for convenience. This is true for shmemPublish as well,

$(INTERFACE_DEST)/shmemPublish.so: ShmemPublishRoadSource.o
        $(MAKE_INTF) $(INTERFACE_DEST)/out.so

Even though logger and shmemPublish are not strictly sub-classes, you handle sub-class compilation and linking the same way

Building test programs

When building reconfigurable interfaces, it is often essential to simultaneously build test programs and utilities that get built with a reconfigurable interface class.

First, you list any object files that are involved in the test programs. In this case, only road_relay.o

EXTRA_DEPENDS = road_relay.o

Then, any executables you want installed in $(INSTALL_DIR)/bin you include in the EXTRA_INSTALLS variable definition line:

EXTRA_INSTALLS = road_relay

If you want an executable that is built by default but not installed, you should add it to the EXTRA_BUILDS line with the interface plug-ins. Any other executables (that have build rules) will need to be built explicitly by name.

For all executables, you need build rules. These rules will almost always look almost identical to

road_relay: road_relay.o $(LIB_NAME)
        $(CXX) $(CXXFLAGS) -o $@ $^

Finishing up

Finally, you need to include the MakeTail makefile fragment that makes the rest of the make magic work,

include ${UTILS_DIR}/include/maketools/MakeTail

Designing Interfaces

There are no restrictions for what your reconfigurable interfaces can be, but there are some common patterns and rules of thumb for interfaces that you can model for most of the interfaces you will be writing.

One useful pattern is to always have corresponding input and output interfaces, such as RoadSource, RoadDest or VehPoseSource, VehPoseDest. It can be tempting to only have source interfaces, such as RoadSource or VehPoseSource, as the destination interfaces are not going to be used often, but, eventually, if the producers of the information consumed by the Source interfaces are not done as reconfigurable interfaces you will end up needlessly duplicating code or overly tying yourself to a particular architecture, as you will have "producer" modules which can only be used for a particular architecture and not be able to easily tease out common code. By using a Dest interface you enable your producer modules to be entirely architecturally independent code with all architecture dependent code hidden in the interface instances.

Destination Interfaces

The simplest design pattern for a reconfigurable interface is a destination interface. Take for example, the RoadDest definition,

UTILS_INTERFACE(RoadDest) {
 public:
  virtual ~RoadDest() {}

  virtual bool outputPoints(utils::Time time,
                            const std::vector<utils::Vec3d>& points) = 0;

  // declare the standard interface static methods it also declares,
  UTILS_INTF_DECL(RoadDest);
};

The class definition starts with the UTILS_INTERFACE macro, which takes care of some behind the scenes bookkeeping for you and declares the RoadDest class. This macro must be used when defining an interface.

The class then continues with an empty virtual destructor. This is required due to some syntactical weirdness in C++.

Then comes the meat of the class, the RoadDest::outputPoints declaration. Your API's will always consist largely of virtual abstract functions, although sometimes you might have a default definition for a method, which means it will just be a virtual function (without the = 0 at the end). For a destination interface, usually all that is needed is some kind of output method which outputs a data structure (such as a list of 3D points in this case or a vehicle pose, in the case of VehPoseDest, another example of a destination interface) tagged with a time. The time is the best estimate for when the data, in this case the points, was measured.

Finally there is another required macro, UTILS_INTF_DECL, which takes care of defining many of the static member functions associated with a reconfigurable interface.

Data Driven Source Interfaces

Often a source interface will be data driven, i.e., it is an interface to a normal sensor which is providing data at periodic intervals, and we are always going to be most interested in the most recent or the next piece of data. A good pattern to follow for a data driven source interface is the RoadSource example:

UTILS_INTERFACE(RoadSource) {
 public:
  virtual ~RoadSource() {}

  virtual bool getPoints(utils::Time& time,
                         std::vector<utils::Vec3d>& points,
                         bool blocking = true) = 0;

  // declare the standard interface static methods
  UTILS_INTF_DECL(RoadSource);
};

This interface class looks much like the RoadDest class, with macros being used to do some interface bookkeeping and a single abstract virtual method providing the meat of the API, but there are some hard-won lessons that go into the exact syntax and semantics of that method.

Experience shows that there are two modes that are useful for accessing sensor data:

• Blocking: Wait for the next sensor data.
• Polling: Return the latest sensor data, and flag it if it is "new."

One realization after working with abstract interfaces was that no matter what the original intent of the interface, it seems that eventually both modes are required. RoadSource::getPoints represents a single call which can suffice for both purposes. By default, it is blocking, i.e., the implementation will wait until new data is available. If an error ocurs while waiting, the routine will return false. If polling is desired, then the user will pass in blocking set to false and just accept that the data is the most recent available data. If the user needs to know that it is new data, they can check the return value, which would be true in that case. If an error occurs while reading the latest data, the time result will be set to zero, i.e., time.isZero() will return true.

Time Driven Source Interfaces

Another type of source interface is time driven rather than data driven. Rather than wanting to know the latest or most recent data from a sensor, with a time driven data source we primarily want to know is, given a time, what is the best interpolated (or extrapolated) value of the data for that time. Examples of this kind of data source are vehicle pose (VehPoseSource), but others can be the pointing direction of a pan/tilt head or velocities and accelerations of a vehicle. These types of interfaces are usually used in conjunction with data driven source interfaces, so we will get a time tagged piece of data and then use the time driven data source to get an the vehicle pose, pointing direction, velocity, etc., at the data's time tag.

UTILS_INTERFACE(VehPoseSource) {
 public:
  virtual ~VehPoseSource() {}

  virtual bool getPose(utils::Time time, VehPose& pose) = 0;

  virtual bool getCurPose(utils::Time& time,
                          VehPose& pose, bool blocking = false);

  
  static void interpolate(const VehPose& prev_pose, 
                          const VehPose& next_pose, double t,
                          VehPose& sensor_pose);

  // declare the standard interface static methods
  UTILS_INTF_DECL(VehPoseSource);
};

The VehPoseSource class has a data driven method, VehPoseSource::getCurPose, which returns the latest or next vehicle pose, but it is a secondary method that most users will not use. The heart of this interface is VehPoseSource::getPose which is given a time and returns the vehicle pose at that time, if possible. If it is not possible to return the vehicle pose at that time, it returns false.

Another element of interest about the VehPoseSource is that it is not purely abstract like VehPoseDest, RoadSource, and RoadDest. There is a default implementation of the VehPoseSource::getCurPose method which simply tries to use VehPoseSource::getPose and ignores the blocking parameter. In addition, there is an VehPoseSource::interpolate static method which is used to support interpolation of vehicle poses, a common operation that will be needed by many different VehPoseSource interface instances. These two method are implemented in the file VehPoseSource.cc. The makefile creation system will look to see if there is a C++ source file with the same name as the interface (with a .cc extension), and if there is, that file will be included in the reconfigurable interface library as code which is common to all instances.

Interface Basics

There are several aspects of interfaces that module developers can be totally oblivious to which are critical for interface developers. For example, all reconfigurable interfaces are really subclasses of utils::Interface, which itself is a subclass of utils::Managed, which means they are reference counted. This means that every interface has with it a reference count. You mark that you are using an interface by invoking its ref method, which increments its reference count, and you release it by invoking its unref method, which decrements its reference count. When an interface's reference count goes to 0, it is deleted. Module developers normally don't have to worry about this because the Module superclass manages this reference counting behind the scenes. When you deal with interfaces within the framework of interface instance development, you will have to manage the interface reference counts yourself.

This section will cover other pieces of knowledge that undergird and are required for interface development. This is another layer off of the onion.

The Symbol Table

You may have noticed that there were instances of utils::SymbolTable being passed around in the modules commonly under the name globals or table. The module developers typically don't have to deal with this, but it can be critical to understand for interface instance developers.

A utils::SymbolTable is simply a dictionary of name/value pairs with some memory management. A module should have one symbol table which acts as the pool of global variables without having global variables. This is normally taken care of by the module infrastructure.

For example, a module's symbol table often holds the last created instance of an interface under a standard name, for example, the last VehPoseSource instance created will be stored under the name "VehPoseSourceIntf". In addition, this instance will have its reference count incremented automatically when it is put into the table and decremented if it is ever overwritten. When the symbol table is destroyed at the end of the run, the vehicle pose source instance will have its reference count decremented as well.

There are several reasons we use this global symbol table rather than using real global variables

• Global variables are icky.
• This method allows dynamic checking and installation of "global" things at runtime rather than at link time.
• It packages all of your "global" variables in a single structure, which makes life much happier when we move to operating systems which share global name spaces across processes, such as vxWorks.

A typical use for the global symbol table from the interface instance developer's point of view is as a back door channel of communications between separate interface classes. For example, imagine a simple 2D simulation which simulates the vehicle driving in a road map. A common way to implement this would be to have a single simulation class which serves up both types of information, since the closes road depends on where the vehicle is. Then you have RoadSource and VehPoseSource interface instances which first check to see if there is a simulator in the symbol table, and if not, creates it. Then both instances use the same simulator instance shared through the global symbol table.

utils::SymbolTable is a very simple class. It is defined in utils/SymbolTable.h. The relevant methods you will work with are

    void* get(const char* key) const;
    bool set(const char* key, const void* data,
             SymbolManager* cleaner = (SymbolManager*) NULL,
             bool overwrite = false);

The get method is fairly obvious: You pass it a name and it returns a pointer which you have to cast to something appropriate, or NULL if there is no symbol installed by that name. The set method is a little trickier, as it takes parameters specifying how to do the memory management in addition to the symbol name and anonymized pointer value. Although you can work at creating your own "cleaners," the normal means for putting a reference counted symbol (sub-classed from utils::Managed) into the symbol table is

globals->set("SymbolName", sym_value, utils::SymbolTable::managedManager,
             true);

This will put sym_value in the symbol table under the name SymbolName and will use a built-in cleaner which does the referencing and dereferencing correctly. Notice the final true means that the value will be overwritten, i.e., if SymbolName already has a set value we will replace it with sym_value.

A common way to set a symbol with a value that is not reference-counted is to use the utils::SymbolDeleter template, i.e.,

MyClass* my_class = new MyClass;
globals->set("SymbolName", my_class, new utils::SymbolDeleter<MyClass>);

This will put an instance of MyClass in the symbol table under SymbolName. When it is overwritten or the symbol table is deleted, the utils::SymbolDeleter template instance will delete my_class. Note that this invocation will not overwrite a pre-existing value of SymbolName as the overwrite parameter is left at its default, false. If the symbol is not set, the set method will return false and the cleaner will delete sym_value.

Generators and Interfaces

The mechanism which takes a specification string and creates an interface is managed by generators, which are instances of the utils::Generator template defined in utils/Generator.h. Every reconfigurable interface has exactly one generator stored in the symbol table. In addition, part of the bookkeeping in creating an interface sets up a type definition so that

utils::Generator<RoadSource>* gen;

and

RoadSourceGenerator* gen;

are equivalent.

A generator's syntax is fairly straightforward. If you have one, then you can use it to create an interface instance like this,

utils::SymbolTable* globals;  // the symbol table
RoadSourceGenerator* gen;  // the generator
const char* spec_string;  // the specification string
.
.
// create the instance
RoadSource* inst = gen->interface(spec_string, globals)

If you generate an instance this way you must reference it immediately,

inst->ref();

and dereference it when you are done,

inst->unref();

Interface Static Methods

The UTILS_INTF_DECL interface definition macro defines a set of static member functions that all reconfigurable interfaces should share. For any reconfigurable interface class T, these include

• const char* intfName()
  Return the symbol name for the last created interface.
• T* create(const char*, utils::SymbolTable* globals)
  Create an interface instance from a string specificaiton if it has not been created already. Once created, put in symbol table afterwards under intfName(). This routine was mainly used for convenience before the Module superclass was put into place, and its use is deprecated in favor of the templated Module::create method.
• T* interface(const char*, utils::SymbolTable* globals)
  Create interface from string if not in symbol table already under the name of intfName().
• T* generate(const char*, utils::SymbolTable* globals)
  Use the interface's generator from globals to create an interface from a specification string.
• TGenerator* generator(utils::SymbolTable* globals)
  Get the interface's generator out of the symbol table globals. If there is no such generator in the symbol table, create it, put it in the symbol table, and return it.
• T* getIntf(utils::SymbolTable* globals)
  Get the last result of create.
• void clear(utils::SymbolTable* globals)
  Clear the interface out of the symbol table.

Interface Instance Recipes

A Basic Interface

To teach you how to build an interface we will start with a concrete example, a "fake" RoadSource which simply repeatedly returns the same fake road when asked. This simple interface can serve as a model for almost any primitive data driven interface, such as a direct interface to a real live sensor. The full example is in FakeRoadSource.cc.

After you include the proper header set of header files for your interface (always including the basic interface header, in this case RoadSource.h as well as usually including the configuration file header, utils/ConfigFile.h), you will define your interface subclass:

class FakeRoadSource : public RoadSource {
 public:
  virtual bool getPoints(utils::Time& time,
                         std::vector<utils::Vec3d>& points,
                         bool blocking = true);

  bool init(utils::ConfigFile& params);

 private:
  std::vector<utils::Vec3d> _points;  // stored points to return
};

You do not need any special macros for this class definition, simply subclass from the reconfigurable interface class.

Then you must define the creation function:

UTILS_INTF_CREATOR(RoadSource, fake, gen, params, globals)
{
  UTILS_INTF_REPORT(RoadSource, fake);
  FakeRoadSource* intf = new FakeRoadSource();
  if (!intf->init(*params)) {
    delete intf;
    return NULL;
  }
  return intf;
}

Remember the reconfigurable interface generator looks for a specifically named symbol in the plug-in. The UTILS_INTF_CREATOR macro takes the class name as the first argument (RoadSource) and the interface instance name as the second argument (fake). The remaining arguments are the variable names for a RoadSourceGenerator* (gen) that can be used to generate new contained interfaces, a utils::ConfigFile* (params) that holds the interfaces parameters, and utils::SymbolTable* (globals) that points to the global symbol table.

The first line of your creation function should always be

UTILS_INTF_REPORT(ClassName, tagname) 

with ClassName and tagname matching the class name and tag name used in the UTILS_INTF_CREATOR invocation. If you leave this line out, then you will not be able to use the INTF_VERBOSE variable to track the generation of this interface.

We have found that the pattern shown in this creation function and interface instance is a good approach. The interface instance class has a constructor with no arguments that just sets up some internal variables (and in this case, does not even exist). Then we invoke the init method, passing in the parameters (and generators and symbol table if required by your creation function as shown in the next section). The init method then attempts to parse the parameters and setup the interface and returns true for success and false for failure. If we have an initialization failure we just delete the interface instance and return NULL, otherwise we return the created, initialized instance.

Composite Interfaces

Many times interfaces are composite, i.e., you have an interface which contains one or more other interfaces. The first one you are likely to build is an "output" interface which is a Source interface which wraps both another Source interface and a Dest interface, with the idea that we transparently pass the result of the contained Source interface to the user and to the Output interface. Thus you can have a Source interface which is wire-tapped to output to shared memory or a log file. The full example file is in OutputRoadSource.cc, and you can probably use this file (or its equivalent, OutputVehPoseSource.cc) as a template for making most composite interfaces.

To create a composite interface, as before you start by including the necessary include files, and then define the interface instance sub-class:

class OutputRoadSource : public RoadSource {
 public:
  OutputRoadSource();
  virtual ~OutputRoadSource();

  virtual bool getPoints(utils::Time& time,
                         std::vector<utils::Vec3d>& points,
                         bool blocking = true);

  bool init(utils::ConfigFile& params,
            RoadSourceGenerator* gen, utils::SymbolTable* globals);

 private:
  RoadSource* _contained;
  RoadDest* _output;
};

As you can see, this is again, a fairly simple subclass, but notice the private member functions for the contained RoadSource and the contained RoadDest.

The required creation function is very similar,

UTILS_INTF_CREATOR(RoadSource, out, gen, params, globals)
{
  UTILS_INTF_REPORT(RoadSource, out);
  OutputRoadSource* intf = new OutputRoadSource();
  if (!intf->init(*params, gen, globals)) {
    delete intf;
    return NULL;
  }
  return intf;
}

Notice that the init method is passed the RoadSourceGenerator (gen) and the symbol table (globals) as well as the parameters. This is because we will need those elements in creating the various contained interface instances.

bool OutputRoadSource::init(utils::ConfigFile& params,
                            RoadSourceGenerator* gen,
                            utils::SymbolTable* globals)
{
  // create the contained interface
  _contained = gen->interface(params.getString("contained"), globals);
  if (!_contained) {
    cerr << "OutputRoadSource::init: could not create contained\n";
    return false;
  }
  _contained->ref();

  _output = RoadDest::generate(params.getString("output", "logger"), globals);
  if (!_output) {
    cerr << "OutputRoadSource::init: could not create output\n";
    return false;
  }
  _output->ref();
  
  return true;
}

We will go through the init method in a little more detail. First it creates the contained interface. Since it is creating an instance of an interface of the same type (RoadSource), it uses the generator passed in from the creation function, gen. To create the output, which is a different interface type, it uses one of RoadDest's standard static members, generate, which simply invokes (creating if necessary) RoadDest's generator to create an interface instance with the parameters gotten from the output parameter.

Note that immediately upon creating the interfaces, we reference them with the ref method to mark that we are using them. To properly manage the contained instances, we have to have a constructor which sets them to NULL and a destructor which, if the containers are not NULL dereferences them to mark that we no longer need them.

OutputRoadSource::OutputRoadSource()
{
  _contained = NULL;
  _output = NULL;
}

OutputRoadSource::~OutputRoadSource()
{
  if (_contained)
    _contained->unref();
  if (_output)
    _output->unref();
}

The actual OutputRoadSource::getPoints method simply wraps the transaction of getting the points from the contained interface, outputting points to the destination interface, and returning the points to the user:

bool OutputRoadSource::getPoints(utils::Time& time,
                               std::vector<utils::Vec3d>& points,
                               bool blocking)
{
  if (_contained->getPoints(time, points, blocking)) {
    _output->outputPoints(time, points);
    return true;
  } 
  return false;
}

One other element of interest in OutputRoadSource is the definition of two "aliased" source tags, logger and shmemPublish. These two tags do not involve any new classes, they simply take their parameters and create the appropriate specification for an OutputRoadSource. For example, the source specification,

spec {logger: 
  string name=PointOutput.raw;
  bool allow_clobber = true;
  spec contained = fake;
}

will be automatically transformed into

spec {output:
  spec output {logger:
    string name=PointOutput.raw;
    bool allow_clobber = true;
  }
  spec contained = fake;
}

Basically all parameters of the "top" level other than the contained specification are assumed to be logger parameters and are put into the output specification of result. If you want to create your own aliased output tags, just copy LoggerRoadSource.cc, change the interface and tag names appropriately, and put your new plug-in in the Makefile. If you want some practice in advanced utils::ConfigFile, feel free to pick the code apart and figure out why it does what it does, but you should be able to just use it.

UTILS_INTF_CREATOR(RoadSource, logger, gen, params, globals)
{
  UTILS_INTF_REPORT(RoadSource, logger);
  // pose output specification inherited from params for the most part
  utils::ConfigFile output_params;
  utils::ConfigFile::copy(*params, output_params);
  output_params.setString("tag", "logger");  // set up logger tag
  output_params.set("contained", "{}"); //we clear contained to avoid confusion
  
  // get the contained specification
  utils::ConfigFile contained_params;
  params->getStruct("contained", contained_params);

  // now create the output instance parameters
  utils::ConfigFile final_params;
  final_params.setString("tag", "output");
  final_params.setStruct("contained", contained_params);
  final_params.setStruct("output", output_params);

  OutputRoadSource* intf = new OutputRoadSource();
  if (!intf->init(final_params, gen, globals)) {
    delete intf;
    return NULL;
  }
  return intf;
}

Data Formatting and Interfaces

A common task in data manipulating is taking a C structure, containing primitives, strings, arrays, and vectors, and marshalling it into a flat array of bytes. This array of bytes can then be logged to a file or sent over a network. The bytes need to be then unmarshalled back into a C structure in order to interpret them. A complicating factor is dealing transparently with different platform and compiler architecures.

The ModUtils package provides generic facilities for performing this marshalling and unmarshalling of data, and it is a key enabler for both the data logging/replay and for interprocess communications. Essentially, we use a simple format string to specify a C structure, and this format string guides the marshalling of data out of the C structure and the marshalling of data back into the C structure. This format syntax was originally developed for TCA (the Task Control Architecture (http://www.cs.cmu.edu/~TCA), and it continues in Reid Simmons' IPC package (http://www.cs.cmu.edu/~IPC). In our group's hands it passed through a package called IPX to a package called IPT, and eventually was stripped out of IPT to form a standalone utility within the ModUtils package.

As an interface developer, the primary aspect of this you will have to understand is the structure format strings, as most of the aspects of how the structure format strings are used will be hidden from you. A structure format string can describe fairly arbitrary C structures (it currently is not set up to handle C++ elements such as STL strings and vectors, although that would be a good extension).

Simple Formats

At their simplest, format strings can specify individual C++ primitive variables.

• int: A four byte integer
• short: A two byte integer
• double: A double precision floating point number
• float: A single precision floating point number
• char: A one byte character
• string: A C string, i.e., a char* pointer

These usually are not standalone (although they could be), and need to be encapsulated in structures, which are just a comma separated list of format specifiers enclosed with curly braces. For example, the structure

struct {
  int a;
  double d;
  char c;
  char* str;
}

Can be specified by the following structure format string,

"{ int, double, char, string }"

Composite Formats

The simplest example of a composite format is a structure which containes another structure, so

struct {
   int a;
   struct {
     double b;
     char* str;
   }
}

can be represented by the format string,

"{ int, { double, string } }"

There are other types of composite structures that can be represented: The fixed array, the variable array, and the pointer.

Fixed Arrays.

A fixed array simply consists of a structure format, a colon, and a size enclosed in square braces. Often you will see a fixed array of some primitive type, so

struct TestStruct {
  int a[20];
  double d;
}

can be represented by the format string

"{ [ int : 20 ], double }"

Of course, you can have arrays of structures as well, so

struct SillyStruct {
         int thing;
   TestStruct[5];
};

corresponds to

"{ int, [ { [ int : 20 ], double } : 5] }"

Variable Arrays.

Often you will want a variable sized array. A variable sized array must reside inside of a structure, and represents the pairing of a pointer to an array of structures with another integer element of the structure which contains the size of the array.

For example, the structure

struct {
   int num_elems;
   double* elems;
};

corresponds to

"{ int, < double : 1 >}"

The numerical "argument" for the variable sized array (after the colon) is the index (starting at 1) of the value containing the size. For your sanity, it is a good idea to have this be the first element, but it doesn't have to be. For example,

struct {
  double* elems;
  int num_elems; 
}

corresponds to

"{ <double : 2 >, int }"

Pointers.

One of the least often used abilities of the format string is the ability to specify a pointer to a structure. If you were sending a C linked list, this might make sense, but it is hardly ever used now. For completeness, its usage looks something like this, where

struct {
  double a;
  TestStruct* ptr;
}

corresponds to

"{ double, *{ [ int : 20 ], double } }"

Note that structured format system can marshall and unmarshall a NULL pointer just fine using the pointer format specifier.

Examples

We will look at to road and vehicle pose examples for some practical instances of the use of structured formats.

Most interfaces will have an "internal" representation for their data for marshalling. This representation may not be what the user sees, as it will be tweaked for compactness and ease of marshalling. For the vehicle pose example, we group these internal structures in the file VehPoseDest/VehPoseStructs.h. We put these common structures in the output interface because it is more standalone than the input interface (VehPoseSource uses VehPoseDest while VehPoseDest does not use VehPoseSource).

In this file, the internal representation structure is

struct VehPoseDataStruct {
  double x;    
  double y;    
  float z;     
  float ori[4]; 
};

Note that even though the "user" representation (VehPose) is completely doubles, in order to make the log files more compact and shared memory transfers more efficient we only represent x and y as doubles, since single precision floating point is sufficient for all the other values.

In this same file we define the data format for the internal representation,

#define VEH_POSE_DATA_FMT "{double, double, float, [float:4]}"

It is a very good idea to keep your data format specification near the C structure it represents, so that you are more likely to change one when changing the other. Someday someone will write a good, integratable tool for automatically generating one from another.

Further down in the file we see another internal structure that is built up from VehPoseDataStruct that is used specifically for shared memory transfer. It is a good example of how to compose structured format strings in a maintainable manner at compile time. The structure is,

struct VehPoseShmemStruct {
  int secs;  
  int usecs; 
  VehPoseDataStruct data;  
};

and the data format string is,

#define VEH_POSE_SHMEM_FMT "{int, int, " VEH_POSE_DATA_FMT " }"

Note the mechanism for embedding VEH_POSE_DATA_FMT in the middle of VEH_POSE_SHMEM_FMT as an embedded structure. This means changes in the internal representation and format string will be propagated automatically to the shared memory representation.

The road example has the same structure, with common structures for marshalling and unmarshing being defined and specified in RoadDest/RoadStructs.h. The road internal structures are a little more complex, with a basic structure for a point defined as,

struct RoadDataPoint {
  double x;  
  double y;  
  float z;   
};

and then the internal structure for representing a point for marshalling as

struct RoadDataStruct {
  int num_points;        
  RoadDataPoint* points; 
};

and the data format for the whole thing being

#define ROAD_DATA_FMT "{int, < {double, double, float } : 1>}"

Again, you can see how the internal representation has been optimized using some basic assumptions for saving space.

One thing to note is that marshalling and unmarshalling "flat" structures like the vehicle pose, which have no pointers in them, is much more efficient than marshalling non-flat structures such as the road points. Note that the VehPoseShmemStruct is still flat even though it contains nested structures, as it contains no nested pointers or strings.

Data logger/player

After you make your fake and sensor interface instances, you will want to make a data logger destination interface and a data player source interface as soon as possible.

The ModUtils package provides several layers of functionality to support the creation of logger and player interfaces. At its lowest level this functionality is built on a canned raw data facility. The recorded data is broken into two separate files, a data file which is simply records of bytes of arbitrary length and then an index file which gives both the time of collection and position in the data file of every record. This index allows us to randomly access the data file by time, which supports a variety of playback functionalities. The two files are often "crunched" into a single file with a .rad extension. You could access raw data files at a very low level through the utils::CannedDataWriter and utils::CannedDataReader, but this is highly unlikely. It is more likely that you will be using the structured canned data facilities built on top of these two classes which use the structured data formats described in the previous section.

One structural piece of advice that is modeled by the examples is to wrap the meat of your logging and playing interfaces in standalone classes. The issue is that eventually, there is a high likelihood that you will need a data analysis utility that explicitly deals with your data files as data files instead of through a veil of abstraction. If your logging and reading facilities are entwined inextricably with your interface instances, you will have a hard time building these standalone analysis tools.

Logger Interfaces

The logger interfaces are built around the utils::Logger class, which wraps the canned data format tools with the structured data formatting tools to connect selected C structures to the logging mechanism. For the RoadDest example, the file logging is implemented in the class RoadLogger.

The RoadLogger class is fairly straightforward:

class RoadLogger {
 public:
  bool open(const char* name, const utils::ConfigFile* header=NULL);

  bool open(utils::ConfigFile& header);

  bool logPoints(utils::Time time, const std::vector< utils::Vec3d > & points);

 private:
  utils::Logger _logger;      
  RoadDataStruct _output_area; 
};

This class's main purpose is to wrap access to the utils::Logger instance, _logger, which will be outputting data derived from lists of points passed in through the RoadLogger::logPoints method to the _output_area structure.

The RoadLogger class must first be opened.

bool RoadLogger::open(const char* name,
                      const utils::ConfigFile* user_header)
{
  utils::ConfigFile header;

  // copy user header information if there is any
  if (user_header) {
    utils::ConfigFile::copy(*user_header, header);
  }

  // set the data format major and minor version numbers in the header
  header.setInt("int DataFormat.version_major", 1);
  header.setInt("int DataFormat.version_minor", 0);

  // make sure logger is closed
  _logger.close();

  // hook up logger to _output_area
  _logger.declare("points", ROAD_DATA_FMT, &_output_area);

  // open the file, and return status as the result
  return _logger.open(name, header);
}

This routine will open the data file named name. The first job is to create the header which will be included with the data file. We can base this header on a user-defined header passed in with user_header, but regardless, the header will have a DataFormat structure which specifies the major and minor version number, which you can use in the player to decide how to interpret the data.

The second major job of the open method is to tell the logger what data to log and how to log it. This is done with the logger's declare method, so

  _logger.declare("points", ROAD_DATA_FMT, &_output_area);

says that there is a data field to be logged which will be tagged as points in the data file, it contains a C structure specified by the ROAD_DATA_FMT structure specifier which will be contained in the _output_area structure.

Finally we use the logger's open method to open the file with the proper header. This will cause the index and data file to be created, and the header's DataFormat structure to be expanded with the formatting information and written to the index file. For our example, the header will end up looking something like

{
  struct DataFormat {
        int version_major = 1;
        int version_minor = 0;
        string names = "points";
        string formats = "{int, < {double, double, float } : 1>}";
        int alignment = 4;
        int byte_order = 0;
  }
}

When you use the cddescribe utility to print out a data file's header, it allows you to check out the version number, but also the fields and field formats of the data, as well as some indication of the platform's byte order and structure alignment (although this is likely only ever needed by the computer, not you).

The RoadLogger::logPoints method transforms the STL vector of 3D points into the internal representation, stores it in the _output_area and invokes the logger, which was connected to _output_area by the open method:

bool RoadLogger::logPoints(utils::Time time,
                           const std::vector<utils::Vec3d>& points)
{
  // setup output area
  _output_area.num_points = (int) points.size();
  _output_area.points = new RoadDataPoint[_output_area.num_points];
  for (int i=0;i<_output_area.num_points;i++) {
    _output_area.points[i].x = points[i].x;
    _output_area.points[i].y = points[i].y;
    _output_area.points[i].z = points[i].z;
  }

  // log _output_area
  bool res = _logger.log(time);

  // cleanup output area
  delete [] _output_area.points;

  // return the result of the log
  return res;
}

As an aside, for clarity, this method does the dynamic allocation and deallocation of the _output_area.points array locally. For a real interface it may be more efficient to create an array of points that stays between invocations and is large enough (or expandable to) hold the points array. The less dynamic memory allocation and deallocation you do on a regular basis, the more predictable your codes performance.

For most of its life, the RoadLogger class will exist to support the LoggerRoadDest interface instance class. This class is defined similarly to the fake interface:

class LoggerRoadDest : public RoadDest {
 public:
  virtual bool outputPoints(utils::Time time,
                            const std::vector<utils::Vec3d>& points);

  bool init(utils::ConfigFile& params);

 private:
  RoadLogger _logger;  
};

Most of its functionality is implemented by the RoadLogger class, so the member definitions are almost trivial:

bool LoggerRoadDest::init(utils::ConfigFile& params)
{
  return _logger.open(params);
}

bool LoggerRoadDest::outputPoints(utils::Time time,
                                  const std::vector<utils::Vec3d>& points)
{
  return _logger.logPoints(time, points);
}

The logger instance for the VehPoseDest interface is structured in a very similar way, with a standalone VehPoseLogger class and a minimal LoggerVehPoseDest interface instance definition. One difference to point out is in VehPoseLogger::open, which looks like

bool VehPoseLogger::open(const char* name,
                         const utils::ConfigFile* user_header)
{
  utils::ConfigFile header;

  // copy user header information if there is any
  if (user_header) {
    utils::ConfigFile::copy(*user_header, header);
  }

  // set the data format major and minor version numbers in the header
  header.setInt("int DataFormat.version_major", 1);
  header.setInt("int DataFormat.version_minor", 0);

  // make sure logger is closed
  _logger.close();

  // hook up logger to _output_area
  _logger.declare("x", "double", &_output_area.x);
  _logger.declare("y", "double", &_output_area.y);
  _logger.declare("z", "float", &_output_area.z);
  _logger.declare("ori", "[float : 4]", &_output_area.ori);

  // open the file, and return status as the result
  return _logger.open(name, header);
}

One way to implement this is exactly as in RoadLogger::open, with a single logging field which will be connected to _output_area as a whole. Instead, in this example, the logger declare invocations show that we break the logging into four fields,

• an x field connected to _output_area.x.
• an y field connected to _output_area.y.
• an z field connected to _output_area.z.
• an ori field connected to _output_area.ori

Breaking the declaration up like this will take no more room in the output records, but has one distinct advantage: It makes it easier for you to change what is logged while still being able to read old files. For example, if later on you expand the VehPose struct to include a velocity, then, while you would probably increment the minor version, you could just add another declare invocation which connects the new velocity field to the _output_area.velocity. Because the fields are broken out like this, the reader can transparently access old files without the velocity field, and it will just never fill in the velocity, most likely leaving it as zero with a well written player.

Player Interfaces

Player interfaces are somewhat trickier than logger interfaces. Player interfaces have to deal with time, i.e., does the interface drive time so that time advances with every piece of data, or is the interface driven by time, so that we examine the current time and look up the appropriate data? To make sure we deal with these issues consistently, the player interface is built on top of the TimeSource::PlayerManager class. This class packages and abstracts all of the parameterization of a replay interface and its interaction with the current time source. The player manager does maintain a utils::Player, and gives you access to it through the TimeSource::PlayerManager::getPlayer method, but you will usually use its pre-packaged methods instead to handle time consistently for replaying data.

Just like with the RoadDest logger example, for RoadSource you will put the meat of your player interface in a standalone class which wraps the player manager and player appropriately. This standalone class is called the RoadPlayer:

class RoadPlayer {
 public:
  bool open(utils::ConfigFile& params, utils::SymbolTable* globals);

  bool advance();

  bool getPoints(utils::Time& time, std::vector< utils::Vec3d > & points);

  bool nextPoints(utils::Time& time, std::vector< utils::Vec3d > & points,
                  bool blocking=true);

  TimeSource::PlayerManager* getManager() { return &_mgr; }

 private:
  TimeSource::PlayerManager _mgr;  
  utils::Player* _player;          
  utils::PlayElem* _play_elem;     

  RoadDataStruct _input_area; 
  utils::Time _play_time;     
};

The open method uses the player manager (_mgr) to open the data file and parameterize the interaction of replay and time. This standard parameterization is detailed in the module developer documentation

The standard method for using the RoadPlayer is simple, just repeatedly use the RoadPlayer::nextPoints method to either read in the next set of data, or read the data at the current time, depending on the RoadPlayer's parameterization. This method consists of using the advance method to get the "next" set of data into the internal data structures and the getPoints method to read that data into the output structure.

In addition, the RoadPlayer source provides the RoadPlayer::getManager access method in case your interface needs to deal more directly with the player manager or the player.

The implementation of the RoadPlayer::open method is important, as it gives a good example of how to deal with the file headers.

bool RoadPlayer::open(utils::ConfigFile& params,
                      utils::SymbolTable* globals)
{
  // setup the player manager
  // the player manager takes care of all of the interactions between the
  // data file and the time source
  if (!_mgr.open("Road.rad", params, globals))
    return false;

  // get the actual canned data reader from the manager
  _player = _mgr.getPlayer();

  // clear the input area
  memset(&_input_area, 0, sizeof(_input_area));

  // check versions
  int major_version = _mgr.getPlayer()->getHeader().
    getInt("int DataFormat.version_major", 1);
  int minor_version = _mgr.getPlayer()->getHeader().
    getInt("int DataFormat.version_minor", 1);
  // note, you don't have to simply reject other version, you can
  // try and adapt here
  if (major_version != 1 && minor_version != 0) {
    printf("RoadPlayer::init:  Cannot read version %d.%d\n",
           major_version, minor_version);
    return false;
  }

  // Tell the player to expect the points.  We store a reference in _play_elem
  // so we can have the player properly manage the memory associated with the
  // points.
  _play_elem = _player->expect("points", ROAD_DATA_FMT, &_input_area);

  // Ready the player for action
  return _player->setup();
}

So, for simple player formats you can just use this almost verbatim. We have provided a place holder here which checks the major and minor version numbers of the header. A common pattern is to be able to read old versions, so if the major and minor version number match a known format, instead of causing an error, we would register a different input area with the old format, flag that we were using that format, and carry on using that old structure instead of the new structure. Thus you should be able to change your file format while still being able to read the old format, if desired. At the very least, keeping the version check is good form.

The RoadPlayer::advance method simply uses the TimeSource::PlayerManager::next method to handle all of the interactions between the data file and the time sources.

bool RoadPlayer::advance()
{
  // advance the file pointer, if necessary, 
  // and cache the last read time for later
  // the player manager takes care of all the necessary interactions with
  // time, i.e., does reading advance time, or do we observe time to see
  // where to read.
  return _mgr.next(_play_time);
}

The RoadPlayer::getPoints method takes care of unpacking the input structure into the output STL vector of points. One thing to take notice of is the explicit release of the memory associated with the road points back to the player underlying the player manager. This is necessary because the road data structure is not flat, i.e., it points to memory for the array of data points. If we did not release the memory, we would have a memory leak. The underlying mechanism does attempt to do some caching, so there will probably not be an explicit allocation and deallocation of memory associated with the acquisition and relase of the data point array.

bool RoadPlayer::getPoints(utils::Time& time,
                           std::vector<utils::Vec3d> & points)
{
  // transfer input area points
  points.clear();
  for (int i=0;i<_input_area.num_points;i++) {
    RoadDataPoint& pt = _input_area.points[i];
    points.push_back(utils::Vec3d(pt.x, pt.y, pt.z));
  }

  // release point memory back to the player
  _player->release(_play_elem);

  // and set time from cached value
  time = _play_time;

  return true;
}

Finally there is the most public face of the RoadPlayer, RoadPlayer::nextPoints:

bool RoadPlayer::nextPoints(utils::Time& time,
                            std::vector<utils::Vec3d> & points,
                            bool blocking)
{
  if (blocking || (!blocking && _mgr.poll())) {
    if (!advance())
      return false;
  }
  return getPoints(time, points);
}

This method takes care of the intricacies of handling blocking versus non-blocking for data files as effectively as possible.

Just as RoadLogger exists primarily to support the LoggerRoadDest interface instance, RoadPlayer exists primarily to support the PlayerRoadSource interface instance.

class PlayerRoadSource : public RoadSource {
 public:
  virtual bool getPoints(utils::Time& time,
                         std::vector<utils::Vec3d>& points,
                         bool blocking = true);

  bool init(utils::ConfigFile& params, utils::SymbolTable* globals);

 private:
  RoadPlayer _player;
};

PlayerRoadSource::init and PlayerRoadSource::getPoints simply cover the RoadPlayer methods RoadPlayer::open and RoadPlayer::nextPoints.

bool PlayerRoadSource::init(utils::ConfigFile& params,
                            utils::SymbolTable* globals)
{
  return _player.open(params, globals);
}
bool PlayerRoadSource::getPoints(utils::Time& time,
                               std::vector<utils::Vec3d>& points,
                               bool blocking)
{
  return _player.nextPoints(time, points, blocking);
}

The player instance for the VehPoseSource interface is structured in a very similar way, with a standalone VehPosePlayer class and a minimal PlayerVehPoseSource interface instance definition. Just as with VehPoseLogger, one difference to point out is in VehPosePlayer::open, we break the connection of data to data structure into named fields, through

  _player->expect("x", "double", &_input_area.x);
  _player->expect("y", "double", &_input_area.y);
  _player->expect("z", "float", &_input_area.z);;
  _player->expect("ori", "[float : 4]", &_input_area.ori);

The other differences have to do with the fact that VehPosePlayer is a time driven sensor. For example, VehPosePlayer::nextPose has exactly the same functionality as RoadPlayer::nextPoints, but VehPosePlayer has an additional method, VehPosePlayer::getPose, which returns a pose at a given time, interpolating if necessary.

bool VehPosePlayer::getPose(utils::Time time, VehPose& pose)
{
  // get pose before target
  utils::Time prev_time = time;
  if (!_player->get(prev_time)) {
    return false;
  }
  VehPose prev_pose;
  set_pose(prev_pose);

  // if we have an exact match, return
  if (time == prev_time) {
    pose = prev_pose;
    return true;
  }

  // get pose after target
  utils::Time next_time;
  if (!_player->next() || !_player->process(next_time)) {
    return false;
  }

  // if we have an exact match, return
  VehPose next_pose;
  set_pose(next_pose);
  if (time == next_time) {
    pose = next_pose;
    return true;
  }

  // if we do not have two different elements, we cannot extrapolate
  if (prev_time == next_time) {
    time = utils::Time();
    return false;
  }

  // attempte to interpolate between the two poses
  double period = (next_time-prev_time).getValue();
  double elapsed = (time-prev_time).getValue();
  double t = elapsed/period;

  VehPoseSource::interpolate(prev_pose, next_pose, t, pose);
  return true;
}  

This method reaches down into the utils::Player level to get successive poses and interpolate as necessary. This is the routine that then provides the meat for the PlayerVehPoseSource::getPose:

bool PlayerVehPoseSource::getPose(utils::Time time, VehPose& pose)
{
  return _player.getPose(time, pose);
}

Communications Interfaces

At this point, module developers can most likely stop. The interface examples covered so far are sufficient to build stand-alone test modules connected to canned data, or even directly to sensors for simple kinds of modules. The next step is to build the interfaces that will connect the module to other modules together in an architecture, which is the realm of the system developers.

Now, the ModUtils package does not restrict you from using any type of communications package to connect your modules together, but it does provide a set of tools with which to implement a particular style and philosophy of communications and robot architectures that we have found to be useful.

Communications Philosophy

The vast majority of interprocess communications in robotics systems can be considered as analogous to signals in electronics. These are repeated estimations of a consistently changing value, such as the most recent line scanner data or most recent local navigation plan. It doesn't truly matter if the recipient misses a signal value: all that matters is the most recent value. What does matter is the minimization of latencies in transporting the signal value from producers to consumers. An appropriate paradigm for propagation of signal type information is global shared memory: A producer sets the memory and a consumer simply reads the most recent value. We have chosen a simple implementation for global shared memory which uses a "single-writer, multiple-reader" model. When processes are communicating on the same machine we use actual System V shared memory, whereas when processes are communicating between machines we transparently propagate changing memory values from writer to readers via the UDP socket protocol.

While signals make up the bulk of the communications, signals are not the only paradigm for interprocess communications in a robotic system. Symbols, i.e., atomic pieces of information, changes in state, or requests for information, are very difficult to communicate via a signal-based communications paradigm. For example, unlike signals, if a symbol value is dropped or missed, then information is lost, state changes don't get noticed, and requests are ignored. The guarantee that a symbol has been transported from writer to reader is worth significant additional latency and complexity in the implementation. Symbolic information is typically communicated in robotic systems via TCP/IP message based packages ranging in complexity from the raw use of socket libraries all the way up to complex, object based systems such as the Common Object Request Broker Architecture (CORBA). In order to limit the complexity and size of our software while still providing some abstraction and flexibility, we have chosen a simple TCP/IP based messaging package developed for the Navlab project: The InterProcess Toolkit (IPT) [IPT].

A key abstraction built using the messaging toolkit is the concept of a central black board. Individual algorithms mainly query the black board for their configuration parameters, but they can also post information in the black board and watch for changes in values on the black board. Thus, the black board becomes a channel for propagating information through the system that has to be generally available, but for which a certain degree of latency is acceptable. In addition, since the system's information is being funneled through the black board, we have chosen to make the black board manager the system process manager. It initiates, parameterizes, and monitors the system processes. Interestingly, this paradigm of a central black board was one of the earliest used in robotics, but it has been often rejected because if the black board is the only means for propagating information through the system, it becomes an intolerable bottleneck for the kind of low-latency, high-bandwidth signal-type information that forms the backbone of information flow for a real robotic system.

Thus we see the core of our communications philosophy: Instead of having one tool or one approach, which must be bent and stretched to handle all possible uses, we select a suite of simple tools, each one narrowly focused on a particular style of communications necessary for the efficient and successful operation of the system.

IPT Overview

IPT was originally written as a package to send point-to-point messages via TCP/.IP. There is an IPT server that brokers the connections, i.e., it provides a repository of contact information and central data base of message names and types. Thus, named clients would connect to the server, request connections to other named clients, and register messages with the server, and send those messages directly to the destination client. Recently, we extended IPT to provide the minimal cross-processor shared memory style of messaging described in the philosophy section.

Much of the IPT code is currently vestigial and, in our opinion, it should be rewritten, but it does represent a package with over a decade of use and debugging, so it is difficult to just abandon.

The IPT messaging capabilities underly much of the behind-the-scenes infrastructure provided by the ModUtils package, such as the remote configuration source capabilities or setting up shared memory, but even as a system developer you should not have to deal with them explicitly. The primary point of contact, at least for interface development, will be through the shared memory facilities, as we have discovered that the vast majority of support for interfaces will be through shared memory.

As background, on a single processor we use real Sys V shared memory to share memory. You can actually specify IPT shared memory that is purely local to the machine and only accessible by a priori knowledge of the Sys V shared memory keys. This is not very flexible, so we provide a IPT shared memory manager (accessed via IPT messages) which has several jobs

• Provide a translation of memory region names to Sys V shared memory keys
• When a shared memory region on a remote machine is requested, it contacts the shared memory manager on that machine and has it publish changes to the memory region via UDP back to this machine, and then propagates those changes through a Sys V shared memory region managed on this machine
• Monitors for changes in "subscribed" memory regions and publishes the changes via UDP.

Thus, you can create "named" memory regions and access those regions by name and machine.

As before, we will take the approach of instruction through annotated examples in order to teach how to build shared memory interfaces.

Publishing To Shared Memory

RoadDest/ShmemRoadDest.cc provides a good example of a simple shared memory output interface instance. First we must include the necessary IPT include files (and they require stdio.h as a prerequisite),

#include <stdio.h>
#include <ipt/ipt.h>
#include <ipt/sharedmem.h>

and then we define the class, including member variables for the shared memory region and an output area for packaging up the road points and define the standard creation function for this interface instance

class ShmemRoadDest : public RoadDest {
 public:
  ShmemRoadDest();
  virtual ~ShmemRoadDest();

  virtual bool outputPoints(utils::Time time,
                            const std::vector<utils::Vec3d>& points);

  bool init(utils::ConfigFile& params, utils::SymbolTable* globals);

 private:
  IPSharedMemory* _shm;  
  RoadShmemStruct _output_area;  
  unsigned int _max_points;      
};

UTILS_INTF_CREATOR(RoadDest, shmem, gen, params, globals)
{
  UTILS_INTF_REPORT(RoadDest, shmem);
  ShmemRoadDest* intf = new ShmemRoadDest();
  if (!intf->init(*params, globals)) {
    delete intf;
    return NULL;
  }
  return intf;
}

Remember that since the road output is a "non-flat" structure including an array of road points, we must deal with this memory somehow. Whereas in the data logger we allocated and deallocated all memory local to the output method, in this case we will preallocate an array of points in output_area.points, and store the number of preallocated points in the member _max_points. If we have to output more points than _max_points we can expand the cache. This pattern minimizes the amount of memory allocation and deallocation necessary to output data.

ShmemRoadDest::ShmemRoadDest()
{
  _max_points = 5;
  _output_area.data.points = new RoadDataPoint[_max_points];
}

When we delete the interface instance, we have to make sure to clean up the point data cache,

ShmemRoadDest::~ShmemRoadDest()
{
  delete [] _output_area.data.points;
}

The most complicated and arcane part about this interface is the initialization routine. The routine first either creates or accesses the created IPT "communicator" in the global symbol table via the IPCommunicator::Communicator static creation method.

bool ShmemRoadDest::init(utils::ConfigFile& params,
                         utils::SymbolTable* globals)
{
  // get or create the IPT communicator
  // If it is created, it is cached in the global symbol table
  IPCommunicator* com =
    IPCommunicator::Communicator(globals, 
                                 params.getString("ipt_spec",
                                                  "unix: int port=0;"));
  if (!com)
    return false;

If the communicator needs to be created, the parameter ipt_spec is used to specify exactly how to do it. By default, the specification is unix: int port=0;, which means we will build a communicator that can use Unix sockets when necessary and will self assign an available port. The default will normally do, but there is an IPT "feature" which means that it is difficult to have two modules with the same name, so, especially in early testing, you may find it necessary to force a different module name than the default on a shared memory user. You can do this by using the IPT specification,

spec {unix:
  int port=0;
  string ModuleName=alternate_name;
}

The init method continues to set up the shared memory region specification.

  const char* mem_name = params.getString("name", ROAD_SHMEM_NAME);
  char buffer[200];
  // first set up the default, which is based on the memory name
  sprintf(buffer, "managed: name=%s; owner=true;", mem_name);
  // and then get the spec given the default (i.e., it can be arbitrarily
  // overridden
  const char* mem_spec = params.getString("mem", buffer);

The point is to have a convenient, yet flexible way to construct a shared memory specification string. The standard way for a client to use the shared memory directly is through the shared memory manager, i.e., you are created "managed" memory, which is memory that has a name managed by the IPT shared memory manager. You provide the name of the memory through the name parameter and the routine will construct the proper managed, owned specification. Alternatively, you can directly specify the memory with the mem parameter.

Then comes the actual creation of the shared memory region.

  int max_points = params.getInt("max_points", 20);
  // create the shared memory region
  _shm =
    com->OpenSharedMemory(mem_spec, ROAD_SHMEM_FMT,
                          sizeof(RoadShmemStruct) +
                          max_points*sizeof(RoadDataPoint));
  if (!_shm) {
    printf("Problem opening shared memory %s\n", mem_spec);
    return false;
  }

  return true;
}

We have to pass the IPCommunicator::OpenSharedMemory method our constructed (or read from mem) memory specification, the structure format of the memory (ROAD_SHMEM_FMT), and the size of the memory region. Note that the shared memory region is a fixed size, while the road data structure can have an arbitrarily sized number of points. You will have to use the max_points method to make a shared memory region big enough for the largest number of points expected.

The ShmemRoadDest::outputPoints method makes sure the point cache can handle the number of points, fills out the _output_area, and then uses the IPSharedMemory::PutFormattedData to output the data.

bool ShmemRoadDest::outputPoints(utils::Time time,
                                 const std::vector<utils::Vec3d>& points)
{
  // make sure there is enough room in the cached points for the data
  if (points.size() > _max_points) {
    _max_points = points.size();
    delete [] _output_area.data.points;
    _output_area.data.points = new RoadDataPoint[_max_points];
  }

  // setup the output area
  _output_area.data.num_points = (int) points.size();
  for (int i=0;i<_output_area.data.num_points;i++) {
    const utils::Vec3d& src = points[i];
    RoadDataPoint& dest = _output_area.data.points[i];
    dest.x = src.x;
    dest.y = src.y;
    dest.z = src.z;
  }
  time.getValue(_output_area.secs, _output_area.usecs);

  // and output to shared memory
  _shm->PutFormattedData((void*) &_output_area);

  return true;
}

Our other example file, VehPoseDest/ShmemVehPoseDest.cc is almost completely analogous, except that since VehPoseShmemStruct is a flat structure, we do not have to do any caching and can create a truly fixed memory region like this,

  _shm =
    com->OpenSharedMemory(mem_spec, VEH_POSE_SHMEM_FMT,
                          sizeof(VehPoseShmemStruct));

Data Driven Interfaces and Shared Memory

Shared memory data driven interface instances are very straightforward to implement. As RoadSource/ShmemRoadSource.cc shows, we simply include many of the same support files and define our class as with any basic interface.

class ShmemRoadSource : public RoadSource {
 public:
  ShmemRoadSource();

  virtual bool getPoints(utils::Time& time,
                         std::vector<utils::Vec3d>& points,
                         bool blocking = true);

  bool init(utils::ConfigFile& params, utils::SymbolTable* globals);

 private:
  IPSharedMemory* _shm;  
  int _last_tag;  
};

Note the _last_tag member. Every piece of memory has associated with it a tag, which is just a number which is incremented whenever new data is received into the shared memory region. We will keep track of the tag of the last data reported to the user so that we can tell the user if this is new data or not. For bookkeeping, this means we have to initialize the last tag to -1:

ShmemRoadSource::ShmemRoadSource()
{
  _last_tag = -1;
}

In the init method, accessing the IPT communicator is done in exactly the same way as in the Dest shared memory interfaces:

bool ShmemRoadSource::init(utils::ConfigFile& params,
                           utils::SymbolTable* globals)
{
  // get or create the IPT communicator
  // If it is created, it is cached in the global symbol table
  IPCommunicator* com =
    IPCommunicator::Communicator(globals, 
                                 params.getString("ipt_spec",
                                                  "unix: int port=0;"));
  if (!com)
    return false;

The shared memory specification for a Source interface is a bit more complicated than for a Dest interface. Now you can provide both a name for the managed shared memory name and the machine and port number for a remote shared memory manager. If you omit the machine parameter, the local host is assumed, and the default value for the port parameter is 1389. The init code takes these parameters and creates a managed shared memory specification which is used unless you use the mem specifier to override it.

  const char* mem_name = params.getString("name", ROAD_SHMEM_NAME);
  const char* machine = params.getString("machine");
  int port = params.getInt("port", 1389);
  char buffer[200];
  if (!*machine) {
    sprintf(buffer, "managed: name=%s;", mem_name);
  } else {
    sprintf(buffer, "managed: name='%s@%s|%d';", mem_name, machine, port);
  }
  const char* mem_spec = params.getString("mem", buffer);

Finally, we access the memory in the same way as in the ShmemRoadDest interface instance. Again, we must cap the variable number of points with the max_points parameter. Note that if we request access to a shared memory region with a larger size than what it was created, the shared memory access initialization will fail.

  int max_points = params.getInt("max_points", 20);
  // create the shared memory region
  _shm =
    com->OpenSharedMemory(mem_spec, ROAD_SHMEM_FMT,
                          sizeof(RoadShmemStruct) +
                          max_points*sizeof(RoadDataPoint));
  if (!_shm) {
    printf("Problem opening shared memory %s\n", mem_spec);
    return false;
  }

  return true;
}

If the blocking parameter is true, the ShmemRoadSource::getPoints first blocks on the shared memory by using the shared memory's Wait method.

bool ShmemRoadSource::getPoints(utils::Time& time,
                                std::vector<utils::Vec3d>& points,
                                bool blocking)
{
  if (blocking) {
    // wait for new data in the shared memory region
    if (!_shm->Wait()) {
      // if waiting failed, mark result with a bad time and return false
      time = utils::Time();
      return false;
    }
  }

Then the ShmemRoadSource::getPoints method goes on to acquire the current data with the shared memory's FormattedData method, which copies the data according to the structure format specifier into the input area.

  RoadShmemStruct input_area;  
  if (!_shm->FormattedData(&input_area)) {
    // if there was a formatting failure, mark as bad and return
    time = utils::Time();
    return false;
  }

Then the method packages the data from the input area to the STL vector of points and time.

  time.setValue(input_area.secs, input_area.usecs);
  points.clear();
  for (int i=0;i<input_area.data.num_points;i++) {
    RoadDataPoint& pt = input_area.data.points[i];
    points.push_back(utils::Vec3d(pt.x, pt.y, pt.z));
  }

Since the road data structure is not flat, we must release the array of points back to the shared memory system so that it can reuse it for the next time, if possible.

  _shm->DeleteContents(&input_area);

And finally, we determine if this is "new" data or not by comparing the shared memory tag (given by the Tag method) with the last stored tag. If this is new data we return true, if not, we return false.

  if (_shm->Tag() != _last_tag) {
    _last_tag = _shm->Tag();
    return true;
  } else 
    return false;
}

Time Driven Interfaces and Shared Memory

Time driven interfaces need to have a more complicated interface to shared memory, as they are not just returning the most recent changes but must keep a record of a history of data in order to interpolate between elements to return an estimate of the data at the requested time.

The example file in VehPoseSource/ShmemVehPoseSource.cc

shows one approach to this. In this example we create a sub-thread to constantly monitor the shared memory region and create a ring buffer of vehicle poses which can be used to interpolate and extrapolate vehicle poses to get the vehicle pose at the requested time.

We implement this class with a variety of support variables and methods to manage and communicate with the sub-thread.

class ShmemVehPoseSource : public VehPoseSource {
public:
  ShmemVehPoseSource();
  virtual ~ShmemVehPoseSource();

  virtual bool getPose(utils::Time time, VehPose& pose);

  virtual bool getCurPose(utils::Time& time,
                          VehPose& pose, bool blocking = false);

  bool init(utils::ConfigFile& params, utils::SymbolTable* globals);

  static void interpolate(const VehPoseShmemStruct& prev_pose, 
                          const VehPoseShmemStruct& next_pose, double t,
                          VehPose& veh_pose);


private:
  static void* thread_entry(void*);
  void collector_thread();
  void error(VehPose&); 
  static void set_pose(const VehPoseShmemStruct& input, VehPose& output);

private:
  IPCommunicator* _com;  // the IPT communicator
  IPSharedMemory* _shm;  // the pose source shared memory region

  bool _collector_running;  // true if the collector thread is running
  pthread_t _collector_t;  // the collector thread ID
  pthread_mutex_t _collector_mutex;  // mutex which guards ring buffer
  pthread_cond_t _collector_cond;  // conditional which signals new pose

  int _history_length;  // the size of the pose ring buffer
  int _num_poses;      // number of poses in the ring buffer
  int _cur_pose_index;  // most recent pose index in the ring buffer
  VehPoseShmemStruct* _poses;  // the ring buffer

  int _last_secs, _last_usecs;  // time stamp of the last pose received
                                // by getCurPose
  float _max_extrapolation;  // how far can we extrapolate pose information?
};

It is a bad idea to initialize a sub-thread until it is really needed, so we initialize many of the bookkeeping variables to indicate nothing is running:

ShmemVehPoseSource::ShmemVehPoseSource()
{
  _shm = NULL;
  _collector_running = false;
  _poses = NULL;
  _last_secs = _last_usecs = -1;
}

and the destructor checks to see if the subthread is running and share memory is setup before trying to destroy anything,

ShmemVehPoseSource::~ShmemVehPoseSource()
{
  // close shared memory, if necessary
  if (_shm)
    _com->CloseSharedMemory(_shm);

  // shutdown collector thread and cleanup, if necessary
  if (_collector_running) {
    _collector_running = false;
    pthread_cancel(_collector_t);
    pthread_mutex_destroy(&_collector_mutex);
    pthread_cond_destroy(&_collector_cond);
  }

  delete [] _poses;
}

Even for this interface, the init method looks very familiar: Setting up the communicator, attaching to the fixed size shared memory region, etc. The major difference comes in the end where we set up the ring buffer and kick of the sub-thread which will access the shared memory. The ring buffer has a maximum size (in elements), history_length, and we also have a parameter for how far into the future we will extrapolate data, max_extrapolation.

  _history_length = params.getInt("history_length",  100);
  _num_poses = 0;
  _cur_pose_index = -1;
  _poses = new VehPoseShmemStruct[_history_length];

  _max_extrapolation = params.getFloat("max_extrapolation", 0.2);

  // and finally, kick off the collector thread
  pthread_mutex_init(&_collector_mutex, NULL);
  pthread_cond_init(&_collector_cond, NULL);
  _collector_running = true;
  pthread_create(&_collector_t, NULL, thread_entry, this);

The sub-thread just sits in a loop in the method ShmemVehPoseSource::collector_thread. This thread simply blocks on the shared memory region and inserts "new" vehicle poses into the history ring buffer. As a side effect it uses the pthread conditional signal, _collector_cond, to flag that new data is coming in for the main thread. If the member variable _collector_running goes false, it exits gracefully.

void ShmemVehPoseSource::collector_thread()
{
  VehPoseShmemStruct incoming;
  while (_collector_running) {
    // wait for new data in the shared memory
    if (!_shm->Wait()) {
      utils::Time::sleep(0.02);  // make sure we don't busy wait
      continue;
    }

    // unmarshal it and stick pose in incoming
    _shm->FormattedData((void*) &incoming);
    // the time tag of the pose can get set to 0 on startup and shutdown
    // of the module producing the poses.  Just skip these.
    if (!incoming.secs && !incoming.usecs)
      continue;

    // Put the new pose in the ring buffer
    pthread_mutex_lock(&_collector_mutex);
    if (_num_poses != _history_length)
      _num_poses++;
    _cur_pose_index = (_cur_pose_index + 1) % _history_length;
    _poses[_cur_pose_index] = incoming;
    pthread_mutex_unlock(&_collector_mutex);

    // signal we have new data for anyone blocking in getCurPose
    pthread_cond_signal(&_collector_cond);
  }
}

The workhorse method for the VehPoseSource interface class is getPose, which returns the vehicle pose at a requested time, if possible. ShmemVehPoseSource::getPose first locks access to the ring buffer with a mutex, and then sees if the requested time is after the latest element in the ring buffer. If it is and the time is less than max_extrapolation seconds in the future it extrapolates based on the last two vehicle poses a new vehicle pose.

bool ShmemVehPoseSource::getPose(utils::Time now, VehPose& veh_pose)
{
  // lock the ring buffer
  pthread_mutex_lock(&_collector_mutex);

  // Get the latest sensor pose
  VehPoseShmemStruct* cur_pose = &_poses[_cur_pose_index];
  utils::Time t(cur_pose->secs, cur_pose->usecs);

  // if the requested time is after the latest sensor pose
  if (now > t) {
    // figure out if we can extrapolate
    double elapsed = (now - t).getValue();
    if (elapsed > _max_extrapolation) {
      fprintf(stderr, "ShmemVehPoseSource::getPose: "
              "Pose lookup time too far in the future, delta = %f\n",
              (now - t).getValue());
      error(veh_pose);
      return false;
    }
    // default all fields to cur_pose value.
    set_pose(*cur_pose, veh_pose);

    // get previous pose
    VehPoseShmemStruct* prev_pose;
    utils::Time pt;
    int prev_index = _cur_pose_index;
    while (1) {
      prev_index--;
      if (prev_index < 0 && _num_poses < _history_length) {
        fprintf(stderr, "ShmemVehPoseSource::getPose: "
                "Cannot extrapolate yet (%5.2f)\n",
                (now - t).getValue());
        error(veh_pose);
        return false;
      }
      prev_pose = &_poses[prev_index];
      pt.setValue(prev_pose->secs, prev_pose->usecs); 
      if (pt > t) {
        fprintf(stderr, "ShmemVehPoseSource::getPose: "
                "Cannot extrapolate with current history (%5.2f)\n",
                (now - t).getValue());
        error(veh_pose);
        return false;
      }
      if (pt != t)
        break;
    }
    
    // extrapolate pose (with t > 1)
    interpolate(*prev_pose, *cur_pose, 1 + double(now-t)/(t-pt), veh_pose);

    // unlock the ring buffer and return
    pthread_mutex_unlock(&_collector_mutex);
    return true;
  }

Note that the extrapolation would be easier and probably more accurate if the VehPose contained an estimate of the velocities of the various quantities as well as the value. Then we could simply use the most recent element all by itself to generate extrapolations instead of having to use two elements to estimate the velocities.

If we are not extrapolating, the method then searches through the ring buffer to find the two entries which bracket the requested time. If there are no such entries, we return false. If there are, we return true with the interpolated vehicle pose set in veh_pose.

  int cur_index = _cur_pose_index;
  VehPoseShmemStruct* prev_pose = cur_pose;
  for (int i=0;i<_num_poses-1;i++) {
    cur_index--;
    if (cur_index < 0)
      cur_index = _history_length-1;
    prev_pose = &_poses[cur_index];
    t.setValue(prev_pose->secs, prev_pose->usecs);
    if (now > t) {
      // we have found the bracketing elements

      utils::Time cur(cur_pose->secs, cur_pose->usecs);
      double dist = (now-t).getValue()/(cur-t).getValue();

      // interpolate appropriately
      interpolate(*prev_pose, *cur_pose, dist, veh_pose);

      // unlock the ring buffer and return
      pthread_mutex_unlock(&_collector_mutex);
      return true;
    }
    cur_pose = prev_pose;
  }

  // requrest time is too far in the past
  fprintf(stderr, "ShmemVehPoseSource::getPose: "
          "State lookup time too old, delta = %f\n",
          (now - TimeSource::now()).getValue());
  error(veh_pose);

  return false;
}

The ShmemVehPoseSource::getCurPose method does not access the shared memory region directly, instead it uses the pthread conditional signal to block if necessary and returns the values stored in the latest element of the ring buffer.

bool ShmemVehPoseSource::getCurPose(utils::Time& time, VehPose& veh_pose,
                                    bool blocking)
{
  // lock the ring buffer
  pthread_mutex_lock(&_collector_mutex);

  if (blocking) {
    // if we are not at the first one and do not have new data
    if (_cur_pose_index < 0 ||
        (_poses[_cur_pose_index].secs == _last_secs &&
         _poses[_cur_pose_index].usecs == _last_usecs)) {
      // wait for new data
      if (pthread_cond_wait(&_collector_cond, &_collector_mutex)) {
        pthread_mutex_unlock(&_collector_mutex);
        perror("ShmemVehPoseSource::getCurPose: waiting for condition");
        error(veh_pose);
        return false;
      }
    }
  } 

  // get the latest sensor pose
  VehPoseShmemStruct& result = _poses[_cur_pose_index];
  set_pose(result, veh_pose);
  time.setValue(result.secs, result.usecs);

  // mark if this is new data or not
  bool res = !(_last_secs == result.secs && _last_usecs == result.usecs);
  _last_secs = result.secs;
  _last_usecs = result.usecs;

  // unlock the ring buffer
  pthread_mutex_unlock(&_collector_mutex);

  return res;
}

Building Interface Plug-ins Independently

A major advantage of the dynamically loadable plug-in approach is that reconfigurable interface instance plug-ins can be built anywhere in your code tree anytime after the original reconfigurable interface library, just as long as they are installed in the proper directory so the run-time loader can find it when generating the instance.

A common usage for this is when you are connecting to a particular hardware device for a particular project. In this case, one of the best patterns found is to define a class to interface with the hardware with all of the hardware-specific parameters for adjusting and parameterizing the device. This class is then embedded in a module which allows the user to graphically adjust the various parameters while collecting data from the device and outputting the data to a reconfigurable "destination" interface. This destination interface can then be configured to log to a file or output through shared memory to the rest of the system as necessary.

Now, if the device-specific class is setup as a sub-class of an appropriate reconfigurable interface "source" class which be initialized from a utils::ConfigFile as well as from methods manipulated by a GUI, we can actually install this interface into the global or project interface directory as a plug-in so that the ability to connect directly to the hardware device becomes a potential part of any module in the system.

As an example, imagine we are connecting to a high quality positioning system such as an Applanix. In this case, we will create an ApplanixPose module which will interface with the applanix via a new ApplanixVehPoseSource class (a new subclass of VehPoseSource). The ApplanixVehPoseSource class uses primitive routines defined in a low level applanix library. ApplanixPose also creates a GUI with FLTK which allows the user to manipulate the Applanix parameters and monitor the Applanix specific status information through new methods in the ApplanixVehPoseSource class not defined in the superclass (and thus not generally available through the VehPoseSource abstract interface).

The makefile for this fictional module should look like this:

include ${UTILS_DIR}/include/maketools/MakeDefs
include $(UTILS_DIR)/include/maketools/fltk.mk
include $(UTILS_DIR)/include/VehPoseSource/VehPoseSource.mk

TARGET = ApplanixPose
TARGET_OBJS = ApplanixPose.o ApplanixGUI.o ApplanixVehPoseSource.o
TARGET_LIBS = -lVehPoseSource -lVehPoseDest
EXTRA_INC_DIRS = $(FLTK_INC)
EXTRA_TARGET_LIBS = $(FLTK_LIB) -lapplanix

INTF_OBJS = ApplanixVehPoseSource.o
EXTRA_BUILDS = $(INTERFACE_DIR)/VehPoseSource/applanix.so
$(INTERFACE_DIR)/VehPoseSource/applanix.so: ApplanixVehPoseSource.o
        $(MAKE_INTF_EXTERNAL) $(VehPoseSource_LIBS) -lapplanix

include ${UTILS_DIR}/include/maketools/MakeTail

As you might expect, this makefile is a blending of a module makefile and an interface makefile. Some important differences:

• You do not define the SRC_MODULE variable. This means that no include files or libraries will be created and installed.
• You do not define the INTERFACE_DEST variable. The INTERFACE_DEST variable ensures that the interface destination directory exists, and since you are building this after the relevant interface library has already been built, this should be taken care of
• You use the MAKE_INTF_EXTERNAL macro in the interface build rules instead of the MAKE_INTF macro. This macro requires you to explicitly list the relevant interface library, in this case as defined by the VehPoseSource_LIBS variable defined in the VehPoseSource.mk makefile fragment
• If the interface objects you are defining are on the TARGET_OBJS list then you do not have to explicity define them on the INTF_OBJS list, but it doesn't hurt.
• Libraries required by the interface and the target (like -lapplanix) must be replicated both on the EXTRA_TARGET_LIBS line and on the MAKE_INTF_EXTERNAL line.

Occasionally when creating interface extensions you will find you want to install header files in with the other reconfigurable interface header files. In this case, for this example you would have something like this to the top of the makefile,

SRC_MODULE = VehPoseSource
NO_MKFILE = true

and later after the inclusion of the MakeDefs fragment,

HEADERS = Applanix.h

The result would install the header file Applanix.h in $(INSTALL_DIR)/include/VehPoseSource. Setting the NO_MKFILE means that you will not overwrite $(INSTALL_DIR)/include/VehPoseSource/VehPoseSource.mk

---