Data Generation Tutorial

## Overview The example program `polysync-data-generator-cpp` is included in the PolySync installation. It is designed to populate PolySync messages with sample data and publish those messages to the global, shared bus. The sample data comes in multiple types, each with its own associated high-level PolySync message: * Radar Targets: `ps_radar_targets_msg` * Classified Objects: `ps_objects_msg` * LiDAR Points: `ps_lidar_points_msg` PolySync messages are defined in multiple data model modules: * [Core Messages](http://docs.polysync.io/#c_data_model-core-messages) * [Control Messages](http://docs.polysync.io/#c_data_model-control-messages) * [Sensor Messages](http://docs.polysync.io/#c_data_model-sensor-messages) ### 1. Running the data generator Begin by starting the PolySync manager in background mode. Open a new terminal and run: ```bash $ polysync-manager ``` Once the PolySync manager is running, start the PolySync data generator (included as part of the PolySync installation) in a separate terminal by running: ```bash $ polysync-data-generator-cpp ``` ### 1.2 Generate data You can find the C++ Data Generation code─by default─in `/usr/local/polysync/examples/cpp/DataGenerator`. ``` /usr/local/polysync/examples/cpp/DataGenerator/ ├── CMakeLists.txt ├── DataGenerator.cpp ├── LidarPointGenerator.cpp ├── LidarPointGenerator.hpp ├── ObjectGenerator.cpp ├── ObjectGenerator.hpp ├── RadarTargetGenerator.cpp └── RadarTargetGenerator.hpp ``` ### 1.2.1 DataGenerator.cpp This is where the PolySync node is defined, and where you can access the main entry point for this node/application. ```cpp #include < iostream > #include < PolySyncNode.hpp > #include < PolySyncDataModel.hpp > #include "LidarPointGenerator.hpp" #include "RadarTargetGenerator.hpp" #include "ObjectGenerator.hpp" using namespace polysync::datamodel; class DataGenerator : public polysync::Node { protected: virtual void okStateEvent() { _lidarPointGenerator->updatePoints(); _lidarPointGenerator->publishPoints(); _radarTargetGenerator->updateTargets(); _radarTargetGenerator->publishTargets(); _objectGenerator->updateObjects(); _objectGenerator->publishObjects(); polysync::sleepMicro( _updateInterval ); } virtual void initStateEvent() { _lidarPointGenerator = std::unique_ptr< LidarPointGenerator >{ new LidarPointGenerator( *this ) }; _radarTargetGenerator = std::unique_ptr< RadarTargetGenerator >{ new RadarTargetGenerator( *this ) }; _objectGenerator = std::unique_ptr< ObjectGenerator >{ new ObjectGenerator( *this ) }; } private: ps_timestamp _updateInterval{ 50000 }; std::unique_ptr< LidarPointGenerator > _lidarPointGenerator; std::unique_ptr< RadarTargetGenerator > _radarTargetGenerator; std::unique_ptr< ObjectGenerator > _objectGenerator; }; int main() { DataGenerator dataGenerator; dataGenerator.connectPolySync(); return 0; } ``` ### 1.2.2 DataGenerator.cpp explained Now let’s break down the DataGenerator.cpp code. The line below defines the [PolySync node](http://docs.polysync.io/#cpp-node) and each of the protected event methods in the node state machine. Each of the states do minimal or no operation unless they are overridden. ```cpp #include < PolySyncNode.hpp > ``` The following [data model](http://docs.polysync.io/#cpp-data-model) defines all types and messages in the PolySync runtime. This **must** be included in all C++ node applications. ```cpp #include < PolySyncDataModel.hpp > ``` The following headers are specific to this application and define the data "generators." They are responsible for creating, populating, and updating the respective message types with data. ```cpp #include "LidarPointGenerator.hpp" #include "RadarTargetGenerator.hpp" #include "ObjectGenerator.hpp" ``` All of our PolySync C++ applications must then subclass the PolySync node object in order to connect to the PolySync bus and publish/subscribe messages. ```cpp class DataGenerator : public polysync::Node ``` The following is called continuously while in the node's OK state, and is where almost all of the action happens for the node. ```cpp virtual void okStateEvent() ``` For each execution loop, each of the three primitive data types will be updated in a new message instance. These are then published to the bus. ```cpp _lidarPointGenerator->updatePoints(); _lidarPointGenerator->publishPoints(); _radarTargetGenerator->updateTargets(); _radarTargetGenerator->publishTargets(); _objectGenerator->updateObjects(); _objectGenerator->publishObjects(); ``` This is called once after the node transitions to the INIT state. The initStateEvent is typically where a node creates messages and initializes any other resources that require a PolySync node reference. ```cpp virtual void initStateEvent() ``` This creates a reference to the new LiDAR point generator instance. ```cpp _lidarPointGenerator = std::unique_ptr< LidarPointGenerator >{ new LidarPointGenerator( *this ) }; ``` The following is how you can control the execution speed of the state machine. The `_updateInterval` is called at the end of each event, whether it has been overridden or not. ```cpp ps_timestamp _updateInterval{ 50000 }; ``` This creates a reference for each of your data generator objects defined in their unique header and source files. ```cpp std::unique_ptr< LidarPointGenerator > _lidarPointGenerator; std::unique_ptr< RadarTargetGenerator > _radarTargetGenerator; std::unique_ptr< ObjectGenerator > _objectGenerator; ``` This creates an instance of your `DataGenerator` object. The second command is a blocking operation, and places the node in the node state machine. If there is a valid license, the node leaves the AUTH state and enters the INIT state, calling the `initStateEvent`. ```cpp DataGenerator dataGenerator; dataGenerator.connectPolySync(); ``` ### 1.3 Generate LiDAR points Th LiDAR point generator defines the class that is used to create, initialize, populate, and publish LiDAR message on PolySync. This is reimplemented in `RadarTargetGenerator.cpp` and `ObjectGenerator.cpp` for a RADAR generator class and an object's generator class. ```cpp #include < PolySyncNode.hpp > #include < PolySyncDataModel.hpp > class LidarPointGenerator { public: LidarPointGenerator( polysync::Node & ); void updatePoints(); void publishPoints(); void initializeMessage(); private: polysync::datamodel::LidarPointsMessage _message; float _relativeTime{ 0.0 }; const float _gridScale{ 10.0 }; const ulong _gridSideLength{ 100 }; const ulong _sensorID{ 11 }; const ulong _numberOfPoints{ 10000 }; const float _sineFrequency{ 4.0 }; }; ``` ### 1.3.1 LidarPointsGenerator.hpp explained Now let’s break down the LidarPointsGenerator.hpp code. ```cpp #include < PolySyncNode.hpp > #include < PolySyncDataModel.hpp > ``` The three primitive data types (LiDAR, RADAR, objects) each create a class with a basic PolySync node constructor and three functions to initialize, update, and publish data to the bus. The LiDAR point generator class is then called. ```cpp class LidarPointGenerator { ... }; ``` This class must be constructed with a valid node reference, which happens in the `initStateEvent` defined in the `DataGenerator.cpp` file. ```cpp public: LidarPointGenerator( polysync::Node & ); }; ``` This will be used to populate the `LidarPointsMessage` with a valid sensor descriptor─describing this node as a publisher on the bus─as well as set the initial header, and message start/end timestamps. Once the message is initialized, you will use function below to call `updatePoints`. ```cpp public: ... void initializeMessage(); }; ``` For each message published to the bus, the node needs to update each of the valid fields within the message. `updatePoints` iterates over a vector of PolySync `LidarPoint` and updates each LiDAR points: * Intensity * x, y and z position Each point in the vector is updated to represent a sine wave. ```cpp public: ... void updatePoints(); }; ``` After you're done setting the header timestamp, the following will publish the `LidarPointsMessage` with an updated vector of points to the PolySync bus. ```cpp public: ... void publishPoints(); }; ``` Next, you will call this high-level message used to store LiDAR points. ```cpp private: polysync::datamodel::LidarPointsMessage _message; }; ``` The variables are then used to calculate the sine wave and determine the number of LiDAR points required to represent it. ```cpp private: ... float _relativeTime{ 0.0 }; const float _gridScale{ 10.0 }; const ulong _gridSideLength{ 100 }; const ulong _numberOfPoints{ 10000 }; const float _sineFrequency{ 4.0 }; }; ``` ### 1.3.2 LidarPointsGenerator.cpp ```cpp #include "LidarPointGenerator.hpp" using namespace std; using namespace polysync::datamodel; LidarPointGenerator::LidarPointGenerator( polysync::Node & node ) : _message( node ), _numberOfPoints( _gridSideLength * _gridSideLength ) { initializeMessage(); } void LidarPointGenerator::initializeMessage() { polysync::datamodel::SensorDescriptor descriptor; descriptor.setTransformParentId( PSYNC_COORDINATE_FRAME_LOCAL ); descriptor.setType( PSYNC_SENSOR_KIND_NOT_AVAILABLE ); _message.setSensorDescriptor( descriptor ); auto time = polysync::getTimestamp(); _message.setHeaderTimestamp( time ); _message.setStartTimestamp( time ); _message.setEndTimestamp( time ); updatePoints(); } void LidarPointGenerator::updatePoints() { auto time = polysync::getTimestamp(); auto timeDelta = time - _message.getStartTimestamp(); auto timeDeltaSeconds = static_cast< float >( timeDelta ) / 1000000.0; _relativeTime += timeDeltaSeconds; _message.setStartTimestamp( time ); _message.setEndTimestamp( time ); std::vector< LidarPoint > outputPoints; outputPoints.reserve( _numberOfPoints ); for( auto pointNum = 0U; pointNum < _numberOfPoints; ++pointNum ) { polysync::datamodel::LidarPoint point; point.setIntensity( 255 ); auto x = pointNum % 100; auto y = pointNum / 100; float u = static_cast< float >( x )/ 100.0; float v = static_cast< float >( y ) / 100.0; // center u/v at origin u = ( u * 2.0 ) - 1.0; v = ( v * 2.0 ) - 1.0; float w = sin( ( u * _sineFrequency ) + _relativeTime ) * cos( ( v * _sineFrequency ) + _relativeTime ) * 0.5; point.setPosition( { u * 10, v * 10, w * 10 } ); outputPoints.emplace_back( point ); } _message.setPoints( outputPoints ); } void LidarPointGenerator::publishPoints() { _message.setHeaderTimestamp( polysync::getTimestamp() ); _message.publish(); } ``` ### 1.3.3 LidarPointsGenerator.cpp explained Now let’s break down the LidarPointsGenerator.cpp code. During the object’s construction, the LiDAR message is initialized with the node reference provided from the `DataGenerator.cpp` `initStateEvent` function call. Once you initialize the message here, the message fields can be set to the desired values. ```cpp LidarPointGenerator::LidarPointGenerator( polysync::Node & node ) : _message( node ), _numberOfPoints( _gridSideLength * _gridSideLength ) { initializeMessage(); } ``` The sensor descriptor is used by the subscribing node to determine which node the message originated from. It is not mandatory, but we highly recommend it as a means to populate the sensor descriptor for each message instance. The `TransformParentId` represents the active [coordinate frame identifier](/articles/?p=1323). In this case, you are setting the active coordinate frame to the local frame, which means no transform corrections are needed for outgoing data. Messages should be initialized with the current UTC timestamp, [multiple timestamps](/articles/?p=573) fields exist. ```cpp void LidarPointGenerator::initializeMessage() { polysync::datamodel::SensorDescriptor descriptor; descriptor.setTransformParentId( PSYNC_COORDINATE_FRAME_LOCAL ); descriptor.setType( PSYNC_SENSOR_KIND_NOT_AVAILABLE ); _message.setSensorDescriptor( descriptor ); auto time = polysync::getTimestamp(); _message.setHeaderTimestamp( time ); _message.setStartTimestamp( time ); _message.setEndTimestamp( time ); updatePoints(); } ``` This function is responsible for setting each point in the LiDAR points message, stored as a vector of PolySync `LidarPoint`. You should be aware that the **namespace is `polysync::data model`**. Timestamp fields are updated for each iteration that the message is updated and subsequently published by `publishPoints()`. PolySync timestamps can be referenced globally across the PolySync bus, and they are used here both to calculate the sine wave, as well as in the message timestamp fields. ```cpp void LidarPointGenerator::updatePoints() { auto time = polysync::getTimestamp(); auto timeDelta = time - _message.getStartTimestamp(); auto timeDeltaSeconds = static_cast< float >( timeDelta ) / 1000000.0; _relativeTime += timeDeltaSeconds; _message.setStartTimestamp( time ); _message.setEndTimestamp( time ); } ``` Each point is iterated over to set the intensity, and x/y/z position values before it's emplaced within the vector. The following `for` loop will be used to calculate the position of each point to represent a sine wave. Once the vector is filled the `setPoints` member function is used to copy the local vector into the message. ```cpp std::vector< LidarPoint > outputPoints; outputPoints.reserve( _numberOfPoints ); for( auto pointNum = 0U; pointNum < _numberOfPoints; ++pointNum ) { polysync::datamodel::LidarPoint point; point.setIntensity( 255 ); auto x = pointNum % 100; auto y = pointNum / 100; float u = static_cast< float >( x )/ 100.0; float v = static_cast< float >( y ) / 100.0; // center u/v at origin u = ( u * 2.0 ) - 1.0; v = ( v * 2.0 ) - 1.0; float w = sin( ( u * _sineFrequency ) + _relativeTime ) * cos( ( v * _sineFrequency ) + _relativeTime ) * 0.5; point.setPosition( { u * 10, v * 10, w * 10 } ); outputPoints.emplace_back( point ); } _message.setPoints( outputPoints ); } ``` Now the header timestamp will be set─immediately before publishing─to represent when the UTC timestamp of this message will be published to the PolySync bus. ```cpp void LidarPointGenerator::publishPoints() { _message.setHeaderTimestamp( polysync::getTimestamp() ); _message.publish(); } ``` ### 2. Visualize data Once a node─in this case `polysync-data-generator-cpp`─is publishing data to the PolySync bus, any other application can access the data by subscribing to the PolySync message type(s). PolySync provides two applications that are already set up to subscribe to the message types published by this node, and visualize the data in multiple ways: * PolySync Studio * A Qt based application that has multiple plugins providing different ways to view the data on the bus. * Viewer Lite * An OpenGL 2D visualizer application that allows you to see primitive data types being transmitted using the PolySync bus. ### 2.1 Viewer Lite To run PolySync Viewer Lite input the following command: ```bash $ polysync-viewer-lite ``` ![DataGen](https://help.polysync.io/releases/2.0.8//wp-content/uploads/2016/08/datagen1.png) Your sample node should display the following: * A LiDAR "point cloud” as the the block in the center * Two “identified objects” as the rectangles on the upper right side of the point cloud. These are similar to those that you might get from an object identification sensor * Two circles representing RADAR targets on the upper left side of the point cloud PolySync Viewer Lite does not allow for much distinction in the point cloud, but updates to the object data and RADAR pings that are immediately obvious. ### 2.2 Studio In this instance, the 3D View plugin is the easiest way to view the provided data. Leave the PolySync Viewer Lite application running, and either double-click the PolySync Studio icon on your desktop or execute the following command in a new terminal: ```bash $ polysync-studio ``` When Studio opens, you will be able to visualize the data that the PolySync Data Generator is creating, select the “3D View” button on the right. With that item selected, PolySync Studio will display something similar to the image: ![DataGen](https://help.polysync.io/releases/2.0.8//wp-content/uploads/2016/08/datagen3.png) ### Conclusion Congratulations! You have now successfully populated and published data from a single node into a PolySync message on the bus, enabled multiple nodes to subscribe to messages to access data asynchronously, and visualized all data being published to the PolySync bus.