Added 3d asynchronous functionality and example

doc/modules/3d_reconstruction_module.md

@@ -109,7 +109,7 @@ It should be similar to the following image.
 You can copy and modify the OpenPose 3-D demo to use any camera brand by:
 
 1. You can optionally turn off the `WITH_FLIR_CAMERA` while compiling CMake.
-2. Copy `examples/tutorial_api_cpp/13_synchronous_custom_input.cpp` (or `17_synchronous_custom_all_and_datum.cpp`).
+2. Copy `examples/tutorial_api_cpp/14_synchronous_custom_input.cpp` (or `18_synchronous_custom_all_and_datum.cpp`).
 3. Modify `WUserInput` and add your custom code there. Your code should fill `Datum::name`, `Datum::cameraMatrix`, `Datum::cvInputData`, and `Datum::cvOutputData` (fill cvOutputData = cvInputData).
 4. Remove `WUserPostProcessing` and `WUserOutput` (unless you want to have your custom post-processing and/or output).
 

doc/release_notes.md

@@ -385,19 +385,26 @@ OpenPose Library - Release Notes
 
 ## Current version - Future OpenPose 1.6.0
 1. Main improvements:
-    1. Created Matrix as container of cv::Mat, and String as container of std::string.
-    2. After replacing cv::Mat by Matrix, headers do not contain any 3rd-party library includes nor functions. This way, OpenPose can be exported without needing 3rd-party includes nor static library files (e.g., lib files in Windows), allowing people to use their own versions of OpenCV, Eigen, etc. without conflicting with OpenPose. Dynamic library files (e.g., `dll` files in Windows, `so` in Ubuntu) are still required.
-    3. Created the `openpose_private` directory with some internal headers that, if exported with OpenPose, would require including 3rd-party headers and static library files.
-    4. Default OpenCV version for Windows upgraded to version 4.2.0, extracted from their oficial website: section `Releases`, subsection `OpenCV - 4.2.0`, `Windows` version.
-    5. In all `*.cpp` files, their include of their analog `*.hpp` file has been moved to the first line of those `*.cpp` files to slightly speed up compiling time.
-    6. String is used in `include/openpose/wrapper/` to avoid std::string to cause errors for using diferent std DLLs.
-    7. Added `ScaleMode::ZeroToOneFixedAspect` and `ScaleMode::PlusMinusOneFixedAspect`. Compared to `ZeroToOne` and `PlusMinusOne`, the new ones also preserve the aspect ratio of each axis.
+    1. Multi-camera (3D) working on Asynchronous mode.
+        1. Functions `WrapperT::waitAndEmplace()` and `WrapperT::tryEmplace()` improved, allowing multi-camera/3-D (`TDatums` of size > 1).
+        2. Added `createMultiviewTDatum()` to auto-generate a `TDatums` for multi-camera/3-D from a single cv::Mat (that is splitted) and the desired camera parameter matrices.
+        3. Added `examples/tutorial_api_cpp/11_asynchronous_custom_input_multi_camera.cpp` for a test example.
+    2. Created Matrix as container of cv::Mat, and String as container of std::string.
+    3. After replacing cv::Mat by Matrix, headers do not contain any 3rd-party library includes nor functions. This way, OpenPose can be exported without needing 3rd-party includes nor static library files (e.g., lib files in Windows), allowing people to use their own versions of OpenCV, Eigen, etc. without conflicting with OpenPose. Dynamic library files (e.g., `dll` files in Windows, `so` in Ubuntu) are still required.
+    4. Created the `openpose_private` directory with some internal headers that, if exported with OpenPose, would require including 3rd-party headers and static library files.
+    5. Default OpenCV version for Windows upgraded to version 4.2.0, extracted from their oficial website: section `Releases`, subsection `OpenCV - 4.2.0`, `Windows` version.
+    6. In all `*.cpp` files, their include of their analog `*.hpp` file has been moved to the first line of those `*.cpp` files to slightly speed up compiling time.
+    7. String is used in `include/openpose/wrapper/` to avoid std::string to cause errors for using diferent std DLLs.
+    8. Added `ScaleMode::ZeroToOneFixedAspect` and `ScaleMode::PlusMinusOneFixedAspect`. Compared to `ZeroToOne` and `PlusMinusOne`, the new ones also preserve the aspect ratio of each axis.
 2. Functions or parameters renamed:
     1. All headers moved into `openpose_private`, all 3rd-party library calls in headers, and std::string calls in `include/openpose/wrapper/`.
     2. Renamed `dLog()` as `opLogIfDebug()`, `log()` as `opLog()`, `check()` as `checkBool()`, and also renamed all the `checkX()` functions in `include/openpose/utilities/check.hpp`. This avoids compiling crashes when exporting OpenPose to other projects which contain other 3rd-party libraries that define functions with the same popular names with `#define`.
 3. Main bugs fixed:
-    1. Removed many Visual Studio (Windows) warnings.
-    2. Natural sort now works properly with filenames containining numbers longer than the limit of an int.
+    1. Debug version of OpenPose actually targets debug lib/DLL files of 3rd-party libraries.
+    2. Debug version no longer prints on console a huge log message from Caffe with the network when starting OpenPose (fixed by using the right debug libraries).
+    3. Removed many Visual Studio (Windows) warnings.
+    4. Natural sort now works properly with filenames containining numbers longer than the limit of an int.
+    5. Optionally auto-generated bin folder only contains the required DLLs (depending on the CMake configuration), instead of all of them.
 4. Changes/additions that affect the compatibility with the OpenPose Unity Plugin:
 
 

examples/tutorial_add_module/1_custom_post_processing.cpp

@@ -20,7 +20,7 @@
 // 4. If extra classes and files are required, add those extra files inside the OpenPose include and src folders,
 // under a new folder (i.e., `include/newMethod/` and `src/newMethod/`), including `namespace op` on those files.
 
-// This example is a sub-case of `tutorial_api_cpp/15_synchronous_custom_postprocessing.cpp`, where only custom post-processing is
+// This example is a sub-case of `tutorial_api_cpp/16_synchronous_custom_postprocessing.cpp`, where only custom post-processing is
 // considered.
 
 // Command-line user interface

examples/tutorial_api_cpp/11_asynchronous_custom_input_multi_camera.cpp

+// ------------------------- OpenPose C++ API Tutorial - Example 10 - Custom Input -------------------------
+// Asynchronous mode: ideal for fast prototyping when performance is not an issue.
+// In this function, the user can implement its own way to create frames (e.g., reading his own folder of images)
+// and emplaces/pushes the frames to OpenPose.
+
+// Third-party dependencies
+#include <opencv2/opencv.hpp>
+// Command-line user interface
+#define OPENPOSE_FLAGS_DISABLE_PRODUCER
+#include <openpose/flags.hpp>
+// OpenPose dependencies
+#include <openpose/headers.hpp>
+
+// Custom OpenPose flags
+// Producer
+DEFINE_string(video,                "3d_4camera_video.avi",
+    "Use a video file instead of the camera. Use `examples/media/video.avi` for our default example video.");
+DEFINE_string(camera_parameter_path,    "models/cameraParameters/flir/",
+    "String with the folder where the camera parameters are located. If there is only 1 XML file (for single"
+    " video, webcam, or images from the same camera), you must specify the whole XML file path (ending in .xml).");
+
+// This worker will just read and return all the basic image file formats in a directory
+class UserInputClass
+{
+public:
+    UserInputClass(const std::string& videoPath, const std::string& cameraParameterPath) :
+        mClosed{false},
+        mFrameCounter{0ull},
+        mVideoCapture{videoPath}
+    {
+        if (!mVideoCapture.isOpened())
+        {
+            mClosed = true;
+            op::error("No video " + videoPath + " opened.", __LINE__, __FUNCTION__, __FILE__);
+        }
+
+        // Create CameraParameterReader
+        mCameraParameterReader.readParameters(cameraParameterPath);
+    }
+
+    std::shared_ptr<std::vector<std::shared_ptr<op::Datum>>> createDatum()
+    {
+        if (mClosed)
+        {
+            op::opLog("Video already closed, nullptr returned.", op::Priority::High);
+            return nullptr;
+        }
+
+        // Read cv::Mat
+        cv::Mat cvInputData;
+        mVideoCapture >> cvInputData;
+        // If empty frame -> return nullptr
+        if (cvInputData.empty())
+        {
+            // Close program when empty frame
+            op::opLog("Empty frame detected, closing program.", op::Priority::High);
+            mClosed = true;
+            return nullptr;
+        }
+
+        // Create new datum and add 3D information (cv::Mat splitted and camera parameters)
+        auto datumsPtr = std::make_shared<std::vector<std::shared_ptr<op::Datum>>>();
+        op::createMultiviewTDatum<op::Datum>(datumsPtr, mFrameCounter, mCameraParameterReader, (void*)&cvInputData);
+
+        return datumsPtr;
+    }
+
+    bool isFinished() const
+    {
+        return mClosed;
+    }
+
+private:
+    bool mClosed;
+    unsigned long long mFrameCounter;
+    cv::VideoCapture mVideoCapture;
+    op::CameraParameterReader mCameraParameterReader;
+};
+
+void configureWrapper(op::Wrapper& opWrapper)
+{
+    try
+    {
+        // Configuring OpenPose
+
+        // logging_level
+        op::checkBool(
+            0 <= FLAGS_logging_level && FLAGS_logging_level <= 255, "Wrong logging_level value.",
+            __LINE__, __FUNCTION__, __FILE__);
+        op::ConfigureLog::setPriorityThreshold((op::Priority)FLAGS_logging_level);
+        op::Profiler::setDefaultX(FLAGS_profile_speed);
+
+        // Applying user defined configuration - GFlags to program variables
+        // outputSize
+        const auto outputSize = op::flagsToPoint(op::String(FLAGS_output_resolution), "-1x-1");
+        // netInputSize
+        const auto netInputSize = op::flagsToPoint(op::String(FLAGS_net_resolution), "-1x368");
+        // faceNetInputSize
+        const auto faceNetInputSize = op::flagsToPoint(op::String(FLAGS_face_net_resolution), "368x368 (multiples of 16)");
+        // handNetInputSize
+        const auto handNetInputSize = op::flagsToPoint(op::String(FLAGS_hand_net_resolution), "368x368 (multiples of 16)");
+        // poseMode
+        const auto poseMode = op::flagsToPoseMode(FLAGS_body);
+        // poseModel
+        const auto poseModel = op::flagsToPoseModel(op::String(FLAGS_model_pose));
+        // JSON saving
+        if (!FLAGS_write_keypoint.empty())
+            op::opLog(
+                "Flag `write_keypoint` is deprecated and will eventually be removed. Please, use `write_json`"
+                " instead.", op::Priority::Max);
+        // keypointScaleMode
+        const auto keypointScaleMode = op::flagsToScaleMode(FLAGS_keypoint_scale);
+        // heatmaps to add
+        const auto heatMapTypes = op::flagsToHeatMaps(FLAGS_heatmaps_add_parts, FLAGS_heatmaps_add_bkg,
+                                                      FLAGS_heatmaps_add_PAFs);
+        const auto heatMapScaleMode = op::flagsToHeatMapScaleMode(FLAGS_heatmaps_scale);
+        // >1 camera view?
+        const auto multipleView = (FLAGS_3d || FLAGS_3d_views > 1);
+        // Face and hand detectors
+        const auto faceDetector = op::flagsToDetector(FLAGS_face_detector);
+        const auto handDetector = op::flagsToDetector(FLAGS_hand_detector);
+        // Enabling Google Logging
+        const bool enableGoogleLogging = true;
+
+        // Pose configuration (use WrapperStructPose{} for default and recommended configuration)
+        const op::WrapperStructPose wrapperStructPose{
+            poseMode, netInputSize, outputSize, keypointScaleMode, FLAGS_num_gpu, FLAGS_num_gpu_start,
+            FLAGS_scale_number, (float)FLAGS_scale_gap, op::flagsToRenderMode(FLAGS_render_pose, multipleView),
+            poseModel, !FLAGS_disable_blending, (float)FLAGS_alpha_pose, (float)FLAGS_alpha_heatmap,
+            FLAGS_part_to_show, op::String(FLAGS_model_folder), heatMapTypes, heatMapScaleMode, FLAGS_part_candidates,
+            (float)FLAGS_render_threshold, FLAGS_number_people_max, FLAGS_maximize_positives, FLAGS_fps_max,
+            op::String(FLAGS_prototxt_path), op::String(FLAGS_caffemodel_path),
+            (float)FLAGS_upsampling_ratio, enableGoogleLogging};
+        opWrapper.configure(wrapperStructPose);
+        // Face configuration (use op::WrapperStructFace{} to disable it)
+        const op::WrapperStructFace wrapperStructFace{
+            FLAGS_face, faceDetector, faceNetInputSize,
+            op::flagsToRenderMode(FLAGS_face_render, multipleView, FLAGS_render_pose),
+            (float)FLAGS_face_alpha_pose, (float)FLAGS_face_alpha_heatmap, (float)FLAGS_face_render_threshold};
+        opWrapper.configure(wrapperStructFace);
+        // Hand configuration (use op::WrapperStructHand{} to disable it)
+        const op::WrapperStructHand wrapperStructHand{
+            FLAGS_hand, handDetector, handNetInputSize, FLAGS_hand_scale_number, (float)FLAGS_hand_scale_range,
+            op::flagsToRenderMode(FLAGS_hand_render, multipleView, FLAGS_render_pose), (float)FLAGS_hand_alpha_pose,
+            (float)FLAGS_hand_alpha_heatmap, (float)FLAGS_hand_render_threshold};
+        opWrapper.configure(wrapperStructHand);
+        // Extra functionality configuration (use op::WrapperStructExtra{} to disable it)
+        const op::WrapperStructExtra wrapperStructExtra{
+            FLAGS_3d, FLAGS_3d_min_views, FLAGS_identification, FLAGS_tracking, FLAGS_ik_threads};
+        opWrapper.configure(wrapperStructExtra);
+        // Output (comment or use default argument to disable any output)
+        const op::WrapperStructOutput wrapperStructOutput{
+            FLAGS_cli_verbose, op::String(FLAGS_write_keypoint), op::stringToDataFormat(FLAGS_write_keypoint_format),
+            op::String(FLAGS_write_json), op::String(FLAGS_write_coco_json), FLAGS_write_coco_json_variants,
+            FLAGS_write_coco_json_variant, op::String(FLAGS_write_images), op::String(FLAGS_write_images_format),
+            op::String(FLAGS_write_video), FLAGS_write_video_fps, FLAGS_write_video_with_audio,
+            op::String(FLAGS_write_heatmaps), op::String(FLAGS_write_heatmaps_format), op::String(FLAGS_write_video_3d),
+            op::String(FLAGS_write_video_adam), op::String(FLAGS_write_bvh), op::String(FLAGS_udp_host),
+            op::String(FLAGS_udp_port)};
+        opWrapper.configure(wrapperStructOutput);
+        // GUI (comment or use default argument to disable any visual output)
+        const op::WrapperStructGui wrapperStructGui{
+            op::flagsToDisplayMode(FLAGS_display, FLAGS_3d), !FLAGS_no_gui_verbose, FLAGS_fullscreen};
+        opWrapper.configure(wrapperStructGui);
+        // Set to single-thread (for sequential processing and/or debugging and/or reducing latency)
+        if (FLAGS_disable_multi_thread)
+            opWrapper.disableMultiThreading();
+    }
+    catch (const std::exception& e)
+    {
+        op::error(e.what(), __LINE__, __FUNCTION__, __FILE__);
+    }
+}
+
+int tutorialApiCpp()
+{
+    try
+    {
+        op::opLog("Starting OpenPose demo...", op::Priority::High);
+        const auto opTimer = op::getTimerInit();
+
+        // Required flags to enable 3-D
+        FLAGS_3d = true;
+        FLAGS_number_people_max = 1;
+        FLAGS_3d_min_views = 3;
+        FLAGS_output_resolution = "320x256"; // Optional, but otherwise it gets too big to render in real time
+        // FLAGS_3d_views = X; // Not required because it only affects OpenPose producers (rather than custom ones)
+
+        // Configuring OpenPose
+        op::opLog("Configuring OpenPose...", op::Priority::High);
+        op::Wrapper opWrapper{op::ThreadManagerMode::AsynchronousIn};
+        configureWrapper(opWrapper);
+
+        // Start, run, and stop processing - exec() blocks this thread until OpenPose wrapper has finished
+        op::opLog("Starting thread(s)...", op::Priority::High);
+        opWrapper.start();
+
+        // User processing
+        UserInputClass userInputClass(FLAGS_video, FLAGS_camera_parameter_path);
+        bool userWantsToExit = false;
+        while (!userWantsToExit && !userInputClass.isFinished())
+        {
+            if (!opWrapper.isRunning())
+            {
+                op::opLog("OpenPose wrapper is no longer running, exiting video.", op::Priority::High);
+                break;
+            }
+            // Push frame
+            auto datumToProcess = userInputClass.createDatum();
+            if (datumToProcess != nullptr)
+            {
+                auto successfullyEmplaced = opWrapper.waitAndEmplace(datumToProcess);
+                if (!successfullyEmplaced)
+                    op::opLog("Processed datum could not be emplaced.", op::Priority::High);
+            }
+        }
+
+        op::opLog("Stopping thread(s)", op::Priority::High);
+        opWrapper.stop();
+
+        // Measuring total time
+        op::printTime(opTimer, "OpenPose demo successfully finished. Total time: ", " seconds.", op::Priority::High);
+
+        // Return
+        return 0;
+    }
+    catch (const std::exception&)
+    {
+        return -1;
+    }
+}
+
+int main(int argc, char *argv[])
+{
+    // Parsing command line flags
+    gflags::ParseCommandLineFlags(&argc, &argv, true);
+
+    // Running tutorialApiCpp
+    return tutorialApiCpp();
+}

examples/tutorial_api_cpp/CMakeLists.txt

@@ -9,13 +9,14 @@ set(EXAMPLE_FILES
     08_heatmaps_from_image.cpp
     09_keypoints_from_heatmaps.cpp
     10_asynchronous_custom_input.cpp
-    11_asynchronous_custom_output.cpp
-    12_asynchronous_custom_input_output_and_datum.cpp
-    13_synchronous_custom_input.cpp
-    14_synchronous_custom_preprocessing.cpp
-    15_synchronous_custom_postprocessing.cpp
-    16_synchronous_custom_output.cpp
-    17_synchronous_custom_all_and_datum.cpp)
+    11_asynchronous_custom_input_multi_camera.cpp
+    12_asynchronous_custom_output.cpp
+    13_asynchronous_custom_input_output_and_datum.cpp
+    14_synchronous_custom_input.cpp
+    15_synchronous_custom_preprocessing.cpp
+    16_synchronous_custom_postprocessing.cpp
+    17_synchronous_custom_output.cpp
+    18_synchronous_custom_all_and_datum.cpp)
 
 include(${CMAKE_SOURCE_DIR}/cmake/Utils.cmake)
 

include/openpose/core/matrix.hpp

@@ -95,6 +95,15 @@ namespace op
     class OP_API Matrix
     {
     public:
+        /**
+         * @param matrixesResized For 3-D OpenPose, if >1, it will assume the image is composed of
+         * numberImagesStackedHorizontally horizontally stacked images. It must be already resized to avoid
+         * internally allocating/removing elements of std::vector (to avoid errors if using different std DLLs)
+         * @param cvMatPtr should be a cv::Mat element or it will provoke a core dumped. Done to
+         * avoid explicitly exposing 3rdparty libraries on the headers.
+         */
+        static void splitCvMatIntoVectorMatrix(std::vector<Matrix>& matrixesResized, const void* const cvMatPtr);
+
         Matrix();
 
         /**

include/openpose/producer/wDatumProducer.hpp

@@ -81,7 +81,7 @@ namespace op
                 opLogIfDebug("", Priority::Low, __LINE__, __FUNCTION__, __FILE__);
             }
             // Equivalent to WQueueSplitter
-            // Queued elements - Multiple views --> Split views into different share pointers
+            // Queued elements - Multiple views --> Split views into different shared pointers
             if (tDatums != nullptr && tDatums->size() > 1)
             {
                 // Add tDatums to mQueuedElements

include/openpose/wrapper/wrapper.hpp

@@ -491,7 +491,26 @@ namespace op
             if (!mUserWs[int(WorkerType::Input)].empty())
                 error("Emplace cannot be called if an input worker was already selected.",
                       __LINE__, __FUNCTION__, __FILE__);
-            return mThreadManager.tryEmplace(tDatums);
+            // tryEmplace for 1 camera
+            if (tDatums->size() < 2)
+            {
+                return mThreadManager.tryEmplace(tDatums);
+            }
+            // tryEmplace for multiview
+            else
+            {
+                bool successfulEmplace = true;
+                for (auto datumIndex = 0u; datumIndex < tDatums->size(); ++datumIndex)
+                {
+                    auto tDatumsSingle = std::make_shared<TDatums>(TDatums({ tDatums->at(datumIndex) }));
+                    if (!tryEmplace(tDatumsSingle))
+                    {
+                        successfulEmplace = false;
+                        break;
+                    }
+                }
+                return successfulEmplace;
+            }
         }
         catch (const std::exception& e)
         {
@@ -508,7 +527,28 @@ namespace op
             if (!mUserWs[int(WorkerType::Input)].empty())
                 error("Emplace cannot be called if an input worker was already selected.",
                       __LINE__, __FUNCTION__, __FILE__);
-            return mThreadManager.waitAndEmplace(tDatums);
+            // waitAndEmplace for 1 camera
+            if (tDatums->size() < 2)
+            {
+                return mThreadManager.waitAndEmplace(tDatums);
+            }
+            // waitAndEmplace for multiview
+            else
+            {
+                bool successfulEmplace = true;
+                for (auto datumIndex = 0u ; datumIndex < tDatums->size() ; ++datumIndex)
+                {
+                    auto tDatumsSingle = std::make_shared<TDatums>(TDatums({tDatums->at(datumIndex)}));
+                    if (!waitAndEmplace(tDatumsSingle))
+                    {
+                        successfulEmplace = false;
+                        opLog("Waiting to emplace for multi-camera failed.",
+                            Priority::High, __LINE__, __FUNCTION__, __FILE__);
+                        break;
+                    }
+                }
+                return successfulEmplace;
+            }
         }
         catch (const std::exception& e)
         {

include/openpose/wrapper/wrapperAuxiliary.hpp

@@ -62,6 +62,17 @@ namespace op
         const WrapperStructOutput& wrapperStructOutput, const WrapperStructGui& wrapperStructGui,
         const std::array<std::vector<TWorker>, int(WorkerType::Size)>& userWs,
         const std::array<bool, int(WorkerType::Size)>& userWsOnNewThread);
+
+    /**
+     * It fills camera parameters and splits the cvMat depending on how many camera parameter matrices are found.
+     * For example usage, check `examples/tutorial_api_cpp/11_asynchronous_custom_input_multi_camera.cpp`
+     */
+    template<typename TDatum,
+             typename TDatums = std::vector<std::shared_ptr<TDatum>>,
+             typename TDatumsSP = std::shared_ptr<TDatums>>
+    void createMultiviewTDatum(
+        TDatumsSP& tDatumsSP, unsigned long long& frameCounter,
+        const CameraParameterReader& cameraParameterReader, const void* const cvMatPtr);
 }
 
 
@@ -1188,6 +1199,55 @@ namespace op
             error(e.what(), __LINE__, __FUNCTION__, __FILE__);
         }
     }
+
+    template<typename TDatum, typename TDatums, typename TDatumsSP>
+    void createMultiviewTDatum(
+        TDatumsSP& tDatumsSP, unsigned long long& frameCounter,
+        const CameraParameterReader& cameraParameterReader, const void* const cvMatPtr)
+    {
+        try
+        {
+            // Sanity check
+            if (tDatumsSP == nullptr)
+                op::error("tDatumsSP was nullptr, it must be initialized.", __LINE__, __FUNCTION__, __FILE__);
+            // Camera parameters
+            const std::vector<op::Matrix>& cameraMatrices = cameraParameterReader.getCameraMatrices();
+            const std::vector<op::Matrix>& cameraIntrinsics = cameraParameterReader.getCameraIntrinsics();
+            const std::vector<op::Matrix>& cameraExtrinsics = cameraParameterReader.getCameraExtrinsics();
+            const auto matrixesSize = cameraMatrices.size();
+            // More sanity checks
+            if (cameraMatrices.size() < 2)
+                op::error("There is less than 2 camera parameter matrices.",
+                    __LINE__, __FUNCTION__, __FILE__);
+            if (cameraMatrices.size() != cameraIntrinsics.size() || cameraMatrices.size() != cameraExtrinsics.size())
+                op::error("Camera parameters must have the same size.", __LINE__, __FUNCTION__, __FILE__);
+            // Split image to process
+            std::vector<op::Matrix> imagesToProcess(matrixesSize);
+            op::Matrix::splitCvMatIntoVectorMatrix(imagesToProcess, cvMatPtr);
+            // Fill tDatumsSP
+            tDatumsSP->resize(cameraMatrices.size());
+            for (auto datumIndex = 0 ; datumIndex < matrixesSize ; ++datumIndex)
+            {
+                auto& datumPtr = tDatumsSP->at(datumIndex);
+                datumPtr = std::make_shared<op::Datum>();
+                datumPtr->frameNumber = frameCounter;
+                datumPtr->cvInputData = imagesToProcess[datumIndex];
+                if (matrixesSize > 1)
+                {
+                    datumPtr->subId = datumIndex;
+                    datumPtr->subIdMax = matrixesSize-1;
+                    datumPtr->cameraMatrix = cameraMatrices[datumIndex];
+                    datumPtr->cameraExtrinsics = cameraExtrinsics[datumIndex];
+                    datumPtr->cameraIntrinsics = cameraIntrinsics[datumIndex];
+                }
+            }
+            ++frameCounter;
+        }
+        catch (const std::exception& e)
+        {
+            error(e.what(), __LINE__, __FUNCTION__, __FILE__);
+        }
+    }
 }
 
 #endif // OPENPOSE_WRAPPER_WRAPPER_AUXILIARY_HPP

scripts/travis/run_tests.sh

@@ -58,32 +58,37 @@ if [[ $RUN_EXAMPLES == true ]] ; then
   ./build/examples/tutorial_api_cpp/10_asynchronous_custom_input.bin --image_dir examples/media/ --net_resolution -1x32 --write_json output/ --write_images output/ --display 0
   echo " "
 
-  echo "Tutorial API C++: Example 11..."
-  ./build/examples/tutorial_api_cpp/11_asynchronous_custom_output.bin --image_dir examples/media/ --net_resolution -1x32 --write_json output/ --write_images output/ --no_display
-  echo " "
+  # # Note: Example 11 would require 3D video and camera parameters
+  # echo "Tutorial API C++: Example 11..."
+  # ./build/examples/tutorial_api_cpp/11_asynchronous_custom_input_multi_camera.bin
+  # echo " "
 
   echo "Tutorial API C++: Example 12..."
-  ./build/examples/tutorial_api_cpp/12_asynchronous_custom_input_output_and_datum.bin --image_dir examples/media/ --net_resolution -1x32 --write_json output/ --write_images output/ --no_display
+  ./build/examples/tutorial_api_cpp/12_asynchronous_custom_output.bin --image_dir examples/media/ --net_resolution -1x32 --write_json output/ --write_images output/ --no_display
   echo " "
 
   echo "Tutorial API C++: Example 13..."
-  ./build/examples/tutorial_api_cpp/13_synchronous_custom_input.bin --image_dir examples/media/ --net_resolution -1x32 --write_json output/ --write_images output/ --display 0
+  ./build/examples/tutorial_api_cpp/13_asynchronous_custom_input_output_and_datum.bin --image_dir examples/media/ --net_resolution -1x32 --write_json output/ --write_images output/ --no_display
   echo " "
 
   echo "Tutorial API C++: Example 14..."
-  ./build/examples/tutorial_api_cpp/14_synchronous_custom_preprocessing.bin --image_dir examples/media/ --net_resolution -1x32 --write_json output/ --write_images output/ --display 0
+  ./build/examples/tutorial_api_cpp/14_synchronous_custom_input.bin --image_dir examples/media/ --net_resolution -1x32 --write_json output/ --write_images output/ --display 0
   echo " "
 
   echo "Tutorial API C++: Example 15..."
-  ./build/examples/tutorial_api_cpp/15_synchronous_custom_postprocessing.bin --image_dir examples/media/ --net_resolution -1x32 --write_json output/ --write_images output/ --display 0
+  ./build/examples/tutorial_api_cpp/15_synchronous_custom_preprocessing.bin --image_dir examples/media/ --net_resolution -1x32 --write_json output/ --write_images output/ --display 0
   echo " "
 
   echo "Tutorial API C++: Example 16..."
-  ./build/examples/tutorial_api_cpp/16_synchronous_custom_output.bin --image_dir examples/media/ --net_resolution -1x32 --write_json output/ --write_images output/ --no_display
+  ./build/examples/tutorial_api_cpp/16_synchronous_custom_postprocessing.bin --image_dir examples/media/ --net_resolution -1x32 --write_json output/ --write_images output/ --display 0
   echo " "
 
   echo "Tutorial API C++: Example 17..."
-  ./build/examples/tutorial_api_cpp/17_synchronous_custom_all_and_datum.bin --image_dir examples/media/ --net_resolution -1x32 --write_json output/ --write_images output/ --no_display
+  ./build/examples/tutorial_api_cpp/17_synchronous_custom_output.bin --image_dir examples/media/ --net_resolution -1x32 --write_json output/ --write_images output/ --no_display
+  echo " "
+
+  echo "Tutorial API C++: Example 18..."
+  ./build/examples/tutorial_api_cpp/18_synchronous_custom_all_and_datum.bin --image_dir examples/media/ --net_resolution -1x32 --write_json output/ --write_images output/ --no_display
   echo " "
 
   # Python examples

src/openpose/3d/poseTriangulation.cpp

@@ -218,7 +218,7 @@ namespace op
             // Sanity checks
             if (cvCameraMatrices.size() < 2)
                 error("3-D reconstruction (`--3d`) requires at least 2 camera views, only found "
-                    + std::to_string(cvCameraMatrices.size()) + "camera parameter matrices." + sFlirErrorMessage,
+                    + std::to_string(cvCameraMatrices.size()) + " camera parameter matrices." + sFlirErrorMessage,
                     __LINE__, __FUNCTION__, __FILE__);
             for (const auto& cameraMatrix : cvCameraMatrices)
                 if (cameraMatrix.empty())

src/openpose/core/matrix.cpp

@@ -9,6 +9,33 @@ namespace op
         cv::Mat mCvMat;
     };
 
+    void Matrix::splitCvMatIntoVectorMatrix(std::vector<Matrix>& matrixesResized, const void* const cvMatPtr)
+    {
+        try
+        {
+            const auto numberImagesStackedHorizontally = matrixesResized.size();
+            // Sanity check
+            if (numberImagesStackedHorizontally < 1)
+                error("matrixesResized.size() must be greater than 0.", __LINE__, __FUNCTION__, __FILE__);
+            // Split cv::Mat
+            cv::Mat matConcatenated = *((cv::Mat*) cvMatPtr);
+            const auto individualWidth = matConcatenated.cols/numberImagesStackedHorizontally;
+            for (auto i = 0u ; i < numberImagesStackedHorizontally ; i++)
+            {
+                cv::Mat cvMat(
+                    matConcatenated,
+                    cv::Rect{
+                        (int)(i*individualWidth), 0,
+                        (int)individualWidth, (int)matConcatenated.rows });
+                matrixesResized[i] = OP_CV2OPMAT(cvMat);
+            }
+        }
+        catch (const std::exception& e)
+        {
+            error(e.what(), __LINE__, __FUNCTION__, __FILE__);
+        }
+    }
+
     Matrix::Matrix() :
         spImpl{std::make_shared<ImplMatrix>()}
     {