提交 eae440bd 编写于 作者: D Donglai Xiang 提交者: Gines

Face Detection by OpenCV (#270)

* first try on OpenCV face

* find a way to visualize detected face; enlarge by 1.5x

* revision for first-time pull request

* faceDetector

* fix little mistake

* Update headers.hpp

* Update CMakeLists.txt

* OpenCV face detector thread save & 4x faster

* Added doc
上级 f54a312b
...@@ -160,7 +160,7 @@ ifeq ($(USE_LMDB), 1) ...@@ -160,7 +160,7 @@ ifeq ($(USE_LMDB), 1)
LIBRARIES += lmdb LIBRARIES += lmdb
endif endif
ifeq ($(USE_OPENCV), 1) ifeq ($(USE_OPENCV), 1)
LIBRARIES += opencv_core opencv_highgui opencv_imgproc LIBRARIES += opencv_core opencv_highgui opencv_imgproc opencv_objdetect
ifeq ($(OPENCV_VERSION), 3) ifeq ($(OPENCV_VERSION), 3)
LIBRARIES += opencv_imgcodecs opencv_videoio LIBRARIES += opencv_imgcodecs opencv_videoio
......
...@@ -143,6 +143,7 @@ Each flag is divided into flag name, default value, and description. ...@@ -143,6 +143,7 @@ Each flag is divided into flag name, default value, and description.
- DEFINE_int32(keypoint_scale, 0, "Scaling of the (x,y) coordinates of the final pose data array, i.e. the scale of the (x,y) coordinates that will be saved with the `write_keypoint` & `write_keypoint_json` flags. Select `0` to scale it to the original source resolution, `1`to scale it to the net output size (set with `net_resolution`), `2` to scale it to the final output size (set with `resolution`), `3` to scale it in the range [0,1], and 4 for range [-1,1]. Non related with `scale_number` and `scale_gap`."); - DEFINE_int32(keypoint_scale, 0, "Scaling of the (x,y) coordinates of the final pose data array, i.e. the scale of the (x,y) coordinates that will be saved with the `write_keypoint` & `write_keypoint_json` flags. Select `0` to scale it to the original source resolution, `1`to scale it to the net output size (set with `net_resolution`), `2` to scale it to the final output size (set with `resolution`), `3` to scale it in the range [0,1], and 4 for range [-1,1]. Non related with `scale_number` and `scale_gap`.");
4. OpenPose Body Pose 4. OpenPose Body Pose
- DEFINE_bool(body_disable, false, "Disable body keypoint detection. Option only possible for faster (but less accurate) face keypoint detection.");
- DEFINE_string(model_pose, "COCO", "Model to be used. E.g. `COCO` (18 keypoints), `MPI` (15 keypoints, ~10% faster), `MPI_4_layers` (15 keypoints, even faster but less accurate)."); - DEFINE_string(model_pose, "COCO", "Model to be used. E.g. `COCO` (18 keypoints), `MPI` (15 keypoints, ~10% faster), `MPI_4_layers` (15 keypoints, even faster but less accurate).");
- DEFINE_string(net_resolution, "656x368", "Multiples of 16. If it is increased, the accuracy potentially increases. If it is decreased, the speed increases. For maximum speed-accuracy balance, it should keep the closest aspect ratio possible to the images or videos to be processed. Using `-1` in any of the dimensions, OP will choose the optimal resolution depending on the other value introduced by the user. E.g. the default `-1x368` is equivalent to `656x368` in 16:9 videos, e.g. full HD (1980x1080) and HD (1280x720) resolutions."); - DEFINE_string(net_resolution, "656x368", "Multiples of 16. If it is increased, the accuracy potentially increases. If it is decreased, the speed increases. For maximum speed-accuracy balance, it should keep the closest aspect ratio possible to the images or videos to be processed. Using `-1` in any of the dimensions, OP will choose the optimal resolution depending on the other value introduced by the user. E.g. the default `-1x368` is equivalent to `656x368` in 16:9 videos, e.g. full HD (1980x1080) and HD (1280x720) resolutions.");
- DEFINE_int32(scale_number, 1, "Number of scales to average."); - DEFINE_int32(scale_number, 1, "Number of scales to average.");
......
...@@ -120,10 +120,11 @@ OpenPose Library - Release Notes ...@@ -120,10 +120,11 @@ OpenPose Library - Release Notes
1. FrameDisplayer accepts variable size images by rescaling every time a frame with bigger width or height is displayed (gui module). 1. FrameDisplayer accepts variable size images by rescaling every time a frame with bigger width or height is displayed (gui module).
2. OpOutputToCvMat & GuiInfoAdder does not require to know the output size at construction time, deduced from each image. 2. OpOutputToCvMat & GuiInfoAdder does not require to know the output size at construction time, deduced from each image.
3. CvMatToOutput and Renderers allow to keep input resolution as output for images (core module). 3. CvMatToOutput and Renderers allow to keep input resolution as output for images (core module).
3. Face and hand keypoint detectors now can return each keypoint heatmap. 3. New standalone face keypoint detector based on OpenCV face detector: much faster if body keypoint detection is not required but much less accurate.
4. COCO JSON file outputs 0 as score for non-detected keypoints. 4. Face and hand keypoint detectors now can return each keypoint heatmap.
5. Added example for OpenPose for user asynchronous output and cleaned all `tutorial_wrapper/` examples. 5. COCO JSON file outputs 0 as score for non-detected keypoints.
6. Added `-1` option for `net_resolution` in order to auto-select the best possible aspect ratio given the user input. 6. Added example for OpenPose for user asynchronous output and cleaned all `tutorial_wrapper/` examples.
7. Added `-1` option for `net_resolution` in order to auto-select the best possible aspect ratio given the user input.
2. Functions or parameters renamed: 2. Functions or parameters renamed:
1. OpenPose able to change its size and initial size: 1. OpenPose able to change its size and initial size:
1. Flag `resolution` renamed as `output_resolution`. 1. Flag `resolution` renamed as `output_resolution`.
......
...@@ -4,11 +4,17 @@ OpenPose Library - Standalone Face Or Hand Keypoint Detector ...@@ -4,11 +4,17 @@ OpenPose Library - Standalone Face Or Hand Keypoint Detector
In case of hand camera views at which the hands are visible but not the rest of the body, or if you do not need the body keypoint detector and want to considerably speed up the process, you can use the OpenPose face or hand keypoint detectors with your own face or hand detectors, rather than using the body keypoint detector as initial detector for those. In case of hand camera views at which the hands are visible but not the rest of the body, or if you do not need the body keypoint detector and want to considerably speed up the process, you can use the OpenPose face or hand keypoint detectors with your own face or hand detectors, rather than using the body keypoint detector as initial detector for those.
## Standalone Face Keypoint Detector ## Standalone Face Keypoint Detector
Note that this method will be much faster than current system, but also much less accurate.
```
./build/examples/openpose/openpose.bin --face --body_disable
```
## Custom Standalone Face Keypoint Detector
There are 2 ways to add the OpenPose face keypoint detector to your own code without using the body pose keypoint extractor as initial face detector: There are 2 ways to add the OpenPose face keypoint detector to your own code without using the body pose keypoint extractor as initial face detector:
1. Easiest solution: Forget about the `OpenPose demo` and `wrapper/wrapper.hpp`, and instead use the `include/openpose/face/faceExtractor.hpp` class with the output of your face detector. Recommended if you do not wanna use any other OpenPose functionality. 1. Easiest solution: Forget about the `OpenPose demo` and `wrapper/wrapper.hpp`, and instead use the `include/openpose/face/faceExtractor.hpp` class with the output of your face detector. Recommended if you do not wanna use any other OpenPose functionality.
2. Elegant solution: If you wanna use the whole OpenPose framework, simply copy `include/wrapper/wrapper.hpp` as e.g. `examples/userCode/wrapperFace.hpp`, and change our `FaceDetector` class by your custom face detector class. If you wanna omit the Pose keypoint detection, you can simply delete it from that custom wrapper too. 2. Elegant solution: If you wanna use the whole OpenPose framework, simply copy `include/wrapper/wrapper.hpp` as e.g. `examples/userCode/wrapperFace.hpp`, and change our `FaceDetector` or `FaceDetectorOpenCV` class by your custom face detector class inside your `WrapperFace` class. If you wanna omit the Pose keypoint detection for a big speed up if you do not need it, you can simply use the `body_disable` flag.
## Standalone Hand Keypoint Detector ## Custom Standalone Hand Keypoint Detector
The analogous steps apply to the hand keypoint detector, but modifying `include/openpose/hand/handExtractor.hpp`. The analogous steps apply to the hand keypoint detector, but modifying `include/openpose/hand/handExtractor.hpp`.
...@@ -64,6 +64,8 @@ DEFINE_int32(keypoint_scale, 0, "Scaling of the (x,y) co ...@@ -64,6 +64,8 @@ DEFINE_int32(keypoint_scale, 0, "Scaling of the (x,y) co
" `resolution`), `3` to scale it in the range [0,1], and 4 for range [-1,1]. Non related" " `resolution`), `3` to scale it in the range [0,1], and 4 for range [-1,1]. Non related"
" with `scale_number` and `scale_gap`."); " with `scale_number` and `scale_gap`.");
// OpenPose Body Pose // OpenPose Body Pose
DEFINE_bool(body_disable, false, "Disable body keypoint detection. Option only possible for faster (but less accurate) face"
" keypoint detection.");
DEFINE_string(model_pose, "COCO", "Model to be used. E.g. `COCO` (18 keypoints), `MPI` (15 keypoints, ~10% faster), " DEFINE_string(model_pose, "COCO", "Model to be used. E.g. `COCO` (18 keypoints), `MPI` (15 keypoints, ~10% faster), "
"`MPI_4_layers` (15 keypoints, even faster but less accurate)."); "`MPI_4_layers` (15 keypoints, even faster but less accurate).");
DEFINE_string(net_resolution, "656x368", "Multiples of 16. If it is increased, the accuracy potentially increases. If it is" DEFINE_string(net_resolution, "656x368", "Multiples of 16. If it is increased, the accuracy potentially increases. If it is"
...@@ -196,7 +198,7 @@ int openPoseDemo() ...@@ -196,7 +198,7 @@ int openPoseDemo()
op::log("Configuring OpenPose wrapper.", op::Priority::Low, __LINE__, __FUNCTION__, __FILE__); op::log("Configuring OpenPose wrapper.", op::Priority::Low, __LINE__, __FUNCTION__, __FILE__);
op::Wrapper<std::vector<op::Datum>> opWrapper; op::Wrapper<std::vector<op::Datum>> opWrapper;
// Pose configuration (use WrapperStructPose{} for default and recommended configuration) // Pose configuration (use WrapperStructPose{} for default and recommended configuration)
const op::WrapperStructPose wrapperStructPose{netInputSize, outputSize, keypointScale, FLAGS_num_gpu, const op::WrapperStructPose wrapperStructPose{!FLAGS_body_disable, netInputSize, outputSize, keypointScale, FLAGS_num_gpu,
FLAGS_num_gpu_start, FLAGS_scale_number, (float)FLAGS_scale_gap, FLAGS_num_gpu_start, FLAGS_scale_number, (float)FLAGS_scale_gap,
op::flagsToRenderMode(FLAGS_render_pose), poseModel, op::flagsToRenderMode(FLAGS_render_pose), poseModel,
!FLAGS_disable_blending, (float)FLAGS_alpha_pose, !FLAGS_disable_blending, (float)FLAGS_alpha_pose,
......
...@@ -49,7 +49,7 @@ int handFromJsonTest() ...@@ -49,7 +49,7 @@ int handFromJsonTest()
op::log("Configuring OpenPose wrapper.", op::Priority::Low, __LINE__, __FUNCTION__, __FILE__); op::log("Configuring OpenPose wrapper.", op::Priority::Low, __LINE__, __FUNCTION__, __FILE__);
op::WrapperHandFromJsonTest<std::vector<op::Datum>> opWrapper; op::WrapperHandFromJsonTest<std::vector<op::Datum>> opWrapper;
// Pose configuration (use WrapperStructPose{} for default and recommended configuration) // Pose configuration (use WrapperStructPose{} for default and recommended configuration)
op::WrapperStructPose wrapperStructPose{op::flagsToPoint("656x368"), op::flagsToPoint("1280x720"), op::WrapperStructPose wrapperStructPose{true, op::flagsToPoint("656x368"), op::flagsToPoint("1280x720"),
op::ScaleMode::InputResolution, FLAGS_num_gpu, FLAGS_num_gpu_start}; op::ScaleMode::InputResolution, FLAGS_num_gpu, FLAGS_num_gpu_start};
wrapperStructPose.modelFolder = FLAGS_model_folder; wrapperStructPose.modelFolder = FLAGS_model_folder;
// Hand configuration (use op::WrapperStructHand{} to disable it) // Hand configuration (use op::WrapperStructHand{} to disable it)
......
...@@ -64,6 +64,8 @@ DEFINE_int32(keypoint_scale, 0, "Scaling of the (x,y) co ...@@ -64,6 +64,8 @@ DEFINE_int32(keypoint_scale, 0, "Scaling of the (x,y) co
" `resolution`), `3` to scale it in the range [0,1], and 4 for range [-1,1]. Non related" " `resolution`), `3` to scale it in the range [0,1], and 4 for range [-1,1]. Non related"
" with `scale_number` and `scale_gap`."); " with `scale_number` and `scale_gap`.");
// OpenPose Body Pose // OpenPose Body Pose
DEFINE_bool(body_disable, false, "Disable body keypoint detection. Option only possible for faster (but less accurate) face"
" keypoint detection.");
DEFINE_string(model_pose, "COCO", "Model to be used. E.g. `COCO` (18 keypoints), `MPI` (15 keypoints, ~10% faster), " DEFINE_string(model_pose, "COCO", "Model to be used. E.g. `COCO` (18 keypoints), `MPI` (15 keypoints, ~10% faster), "
"`MPI_4_layers` (15 keypoints, even faster but less accurate)."); "`MPI_4_layers` (15 keypoints, even faster but less accurate).");
DEFINE_string(net_resolution, "656x368", "Multiples of 16. If it is increased, the accuracy potentially increases. If it is" DEFINE_string(net_resolution, "656x368", "Multiples of 16. If it is increased, the accuracy potentially increases. If it is"
...@@ -288,7 +290,7 @@ int openPoseTutorialWrapper3() ...@@ -288,7 +290,7 @@ int openPoseTutorialWrapper3()
op::log("Configuring OpenPose wrapper.", op::Priority::Low, __LINE__, __FUNCTION__, __FILE__); op::log("Configuring OpenPose wrapper.", op::Priority::Low, __LINE__, __FUNCTION__, __FILE__);
op::Wrapper<std::vector<UserDatum>> opWrapper{op::ThreadManagerMode::AsynchronousOut}; op::Wrapper<std::vector<UserDatum>> opWrapper{op::ThreadManagerMode::AsynchronousOut};
// Pose configuration (use WrapperStructPose{} for default and recommended configuration) // Pose configuration (use WrapperStructPose{} for default and recommended configuration)
const op::WrapperStructPose wrapperStructPose{netInputSize, outputSize, keypointScale, FLAGS_num_gpu, const op::WrapperStructPose wrapperStructPose{!FLAGS_body_disable, netInputSize, outputSize, keypointScale, FLAGS_num_gpu,
FLAGS_num_gpu_start, FLAGS_scale_number, (float)FLAGS_scale_gap, FLAGS_num_gpu_start, FLAGS_scale_number, (float)FLAGS_scale_gap,
op::flagsToRenderMode(FLAGS_render_pose), poseModel, op::flagsToRenderMode(FLAGS_render_pose), poseModel,
!FLAGS_disable_blending, (float)FLAGS_alpha_pose, !FLAGS_disable_blending, (float)FLAGS_alpha_pose,
......
...@@ -47,6 +47,8 @@ DEFINE_int32(keypoint_scale, 0, "Scaling of the (x,y) co ...@@ -47,6 +47,8 @@ DEFINE_int32(keypoint_scale, 0, "Scaling of the (x,y) co
" `resolution`), `3` to scale it in the range [0,1], and 4 for range [-1,1]. Non related" " `resolution`), `3` to scale it in the range [0,1], and 4 for range [-1,1]. Non related"
" with `scale_number` and `scale_gap`."); " with `scale_number` and `scale_gap`.");
// OpenPose Body Pose // OpenPose Body Pose
DEFINE_bool(body_disable, false, "Disable body keypoint detection. Option only possible for faster (but less accurate) face"
" keypoint detection.");
DEFINE_string(model_pose, "COCO", "Model to be used. E.g. `COCO` (18 keypoints), `MPI` (15 keypoints, ~10% faster), " DEFINE_string(model_pose, "COCO", "Model to be used. E.g. `COCO` (18 keypoints), `MPI` (15 keypoints, ~10% faster), "
"`MPI_4_layers` (15 keypoints, even faster but less accurate)."); "`MPI_4_layers` (15 keypoints, even faster but less accurate).");
DEFINE_string(net_resolution, "656x368", "Multiples of 16. If it is increased, the accuracy potentially increases. If it is" DEFINE_string(net_resolution, "656x368", "Multiples of 16. If it is increased, the accuracy potentially increases. If it is"
...@@ -380,7 +382,7 @@ int openPoseTutorialWrapper2() ...@@ -380,7 +382,7 @@ int openPoseTutorialWrapper2()
opWrapper.setWorkerOutput(wUserOutput, workerOutputOnNewThread); opWrapper.setWorkerOutput(wUserOutput, workerOutputOnNewThread);
// Configure OpenPose // Configure OpenPose
op::log("Configuring OpenPose wrapper.", op::Priority::Low, __LINE__, __FUNCTION__, __FILE__); op::log("Configuring OpenPose wrapper.", op::Priority::Low, __LINE__, __FUNCTION__, __FILE__);
const op::WrapperStructPose wrapperStructPose{netInputSize, outputSize, keypointScale, FLAGS_num_gpu, const op::WrapperStructPose wrapperStructPose{!FLAGS_body_disable, netInputSize, outputSize, keypointScale, FLAGS_num_gpu,
FLAGS_num_gpu_start, FLAGS_scale_number, (float)FLAGS_scale_gap, FLAGS_num_gpu_start, FLAGS_scale_number, (float)FLAGS_scale_gap,
op::flagsToRenderMode(FLAGS_render_pose), poseModel, op::flagsToRenderMode(FLAGS_render_pose), poseModel,
!FLAGS_disable_blending, (float)FLAGS_alpha_pose, !FLAGS_disable_blending, (float)FLAGS_alpha_pose,
......
...@@ -47,6 +47,8 @@ DEFINE_int32(keypoint_scale, 0, "Scaling of the (x,y) co ...@@ -47,6 +47,8 @@ DEFINE_int32(keypoint_scale, 0, "Scaling of the (x,y) co
" `resolution`), `3` to scale it in the range [0,1], and 4 for range [-1,1]. Non related" " `resolution`), `3` to scale it in the range [0,1], and 4 for range [-1,1]. Non related"
" with `scale_number` and `scale_gap`."); " with `scale_number` and `scale_gap`.");
// OpenPose Body Pose // OpenPose Body Pose
DEFINE_bool(body_disable, false, "Disable body keypoint detection. Option only possible for faster (but less accurate) face"
" keypoint detection.");
DEFINE_string(model_pose, "COCO", "Model to be used. E.g. `COCO` (18 keypoints), `MPI` (15 keypoints, ~10% faster), " DEFINE_string(model_pose, "COCO", "Model to be used. E.g. `COCO` (18 keypoints), `MPI` (15 keypoints, ~10% faster), "
"`MPI_4_layers` (15 keypoints, even faster but less accurate)."); "`MPI_4_layers` (15 keypoints, even faster but less accurate).");
DEFINE_string(net_resolution, "656x368", "Multiples of 16. If it is increased, the accuracy potentially increases. If it is" DEFINE_string(net_resolution, "656x368", "Multiples of 16. If it is increased, the accuracy potentially increases. If it is"
...@@ -322,7 +324,7 @@ int openPoseTutorialWrapper1() ...@@ -322,7 +324,7 @@ int openPoseTutorialWrapper1()
// Configure OpenPose // Configure OpenPose
op::Wrapper<std::vector<UserDatum>> opWrapper{op::ThreadManagerMode::Asynchronous}; op::Wrapper<std::vector<UserDatum>> opWrapper{op::ThreadManagerMode::Asynchronous};
// Pose configuration (use WrapperStructPose{} for default and recommended configuration) // Pose configuration (use WrapperStructPose{} for default and recommended configuration)
const op::WrapperStructPose wrapperStructPose{netInputSize, outputSize, keypointScale, FLAGS_num_gpu, const op::WrapperStructPose wrapperStructPose{!FLAGS_body_disable, netInputSize, outputSize, keypointScale, FLAGS_num_gpu,
FLAGS_num_gpu_start, FLAGS_scale_number, (float)FLAGS_scale_gap, FLAGS_num_gpu_start, FLAGS_scale_number, (float)FLAGS_scale_gap,
op::flagsToRenderMode(FLAGS_render_pose), poseModel, op::flagsToRenderMode(FLAGS_render_pose), poseModel,
!FLAGS_disable_blending, (float)FLAGS_alpha_pose, !FLAGS_disable_blending, (float)FLAGS_alpha_pose,
......
...@@ -47,6 +47,8 @@ DEFINE_int32(keypoint_scale, 0, "Scaling of the (x,y) co ...@@ -47,6 +47,8 @@ DEFINE_int32(keypoint_scale, 0, "Scaling of the (x,y) co
" `resolution`), `3` to scale it in the range [0,1], and 4 for range [-1,1]. Non related" " `resolution`), `3` to scale it in the range [0,1], and 4 for range [-1,1]. Non related"
" with `scale_number` and `scale_gap`."); " with `scale_number` and `scale_gap`.");
// OpenPose Body Pose // OpenPose Body Pose
DEFINE_bool(body_disable, false, "Disable body keypoint detection. Option only possible for faster (but less accurate) face"
" keypoint detection.");
DEFINE_string(model_pose, "COCO", "Model to be used. E.g. `COCO` (18 keypoints), `MPI` (15 keypoints, ~10% faster), " DEFINE_string(model_pose, "COCO", "Model to be used. E.g. `COCO` (18 keypoints), `MPI` (15 keypoints, ~10% faster), "
"`MPI_4_layers` (15 keypoints, even faster but less accurate)."); "`MPI_4_layers` (15 keypoints, even faster but less accurate).");
DEFINE_string(net_resolution, "656x368", "Multiples of 16. If it is increased, the accuracy potentially increases. If it is" DEFINE_string(net_resolution, "656x368", "Multiples of 16. If it is increased, the accuracy potentially increases. If it is"
...@@ -191,7 +193,7 @@ int openpose3d() ...@@ -191,7 +193,7 @@ int openpose3d()
const auto workerOutputOnNewThread = true; const auto workerOutputOnNewThread = true;
opWrapper.setWorkerOutput(wRender3D, workerOutputOnNewThread); opWrapper.setWorkerOutput(wRender3D, workerOutputOnNewThread);
// Configure OpenPose // Configure OpenPose
const op::WrapperStructPose wrapperStructPose{netInputSize, outputSize, keypointScale, FLAGS_num_gpu, const op::WrapperStructPose wrapperStructPose{!FLAGS_body_disable, netInputSize, outputSize, keypointScale, FLAGS_num_gpu,
FLAGS_num_gpu_start, FLAGS_scale_number, (float)FLAGS_scale_gap, FLAGS_num_gpu_start, FLAGS_scale_number, (float)FLAGS_scale_gap,
op::flagsToRenderMode(FLAGS_render_pose), poseModel, op::flagsToRenderMode(FLAGS_render_pose), poseModel,
!FLAGS_disable_blending, (float)FLAGS_alpha_pose, !FLAGS_disable_blending, (float)FLAGS_alpha_pose,
......
...@@ -10,21 +10,21 @@ ...@@ -10,21 +10,21 @@
*/ */
cv::Mat pointGreyToCvMat(const Spinnaker::ImagePtr &imagePtr) cv::Mat pointGreyToCvMat(const Spinnaker::ImagePtr &imagePtr)
{ {
try try
{ {
const auto XPadding = imagePtr->GetXPadding(); const auto XPadding = imagePtr->GetXPadding();
const auto YPadding = imagePtr->GetYPadding(); const auto YPadding = imagePtr->GetYPadding();
const auto rowsize = imagePtr->GetWidth(); const auto rowsize = imagePtr->GetWidth();
const auto colsize = imagePtr->GetHeight(); const auto colsize = imagePtr->GetHeight();
// image data contains padding. When allocating cv::Mat container size, you need to account for the X,Y image data padding. // image data contains padding. When allocating cv::Mat container size, you need to account for the X,Y image data padding.
return cv::Mat((int)(colsize + YPadding), (int)(rowsize + XPadding), CV_8UC3, imagePtr->GetData(), imagePtr->GetStride()); return cv::Mat((int)(colsize + YPadding), (int)(rowsize + XPadding), CV_8UC3, imagePtr->GetData(), imagePtr->GetStride());
} }
catch (const std::exception& e) catch (const std::exception& e)
{ {
op::error(e.what(), __LINE__, __FUNCTION__, __FILE__); op::error(e.what(), __LINE__, __FUNCTION__, __FILE__);
return cv::Mat(); return cv::Mat();
} }
} }
// This function configures the camera to use a trigger. First, trigger mode is // This function configures the camera to use a trigger. First, trigger mode is
...@@ -35,24 +35,24 @@ int configureTrigger(Spinnaker::GenApi::INodeMap &iNodeMap) ...@@ -35,24 +35,24 @@ int configureTrigger(Spinnaker::GenApi::INodeMap &iNodeMap)
{ {
try try
{ {
int result = 0; int result = 0;
op::log("*** CONFIGURING TRIGGER ***", op::Priority::High); op::log("*** CONFIGURING TRIGGER ***", op::Priority::High);
op::log("Configuring hardware trigger...", op::Priority::High); op::log("Configuring hardware trigger...", op::Priority::High);
// Ensure trigger mode off // Ensure trigger mode off
// *** NOTES *** // *** NOTES ***
// The trigger must be disabled in order to configure whether the source // The trigger must be disabled in order to configure whether the source
// is software or hardware. // is software or hardware.
Spinnaker::GenApi::CEnumerationPtr ptrTriggerMode = iNodeMap.GetNode("TriggerMode"); Spinnaker::GenApi::CEnumerationPtr ptrTriggerMode = iNodeMap.GetNode("TriggerMode");
if (!Spinnaker::GenApi::IsAvailable(ptrTriggerMode) || !Spinnaker::GenApi::IsReadable(ptrTriggerMode)) if (!Spinnaker::GenApi::IsAvailable(ptrTriggerMode) || !Spinnaker::GenApi::IsReadable(ptrTriggerMode))
op::error("Unable to disable trigger mode (node retrieval). Aborting...", __LINE__, __FUNCTION__, __FILE__); op::error("Unable to disable trigger mode (node retrieval). Aborting...", __LINE__, __FUNCTION__, __FILE__);
Spinnaker::GenApi::CEnumEntryPtr ptrTriggerModeOff = ptrTriggerMode->GetEntryByName("Off"); Spinnaker::GenApi::CEnumEntryPtr ptrTriggerModeOff = ptrTriggerMode->GetEntryByName("Off");
if (!Spinnaker::GenApi::IsAvailable(ptrTriggerModeOff) || !Spinnaker::GenApi::IsReadable(ptrTriggerModeOff)) if (!Spinnaker::GenApi::IsAvailable(ptrTriggerModeOff) || !Spinnaker::GenApi::IsReadable(ptrTriggerModeOff))
op::error("Unable to disable trigger mode (enum entry retrieval). Aborting...", __LINE__, __FUNCTION__, __FILE__); op::error("Unable to disable trigger mode (enum entry retrieval). Aborting...", __LINE__, __FUNCTION__, __FILE__);
ptrTriggerMode->SetIntValue(ptrTriggerModeOff->GetValue()); ptrTriggerMode->SetIntValue(ptrTriggerModeOff->GetValue());
op::log("Trigger mode disabled...", op::Priority::High); op::log("Trigger mode disabled...", op::Priority::High);
// Select trigger source // Select trigger source
// *** NOTES *** // *** NOTES ***
...@@ -60,16 +60,16 @@ int configureTrigger(Spinnaker::GenApi::INodeMap &iNodeMap) ...@@ -60,16 +60,16 @@ int configureTrigger(Spinnaker::GenApi::INodeMap &iNodeMap)
// mode is off. // mode is off.
Spinnaker::GenApi::CEnumerationPtr ptrTriggerSource = iNodeMap.GetNode("TriggerSource"); Spinnaker::GenApi::CEnumerationPtr ptrTriggerSource = iNodeMap.GetNode("TriggerSource");
if (!Spinnaker::GenApi::IsAvailable(ptrTriggerSource) || !Spinnaker::GenApi::IsWritable(ptrTriggerSource)) if (!Spinnaker::GenApi::IsAvailable(ptrTriggerSource) || !Spinnaker::GenApi::IsWritable(ptrTriggerSource))
op::error("Unable to set trigger mode (node retrieval). Aborting...", __LINE__, __FUNCTION__, __FILE__); op::error("Unable to set trigger mode (node retrieval). Aborting...", __LINE__, __FUNCTION__, __FILE__);
// Set trigger mode to hardware ('Line0') // Set trigger mode to hardware ('Line0')
Spinnaker::GenApi::CEnumEntryPtr ptrTriggerSourceHardware = ptrTriggerSource->GetEntryByName("Line0"); Spinnaker::GenApi::CEnumEntryPtr ptrTriggerSourceHardware = ptrTriggerSource->GetEntryByName("Line0");
if (!Spinnaker::GenApi::IsAvailable(ptrTriggerSourceHardware) || !Spinnaker::GenApi::IsReadable(ptrTriggerSourceHardware)) if (!Spinnaker::GenApi::IsAvailable(ptrTriggerSourceHardware) || !Spinnaker::GenApi::IsReadable(ptrTriggerSourceHardware))
op::error("Unable to set trigger mode (enum entry retrieval). Aborting...", __LINE__, __FUNCTION__, __FILE__); op::error("Unable to set trigger mode (enum entry retrieval). Aborting...", __LINE__, __FUNCTION__, __FILE__);
ptrTriggerSource->SetIntValue(ptrTriggerSourceHardware->GetValue()); ptrTriggerSource->SetIntValue(ptrTriggerSourceHardware->GetValue());
op::log("Trigger source set to hardware...", op::Priority::High); op::log("Trigger source set to hardware...", op::Priority::High);
// Turn trigger mode on // Turn trigger mode on
// *** LATER *** // *** LATER ***
...@@ -78,26 +78,26 @@ int configureTrigger(Spinnaker::GenApi::INodeMap &iNodeMap) ...@@ -78,26 +78,26 @@ int configureTrigger(Spinnaker::GenApi::INodeMap &iNodeMap)
Spinnaker::GenApi::CEnumEntryPtr ptrTriggerModeOn = ptrTriggerMode->GetEntryByName("On"); Spinnaker::GenApi::CEnumEntryPtr ptrTriggerModeOn = ptrTriggerMode->GetEntryByName("On");
if (!Spinnaker::GenApi::IsAvailable(ptrTriggerModeOn) || !Spinnaker::GenApi::IsReadable(ptrTriggerModeOn)) if (!Spinnaker::GenApi::IsAvailable(ptrTriggerModeOn) || !Spinnaker::GenApi::IsReadable(ptrTriggerModeOn))
{ {
op::error("Unable to enable trigger mode (enum entry retrieval). Aborting...", __LINE__, __FUNCTION__, __FILE__); op::error("Unable to enable trigger mode (enum entry retrieval). Aborting...", __LINE__, __FUNCTION__, __FILE__);
return -1; return -1;
} }
ptrTriggerMode->SetIntValue(ptrTriggerModeOn->GetValue()); ptrTriggerMode->SetIntValue(ptrTriggerModeOn->GetValue());
op::log("Trigger mode turned back on...", op::Priority::High); op::log("Trigger mode turned back on...", op::Priority::High);
return result; return result;
}
catch (const Spinnaker::Exception& e)
{
op::error(e.what(), __LINE__, __FUNCTION__, __FILE__);
return -1;
}
catch (const std::exception& e)
{
op::error(e.what(), __LINE__, __FUNCTION__, __FILE__);
return -1;
} }
catch (const Spinnaker::Exception& e)
{
op::error(e.what(), __LINE__, __FUNCTION__, __FILE__);
return -1;
}
catch (const std::exception& e)
{
op::error(e.what(), __LINE__, __FUNCTION__, __FILE__);
return -1;
}
} }
// This function returns the camera to a normal state by turning off trigger // This function returns the camera to a normal state by turning off trigger
...@@ -106,7 +106,7 @@ int resetTrigger(Spinnaker::GenApi::INodeMap &iNodeMap) ...@@ -106,7 +106,7 @@ int resetTrigger(Spinnaker::GenApi::INodeMap &iNodeMap)
{ {
try try
{ {
int result = 0; int result = 0;
// //
// Turn trigger mode back off // Turn trigger mode back off
// //
...@@ -116,28 +116,28 @@ int resetTrigger(Spinnaker::GenApi::INodeMap &iNodeMap) ...@@ -116,28 +116,28 @@ int resetTrigger(Spinnaker::GenApi::INodeMap &iNodeMap)
// //
Spinnaker::GenApi::CEnumerationPtr ptrTriggerMode = iNodeMap.GetNode("TriggerMode"); Spinnaker::GenApi::CEnumerationPtr ptrTriggerMode = iNodeMap.GetNode("TriggerMode");
if (!Spinnaker::GenApi::IsAvailable(ptrTriggerMode) || !Spinnaker::GenApi::IsReadable(ptrTriggerMode)) if (!Spinnaker::GenApi::IsAvailable(ptrTriggerMode) || !Spinnaker::GenApi::IsReadable(ptrTriggerMode))
op::error("Unable to disable trigger mode (node retrieval). Non-fatal error...", __LINE__, __FUNCTION__, __FILE__); op::error("Unable to disable trigger mode (node retrieval). Non-fatal error...", __LINE__, __FUNCTION__, __FILE__);
Spinnaker::GenApi::CEnumEntryPtr ptrTriggerModeOff = ptrTriggerMode->GetEntryByName("Off"); Spinnaker::GenApi::CEnumEntryPtr ptrTriggerModeOff = ptrTriggerMode->GetEntryByName("Off");
if (!Spinnaker::GenApi::IsAvailable(ptrTriggerModeOff) || !Spinnaker::GenApi::IsReadable(ptrTriggerModeOff)) if (!Spinnaker::GenApi::IsAvailable(ptrTriggerModeOff) || !Spinnaker::GenApi::IsReadable(ptrTriggerModeOff))
op::error("Unable to disable trigger mode (enum entry retrieval). Non-fatal error...", __LINE__, __FUNCTION__, __FILE__); op::error("Unable to disable trigger mode (enum entry retrieval). Non-fatal error...", __LINE__, __FUNCTION__, __FILE__);
ptrTriggerMode->SetIntValue(ptrTriggerModeOff->GetValue()); ptrTriggerMode->SetIntValue(ptrTriggerModeOff->GetValue());
// op::log("Trigger mode disabled...", op::Priority::High); // op::log("Trigger mode disabled...", op::Priority::High);
return result; return result;
} }
catch (Spinnaker::Exception &e) catch (Spinnaker::Exception &e)
{ {
op::error(e.what(), __LINE__, __FUNCTION__, __FILE__); op::error(e.what(), __LINE__, __FUNCTION__, __FILE__);
return -1;
}
catch (const std::exception& e)
{
op::error(e.what(), __LINE__, __FUNCTION__, __FILE__);
return -1; return -1;
} }
catch (const std::exception& e)
{
op::error(e.what(), __LINE__, __FUNCTION__, __FILE__);
return -1;
}
} }
// This function acquires and displays images from each device. // This function acquires and displays images from each device.
...@@ -149,7 +149,7 @@ std::vector<cv::Mat> acquireImages(Spinnaker::CameraList &cameraList) ...@@ -149,7 +149,7 @@ std::vector<cv::Mat> acquireImages(Spinnaker::CameraList &cameraList)
if (cameraList.GetSize() != INTRINSICS.size()) if (cameraList.GetSize() != INTRINSICS.size())
op::checkE(cameraList.GetSize(), INTRINSICS.size(), "The number of cameras must be the same as the INTRINSICS vector size.", __LINE__, __FUNCTION__, __FILE__); op::checkE(cameraList.GetSize(), INTRINSICS.size(), "The number of cameras must be the same as the INTRINSICS vector size.", __LINE__, __FUNCTION__, __FILE__);
std::vector<cv::Mat> cvMats; std::vector<cv::Mat> cvMats;
// Retrieve, convert, and return an image for each camera // Retrieve, convert, and return an image for each camera
// In order to work with simultaneous camera streams, nested loops are // In order to work with simultaneous camera streams, nested loops are
...@@ -176,7 +176,7 @@ std::vector<cv::Mat> acquireImages(Spinnaker::CameraList &cameraList) ...@@ -176,7 +176,7 @@ std::vector<cv::Mat> acquireImages(Spinnaker::CameraList &cameraList)
for (auto i = 0u; i < cameraPtrs.size(); i++) for (auto i = 0u; i < cameraPtrs.size(); i++)
imagePtrs.at(i) = cameraPtrs.at(i)->GetNextImage(); imagePtrs.at(i) = cameraPtrs.at(i)->GetNextImage();
durationMs = std::chrono::duration_cast<std::chrono::nanoseconds>(std::chrono::high_resolution_clock::now()-begin).count() * 1e-6; durationMs = std::chrono::duration_cast<std::chrono::nanoseconds>(std::chrono::high_resolution_clock::now()-begin).count() * 1e-6;
// op::log("Time extraction (ms): " + std::to_string(durationMs), op::Priority::High, __LINE__, __FUNCTION__, __FILE__); // op::log("Time extraction (ms): " + std::to_string(durationMs), op::Priority::High);
} }
// Original format -> RGB8 // Original format -> RGB8
...@@ -186,7 +186,7 @@ std::vector<cv::Mat> acquireImages(Spinnaker::CameraList &cameraList) ...@@ -186,7 +186,7 @@ std::vector<cv::Mat> acquireImages(Spinnaker::CameraList &cameraList)
if (imagePtr->IsIncomplete()) if (imagePtr->IsIncomplete())
{ {
op::log("Image incomplete with image status " + std::to_string(imagePtr->GetImageStatus()) + "...", op::log("Image incomplete with image status " + std::to_string(imagePtr->GetImageStatus()) + "...",
op::Priority::High, __LINE__, __FUNCTION__, __FILE__); op::Priority::High, __LINE__, __FUNCTION__, __FILE__);
imagesExtracted = false; imagesExtracted = false;
break; break;
} }
...@@ -222,7 +222,7 @@ std::vector<cv::Mat> acquireImages(Spinnaker::CameraList &cameraList) ...@@ -222,7 +222,7 @@ std::vector<cv::Mat> acquireImages(Spinnaker::CameraList &cameraList)
// imagePtr = imagePtr; // imagePtr = imagePtr;
// } // }
// durationMs = std::chrono::duration_cast<std::chrono::nanoseconds>(std::chrono::high_resolution_clock::now()-begin).count() * 1e-6; // durationMs = std::chrono::duration_cast<std::chrono::nanoseconds>(std::chrono::high_resolution_clock::now()-begin).count() * 1e-6;
// op::log("Time conversion (ms): " + std::to_string(durationMs / reps), op::Priority::High, __LINE__, __FUNCTION__, __FILE__); // op::log("Time conversion (ms): " + std::to_string(durationMs / reps), op::Priority::High);
} }
} }
...@@ -241,18 +241,18 @@ std::vector<cv::Mat> acquireImages(Spinnaker::CameraList &cameraList) ...@@ -241,18 +241,18 @@ std::vector<cv::Mat> acquireImages(Spinnaker::CameraList &cameraList)
} }
} }
return cvMats; return cvMats;
} }
catch (Spinnaker::Exception &e) catch (Spinnaker::Exception &e)
{ {
op::error(e.what(), __LINE__, __FUNCTION__, __FILE__); op::error(e.what(), __LINE__, __FUNCTION__, __FILE__);
return {}; return {};
} }
catch (const std::exception& e) catch (const std::exception& e)
{ {
op::error(e.what(), __LINE__, __FUNCTION__, __FILE__); op::error(e.what(), __LINE__, __FUNCTION__, __FILE__);
return {}; return {};
} }
} }
// This function prints the device information of the camera from the transport // This function prints the device information of the camera from the transport
...@@ -262,7 +262,7 @@ int printDeviceInfo(Spinnaker::GenApi::INodeMap &iNodeMap, const unsigned int ca ...@@ -262,7 +262,7 @@ int printDeviceInfo(Spinnaker::GenApi::INodeMap &iNodeMap, const unsigned int ca
{ {
int result = 0; int result = 0;
op::log("Printing device information for camera " + std::to_string(camNum) + "...\n", op::Priority::High); op::log("Printing device information for camera " + std::to_string(camNum) + "...\n", op::Priority::High);
Spinnaker::GenApi::FeatureList_t features; Spinnaker::GenApi::FeatureList_t features;
Spinnaker::GenApi::CCategoryPtr cCategoryPtr = iNodeMap.GetNode("DeviceInformation"); Spinnaker::GenApi::CCategoryPtr cCategoryPtr = iNodeMap.GetNode("DeviceInformation");
...@@ -275,12 +275,12 @@ int printDeviceInfo(Spinnaker::GenApi::INodeMap &iNodeMap, const unsigned int ca ...@@ -275,12 +275,12 @@ int printDeviceInfo(Spinnaker::GenApi::INodeMap &iNodeMap, const unsigned int ca
{ {
Spinnaker::GenApi::CNodePtr pfeatureNode = *it; Spinnaker::GenApi::CNodePtr pfeatureNode = *it;
const auto cValuePtr = (Spinnaker::GenApi::CValuePtr)pfeatureNode; const auto cValuePtr = (Spinnaker::GenApi::CValuePtr)pfeatureNode;
op::log(pfeatureNode->GetName() + " : " + (IsReadable(cValuePtr) ? cValuePtr->ToString() : "Node not readable"), op::Priority::High); op::log(pfeatureNode->GetName() + " : " + (IsReadable(cValuePtr) ? cValuePtr->ToString() : "Node not readable"), op::Priority::High);
} }
} }
else else
op::log("Device control information not available.", op::Priority::High); op::log("Device control information not available.", op::Priority::High);
op::log(" ", op::Priority::High); op::log(" ", op::Priority::High);
return result; return result;
} }
...@@ -335,7 +335,7 @@ WPointGrey::~WPointGrey() ...@@ -335,7 +335,7 @@ WPointGrey::~WPointGrey()
// Reset trigger // Reset trigger
auto result = resetTrigger(iNodeMap); auto result = resetTrigger(iNodeMap);
if (result < 0) if (result < 0)
op::error("Error happened..." + std::to_string(result), __LINE__, __FUNCTION__, __FILE__); op::error("Error happened..." + std::to_string(result), __LINE__, __FUNCTION__, __FILE__);
// Deinitialize each camera // Deinitialize each camera
// Each camera must be deinitialized separately by first // Each camera must be deinitialized separately by first
...@@ -343,7 +343,7 @@ WPointGrey::~WPointGrey() ...@@ -343,7 +343,7 @@ WPointGrey::~WPointGrey()
cameraPtr->DeInit(); cameraPtr->DeInit();
} }
op::log("Completed. Releasing...", op::Priority::High); op::log("Completed. Releasing...", op::Priority::High);
// Clear camera list before releasing mSystemPtr // Clear camera list before releasing mSystemPtr
mCameraList.Clear(); mCameraList.Clear();
...@@ -352,7 +352,7 @@ WPointGrey::~WPointGrey() ...@@ -352,7 +352,7 @@ WPointGrey::~WPointGrey()
mSystemPtr->ReleaseInstance(); mSystemPtr->ReleaseInstance();
} }
op::log("Done! Exitting...", op::Priority::High); op::log("Done! Exitting...", op::Priority::High);
} }
catch (const Spinnaker::Exception& e) catch (const Spinnaker::Exception& e)
{ {
...@@ -371,7 +371,7 @@ void WPointGrey::initializationOnThread() ...@@ -371,7 +371,7 @@ void WPointGrey::initializationOnThread()
initialized = true; initialized = true;
// Print application build information // Print application build information
op::log(std::string{ "Application build date: " } + __DATE__ + " " + __TIME__, op::Priority::High, __LINE__, __FUNCTION__, __FILE__); op::log(std::string{ "Application build date: " } + __DATE__ + " " + __TIME__, op::Priority::High);
// Retrieve singleton reference to mSystemPtr object // Retrieve singleton reference to mSystemPtr object
mSystemPtr = Spinnaker::System::GetInstance(); mSystemPtr = Spinnaker::System::GetInstance();
...@@ -381,7 +381,7 @@ void WPointGrey::initializationOnThread() ...@@ -381,7 +381,7 @@ void WPointGrey::initializationOnThread()
unsigned int numCameras = mCameraList.GetSize(); unsigned int numCameras = mCameraList.GetSize();
op::log("Number of cameras detected: " + std::to_string(numCameras), op::Priority::High, __LINE__, __FUNCTION__, __FILE__); op::log("Number of cameras detected: " + std::to_string(numCameras), op::Priority::High);
// Finish if there are no cameras // Finish if there are no cameras
if (numCameras == 0) if (numCameras == 0)
...@@ -392,12 +392,12 @@ void WPointGrey::initializationOnThread() ...@@ -392,12 +392,12 @@ void WPointGrey::initializationOnThread()
// Release mSystemPtr // Release mSystemPtr
mSystemPtr->ReleaseInstance(); mSystemPtr->ReleaseInstance();
op::log("Not enough cameras!\nPress Enter to exit...", op::Priority::High); op::log("Not enough cameras!\nPress Enter to exit...", op::Priority::High);
getchar(); getchar();
op::error("No cameras detected.", __LINE__, __FUNCTION__, __FILE__); op::error("No cameras detected.", __LINE__, __FUNCTION__, __FILE__);
} }
op::log("Camera system initialized...", op::Priority::High); op::log("Camera system initialized...", op::Priority::High);
// //
// Retrieve transport layer nodemaps and print device information for // Retrieve transport layer nodemaps and print device information for
...@@ -409,7 +409,7 @@ void WPointGrey::initializationOnThread() ...@@ -409,7 +409,7 @@ void WPointGrey::initializationOnThread()
// serial number. Rather than caching the nodemap, each nodemap is // serial number. Rather than caching the nodemap, each nodemap is
// retrieved both times as needed. // retrieved both times as needed.
// //
op::log("\n*** DEVICE INFORMATION ***\n", op::Priority::High); op::log("\n*** DEVICE INFORMATION ***\n", op::Priority::High);
for (int i = 0; i < mCameraList.GetSize(); i++) for (int i = 0; i < mCameraList.GetSize(); i++)
{ {
...@@ -445,7 +445,7 @@ void WPointGrey::initializationOnThread() ...@@ -445,7 +445,7 @@ void WPointGrey::initializationOnThread()
// // Configure trigger // // Configure trigger
// result = configureTrigger(iNodeMap); // result = configureTrigger(iNodeMap);
// if (result < 0) // if (result < 0)
// op::error("Result > 0, error " + std::to_string(result) + " occurred...", __LINE__, __FUNCTION__, __FILE__); // op::error("Result > 0, error " + std::to_string(result) + " occurred...", __LINE__, __FUNCTION__, __FILE__);
// // Configure chunk data // // Configure chunk data
// result = configureChunkData(iNodeMap); // result = configureChunkData(iNodeMap);
...@@ -456,11 +456,11 @@ void WPointGrey::initializationOnThread() ...@@ -456,11 +456,11 @@ void WPointGrey::initializationOnThread()
Spinnaker::GenApi::INodeMap& snodeMap = cameraPtr->GetTLStreamNodeMap(); Spinnaker::GenApi::INodeMap& snodeMap = cameraPtr->GetTLStreamNodeMap();
Spinnaker::GenApi::CEnumerationPtr ptrBufferHandlingMode = snodeMap.GetNode("StreamBufferHandlingMode"); Spinnaker::GenApi::CEnumerationPtr ptrBufferHandlingMode = snodeMap.GetNode("StreamBufferHandlingMode");
if (!Spinnaker::GenApi::IsAvailable(ptrBufferHandlingMode) || !Spinnaker::GenApi::IsWritable(ptrBufferHandlingMode)) if (!Spinnaker::GenApi::IsAvailable(ptrBufferHandlingMode) || !Spinnaker::GenApi::IsWritable(ptrBufferHandlingMode))
op::error("Unable to change buffer handling mode", __LINE__, __FUNCTION__, __FILE__); op::error("Unable to change buffer handling mode", __LINE__, __FUNCTION__, __FILE__);
Spinnaker::GenApi::CEnumEntryPtr ptrBufferHandlingModeNewest = ptrBufferHandlingMode->GetEntryByName("NewestFirstOverwrite"); Spinnaker::GenApi::CEnumEntryPtr ptrBufferHandlingModeNewest = ptrBufferHandlingMode->GetEntryByName("NewestFirstOverwrite");
if (!Spinnaker::GenApi::IsAvailable(ptrBufferHandlingModeNewest) || !IsReadable(ptrBufferHandlingModeNewest)) if (!Spinnaker::GenApi::IsAvailable(ptrBufferHandlingModeNewest) || !IsReadable(ptrBufferHandlingModeNewest))
op::error("Unable to set buffer handling mode to newest (entry 'NewestFirstOverwrite' retrieval). Aborting...", __LINE__, __FUNCTION__, __FILE__); op::error("Unable to set buffer handling mode to newest (entry 'NewestFirstOverwrite' retrieval). Aborting...", __LINE__, __FUNCTION__, __FILE__);
int64_t bufferHandlingModeNewest = ptrBufferHandlingModeNewest->GetValue(); int64_t bufferHandlingModeNewest = ptrBufferHandlingModeNewest->GetValue();
ptrBufferHandlingMode->SetIntValue(bufferHandlingModeNewest); ptrBufferHandlingMode->SetIntValue(bufferHandlingModeNewest);
...@@ -486,22 +486,22 @@ void WPointGrey::initializationOnThread() ...@@ -486,22 +486,22 @@ void WPointGrey::initializationOnThread()
// Set acquisition mode to continuous // Set acquisition mode to continuous
Spinnaker::GenApi::CEnumerationPtr ptrAcquisitionMode = cameraPtr->GetNodeMap().GetNode("AcquisitionMode"); Spinnaker::GenApi::CEnumerationPtr ptrAcquisitionMode = cameraPtr->GetNodeMap().GetNode("AcquisitionMode");
if (!Spinnaker::GenApi::IsAvailable(ptrAcquisitionMode) || !Spinnaker::GenApi::IsWritable(ptrAcquisitionMode)) if (!Spinnaker::GenApi::IsAvailable(ptrAcquisitionMode) || !Spinnaker::GenApi::IsWritable(ptrAcquisitionMode))
op::error("Unable to set acquisition mode to continuous (node retrieval; camera " + std::to_string(i) + "). Aborting...", __LINE__, __FUNCTION__, __FILE__); op::error("Unable to set acquisition mode to continuous (node retrieval; camera " + std::to_string(i) + "). Aborting...", __LINE__, __FUNCTION__, __FILE__);
Spinnaker::GenApi::CEnumEntryPtr ptrAcquisitionModeContinuous = ptrAcquisitionMode->GetEntryByName("Continuous"); Spinnaker::GenApi::CEnumEntryPtr ptrAcquisitionModeContinuous = ptrAcquisitionMode->GetEntryByName("Continuous");
if (!Spinnaker::GenApi::IsAvailable(ptrAcquisitionModeContinuous) || !Spinnaker::GenApi::IsReadable(ptrAcquisitionModeContinuous)) if (!Spinnaker::GenApi::IsAvailable(ptrAcquisitionModeContinuous) || !Spinnaker::GenApi::IsReadable(ptrAcquisitionModeContinuous))
op::error("Unable to set acquisition mode to continuous (entry 'continuous' retrieval " + std::to_string(i) + "). Aborting...", __LINE__, __FUNCTION__, __FILE__); op::error("Unable to set acquisition mode to continuous (entry 'continuous' retrieval " + std::to_string(i) + "). Aborting...", __LINE__, __FUNCTION__, __FILE__);
int64_t acquisitionModeContinuous = ptrAcquisitionModeContinuous->GetValue(); int64_t acquisitionModeContinuous = ptrAcquisitionModeContinuous->GetValue();
ptrAcquisitionMode->SetIntValue(acquisitionModeContinuous); ptrAcquisitionMode->SetIntValue(acquisitionModeContinuous);
op::log("Camera " + std::to_string(i) + " acquisition mode set to continuous...", op::Priority::High); op::log("Camera " + std::to_string(i) + " acquisition mode set to continuous...", op::Priority::High);
// Begin acquiring images // Begin acquiring images
cameraPtr->BeginAcquisition(); cameraPtr->BeginAcquisition();
op::log("Camera " + std::to_string(i) + " started acquiring images...", op::Priority::High); op::log("Camera " + std::to_string(i) + " started acquiring images...", op::Priority::High);
// Retrieve device serial number for filename // Retrieve device serial number for filename
strSerialNumbers[i] = ""; strSerialNumbers[i] = "";
...@@ -513,10 +513,10 @@ void WPointGrey::initializationOnThread() ...@@ -513,10 +513,10 @@ void WPointGrey::initializationOnThread()
strSerialNumbers[i] = ptrStringSerial->GetValue(); strSerialNumbers[i] = ptrStringSerial->GetValue();
op::log("Camera " + std::to_string(i) + " serial number set to " + strSerialNumbers[i].c_str() + "...", op::Priority::High); op::log("Camera " + std::to_string(i) + " serial number set to " + strSerialNumbers[i].c_str() + "...", op::Priority::High);
} }
op::log(" ", op::Priority::High); op::log(" ", op::Priority::High);
} }
op::log("\nRunning for all cameras...\n\n*** IMAGE ACQUISITION ***\n", op::Priority::High); op::log("\nRunning for all cameras...\n\n*** IMAGE ACQUISITION ***\n", op::Priority::High);
} }
catch (const Spinnaker::Exception& e) catch (const Spinnaker::Exception& e)
{ {
......
#ifndef OPENPOSE_FACE_FACE_DETECTOR_OPENCV_HPP
#define OPENPOSE_FACE_FACE_DETECTOR_OPENCV_HPP
#include <opencv2/core/core.hpp>
#include <opencv2/objdetect/objdetect.hpp>
#include <openpose/core/common.hpp>
namespace op
{
class OP_API FaceDetectorOpenCV
{
public:
explicit FaceDetectorOpenCV(const std::string& modelFolder);
// No thread-save
std::vector<Rectangle<float>> detectFaces(const cv::Mat& cvInputData);
private:
cv::CascadeClassifier mFaceCascade;
DELETE_COPY(FaceDetectorOpenCV);
};
}
#endif // OPENPOSE_FACE_FACE_DETECTOR_OPENCV_HPP
...@@ -3,6 +3,7 @@ ...@@ -3,6 +3,7 @@
// face module // face module
#include <openpose/face/faceDetector.hpp> #include <openpose/face/faceDetector.hpp>
#include <openpose/face/faceDetectorOpenCV.hpp>
#include <openpose/face/faceExtractor.hpp> #include <openpose/face/faceExtractor.hpp>
#include <openpose/face/faceParameters.hpp> #include <openpose/face/faceParameters.hpp>
#include <openpose/face/faceCpuRenderer.hpp> #include <openpose/face/faceCpuRenderer.hpp>
...@@ -10,6 +11,7 @@ ...@@ -10,6 +11,7 @@
#include <openpose/face/faceRenderer.hpp> #include <openpose/face/faceRenderer.hpp>
#include <openpose/face/renderFace.hpp> #include <openpose/face/renderFace.hpp>
#include <openpose/face/wFaceDetector.hpp> #include <openpose/face/wFaceDetector.hpp>
#include <openpose/face/wFaceDetectorOpenCV.hpp>
#include <openpose/face/wFaceExtractor.hpp> #include <openpose/face/wFaceExtractor.hpp>
#include <openpose/face/wFaceRenderer.hpp> #include <openpose/face/wFaceRenderer.hpp>
......
#ifndef OPENPOSE_FACE_W_FACE_EXTRACTOR_OPENCV_HPP
#define OPENPOSE_FACE_W_FACE_EXTRACTOR_OPENCV_HPP
#include <openpose/core/common.hpp>
#include <openpose/face/faceRenderer.hpp>
#include <openpose/thread/worker.hpp>
namespace op
{
template<typename TDatums>
class WFaceDetectorOpenCV : public Worker<TDatums>
{
public:
explicit WFaceDetectorOpenCV(const std::shared_ptr<FaceDetectorOpenCV>& faceDetectorOpenCV);
void initializationOnThread();
void work(TDatums& tDatums);
private:
std::shared_ptr<FaceDetectorOpenCV> spFaceDetectorOpenCV;
DELETE_COPY(WFaceDetectorOpenCV);
};
}
// Implementation
#include <openpose/utilities/pointerContainer.hpp>
namespace op
{
template<typename TDatums>
WFaceDetectorOpenCV<TDatums>::WFaceDetectorOpenCV(const std::shared_ptr<FaceDetectorOpenCV>& faceDetectorOpenCV) :
spFaceDetectorOpenCV{faceDetectorOpenCV}
{
}
template<typename TDatums>
void WFaceDetectorOpenCV<TDatums>::initializationOnThread()
{
}
template<typename TDatums>
void WFaceDetectorOpenCV<TDatums>::work(TDatums& tDatums)
{
try
{
if (checkNoNullNorEmpty(tDatums))
{
// Debugging log
dLog("", Priority::Low, __LINE__, __FUNCTION__, __FILE__);
// Profiling speed
const auto profilerKey = Profiler::timerInit(__LINE__, __FUNCTION__, __FILE__);
// Detect people face
for (auto& tDatum : *tDatums)
tDatum.faceRectangles = spFaceDetectorOpenCV->detectFaces(tDatum.cvInputData);
// Profiling speed
Profiler::timerEnd(profilerKey);
Profiler::printAveragedTimeMsOnIterationX(profilerKey, __LINE__, __FUNCTION__, __FILE__, Profiler::DEFAULT_X);
// Debugging log
dLog("", Priority::Low, __LINE__, __FUNCTION__, __FILE__);
}
}
catch (const std::exception& e)
{
this->stop();
tDatums = nullptr;
error(e.what(), __LINE__, __FUNCTION__, __FILE__);
}
}
COMPILE_TEMPLATE_DATUM(WFaceDetectorOpenCV);
}
#endif // OPENPOSE_FACE_W_FACE_EXTRACTOR_OPENCV_HPP
...@@ -104,7 +104,7 @@ namespace op ...@@ -104,7 +104,7 @@ namespace op
const std::string commonMessage{"Input images must be 3-channel BGR."}; const std::string commonMessage{"Input images must be 3-channel BGR."};
if (datum.cvInputData.channels() == 1) if (datum.cvInputData.channels() == 1)
{ {
log(commonMessage + " Converting your grey image into BGR.", Priority::High, __LINE__, __FUNCTION__, __FILE__); log(commonMessage + " Converting grey image into BGR.", Priority::High);
cv::cvtColor(datum.cvInputData, datum.cvInputData, CV_GRAY2BGR); cv::cvtColor(datum.cvInputData, datum.cvInputData, CV_GRAY2BGR);
} }
else else
......
...@@ -524,6 +524,14 @@ namespace op ...@@ -524,6 +524,14 @@ namespace op
if (!wrapperStructOutput.writeVideo.empty() && wrapperStructInput.producerSharedPtr == nullptr) if (!wrapperStructOutput.writeVideo.empty() && wrapperStructInput.producerSharedPtr == nullptr)
error("Writting video is only available if the OpenPose producer is used (i.e." error("Writting video is only available if the OpenPose producer is used (i.e."
" wrapperStructInput.producerSharedPtr cannot be a nullptr).", __LINE__, __FUNCTION__, __FILE__); " wrapperStructInput.producerSharedPtr cannot be a nullptr).", __LINE__, __FUNCTION__, __FILE__);
if (!wrapperStructPose.enable)
{
if (!wrapperStructFace.enable)
error("Body keypoint detection must be enabled.", __LINE__, __FUNCTION__, __FILE__);
if (wrapperStructHand.enable)
error("Body keypoint detection must be enabled in order to run hand keypoint detection.",
__LINE__, __FUNCTION__, __FILE__);
}
// Get number GPUs // Get number GPUs
auto gpuNumber = wrapperStructPose.gpuNumber; auto gpuNumber = wrapperStructPose.gpuNumber;
...@@ -545,6 +553,7 @@ namespace op ...@@ -545,6 +553,7 @@ namespace op
const auto writeKeypointCleaned = formatAsDirectory(wrapperStructOutput.writeKeypoint); const auto writeKeypointCleaned = formatAsDirectory(wrapperStructOutput.writeKeypoint);
const auto writeKeypointJsonCleaned = formatAsDirectory(wrapperStructOutput.writeKeypointJson); const auto writeKeypointJsonCleaned = formatAsDirectory(wrapperStructOutput.writeKeypointJson);
const auto writeHeatMapsCleaned = formatAsDirectory(wrapperStructOutput.writeHeatMaps); const auto writeHeatMapsCleaned = formatAsDirectory(wrapperStructOutput.writeHeatMaps);
const auto modelFolder = formatAsDirectory(wrapperStructPose.modelFolder);
// Common parameters // Common parameters
auto finalOutputSize = wrapperStructPose.outputSize; auto finalOutputSize = wrapperStructPose.outputSize;
...@@ -597,46 +606,50 @@ namespace op ...@@ -597,46 +606,50 @@ namespace op
else else
wDatumProducer = nullptr; wDatumProducer = nullptr;
// Pose estimators // Pose estimators & renderers
const Point<int>& poseNetOutputSize = poseNetInputSize; const Point<int>& poseNetOutputSize = poseNetInputSize;
std::vector<std::shared_ptr<PoseExtractor>> poseExtractors; std::vector<std::shared_ptr<PoseExtractor>> poseExtractors;
for (auto gpuId = 0; gpuId < gpuNumber; gpuId++)
poseExtractors.emplace_back(std::make_shared<PoseExtractorCaffe>(
poseNetInputSize, poseNetOutputSize, finalOutputSize, wrapperStructPose.scalesNumber,
wrapperStructPose.poseModel, wrapperStructPose.modelFolder, gpuId + gpuNumberStart,
wrapperStructPose.heatMapTypes, wrapperStructPose.heatMapScale
));
// Pose renderers
std::vector<std::shared_ptr<PoseGpuRenderer>> poseGpuRenderers; std::vector<std::shared_ptr<PoseGpuRenderer>> poseGpuRenderers;
std::shared_ptr<PoseCpuRenderer> poseCpuRenderer; std::shared_ptr<PoseCpuRenderer> poseCpuRenderer;
std::vector<TWorker> cpuRenderers; std::vector<TWorker> cpuRenderers;
if (renderOutputGpu || wrapperStructPose.renderMode == RenderMode::Cpu) if (wrapperStructPose.enable)
{ {
// If wrapperStructPose.renderMode != RenderMode::Gpu but renderOutput, then we create an alpha = 0 // Pose estimators
// pose renderer in order to keep the removing background option for (auto gpuId = 0; gpuId < gpuNumber; gpuId++)
const auto alphaKeypoint = (wrapperStructPose.renderMode != RenderMode::None poseExtractors.emplace_back(std::make_shared<PoseExtractorCaffe>(
? wrapperStructPose.alphaKeypoint : 0.f); poseNetInputSize, poseNetOutputSize, finalOutputSize, wrapperStructPose.scalesNumber,
const auto alphaHeatMap = (wrapperStructPose.renderMode != RenderMode::None wrapperStructPose.poseModel, modelFolder, gpuId + gpuNumberStart,
? wrapperStructPose.alphaHeatMap : 0.f); wrapperStructPose.heatMapTypes, wrapperStructPose.heatMapScale
// GPU rendering ));
if (renderOutputGpu)
// Pose renderers
if (renderOutputGpu || wrapperStructPose.renderMode == RenderMode::Cpu)
{ {
for (auto gpuId = 0u; gpuId < poseExtractors.size(); gpuId++) // If wrapperStructPose.renderMode != RenderMode::Gpu but renderOutput, then we create an alpha = 0
// pose renderer in order to keep the removing background option
const auto alphaKeypoint = (wrapperStructPose.renderMode != RenderMode::None
? wrapperStructPose.alphaKeypoint : 0.f);
const auto alphaHeatMap = (wrapperStructPose.renderMode != RenderMode::None
? wrapperStructPose.alphaHeatMap : 0.f);
// GPU rendering
if (renderOutputGpu)
{ {
poseGpuRenderers.emplace_back(std::make_shared<PoseGpuRenderer>( for (auto gpuId = 0u; gpuId < poseExtractors.size(); gpuId++)
poseNetOutputSize, wrapperStructPose.poseModel, poseExtractors[gpuId], {
wrapperStructPose.renderThreshold, wrapperStructPose.blendOriginalFrame, alphaKeypoint, poseGpuRenderers.emplace_back(std::make_shared<PoseGpuRenderer>(
alphaHeatMap, wrapperStructPose.defaultPartToRender poseNetOutputSize, wrapperStructPose.poseModel, poseExtractors[gpuId],
)); wrapperStructPose.renderThreshold, wrapperStructPose.blendOriginalFrame, alphaKeypoint,
alphaHeatMap, wrapperStructPose.defaultPartToRender
));
}
}
// CPU rendering
if (wrapperStructPose.renderMode == RenderMode::Cpu)
{
poseCpuRenderer = std::make_shared<PoseCpuRenderer>(wrapperStructPose.poseModel,
wrapperStructPose.blendOriginalFrame);
cpuRenderers.emplace_back(std::make_shared<WPoseRenderer<TDatumsPtr>>(poseCpuRenderer));
} }
}
// CPU rendering
if (wrapperStructPose.renderMode == RenderMode::Cpu)
{
poseCpuRenderer = std::make_shared<PoseCpuRenderer>(wrapperStructPose.poseModel,
wrapperStructPose.blendOriginalFrame);
cpuRenderers.emplace_back(std::make_shared<WPoseRenderer<TDatumsPtr>>(poseCpuRenderer));
} }
} }
log("", Priority::Low, __LINE__, __FUNCTION__, __FILE__); log("", Priority::Low, __LINE__, __FUNCTION__, __FILE__);
...@@ -650,22 +663,46 @@ namespace op ...@@ -650,22 +663,46 @@ namespace op
spWCvMatToOpOutput = std::make_shared<WCvMatToOpOutput<TDatumsPtr>>(cvMatToOpOutput); spWCvMatToOpOutput = std::make_shared<WCvMatToOpOutput<TDatumsPtr>>(cvMatToOpOutput);
// Pose extractor(s) // Pose extractor(s)
spWPoses.resize(poseExtractors.size()); if (wrapperStructPose.enable)
for (auto i = 0u; i < spWPoses.size(); i++) {
spWPoses.at(i) = {std::make_shared<WPoseExtractor<TDatumsPtr>>(poseExtractors.at(i))}; spWPoses.resize(poseExtractors.size());
for (auto i = 0u; i < spWPoses.size(); i++)
spWPoses.at(i) = {std::make_shared<WPoseExtractor<TDatumsPtr>>(poseExtractors.at(i))};
}
else
spWPoses.resize(gpuNumber);
// Face extractor(s) // Face extractor(s)
if (wrapperStructFace.enable) if (wrapperStructFace.enable)
{ {
const auto faceDetector = std::make_shared<FaceDetector>(wrapperStructPose.poseModel); // Face detector
// OpenPose face detector
if (wrapperStructPose.enable)
{
const auto faceDetector = std::make_shared<FaceDetector>(wrapperStructPose.poseModel);
for (auto gpu = 0u; gpu < spWPoses.size(); gpu++)
spWPoses.at(gpu).emplace_back(std::make_shared<WFaceDetector<TDatumsPtr>>(faceDetector));
}
// OpenCV face detector
else
{
log("Body keypoint detection is disabled. Hence, using OpenCV face detector (much less accurate"
" but faster).", Priority::High);
for (auto gpu = 0u; gpu < spWPoses.size(); gpu++)
{
// 1 FaceDetectorOpenCV per thread, OpenCV face detector is not thread-safe
const auto faceDetectorOpenCV = std::make_shared<FaceDetectorOpenCV>(modelFolder);
spWPoses.at(gpu).emplace_back(std::make_shared<WFaceDetectorOpenCV<TDatumsPtr>>(faceDetectorOpenCV));
}
}
// Face keypoint extractor
for (auto gpu = 0u; gpu < spWPoses.size(); gpu++) for (auto gpu = 0u; gpu < spWPoses.size(); gpu++)
{ {
// Face detector
spWPoses.at(gpu).emplace_back(std::make_shared<WFaceDetector<TDatumsPtr>>(faceDetector));
// Face keypoint extractor // Face keypoint extractor
const auto netOutputSize = wrapperStructFace.netInputSize; const auto netOutputSize = wrapperStructFace.netInputSize;
const auto faceExtractor = std::make_shared<FaceExtractor>( const auto faceExtractor = std::make_shared<FaceExtractor>(
wrapperStructFace.netInputSize, netOutputSize, wrapperStructPose.modelFolder, wrapperStructFace.netInputSize, netOutputSize, modelFolder,
gpu + gpuNumberStart, wrapperStructPose.heatMapTypes, wrapperStructPose.heatMapScale gpu + gpuNumberStart, wrapperStructPose.heatMapTypes, wrapperStructPose.heatMapScale
); );
spWPoses.at(gpu).emplace_back(std::make_shared<WFaceExtractor<TDatumsPtr>>(faceExtractor)); spWPoses.at(gpu).emplace_back(std::make_shared<WFaceExtractor<TDatumsPtr>>(faceExtractor));
...@@ -690,7 +727,7 @@ namespace op ...@@ -690,7 +727,7 @@ namespace op
// Hand keypoint extractor // Hand keypoint extractor
const auto netOutputSize = wrapperStructHand.netInputSize; const auto netOutputSize = wrapperStructHand.netInputSize;
const auto handExtractor = std::make_shared<HandExtractor>( const auto handExtractor = std::make_shared<HandExtractor>(
wrapperStructHand.netInputSize, netOutputSize, wrapperStructPose.modelFolder, wrapperStructHand.netInputSize, netOutputSize, modelFolder,
gpu + gpuNumberStart, wrapperStructHand.scalesNumber, wrapperStructHand.scaleRange, gpu + gpuNumberStart, wrapperStructHand.scalesNumber, wrapperStructHand.scaleRange,
wrapperStructPose.heatMapTypes, wrapperStructPose.heatMapScale wrapperStructPose.heatMapTypes, wrapperStructPose.heatMapScale
); );
......
...@@ -10,11 +10,18 @@ namespace op ...@@ -10,11 +10,18 @@ namespace op
{ {
/** /**
* WrapperStructPose: Pose estimation and rendering configuration struct. * WrapperStructPose: Pose estimation and rendering configuration struct.
* WrapperStructPose allows the user to set up the pose estimation and rendering parameters that will be used for the OpenPose Wrapper * WrapperStructPose allows the user to set up the pose estimation and rendering parameters that will be used for
* class. * the OpenPose Wrapper class.
*/ */
struct OP_API WrapperStructPose struct OP_API WrapperStructPose
{ {
/**
* Whether to extract body.
* It might be optionally disabled if only face keypoint detection is required. Otherwise, it must be always
* true.
*/
bool enable;
/** /**
* CCN (Conv Net) input size. * CCN (Conv Net) input size.
* The greater, the slower and more memory it will be needed, but it will potentially increase accuracy. * The greater, the slower and more memory it will be needed, but it will potentially increase accuracy.
...@@ -25,21 +32,23 @@ namespace op ...@@ -25,21 +32,23 @@ namespace op
/** /**
* Output size of the final rendered image. * Output size of the final rendered image.
* It barely affects performance compared to netInputSize. * It barely affects performance compared to netInputSize.
* The final Datum.poseKeypoints can be scaled with respect to outputSize if `keypointScale` is set to ScaleMode::OutputResolution, even if the * The final Datum.poseKeypoints can be scaled with respect to outputSize if `keypointScale` is set to
* rendering is disabled. * ScaleMode::OutputResolution, even if the rendering is disabled.
*/ */
Point<int> outputSize; Point<int> outputSize;
/** /**
* Final scale of the Array<float> Datum.poseKeypoints and the writen pose data. * Final scale of the Array<float> Datum.poseKeypoints and the writen pose data.
* The final Datum.poseKeypoints can be scaled with respect to input size (ScaleMode::InputResolution), net output size (ScaleMode::NetOutputResolution), * The final Datum.poseKeypoints can be scaled with respect to input size (ScaleMode::InputResolution), net
* output rendering size (ScaleMode::OutputResolution), from 0 to 1 (ScaleMode::ZeroToOne), and -1 to 1 (ScaleMode::PlusMinusOne). * output size (ScaleMode::NetOutputResolution), output rendering size (ScaleMode::OutputResolution), from 0 to
* 1 (ScaleMode::ZeroToOne), and -1 to 1 (ScaleMode::PlusMinusOne).
*/ */
ScaleMode keypointScale; ScaleMode keypointScale;
/** /**
* Number of GPUs processing in parallel. * Number of GPUs processing in parallel.
* The greater, the faster the algorithm will run, but potentially higher lag will appear (which only affects in real-time webcam scenarios). * The greater, the faster the algorithm will run, but potentially higher lag will appear (which only affects
* in real-time webcam scenarios).
*/ */
int gpuNumber; int gpuNumber;
...@@ -52,13 +61,15 @@ namespace op ...@@ -52,13 +61,15 @@ namespace op
/** /**
* Number of scales to process. * Number of scales to process.
* The greater, the slower and more memory it will be needed, but it will potentially increase accuracy. * The greater, the slower and more memory it will be needed, but it will potentially increase accuracy.
* This parameter is related with scaleGap, such as the final pose estimation will be an average of the predicted results for each scale. * This parameter is related with scaleGap, such as the final pose estimation will be an average of the
* predicted results for each scale.
*/ */
int scalesNumber; int scalesNumber;
/** /**
* Gap between successive scales. * Gap between successive scales.
* The pose estimation will be estimation for the scales in the range [1, 1-scaleGap*scalesNumber], with a gap of scaleGap. * The pose estimation will be estimation for the scales in the range [1, 1-scaleGap*scalesNumber], with a gap
* of scaleGap.
*/ */
float scaleGap; float scaleGap;
...@@ -70,8 +81,8 @@ namespace op ...@@ -70,8 +81,8 @@ namespace op
/** /**
* Pose model, it affects the number of body parts to render * Pose model, it affects the number of body parts to render
* Select PoseModel::COCO_18 for 18 body-part COCO, PoseModel::MPI_15 for 15 body-part MPI, PoseModel::MPI_15_4 for faster version * Select PoseModel::COCO_18 for 18 body-part COCO, PoseModel::MPI_15 for 15 body-part MPI, PoseModel::MPI_15_4
* of MPI, etc.). * for faster version of MPI, etc.).
*/ */
PoseModel poseModel; PoseModel poseModel;
...@@ -87,7 +98,8 @@ namespace op ...@@ -87,7 +98,8 @@ namespace op
float alphaKeypoint; float alphaKeypoint;
/** /**
* Rendering blending alpha value of the heat maps (body part, background or PAF) with respect to the background image. * Rendering blending alpha value of the heat maps (body part, background or PAF) with respect to the
* background image.
* Value in the range [0, 1]. 0 will only render the background, 1 will only render the heat map. * Value in the range [0, 1]. 0 will only render the background, 1 will only render the heat map.
*/ */
float alphaHeatMap; float alphaHeatMap;
...@@ -95,8 +107,9 @@ namespace op ...@@ -95,8 +107,9 @@ namespace op
/** /**
* Element to initially render. * Element to initially render.
* Set 0 for pose, [1, #body parts] for each body part following the order on POSE_BODY_PART_MAPPING on * Set 0 for pose, [1, #body parts] for each body part following the order on POSE_BODY_PART_MAPPING on
* `include/pose/poseParameters.hpp`, #body parts+1 for background, #body parts+2 for all body parts overlapped, * `include/pose/poseParameters.hpp`, #body parts+1 for background, #body parts+2 for all body parts
* #body parts+3 for all PAFs, and [#body parts+4, #body parts+4+#pair pairs] for each PAF following the order on POSE_BODY_PART_PAIRS. * overlapped, #body parts+3 for all PAFs, and [#body parts+4, #body parts+4+#pair pairs] for each PAF
* following the order on POSE_BODY_PART_PAIRS.
*/ */
int defaultPartToRender; int defaultPartToRender;
...@@ -107,21 +120,24 @@ namespace op ...@@ -107,21 +120,24 @@ namespace op
/** /**
* Whether and which heat maps to save on the Array<float> Datum.heatmaps. * Whether and which heat maps to save on the Array<float> Datum.heatmaps.
* Use HeatMapType::Parts for body parts, HeatMapType::Background for the background, and HeatMapType::PAFs for the Part Affinity Fields. * Use HeatMapType::Parts for body parts, HeatMapType::Background for the background, and HeatMapType::PAFs for
* the Part Affinity Fields.
*/ */
std::vector<HeatMapType> heatMapTypes; std::vector<HeatMapType> heatMapTypes;
/** /**
* Scale of the Datum.heatmaps. * Scale of the Datum.heatmaps.
* Select ScaleMode::ZeroToOne for range [0,1], ScaleMode::PlusMinusOne for [-1,1] and ScaleMode::UnsignedChar for [0, 255] * Select ScaleMode::ZeroToOne for range [0,1], ScaleMode::PlusMinusOne for [-1,1] and ScaleMode::UnsignedChar
* for [0, 255].
* If heatMapTypes.empty(), then this parameters makes no effect. * If heatMapTypes.empty(), then this parameters makes no effect.
*/ */
ScaleMode heatMapScale; ScaleMode heatMapScale;
/** /**
* Rendering threshold. Only estimated keypoints whose score confidences are higher than this value will be rendered. Generally, a * Rendering threshold. Only estimated keypoints whose score confidences are higher than this value will be
* high threshold (> 0.5) will only render very clear body parts; while small thresholds (~0.1) will also output guessed and occluded * rendered. Generally, a high threshold (> 0.5) will only render very clear body parts; while small thresholds
* keypoints, but also more false positives (i.e. wrong detections). * (~0.1) will also output guessed and occluded keypoints, but also more false positives (i.e. wrong
* detections).
*/ */
float renderThreshold; float renderThreshold;
...@@ -130,7 +146,7 @@ namespace op ...@@ -130,7 +146,7 @@ namespace op
* It has the recommended and default values we recommend for each element of the struct. * It has the recommended and default values we recommend for each element of the struct.
* Since all the elements of the struct are public, they can also be manually filled. * Since all the elements of the struct are public, they can also be manually filled.
*/ */
WrapperStructPose(const Point<int>& netInputSize = Point<int>{656, 368}, WrapperStructPose(const bool enable = true, const Point<int>& netInputSize = Point<int>{656, 368},
const Point<int>& outputSize = Point<int>{1280, 720}, const Point<int>& outputSize = Point<int>{1280, 720},
const ScaleMode keypointScale = ScaleMode::InputResolution, const ScaleMode keypointScale = ScaleMode::InputResolution,
const int gpuNumber = -1, const int gpuNumberStart = 0, const int scalesNumber = 1, const int gpuNumber = -1, const int gpuNumberStart = 0, const int scalesNumber = 1,
......
此差异已折叠。
set(SOURCES set(SOURCES
defineTemplates.cpp defineTemplates.cpp
faceDetector.cpp faceDetector.cpp
faceDetectorOpenCV.cpp
faceExtractor.cpp faceExtractor.cpp
faceCpuRenderer.cpp faceCpuRenderer.cpp
faceGpuRenderer.cpp faceGpuRenderer.cpp
......
...@@ -5,4 +5,5 @@ namespace op ...@@ -5,4 +5,5 @@ namespace op
DEFINE_TEMPLATE_DATUM(WFaceDetector); DEFINE_TEMPLATE_DATUM(WFaceDetector);
DEFINE_TEMPLATE_DATUM(WFaceExtractor); DEFINE_TEMPLATE_DATUM(WFaceExtractor);
DEFINE_TEMPLATE_DATUM(WFaceRenderer); DEFINE_TEMPLATE_DATUM(WFaceRenderer);
DEFINE_TEMPLATE_DATUM(WFaceDetectorOpenCV);
} }
#include <opencv2/imgproc/imgproc.hpp> // cv::COLOR_BGR2GRAY
#include <openpose/pose/poseParameters.hpp>
#include <openpose/face/faceDetectorOpenCV.hpp>
namespace op
{
FaceDetectorOpenCV::FaceDetectorOpenCV(const std::string& modelFolder)
{
try
{
const std::string faceDetectorModelPath{modelFolder + "face/haarcascade_frontalface_alt.xml"};
if (!mFaceCascade.load(faceDetectorModelPath))
error("Face detector model not found at: " + faceDetectorModelPath, __LINE__, __FUNCTION__, __FILE__);
}
catch (const std::exception& e)
{
error(e.what(), __LINE__, __FUNCTION__, __FILE__);
}
}
std::vector<Rectangle<float>> FaceDetectorOpenCV::detectFaces(const cv::Mat& cvInputData)
{
try
{
// Image to grey and pyrDown
cv::Mat frameGray;
cv::cvtColor(cvInputData, frameGray, cv::COLOR_BGR2GRAY);
auto multiplier = 1;
while (frameGray.cols * frameGray.rows > 640*360)
{
cv::pyrDown(frameGray, frameGray);
multiplier *= 2;
}
// Face detection - Example from:
// http://docs.opencv.org/2.4/doc/tutorials/objdetect/cascade_classifier/cascade_classifier.html
std::vector<cv::Rect> detectedFaces;
mFaceCascade.detectMultiScale(frameGray, detectedFaces, 1.2, 3, 0|CV_HAAR_SCALE_IMAGE);
// Rescale rectangles
std::vector<Rectangle<float>> faceRectangles(detectedFaces.size());
for(auto i = 0u; i < detectedFaces.size(); i++)
{
// Enlarge detected rectangle by 1.5x, so that it covers the whole face
faceRectangles.at(i).x = detectedFaces.at(i).x - 0.25*detectedFaces.at(i).width;
faceRectangles.at(i).y = detectedFaces.at(i).y - 0.25*detectedFaces.at(i).height;
faceRectangles.at(i).width = 1.5*detectedFaces.at(i).width;
faceRectangles.at(i).height = 1.5*detectedFaces.at(i).height;
faceRectangles.at(i) *= multiplier;
}
return faceRectangles;
}
catch (const std::exception& e)
{
error(e.what(), __LINE__, __FUNCTION__, __FILE__);
return {};
}
}
}
...@@ -69,7 +69,7 @@ namespace op ...@@ -69,7 +69,7 @@ namespace op
// Warnings // Warnings
if (!mHeatMapTypes.empty()) if (!mHeatMapTypes.empty())
log("Note only keypoint heatmaps are available with face heatmaps (no background nor PAFs).", log("Note only keypoint heatmaps are available with face heatmaps (no background nor PAFs).",
Priority::Low, __LINE__, __FUNCTION__, __FILE__); Priority::High);
} }
catch (const std::exception& e) catch (const std::exception& e)
{ {
......
...@@ -156,8 +156,8 @@ namespace op ...@@ -156,8 +156,8 @@ namespace op
__LINE__, __FUNCTION__, __FILE__); __LINE__, __FUNCTION__, __FILE__);
// Warnings // Warnings
if (!mHeatMapTypes.empty()) if (!mHeatMapTypes.empty())
log("Note only keypoint heatmaps are available with face heatmaps (no background nor PAFs).", log("Note only keypoint heatmaps are available with hand heatmaps (no background nor PAFs).",
Priority::Low, __LINE__, __FUNCTION__, __FILE__); Priority::High);
} }
catch (const std::exception& e) catch (const std::exception& e)
{ {
......
...@@ -2,15 +2,17 @@ ...@@ -2,15 +2,17 @@
namespace op namespace op
{ {
WrapperStructPose::WrapperStructPose(const Point<int>& netInputSize_, const Point<int>& outputSize_, WrapperStructPose::WrapperStructPose(const bool enable_, const Point<int>& netInputSize_,
const ScaleMode keypointScale_, const int gpuNumber_, const Point<int>& outputSize_, const ScaleMode keypointScale_,
const int gpuNumberStart_, const int scalesNumber_, const float scaleGap_, const int gpuNumber_, const int gpuNumberStart_,
const int scalesNumber_, const float scaleGap_,
const RenderMode renderMode_, const PoseModel poseModel_, const RenderMode renderMode_, const PoseModel poseModel_,
const bool blendOriginalFrame_, const float alphaKeypoint_, const bool blendOriginalFrame_, const float alphaKeypoint_,
const float alphaHeatMap_, const int defaultPartToRender_, const float alphaHeatMap_, const int defaultPartToRender_,
const std::string& modelFolder_, const std::string& modelFolder_,
const std::vector<HeatMapType>& heatMapTypes_, const std::vector<HeatMapType>& heatMapTypes_,
const ScaleMode heatMapScale_, const float renderThreshold_) : const ScaleMode heatMapScale_, const float renderThreshold_) :
enable{enable_},
netInputSize{netInputSize_}, netInputSize{netInputSize_},
outputSize{outputSize_}, outputSize{outputSize_},
keypointScale{keypointScale_}, keypointScale{keypointScale_},
......
Markdown is supported
0% .
You are about to add 0 people to the discussion. Proceed with caution.
先完成此消息的编辑!
想要评论请 注册