提交 7c1e8e40 编写于 作者: G Gines Hidalgo

Sanity checks for flir camera with 3-d #1254

上级 13ee82fc
...@@ -210,7 +210,7 @@ Each flag is divided into flag name, default value, and description. ...@@ -210,7 +210,7 @@ Each flag is divided into flag name, default value, and description.
- DEFINE_bool(disable_blending, false, "If enabled, it will render the results (keypoint skeletons or heatmaps) on a black background, instead of being rendered into the original image. Related: `part_to_show`, `alpha_pose`, and `alpha_pose`."); - DEFINE_bool(disable_blending, false, "If enabled, it will render the results (keypoint skeletons or heatmaps) on a black background, instead of being rendered into the original image. Related: `part_to_show`, `alpha_pose`, and `alpha_pose`.");
11. OpenPose Rendering Pose 11. OpenPose Rendering Pose
- DEFINE_double(render_threshold, 0.05, "Only estimated keypoints whose score confidences are higher than this threshold will be rendered. Generally, a high threshold (> 0.5) will only render very clear body parts; while small thresholds (~0.1) will also output guessed and occluded keypoints, but also more false positives (i.e., wrong detections)."); - DEFINE_double(render_threshold, 0.05, "Only estimated keypoints whose score confidences are higher than this threshold will be rendered. Note: Rendered refers only to visual display in the OpenPose basic GUI, not in the saved results. Generally, a high threshold (> 0.5) will only render very clear body parts; while small thresholds (~0.1) will also output guessed and occluded keypoints, but also more false positives (i.e., wrong detections).");
- DEFINE_int32(render_pose, -1, "Set to 0 for no rendering, 1 for CPU rendering (slightly faster), and 2 for GPU rendering (slower but greater functionality, e.g., `alpha_X` flags). If -1, it will pick CPU if CPU_ONLY is enabled, or GPU if CUDA is enabled. If rendering is enabled, it will render both `outputData` and `cvOutputData` with the original image and desired body part to be shown (i.e., keypoints, heat maps or PAFs)."); - DEFINE_int32(render_pose, -1, "Set to 0 for no rendering, 1 for CPU rendering (slightly faster), and 2 for GPU rendering (slower but greater functionality, e.g., `alpha_X` flags). If -1, it will pick CPU if CPU_ONLY is enabled, or GPU if CUDA is enabled. If rendering is enabled, it will render both `outputData` and `cvOutputData` with the original image and desired body part to be shown (i.e., keypoints, heat maps or PAFs).");
- DEFINE_double(alpha_pose, 0.6, "Blending factor (range 0-1) for the body part rendering. 1 will show it completely, 0 will hide it. Only valid for GPU rendering."); - DEFINE_double(alpha_pose, 0.6, "Blending factor (range 0-1) for the body part rendering. 1 will show it completely, 0 will hide it. Only valid for GPU rendering.");
- DEFINE_double(alpha_heatmap, 0.7, "Blending factor (range 0-1) between heatmap and original frame. 1 will only show the heatmap, 0 will only show the frame. Only valid for GPU rendering."); - DEFINE_double(alpha_heatmap, 0.7, "Blending factor (range 0-1) between heatmap and original frame. 1 will only show the heatmap, 0 will only show the frame. Only valid for GPU rendering.");
......
...@@ -18,6 +18,7 @@ OpenPose - Frequently Asked Question (FAQ) ...@@ -18,6 +18,7 @@ OpenPose - Frequently Asked Question (FAQ)
12. [CMake-GUI Error While Getting Default Caffe](#cmake-gui-error-while-getting-default-caffe) 12. [CMake-GUI Error While Getting Default Caffe](#cmake-gui-error-while-getting-default-caffe)
13. [Libgomp Out of Memory Error](#libgomp-out-of-memory-error) 13. [Libgomp Out of Memory Error](#libgomp-out-of-memory-error)
14. [Runtime Error with Turing GPU (Tesla T4) or Volta GPU][#runtime-error-with-turing-gpu-teslat4-or-volta-gpu) 14. [Runtime Error with Turing GPU (Tesla T4) or Volta GPU][#runtime-error-with-turing-gpu-teslat4-or-volta-gpu)
15. [Obscure CMake Error about Caffe or Pybind](#obscure-cmake-error-about-caffe-or-pybind).
2. [Speed Performance Issues](#speed-performance-issues) 2. [Speed Performance Issues](#speed-performance-issues)
1. [Speed Up, Memory Reduction, and Benchmark](#speed-up-memory-reduction-and-benchmark) 1. [Speed Up, Memory Reduction, and Benchmark](#speed-up-memory-reduction-and-benchmark)
2. [How to Measure the Latency Time?](#how-to-measure-the-latency-time) 2. [How to Measure the Latency Time?](#how-to-measure-the-latency-time)
...@@ -171,6 +172,14 @@ git submodle update ...@@ -171,6 +172,14 @@ git submodle update
#### Obscure CMake Error about Caffe or Pybind
**Q:** There appear some weird and obscure errors on CMake about Caffe and/or Pybind.
**A**: Check [doc/prerequisites.md#general-tips](./prerequisites.md#general-tips) to run the `git submodule` command.
### Speed Performance Issues ### Speed Performance Issues
#### Speed Up, Memory Reduction, and Benchmark #### Speed Up, Memory Reduction, and Benchmark
......
...@@ -330,9 +330,10 @@ Also as a side note, if the default installation fails (i.e., the one explained ...@@ -330,9 +330,10 @@ Also as a side note, if the default installation fails (i.e., the one explained
3. `Caffe_LIBS` set to `/usr/local/lib/libcaffe.dylib`. 3. `Caffe_LIBS` set to `/usr/local/lib/libcaffe.dylib`.
4. Run `Configure` and `Generate` from CMake GUI. 4. Run `Configure` and `Generate` from CMake GUI.
You may also have to apply the following patch if you have the latest OSX 10.14. It can be done as follows: In addition, if you face an OpenCV error during compiling time similar to `fatal error: 'opencv2/highgui/highgui.hpp' file not found`, please apply the following patch (this error has been reported in the latest OSX 10.14):
```
`cd 3rdparty/caffe; git apply ../../scripts/osx/mac_opencl_patch.txt` cd 3rdparty/caffe; git apply ../../scripts/osx/mac_opencl_patch.txt
```
#### 3D Reconstruction Module #### 3D Reconstruction Module
......
...@@ -100,7 +100,7 @@ The visual GUI should show 3 screens. ...@@ -100,7 +100,7 @@ The visual GUI should show 3 screens.
It should be similar to the following image. It should be similar to the following image.
<p align="center"> <p align="center">
<img src="media/openpose3d.png"> <img src="../media/openpose3d.gif">
</p> </p>
......
...@@ -184,9 +184,10 @@ DEFINE_bool(disable_blending, false, "If enabled, it will ren ...@@ -184,9 +184,10 @@ DEFINE_bool(disable_blending, false, "If enabled, it will ren
" `alpha_pose`, and `alpha_pose`."); " `alpha_pose`, and `alpha_pose`.");
// OpenPose Rendering Pose // OpenPose Rendering Pose
DEFINE_double(render_threshold, 0.05, "Only estimated keypoints whose score confidences are higher than this threshold will be" DEFINE_double(render_threshold, 0.05, "Only estimated keypoints whose score confidences are higher than this threshold will be"
" rendered. Generally, a high threshold (> 0.5) will only render very clear body parts;" " rendered. Note: Rendered refers only to visual display in the OpenPose basic GUI, not in"
" while small thresholds (~0.1) will also output guessed and occluded keypoints, but also" " the saved results. Generally, a high threshold (> 0.5) will only render very clear body"
" more false positives (i.e., wrong detections)."); " parts; while small thresholds (~0.1) will also output guessed and occluded keypoints,"
" but also more false positives (i.e., wrong detections).");
DEFINE_int32(render_pose, -1, "Set to 0 for no rendering, 1 for CPU rendering (slightly faster), and 2 for GPU rendering" DEFINE_int32(render_pose, -1, "Set to 0 for no rendering, 1 for CPU rendering (slightly faster), and 2 for GPU rendering"
" (slower but greater functionality, e.g., `alpha_X` flags). If -1, it will pick CPU if" " (slower but greater functionality, e.g., `alpha_X` flags). If -1, it will pick CPU if"
" CPU_ONLY is enabled, or GPU if CUDA is enabled. If rendering is enabled, it will render" " CPU_ONLY is enabled, or GPU if CUDA is enabled. If rendering is enabled, it will render"
......
...@@ -23,6 +23,10 @@ namespace op ...@@ -23,6 +23,10 @@ namespace op
std::vector<cv::Mat> getRawFrames(); std::vector<cv::Mat> getRawFrames();
/**
* Note: The camera parameters are only read if undistortImage is true. This should be changed to add a
* new bool flag in the constructor, e.g., readCameraParameters
*/
std::vector<cv::Mat> getCameraMatrices() const; std::vector<cv::Mat> getCameraMatrices() const;
std::vector<cv::Mat> getCameraExtrinsics() const; std::vector<cv::Mat> getCameraExtrinsics() const;
......
...@@ -70,8 +70,13 @@ namespace op ...@@ -70,8 +70,13 @@ namespace op
float getKeypointsRoi(const Array<T>& keypoints, const int personA, const int personB, const T threshold); float getKeypointsRoi(const Array<T>& keypoints, const int personA, const int personB, const T threshold);
template <typename T> template <typename T>
float getKeypointsRoi(const Array<T>& keypointsA, const int personA, const Array<T>& keypointsB, const int personB, float getKeypointsRoi(
const T threshold); const Array<T>& keypointsA, const int personA, const Array<T>& keypointsB, const int personB,
const T threshold);
template <typename T>
float getKeypointsRoi(
const Rectangle<T>& rectangleA, const Rectangle<T>& rectangleB);
} }
#endif // OPENPOSE_UTILITIES_KEYPOINT_HPP #endif // OPENPOSE_UTILITIES_KEYPOINT_HPP
...@@ -54,7 +54,8 @@ namespace op ...@@ -54,7 +54,8 @@ namespace op
/** /**
* Rendering threshold. Only estimated keypoints whose score confidences are higher than this value will be * Rendering threshold. Only estimated keypoints whose score confidences are higher than this value will be
* rendered. Generally, a high threshold (> 0.5) will only render very clear body parts; while small thresholds * rendered. Note: Rendered refers only to visual display in the OpenPose basic GUI, not in the saved results.
* Generally, a high threshold (> 0.5) will only render very clear body parts; while small thresholds
* (~0.1) will also output guessed and occluded keypoints, but also more false positives (i.e., wrong * (~0.1) will also output guessed and occluded keypoints, but also more false positives (i.e., wrong
* detections). * detections).
*/ */
......
...@@ -73,7 +73,8 @@ namespace op ...@@ -73,7 +73,8 @@ namespace op
/** /**
* Rendering threshold. Only estimated keypoints whose score confidences are higher than this value will be * Rendering threshold. Only estimated keypoints whose score confidences are higher than this value will be
* rendered. Generally, a high threshold (> 0.5) will only render very clear body parts; while small thresholds * rendered. Note: Rendered refers only to visual display in the OpenPose basic GUI, not in the saved results.
* Generally, a high threshold (> 0.5) will only render very clear body parts; while small thresholds
* (~0.1) will also output guessed and occluded keypoints, but also more false positives (i.e., wrong * (~0.1) will also output guessed and occluded keypoints, but also more false positives (i.e., wrong
* detections). * detections).
*/ */
......
...@@ -143,7 +143,8 @@ namespace op ...@@ -143,7 +143,8 @@ namespace op
/** /**
* Rendering threshold. Only estimated keypoints whose score confidences are higher than this value will be * Rendering threshold. Only estimated keypoints whose score confidences are higher than this value will be
* rendered. Generally, a high threshold (> 0.5) will only render very clear body parts; while small thresholds * rendered. Note: Rendered refers only to visual display in the OpenPose basic GUI, not in the saved results.
* Generally, a high threshold (> 0.5) will only render very clear body parts; while small thresholds
* (~0.1) will also output guessed and occluded keypoints, but also more false positives (i.e., wrong * (~0.1) will also output guessed and occluded keypoints, but also more false positives (i.e., wrong
* detections). * detections).
*/ */
......
...@@ -505,8 +505,8 @@ namespace op ...@@ -505,8 +505,8 @@ namespace op
{ {
try try
{ {
return reconstructArray(std::vector<std::vector<Array<float>>>{keypointsVector}, return reconstructArray(
cameraMatrices, imageSizes).at(0); std::vector<std::vector<Array<float>>>{keypointsVector}, cameraMatrices, imageSizes).at(0);
} }
catch (const std::exception& e) catch (const std::exception& e)
{ {
...@@ -515,6 +515,9 @@ namespace op ...@@ -515,6 +515,9 @@ namespace op
} }
} }
const std::string sFlirErrorMessage{
" If you are simultaneously using FLIR cameras (`--flir_camera`) and the 3-D reconstruction module"
" (`--3d), you should also enable `--frame_undistort` so their camera parameters are read."};
std::vector<Array<float>> PoseTriangulation::reconstructArray( std::vector<Array<float>> PoseTriangulation::reconstructArray(
const std::vector<std::vector<Array<float>>>& keypointsVectors, const std::vector<std::vector<Array<float>>>& keypointsVectors,
const std::vector<cv::Mat>& cameraMatrices, const std::vector<cv::Mat>& cameraMatrices,
...@@ -522,6 +525,20 @@ namespace op ...@@ -522,6 +525,20 @@ namespace op
{ {
try try
{ {
// Sanity checks
if (cameraMatrices.size() < 2)
error("3-D reconstruction (`--3d`) requires at least 2 camera views, only found "
+ std::to_string(cameraMatrices.size()) + "camera parameter matrices." + sFlirErrorMessage,
__LINE__, __FUNCTION__, __FILE__);
for (const auto& cameraMatrix : cameraMatrices)
if (cameraMatrix.empty())
error("Camera matrix was found empty during 3-D reconstruction (`--3d`)." + sFlirErrorMessage,
__LINE__, __FUNCTION__, __FILE__);
if (cameraMatrices.size() != imageSizes.size())
error("The camera parameters and number of images must be the same ("
+ std::to_string(cameraMatrices.size()) + " vs. " + std::to_string(imageSizes.size()) + ").",
__LINE__, __FUNCTION__, __FILE__);
// Run 3-D reconstruction
bool keypointsReconstructed = false; bool keypointsReconstructed = false;
std::vector<Array<float>> keypoints3Ds(keypointsVectors.size()); std::vector<Array<float>> keypoints3Ds(keypointsVectors.size());
// std::vector<std::thread> threads(keypointsVectors.size()-1); // std::vector<std::thread> threads(keypointsVectors.size()-1);
......
...@@ -27,7 +27,7 @@ namespace op ...@@ -27,7 +27,7 @@ namespace op
__LINE__, __FUNCTION__, __FILE__); __LINE__, __FUNCTION__, __FILE__);
// Warnings // Warnings
if (!mHeatMapTypes.empty()) if (!mHeatMapTypes.empty())
log("Note only keypoint heatmaps are available with face heatmaps (no background nor PAFs).", log("Note that only the keypoint heatmaps are available with face heatmaps (no background nor PAFs).",
Priority::High); Priority::High);
} }
catch (const std::exception& e) catch (const std::exception& e)
......
...@@ -29,7 +29,7 @@ namespace op ...@@ -29,7 +29,7 @@ namespace op
__LINE__, __FUNCTION__, __FILE__); __LINE__, __FUNCTION__, __FILE__);
// Warnings // Warnings
if (!mHeatMapTypes.empty()) if (!mHeatMapTypes.empty())
log("Note only keypoint heatmaps are available with hand heatmaps (no background nor PAFs).", log("Note that only the keypoint heatmaps are available with hand heatmaps (no background nor PAFs).",
Priority::High); Priority::High);
} }
catch (const std::exception& e) catch (const std::exception& e)
......
...@@ -83,12 +83,12 @@ namespace op ...@@ -83,12 +83,12 @@ namespace op
void getRoiDiameterAndBounds( void getRoiDiameterAndBounds(
Rectangle<int>& roi, int& diameter, int& partFirstNon0, int& partLastNon0, Rectangle<int>& roi, int& diameter, int& partFirstNon0, int& partLastNon0,
const std::vector<int>& personVector, const T* const peaksPtr, const std::vector<int>& personVector, const T* const peaksPtr,
const int partInit, const int partEnd) const int partInit, const int partEnd, const float margin)
{ {
try try
{ {
// Find ROI, partFirstNon0, and partLastNon0 // Find ROI, partFirstNon0, and partLastNon0
roi = Rectangle<int>{0,0,0,0}; roi = Rectangle<int>{std::numeric_limits<int>::max(),std::numeric_limits<int>::max(),0,0};
partFirstNon0 = -1; partFirstNon0 = -1;
partLastNon0 = -1; partLastNon0 = -1;
for (auto part = partInit ; part < partEnd ; part++) for (auto part = partInit ; part < partEnd ; part++)
...@@ -96,7 +96,7 @@ namespace op ...@@ -96,7 +96,7 @@ namespace op
const auto x = peaksPtr[personVector[part]-2]; const auto x = peaksPtr[personVector[part]-2];
const auto y = peaksPtr[personVector[part]-1]; const auto y = peaksPtr[personVector[part]-1];
const auto score = peaksPtr[personVector[part]]; const auto score = peaksPtr[personVector[part]];
if (score > 0) if (personVector[part] > 0 && score > 0)
{ {
// ROI // ROI
if (roi.x > x) if (roi.x > x)
...@@ -105,7 +105,7 @@ namespace op ...@@ -105,7 +105,7 @@ namespace op
roi.y = y; roi.y = y;
if (roi.width < x) if (roi.width < x)
roi.width = x; roi.width = x;
if (roi.height > y) if (roi.height < y)
roi.height = y; roi.height = y;
// First keypoint? // First keypoint?
if (partFirstNon0 < 0) if (partFirstNon0 < 0)
...@@ -114,11 +114,24 @@ namespace op ...@@ -114,11 +114,24 @@ namespace op
partLastNon0 = part; partLastNon0 = part;
} }
} }
// From [p1, p2] to [p1, width, height] if (partLastNon0 > -1)
roi.width -= roi.x; {
roi.height -= roi.y; // Add margin
// diameter const auto marginX = roi.width * margin;
diameter = fastMax(roi.width, roi.height); const auto marginY = roi.height * margin;
roi.x -= marginX;
roi.y -= marginY;
roi.width += 2*marginX;
roi.height += 2*marginY;
// partFirstNon0+1 for for loops
partLastNon0++;
// From [p1, p2] to [p1, width, height]
// +1 to account for 1-line keypoints
roi.width += 1 - roi.x;
roi.height += 1 - roi.y;
// diameter
diameter = fastMax(roi.width, roi.height);
}
} }
catch (const std::exception& e) catch (const std::exception& e)
{ {
...@@ -712,7 +725,8 @@ namespace op ...@@ -712,7 +725,8 @@ namespace op
faceValidSubsetIndexes.reserve(peopleVector.size()); faceValidSubsetIndexes.reserve(peopleVector.size());
// Face invalid sets // Face invalid sets
std::vector<int> faceInvalidSubsetIndexes; std::vector<int> faceInvalidSubsetIndexes;
faceInvalidSubsetIndexes.reserve(peopleVector.size()); if (numberBodyParts >= 135)
faceInvalidSubsetIndexes.reserve(peopleVector.size());
// For each person candidate // For each person candidate
for (auto person = 0u ; person < peopleVector.size() ; person++) for (auto person = 0u ; person < peopleVector.size() ; person++)
{ {
...@@ -768,6 +782,77 @@ namespace op ...@@ -768,6 +782,77 @@ namespace op
error("Bad personCounter (" + std::to_string(personCounter) + "). Bug in this" error("Bad personCounter (" + std::to_string(personCounter) + "). Bug in this"
" function if this happens.", __LINE__, __FUNCTION__, __FILE__); " function if this happens.", __LINE__, __FUNCTION__, __FILE__);
} }
// Random standalone facial keypoints --> Merge into a more complete face
if (numberPeople > 0)
{
// Check invalid faces
for (const auto& personInvalid : faceInvalidSubsetIndexes)
{
// Get ROI of current face
Rectangle<int> roiInvalid;
int diameterInvalid = -1;
int partFirstNon0Invalid = -1;
int partLastNon0Invalid = -1;
getRoiDiameterAndBounds(
roiInvalid, diameterInvalid, partFirstNon0Invalid, partLastNon0Invalid,
peopleVector[personInvalid].first, peaksPtr, 65, 135, 0.2f);
// Check all valid faces to find best candidate
float keypointsRoiBest = 0.f;
auto keypointsRoiBestIndex = -1;
for (auto personId = 0u ; personId < faceValidSubsetIndexes.size() ; personId++)
{
const auto& personValid = faceValidSubsetIndexes[personId];
// Get ROI of current face
Rectangle<int> roiValid;
int diameterValid = -1;
int partFirstNon0Valid = -1;
int partLastNon0Valid = -1;
getRoiDiameterAndBounds(
roiValid, diameterValid, partFirstNon0Valid, partLastNon0Valid, peopleVector[personValid].first,
peaksPtr, 65, 135, 0.1f);
// Get ROI between both faces
const auto keypointsRoi = getKeypointsRoi(roiValid, roiInvalid);
// Update best so far
if (keypointsRoiBest < keypointsRoi)
{
keypointsRoiBest = keypointsRoi;
keypointsRoiBestIndex = personId;
}
}
// If invalid and best valid candidate overlap enough --> Merge them
if (keypointsRoiBest > 0.3f || (keypointsRoiBest > 0.01f && faceValidSubsetIndexes.size() < 3))
{
const auto& personValid = faceValidSubsetIndexes[keypointsRoiBestIndex];
// If it is from that face --> Combine invalid face keypoints into valid face
for (auto part = partFirstNon0Invalid ; part < partLastNon0Invalid ; part++)
{
auto& personVectorValid = peopleVector[personValid].first;
const auto scoreValid = peaksPtr[personVectorValid[part]];
const auto& personVectorInvalid = peopleVector[personInvalid].first;
const auto scoreInvalid = peaksPtr[personVectorInvalid[part]];
// If the new one has a keypoint...
if (personVectorInvalid[part] != 0)
{
// ... and the original face does not have it, then add it to it
if (personVectorValid[part] == 0)
{
if (personVectorInvalid[part] != 0)
{
personVectorValid[part] = personVectorInvalid[part];
peopleVector[personValid].second += scoreInvalid;
}
}
// ... and its score is higher than the original one, then replace it
else if (scoreValid < scoreInvalid)
{
personVectorValid[part] = personVectorInvalid[part];
peopleVector[personValid].second += scoreInvalid - scoreValid;
}
}
}
}
}
}
// If no people found --> Repeat with maximizePositives = true // If no people found --> Repeat with maximizePositives = true
// Result: Increased COCO mAP because we catch more foot-only images // Result: Increased COCO mAP because we catch more foot-only images
if (numberPeople == 0 && !maximizePositives) if (numberPeople == 0 && !maximizePositives)
......
...@@ -716,7 +716,11 @@ namespace op ...@@ -716,7 +716,11 @@ namespace op
return 0.01f; return 0.01f;
// Non-MPI models // Non-MPI models
else else
return (maximizePositives ? 0.01f : 0.05f); // return (maximizePositives ? 0.01f : 0.5f); // 0.485 but much less false positive connections
// return (maximizePositives ? 0.01f : 0.1f); // 0.518
// return (maximizePositives ? 0.01f : 0.075f); // 0.521
return (maximizePositives ? 0.01f : 0.05f); // 0.523
// return (maximizePositives ? 0.01f : 0.01f); // 0.527 but huge amount of false positives joints
} }
catch (const std::exception& e) catch (const std::exception& e)
{ {
......
...@@ -828,9 +828,9 @@ namespace op ...@@ -828,9 +828,9 @@ namespace op
} }
} }
void renderPosePAFGpu(float* framePtr, const PoseModel poseModel, const Point<int>& frameSize, void renderPosePAFGpu(
const float* const heatMapPtr, const Point<int>& heatMapSize, const float scaleToKeepRatio, float* framePtr, const PoseModel poseModel, const Point<int>& frameSize, const float* const heatMapPtr,
const int part, const float alphaBlending) const Point<int>& heatMapSize, const float scaleToKeepRatio, const int part, const float alphaBlending)
{ {
try try
{ {
...@@ -843,16 +843,16 @@ namespace op ...@@ -843,16 +843,16 @@ namespace op
} }
} }
void renderPosePAFsGpu(float* framePtr, const PoseModel poseModel, const Point<int>& frameSize, void renderPosePAFsGpu(
const float* const heatMapPtr, const Point<int>& heatMapSize, const float scaleToKeepRatio, float* framePtr, const PoseModel poseModel, const Point<int>& frameSize, const float* const heatMapPtr,
const float alphaBlending) const Point<int>& heatMapSize, const float scaleToKeepRatio, const float alphaBlending)
{ {
try try
{ {
const auto numberBodyPartPairs = (int)getPosePartPairs(poseModel).size()/2; const auto numberBodyPartPairs = (int)getPosePartPairs(poseModel).size()/2;
renderPosePAFGpuAux( renderPosePAFGpuAux(
framePtr, poseModel, frameSize, heatMapPtr, heatMapSize, scaleToKeepRatio, framePtr, poseModel, frameSize, heatMapPtr, heatMapSize, scaleToKeepRatio,
getPoseNumberBodyParts(poseModel) + (poseModel != PoseModel::BODY_25B ? 1 : 0), getPoseNumberBodyParts(poseModel) + (addBkgChannel(poseModel) ? 1 : 0),
numberBodyPartPairs, alphaBlending); numberBodyPartPairs, alphaBlending);
} }
catch (const std::exception& e) catch (const std::exception& e)
...@@ -861,9 +861,9 @@ namespace op ...@@ -861,9 +861,9 @@ namespace op
} }
} }
void renderPoseDistanceGpu(float* framePtr, const Point<int>& frameSize, const float* const heatMapPtr, void renderPoseDistanceGpu(
const Point<int>& heatMapSize, const float scaleToKeepRatio, const unsigned int part, float* framePtr, const Point<int>& frameSize, const float* const heatMapPtr, const Point<int>& heatMapSize,
const float alphaBlending) const float scaleToKeepRatio, const unsigned int part, const float alphaBlending)
{ {
try try
{ {
......
...@@ -551,8 +551,9 @@ namespace op ...@@ -551,8 +551,9 @@ namespace op
const Array<double>& keypoints, const int personA, const int personB, const double threshold); const Array<double>& keypoints, const int personA, const int personB, const double threshold);
template <typename T> template <typename T>
float getKeypointsRoi(const Array<T>& keypointsA, const int personA, const Array<T>& keypointsB, const int personB, float getKeypointsRoi(
const T threshold) const Array<T>& keypointsA, const int personA, const Array<T>& keypointsB, const int personB,
const T threshold)
{ {
try try
{ {
...@@ -566,13 +567,50 @@ namespace op ...@@ -566,13 +567,50 @@ namespace op
// Get ROI // Get ROI
const auto rectangleA = getKeypointsRectangle(keypointsA, personA, threshold); const auto rectangleA = getKeypointsRectangle(keypointsA, personA, threshold);
const auto rectangleB = getKeypointsRectangle(keypointsB, personB, threshold); const auto rectangleB = getKeypointsRectangle(keypointsB, personB, threshold);
return getKeypointsRoi(rectangleA, rectangleB);
}
catch (const std::exception& e)
{
error(e.what(), __LINE__, __FUNCTION__, __FILE__);
return 0.f;
}
}
template OP_API float getKeypointsRoi(
const Array<float>& keypointsA, const int personA, const Array<float>& keypointsB, const int personB,
const float threshold);
template OP_API float getKeypointsRoi(
const Array<double>& keypointsA, const int personA, const Array<double>& keypointsB, const int personB,
const double threshold);
template <typename T>
float getKeypointsRoi(const Rectangle<T>& rectangleA, const Rectangle<T>& rectangleB)
{
try
{
// Check if negative values, then normalize it
auto rectangleANorm = rectangleA;
auto rectangleBNorm = rectangleB;
// E.g., [-10,-10,W1,H1] and [-20,-20,W2,H2] should be equivalent to [10,10,W1,H1] and [0,0,W2,H2]
const auto biasX = std::min(std::min(T{0}, rectangleA.x), rectangleB.x);
if (biasX != 0)
{
rectangleANorm.x -= biasX;
rectangleBNorm.x -= biasX;
}
const auto biasY = std::min(std::min(T{0}, rectangleA.y), rectangleB.y);
if (biasY != 0)
{
rectangleANorm.y -= biasY;
rectangleBNorm.y -= biasY;
}
// Get ROI
const Point<T> pointAIntersection{ const Point<T> pointAIntersection{
fastMax(rectangleA.x, rectangleB.x), fastMax(rectangleANorm.x, rectangleBNorm.x),
fastMax(rectangleA.y, rectangleB.y) fastMax(rectangleANorm.y, rectangleBNorm.y)
}; };
const Point<T> pointBIntersection{ const Point<T> pointBIntersection{
fastMin(rectangleA.x+rectangleA.width, rectangleB.x+rectangleB.width), fastMin(rectangleANorm.x+rectangleANorm.width, rectangleBNorm.x+rectangleBNorm.width),
fastMin(rectangleA.y+rectangleA.height, rectangleB.y+rectangleB.height) fastMin(rectangleANorm.y+rectangleANorm.height, rectangleBNorm.y+rectangleBNorm.height)
}; };
// Make sure there is overlap // Make sure there is overlap
if (pointAIntersection.x < pointBIntersection.x && pointAIntersection.y < pointBIntersection.y) if (pointAIntersection.x < pointBIntersection.x && pointAIntersection.y < pointBIntersection.y)
...@@ -583,13 +621,14 @@ namespace op ...@@ -583,13 +621,14 @@ namespace op
pointBIntersection.x-pointAIntersection.x, pointBIntersection.x-pointAIntersection.x,
pointBIntersection.y-pointAIntersection.y pointBIntersection.y-pointAIntersection.y
}; };
const auto areaA = rectangleA.area(); const auto areaA = rectangleANorm.area();
const auto areaB = rectangleB.area(); const auto areaB = rectangleBNorm.area();
const auto intersection = rectangleIntersection.area(); const auto intersection = rectangleIntersection.area();
return float(intersection) / float(areaA + areaB - intersection); return float(intersection) / float(areaA + areaB - intersection);
} }
// If non overlap --> Return 0 // If non overlap --> Return 0
return 0.f; else
return 0.f;
} }
catch (const std::exception& e) catch (const std::exception& e)
{ {
...@@ -598,9 +637,11 @@ namespace op ...@@ -598,9 +637,11 @@ namespace op
} }
} }
template OP_API float getKeypointsRoi( template OP_API float getKeypointsRoi(
const Array<float>& keypointsA, const int personA, const Array<float>& keypointsB, const int personB, const Rectangle<int>& rectangleA, const Rectangle<int>& rectangleB);
const float threshold);
template OP_API float getKeypointsRoi( template OP_API float getKeypointsRoi(
const Array<double>& keypointsA, const int personA, const Array<double>& keypointsB, const int personB, const Rectangle<unsigned int>& rectangleA, const Rectangle<unsigned int>& rectangleB);
const double threshold); template OP_API float getKeypointsRoi(
const Rectangle<float>& rectangleA, const Rectangle<float>& rectangleB);
template OP_API float getKeypointsRoi(
const Rectangle<double>& rectangleA, const Rectangle<double>& rectangleB);
} }
Markdown is supported
0% .
You are about to add 0 people to the discussion. Proceed with caution.
先完成此消息的编辑!
想要评论请 注册