@@ -15,7 +15,7 @@ Note: Currently using [travis-matrix-badges](https://github.com/bjfish/travis-ma
[**OpenPose**](https://github.com/CMU-Perceptual-Computing-Lab/openpose) represents the **first real-time multi-person system to jointly detect human body, hand, facial, and foot keypoints (in total 135 keypoints) on single images**.
It is **authored by [Gines Hidalgo](https://www.gineshidalgo.com), [Zhe Cao](https://people.eecs.berkeley.edu/~zhecao), [Tomas Simon](http://www.cs.cmu.edu/~tsimon), [Shih-En Wei](https://scholar.google.com/citations?user=sFQD3k4AAAAJ&hl=en), [Hanbyul Joo](https://jhugestar.github.io), and [Yaser Sheikh](http://www.cs.cmu.edu/~yaser)**. Currently, it is being **maintained by [Gines Hidalgo](https://www.gineshidalgo.com) and [Yaadhav Raaj](https://www.linkedin.com/in/yaadhavraaj)**. In addition, OpenPose would not be possible without the [**CMU Panoptic Studio dataset**](http://domedb.perception.cs.cmu.edu). We would also like to thank all the people who helped OpenPose in any way. The main contributors are listed in [doc/contributors.md](doc/contributors.md).
It is **authored by [Gines Hidalgo](https://www.gineshidalgo.com), [Zhe Cao](https://people.eecs.berkeley.edu/~zhecao), [Tomas Simon](http://www.cs.cmu.edu/~tsimon), [Shih-En Wei](https://scholar.google.com/citations?user=sFQD3k4AAAAJ&hl=en), [Hanbyul Joo](https://jhugestar.github.io), and [Yaser Sheikh](http://www.cs.cmu.edu/~yaser)**. Currently, it is being **maintained by [Gines Hidalgo](https://www.gineshidalgo.com) and [Yaadhav Raaj](https://www.raaj.tech)**. In addition, OpenPose would not be possible without the [**CMU Panoptic Studio dataset**](http://domedb.perception.cs.cmu.edu). We would also like to thank all the people who helped OpenPose in any way. The main contributors are listed in [doc/contributors.md](doc/contributors.md).
<!-- The [original CVPR 2017 repo](https://github.com/ZheC/Multi-Person-Pose-Estimation) includes Matlab and Python versions, as well as the training code. The body pose estimation work is based on [the original ECCV 2016 demo](https://github.com/CMU-Perceptual-Computing-Lab/caffe_rtpose). -->
@@ -11,7 +11,7 @@ OpenPose is authored by [Gines Hidalgo](https://www.gineshidalgo.com/), [Zhe Cao
### Contributors
We would also like to thank the following people who have highly contributed to OpenPose:
1.[Yaadhav Raaj](https://www.linkedin.com/in/yaadhavraaj): OpenPose maintainer, CPU version, OpenCL version, Mac version, Python API, and person tracker.
1.[Yaadhav Raaj](https://www.raaj.tech): OpenPose maintainer, CPU version, OpenCL version, Mac version, Python API, and person tracker.
2.[Bikramjot Hanzra](https://www.linkedin.com/in/bikz05): Former OpenPose maintainer, CMake (Ubuntu and Windows) version, and initial Travis Build version for Ubuntu.
3.[Donglai Xiang](https://xiangdonglai.github.io): Camera calibration toolbox improvement, including the implementation of its bundle adjustment algorithm.
4.[Luis Fernando Fraga](https://github.com/fragalfernando): Implementation of Lukas-Kanade algorith and person ID extractor.
@@ -166,11 +166,12 @@ Each flag is divided into flag name, default value, and description.
- DEFINE_double(fps_max, -1., "Maximum processing frame rate. By default (-1), OpenPose will process frames as fast as possible. Example usage: If OpenPose is displaying images too quickly, this can reduce the speed so the user can analyze better each frame from the GUI.");
4. OpenPose Body Pose
- DEFINE_bool(body_disable, false, "Disable body keypoint detection. Option only possible for faster (but less accurate) face keypoint detection.");
- DEFINE_int32(body, 1, "Select 0 to disable body keypoint detection (e.g., for faster but less accurate face keypoint detection, custom hand detector, etc.), 1 (default) for body keypoint estimation, and 2 to disable its internal body pose estimation network but still still run the greedy association parsing algorithm");
- DEFINE_string(model_pose, "BODY_25", "Model to be used. E.g., `COCO` (18 keypoints), `MPI` (15 keypoints, ~10% faster), `MPI_4_layers` (15 keypoints, even faster but less accurate).");
- DEFINE_string(net_resolution, "-1x368", "Multiples of 16. If it is increased, the accuracy potentially increases. If it is decreased, the speed increases. For maximum speed-accuracy balance, it should keep the closest aspect ratio possible to the images or videos to be processed. Using `-1` in any of the dimensions, OP will choose the optimal aspect ratio depending on the user's input value. E.g., the default `-1x368` is equivalent to `656x368` in 16:9 resolutions, e.g., full HD (1980x1080) and HD (1280x720) resolutions.");
- DEFINE_int32(scale_number, 1, "Number of scales to average.");
- DEFINE_double(scale_gap, 0.25, "Scale gap between scales. No effect unless scale_number > 1. Initial scale is always 1. If you want to change the initial scale, you actually want to multiply the `net_resolution` by your desired initial scale.");
- DEFINE_double(upsampling_ratio, 0., "Upsampling ratio between the `net_resolution` and the output net results. A value less or equal than 0 (default) will use the network default value (recommended).");
5. OpenPose Body Pose Heatmaps and Part Candidates
- DEFINE_bool(heatmaps_add_parts, false, "If true, it will fill op::Datum::poseHeatMaps array with the body part heatmaps, and analogously face & hand heatmaps to op::Datum::faceHeatMaps & op::Datum::handHeatMaps. If more than one `add_heatmaps_X` flag is enabled, it will place then in sequential memory order: body parts + bkg + PAFs. It will follow the order on POSE_BODY_PART_MAPPING in `src/openpose/pose/poseParameters.cpp`. Program speed will considerably decrease. Not required for OpenPose, enable it only if you intend to explicitly use this information later.");
3. Modify `WUserInput` and add your custom code there. Your code should fill `Datum::name`, `Datum::cameraMatrix`, `Datum::cvInputData`, and `Datum::cvOutputData` (fill cvOutputData = cvInputData).
4. Remove `WUserPostProcessing` and `WUserOutput` (unless you want to have your custom post-processing and/or output).
8. Given that display can be disabled in all examples, they all have been added to the Travis build so they can be tested.
7. Added a virtual destructor to almost all clases, so they can be inherited. Exceptions (for performance reasons): Array, Point, Rectangle, CvMatToOpOutput, OpOutputToCvMat.
8. Auxiliary classes in errorAndLog turned into namespaces (Profiler must be kept as class to allow static parameters).
9. Added flag `--frame_step` to allow the user to select the step or gap between processed frames. E.g., `--frame_step 5` would read and process frames 0, 5, 10, etc.
10. Added sanity checks to avoid `--frame_last` to be smaller than `--frame_first` or higher than the number of total frames.
11. Array improvements for Pybind11 compatibility:
9. Added flags:
1. Added flag `--frame_step` to allow the user to select the step or gap between processed frames. E.g., `--frame_step 5` would read and process frames 0, 5, 10, etc.
2. Previously hardcoded `COCO_CHALLENGE` variable turned into user configurable flag `--maximize_positives`.
3. Added flag `--verbose` to plot the progress.
4. Added flag `--fps_max` to limit the maximum processing frame rate of OpenPose (useful to display results at a maximum desired speed).
5. Added sanit30. Added the flags `--prototxt_path` and `--caffemodel_path` to allow custom ProtoTxt and CaffeModel paths.
6. Added the flags `--face_detector` and `--hand_detector`, that enable the user to select the face/hand rectangle detector that is used for the later face/hand keypoint detection. It includes OpenCV (for face), and also allows the user to provide its own input. Flag `--hand_tracking` is removed and integrated into this flag too.
y checks to avoid `--frame_last` to be smaller than `--frame_first` or higher than the number of total frames.
7. Added the flag `--upsampling_ratio`, which controls the upsampling than OpenPose will perform to the frame before the greedy association parsing algorithm.
8. Added the flag `--body` (replacing `--body_disable`), which adds the possibility of disabling the OpenPose pose network but still running the greedy association parsing algorithm (on top of the user heatmaps, see the associated `tutorial_api_cpp` example).
10. Array improvements for Pybind11 compatibility:
1. Array::getStride() to get step size of each dimension of the array.
2. Array::getPybindPtr() to get an editable const pointer.
3. Array::pData as binding of spData.
4. Array::Array that takes as input a pointer, so it does not re-allocate memory.
12. Producer defined inside Wrapper rather than being defined on each example.
13. Reduced many Visual Studio warnings (e.g., uncontrolled conversions between types).
14. Added new keypoint-related auxiliary functions in `utilities/keypoints.hpp`.
15. Function `resizeFixedAspectRatio` can take already allocated memory (e.g., faster if target is an Array<T> object, no intermediate cv::Mat required).
16. Added compatibility for OpenCV 4.0, while preserving 2.4.X and 3.X compatibility.
17. Improved and added several functions to `utilities/keypoints.hpp` and Array to simplify keypoint post-processing.
18. Removed warnings from Spinnaker SDK at compiling time.
19. All bash scripts incorporate `#!/bin/bash` to tell the terminal that they are bash scripts.
20. Added flag `--verbose` to plot the progress.
21. Added find_package(Protobuf) to allow specific versions of Protobuf.
22. Video saving improvements:
11. Producer defined inside Wrapper rather than being defined on each example.
12. Reduced many Visual Studio warnings (e.g., uncontrolled conversions between types).
13. Added new keypoint-related auxiliary functions in `utilities/keypoints.hpp`.
14. Function `resizeFixedAspectRatio` can take already allocated memory (e.g., faster if target is an Array<T> object, no intermediate cv::Mat required).
15. Added compatibility for OpenCV 4.0, while preserving 2.4.X and 3.X compatibility.
16. Improved and added several functions to `utilities/keypoints.hpp` and Array to simplify keypoint post-processing.
17. Removed warnings from Spinnaker SDK at compiling time.
18. All bash scripts incorporate `#!/bin/bash` to tell the terminal that they are bash scripts.
19. Added find_package(Protobuf) to allow specific versions of Protobuf.
20. Video saving improvements:
1. Video (`--write_video`) can be generated from images (`--image_dir`), as long as they maintain the same resolution.
2. Video with the 3D output can be saved with the new `--write_video_3d` flag.
3. Added the capability of saving videos in MP4 format (by using the ffmpeg library).
4. Added the flag `write_video_with_audio` to enable saving these output MP4 videos with audio.
23. Added `--fps_max` flag to limit the maximum processing frame rate of OpenPose (useful to display results at a maximum desired speed).
24. Frame undistortion can be applied not only to FLIR cameras, but also to all other input sources (image, webcam, video, etc.).
25. Calibration improvements:
21. Frame undistortion can be applied not only to FLIR cameras, but also to all other input sources (image, webcam, video, etc.).
22. Calibration improvements:
1. Improved chessboard orientation detection, more robust and less errors.
2. Triangulation functions (triangulate and triangulateWithOptimization) public, so calibration can use them for bundle adjustment.
3. Added bundle adjustment refinement for camera extrinsic calibration.
4. Added `CameraMatrixInitial` field into the XML calibration files to keep the information of the original camera extrinsic parameters when bundle adjustment is run.
26. Added Mac OpenCL compatibility.
27. Added documentation for Nvidia TX2 with JetPack 3.3.
28. Added Travis build check for several configurations: Ubuntu (14/16)/Mac/Windows, CPU/CUDA/OpenCL, with/without Python, and Release/Debug.
29. Assigned 755 access to all sh scripts (some of them were only 644).
30. Added the flags `--prototxt_path` and `--caffemodel_path` to allow custom ProtoTxt and CaffeModel paths.
31. Replaced the old Python wrapper for an updated Pybind11 wrapper version, that includes all the functionality of the C++ API.
32. Function getFilesOnDirectory() can extra all basic image file types at once without requiring to manually enumerate them.
33. Added the flags `--face_detector` and `--hand_detector`, that enable the user to select the face/hand rectangle detector that is used for the later face/hand keypoint detection. It includes OpenCV (for face), and also allows the user to provide its own input. Flag `--hand_tracking` is removed and integrated into this flag too.
34. Maximum queue size per OpenPose thread is configurable through the Wrapper class.
35. Added pre-processing capabilities to Wrapper (WorkerType::PreProcessing), which will be run right after the image has been read.
36. Removed boost::shared_ptr and caffe::Blob dependencies from the headers. No 3rdparty dependencies left on headers (except dim3 for CUDA).
37. Added `poseNetOutput` to Datum so that user can introduce his custom network output.
23. Added Mac OpenCL compatibility.
24. Added documentation for Nvidia TX2 with JetPack 3.3.
25. Added Travis build check for several configurations: Ubuntu (14/16)/Mac/Windows, CPU/CUDA/OpenCL, with/without Python, and Release/Debug.
26. Assigned 755 access to all sh scripts (some of them were only 644).
27. Replaced the old Python wrapper for an updated Pybind11 wrapper version, that includes all the functionality of the C++ API.
28. Function getFilesOnDirectory() can extra all basic image file types at once without requiring to manually enumerate them.
29. Maximum queue size per OpenPose thread is configurable through the Wrapper class.
30. Added pre-processing capabilities to Wrapper (WorkerType::PreProcessing), which will be run right after the image has been read.
31. Removed boost::shared_ptr and caffe::Blob dependencies from the headers. No 3rdparty dependencies left on headers (except dim3 for CUDA).
32. Added Array `poseNetOutput` to Datum so that user can introduce his custom network output.
2. Functions or parameters renamed:
1. By default, python example `tutorial_developer/python_2_pose_from_heatmaps.py` was using 2 scales starting at -1x736, changed to 1 scale at -1x368.
2. WrapperStructPose default parameters changed to match those of the OpenPose demo binary.
3. WrapperT.configure() changed from 1 function that requries all arguments to individual functions that take 1 argument each.
4. Added `Forward` to all net classes that automatically selects between CUDA, OpenCL, or CPU-only version depending on the defines.
5.Previously hardcoded `COCO_CHALLENGE` variable turned into user configurable flag `--maximize_positives`.
6.Removed old COCO 2014 validation scripts.
7.WrapperStructOutput split into WrapperStructOutput and WrapperStructGui.
8. Replaced `--camera_fps` flag by `--write_video_fps`, given that it was a confusing name: It did not affect the webcam FPS, but only the FPS of the output video. In addition, default value changed from 30 to -1.
9. Renamed `--frame_keep_distortion` as `--frame_undistort`, which performs the opposite operation (the default value has been also changed to the opposite).
10. Renamed `--camera_parameter_folder` as `--camera_parameter_path` because it could also take a whole XML file path rather than its parent folder.
11. Default value of flag `--scale_gap` changed from 0.3 to 0.25.
12. Moved most sh scripts into the `scripts/` folder. Only models/getModels.sh and the `*.bat` files are kept under `models/` and `3rdparty/windows`.
13. For Python compatibility and scalability increase, template `TDatums` used for `include/openpose/wrapper/wrapper.hpp` has changed from `std::vector<Datum>` to `std::vector<std::shared_ptr<Datum>>`, including the respective changes in all the worker classes. In addition, some template classes have been simplified to only take 1 template parameter for user simplicity.
14. Renamed intRound, charRound, etc. by positiveIntRound, positiveCharRound, etc. so that people can realize it is not safe for negative numbers.
15. Flag `--hand_tracking` is a subcase of `--hand_detector`, so it has been removed and incorporated as `--hand_detector 3`.
5.Removed old COCO 2014 validation scripts.
6.WrapperStructOutput split into WrapperStructOutput and WrapperStructGui.
7.Replaced flags:
1. Replaced `--camera_fps` flag by `--write_video_fps`, given that it was a confusing name: It did not affect the webcam FPS, but only the FPS of the output video. In addition, default value changed from 30 to -1.
2. Flag `--hand_tracking` is a subcase of `--hand_detector`, so it has been removed and incorporated as `--hand_detector 3`.
8. Renamed `--frame_keep_distortion` as `--frame_undistort`, which performs the opposite operation (the default value has been also changed to the opposite).
9. Renamed `--camera_parameter_folder` as `--camera_parameter_path` because it could also take a whole XML file path rather than its parent folder.
10. Default value of flag `--scale_gap` changed from 0.3 to 0.25.
11. Moved most sh scripts into the `scripts/` folder. Only models/getModels.sh and the `*.bat` files are kept under `models/` and `3rdparty/windows`.
12. For Python compatibility and scalability increase, template `TDatums` used for `include/openpose/wrapper/wrapper.hpp` has changed from `std::vector<Datum>` to `std::vector<std::shared_ptr<Datum>>`, including the respective changes in all the worker classes. In addition, some template classes have been simplified to only take 1 template parameter for user simplicity.
13. Renamed intRound, charRound, etc. by positiveIntRound, positiveCharRound, etc. so that people can realize it is not safe for negative numbers.
3. Main bugs fixed:
1. CMake-GUI was forcing to Release mode, allowed Debug modes too.
2. NMS returns in index 0 the number of found peaks. However, while the number of peaks was truncated to a maximum of 127, this index 0 was saving the real number instead of the truncated one.
# Change these variables to point to the correct folder (Release/x64 etc.)
sys.path.append('../../python');
# If you run `make install` (default path is `/usr/local/python` for Ubuntu), you can also access the OpenPose/python module from there. This will install OpenPose and the python library at your desired installation path. Ensure that this is in your python path in order to use it.
# sys.path.append('/usr/local/python')
fromopenposeimportpyopenposeasop
exceptImportErrorase:
print('Error: OpenPose library could not be found. Did you enable `BUILD_PYTHON` in CMake and have this Python script in the right folder?')
raisee
# Flags
parser=argparse.ArgumentParser()
parser.add_argument("--image_path",default="../../../examples/media/COCO_val2014_000000000192.jpg",help="Process an image. Read all standard formats (jpg, png, bmp, etc.).")
args=parser.parse_known_args()
# Custom Params (refer to include/openpose/flags.hpp for more parameters)
# Change these variables to point to the correct folder (Release/x64 etc.)
sys.path.append('../../python');
# If you run `make install` (default path is `/usr/local/python` for Ubuntu), you can also access the OpenPose/python module from there. This will install OpenPose and the python library at your desired installation path. Ensure that this is in your python path in order to use it.
# sys.path.append('/usr/local/python')
fromopenposeimportpyopenposeasop
exceptImportErrorase:
print('Error: OpenPose library could not be found. Did you enable `BUILD_PYTHON` in CMake and have this Python script in the right folder?')
raisee
# Flags
parser=argparse.ArgumentParser()
parser.add_argument("--image_dir",default="../../../examples/media/",help="Process a directory of images. Read all standard formats (jpg, png, bmp, etc.).")
parser.add_argument("--no_display",default=False,help="Enable to disable the visual display.")
args=parser.parse_known_args()
# Custom Params (refer to include/openpose/flags.hpp for more parameters)
# Change these variables to point to the correct folder (Release/x64 etc.)
sys.path.append('../../python');
# If you run `make install` (default path is `/usr/local/python` for Ubuntu), you can also access the OpenPose/python module from there. This will install OpenPose and the python library at your desired installation path. Ensure that this is in your python path in order to use it.
# sys.path.append('/usr/local/python')
fromopenposeimportpyopenposeasop
exceptImportErrorase:
print('Error: OpenPose library could not be found. Did you enable `BUILD_PYTHON` in CMake and have this Python script in the right folder?')
raisee
# Flags
parser=argparse.ArgumentParser()
parser.add_argument("--image_dir",default="../../../examples/media/",help="Process a directory of images. Read all standard formats (jpg, png, bmp, etc.).")
parser.add_argument("--no_display",default=False,help="Enable to disable the visual display.")
args=parser.parse_known_args()
# Custom Params (refer to include/openpose/flags.hpp for more parameters)
