提交 3a7cd1db 编写于 作者: G Gines Hidalgo

Deprecated non-API examples and further simplified readme/doc

Signed-off-by: NGines Hidalgo <gineshidalgo99@gmail.com>
上级 f20d5ee0
......@@ -13,53 +13,31 @@
Note: Currently using [travis-matrix-badges](https://github.com/bjfish/travis-matrix-badges) vs. traditional [![Build Status](https://travis-ci.org/CMU-Perceptual-Computing-Lab/openpose.svg?branch=master)](https://travis-ci.org/CMU-Perceptual-Computing-Lab/openpose)
-->
[**OpenPose**](https://github.com/CMU-Perceptual-Computing-Lab/openpose) represents the **first real-time multi-person system to jointly detect human body, hand, facial, and foot keypoints (in total 135 keypoints) on single images**.
[**OpenPose**](https://github.com/CMU-Perceptual-Computing-Lab/openpose) has represented the **first real-time multi-person system to jointly detect human body, hand, facial, and foot keypoints (in total 135 keypoints) on single images**, and it is currently being actively used in many projects and research areas!
It is **authored by [Gines Hidalgo](https://www.gineshidalgo.com), [Zhe Cao](https://people.eecs.berkeley.edu/~zhecao), [Tomas Simon](http://www.cs.cmu.edu/~tsimon), [Shih-En Wei](https://scholar.google.com/citations?user=sFQD3k4AAAAJ&hl=en), [Hanbyul Joo](https://jhugestar.github.io), and [Yaser Sheikh](http://www.cs.cmu.edu/~yaser)**. Currently, it is being **maintained by [Gines Hidalgo](https://www.gineshidalgo.com) and [Yaadhav Raaj](https://www.raaj.tech)**. In addition, OpenPose would not be possible without the [**CMU Panoptic Studio dataset**](http://domedb.perception.cs.cmu.edu). We would also like to thank all the people who helped OpenPose in any way. The main contributors are listed in [doc/contributors.md](doc/contributors.md).
It is **authored by [Gines Hidalgo](https://www.gineshidalgo.com), [Zhe Cao](https://people.eecs.berkeley.edu/~zhecao), [Tomas Simon](http://www.cs.cmu.edu/~tsimon), [Shih-En Wei](https://scholar.google.com/citations?user=sFQD3k4AAAAJ&hl=en), [Hanbyul Joo](https://jhugestar.github.io), and [Yaser Sheikh](http://www.cs.cmu.edu/~yaser)**, and **maintained by [Gines Hidalgo](https://www.gineshidalgo.com) and [Yaadhav Raaj](https://www.raaj.tech)**. OpenPose would not be possible without the [**CMU Panoptic Studio dataset**](http://domedb.perception.cs.cmu.edu). We would also like to thank all the people who helped OpenPose in any way ([doc/contributors.md](doc/contributors.md)).
<!-- The [original CVPR 2017 repo](https://github.com/ZheC/Multi-Person-Pose-Estimation) includes Matlab and Python versions, as well as the training code. The body pose estimation work is based on [the original ECCV 2016 demo](https://github.com/CMU-Perceptual-Computing-Lab/caffe_rtpose). -->
<p align="center">
<img src=".github/media/pose_face_hands.gif", width="480">
<br>
<sup>Authors <a href="https://www.gineshidalgo.com" target="_blank">Gines Hidalgo</a> (left) and <a href="https://jhugestar.github.io" target="_blank">Hanbyul Joo</a> (right) in front of the <a href="http://domedb.perception.cs.cmu.edu" target="_blank">CMU Panoptic Studio</a></sup>
</p>
## Features
- **Functionality**:
- **2D real-time multi-person keypoint detection**:
- 15, 18 or **25-keypoint body/foot keypoint estimation**. **Running time invariant to number of detected people**.
- **6-keypoint foot keypoint estimation**. Integrated together with the 25-keypoint body/foot keypoint detector.
- **2x21-keypoint hand keypoint estimation**. Currently, **running time depends** on **number of detected people**.
- **70-keypoint face keypoint estimation**. Currently, **running time depends** on **number of detected people**.
- **3D real-time single-person keypoint detection**:
- 3-D triangulation from multiple single views.
- Synchronization of Flir cameras handled.
- Compatible with Flir/Point Grey cameras, but provided C++ demos to add your custom input.
- **Calibration toolbox**:
- Easy estimation of distortion, intrinsic, and extrinsic camera parameters.
- **Single-person tracking** for further speed up or visual smoothing.
- **Input**: Image, video, webcam, Flir/Point Grey and IP camera. Included C++ demos to add your custom input.
- **Output**: Basic image + keypoint display/saving (PNG, JPG, AVI, ...), keypoint saving (JSON, XML, YML, ...), and/or keypoints as array class.
- **OS**: Ubuntu (20, 18, 16, 14), Windows (10, 8), Mac OSX, Nvidia TX2.
- **Training and datasets**:
- [**OpenPose Training**](https://github.com/CMU-Perceptual-Computing-Lab/openpose_train).
- [**Foot dataset website**](https://cmu-perceptual-computing-lab.github.io/foot_keypoint_dataset/).
- **Others**:
- Available: command-line demo, C++ wrapper, and C++ API.
- [**Python API**](doc/python_module.md).
- [**Unity Plugin**](https://github.com/CMU-Perceptual-Computing-Lab/openpose_unity_plugin).
- CUDA (Nvidia GPU), OpenCL (AMD GPU), and CPU-only (no GPU) versions.
For further details, check [all released features](doc/released_features.md) and [release notes](doc/release_notes.md).
## Related Work
- Since Sep 2019: [**Training code**](https://github.com/CMU-Perceptual-Computing-Lab/openpose_train)!
- Since Jan 2019: [**Unity plugin**](https://github.com/CMU-Perceptual-Computing-Lab/openpose_unity_plugin)!
- Since Dec 2018: [**Foot dataset**](https://cmu-perceptual-computing-lab.github.io/foot_keypoint_dataset) and [**new paper released**](https://arxiv.org/abs/1812.08008)!
## Contents
1. [Results](#results)
2. [Features](#features)
3. [Related Work](#related-work)
4. [Installation](#installation)
5. [Quick Start](#quick-start)
6. [Send Us Feedback!](#send-us-feedback)
7. [Citation](#citation)
8. [License](#license)
......@@ -71,7 +49,7 @@ For further details, check [all released features](doc/released_features.md) and
<sup>Testing the <a href="https://www.youtube.com/watch?v=2DiQUX11YaY" target="_blank"><i>Crazy Uptown Funk flashmob in Sydney</i></a> video sequence with OpenPose</sup>
</p>
### 3-D Reconstruction Module (Body, Foot, Face, and Hands)
### 3D Reconstruction Module (Body, Foot, Face, and Hands)
<p align="center">
<img src=".github/media/openpose3d.gif", width="360">
<br>
......@@ -96,94 +74,98 @@ For further details, check [all released features](doc/released_features.md) and
</p>
### Runtime Analysis
Inference time comparison between the 3 available pose estimation libraries: OpenPose, Alpha-Pose (fast Pytorch version), and Mask R-CNN:
In the following image, we show an inference time comparison between the 3 available pose estimation libraries: OpenPose, Alpha-Pose (fast Pytorch version), and Mask R-CNN. The OpenPose runtime is constant, while the runtime of Alpha-Pose and Mask R-CNN grow linearly with the number of people. The analysis was performed using the same hardware (Nvidia 1080 Ti) and number of images per algorithm. More details [**here**](https://arxiv.org/abs/1812.08008).
<p align="center">
<img src=".github/media/openpose_vs_competition.png", width="360">
</p>
This analysis was performed using the same images for each algorithm and a batch size of 1. Each analysis was repeated 1000 times and then averaged. This was all performed on a system with a Nvidia 1080 Ti and CUDA 8. Megvii (Face++) and MSRA GitHub repositories were excluded because they only provide pose estimation results given a cropped person. However, they suffer the same problem than Alpha-Pose and Mask R-CNN, their runtimes grow linearly with the number of people.
## Contents
1. [Features](#features)
2. [Related Work](#related-work)
3. [Results](#results)
4. [Installation, Reinstallation and Uninstallation](#installation-reinstallation-and-uninstallation)
5. [Quick Start](#quick-start)
6. [Output](#output)
7. [Speeding Up OpenPose and Benchmark](#speeding-up-openpose-and-benchmark)
8. [Training Code and Foot Dataset](#training-code-and-foot-dataset)
9. [Send Us Failure Cases and Feedback!](#send-us-failure-cases-and-feedback)
10. [Citation](#citation)
11. [License](#license)
## Features
- **Main Functionality**:
- **2D real-time multi-person keypoint detection**:
- 15, 18 or **25-keypoint body/foot keypoint estimation**, including **6 foot keypoints**. **Runtime invariant to number of detected people**.
- **2x21-keypoint hand keypoint estimation**. **Runtime depends on number of detected people**. See [**OpenPose Training**](https://github.com/CMU-Perceptual-Computing-Lab/openpose_train) for a runtime invariant alternative.
- **70-keypoint face keypoint estimation**. **Runtime depends on number of detected people**. See [**OpenPose Training**](https://github.com/CMU-Perceptual-Computing-Lab/openpose_train) for a runtime invariant alternative.
- [**3D real-time single-person keypoint detection**](doc/advanced/3d_reconstruction_module.md):
- 3D triangulation from multiple single views.
- Synchronization of Flir cameras handled.
- Compatible with Flir/Point Grey cameras.
- [**Calibration toolbox**](doc/advanced/calibration_module.md): Estimation of distortion, intrinsic, and extrinsic camera parameters.
- **Single-person tracking** for further speedup or visual smoothing.
- **Input**: Image, video, webcam, Flir/Point Grey and IP camera.
- **Output**: Basic image + keypoint display/saving (PNG, JPG, AVI, ...), keypoint saving (JSON, XML, YML, ...), and/or keypoints as array class.
- Included [C++](examples/tutorial_api_cpp/)/[python](examples/tutorial_api_python/) examples to add your custom input and/or output, as well as pre or post-processing steps.
- **OS**: Ubuntu (20, 18, 16, 14), Windows (10, 8), Mac OSX, Nvidia TX2.
- **Hardware compatibility**: CUDA (Nvidia GPU), OpenCL (AMD GPU), and non-GPU (CPU-only) versions.
- **Others**:
- Available: command-line demo, C++ wrapper, and C++ API.
- [**Python API**](doc/python_module.md).
For further details, check [all released features](doc/released_features.md) and [release notes](doc/release_notes.md).
## Related Work
- [**OpenPose training code**](https://github.com/CMU-Perceptual-Computing-Lab/openpose_train).
- [**OpenPose foot dataset**](https://cmu-perceptual-computing-lab.github.io/foot_keypoint_dataset/).
- [**Unity Plugin**](https://github.com/CMU-Perceptual-Computing-Lab/openpose_unity_plugin).
- OpenPose papers published in [**IEEE TPAMI** and **CVPR**](#citation). [Cite them](#citation) in your publications if OpenPose helps your research!
## Installation, Reinstallation and Uninstallation
**Windows portable version**: Simply download and use the latest version from the [Releases](https://github.com/CMU-Perceptual-Computing-Lab/openpose/releases) section.
## Installation
If you want to use OpenPose without compiling or writing any code, simply [download and use the latest Windows portable version of OpenPose](doc/installation/README.md#windows-portable-demo))! Otherwise, you can also [build OpenPose from source](doc/installation/README.md#compiling-and-running-openpose-from-source-on-windows-ubuntu-and-mac).
Otherwise, check [doc/installation/installation.md](doc/installation/installation.md) for instructions on how to build OpenPose from source.
See [doc/installation/README.md](doc/installation/README.md) for more details.
## Quick Start
Most users do not need the OpenPose C++/Python API, but can simply use the OpenPose Demo:
- **OpenPose Demo**: To easily process images/video/webcam and display/save the results. See [doc/quick_start.md](./doc/quick_start.md) and [doc/demo_overview.md](./doc/demo_overview.md). E.g., run OpenPose in a video with:
- [**Output information**](doc/output.md): Learn about the output format, keypoint index ordering, etc.
- **OpenPose Demo**: Choose your input (e.g., images, video, webcam), set of algorithms (body, hand, face), and output (e.g., display, JSON keypoint saving, image+keypoints). See [doc/quick_start.md](doc/quick_start.md) and [doc/demo_overview.md](doc/demo_overview.md).
- E.g., in your terminal or PowerShell: Given an input video (`--video`), extract body (by default), face (`--face`) and hand (`--hand`) keypoints, save the keypoints in a JSON file (`--write_json`), and display (by default) the results in the screen. Remove any of the flags to remove that particular functionality.
```
# Ubuntu
./build/examples/openpose/openpose.bin --video examples/media/video.avi
./build/examples/openpose/openpose.bin --video examples/media/video.avi --face --hand --write_json output_json_folder/
:: Windows - Portable Demo
bin\OpenPoseDemo.exe --video examples\media\video.avi
bin\OpenPoseDemo.exe --video examples\media\video.avi --face --hand --write_json output_json_folder/
```
- **OpenPose C++ API**: If you want to read a specific input, and/or add your custom post-processing function, and/or implement your own display/saving, check the C++ API tutorial on [examples/tutorial_api_cpp/](examples/tutorial_api_cpp/).
- [**OpenPose C++ API**](examples/tutorial_api_cpp/): If you want to read a specific input, and/or add your custom post-processing function, and/or implement your own display/saving.
- For quick prototyping: You can easily **create your custom code** on [examples/user_code/](examples/user_code/) and CMake will automatically compile it together with the whole OpenPose project. See [examples/user_code/README.md](examples/user_code/README.md) for more details.
- **OpenPose Python API**: Analogously to the C++ API, find the tutorial for the Python API on [examples/tutorial_api_python/](examples/tutorial_api_python/).
- For quick prototyping: You can simply duplicate any of the [examples/tutorial_api_python/](examples/tutorial_api_python/) files and rename it within the same folder.
- [**OpenPose Python API**](examples/tutorial_api_python/): Almost the exact same functionality than the C++ API, but in Python!
- For quick prototyping: You can simply duplicate and rename any of the [existing example files](examples/tutorial_api_python/) within that same folder.
- [**Speeding up OpenPose and benchmark**](doc/speed_up_openpose.md): Check the OpenPose Benchmark as well as some hints to speed up and/or reduce the memory requirements for OpenPose.
- **Calibration toolbox**: To easily calibrate your cameras for 3-D OpenPose or any other stereo vision task. See [doc/advanced/calibration_module.md](doc/advanced/calibration_module.md).
- [**Calibration toolbox**](doc/advanced/calibration_module.md) and [**3D OpenPose**](doc/advanced/3d_reconstruction_module.md): Calibrate your cameras for 3D OpenPose (or any other stereo vision tasks) and start obtaining 3D keypoints!
- **Standalone face or hand detector**. See [doc/advanced/standalone_face_or_hand_keypoint_detector.md](doc/advanced/standalone_face_or_hand_keypoint_detector.md) if you want to do any of the following:
- [**Standalone face or hand detector**doc/advanced/standalone_face_or_hand_keypoint_detector.md](doc/advanced/standalone_face_or_hand_keypoint_detector.md) is useful if you want to do any of the following:
- **Face** keypoint detection **without body** keypoint detection: Pros: Speedup and RAM/GPU memory reduction. Cons: Worse accuracy and less detected number of faces).
- **Use your own face/hand detector**: You can use the hand and/or face keypoint detectors with your own face or hand detectors, rather than using the body detector. E.g., useful for camera views at which the hands are visible but not the body (OpenPose detector would fail).
## Output
Output (format, keypoint index ordering, etc.) in [doc/output.md](doc/output.md).
## Speeding Up OpenPose and Benchmark
Check the OpenPose Benchmark as well as some hints to speed up and/or reduce the memory requirements for OpenPose on [doc/speed_up_openpose.md](doc/speed_up_openpose.md).
## Training Code and Foot Dataset
For training OpenPose, check [github.com/CMU-Perceptual-Computing-Lab/openpose_train](https://github.com/CMU-Perceptual-Computing-Lab/openpose_train).
For the foot dataset, check the [foot dataset website](https://cmu-perceptual-computing-lab.github.io/foot_keypoint_dataset/) and new [OpenPose paper](https://arxiv.org/abs/1812.08008) for more information.
## Send Us Failure Cases and Feedback!
## Send Us Feedback!
Our library is open source for research purposes, and we want to continuously improve it! So please, let us know if...
1. ... you find any bug (in functionality or speed).
2. ... you added some functionality on top of OpenPose which we might want to add.
3. ... you know how to speed up or improve any part of OpenPose.
4. ... you want to share your cool demo or project made on top of OpenPose (you can email it to us too!).
5. ... etc.
1. ... you find videos or images where OpenPose does not seems to work well. Feel free to send them to openposecmu@gmail.com (email only for failure cases!), we will use them to improve the quality of the algorithm!
2. ... you find any bug (in functionality or speed).
3. ... you added some functionality to some class or some new Worker<T> subclass which we might potentially incorporate.
4. ... you know how to speed up or improve any part of the library.
5. ... you have a request about possible functionality.
6. ... etc.
Just comment on GitHub or make a pull request and we will answer as soon as possible! Send us an email if you use the library to make a cool demo or YouTube video!
Just create a new GitHub issue or a pull request and we will answer as soon as possible!
## Citation
Please cite these papers in your publications if it helps your research. Most of OpenPose is based on `[8765346]`. In addition, the hand and face keypoint detectors are a combination of `[8765346]` and `[Simon et al. 2017]` (the face detector was trained using the same procedure than the hand detector).
Please cite these papers in your publications if it helps your research. All of OpenPose is based on [OpenPose: Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields](https://arxiv.org/abs/1812.08008), while the hand and face detectors also use [Hand Keypoint Detection in Single Images using Multiview Bootstrapping](https://arxiv.org/abs/1704.07809) (the face detector was trained using the same procedure than the hand detector).
@article{8765346,
author = {Z. {Cao} and G. {Hidalgo Martinez} and T. {Simon} and S. {Wei} and Y. A. {Sheikh}},
......@@ -213,8 +195,7 @@ Please cite these papers in your publications if it helps your research. Most of
year = {2016}
}
Links to the papers:
Paper links:
- OpenPose: Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields:
- [IEEE TPAMI](https://ieeexplore.ieee.org/document/8765346)
- [ArXiv](https://arxiv.org/abs/1812.08008)
......
......@@ -23,13 +23,13 @@ This module performs 3-D keypoint (body, face, and hand) reconstruction and rend
## Installation
Check [doc/installation/installation.md#3d-reconstruction-module](../installation/installation.md#3d-reconstruction-module) for installation steps.
Check [doc/installation/README.md#3d-reconstruction-module](../installation/README.md#3d-reconstruction-module) for installation steps.
## Non Linear Optimization
In order to increase the 3-D reconstruction accuracy, OpenPose optionally performs non-linear optimization if Ceres solver support is enabled (only available in Ubuntu for now). To enable it, check [doc/installation/installation.md#3d-reconstruction-module](../installation/installation.md#3d-reconstruction-module) for more details.
In order to increase the 3-D reconstruction accuracy, OpenPose optionally performs non-linear optimization if Ceres solver support is enabled (only available in Ubuntu for now). To enable it, check [doc/installation/README.md#3d-reconstruction-module](../installation/README.md#3d-reconstruction-module) for more details.
......@@ -69,7 +69,7 @@ This demo assumes n arbitrary stereo cameras from the FLIR company (formerly Poi
The user must manually get the intrinsic and extrinsic parameters of the stereo-cameras. Note, we will assume `Flir` cameras, which is specified by default with the flag `--camera_parameter_path "models/cameraParameters/flir/"`. Otherwise, change the path to your camera name accordingly.
There are 2 alternatives to calibrate the cameras:
1. Using the OpenPose calibration toolbox, [doc/advanced/calibration_module.md](./calibration_module.md).
1. Using the OpenPose calibration toolbox, [doc/advanced/calibration_module.md](calibration_module.md).
2. Using your own calibration toolbox (or if you already know the camera parameters of your cameras):
1. Create a xml file for each camera named as `models/cameraParameters/flir/{camera_serial_number}.xml`.
2. The elements inside each xml file are the extrinsic parameters of the camera (`CameraMatrix`), the intrinsic parameters (`Intrinsics`), and the distortion coefficients (`Distortion`). Copy the format from `models/cameraParameters/flir/17012332.xml.example`. For the extrinsic parameters of the camera, it allows you to set the coordinate origin (so that 3-d keypoints are distances with respect to that origin).
......
......@@ -25,13 +25,13 @@ Note: We are not aiming to have the best calibration toolbox, but the simplest o
## Example Chessboard
[doc/Chessboard_in_PDF/](./Chessboard_in_PDF/) contains a chessboard example that you can use to follow this documentation. You can simply print the PDF version ([doc/Chessboard_in_PDF/pattern.pdf](./Chessboard_in_PDF/pattern.pdf)), making your your printer maintains the aspect ratio and, if possible, does not zoom in/out the image. Simply replace the "127" used below by the size of each square on your printed chessboard.
[doc/Chessboard_in_PDF/](Chessboard_in_PDF/) contains a chessboard example that you can use to follow this documentation. You can simply print the PDF version ([doc/Chessboard_in_PDF/pattern.pdf](Chessboard_in_PDF/pattern.pdf)), making your your printer maintains the aspect ratio and, if possible, does not zoom in/out the image. Simply replace the "127" used below by the size of each square on your printed chessboard.
## Installing the Calibration Module
Check [doc/installation/installation.md#calibration-module](../installation/installation.md#calibration-module) for installation steps.
Check [doc/installation/README.md#calibration-module](../installation/README.md#calibration-module) for installation steps.
......
......@@ -18,7 +18,7 @@ Starting in OpenPose 1.6.0 (GitHub code in or after October 2019), OpenPose has
### Third-Party Libraries
While compiling OpenPose from source, the static library files (`*.a` for Ubuntu, `*.lib` for Windows, etc.) and `include/` directories of all the third-party libraries detailed in [doc/installation/installation.md](../installation/installation.md) are required (GFlags, Glog, OpenCV, Caffe, etc.). However, when deploying OpenPose, fewer dependencies are required:
While compiling OpenPose from source, the static library files (`*.a` for Ubuntu, `*.lib` for Windows, etc.) and `include/` directories of all the third-party libraries detailed in [doc/installation/README.md](../installation/README.md) are required (GFlags, Glog, OpenCV, Caffe, etc.). However, when deploying OpenPose, fewer dependencies are required:
- GFLags and Glog are required only if the `include/openpose/flags.hpp` file is going to be used (e.g., when intenting to use the command-line interface).
- OpenCV can be optionally included if your project already uses it (but make sure to use the same binaries and include directory of OpenCV for both OpenPose and your project or weird runtime crashes will occur!). Including OpenCV does not increase the functionality of OpenPose, but it makes it easier to use by adding some functions that directly take cv::Mat matrices as input (rather than raw pointers). However, it is optional starting in OpenPose 1.6.0.
- Caffe or any other 3rd-party libraries are not required.
......
......@@ -7,7 +7,7 @@ Even if you want to e.g., change internal functions and/or extend the OpenPose f
In order to learn the basics about how OpenPose works internally:
1. Take a look at the [library Quick Start section](../../README.md#quick-start) from the main README.
2. OpenPose Overview: Learn the basics about the library source code in [doc/advanced/library_structure/library_overview.md](./library_overview.md).
3. Extending Functionality: Learn how to extend the library in [doc/advanced/library_structure/library_extend_functionality.md](./library_extend_functionality.md).
4. Adding An Extra Module: Learn how to add an extra module in [doc/advanced/library_structure/library_add_new_module.md](./library_add_new_module.md).
2. OpenPose Overview: Learn the basics about the library source code in [doc/advanced/library_structure/library_overview.md](library_overview.md).
3. Extending Functionality: Learn how to extend the library in [doc/advanced/library_structure/library_extend_functionality.md](library_extend_functionality.md).
4. Adding An Extra Module: Learn how to add an extra module in [doc/advanced/library_structure/library_add_new_module.md](library_add_new_module.md).
5. See the Doxygen documentation on [http://cmu-perceptual-computing-lab.github.io/openpose/html/index.html](http://cmu-perceptual-computing-lab.github.io/openpose/html/index.html) or build it from the source code.
......@@ -11,7 +11,7 @@ If you intend to extend the functionality of our library:
4. Read, understand and play with the other tutorials in [examples/](../../../examples/). It includes more specific examples.
5. Check the basic UML diagram on the [doc/advanced/library_structure/UML](./UML/) to get an idea of each module relations.
5. Check the basic UML diagram on the [doc/advanced/library_structure/UML](UML/) to get an idea of each module relations.
6. Take a look to the stucuture of the already existing modules.
......
......@@ -3,7 +3,7 @@ OpenPose C++ API - Basic Overview
Note: Read [doc/advanced/library_structure/README.md](./README.md) before this page.
Note: Read [doc/advanced/library_structure/README.md](README.md) before this page.
......@@ -15,7 +15,7 @@ Note: Read [doc/advanced/library_structure/README.md](./README.md) before this p
## Main Modules
In order to use and/or slightly extend the OpenPose library, we try to explain the 2 main components on this section. [doc/advanced/library_structure/UML](./UML/) contains the class diagram of all these modules.
In order to use and/or slightly extend the OpenPose library, we try to explain the 2 main components on this section. [doc/advanced/library_structure/UML](UML/) contains the class diagram of all these modules.
1. The basic module: `core`.
......
......@@ -11,17 +11,17 @@ In Ubuntu, Mac, and other Unix systems, use any command-line interface, such as
## Running on Images, Video or Webcam
See [doc/quick_start.md#quick-start](./quick_start.md#quick-start).
See [doc/quick_start.md#quick-start](quick_start.md#quick-start).
## Pose + Face + Hands
See [doc/quick_start.md#quick-start](./quick_start.md#quick-start).
See [doc/quick_start.md#quick-start](quick_start.md#quick-start).
## Maximum Accuracy Configuration
See [doc/quick_start.md#maximum-accuracy-configuration](./quick_start.md#maximum-accuracy-configuration).
See [doc/quick_start.md#maximum-accuracy-configuration](quick_start.md#maximum-accuracy-configuration).
......@@ -29,7 +29,7 @@ See [doc/quick_start.md#maximum-accuracy-configuration](./quick_start.md#maximum
In general, there are 3 ways to reduce the latency (with some drawbacks each one):
- Reducing `--output_resolution`: It will slightly reduce the latency and increase the FPS. But the quality of the displayed image will deteriorate.
- Reducing `--net_resolution` and/or `--face_net_resolution` and/or `--hand_net_resolution`: It will increase the FPS and reduce the latency. But the accuracy will drop, specially for small people in the image. Note: For maximum accuracy, follow [doc/quick_start.md#maximum-accuracy-configuration](./quick_start.md#maximum-accuracy-configuration).
- Reducing `--net_resolution` and/or `--face_net_resolution` and/or `--hand_net_resolution`: It will increase the FPS and reduce the latency. But the accuracy will drop, specially for small people in the image. Note: For maximum accuracy, follow [doc/quick_start.md#maximum-accuracy-configuration](quick_start.md#maximum-accuracy-configuration).
- Enabling `--disable_multi_thread`: The latency should be reduced. But the speed will drop to 1-GPU speed (as it will only use 1 GPU). Note that it's practical only for body, if hands and face are also extracted, it's usually not worth it.
......@@ -40,7 +40,7 @@ Since the Windows 10 Anniversary, Kinect 2.0 can be read as a normal webcam. All
## JSON Output with No Visualization
The following example runs the demo video `video.avi` and outputs JSON files in `output/`. Note: see [doc/output.md](./output.md) to understand the format of the JSON files.
The following example runs the demo video `video.avi` and outputs JSON files in `output/`. Note: see [doc/output.md](output.md) to understand the format of the JSON files.
```
# Only body
./build/examples/openpose/openpose.bin --video examples/media/video.avi --write_json output/ --display 0 --render_pose 0
......@@ -51,7 +51,7 @@ The following example runs the demo video `video.avi` and outputs JSON files in
## JSON Output + Rendered Images Saving
The following example runs the demo video `video.avi`, renders image frames on `output/result.avi`, and outputs JSON files in `output/`. Note: see [doc/output.md](./output.md) to understand the format of the JSON files.
The following example runs the demo video `video.avi`, renders image frames on `output/result.avi`, and outputs JSON files in `output/`. Note: see [doc/output.md](output.md) to understand the format of the JSON files.
```
./build/examples/openpose/openpose.bin --video examples/media/video.avi --write_video output/result.avi --write_json output/
```
......
......@@ -42,7 +42,7 @@ OpenPose - Frequently Asked Question (FAQ)
### Errors
#### Download Server Down
**Q: The download server is down, Download Hash missmatch, Error 502: Bad Gateway, etc. I.e., I cannot download the OpenPose models and/or 3rd party libraries.**
**A**: The are 2 alternatives. Option 1 (recommended): Download the links from [1602#issuecomment-641653411](https://github.com/CMU-Perceptual-Computing-Lab/openpose/issues/1602#issuecomment-641653411) and download them in the places indicated by [doc/prerequisites.md](./prerequisites.md). Option 2: If you downloaded the models and dependencies to some local server, you could set the advanced CMake property `DOWNLOAD_SERVER` to link OpenPose to your local mirror server instead.
**A**: The are 2 alternatives. Option 1 (recommended): Download the links from [1602#issuecomment-641653411](https://github.com/CMU-Perceptual-Computing-Lab/openpose/issues/1602#issuecomment-641653411) and download them in the places indicated by [doc/prerequisites.md](prerequisites.md). Option 2: If you downloaded the models and dependencies to some local server, you could set the advanced CMake property `DOWNLOAD_SERVER` to link OpenPose to your local mirror server instead.
......@@ -67,7 +67,7 @@ Also, hands and face increases the GPU memory requeriments, and 4 GB GPUs might
#### Cannot Find OpenPose.dll Error (Windows)
**Q: System cannot find the file specified (Openpose.dll) error when trying to release** - Using a folder with images does work, but the video and/or the webcam do not. Note: often on Windows.
**A**: Visual Studio (VS) and the [doc/installation/installation.md](./installation/installation.md) section is only intended if you plan to modify the OpenPose code or integrate it with another library or project. If you just want to use the OpenPose demo, simply follow [doc/installation/installation.md#windows-portable-demo](./installation/installation.md#windows-portable-demo) and download the OpenPose binaries in the [Releases](https://github.com/CMU-Perceptual-Computing-Lab/openpose/releases) section.
**A**: Visual Studio (VS) and the [doc/installation/README.md](installation/README.md) section is only intended if you plan to modify the OpenPose code or integrate it with another library or project. If you just want to use the OpenPose demo, simply follow [doc/installation/README.md#windows-portable-demo](installation/README.md#windows-portable-demo) and download the OpenPose binaries in the [Releases](https://github.com/CMU-Perceptual-Computing-Lab/openpose/releases) section.
If you need to compile it with Visual Studio (VS), then keep reading. In this error, VS is simply saying that there were errors while compiling the OpenPose library. Try compiling only the OpenPose library (not the demo), by right clicking on it, then `Set as StartUp Project`, and finally right click + `Build`. Then, at the bottom left part of VS, press `Error list` and then you should see which errors VS encountered while compiling. In that way, VS gives you the exact error so you can know it and share the exact issue.
......@@ -96,12 +96,12 @@ Note: OpenPose library is not an executable, but a library. So instead clicking
**A**: This answer assumes that never a single person is detected. If in your case it works sometimes, then check [Always Zero People Detected](#always-zero-people-detected). This always-0-people problem usually occurs in 2 situations: 1) When you selection `--num_gpu 0`, and 2) when the caffemodel has not been properly downloaded. E.g., if the connection drops when downloading the models.
For problem 1, setting `--num_gpu 0` means that no processing is done, so you can use this setting e.g., to record webcam. This functionality is kept for back-compatibility. You are most probably trying to run on CPU-only mode, for that, install OpenPose in CPU-only mode following [doc/installation/installation.md](./installation/installation.md).
For problem 1, setting `--num_gpu 0` means that no processing is done, so you can use this setting e.g., to record webcam. This functionality is kept for back-compatibility. You are most probably trying to run on CPU-only mode, for that, install OpenPose in CPU-only mode following [doc/installation/README.md](installation/README.md).
For problem 2, try the following solutions (in this order):
1. Assuming that default OpenPose (i.e., BODY_25 model) failed, try with `--model_pose COCO` and `--model_pose MPII` models. If any of them work, the `caffemodel` files of the other models were corrupted while being downloaded. Otherwise, it will most probably be a Caffe/protobuf issue.
2. Assuming that the model is corrupted, remove the current models in the model folder, and download them manually from the links in [doc/installation/installation.md](./installation/installation.md). Alternatively, remove them and re-run Cmake again. If this does not work, try downloading the COCO_25 model from the browser following the download link on this [Dropbox link](https://www.dropbox.com/s/03r8pa8sikrqv62/pose_iter_584000.caffemodel).
2. Assuming that the model is corrupted, remove the current models in the model folder, and download them manually from the links in [doc/installation/README.md](installation/README.md). Alternatively, remove them and re-run Cmake again. If this does not work, try downloading the COCO_25 model from the browser following the download link on this [Dropbox link](https://www.dropbox.com/s/03r8pa8sikrqv62/pose_iter_584000.caffemodel).
3. If none of the OpenPose models are working, make sure Caffe is working properly and that you can run the Caffe examples with other caffemodel / prototxt files.
......@@ -109,7 +109,7 @@ For problem 2, try the following solutions (in this order):
#### Very Few People Detected
**Q: Low detection rate. It can detect the person on some images (usually higher contrast, with bigger people), but it will fail for most of images with low resolution or small people.**
**A**: Images with low resolution, or with people too tiny will simply not work too well. However, it can be highly improved by using the maximum accuracy configuration detailed in [doc/quick_start.md#maximum-accuracy-configuration](./quick_start.md#maximum-accuracy-configuration).
**A**: Images with low resolution, or with people too tiny will simply not work too well. However, it can be highly improved by using the maximum accuracy configuration detailed in [doc/quick_start.md#maximum-accuracy-configuration](quick_start.md#maximum-accuracy-configuration).
......@@ -136,7 +136,7 @@ F0821 14:26:29.665053 22812 upgrade_proto.cpp:97] Check failed: ReadProtoFromBin
**A**: This error has been solved in the latest OpenPose versions. Completely remove OpenPose and re-download the latest version (just cleaning the compilation or removing the `build/` folder will not work).
If you wanna use your custom Caffe and it has this error: This error only happens in some Ubuntu machines. Following #787, compile your own Caffe with an older version of it. The hacky (quick but not recommended way) is to follow [#787#issuecomment-415476837](https://github.com/CMU-Perceptual-Computing-Lab/openpose/issues/787#issuecomment-415476837), the elegant way (compatible with future OpenPose versions) is to build your own Caffe independently, following [doc/installation/installation.md#custom-caffe-ubuntu-only](./installation/installation.md#custom-caffe-ubuntu-only).
If you wanna use your custom Caffe and it has this error: This error only happens in some Ubuntu machines. Following #787, compile your own Caffe with an older version of it. The hacky (quick but not recommended way) is to follow [#787#issuecomment-415476837](https://github.com/CMU-Perceptual-Computing-Lab/openpose/issues/787#issuecomment-415476837), the elegant way (compatible with future OpenPose versions) is to build your own Caffe independently, following [doc/installation/README.md#custom-caffe-ubuntu-only](installation/README.md#custom-caffe-ubuntu-only).
......@@ -156,7 +156,7 @@ CUDA_cublas_device_LIBRARY (ADVANCED)
linked by target "caffe" in directory /home/jakebmalis/Documents/openpose/3rdparty/caffe/src/caffe
```
**A**: Make sure to download and install CMake-GUI following the [doc/prerequisites.md](./prerequisites.md) section. This is a known problem with CMake-GUI versions from 3.8 to 3.11 (unfortunately, default Ubuntu 18 CMake-GUI uses 3.10). You will need a CMake version >= 3.12.
**A**: Make sure to download and install CMake-GUI following the [doc/prerequisites.md](prerequisites.md) section. This is a known problem with CMake-GUI versions from 3.8 to 3.11 (unfortunately, default Ubuntu 18 CMake-GUI uses 3.10). You will need a CMake version >= 3.12.
......@@ -194,7 +194,7 @@ git submodule update
#### Obscure CMake Error about Caffe or Pybind
**Q:** There appear some weird and obscure errors on CMake about Caffe and/or Pybind.
**A**: Check [doc/installation/installation.md](./installation/installation.md) to run the `git submodule` command, i.e.,
**A**: Check [doc/installation/README.md](installation/README.md) to run the `git submodule` command, i.e.,
```
git submodule update --init --recursive --remote
```
......@@ -207,7 +207,7 @@ git submodule update --init --recursive --remote
#### Speed Up, Memory Reduction, and Benchmark
**Q: Low speed** - OpenPose is quite slow, is it normal? How can I speed it up?
**A**: Check [doc/speed_up_openpose.md](./speed_up_openpose.md) to discover the approximate speed of your graphics card and some speed tips.
**A**: Check [doc/speed_up_openpose.md](speed_up_openpose.md) to discover the approximate speed of your graphics card and some speed tips.
......@@ -221,19 +221,19 @@ git submodule update --init --recursive --remote
#### CPU Version Too Slow
**Q: The CPU version is insanely slow compared to the GPU version.**
**A**: Check [doc/speed_up_openpose.md#cpu-version](./speed_up_openpose.md#cpu-version) to discover the approximate speed and some speed tips.
**A**: Check [doc/speed_up_openpose.md#cpu-version](speed_up_openpose.md#cpu-version) to discover the approximate speed and some speed tips.
#### Profiling Speed and Estimating FPS without Display
Check the [doc/speed_up_openpose.md#profiling-speed](./speed_up_openpose.md#profiling-speed) section.
Check the [doc/speed_up_openpose.md#profiling-speed](speed_up_openpose.md#profiling-speed) section.
#### Webcam Slower than Images
**Q: Webcam is slow** - Using a folder with images matches the speed FPS benchmarks, but the webcam has lower FPS. Note: often on Windows.
**A**: OpenCV has some issues with some camera drivers (specially on Windows). The first step should be to compile OpenCV by your own and re-compile OpenPose after that (following the [doc/installation/installation.md#reinstallation](./installation/installation.md#reinstallation) section). If the speed is still slower, you can better debug it by running a webcam OpenCV example (e.g. [this C++ example](http://answers.opencv.org/question/1/how-can-i-get-frames-from-my-webcam/)). If you are able to get the proper FPS with the OpenCV demo but OpenPose is still low, then let us know!
**A**: OpenCV has some issues with some camera drivers (specially on Windows). The first step should be to compile OpenCV by your own and re-compile OpenPose after that (following the [doc/installation/README.md#reinstallation](installation/README.md#reinstallation) section). If the speed is still slower, you can better debug it by running a webcam OpenCV example (e.g. [this C++ example](http://answers.opencv.org/question/1/how-can-i-get-frames-from-my-webcam/)). If you are able to get the proper FPS with the OpenCV demo but OpenPose is still low, then let us know!
......@@ -258,7 +258,12 @@ Check the [doc/speed_up_openpose.md#profiling-speed](./speed_up_openpose.md#prof
#### Difference between BODY_25 vs. COCO vs. MPI
COCO model will eventually be removed. BODY_25 model is faster, more accurate, and it includes foot keypoints. However, COCO requires less memory on GPU (being able to fit into 2GB GPUs with the default settings) and it runs faster on CPU-only mode. MPI model is only meant for people requiring the MPI-keypoint structure. It is also slower than BODY_25 and far less accurate.
COCO model will eventually be removed. BODY_25 model is faster, more accurate, and it includes foot keypoints. However, `COCO` requires less memory on GPU (being able to fit into 2GB GPUs with the default settings) and it runs faster on CPU-only mode. `MPI` model is only meant for people requiring the MPI-keypoint structure. It is also slower than BODY_25 and far less accurate.
Key differences:
- `BODY_25` is the fastest one on the GPU, the most accurate one, and the one with the highest number of keypoints (including foot keypoints!), highly recommended. It also uses the most ammount of RAM/GPU memory.
- `COCO` is faster than `BODY_25` on the CPU but slower on the GPU and less accurate.
- `MPI` is the fastest on the CPU but the least accurate of all 3.
......
......@@ -15,28 +15,28 @@ OpenPose - Installation (deprecated)
## Operating Systems
See [doc/installation/installation.md#operating-systems](../installation.md#operating-systems).
See [doc/installation/README.md#operating-systems](../README.md#operating-systems).
## Requirements
See [doc/installation/installation.md#requirements](../installation.md#requirements).
See [doc/installation/README.md#requirements](../README.md#requirements).
## Clone OpenPose
See [doc/installation/installation.md#clone-openpose](../installation.md#clone-openpose).
See [doc/installation/README.md#clone-openpose](../README.md#clone-openpose).
## Update OpenPose
See [doc/installation/installation.md#update-openpose](../installation.md#update-openpose).
See [doc/installation/README.md#update-openpose](../README.md#update-openpose).
## Ubuntu
### Installation - CMake
Recommended installation method, it is simpler and offers more customization settings. See [doc/installation/installation.md](../installation.md).
Recommended installation method, it is simpler and offers more customization settings. See [doc/installation/README.md](../README.md).
......@@ -56,7 +56,7 @@ Build Caffe & the OpenPose library + download the required Caffe models for Ubun
```bash
bash scripts/ubuntu_deprecated/install_caffe_and_openpose_if_cuda8.sh
```
**Highly important**: This script only works with CUDA 8 and Ubuntu 14 or 16. Otherwise, see [doc/installation/installation.md](../installation.md) or [Installation - Manual Compilation](#installation---manual-compilation).
**Highly important**: This script only works with CUDA 8 and Ubuntu 14 or 16. Otherwise, see [doc/installation/README.md](../README.md) or [Installation - Manual Compilation](#installation---manual-compilation).
......@@ -126,7 +126,7 @@ You just need to remove the OpenPose folder, by default called `openpose/`. E.g.
3. [**cuDNN 5.1**](https://developer.nvidia.com/cudnn): Once you have downloaded it, just unzip it and copy (merge) the contents on the CUDA folder, `C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v8.0`.
#### CMake Installer
Recommended installation method, it is simpler and offers more customization settings. See [doc/installation/installation.md](../installation.md). Note that it is a beta version, post in GitHub any issue you find.
Recommended installation method, it is simpler and offers more customization settings. See [doc/installation/README.md](../README.md). Note that it is a beta version, post in GitHub any issue you find.
#### Deprecated Windows Installer
......@@ -174,7 +174,7 @@ If you updated some software that our library or 3rdparty use, or you simply wan
## Doxygen Documentation Autogeneration
See [doc/installation/installation.md#doxygen-documentation-autogeneration-ubuntu-only](../installation.md#doxygen-documentation-autogeneration-ubuntu-only).
See [doc/installation/README.md#doxygen-documentation-autogeneration-ubuntu-only](../README.md#doxygen-documentation-autogeneration-ubuntu-only).
......@@ -196,4 +196,4 @@ We only modified some Caffe compilation flags and minor details. You can use you
## Compiling without cuDNN
See [doc/installation/installation.md#compiling-without-cudnn](../installation.md#compiling-without-cudnn).
See [doc/installation/README.md#compiling-without-cudnn](../README.md#compiling-without-cudnn).
......@@ -10,16 +10,14 @@ OpenPose - Prerequisites
### General Tips
**Very important**: New Nvidia model GPUs (e.g., Nvidia V, GTX 2080, v100, any Nvidia with Volta or Turing architecture, etc.) require (at least) CUDA 10. CUDA 8 would fail!
In addition, CMake automatically downloads all the OpenPose models. However, **some firewall or company networks block these downloads**.
You might prefer to download them manually:
- [BODY_25 model](http://posefs1.perception.cs.cmu.edu/OpenPose/models/pose/body_25/pose_iter_584000.caffemodel): download in `models/pose/body_25/`.
- [COCO model](http://posefs1.perception.cs.cmu.edu/OpenPose/models/pose/coco/pose_iter_440000.caffemodel): download in `models/pose/coco/`.
- [MPI model](http://posefs1.perception.cs.cmu.edu/OpenPose/models/pose/mpi/pose_iter_160000.caffemodel): download in `models/pose/mpi/`.
- [Face model](http://posefs1.perception.cs.cmu.edu/OpenPose/models/face/pose_iter_116000.caffemodel): download in `models/face/`.
- [Hands model](http://posefs1.perception.cs.cmu.edu/OpenPose/models/hand/pose_iter_102000.caffemodel): download in `models/hand/`.
These tips are **very important** and avoid many bugs:
- Install the latest CUDA version or make sure your GPU is compatible with the CUDA version you have in your system. E.g., Nvidia 30XX GPUs require at least CUDA 11, others (GTX 20XX, V100, Volta or Turing GPUs) require at least CUDA 10.
- CMake automatically downloads all the OpenPose models. However, **some firewall or company networks block these downloads**. If so, you might need to download them manually:
- [BODY_25 model](http://posefs1.perception.cs.cmu.edu/OpenPose/models/pose/body_25/pose_iter_584000.caffemodel): download in `models/pose/body_25/`.
- [COCO model](http://posefs1.perception.cs.cmu.edu/OpenPose/models/pose/coco/pose_iter_440000.caffemodel): download in `models/pose/coco/`.
- [MPI model](http://posefs1.perception.cs.cmu.edu/OpenPose/models/pose/mpi/pose_iter_160000.caffemodel): download in `models/pose/mpi/`.
- [Face model](http://posefs1.perception.cs.cmu.edu/OpenPose/models/face/pose_iter_116000.caffemodel): download in `models/face/`.
- [Hands model](http://posefs1.perception.cs.cmu.edu/OpenPose/models/hand/pose_iter_102000.caffemodel): download in `models/hand/`.
......@@ -36,7 +34,7 @@ You might prefer to download them manually:
- Run `./configure --qt-gui`. Make sure no error occurred.
- Run ``./bootstrap && make -j`nproc` && sudo make install -j`nproc` ``. Make sure no error occurred.
- Assuming your CMake downloaded folder is in {CMAKE_FOLDER_PATH}, everytime these instructions mentions `cmake-gui`, you will have to replace that line by `{CMAKE_FOLDER_PATH}/bin/cmake-gui`.
- Ubuntu 14 or 16: Run the command `sudo apt-get install cmake-qt-gui`. Note: If you prefer to use CMake through the command line, see [doc/installation/installation.md#CMake-Command-Line-Configuration-(Ubuntu-Only)](./installation/installation.md#cmake-command-line-configuration-ubuntu-only).
- Ubuntu 14 or 16: Run the command `sudo apt-get install cmake-qt-gui`. Note: If you prefer to use CMake through the command line, see [doc/installation/README.md#CMake-Command-Line-Configuration-(Ubuntu-Only)](installation/README.md#cmake-command-line-configuration-ubuntu-only).
3. Nvidia GPU version prerequisites:
1. **Note: OpenPose has been tested extensively with CUDA 8.0 (cuDNN 5.1) for Ubuntu 14 and 16, CUDA 10.1 (cuDNN 7.5.1) for Ubuntu 18, and CUDA 11 for Ubuntu 20**. We highly recommend using those versions for those Operating Systems to minimize potential installation issues. Other versions should also work, but we do not provide support about any CUDA/cuDNN installation/compilation issue, as well as problems related to their integration into OpenPose.
2. **CUDA**:
......@@ -50,7 +48,7 @@ You might prefer to download them manually:
- Ubuntu 14 or 16 (**cuDNN 5.1 or 7.2**): Run `sudo ./scripts/ubuntu/install_cudnn.sh` (if Ubuntu 16 or 14 and for Graphic cards up to 10XX) or alternatively [download it from their website](https://developer.nvidia.com/rdp/cudnn-archive).
- And install it:
- In order to manually install it (any version), just unzip it and copy (merge) its contents on the CUDA folder, usually `/usr/local/cuda-{version}/` in Ubuntu and `C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v{version}\` in Windows.
5. AMD GPU version prerequisites (only if you do not have an Nvidia GPU and want to run on AMD graphic cards):
5. OpenCL / AMD GPU version prerequisites (only if you do not have an Nvidia GPU and want to run on AMD graphic cards):
- Ubuntu 20 or 18: Not tested and not officially supported. Try at your own risk. You might want to use the CPU version if no Nvidia GPU is available.
- Ubuntu 14 or 16:
1. Download 3rd party ROCM driver for Ubuntu from [**AMD - OpenCL**](https://rocm.github.io/ROCmInstall.html).
......@@ -58,12 +56,16 @@ You might prefer to download them manually:
6. Install **Caffe, OpenCV, and Caffe prerequisites**:
- OpenCV must be already installed on your machine. It can be installed with `sudo apt-get install libopencv-dev`. You could also use your own compiled OpenCV version.
- Caffe prerequisites: By default, OpenPose uses Caffe under the hood. If you have not used Caffe previously, install its dependencies by running `sudo bash ./scripts/ubuntu/install_deps.sh` after installing your desired CUDA and cuDNN versions. If you are using Ubuntu 14 or 16, you can simply run `sudo bash ./scripts/ubuntu/install_deps_and_cuda.sh` (if Ubuntu 16 or 14 and for Graphic cards up to 10XX).
7. **Eigen prerequisite** (optional, only required for some specific extra functionality, such as extrinsic camera calibration). You can perform any of the 2 following options (but only 1 of them!)
- Recommended: Simply set the `WITH_EIGEN` flag to `AUTOBUILD`. CMake will automatically download it and configure OpenPose to use it. If you prefer to download it manually (or if your firewall blocks CMake from downloading it):
- [Eigen 3.3.8](http://posefs1.perception.cs.cmu.edu/OpenPose/3rdparty/eigen_2020_11_18.zip): Unzip as `3rdparty/eigen/`.
- Advanced (not recommended): If you set `WITH_EIGEN` to `FIND`, you must have Eigen already installed in your system. Note that [Eigen <= 3.3.6 is not supported by CUDA >=9.1](https://bitbucket.org/eigen/eigen/commits/034b6c3e101792a3cc3ccabd9bfaddcabe85bb58?at=default). In order to install it (make sure that Eigen version is compatible with CUDA!):
- Run `sudo apt-get install libeigen3-dev` and link CMake to the right CMake.
- Advanced (not recommended): Or you could also use your own version of Eigen by setting `WITH_EIGEN` to `AUTOBUILD`, click `Configure` to let CMake download the zip file, and replace `3rdparty/eigen/` by your own version.
7. Python prerequisites (optional, only if you plan to use the Python API): python-dev, Numpy (for array management), and OpenCV (for image loading).
```
# Python 3 (default and recommended)
sudo apt-get install python3-dev
sudo pip3 install numpy opencv-python
# Python 2
sudo apt-get install python-dev
sudo pip install numpy opencv-python
```
......@@ -71,10 +73,6 @@ You might prefer to download them manually:
1. If you don't have `brew`, install it by running `bash scripts/osx/install_brew.sh` on your terminal.
2. Install **CMake GUI**: Run the command `brew cask install cmake`.
3. Install **Caffe, OpenCV, and Caffe prerequisites**: Run `bash scripts/osx/install_deps.sh`.
4. **Eigen prerequisite** (optional, only required for some specific extra functionality, such as extrinsic camera calibration):
- Enable the `WITH_EIGEN` flag when running CMake, and set it to `AUTOBUILD`.
- CMake will automatically download Eigen.
- Alternatively, you can manually download it from the [Eigen3 website](http://posefs1.perception.cs.cmu.edu/OpenPose/3rdparty/eigen_2020_11_18.zip), and unzip as `3rdparty/eigen/`.
......@@ -107,7 +105,7 @@ NOTE: These instructions are only required when compiling OpenPose brom source.
- [OpenCL Caffe](http://posefs1.perception.cs.cmu.edu/OpenPose/3rdparty/windows/caffe_opencl_2018_02_13.zip): Unzip as `3rdparty/windows/caffe_opencl/`.
- [Caffe dependencies](http://posefs1.perception.cs.cmu.edu/OpenPose/3rdparty/windows/caffe3rdparty_16_2020_11_14.zip): Unzip as `3rdparty/windows/caffe3rdparty/`.
- [OpenCV 4.2.0](http://posefs1.perception.cs.cmu.edu/OpenPose/3rdparty/windows/opencv_450_v15_2020_11_18.zip): Unzip as `3rdparty/windows/opencv/`.
6. **Eigen prerequisite** (optional, only required for some specific extra functionality, such as extrinsic camera calibration):
- Set the `WITH_EIGEN` flag in CMake to `AUTOBUILD`.
- CMake will automatically download Eigen.
- Alternatively, you can manually download it from the [Eigen3 website](http://posefs1.perception.cs.cmu.edu/OpenPose/3rdparty/eigen_2020_11_18.zip), run CMake so that OpenPose downloads the zip file, and then replace the contents of `3rdparty/eigen/` by your own version.
7. Python prerequisites (optional, only if you plan to use the Python API): Install any [Python 3.X](https://www.python.org/downloads/windows/) version for Windows, and then:
```
sudo pip install numpy opencv-python
```
......@@ -273,4 +273,4 @@ There are 3 different keypoint `Array<float>` elements in the `Datum` class:
```
## Camera Matrix Output Format
Check [doc/advanced/calibration_module.md#camera-matrix-output-format](./advanced/calibration_module.md#camera-matrix-output-format).
Check [doc/advanced/calibration_module.md#camera-matrix-output-format](advanced/calibration_module.md#camera-matrix-output-format).
......@@ -50,19 +50,7 @@ If run via the command line, you may need to run cmake twice in order for this c
## Installation
Check [doc/installation/installation.md#python-api](./installation/installation.md#python-api) for installation steps.
The Python API requires python-dev, Numpy (for array management), and OpenCV (for image loading). They can be installed via:
```
# Python 3 (default and recommended)
sudo apt-get install python3-dev
sudo pip3 install numpy opencv-python
# Python 2
sudo apt-get install python-dev
sudo pip install numpy opencv-python
```
Make sure you followed the Python steps in [doc/installation/README.md#cmake-configuration](installation/README.md#cmake-configuration).
......
......@@ -4,17 +4,20 @@ OpenPose - Quick Start
## Contents
1. [Quick Start](#quick-start)
1. [BODY_25 vs. COCO vs. MPI Models](#body-25-vs-coco-vs-mpi-models)
2. [Running on Video](#running-on-video)
3. [Running on Webcam](#running-on-webcam)
4. [Running on Images](#running-on-images)
5. [Maximum Accuracy Configuration](#maximum-accuracy-configuration)
6. [3-D Reconstruction](#3-d-reconstruction)
7. [Tracking](#tracking)
2. [Mac OSX Version Additional Steps](#mac-osx-version-additional-steps)
3. [Running on Video](#running-on-video)
4. [Running on Webcam](#running-on-webcam)
5. [Running on Images](#running-on-images)
6. [Maximum Accuracy Configuration](#maximum-accuracy-configuration)
7. [3-D Reconstruction](#3-d-reconstruction)
8. [Tracking](#tracking)
2. [Expected Visual Results](#expected-visual-results)
## Quick Start
Check OpenPose was properly installed by running it on the default images, video, or webcam.
Check that the library is working properly by running any of the following commands on any command-line interface program. In Ubuntu, Mac, and other Unix systems, use any command-line interface, such as `Terminal` or `Terminator`. In Windows, open the `PowerShell` (recommended) or Windows Command Prompt (CMD). They can be open by pressing the Windows button + X, and then A. Feel free to watch any Youtube video tutorial if you are not familiar with these non-GUI tools. Make sure that you are in the **root directory of the project** (i.e., in the OpenPose folder, not inside `build/` nor `windows/` nor `bin/`). In addition, `examples/media/video.avi` and `examples/media` do exist, no need to change the paths.
......@@ -26,6 +29,25 @@ There is an exception, for CPU version, the COCO and MPI models seems to be fast
### Mac OSX Version Additional Steps
If you compiled Mac with `OPENCL` support, and have inbuilt AMD graphics card, you have to manually select your AMD GPU. To do that, first note which device your Graphics card is set under. Most likely, your AMD device will be device 2.
```bash
clinfo
```
For any OpenPose command you run, add the following 2 flags to use your AMD card for acceleration.
```bash
./build/examples/openpose/openpose.bin --num_gpu 1 --num_gpu_start 2
```
If you only have an integrated Intel Graphics card, then it will most probably be the device 1:
```bash
./build/examples/openpose/openpose.bin --num_gpu 1 --num_gpu_start 1
```
### Running on Video
```
# Ubuntu and Mac
......@@ -40,7 +62,7 @@ bin\OpenPoseDemo.exe --video examples\media\video.avi
bin\OpenPoseDemo.exe --video examples\media\video.avi --face --hand
```
```
:: Windows - Library - Assuming you copied the DLLs following doc/installation/installation.md#windows
:: Windows - Library - Assuming you copied the DLLs following doc/installation/README.md#windows
build\x64\Release\OpenPoseDemo.exe --video examples\media\video.avi
:: With face and hands
build\x64\Release\OpenPoseDemo.exe --video examples\media\video.avi --face --hand
......@@ -62,7 +84,7 @@ bin\OpenPoseDemo.exe
bin\OpenPoseDemo.exe --face --hand
```
```
:: Windows - Library - Assuming you copied the DLLs following doc/installation/installation.md#windows
:: Windows - Library - Assuming you copied the DLLs following doc/installation/README.md#windows
build\x64\Release\OpenPoseDemo.exe
:: With face and hands
build\x64\Release\OpenPoseDemo.exe --face --hand
......@@ -84,7 +106,7 @@ bin\OpenPoseDemo.exe --image_dir examples\media\
bin\OpenPoseDemo.exe --image_dir examples\media\ --face --hand
```
```
:: Windows - Library - Assuming you copied the DLLs following doc/installation/installation.md#windows
:: Windows - Library - Assuming you copied the DLLs following doc/installation/README.md#windows
build\x64\Release\OpenPoseDemo.exe --image_dir examples\media\
:: With face and hands
build\x64\Release\OpenPoseDemo.exe --image_dir examples\media\ --face --hand
......@@ -118,9 +140,9 @@ bin\OpenPoseDemo.exe --net_resolution "1312x736" --scale_number 4 --scale_gap 0.
bin\OpenPoseDemo.exe --net_resolution "1312x736" --scale_number 4 --scale_gap 0.25 --hand --hand_scale_number 6 --hand_scale_range 0.4 --face
```
```
:: Windows - Library - Assuming you copied the DLLs following doc/installation/installation.md#windows: Body
:: Windows - Library - Assuming you copied the DLLs following doc/installation/README.md#windows: Body
build\x64\Release\OpenPoseDemo.exe --net_resolution "1312x736" --scale_number 4 --scale_gap 0.25
:: Windows - Library - Assuming you copied the DLLs following doc/installation/installation.md#windows: Body + Hand + Face
:: Windows - Library - Assuming you copied the DLLs following doc/installation/README.md#windows: Body + Hand + Face
build\x64\Release\OpenPoseDemo.exe --net_resolution "1312x736" --scale_number 4 --scale_gap 0.25 --hand --hand_scale_number 6 --hand_scale_range 0.4 --face
```
......@@ -147,7 +169,7 @@ bin\OpenPoseDemo.exe --flir_camera --3d --number_people_max 1
bin\OpenPoseDemo.exe --flir_camera --3d --number_people_max 1 --face --hand
```
```
:: Windows - Library - Assuming you copied the DLLs following doc/installation/installation.md#windows
:: Windows - Library - Assuming you copied the DLLs following doc/installation/README.md#windows
build\x64\Release\OpenPoseDemo.exe --flir_camera --3d --number_people_max 1
:: With face and hands
build\x64\Release\OpenPoseDemo.exe --flir_camera --3d --number_people_max 1 --face --hand
......
......@@ -439,8 +439,10 @@ OpenPose Library - Release Notes
2. Eigen: From 3.3.4 to 3.3.8.
3. `wget` (Windows): From 1.19.1 to 1.20.3.
3. AppVeyor now tests the actual OpenPoseDemo.exe example (Windows).
4. Doc restructured to improve simplicity. It is now divided into subdirectories (`advanced`, `deprecated`, `installation`, etc).
5. Calibration doc links to included chessboard pdf.
4. Highly simplified README.md.
5. Highly simplified doc, restructured to improve simplicity and now divided into subdirectories (`advanced`, `deprecated`, `installation`, etc).
6. Calibration doc links to included chessboard pdf.
7. Deprecated examples directories `tutorial_add_module` and `tutorial_api_thread` (and renamed as `deprecated`). They still compile, but we no longer support them.
2. Functions or parameters renamed:
3. Main bugs fixed:
1. Eigen working again on Windows.
......
......@@ -5,18 +5,18 @@ OpenPose Library - All Major Released Features
- Jan 2019: [**Unity plugin released**](https://github.com/CMU-Perceptual-Computing-Lab/openpose_unity_plugin)!
- Jan 2019: [**Improved Python API**](doc/python_module.md) released! Including body, face, hands, and all the functionality of the C++ API!
- Dec 2018: [**Foot dataset released**](https://cmu-perceptual-computing-lab.github.io/foot_keypoint_dataset) and [**new paper released**](https://arxiv.org/abs/1812.08008)!
- Sep 2018: [**Experimental tracker**](./quick_start.md#tracking)!
- Jun 2018: [**Combined body-foot model released! 40% faster and 5% more accurate**](./installation/installation.md)!
- Jun 2018: [**Python API**](./python_module.md) released!
- Jun 2018: [**OpenCL/AMD graphic card version**](./installation/installation.md) released!
- Jun 2018: [**Calibration toolbox**](./advanced/calibration_module.md) released!
- Jun 2018: [**Mac OSX version (CPU)**](./installation/installation.md) released!
- Mar 2018: [**CPU version**](./installation/installation.md#cpu-version)!
- Mar 2018: [**3-D keypoint reconstruction module**](./advanced/3d_reconstruction_module.md) (from multiple camera views)!
- Sep 2017: [**CMake**](./installation/installation.md) installer and **IP camera** support!
- Sep 2018: [**Experimental tracker**](quick_start.md#tracking)!
- Jun 2018: [**Combined body-foot model released! 40% faster and 5% more accurate**](installation/README.md)!
- Jun 2018: [**Python API**](python_module.md) released!
- Jun 2018: [**OpenCL/AMD graphic card version**](installation/README.md) released!
- Jun 2018: [**Calibration toolbox**](advanced/calibration_module.md) released!
- Jun 2018: [**Mac OSX version (CPU)**](installation/README.md) released!
- Mar 2018: [**CPU version**](installation/README.md)!
- Mar 2018: [**3-D keypoint reconstruction module**](advanced/3d_reconstruction_module.md) (from multiple camera views)!
- Sep 2017: [**CMake**](installation/README.md) installer and **IP camera** support!
- Jul 2017: [**Windows portable binaries and demo**](https://github.com/CMU-Perceptual-Computing-Lab/openpose/releases)!
- Jul 2017: **Hands** released!
- Jun 2017: **Face** released!
- May 2017: **Windows** version!
- Apr 2017: **Body** released!
For further details, check the [release notes](./release_notes.md).
For further details, check the [release notes](release_notes.md).
......@@ -20,6 +20,8 @@ Check the [OpenPose Benchmark](https://docs.google.com/spreadsheets/d/1-DynFGvoS
### CPU Version
The CPU version runs at about 0.3 FPS on the COCO model, and at about 0.1 FPS (i.e., about 15 sec / frame) on the default BODY_25 model. Switch to COCO model and/or reduce the `net_resolution` as indicated above. Contradictory fact: BODY_25 model is about 5x slower than COCO on CPU-only version, but it is about 40% faster on GPU version.
On Ubuntu (for OS versions older than 20), you can also boost CPU-only speed by 2-3x by following [installation/README.md#faster-cpu-version-ubuntu-only](installation/README.md#faster-cpu-version-ubuntu-only).
### Profiling Speed
......@@ -28,7 +30,7 @@ OpenPose displays the FPS in the basic GUI. However, more complex speed metrics
- Time measurement for 1 graphic card: The FPS will be the slowest time displayed in your terminal command line (as OpenPose is multi-threaded). Times are in milliseconds, so `FPS = 1000/millisecond_measurement`.
- Time measurement for >1 graphic cards: Assuming `n` graphic cards, you will have to wait up to `n` x `F` frames to visualize each graphic card speed (as the frames are splitted among them). In addition, the FPS would be: `FPS = minFPS(speed_per_GPU/n, worst_time_measurement_other_than_GPUs)`. For < 4 GPUs, this is usually `FPS = speed_per_GPU/n`.
Make sure that `wPoseExtractor` time is the slowest timing. Otherwise the input producer (video/webcam codecs issues with OpenCV, images too big, etc.) or the GUI display (use OpenGL support as detailed in [doc/speed_up_openpose.md](./speed_up_openpose.md)) might not be optimized.
Make sure that `wPoseExtractor` time is the slowest timing. Otherwise the input producer (video/webcam codecs issues with OpenCV, images too big, etc.) or the GUI display (use OpenGL support as detailed in [doc/speed_up_openpose.md](speed_up_openpose.md)) might not be optimized.
......@@ -46,7 +48,7 @@ Some speed tips to maximize the OpenPose runtime speed while preserving the accu
## Speed Up and Memory Reduction
Some speed tips to highly maximize the OpenPose speed, but keep in mind the accuracy trade-off:
1. Reduce the `--net_resolution` (e.g., to 320x176) (lower accuracy). Note: For maximum accuracy, follow [doc/quick_start.md#maximum-accuracy-configuration](./quick_start.md#maximum-accuracy-configuration).
1. Reduce the `--net_resolution` (e.g., to 320x176) (lower accuracy). Note: For maximum accuracy, follow [doc/quick_start.md#maximum-accuracy-configuration](quick_start.md#maximum-accuracy-configuration).
2. For face, reduce the `--face_net_resolution`. The resolution 320x320 usually works pretty decently.
3. Points 1-2 will also reduce the GPU memory usage (or RAM memory for CPU version).
4. Use the `BODY_25` model for maximum speed. Use `MPI_4_layers` model for minimum GPU memory usage (but lower accuracy, speed, and number of parts).
add_subdirectory(calibration)
add_subdirectory(deprecated)
add_subdirectory(openpose)
add_subdirectory(tutorial_add_module)
add_subdirectory(tutorial_api_cpp)
add_subdirectory(tutorial_api_python)
add_subdirectory(tutorial_api_thread)
add_subdirectory(user_code)
if (UNIX OR APPLE)
add_subdirectory(tests)
......
set(EXAMPLE_FILES
1_custom_post_processing.cpp)
tutorial_add_module_custom_post_processing.cpp
tutorial_api_thread_1_user_processing_function.cpp
tutorial_api_thread_2_user_input_processing_output_and_datum.cpp)
include(${CMAKE_SOURCE_DIR}/cmake/Utils.cmake)
......@@ -18,7 +20,7 @@ foreach(EXAMPLE_FILE ${EXAMPLE_FILES})
target_link_libraries(${EXE_NAME} openpose ${examples_3rdparty_libraries})
if (WIN32)
set_property(TARGET ${EXE_NAME} PROPERTY FOLDER "Examples/Tutorial/AddModule")
set_property(TARGET ${EXE_NAME} PROPERTY FOLDER "Examples/Tutorial/Deprecated")
configure_file(${CMAKE_SOURCE_DIR}/cmake/OpenPose${VCXPROJ_FILE_GPU_MODE}.vcxproj.user
${CMAKE_CURRENT_BINARY_DIR}/${EXE_NAME}.vcxproj.user @ONLY)
# Properties->General->Output Directory
......
# Thread API Examples
**NOTE**: This directory only provides examples for the basic OpenPose thread mechanism API, and it is only meant for people interested in the multi-thread architecture without been interested in the OpenPose pose estimation algorithm.
# Deprecated Examples
You are most probably looking for the [examples/tutorial_api_cpp/](../tutorial_api_cpp/) or [examples/tutorial_api_python/](../tutorial_api_python/), which provide examples of the thread API already applied to body pose estimation.
This directory is for OpenPose deployers, not for OpenPose uses, as it only provides (potentially not updated) examples for:
- The basic OpenPose thread mechanism API, and it is only meant for people interested in the multi-thread architecture without been interested in the OpenPose pose estimation algorithm.
- How to add a new module to OpenPose. And this is already and better explained in [examples/tutorial_api_cpp/](../tutorial_api_cpp/) or [examples/tutorial_api_python/](../tutorial_api_python/).
......@@ -7,9 +7,9 @@
// 2. `wUserPostProcessing.hpp`: Change 1 line in `work(TDatums& tDatums)` to use your custom function from
// `userPostProcessing.hpp`.
// 3. `userDatum.hpp`: Add any required output of your post-processing class there.
// 4. `1_custom_post_processing.cpp`: Change 1 line, when creating the `UserPostProcessing` class, add any specific
// 4. `tutorial_add_module_custom_post_processing.cpp`: Change 1 line, when creating the `UserPostProcessing` class, add any specific
// arguments/parameters than your class need.
// 5. `1_custom_post_processing.bin`: Compile and run this file (as you usually run the OpenPose demo) in order to
// 5. `tutorial_add_module_custom_post_processing.bin`: Compile and run this file (as you usually run the OpenPose demo) in order to
// test your custom functionality.
// Syntax rules
......@@ -27,8 +27,8 @@
#include <openpose/flags.hpp>
// OpenPose dependencies
#include <openpose/headers.hpp>
#include "userDatum.hpp"
#include "wUserPostProcessing.hpp"
#include "tutorial_add_module/userDatum.hpp"
#include "tutorial_add_module/wUserPostProcessing.hpp"
void configureWrapper(op::WrapperT<op::UserDatum>& opWrapperT)
{
......
set(EXAMPLE_FILES
1_thread_user_processing_function.cpp
2_thread_user_input_processing_output_and_datum.cpp)
foreach(EXAMPLE_FILE ${EXAMPLE_FILES})
get_filename_component(SOURCE_NAME ${EXAMPLE_FILE} NAME_WE)
if (UNIX OR APPLE)
set(EXE_NAME "${SOURCE_NAME}.bin")
elseif (WIN32)
set(EXE_NAME "${SOURCE_NAME}")
endif ()
message(STATUS "Adding Example ${EXE_NAME}")
add_executable(${EXE_NAME} ${EXAMPLE_FILE})
target_link_libraries(${EXE_NAME} openpose ${examples_3rdparty_libraries})
if (WIN32)
set_property(TARGET ${EXE_NAME} PROPERTY FOLDER "Examples/Tutorial/Thread API")
configure_file(${CMAKE_SOURCE_DIR}/cmake/OpenPose${VCXPROJ_FILE_GPU_MODE}.vcxproj.user
${CMAKE_CURRENT_BINARY_DIR}/${EXE_NAME}.vcxproj.user @ONLY)
# Properties->General->Output Directory
set_property(TARGET ${EXE_NAME} PROPERTY RUNTIME_OUTPUT_DIRECTORY_RELEASE ${PROJECT_BINARY_DIR}/$(Platform)/$(Configuration))
set_property(TARGET ${EXE_NAME} PROPERTY RUNTIME_OUTPUT_DIRECTORY_DEBUG ${PROJECT_BINARY_DIR}/$(Platform)/$(Configuration))
endif (WIN32)
endforeach()
......@@ -11,7 +11,7 @@ You can quickly add your custom code into this folder so that quick prototypes c
## How-to
1. Install/compile OpenPose as usual.
2. Add your custom cpp / hpp files in this folder. Hint: You might want to start by copying the [OpenPoseDemo](../openpose/openpose.cpp) example or any of the [examples/tutorial_api_cpp/](../tutorial_api_cpp/) examples.
3. Add the name of your custom cpp / hpp files at the top of the [examples/user_code/CMakeLists.txt](./CMakeLists.txt) file.
3. Add the name of your custom cpp / hpp files at the top of the [examples/user_code/CMakeLists.txt](CMakeLists.txt) file.
4. Re-run CMake (`Configure` and `Generate`).
5. Then, you can simply modify their content.
6. You can know compile OpenPose after each change in your files:
......
......@@ -14,8 +14,8 @@ if [[ $RUN_EXAMPLES == true ]] ; then
./build/examples/openpose/openpose.bin --image_dir examples/media/ --net_resolution -1x32 --write_json output/ --write_images output/ --display 0
echo " "
echo "Tutorial Add Module: Example 1..."
./build/examples/tutorial_add_module/1_custom_post_processing.bin --image_dir examples/media/ --net_resolution -1x32 --write_json output/ --write_images output/ --display 0
echo "(Deprecated) Tutorial Add Module: Example 1..."
./build/examples/deprecated/tutorial_add_module_custom_post_processing.bin --image_dir examples/media/ --net_resolution -1x32 --write_json output/ --write_images output/ --display 0
echo " "
# # Note: Examples 1-2 require the whole OpenPose resolution (too much RAM memory) and the GUI
......
Markdown is supported
0% .
You are about to add 0 people to the discussion. Proceed with caution.
先完成此消息的编辑!
想要评论请 注册