Note: Currently using [travis-matrix-badges](https://github.com/bjfish/travis-matrix-badges) vs. traditional [](https://travis-ci.org/CMU-Perceptual-Computing-Lab/openpose)
-->
[**OpenPose**](https://github.com/CMU-Perceptual-Computing-Lab/openpose) has represented the **first real-time multi-person system to jointly detect human body, hand, facial, and foot keypoints (in total 135 keypoints) on single images**, and it is currently being actively used in many projects and research areas!
[**OpenPose**](https://github.com/CMU-Perceptual-Computing-Lab/openpose) has represented the **first real-time multi-person system to jointly detect human body, hand, facial, and foot keypoints (in total 135 keypoints) on single images**.
It is **authored by [Gines Hidalgo](https://www.gineshidalgo.com), [Zhe Cao](https://people.eecs.berkeley.edu/~zhecao), [Tomas Simon](http://www.cs.cmu.edu/~tsimon), [Shih-En Wei](https://scholar.google.com/citations?user=sFQD3k4AAAAJ&hl=en), [Hanbyul Joo](https://jhugestar.github.io), and [Yaser Sheikh](http://www.cs.cmu.edu/~yaser)**, and **maintained by [Gines Hidalgo](https://www.gineshidalgo.com) and [Yaadhav Raaj](https://www.raaj.tech)**. OpenPose would not be possible without the [**CMU Panoptic Studio dataset**](http://domedb.perception.cs.cmu.edu). We would also like to thank all the people who helped OpenPose in any way ([doc/contributors.md](doc/contributors.md)).
...
...
@@ -74,7 +74,7 @@ It is **authored by [Gines Hidalgo](https://www.gineshidalgo.com), [Zhe Cao](htt
</p>
### Runtime Analysis
In the following image, we show an inference time comparison between the 3 available pose estimation libraries: OpenPose, Alpha-Pose (fast Pytorch version), and Mask R-CNN. The OpenPose runtime is constant, while the runtime of Alpha-Pose and Mask R-CNN grow linearly with the number of people. The analysis was performed using the same hardware (Nvidia 1080 Ti) and number of images per algorithm. More details [**here**](https://arxiv.org/abs/1812.08008).
We show an inference time comparison between the 3 available pose estimation libraries (same hardware and conditions): OpenPose, Alpha-Pose (fast Pytorch version), and Mask R-CNN. The OpenPose runtime is constant, while the runtime of Alpha-Pose and Mask R-CNN grow linearly with the number of people. More details [**here**](https://arxiv.org/abs/1812.08008).
- Included [C++](examples/tutorial_api_cpp/)/[python](examples/tutorial_api_python/) examples to add your custom input and/or output, as well as pre or post-processing steps.
-**Input**: Image, video, webcam, Flir/Point Grey, IP camera, and support to add your own custom input source (e.g., depth camera).
-**Output**: Basic image + keypoint display/saving (PNG, JPG, AVI, ...), keypoint saving (JSON, XML, YML, ...), keypoints as array class, and support to add your own custom output code (e.g., some fancy UI).
-**OS**: Ubuntu (20, 18, 16, 14), Windows (10, 8), Mac OSX, Nvidia TX2.
-**Hardware compatibility**: CUDA (Nvidia GPU), OpenCL (AMD GPU), and non-GPU (CPU-only) versions.
-**Others**:
-Available: command-line demo, C++ wrapper, and C++ API.
-[**Python API**](doc/python_module.md).
-**Usage Alternatives**:
-[**Command-line demo**](doc/demo_quick_start.md) for built-in functionality.
-[**C++ API**](examples/tutorial_api_cpp/) and [**Python API**](doc/python_module.md) for custom functionality. E.g., adding your custom inputs, pre-processing, post-posprocessing, and output steps.
For further details, check [all released features](doc/released_features.md) and [release notes](doc/release_notes.md).
## Related Work
-[**OpenPose training code**](https://github.com/CMU-Perceptual-Computing-Lab/openpose_train).
- OpenPose papers published in [**IEEE TPAMI** and **CVPR**](#citation). [Cite them](#citation) in your publications if it helps your research!
...
...
@@ -122,12 +121,16 @@ See [doc/installation/README.md](doc/installation/README.md) for more details.
## Quick Start
Most users do not need the OpenPose C++/Python API, but can simply use the OpenPose Demo:
Most users do not need to know C++ or Python, they can simply use the OpenPose Demo in their command-line tool (e.g., PowerShell/Terminal). E.g., this would run OpenPose on the webcam and display the body keypoints:
```
# Ubuntu
./build/examples/openpose/openpose.bin
-[**Output information**](doc/output.md): Learn about the output format, keypoint index ordering, etc.
-**OpenPose Demo**: Choose your input (e.g., images, video, webcam), set of algorithms (body, hand, face), and output (e.g., display, JSON keypoint saving, image+keypoints). See [doc/quick_start.md](doc/quick_start.md) and [doc/demo_overview.md](doc/demo_overview.md).
- E.g., in your terminal or PowerShell: Given an input video (`--video`), extract body (by default), face (`--face`) and hand (`--hand`) keypoints, save the keypoints in a JSON file (`--write_json`), and display (by default) the results in the screen. Remove any of the flags to remove that particular functionality.
You can also add any of the available flags in any order. Do you also want to add face and/or hands? Add the `--face` and/or `--hand` flags. Do you also want to save the output keypoints on JSON files on disk? Add the `--write_json` flag, etc.
-[**OpenPose C++ API**](examples/tutorial_api_cpp/): If you want to read a specific input, and/or add your custom post-processing function, and/or implement your own display/saving.
- For quick prototyping: You can easily **create your custom code** on [examples/user_code/](examples/user_code/) and CMake will automatically compile it together with the whole OpenPose project. See [examples/user_code/README.md](examples/user_code/README.md) for more details.
-[**OpenPose Python API**](examples/tutorial_api_python/): Almost the exact same functionality than the C++ API, but in Python!
- For quick prototyping: You can simply duplicate and rename any of the [existing example files](examples/tutorial_api_python/) within that same folder.
-[**Speeding up OpenPose and benchmark**](doc/speed_up_openpose.md): Check the OpenPose Benchmark as well as some hints to speed up and/or reduce the memory requirements for OpenPose.
-[**Calibration toolbox**](doc/advanced/calibration_module.md) and [**3D OpenPose**](doc/advanced/3d_reconstruction_module.md): Calibrate your cameras for 3D OpenPose (or any other stereo vision tasks) and start obtaining 3D keypoints!
-[**Standalone face or hand detector**doc/advanced/standalone_face_or_hand_keypoint_detector.md](doc/advanced/standalone_face_or_hand_keypoint_detector.md) is useful if you want to do any of the following:
-**Face** keypoint detection **without body** keypoint detection: Pros: Speedup and RAM/GPU memory reduction. Cons: Worse accuracy and less detected number of faces).
-**Use your own face/hand detector**: You can use the hand and/or face keypoint detectors with your own face or hand detectors, rather than using the body detector. E.g., useful for camera views at which the hands are visible but not the body (OpenPose detector would fail).
After [installing](#installation) OpenPose, check [doc/README.md](doc/README.md) for a quick overview of all the alternatives and tutorials.
## Send Us Feedback!
Our library is open source for research purposes, and we want to continuously improve it! So please, let us know if...
1. ... you find any bug (in functionality or speed).
2. ... you added some functionality on top of OpenPose which we might want to add.
3. ... you know how to speed up or improve any part of OpenPose.
4. ... you want to share your cool demo or project made on top of OpenPose (you can email it to us too!).
5. ... etc.
Our library is open source for research purposes, and we want to continuously improve it! So let us know if you...
1. Find any bug (in functionality or speed).
2. Add some functionality on top of OpenPose which we might want to add.
3. Know how to speed up or improve any part of OpenPose.
4. Want to share your cool demo or project made on top of OpenPose (you can email it to us too!).
Just create a new GitHub issue or a pull request and we will answer as soon as possible!
Most users can simply use the OpenPose Demo without reading any C++/Python code. Users that need to add additional functionality (new inputs, outputs, etc) should check the C++/Python APIs:
- The first step for any software, [installing](installation.md) it!
-[**Output information**](doc/output.md): Learn about the output format, keypoint index ordering, etc.
-[**OpenPose Demo**](demo_quick_start.md) for your terminal or PowerShell: Choose your input (e.g., images, video, webcam), set of algorithms (body, hand, face), and output (e.g., display, JSON keypoint saving, image+keypoints).
- E.g.: Given an input video (`--video`), extract body (by default), face (`--face`) and hand (`--hand`) keypoints, save the keypoints in a JSON file (`--write_json`), and display (by default) the results in the screen. You can remove any of the flags to remove that particular functionality or add any other.
-[**OpenPose C++ API**](examples/tutorial_api_cpp/): If you want to read a specific input, and/or add your custom post-processing function, and/or implement your own display/saving.
- You should be familiar with the [**OpenPose Demo**](demo_quick_start.md) and the main OpenPose flags before trying to read the C++ or Python API examples. Otherwise, it will be way harder to follow.
- For quick prototyping: You can easily **create your custom code** on [examples/user_code/](examples/user_code/) and CMake will automatically compile it together with the whole OpenPose project. See [examples/user_code/README.md](examples/user_code/README.md) for more details.
-[**OpenPose Python API**](examples/tutorial_api_python/): Almost the exact same functionality than the C++ API, but in Python!
- You should be familiar with the [**OpenPose Demo**](demo_quick_start.md) and the main OpenPose flags before trying to read the C++ or Python API examples. Otherwise, it will be way harder to follow.
- For quick prototyping: You can simply duplicate and rename any of the [existing example files](examples/tutorial_api_python/) within that same folder.
-[**Speeding up OpenPose and benchmark**](doc/speed_up_openpose.md): Check the OpenPose Benchmark as well as some hints to speed up and/or reduce the memory requirements for OpenPose.
-[**Calibration toolbox**](doc/advanced/calibration_module.md) and [**3D OpenPose**](doc/advanced/3d_reconstruction_module.md): Calibrate your cameras for 3D OpenPose (or any other stereo vision tasks) and start obtaining 3D keypoints!
-[**Standalone face or hand detector**doc/advanced/standalone_face_or_hand_keypoint_detector.md](doc/advanced/standalone_face_or_hand_keypoint_detector.md) is useful if you want to do any of the following:
-**Face** keypoint detection **without body** keypoint detection: Pros: Speedup and RAM/GPU memory reduction. Cons: Worse accuracy and less detected number of faces).
-**Use your own face/hand detector**: You can use the hand and/or face keypoint detectors with your own face or hand detectors, rather than using the body detector. E.g., useful for camera views at which the hands are visible but not the body (OpenPose detector would fail).
@@ -192,7 +192,7 @@ In order to be initialized, `PoseExtractorCaffe` has the following constructor a
3.`outputSize` is the final desired resolution to be used. The human pose keypoint locations will be scaled to this output size. However, the heat-maps will have the `netOutputSize` size due to performance.
4.`scaleNumber` and `scaleGap` specify the multi-scale parameters. Explained in [doc/demo_overview.md](../../demo_overview.md).
4.`scaleNumber` and `scaleGap` specify the multi-scale parameters. Explained in [doc/demo_not_quick_start.md](../../demo_not_quick_start.md).
5.`poseModel` specifies the model to load (e.g., COCO or MPI).
Forget about the OpenPose library code, just compile the library and use the demo `./build/examples/openpose/openpose.bin`.
This document is a more detailed continuation of [doc/demo_quick_start.md](demo_quick_start.md), and it assumes the user is quite familiar with the OpenPose demo and the contents of [doc/demo_quick_start.md](demo_quick_start.md).
In order to learn how to use it, run `./build/examples/openpose/openpose.bin --help` in your favorite command-line interface tool and read all the available flags (check only the flags for `examples/openpose/openpose.cpp` itself, i.e., the section `Flags from examples/openpose/openpose.cpp:`). We detail some of them in the following sections.
In Ubuntu, Mac, and other Unix systems, use any command-line interface, such as `Terminal` or `Terminator`. In Windows, open the `PowerShell` (recommended) or Windows Command Prompt (CMD). They can be open by pressing the Windows button + X, and then A. Feel free to watch any Youtube video tutorial if you are not familiar with these non-GUI tools. Make sure that you are in the **root directory of the project** (i.e., in the OpenPose folder, not inside `build/` nor `windows/` nor `bin/`).
## Contents
1.[More Advanced Common Settings](#more-advanced-common-settings)
1.[Reducing Latency/Lag](#reducing-latencylag)
2.[Advanced Hands](#advanced-hands)
3.[Rendering Face and Hands without Pose](#rendering-face-and-hands-without-pose)
4.[Debugging Information](#debugging-information)
5.[Heat Maps Storing](#heat-maps-storing)
6.[BODY_25 vs. COCO vs. MPI Models](#body-25-vs-coco-vs-mpi-models)
2.[Main Flags](#main-flags)
3.[Help Flag](#help-flag)
4.[All Flags](#all-flags)
## Running on Images, Video or Webcam
See [doc/quick_start.md#quick-start](quick_start.md#quick-start).
## Pose + Face + Hands
See [doc/quick_start.md#quick-start](quick_start.md#quick-start).
## Maximum Accuracy Configuration
See [doc/quick_start.md#maximum-accuracy-configuration](quick_start.md#maximum-accuracy-configuration).
## Reducing Latency/Lag
## More Advanced Common Settings
### Reducing Latency/Lag
In general, there are 3 ways to reduce the latency (with some drawbacks each one):
- Reducing `--output_resolution`: It will slightly reduce the latency and increase the FPS. But the quality of the displayed image will deteriorate.
- Reducing `--net_resolution` and/or `--face_net_resolution` and/or `--hand_net_resolution`: It will increase the FPS and reduce the latency. But the accuracy will drop, specially for small people in the image. Note: For maximum accuracy, follow [doc/quick_start.md#maximum-accuracy-configuration](quick_start.md#maximum-accuracy-configuration).
- Reducing `--net_resolution` and/or `--face_net_resolution` and/or `--hand_net_resolution`: It will increase the FPS and reduce the latency. But the accuracy will drop, specially for small people in the image. Note: For maximum accuracy, follow [doc/demo_quick_start.md#maximum-accuracy-configuration](demo_quick_start.md#maximum-accuracy-configuration).
- Enabling `--disable_multi_thread`: The latency should be reduced. But the speed will drop to 1-GPU speed (as it will only use 1 GPU). Note that it's practical only for body, if hands and face are also extracted, it's usually not worth it.
## Kinect 2.0 as Webcam on Windows 10
Since the Windows 10 Anniversary, Kinect 2.0 can be read as a normal webcam. All you need to do is go to `device manager`, expand the `kinect sensor devices` tab, right click and update driver of `WDF kinectSensor Interface`. If you already have another webcam, disconnect it or use `--camera 2`.
## JSON Output with No Visualization
The following example runs the demo video `video.avi` and outputs JSON files in `output/`. Note: see [doc/output.md](output.md) to understand the format of the JSON files.
The following example runs the demo video `video.avi`, renders image frames on `output/result.avi`, and outputs JSON files in `output/`. Note: see [doc/output.md](output.md) to understand the format of the JSON files.
@@ -92,19 +65,20 @@ The following example runs the demo video `video.avi`, renders image frames on `
## Selecting Some GPUs
The following example runs the demo video `video.avi`, parallelizes it over 2 GPUs, GPUs 1 and 2 (note that it will skip GPU 0):
### Heat Maps Storing
The following command will save all the body part heat maps, background heat map and Part Affinity Fields (PAFs) in the folder `output_heatmaps_folder`. It will save them on PNG format. Instead of individually saving each of the 67 heatmaps (18 body parts + background + 2 x 19 PAFs) individually, the library concatenate them vertically into a huge (width x #heatmaps) x (height) matrix. The PAFs channels are multiplied by 2 because there is one heatmpa for the x-coordinates and one for the y-coordinates. The order is body parts + bkg + PAFs. It will follow the sequence on POSE_BODY_PART_MAPPING in [include/openpose/pose/poseParameters.hpp](../include/openpose/pose/poseParameters.hpp).
The following command will save all the body part heat maps, background heat map and Part Affinity Fields (PAFs) in the folder `output_heatmaps_folder`. It will save them on PNG format. Instead of individually saving each of the 67 heatmaps (18 body parts + background + 2 x 19 PAFs) individually, the library concatenate them vertically into a huge (width x #heatmaps) x (height) matrix. The PAFs channels are multiplied by 2 because there is one heatmpa for the x-coordinates and one for the y-coordinates. The order is body parts + bkg + PAFs. It will follow the sequence on POSE_BODY_PART_MAPPING in [include/openpose/pose/poseParameters.hpp](../include/openpose/pose/poseParameters.hpp).
The `BODY_25` model (`--model_pose BODY_25`) includes both body and foot keypoints and it is based in [OpenPose: Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields](https://arxiv.org/abs/1812.08008). COCO and MPI models are slower, less accurate, and do not contain foot keypoints. They are based in our older paper [Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields](https://arxiv.org/abs/1611.08050). We highly recommend only using the `BODY_25` model.
There is an exception, for CPU version, the COCO and MPI models seems to be faster. Accuracy is still better for the `BODY_25` model.
...
...
@@ -130,7 +104,24 @@ We enumerate some of the most important flags, check the `Flags Detailed Descrip
## Flag Description
## Help Flag
Now that you are more familiar with OpenPose, you can add the flag `--help` to see all the available OpenPose flags. Check only the flags for `examples/openpose/openpose.cpp` itself (i.e., the ones in the section `Flags from examples/openpose/openpose.cpp:`). [All Flags](#all-flags) shows them all in this document.
```
# Ubuntu and Mac
./build/examples/openpose/openpose.bin --help
```
```
:: Windows - Portable Demo
bin\OpenPoseDemo.exe --help
```
## All Flags
Each flag is divided into flag name, default value, and description.
Forget about the OpenPose code, just download the portable Windows binaries (or compile the code from source) and use the demo by following this tutorial!
## Contents
1.[Quick Start](#quick-start)
1.[BODY_25 vs. COCO vs. MPI Models](#body-25-vs-coco-vs-mpi-models)
2.[Mac OSX Version Additional Steps](#mac-osx-version-additional-steps)
Check OpenPose was properly installed by running it on the default images, video, or webcam.
Check that the library is working properly by running any of the following commands on any command-line interface program. In Ubuntu, Mac, and other Unix systems, use any command-line interface, such as `Terminal` or `Terminator`. In Windows, open the `PowerShell` (recommended) or Windows Command Prompt (CMD). They can be open by pressing the Windows button + X, and then A. Feel free to watch any Youtube video tutorial if you are not familiar with these non-GUI tools. Make sure that you are in the **root directory of the project** (i.e., in the OpenPose folder, not inside `build/` nor `windows/` nor `bin/`). In addition, `examples/media/video.avi` and `examples/media` do exist, no need to change the paths.
8.[JSON Output](json-output)
9.[JSON Output with No Visualization](json-output-with-no-visualization)
10.[Not Running All GPUs](#not-running-all-gpus)
11.[Kinect 2.0 as Webcam on Windows 10](#kinect-20-as-webcam-on-windows-10)
12.[Tracking](#tracking)
3.[Advanced Quick Start](#advanced-quick-start)
### BODY_25 vs. COCO vs. MPI Models
The BODY_25 model (`--model_pose BODY_25`) includes both body and foot keypoints and it is based in [OpenPose: Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields](https://arxiv.org/abs/1812.08008). COCO and MPI models are slower, less accurate, and do not contain foot keypoints. They are based in our older paper [Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields](https://arxiv.org/abs/1611.08050). We highly recommend only using the BODY_25 model.
There is an exception, for CPU version, the COCO and MPI models seems to be faster. Accuracy is still better for the BODY_25 model.
### Mac OSX Version Additional Steps
## Mac OSX Version Additional Step
If you are using a Mac and selected `CPU_ONLY`, you can skip this section.
If you compiled Mac with `OPENCL` support, and have inbuilt AMD graphics card, you have to manually select your AMD GPU. To do that, first note which device your Graphics card is set under. Most likely, your AMD device will be device 2.
If you are using a Mac and selected `OPENCL` support, and it has an inbuilt AMD graphics card, you have to manually select your AMD GPU. To do that, first note which device your Graphics card is set under. Most likely, your AMD device will be device 2.
```bash
clinfo
```
For any OpenPose command you run, add the following 2 flags to use your AMD card for acceleration.
For any OpenPose command you run, add the following 2 flags to use your AMD card for acceleration (there `num_gpu_start` should be the number given above).
If you only have an integrated Intel Graphics card, then it will most probably be the device 1:
If you only have an integrated Intel Graphics card, then it will most probably be the device 1. Then, always add the following 2 flags to use your AMD card for acceleration.
Check that OpenPose was properly installed by running any of the following 3 examples (image folder, video, or webcam). The expected visual result should look like [doc/output.md#expected-visual-results](output.md#expected-visual-results).
In Ubuntu, Mac, and other Unix systems, use any command-line interface, such as `Terminal` or `Terminator`. In Windows, open the `PowerShell`. You can do so with right-click on the Windows button, and `Windows PowerShell` (or pressing the Windows button + X, and then A). Feel free to watch any Youtube video tutorial if you are not familiar with these non-GUI tools.
Make sure that you are in the **root directory of the project** when running any command (i.e., in the OpenPose folder, not inside `build/` nor `windows/` nor `bin/`). In addition, `examples/media/video.avi` and `examples/media` already exist, so there is no need to change any lines of code on this tutorial. You can test OpenPose by running:
If these fail with an out of memory error, do not worry, the next example will fix this issue.
### Improving Memory and Speed but Decreasing Accuracy
If you have a Nvidia GPU that does not goes out of memory when running, **you should skip this step!**
**Use at your own risk**: If your GPU runs out of memory or you do not have a Nvidia GPU, you can reduce `--net_resolution` to improve the speed and reduce the memory requirements, but it will also highly reduce accuracy! The lower the resolution, the lower accuracy, but also improved speed and memory:
The following example runs the demo video `video.avi`, renders image frames on `output/result.avi`, and outputs JSON files in `output/`. Note: see [doc/output.md](output.md) to understand the format of the JSON files.
The following example runs the demo video `video.avi` and outputs JSON files in `output/`. Note: see [doc/output.md](output.md) to understand the format of the JSON files.
By default, OpenPose will use all the GPUs available in your machine. The following example runs the demo video `video.avi`, parallelizes it over 2 GPUs, GPUs 1 and 2 (note that it will skip GPU 0):
Since the Windows 10 Anniversary, Kinect 2.0 can be read as a normal webcam. All you need to do is go to `device manager`, expand the `kinect sensor devices` tab, right click and update driver of `WDF kinectSensor Interface`. If you already have another webcam, disconnect it or use `--camera 2`.
### Tracking
1. Runtime huge speed up by reducing the accuracy:
```
# Ubuntu and Mac (same flags for Windows version)
# Using OpenPose 1 frame, tracking the following e.g., 5 frames
The visual GUI should show the original image with the poses blended on it, similarly to the pose of this gif:
<palign="center">
<imgsrc="media/shake.gif",width="720">
</p>
If you choose to visualize a body part or a PAF (Part Affinity Field) heat map with the command option `--part_to_show`, the result should be similar to one of the following images:
<palign="center">
<imgsrc="media/body_heat_maps.png",width="720">
</p>
<palign="center">
<imgsrc="media/paf_heat_maps.png",width="720">
</p>
## Advanced Quick Start
In order to learn about many more flags, check [doc/demo_not_quick_start.md](demo_not_quick_start.md).
@@ -109,7 +109,7 @@ For problem 2, try the following solutions (in this order):
#### Very Few People Detected
**Q: Low detection rate. It can detect the person on some images (usually higher contrast, with bigger people), but it will fail for most of images with low resolution or small people.**
**A**: Images with low resolution, or with people too tiny will simply not work too well. However, it can be highly improved by using the maximum accuracy configuration detailed in [doc/quick_start.md#maximum-accuracy-configuration](quick_start.md#maximum-accuracy-configuration).
**A**: Images with low resolution, or with people too tiny will simply not work too well. However, it can be highly improved by using the maximum accuracy configuration detailed in [doc/demo_quick_start.md#maximum-accuracy-configuration](demo_quick_start.md#maximum-accuracy-configuration).
...
...
@@ -241,7 +241,7 @@ Check the [doc/speed_up_openpose.md#profiling-speed](speed_up_openpose.md#profil
### Accuracy Issues
#### Is Maximum Accuracy Configuration Possible on Lower End GPUs?
**Q**: I've read that this command provides the most accurate results possible on Openpose so far: https://github.com/CMU-Perceptual-Computing-Lab/openpose/blob/master/doc/quick_start.md#maximum-accuracy-configuration. However, a 8GB GPU (e.g., 1080 or 2080) will run out of memory, is there any method to achieve the same accuracy on GPU using less memory even if it meant sacrificing speed?
**Q**: I've read that this command provides the most accurate results possible on Openpose so far: https://github.com/CMU-Perceptual-Computing-Lab/openpose/blob/master/doc/demo_quick_start.md#maximum-accuracy-configuration. However, a 8GB GPU (e.g., 1080 or 2080) will run out of memory, is there any method to achieve the same accuracy on GPU using less memory even if it meant sacrificing speed?
**A**: Unfortunately no, there is no way at the moment. Caffe just takes so much memory doing that. You can try with `--scale_number 3` instead of 4, reducing a bit the `net_resolution` (e.g. `720` vs. `736`) and starting the computer without GUI (which also takes about 1GB of memory just to keep the computer GUI running).
**If you just want to use OpenPose in Windows** without compiling or writing any code, simply download and use the latest portable version of OpenPose. You can download it in the [Releases](https://github.com/CMU-Perceptual-Computing-Lab/openpose/releases) section. Read the `Instructions.txt` inside the downloaded files to learn to download the models required by OpenPose (about 500 Mb).
**If you just want to use OpenPose** without compiling or writing any code, simply use the latest portable version of OpenPose for Windows.
1. Download it from the [Releases](https://github.com/CMU-Perceptual-Computing-Lab/openpose/releases) section.
2. Follow the `Instructions.txt` file inside the downloaded zip file to download the models required by OpenPose (about 500 Mb).
3. Then, you can run OpenPose from the PowerShell command-line by following [doc/demo_quick_start.md](demo_quick_start.md).
...
...
@@ -108,15 +111,15 @@ cmake-gui ..
```
2. Select the OpenPose directory as project source directory, and a non-existing or empty sub-directory (e.g., `build`) where the Makefile files (Ubuntu) or Visual Studio solution (Windows) will be generated. If `build` does not exist, it will ask you whether to create it. Press `Yes`.
3. Press the `Configure` button, keep the generator in `Unix Makefiles` (Ubuntu) or set it to your 64-bit Visual Studio version (Windows), and press `Finish`. Note for Windows users: CMake-GUI has changed their design after version 14. For versions older than 14, you usually select `Visual Studio XX 20XX Win64` as the generator (`X` depends on your VS version), while the `Optional toolset to use` must be empty. However, new CMake versions require you to select only the VS version as the generator, e.g., `Visual Studio 16 2019`, and then you must manually choose `x64` for the `Optional platform for generator`. See the following images as example.
4. Enabling Python (optional step, only apply it if you plan to use the Python API): Enable the `BUILD_PYTHON` flag and click `Configure` again.
...
...
@@ -130,8 +133,8 @@ cmake-gui ..
6. If this step is successful, the `Configuring done` text will appear in the bottom box in the last line. Otherwise, some red text will appear in that same bottom box.
7. Press the `Generate` button and proceed to [Compilation](#compilation). You can now close CMake.
...
...
@@ -203,7 +206,7 @@ We welcome users to send us their installation videos (e.g., sharing them as Git
### Running OpenPose
Check OpenPose was properly installed by running any demo example: [doc/quick_start.md#quick-start](quick_start.md#quick-start).
Check OpenPose was properly installed by running any demo example: [doc/demo_quick_start.md](demo_quick_start.md).
...
...
@@ -349,7 +352,7 @@ However, the OpenPose Unity version will crash if if faces an error while it is
### Compiling without cuDNN
The [cuDNN](https://developer.nvidia.com/cudnn) library is not mandatory, but required for full keypoint detection accuracy. In case your graphics card is not compatible with cuDNN, you can disable it by unchecking `USE_CUDNN` in CMake.
Then, you would have to reduce the `--net_resolution` flag to fit the model into the GPU memory. You can try values like `640x320`, `320x240`, `320x160`, or `160x80` to see your GPU memory capabilities. After finding the maximum approximate resolution that your GPU can handle without throwing an out-of-memory error, adjust the `net_resolution` ratio to your image or video to be processed (see the `--net_resolution` explanation from [doc/demo_overview.md](demo_overview.md)), or use `-1` (e.g., `--net_resolution -1x320`).
Then, you would have to reduce the `--net_resolution` flag to fit the model into the GPU memory. You can try values like `640x320`, `320x240`, `320x160`, or `160x80` to see your GPU memory capabilities. After finding the maximum approximate resolution that your GPU can handle without throwing an out-of-memory error, adjust the `net_resolution` ratio to your image or video to be processed (see the `--net_resolution` explanation from [doc/demo_not_quick_start.md](demo_not_quick_start.md)), or use `-1` (e.g., `--net_resolution -1x320`).
...
...
@@ -358,7 +361,7 @@ OpenPose uses a [custom fork of Caffe](https://github.com/CMU-Perceptual-Computi
Alternatively, you can use your own Caffe distribution on Ubuntu/Mac by 1) disabling `BUILD_CAFFE`, 2) setting `Caffe_INCLUDE_DIRS` to `{CAFFE_PATH}/include/caffe`, and 3) setting `Caffe_LIBS` to `{CAFFE_PATH}/build/lib/libcaffe.so`, as shown in the image below. Note that cuDNN-compatible Caffe version is required in order to get the maximum possible accuracy in OpenPose.
@@ -154,7 +154,7 @@ Note: This installer will not incorporate any new features, we recommend to use
3. Open the Windows cmd (Windows button + <kbd>X</kbd>, then <kbd>A</kbd>).
4. Go to the OpenPose directory, assuming OpenPose has been downloaded on `C:\openpose`: `cd C:\openpose\`.
5. Run the tutorial commands.
6. Check OpenPose was properly installed by running it on the default images, video or webcam: [doc/quick_start.md#quick-start](../../quick_start.md#quick-start).
6. Check OpenPose was properly installed by running it on the default images, video or webcam: [doc/demo_quick_start.md](../../demo_quick_start.md).
The visual GUI should show the original image with the poses blended on it, similarly to the pose of this gif:
<palign="center">
<imgsrc="../.github/media/shake.gif",width="720">
</p>
If you choose to visualize a body part or a PAF (Part Affinity Field) heat map with the command option `--part_to_show`, the result should be similar to one of the following images:
@@ -5,7 +5,7 @@ OpenPose Library - All Major Released Features
- Jan 2019: [**Unity plugin released**](https://github.com/CMU-Perceptual-Computing-Lab/openpose_unity_plugin)!
- Jan 2019: [**Improved Python API**](doc/python_module.md) released! Including body, face, hands, and all the functionality of the C++ API!
- Dec 2018: [**Foot dataset released**](https://cmu-perceptual-computing-lab.github.io/foot_keypoint_dataset) and [**new paper released**](https://arxiv.org/abs/1812.08008)!
@@ -48,7 +48,7 @@ Some speed tips to maximize the OpenPose runtime speed while preserving the accu
## Speed Up and Memory Reduction
Some speed tips to highly maximize the OpenPose speed, but keep in mind the accuracy trade-off:
1. Reduce the `--net_resolution` (e.g., to 320x176) (lower accuracy). Note: For maximum accuracy, follow [doc/quick_start.md#maximum-accuracy-configuration](quick_start.md#maximum-accuracy-configuration).
1. Reduce the `--net_resolution` (e.g., to 320x176) (lower accuracy). Note: For maximum accuracy, follow [doc/demo_quick_start.md#maximum-accuracy-configuration](demo_quick_start.md#maximum-accuracy-configuration).
2. For face, reduce the `--face_net_resolution`. The resolution 320x320 usually works pretty decently.
3. Points 1-2 will also reduce the GPU memory usage (or RAM memory for CPU version).
4. Use the `BODY_25` model for maximum speed. Use `MPI_4_layers` model for minimum GPU memory usage (but lower accuracy, speed, and number of parts).
