- Jul 2017: **Windows**, New [**portable demo**](doc/installation.md#installation---demo)**and**[**easier library installation**](doc/installation.md#installation---library)!
- Jul 2017: **Hands** released!
...
...
@@ -14,6 +16,8 @@ OpenPose is a **library for real-time multi-person keypoint detection and multi-
- Apr 2017: **Body** released!
- Check all the [release notes](doc/release_notes.md).
## Results
### Body Estimation
<palign="center">
...
...
@@ -30,30 +34,32 @@ OpenPose is a **library for real-time multi-person keypoint detection and multi-
<imgsrc="doc/media/pose_hands.gif",width="480">
</p>
## Contents
1.[Latest News](#latest-news)
2.[Results](#results)
3.[Introduction](#introduction)
4.[Functionality](#functionality)
5.[Operating Systems](#operating-systems)
6.[Installation, Reinstallation and Uninstallation](#installation-reinstallation-and-uninstallation)
7.[Quick Start](#quick-start)
5.[Installation, Reinstallation and Uninstallation](#installation-reinstallation-and-uninstallation)
6.[Quick Start](#quick-start)
1.[Demo](#demo)
2.[OpenPose Wrapper](#openpose-wrapper)
3.[OpenPose Library](#openpose-library)
8.[Output](#output)
9.[Standalone Face Or Hand Keypoint Detector](#standalone-face-or-hand-keypoint-detector)
10.[Speed Up Openpose And Benchmark](#speed-up-openpose-and-benchmark)
11.[Send Us Failure Cases!](#send-us-failure-cases)
12.[Send Us Your Feedback!](#send-us-your-feedback)
13.[Citation](#citation)
14.[Other Contributors](#other-contributors)
7.[Output](#output)
8.[Standalone Face Or Hand Keypoint Detector](#standalone-face-or-hand-keypoint-detector)
9.[Speed Up Openpose And Benchmark](#speed-up-openpose-and-benchmark)
10.[Send Us Failure Cases!](#send-us-failure-cases)
11.[Send Us Your Feedback!](#send-us-your-feedback)
12.[Citation](#citation)
12.[Other Contributors](#other-contributors)
## Introduction
OpenPose represents the **first real-time system to jointly detect human body, hand and facial keypoints (in total 130 keypoints) on single images**. In addition, the system computational performance on body keypoint estimation is invariant to the number of detected people in the image. It uses Caffe, but it could easily be ported to other frameworks (Tensorflow, Torch, etc.). If you implement any of those, feel free to make a pull request!
OpenPose is authored by [Gines Hidalgo](http://gines-hidalgo.site123.me/), [Zhe Cao](http://www.andrew.cmu.edu/user/zhecao), [Tomas Simon](http://www.cs.cmu.edu/~tsimon/), [Shih-En Wei](https://scholar.google.com/citations?user=sFQD3k4AAAAJ&hl=en), [Hanbyul Joo](http://www.cs.cmu.edu/~hanbyulj/) and [Yaser Sheikh](http://www.cs.cmu.edu/~yaser/).
OpenPose is authored by [Gines Hidalgo](http://gines-hidalgo.site123.me/), [Zhe Cao](http://www.andrew.cmu.edu/user/zhecao), [Tomas Simon](http://www.cs.cmu.edu/~tsimon/), [Shih-En Wei](https://scholar.google.com/citations?user=sFQD3k4AAAAJ&hl=en), [Hanbyul Joo](http://www.cs.cmu.edu/~hanbyulj/) and [Yaser Sheikh](http://www.cs.cmu.edu/~yaser/). In addition, it is being maintained by [Gines Hidalgo](http://gines-hidalgo.site123.me/) and [Bikramjot Singh Hanzra](https://www.linkedin.com/in/bikz05).
It is freely available for free non-commercial use, and may be redistributed under these conditions. Please, see the [license](LICENSE) for further details. [Interested in a commercial license? Check this link](https://flintbox.com/public/project/47343/). For commercial queries, contact [Yaser Sheikh](http://www.cs.cmu.edu/~yaser/).
...
...
@@ -63,28 +69,14 @@ The pose estimation work is based on the C++ code from [the ECCV 2016 demo](http
## Functionality
* Multi-person 15 or **18-keypoint body pose** estimation and rendering. **Running time invariant to number of people** on the image.
* Multi-person **2x21-keypoint hand** estimation and rendering. Note: In this initial version, **running time** linearly **depends** on the **number of people** on the image. **Coming soon (in around 1-5 days)!**
* Multi-person **70-keypoint face** estimation and rendering. Note: In this initial version, **running time** linearly **depends** on the **number of people** on the image.
* Flexible and easy-to-configure **multi-threading** module.
* Image, video, and webcam reader.
* Able to save and load the results in various formats (JSON, XML, PNG, JPG, ...).
* Small display and GUI for simple result visualization.
* All the functionality is wrapped into a **simple-to-use OpenPose Wrapper class**.
## Operating Systems
1.**Ubuntu** 14 and 16.
2.**Windows** 8 and 10.
3. OpenPose has also been used on **Windows 7**, **Mac**, **CentOS**, and **Nvidia Jetson (TK1 and TX1)** embedded systems. However, we do not officially support them at the moment.
- Multi-person 15 or **18-keypoint body pose** estimation and rendering. **Running time invariant to number of people** on the image.
- Multi-person **2x21-keypoint hand** estimation and rendering. Note: In this initial version, **running time** linearly **depends** on the **number of people** on the image. **Coming soon (in around 1-5 days)!**
- Multi-person **70-keypoint face** estimation and rendering. Note: In this initial version, **running time** linearly **depends** on the **number of people** on the image.
- Flexible and easy-to-configure **multi-threading** module.
- Image, video, and webcam reader.
- Able to save and load the results in various formats (JSON, XML, PNG, JPG, ...).
- Small display and GUI for simple result visualization.
- All the functionality is wrapped into a **simple-to-use OpenPose Wrapper class**.
...
...
@@ -97,14 +89,13 @@ You can find the installation, reinstallation and uninstallation steps on: [doc/
Most users cases should not need to dive deep into the library, they might just be able to use the [Demo](#demo) or the simple [OpenPose Wrapper](#openpose-wrapper). So you can most probably skip the library details in [OpenPose Library](#openpose-library).
#### Demo
### Demo
Your case if you just want to process a folder of images or video or webcam and display or save the pose results.
Forget about the OpenPose library details and just read the [doc/demo_overview.md](doc/demo_overview.md) 1-page section.
#### OpenPose Wrapper
### OpenPose Wrapper
Your case if you want to read a specific format of image source and/or add a specific post-processing function and/or implement your own display/saving.
(Almost) forget about the library, just take a look to the `Wrapper` tutorial on [examples/tutorial_wrapper/](examples/tutorial_wrapper/).
...
...
@@ -112,7 +103,7 @@ Your case if you want to read a specific format of image source and/or add a spe
Note: you should not need to modify the OpenPose source code nor examples. In this way, you are able to directly upgrade OpenPose anytime in the future without changing your code. You might create your custom code on [examples/user_code/](examples/user_code/) and compile it by using `make all` in the OpenPose folder.
#### OpenPose Library
### OpenPose Library
Your case if you want to change internal functions and/or extend its functionality. First, take a look at the [Demo](#demo) and [OpenPose Wrapper](#openpose-wrapper). Second, read the 2 following subsections: OpenPose Overview and Extending Functionality.
1. OpenPose Overview: Learn the basics about the library source code in [doc/library_overview.md](doc/library_overview.md).
...
...
@@ -122,7 +113,7 @@ Your case if you want to change internal functions and/or extend its functionali
3. Adding An Extra Module: Learn how to add an extra module in [doc/library_add_new_module.md](doc/library_add_new_module.md).
#### Doxygen Documentation Autogeneration
### Doxygen Documentation Autogeneration
You can generate the documentation by running the following command. The documentation will be generated in `doc/doxygen/html/index.html`. You can simply open it with double-click (your default browser should automatically display it).
2.[Clone and Update the Repository](#clone-and-update-the-repository)
3.[Ubuntu](#ubuntu)
4.[Windows](#windows)
5.[OpenPose 3D Demo](#openpose-3d-demo)
6.[Custom Caffe](#custom-caffe)
7.[FAQ](#faq)
1.[Operating Systems](#operating-systems)
2.[Requirements](#requirements)
3.[Clone and Update the Repository](#clone-and-update-the-repository)
4.[Ubuntu](#ubuntu)
5.[Windows](#windows)
6.[OpenPose 3D Demo](#openpose-3d-demo)
7.[Custom Caffe](#custom-caffe)
8.[Compiling without cuDNN](#compiling-without-cudnn)
9.[FAQ](#faq)
## Operating Systems
-**Ubuntu** 14 and 16.
-**Windows** 8 and 10.
-**Nvidia Jetson TX2**, installation instructions in [doc/installation_jetson_tx2](./installation_jetson_tx2).
- OpenPose has also been used on **Windows 7**, **Mac**, **CentOS**, and **Nvidia Jetson (TK1 and TX1)** embedded systems. However, we do not officially support them at the moment.
## Requirements
- Operating systems:
- Ubuntu (tested on 14 and 16).
- Windows (tested on 10).
- Nvidia Jetson TX2, installation instructions in [doc/installation_jetson_tx2](./installation_jetson_tx2).
- We do not officially support any other OS, but the community has been able to install it on: CentOS, Nvidia Jetson, Windows 7, and Windows 8.
- NVIDIA graphics card with at least 1.6 GB available (the `nvidia-smi` command checks the available GPU memory in Ubuntu).
- At least 2 GB of free RAM memory.
- Highly recommended: A CPU with at least 8 cores.
- Highly recommended: cuDNN and a CPU with at least 8 cores.
Note: These requirements assume the default configuration (i.e. `--net_resolution "656x368"` and `scale_number 1`). You might need more (with a greater net resolution and/or number of scales) or less resources (with smaller net resolution and/or using the MPI and MPI_4 models).
...
...
@@ -38,12 +45,14 @@ OpenPose can be easily updated by clicking the `synchronization` button at the t
## Ubuntu
### Installation - Script Compilation
**Highly important**: This script only works with CUDA 8 and Ubuntu 14 or 16. Otherwise, check [Manual Compilation](#manual-compilation).
1. Required: CUDA, cuDNN, OpenCV and Atlas must be already installed on your machine.
1.[CUDA](https://developer.nvidia.com/cuda-downloads) must be installed. You should reboot your machine after installing CUDA.
2.[cuDNN](https://developer.nvidia.com/cudnn): Once you have downloaded it, just unzip it and copy (merge) the contents on the CUDA folder, e.g. `/usr/local/cuda-8.0/`. Note: We found OpenPose working ~10% faster with cuDNN 5.1 compared to cuDNN 6.
2.[cuDNN](https://developer.nvidia.com/cudnn): Once you have downloaded it, just unzip it and copy (merge) the contents on the CUDA folder, e.g. `/usr/local/cuda-8.0/`. Note: We found OpenPose working ~10% faster with cuDNN 5.1 compared to cuDNN 6. Otherwise, check [Compiling without cuDNN](#compiling-without-cudnn).
3. OpenCV can be installed with `apt-get install libopencv-dev`. If you have compiled OpenCV 3 by your own, follow [Manual Compilation](#manual-compilation). After both Makefile.config files have been generated, edit them and uncomment the line `# OPENCV_VERSION := 3`. You might alternatively modify all `Makefile.config.UbuntuXX` files and then run the scripts in step 2.
4. In addition, OpenCV 3 does not incorporate the `opencv_contrib` module by default. Assuming you have OpenCV 3 compiled with the contrib module and you want to use it, append `opencv_contrib` at the end of the line `LIBRARIES += opencv_core opencv_highgui opencv_imgproc` in the `Makefile` file.
5. Atlas can be installed with `sudo apt-get install libatlas-base-dev`. Instead of Atlas, you can use OpenBLAS or Intel MKL by modifying the line `BLAS := atlas` in the same way as previosuly mentioned for the OpenCV version selection.
...
...
@@ -162,22 +171,14 @@ If you updated some software that our library or 3rdparty use, or you simply wan
## Compiling without cuDNN
The [cuDNN](https://developer.nvidia.com/cudnn) library is not mandatory, but required for full keypoint detection accuracy. In case your graphics card is not compatible with cuDNN, you can disable it by:
- Ubuntu: Modifying the `Makefile.config` files in both the OpenPose and `3rdparty/caffe` folders.
- Windows: Modifying the `Makefile.config` files in both the OpenPose and `3rdparty/caffe` folders.
Then, you would have to reduce the `--net_resolution` flag to fit the model into the GPU memory. You can try values like "640x320", "320x240", "320x160", or "160x80" to see your GPU memory capabilities. After finding the maximum approximate resolution that your GPU can handle without throwing an out-of-memory error, adjust the `net_resolution` ratio to your image or video to be processed (see the `--net_resolution` explanation from [doc/demo_overview.md](./demo_overview.md)).
## OpenPose 3D Demo
If you want to try our OpenPose 3-D reconstruction demo, see [doc/openpose_3d_reconstruction_demo.md](./openpose_3d_reconstruction_demo.md).
## Custom Caffe
We only modified some Caffe compilation flags and minor details. You can use your own Caffe distribution, these are the files we added and modified:
...
...
@@ -193,6 +194,18 @@ We only modified some Caffe compilation flags and minor details. You can use you
## Compiling without cuDNN
The [cuDNN](https://developer.nvidia.com/cudnn) library is not mandatory, but required for full keypoint detection accuracy. In case your graphics card is not compatible with cuDNN, you can disable it by:
- Ubuntu: Modifying the `Makefile.config` files in both the OpenPose and `3rdparty/caffe` folders, disabling `USE_CUDNN`.
- Windows: Compiling Caffe by your own with without cuDNN support and removing the `USE_CUDNN` define from the OpenPose project solution in Visual Studio.
Then, you would have to reduce the `--net_resolution` flag to fit the model into the GPU memory. You can try values like "640x320", "320x240", "320x160", or "160x80" to see your GPU memory capabilities. After finding the maximum approximate resolution that your GPU can handle without throwing an out-of-memory error, adjust the `net_resolution` ratio to your image or video to be processed (see the `--net_resolution` explanation from [doc/demo_overview.md](./demo_overview.md)).
## FAQ
**Q: Out of memory error** - I get an error similar to: `Check failed: error == cudaSuccess (2 vs. 0) out of memory`.
