From 691693aa38546e65081f9f485293675864e52ee4 Mon Sep 17 00:00:00 2001 From: gineshidalgo99 Date: Sat, 8 Jul 2017 13:34:49 -0400 Subject: [PATCH] Improved doc --- README.md | 20 +++++--- doc/contributors.md | 4 +- doc/installation.md | 17 ++++--- doc/output.md | 64 +++++++++++++++++++++++++ doc/release_notes.md | 23 ++++----- include/openpose/gui/frameDisplayer.hpp | 2 +- 6 files changed, 101 insertions(+), 29 deletions(-) diff --git a/README.md b/README.md index a910f053..ad010928 100644 --- a/README.md +++ b/README.md @@ -5,13 +5,20 @@ OpenPose - Apr 2017: Body released! - May 2017: Windows version released! - Jun 2017: Face released! +- Jul 2017: Easier Windows installation! - Check all the [release notes](doc/release_notes.md). - Interested in an internship on CMU as OpenPose programmer? See [this link](https://docs.google.com/document/d/14SygG39NjIRZfx08clewTdFMGwVdtRu2acyCi3TYcHs/edit?usp=sharing) for details. -## Introduction +## Operating Systems +1. **Ubuntu** 14 and 16. +2. **Windows** 10. +3. Other people have been able to install it on **Windows 7 and 8**, **Mac**, **CentOS**, and **Nvidia Jetson (TK1 and TX1)** embedded systems. However, we do not support them at the moment. + + +## Introduction OpenPose is a **library for real-time multi-person keypoint detection and multi-threading written in C++** using OpenCV and Caffe*, authored by [Gines Hidalgo](https://www.linkedin.com/in/gineshidalgo/), [Zhe Cao](http://www.andrew.cmu.edu/user/zhecao), [Tomas Simon](http://www.cs.cmu.edu/~tsimon/), [Shih-En Wei](https://scholar.google.com/citations?user=sFQD3k4AAAAJ&hl=en), [Hanbyul Joo](http://www.cs.cmu.edu/~hanbyulj/) and [Yaser Sheikh](http://www.cs.cmu.edu/~yaser/). \* It uses Caffe, but the code is ready to be ported to other frameworks (Tensorflow, Torch, etc.). If you implement any of those, feel free to make a pull request! @@ -20,13 +27,15 @@ OpenPose represents the **first real-time system to jointly detect human body, h OpenPose is freely available for free non-commercial use, and may be redistributed under these conditions. Please, see the [license](LICENSE) for further details. Contact [Yaser Sheikh](http://www.cs.cmu.edu/~yaser/) for commercial purposes. +In addition, OpenPose would not be possible without the [CMU Panoptic Studio](http://domedb.perception.cs.cmu.edu/). + Library main functionality: * Multi-person 15 or **18-keypoint body pose** estimation and rendering. **Running time invariant to number of people** on the image. -* Multi-person **2x21-keypoint hand** estimation and rendering. Note: In this initial version, **running time** linearly **depends** on the **number of people** on the image. **Coming soon (in around 1-5 weeks)!** +* Multi-person **2x21-keypoint hand** estimation and rendering. Note: In this initial version, **running time** linearly **depends** on the **number of people** on the image. **Coming soon (in around 1-5 days)!** * Multi-person **70-keypoint face** estimation and rendering. Note: In this initial version, **running time** linearly **depends** on the **number of people** on the image. @@ -45,7 +54,6 @@ The pose estimation work is based on the C++ code from [the ECCV 2016 demo](http ## Results - ### Body Estimation

@@ -90,7 +98,7 @@ You can find the installation, reinstallation and uninstallation steps on: [doc/ ## Custom Caffe -We only modified some Caffe compilation flags and minor details. You can use use your own Caffe distribution, these are the files we added and modified: +We only modified some Caffe compilation flags and minor details. You can use your own Caffe distribution, these are the files we added and modified: 1. Added files: `install_caffe.sh`; as well as `Makefile.config.Ubuntu14.example`, `Makefile.config.Ubuntu16.example`, `Makefile.config.Ubuntu14_cuda_7.example` and `Makefile.config.Ubuntu16_cuda_7.example` (extracted from `Makefile.config.example`). Basically, you must enable cuDNN. 2. Edited file: Makefile. Search for "# OpenPose: " to find the edited code. We basically added the C++11 flag to avoid issues in some old computers. @@ -117,7 +125,7 @@ Your case if you want to read a specific format of image source and/or add a spe (Almost) forget about the library, just take a look to the `Wrapper` tutorial on [examples/tutorial_wrapper/](examples/tutorial_wrapper/). -Note: you should not need to modify OpenPose source code or examples, so that you can directly upgrade the OpenPose library anytime in the future without changing your code. You might create your custom code on [examples/user_code/](examples/user_code/) and compile it by using `make all` in the OpenPose folder. +Note: you should not need to modify the OpenPose source code nor examples. In this way, you are able to directly upgrade OpenPose anytime in the future without changing your code. You might create your custom code on [examples/user_code/](examples/user_code/) and compile it by using `make all` in the OpenPose folder. #### OpenPose Library Your case if you want to change internal functions and/or extend its functionality. First, take a look at the [Demo](#demo) and [OpenPose Wrapper](#openpose-wrapper). Second, read the 2 following subsections: OpenPose Overview and Extending Functionality. @@ -129,7 +137,7 @@ Your case if you want to change internal functions and/or extend its functionali 3. Adding An Extra Module: Learn how to add an extra module in [doc/library_add_new_module.md](doc/library_add_new_module.md). #### Doxygen Documentation Autogeneration -You can generate the documentation by running the following command. The documentation will be generated in `doc/doxygen/html/index.html`. You can simply open it with double click (your default browser should automatically display it). +You can generate the documentation by running the following command. The documentation will be generated in `doc/doxygen/html/index.html`. You can simply open it with double-click (your default browser should automatically display it). ``` cd doc/ doxygen doc_autogeneration.doxygen diff --git a/doc/contributors.md b/doc/contributors.md index 8af73823..3b283937 100644 --- a/doc/contributors.md +++ b/doc/contributors.md @@ -9,6 +9,6 @@ OpenPose is authored by [Gines Hidalgo](https://www.linkedin.com/in/gineshidalgo ### Contributors -We would like to thank the following people who have highly contributed to OpenPose: +We would also like to thank the following people who have highly contributed to OpenPose: -1. [Helen Medina](https://github.com/helen-medina): Windows branch creator and making OpenPose stable on Windows. +1. [Helen Medina](https://github.com/helen-medina): Original Windows branch creator. She did the first stable OpenPose version on Windows. diff --git a/doc/installation.md b/doc/installation.md index 7ad4cfcb..14073a52 100644 --- a/doc/installation.md +++ b/doc/installation.md @@ -131,7 +131,7 @@ You just need to remove the OpenPose folder, by default called `openpose/`. E.g. -### Installation - Library - Deprecated Old Version +### Installation - Library - Old Deprecated Version Note: This version will not be updated anymore and removed in the future. 1. Install the pre-requisites: @@ -142,7 +142,7 @@ Note: This version will not be updated anymore and removed in the future. - In addition, open the Windows cmd (Windows button + X, then A), and install some Python libraries with this command: `pip install numpy protobuf hypothesis`. 4. [Cmake](https://cmake.org/download/): Select the option to add it to the Windows PATH. 5. [Ninja](https://ninja-build.org/): Select the option to add it to the Windows PATH. -2. Download the `Windows` branch of Openpose by either cliking on `Download ZIP` on [openpose/tree/windows](https://github.com/CMU-Perceptual-Computing-Lab/openpose/tree/windows) or cloning the repository: `git clone https://github.com/CMU-Perceptual-Computing-Lab/openpose/ && cd openpose && git checkout windows`. +2. Download the `Windows` branch of Openpose by either cliking on `Download ZIP` on [openpose/tree/windows](https://github.com/CMU-Perceptual-Computing-Lab/openpose/tree/windows) or cloning the repository: `git clone https://github.com/CMU-Perceptual-Computing-Lab/openpose/ && cd openpose && git checkout old_deprecated_windows`. 3. Install Caffe on Windows: 1. Open the Windows cmd (Windows button + X, then A). 2. Go to the Caffe directory, assuming OpenPose has been downloaded on `C:\openpose`: `cd C:\openpose\3rdparty\caffe\caffe-windows`. @@ -177,11 +177,11 @@ Check that the library is working properly by using any of the following command ``` ``` :: Windows - Demo -bin\OpenPoseDemo.exe --video examples/media/video.avi +bin\OpenPoseDemo.exe --video examples\media\video.avi ``` ``` :: Windows - Library -windows_project\x64\Release\OpenPoseDemo.exe --video examples/media/video.avi +windows_project\x64\Release\OpenPoseDemo.exe --video examples\media\video.avi ``` **2. Running on Webcam** @@ -205,11 +205,11 @@ windows_project\x64\Release\OpenPoseDemo.exe ``` ``` :: Windows - Demo -bin\OpenPoseDemo.exe --image_dir examples/media/ +bin\OpenPoseDemo.exe --image_dir examples\media\ ``` ``` :: Windows - Library -windows_project\x64\Release\OpenPoseDemo.exe --image_dir examples/media/ +windows_project\x64\Release\OpenPoseDemo.exe --image_dir examples\media\ ``` **4. Body Pose and Face Keypoint Detector (e.g. on Video)** @@ -219,12 +219,11 @@ windows_project\x64\Release\OpenPoseDemo.exe --image_dir examples/media/ ``` ``` :: Windows - Demo -bin\OpenPoseDemo.exe --video examples/media/video.avi --face +bin\OpenPoseDemo.exe --video examples\media\video.avi --face ``` ``` :: Windows - Library -windows_project\x64\Release\OpenPoseDemo.exe --video examples/media/video.avi --face -``` +windows_project\x64\Release\OpenPoseDemo.exe --video examples\media\video.avi --face The visual GUI should show the original image with the poses blended on it, similarly to the pose of this gif:

diff --git a/doc/output.md b/doc/output.md index a0baa8ac..fc1bd860 100644 --- a/doc/output.md +++ b/doc/output.md @@ -55,6 +55,70 @@ The output format is analogous for hand (`hand_left_keypoints` and `hand_right_k +## Keypoint Format on the op::Datum Class +There are 3 different keypoint Array elements on this class: + +1. Array **poseKeypoints**: In order to access person `person` and body part `part` (where the index matches `POSE_COCO_BODY_PARTS` or `POSE_MPI_BODY_PARTS`), you can simply output: +``` + // Common parameters needed + const auto numberPeopleDetected = poseKeypoints.getSize(0); + const auto numberBodyParts = poseKeypoints.getSize(1); + // Easy version + const auto x = poseKeypoints[{person, part, 0}]; + const auto y = poseKeypoints[{person, part, 1}]; + const auto score = poseKeypoints[{person, part, 2}]; + // Slightly more efficient version + // If you want to access these elements on a huge loop, it is slightly faster (but usually not faster enough to be worthy) to get the index by your own + const auto baseIndex = poseKeypoints.getSize(2)*(person*numberBodyParts + part); + const auto x = poseKeypoints[baseIndex]; + const auto y = poseKeypoints[baseIndex + 1]; + const auto score = poseKeypoints[baseIndex + 2]; +``` +2. Array **faceKeypoints**: It is completely analogous to poseKeypoints. +``` + // Common parameters needed + const auto numberPeopleDetected = faceKeypoints.getSize(0); + const auto numberFaceParts = faceKeypoints.getSize(1); + // Easy version + const auto x = faceKeypoints[{person, part, 0}]; + const auto y = faceKeypoints[{person, part, 1}]; + const auto score = faceKeypoints[{person, part, 2}]; + // Slightly more efficient version + const auto baseIndex = faceKeypoints.getSize(2)*(person*numberFaceParts + part); + const auto x = faceKeypoints[baseIndex]; + const auto y = faceKeypoints[baseIndex + 1]; + const auto score = faceKeypoints[baseIndex + 2]; +``` +3. std::array, 2> **handKeypoints**, where handKeypoints[0] corresponds to the left hand and handKeypoints[1] to the right one. Each handKeypoints[i] is analogous to poseKeypoints and faceKeypoints: +``` + // Common parameters needed + const auto numberPeopleDetected = handKeypoints[0].getSize(0); // = handKeypoints[1].getSize(0) + const auto numberHandParts = handKeypoints[0].getSize(1); // = handKeypoints[1].getSize(1) + + // Easy version + // Left Hand + const auto xL = handKeypoints[0][{person, part, 0}]; + const auto yL = handKeypoints[0][{person, part, 1}]; + const auto scoreL = handKeypoints[0][{person, part, 2}]; + // Right Hand + const auto xR = handKeypoints[1][{person, part, 0}]; + const auto yR = handKeypoints[1][{person, part, 1}]; + const auto scoreR = handKeypoints[1][{person, part, 2}]; + + // Slightly more efficient version + const auto baseIndex = handKeypoints[0].getSize(2)*(person*numberHandParts + part); + // Left Hand + const auto xL = handKeypoints[0][baseIndex]; + const auto yL = handKeypoints[0][baseIndex + 1]; + const auto scoreL = handKeypoints[0][baseIndex + 2]; + // Right Hand + const auto xR = handKeypoints[1][baseIndex]; + const auto yR = handKeypoints[1][baseIndex + 1]; + const auto scoreR = handKeypoints[1][baseIndex + 2]; +``` + + + ## Reading Saved Results We use standard formats (JSON, XML, PNG, JPG, ...) to save our results, so there will be lots of frameworks to read them later, but you might also directly use our functions in [include/openpose/filestream.hpp](../include/openpose/filestream.hpp). In particular, `loadData` (for JSON, XML and YML files) and `loadImage` (for image formats such as PNG or JPG) to load the data into cv::Mat format. diff --git a/doc/release_notes.md b/doc/release_notes.md index ea88adae..bfffb393 100644 --- a/doc/release_notes.md +++ b/doc/release_notes.md @@ -46,17 +46,18 @@ OpenPose Library - Release Notes ## OpenPose 1.0.0 1. Main improvements: - 1. Hands and face now use `Maximum` instead of `Nms`, since there is only 1 person / detection. - 2. Increased accuracy on multi-scale (added `Datum::scaleRatios` to save the relative scale ratio when multi-scale). - 3. Increased speed ~5% by adding CPU rendering (but GPU is the default rendering). - 4. Rendering colors modified, visually better results. - 5. Check() functions give more feedback. - 6. WCocoJsonSaver finished and removed its 3599-image limit. - 7. Added `camera_fps` so generated video will use that frame rate. - 8. Reduced the number of printed information messages. Default logging priority threshold increased to Priority::Max. - 9. Google flags to OpenPose configuration parameters reader moved from each demo to utilities/flagsToOpenPose. - 10. Nms classes do not use `numberParts` for `Reshape`, they deduce the value. - 11. Improved documentation. + 1. Windows branch merged to master branch. + 2. Face and hands use `Maximum` instead of `Nms`, since there is only 1 person / detection. + 3. Increased accuracy on multi-scale (added `Datum::scaleRatios` to save the relative scale ratio when multi-scale). + 4. Increased speed ~5% by adding CPU rendering (but GPU is the default rendering). + 5. Rendering colors modified, visually better results. + 6. Check() functions give more feedback. + 7. WCocoJsonSaver finished and removed its 3599-image limit. + 8. Added `camera_fps` so generated video will use that frame rate. + 9. Reduced the number of printed information messages. Default logging priority threshold increased to Priority::Max. + 10. Google flags to OpenPose configuration parameters reader moved from each demo to utilities/flagsToOpenPose. + 11. Nms classes do not use `numberParts` for `Reshape`, they deduce the value. + 12. Improved documentation. 2. Functions or parameters renamed: 1. Render flags renamed in the demo in order to incorporate the CPU/GPU rendering. 2. Keypoints saved in JSON files (`write_keypoint_json`) are now saved as `pose_keypoints`, `face_keypoints`, `hand_left_keypoints`, and `hand_right_keypoints`. They all were previously saved as `body_parts`. diff --git a/include/openpose/gui/frameDisplayer.hpp b/include/openpose/gui/frameDisplayer.hpp index a3e6a9b9..58e58087 100644 --- a/include/openpose/gui/frameDisplayer.hpp +++ b/include/openpose/gui/frameDisplayer.hpp @@ -32,7 +32,7 @@ namespace op void setGuiDisplayMode(const GuiDisplayMode displayMode); /** - * This function switch between full screen and windowed modes (e.g. when double click on video players or Ctrt+Enter are presed). + * This function switch between full screen and windowed modes (e.g. when double-click on video players or Ctrt+Enter are presed). */ void switchGuiDisplayMode(); -- GitLab