提交 896167d3 编写于 作者: G gineshidalgo99

Doc improved, Mac bug fixed

上级 4c4909c1
......@@ -36,13 +36,13 @@
## Latest Features
- Jun 2018: [**Combined body-foot model released! Faster and more accurate.**](doc/installation.md)!
- Jun 2018: [**Python version**](doc/python_module.md) released!
- Jun 2018: [**Combined body-foot model released! Faster and more accurate**](doc/installation.md)!
- Jun 2018: [**Python version**](doc/modules/python_module.md) released!
- Jun 2018: [**AMD graphic card version**](doc/installation.md) released!
- Jun 2018: [**Calibration toolbox**](doc/calibration_demo.md) released!
- Jun 2018: [**Calibration toolbox**](doc/modules/calibration_module.md) released!
- Jun 2018: [**Mac OSX version (CPU)**](doc/installation.md) released!
- Mar 2018: [**CPU version**](doc/installation.md#cpu-version)!
- Mar 2018: [**3-D keypoint reconstruction module**](doc/3d_reconstruction_demo.md) (from multiple camera views)!
- Mar 2018: [**3-D keypoint reconstruction module**](doc/modules/3d_reconstruction_module.md) (from multiple camera views)!
- Sep 2017: [**CMake**](doc/installation.md) installer and **IP camera** support!
- Jul 2017: [**Windows portable binaries and demo**](https://github.com/CMU-Perceptual-Computing-Lab/openpose/releases)!
- Jul 2017: **Hands** released!
......@@ -67,19 +67,19 @@ For further details, check [all released features](doc/released_features.md) and
## Results
### 3-D Reconstruction Module
### Body-Foot Estimation
<p align="center">
<img src="doc/media/openpose3d.png", width="360">
<img src="doc/media/dance_foot.gif", width="360">
</p>
### Body Estimation
### Body, Face, and Hands Estimation
<p align="center">
<img src="doc/media/dance.gif", width="360">
<img src="doc/media/pose_face.gif", width="360">
</p>
### Body, Face, and Hands Estimation
### 3-D Reconstruction Module
<p align="center">
<img src="doc/media/pose_face.gif", width="360">
<img src="doc/media/openpose3d.gif", width="360">
</p>
### Body and Hands Estimation
......@@ -87,6 +87,11 @@ For further details, check [all released features](doc/released_features.md) and
<img src="doc/media/pose_hands.gif", width="360">
</p>
### Body Estimation
<p align="center">
<img src="doc/media/dance.gif", width="360">
</p>
## Installation, Reinstallation and Uninstallation
......@@ -107,9 +112,9 @@ Most users do not need the [OpenPose C++ API](#openpose-c-api), but they can sim
bin\OpenPoseDemo.exe --video examples\media\video.avi
```
- **Calibration toolbox**: To easily calibrate your cameras for 3-D OpenPose or any other stereo vision task. See [doc/calibration_demo.md](doc/calibration_demo.md).
- **Calibration toolbox**: To easily calibrate your cameras for 3-D OpenPose or any other stereo vision task. See [doc/modules/calibration_module.md](doc/modules/calibration_module.md).
- **OpenPose Wrapper**: If you want to read a specific input, and/or add your custom post-processing function, and/or implement your own display/saving, check the `Wrapper` tutorial on [examples/tutorial_wrapper/](examples/tutorial_wrapper/). You can create your custom code on [examples/user_code/](examples/user_code/) and quickly compile it by using `make all` in the OpenPose folder (assuming Makefile installer).
- **OpenPose Wrapper**: If you want to read a specific input, and/or add your custom post-processing function, and/or implement your own display/saving, check the `Wrapper` tutorial on [examples/tutorial_wrapper/](examples/tutorial_wrapper/). You can create your custom code on [examples/user_code/](examples/user_code/) and quickly compile it with CMake when compiling the whole OpenPose project. Quickly **add your custom code**: See [examples/user_code/README.md](examples/user_code/README.md) for further details.
- **OpenPose C++ API**: See [doc/library_introduction.md](doc/library_introduction.md).
......
......@@ -12,16 +12,17 @@ OpenPose - Installation
8. [Uninstallation](#uninstallation)
9. [Optional Settings](#optional-settings)
1. [Profiling Speed](#profiling-speed)
2. [COCO and MPI Model](#coco-and-mpi-model)
3. [CPU Version](#cpu-version)
4. [OpenCL Version](#opencl-version)
5. [3D Reconstruction Module](#3d-reconstruction-module)
6. [Calibration Module](#calibration-module)
7. [Compiling without cuDNN](#compiling-without-cudnn)
8. [Custom Caffe (Ubuntu Only)](#custom-caffe-ubuntu-only)
9. [Custom OpenCV (Ubuntu Only)](#custom-opencv-ubuntu-only)
10. [Doxygen Documentation Autogeneration (Ubuntu Only)](#doxygen-documentation-autogeneration-ubuntu-only)
11. [CMake Command Line Configuration (Ubuntu Only)](#cmake-command-line-configuration-ubuntu-only)
2. [COCO and MPI Models](#coco-and-mpi-models)
3. [Python API](#python-api)
4. [CPU Version](#cpu-version)
5. [OpenCL Version](#opencl-version)
6. [3D Reconstruction Module](#3d-reconstruction-module)
7. [Calibration Module](#calibration-module)
8. [Compiling without cuDNN](#compiling-without-cudnn)
9. [Custom Caffe (Ubuntu Only)](#custom-caffe-ubuntu-only)
10. [Custom OpenCV (Ubuntu Only)](#custom-opencv-ubuntu-only)
11. [Doxygen Documentation Autogeneration (Ubuntu Only)](#doxygen-documentation-autogeneration-ubuntu-only)
12. [CMake Command Line Configuration (Ubuntu Only)](#cmake-command-line-configuration-ubuntu-only)
......@@ -248,11 +249,16 @@ OpenPose displays the FPS in the basic GUI. However, more complex speed metrics
#### COCO and MPI Model
#### COCO and MPI Models
By default, the body COCO and MPI models are not downloaded. You can download them by turning on the `DOWNLOAD_BODY_COCO_MODEL` or `DOWNLOAD_BODY_MPI_MODEL` flags. It's slightly faster but less accurate and has less keypoints than the COCO body model.
#### Python API
To install the Python API, ensure that the `BUILD_PYTHON` flag is turned on while running CMake GUI. Check [doc/modules/python_module.md](./modules/python_module.md) for further details.
#### CPU Version
To manually select the CPU Version, open CMake GUI mentioned above, and set the `GPU_MODE` flag to `CPU_ONLY`. **NOTE: Accuracy of the CPU version is ~1% higher than CUDA version, so the results will vary.**
......@@ -305,14 +311,14 @@ You can include the 3D reconstruction module by:
3. Follow the CMake installation steps. In addition, set the `WITH_FLIR_CAMERA` (only if Spinnaker was installed) and `WITH_3D_RENDERER` options.
4. Increased accuracy with Ceres solver (Ubuntu only): For extra 3-D reconstruction accuracy, run `sudo apt-get install libeigen3-dev`, install [Ceres solver](http://ceres-solver.org/installation.html), and enable `WITH_CERES` in CMake when installing OpenPose. Ceres is harder to install in Windows, so we have not tested it so far in there. Feel free to make a pull request if you do.
After installation, check the [doc/3d_reconstruction_demo.md](./3d_reconstruction_demo.md) instructions.
After installation, check the [doc/modules/3d_reconstruction_module.md](./modules/3d_reconstruction_module.md) instructions.
#### Calibration Module
The calibration module is included by default, but you must also enable `WITH_EIGEN` if you intend to use the extrinsic camera parameter estimation tool. You can set that flag to 2 different values: `APT_GET` or `BUILD`, check [Requirements](#requirements) for more information.
After installation, check the [doc/calibration_demo.md](./calibration_demo.md) instructions.
After installation, check the [doc/modules/calibration_module.md](./modules/calibration_module.md) instructions.
......
......@@ -3,7 +3,7 @@
## Contents
1. [Introduction](#introduction)
2. [Installing the OpenPose 3-D Reconstruction Module](#installing-the-openpose-3-d-reconstruction-module)
2. [Installation](#installation)
3. [Non Linear Optimization](#non-linear-optimization)
4. [Features](#features)
5. [Required Hardware](#required-hardware)
......@@ -22,7 +22,7 @@ This module performs 3-D keypoint (body, face, and hand) reconstruction and rend
## Installing the OpenPose 3-D Reconstruction Module
## Installation
Check [doc/installation.md#3d-reconstruction-module](./installation.md#3d-reconstruction-module) for installation steps.
......@@ -70,7 +70,7 @@ The user must manually get the intrinsic and extrinsic parameters of the stereo-
There are 2 alternatives to calibrate the cameras:
1. Using the [OpenPose calibration toolbox](./calibration_demo.md#).
1. Using the [OpenPose calibration toolbox](./modules/calibration_module.md#).
2. Using your own calibration toolbox (or if you already know the camera parameters of your cameras):
1. Create a xml file for each camera named as `models/cameraParameters/flir/{camera_serial_number}.xml`.
2. The elements inside each xml file are the extrinsic parameters of the camera (`CameraMatrix`), the intrinsic parameters (`Intrinsics`), and the distortion coefficients (`Distortion`). Copy the format from `models/cameraParameters/flir/17012332.xml.example`. For the extrinsic parameters of the camera, it allows you to set the coordinate origin (so that 3-d keypoints are distances with respect to that origin).
......
......@@ -40,26 +40,26 @@ Note: In order to maximize calibration quality, **do not reuse the same video se
1. Webcam calibration: `./build/examples/openpose/openpose.bin --num_gpu 0 --frame_keep_distortion --write_images {intrinsic_images_folder_path}`.
2. Flir camera calibration: Add the flags `--flir_camera --flir_camera_index 0` (or the desired flir camera index) to the webcam command.
3. Calibration from video sequence: Add the flag `--video {video_path}` to the webcam command.
4. Any other camera brand: Simply save your images in {intrinsic_images_folder_path}, file names are not relevant.
4. Any other camera brand: Simply save your images in `{intrinsic_images_folder_path}`, file names are not relevant.
2. Get familiar with the calibration parameters used in point 3 (i.e., `grid_square_size_mm`, `grid_number_inner_corners`, etc.) by running the `--help` flag:
```
```sh
./build/examples/calibration/calibration.bin --help
```
3. Extract and save the intrinsic parameters:
```
```sh
./build/examples/calibration/calibration.bin --mode 1 --grid_square_size_mm 40.0 --grid_number_inner_corners "9x5" --camera_serial_number 18079958 --calibration_image_dir {intrinsic_images_folder_path}
```
4. In this case, the intrinsic parameters would have been generated as {intrinsic_images_folder_path}/18079958.xml.
4. In this case, the intrinsic parameters would have been generated as `{intrinsic_images_folder_path}/18079958.xml`.
5. Run steps 1-4 for each one of your cameras.
6. After you calibrate the camera intrinsics, when you run OpenPose with those cameras, you should see the lines in real-life to be (almost) perfect lines in the image. Otherwise, the calibration was not good. Try checking straight patterns such us wall corners or ceilings:
```
```sh
# With distortion (lines might seem with a more circular shape)
./build/examples/openpose/openpose.bin --num_gpu 0 --flir_camera --flir_camera_index 0 --frame_keep_distortion
# Without distortion (lines should look as lines)
./build/examples/openpose/openpose.bin --num_gpu 0 --flir_camera --flir_camera_index 0
```
7. Full example for 4 Flir/Point Grey cameras:
```
```sh
# Get images for calibration
./build/examples/openpose/openpose.bin --num_gpu 0 --frame_keep_distortion --flir_camera --flir_camera_index 0 --write_images ~/Desktop/intrinsics_0
./build/examples/openpose/openpose.bin --num_gpu 0 --frame_keep_distortion --flir_camera --flir_camera_index 1 --write_images ~/Desktop/intrinsics_1
......@@ -80,13 +80,13 @@ Note: In order to maximize calibration quality, **do not reuse the same video se
### Step 2 - Extrinsic Parameter Calibration
1. After intrinsics calibration, save undirtoted images for all the camera views:
```
```sh
./build/examples/openpose/openpose.bin --num_gpu 0 --flir_camera --write_images ~/Desktop/extrinsics
```
2. Run the extrinsic calibration tool between each pair of close cameras. In this example:
- We assume camera 0 to the right, 1 in the middle-right, 2 in the middle-left, and 3 in the left.
- We assume camera 1 as the coordinate origin.
```
```sh
./build/examples/calibration/calibration.bin --mode 2 --grid_square_size_mm 127.0 --grid_number_inner_corners 9x6 --omit_distortion --calibration_image_dir ~/Desktop/extrinsics/ --cam0 1 --cam1 0
./build/examples/calibration/calibration.bin --mode 2 --grid_square_size_mm 127.0 --grid_number_inner_corners 9x6 --omit_distortion --calibration_image_dir ~/Desktop/extrinsics/ --cam0 1 --cam1 2
./build/examples/calibration/calibration.bin --mode 2 --grid_square_size_mm 127.0 --grid_number_inner_corners 9x6 --omit_distortion --calibration_image_dir ~/Desktop/extrinsics/ --cam0 1 --cam1 3
......@@ -109,4 +109,4 @@ Note: In order to maximize calibration quality, **do not reuse the same video se
## Using a Different Camera Brand
If you plan to use the calibration tool without using OpenPose, you can manually save a video sequence of your desired camera into each of the camera image folders (i.e., in the above example, the `~/Desktop/intrinsics_0`, `~/Desktop/intrinsics_1`, etc. folders).
If you wanna eventually run that camera with OpenPose, check [doc/3d_reconstruction_demo.md#using-a-different-camera-brand](./3d_reconstruction_demo.md#using-a-different-camera-brand).
If you wanna eventually run that camera with OpenPose, check [doc/modules/3d_reconstruction_module.md#using-a-different-camera-brand](./modules/3d_reconstruction_module.md#using-a-different-camera-brand).
......@@ -3,23 +3,52 @@ OpenPose Python Module
## Contents
1. [Introduction](#introduction)
2. [Testing and Installation](#testing-and-installation)
2. [Installation](#installation)
3. [Compatibility](#compatibility)
4. [Testing](#testing)
## Introduction
This experimental module exposes a Python API for OpenPose. This allows you to construct an OpenPose object, pass in a numpy array for an image, and get a numpy array of the pose positions. This API also exposes an API that allows you to directly pass in heatmaps from a network and extract poses out of it.
## Testing and Installation
To install the API so that it can be used globally, ensure that the `BUILD_PYTHON` flag is turned on, and run `make install` after compilation. This will install the python library at your desired installation path. (default is `/usr/local/python`) Ensure that this is in your python path in order to use it.
Two examples can be found in `build/examples/tutorial_python` in your build folder. Navigate directly to this path to run examples. `1_extract_pose` demonstrates a simple use of the API. `2_pose_from_heatmaps` demonstrates constructing pose from heatmaps from the caffe network.
## Installation
Check [doc/installation.md#python-module](./installation.md#python-module) for installation steps.
## Compatibility
The OpenPose Python module is compatible with both Python 2 and Python 3. In addition, it will also run in all OpenPose compatible operating systems.
## Testing
Two examples can be found in `build/examples/tutorial_python` in your build folder. Navigate directly to this path to run examples.
- `1_extract_pose` demonstrates a simple use of the API.
- `2_pose_from_heatmaps` demonstrates constructing pose from heatmaps from the caffe network.
```
# From command line
cd build/examples/tutorial_python
python
```
```python
import sys
# From Python
# It requires OpenCV installed for Python
import cv2
import os
import sys
# Remember to add your installation path here
sys.path.append('/usr/local/python')
# Option a
sys.path.append('{OpenPose_path}/python')
# Option b
# If you run `make install` (default path is `/usr/local/python` for Ubuntu), you can also access the OpenPose/python module from there. This will install OpenPose and the python library at your desired installation path. Ensure that this is in your python path in order to use it.
# sys.path.append('/usr/local/python')
from openpose import *
# Parameters for OpenPose. Take a look at C++ OpenPose example for meaning of components. Ensure all below are filled
......@@ -27,7 +56,7 @@ params = dict()
params["logging_level"] = 3
params["output_resolution"] = "-1x-1"
params["net_resolution"] = "-1x368"
params["model_pose"] = "COCO"
params["model_pose"] = "BODY_25"
params["alpha_pose"] = 0.6
params["scale_gap"] = 0.3
params["scale_number"] = 1
......@@ -38,15 +67,15 @@ params["disable_blending"] = False
params["default_model_folder"] = "/home/user/openpose/models"
# Construct OpenPose object allocates GPU memory
openpose = OpenPose(params)
img = cv2.imread("image.png")
arr, output_image = openpose.forward(img, True)
print arr
while 1:
# Read new image
img = cv2.imread("image.png")
# Output keypoints and the image with the human skeleton blended on it
keypoints, output_image = openpose.forward(img, True)
# Print the human pose keypoints, i.e., a [#people x #keypoints x 3]-dimensional numpy object with the keypoints of all the people on that image
print keypoints
# Display the image
cv2.imshow("output", output_image)
cv2.waitKey(15)
```
......@@ -2,11 +2,12 @@ OpenPose Library - Latest Released Features
====================================
- Jun 2018: [**Combined body-foot model released! Faster and more accurate.**](./installation.md)!
- Jun 2018: [**AMD graphic card version**](./calibration_demo.md) released!
- Jun 2018: [**Calibration toolbox**](./calibration_demo.md) released!
- Jun 2018: [**Python version**](./modules/python_module.md) released!
- Jun 2018: [**AMD graphic card version**](./modules/calibration_module.md) released!
- Jun 2018: [**Calibration toolbox**](./modules/calibration_module.md) released!
- Jun 2018: [**Mac OSX version (CPU)**](./installation.md) released!
- Mar 2018: [**CPU version**](./installation.md#cpu-version)!
- Mar 2018: [**3-D keypoint reconstruction module**](./3d_reconstruction_demo.md) (from multiple camera views)!
- Mar 2018: [**3-D keypoint reconstruction module**](./modules/3d_reconstruction_module.md) (from multiple camera views)!
- Sep 2017: [**CMake**](./installation.md) installer and **IP camera** support!
- Jul 2017: [**Windows portable binaries and demo**](https://github.com/CMU-Perceptual-Computing-Lab/openpose/releases)!
- Jul 2017: **Hands** released!
......
......@@ -5,6 +5,7 @@ add_subdirectory(tutorial_pose)
add_subdirectory(tutorial_python)
add_subdirectory(tutorial_thread)
add_subdirectory(tutorial_wrapper)
add_subdirectory(user_code)
if (UNIX OR APPLE)
add_subdirectory(tests)
endif (UNIX OR APPLE)
// ------------------------- OpenPose Calibration Toolbox -------------------------
// Check `doc/calibration_demo.md`.
// Check `doc/modules/calibration_module.md`.
// Implemented on top of OpenCV.
// It computes and saves the intrinsics parameters of the input images.
......@@ -35,7 +35,7 @@ DEFINE_string(camera_serial_number, "18079958", "Camera serial number.")
DEFINE_bool(omit_distortion, false, "Set to true if image views are already undistorted (e.g., if recorded from OpenPose"
" after intrinsic parameter calibration).");
DEFINE_bool(combine_cam0_extrinsics, false, "Set to true if cam0 extrinsics are not [R=I, t=0]. I will make no effect if cam0 is"
" already the origin. See doc/calibration_demo.md for an example.");
" already the origin. See doc/modules/calibration_module.md for an example.");
DEFINE_int32(cam0, 1, "Baseline camera for extrinsic calibration, cam1 will be calibrated assuming cam0 the"
" world coordinate origin.");
DEFINE_int32(cam1, 0, "Target camera to estimate its extrinsic parameters, it will be calibrated assuming cam0"
......
# Uncomment these lines with your custom file names
# set(USER_CODE_FILES
# ADD_HERE_YOUR_FILE1.cpp
# ADD_HERE_YOUR_FILE1.hpp
# ADD_HERE_YOUR_FILE2.cpp
# # ...
# ADD_HERE_YOUR_FILEn.cpp)
foreach(USER_CODE_FILE ${USER_CODE_FILES})
get_filename_component(SOURCE_NAME ${USER_CODE_FILE} NAME_WE)
if (UNIX OR APPLE)
set(EXE_NAME "${SOURCE_NAME}.bin")
elseif (WIN32)
set(EXE_NAME "UserCustomCode")
endif ()
message(STATUS "Adding Example ${EXE_NAME}")
add_executable(${EXE_NAME} ${USER_CODE_FILE})
target_link_libraries(${EXE_NAME} openpose ${examples_3rdparty_libraries})
if (WIN32)
set_property(TARGET ${EXE_NAME} PROPERTY FOLDER "User Code")
configure_file(${CMAKE_SOURCE_DIR}/cmake/OpenPose${VCXPROJ_FILE_GPU_MODE}.vcxproj.user
${CMAKE_CURRENT_BINARY_DIR}/${EXE_NAME}.vcxproj.user @ONLY)
# Properties->General->Output Directory
set_property(TARGET ${EXE_NAME} PROPERTY RUNTIME_OUTPUT_DIRECTORY_RELEASE ${PROJECT_BINARY_DIR}/$(Platform)/$(Configuration))
set_property(TARGET ${EXE_NAME} PROPERTY RUNTIME_OUTPUT_DIRECTORY_DEBUG ${PROJECT_BINARY_DIR}/$(Platform)/$(Configuration))
endif (WIN32)
endforeach()
Adding and Testing Custom Code
====================================
## Purpose
You can quickly add your custom code into this folder so that quick prototypes can be easily tested without having to create a whoel new project just for it.
## How-to
1. Install/compile OpenPose as usual.
2. Add your custom *.cpp / *.hpp files here,. Hint: You might want to start by copying the [OpenPoseDemo](../openpose/openpose.cpp) example or any of the [examples/tutorial_wrapper/](../tutorial_wrapper/) examples. Then, you can simply modify their content.
3. Add the name of your custom *.cpp / *.hpp files at the top of the [examples/user_code/CMakeLists.txt](./CMakeLists.txt) file.
4. Re-compile OpenPose.
```
# Ubuntu/Mac
cd build/
make -j`nproc`
# Windows
# Close Visual Studio, re-run CMake, and re-compile the project in Visual Studio
```
5. **Run step 4 every time that you make changes into your code**.
## Running your Custom Code
Run:
```
./build/examples/user_code/{your_custom_file_name}
```
// #include <thread>
#include <numeric> // std::accumulate
#ifdef USE_CERES
#include <ceres/ceres.h>
#include <ceres/rotation.h>
......
......@@ -143,21 +143,36 @@ namespace op
// Empirically tested - OpenCV is more efficient normalizing a whole matrix/image (it uses AVX and
// other optimized instruction sets).
// In addition, the following if statement does not copy the pointer to a cv::Mat, just wrapps it.
// VGG
if (normalize == 1)
{
cv::Mat floatPtrImageCvWrapper(height, width, CV_32FC3, floatPtrImage);
floatPtrImageCvWrapper = floatPtrImageCvWrapper/256.f - 0.5f;
}
// // ResNet
// else if (normalize == 2)
// {
// const int imageArea = width * height;
// const std::array<float,3> means{102.9801, 115.9465, 122.7717};
// for (auto i = 0 ; i < 3 ; i++)
// {
// cv::Mat floatPtrImageCvWrapper(height, width, CV_32FC1, floatPtrImage + i*imageArea);
// floatPtrImageCvWrapper = floatPtrImageCvWrapper - means[i];
// }
// }
// DenseNet
else if (normalize == 2)
{
const auto scaleDenseNet = 0.017;
const int imageArea = width * height;
const std::array<float,3> means{102.9801, 115.9465, 122.7717};
const std::array<float,3> means{103.94,116.78,123.68};
for (auto i = 0 ; i < 3 ; i++)
{
cv::Mat floatPtrImageCvWrapper(height, width, CV_32FC1, floatPtrImage + i*imageArea);
floatPtrImageCvWrapper = floatPtrImageCvWrapper - means[i];
floatPtrImageCvWrapper = scaleDenseNet*(floatPtrImageCvWrapper - means[i]);
}
}
// Unknown
else if (normalize != 0)
error("Unknown normalization value (" + std::to_string(normalize) + ").",
__LINE__, __FUNCTION__, __FILE__);
......
Markdown is supported
0% .
You are about to add 0 people to the discussion. Proceed with caution.
先完成此消息的编辑!
想要评论请 注册