README.md 4.0 KB
Newer Older
D
daminglu 已提交
1
<p align="center">
Y
Yan Chunwei 已提交
2
  <img src="./docs/images/vs-logo.png" width="60%" />
D
daminglu 已提交
3 4 5
</p>

## Introduction
Q
Qiao Longfei 已提交
6 7 8
VisualDL is a deep learning visualization tool that can help design deep learning jobs.
It includes features such as scalar, parameter distribution, model structure and image visualization.
Currently it is being developed at a high pace.
D
daminglu 已提交
9 10
New features will be continuously added.

Q
Qiao Longfei 已提交
11
At present, most DNN frameworks use Python as their primary language. VisualDL supports Python by nature.
D
daminglu 已提交
12
Users can get plentiful visualization results by simply add a few lines of Python code into their model before training.
Q
Qiao Longfei 已提交
13 14 15


Besides Python SDK, VisualDL was writen in C++ on the low level. It also provides C++ SDK that
D
daminglu 已提交
16 17 18 19 20 21 22 23 24 25 26 27
can be integrated into other platforms.  


## Component
VisualDL now provides 4 components:

- graph
- scalar
- image
- histogram

### Graph
28
Graph is compatible with ONNX ([Open Neural Network Exchange](https://github.com/onnx/onnx)),
Q
Qiao Longfei 已提交
29
Cooperated with Python SDK, VisualDL can be compatible with most major DNN frameworks, including
D
daminglu 已提交
30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57
PaddlePaddle, PyTorch and MXNet.

<p align="center">
  <img src="https://github.com/daming-lu/large_files/blob/master/graph_demo.gif" width="60%" />
</p>

### Scalar
Scalar can be used to show the trends of error during training.


<p align="center">
<img src="https://github.com/daming-lu/large_files/blob/master/loss_scalar.gif" width="60%"/>
</p>

### Image
Image can be used to visualize any tensor or intermediate generated image.

<p align="center">
<img src="https://github.com/daming-lu/large_files/blob/master/loss_image.gif" width="60%"/>
</p>

### Histogram
Histogram can be used to visualize parameter distribution and trends for any tensor.

<p align="center">
<img src="https://github.com/daming-lu/large_files/blob/master/histogram.gif" width="60%"/>
</p>

Q
Qiao Longfei 已提交
58 59 60 61 62 63 64 65 66 67 68 69
## Quick Start
```
# install
pip install --upgrade visualdl

# run a demo
vdl_scratch.py
visualDL --logdir=scratch_log --port=8080

# visit http://127.0.0.1:8080
```

D
daminglu 已提交
70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87
## SDK
VisualDL provides both Python SDK and C++ SDK in order to fit more use cases.


### Python SDK
Below is an example of creating a simple Scalar component and inserting data from different timestamps:

```python
import random
from visualdl import LogWriter

logdir = "./tmp"
logger = LogWriter(dir, sync_cycle=10)

# mark the components with 'train' label.
with logger.mode("train"):
    # create a scalar component called 'scalars/scalar0'
    scalar0 = logger.scalar("scalars/scalar0")
Q
Qiao Longfei 已提交
88

Y
Yan Chunwei 已提交
89 90 91
# add some records during DL model running.
for step in range(100):
    scalar0.add_record(step, random.random())
D
daminglu 已提交
92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107
```

### C++ SDK
Here is the C++ SDK identical to the Python SDK example above:

```c++
#include <cstdlib>
#include <string>
#include "visualdl/sdk.h"

namespace vs = visualdl;
namepsace cp = visualdl::components;

int main() {
  const std::string dir = "./tmp";
  vs::LogWriter logger(dir, 10);
Q
Qiao Longfei 已提交
108

D
daminglu 已提交
109 110
  logger.SetMode("train");
  auto tablet = logger.NewTablet("scalars/scalar0");
Q
Qiao Longfei 已提交
111

D
daminglu 已提交
112
  cp::Scalar<float> scalar0(tablet);
Q
Qiao Longfei 已提交
113

D
daminglu 已提交
114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135
  for (int step = 0; step < 1000; step++) {
    float v = (float)std::rand() / RAND_MAX;
    scalar0.AddRecord(step, v);
  }

  return 0;
}
```

## Launch Board
After some logs have been generated during training, users can launch board to see real-time data visualization.


```
visualDL --logdir <some log dir>
```

Board also supports the parameters below for remote access:

- `--host` set IP
- `--port` set port
- `--model_pb` specify ONNX format for model file
Q
qiaolongfei 已提交
136

Q
Qiao Longfei 已提交
137 138 139 140
### How to install from pypi
```
pip install --upgrade visualdl
```
Q
qiaolongfei 已提交
141

Q
Qiao Longfei 已提交
142
### How to build and install locally
Q
qiaolongfei 已提交
143
```
Q
Qiao Longfei 已提交
144 145 146
git clone https://github.com/PaddlePaddle/VisualDL.git
cd VisualDL

Y
Yan Chunwei 已提交
147
python setup.py bdist_wheel
Q
Qiao Longfei 已提交
148
pip install --upgrade dist/visualdl-*.whl
Q
Qiao Longfei 已提交
149 150
```

Y
Yan Chunwei 已提交
151
### Run a demo from scratch
Q
qiaolongfei 已提交
152
```
Y
Yan Chunwei 已提交
153
vdl_scratch.py
Q
Qiao Longfei 已提交
154
visualDL --logdir=scratch_log --port=8080
Q
Qiao Longfei 已提交
155
```
Q
Qiao Longfei 已提交
156 157
that will start a server locally on port 8080, then
you can visit http://127.0.0.1:8080 the see the visualdl board.
D
daminglu 已提交
158 159 160 161


### Contribute

Q
Qiao Longfei 已提交
162 163
VisualDL is initially created by [PaddlePaddle](http://www.paddlepaddle.org/) and
[ECharts](http://echarts.baidu.com/).
D
daminglu 已提交
164
We welcome everyone to use, comment and contribute to Visual DL :)