diff --git a/.github/ISSUE_TEMPLATE/caffe2paddle.md b/.github/ISSUE_TEMPLATE/caffe2paddle.md new file mode 100644 index 0000000000000000000000000000000000000000..d650fcfbd85663600f57c2055211fa9c2d4b20d4 --- /dev/null +++ b/.github/ISSUE_TEMPLATE/caffe2paddle.md @@ -0,0 +1,8 @@ +--- +name: caffe2paddle +about: Caffe模型转换至PaddlePaddle，请说明Caffe模型的来源，模型类型（例如图像分类、目标检测等） +--- + +Caffe模型转换至PaddlePaddle，请说明Caffe模型的来源，模型类型（例如图像分类、目标检测等） + +如有原模型文件或github链接，如方便可一并附上，方便开发人员分析。 diff --git a/.github/ISSUE_TEMPLATE/onnx2paddle.md b/.github/ISSUE_TEMPLATE/onnx2paddle.md new file mode 100644 index 0000000000000000000000000000000000000000..de8a7aebce34994f1269adad7d326a71851e9700 --- /dev/null +++ b/.github/ISSUE_TEMPLATE/onnx2paddle.md @@ -0,0 +1,7 @@ +--- +name: onnx2paddle +about: ONNX模型转换至PaddlePaddle，请说明ONNX模型的来源，模型类型（例如图像分类、目标检测等) +--- +ONNX模型转换至PaddlePaddle，请说明ONNX模型的来源，模型类型（例如图像分类、目标检测等） + +如有原模型文件或github链接，如方便可一并附上，方便开发人员分析。 diff --git a/.github/ISSUE_TEMPLATE/other.md b/.github/ISSUE_TEMPLATE/other.md new file mode 100644 index 0000000000000000000000000000000000000000..7202cce897890f3d4a8e970cc4e379ba5db02821 --- /dev/null +++ b/.github/ISSUE_TEMPLATE/other.md @@ -0,0 +1,4 @@ +--- +name: 其它类型 +about: 例如Bug类，需求建议类 +--- diff --git a/.github/ISSUE_TEMPLATE/paddle2onnx.md b/.github/ISSUE_TEMPLATE/paddle2onnx.md new file mode 100644 index 0000000000000000000000000000000000000000..4f589a66df1a653995d17d0d6e422dff3764bcd7 --- /dev/null +++ b/.github/ISSUE_TEMPLATE/paddle2onnx.md @@ -0,0 +1,7 @@ +--- +name: paddle2onnx +about: Paddle模型转换至ONNX，请说明Paddle模型的来源，模型类型（例如图像分类、目标检测等） +--- +Paddle模型转换至ONNX，请说明Paddle模型的来源，模型类型（例如图像分类、目标检测等） + +如有原模型文件或github链接，如方便可一并附上，方便开发人员分析，同时建议说明模型转换的使用场景:) diff --git a/.github/ISSUE_TEMPLATE/tensorflow2paddle.md b/.github/ISSUE_TEMPLATE/tensorflow2paddle.md new file mode 100644 index 0000000000000000000000000000000000000000..88af18bf67a737206b6757ec55f1d581d62460b3 --- /dev/null +++ b/.github/ISSUE_TEMPLATE/tensorflow2paddle.md @@ -0,0 +1,8 @@ +--- +name: tensorflow2paddle +about: TensorFlow模型转换至PaddlePaddle，请说明TensorFlow模型的来源，模型类型（例如图像分类、目标检测等） +--- + +TensorFlow模型转换至PaddlePaddle，请说明TensorFlow模型的来源，模型类型（例如图像分类、目标检测等） + +如有原模型文件或github链接，可以一并附上，方便开发人员分析。 diff --git a/FAQ.md b/FAQ.md index a66ac9b0f6c06652f96af7d417effedbf2261730..4247a15752abb8e68fd8ee6307b5eb3930b40a9f 100644 --- a/FAQ.md +++ b/FAQ.md @@ -14,3 +14,12 @@ x2paddle -f tensorflow -m tf.pb -s pd-model --without_data_format_optimization - **Q3. ONNX模型转换过程中，提示『Unknown shape for input tensor[tensor name: "input"] -> shape: ['batch', 'sequence']， Please define shape of input here』** A：该提示信息表示从ONNX的模型中获取到输入tensor(tensor名为"input:)的shape是语义象征性的['batch', 'sequence']，而不是dim为int类型的shape，从而可能会因为部分node的shape无法推理，导致转换失败。所以用户可以尝试手动在提示后输入详细的shape信息，如:-1,3,224,224 其中-1表示Batch + +**Q4. Paddle模型转至ONNX模型过程中，提示『The parameter normalized of multiclass_nms OP of Paddle is False, which has diff with ONNX』** +A: 此提示为警告信息，模型仍然会正常进行转换。Paddle中`fluid.layers.multiclass_nms`算子中提供了`normalized`参数，用于表示输入box是否进行了归一化。而ONNX中的NMS算子只支持`normalized`参数为True的情况，当你转换的模型（一般是YOLOv3模型）中该参数为`False`的情况下，转换后的模型可能会与原模型存在diff。 + +**Q5. Paddle模型转至ONNX模型过程中，提示『Converting this model to ONNX need with static input shape, please fix input shape of this model』** +A: 此提示为错误信息，表示该模型的转换需要固定的输入大小: +> 1. 模型来源于PaddleX导出，可以在导出的命令中，指定--fixed_input_shape=[Height,Width]，详情可见：[PaddleX模型导出文档](https://github.com/PaddlePaddle/PaddleX/blob/develop/docs/deploy/export_model.md)。 +> 2. 模型来源于PaddleDetection导出，可以在导出模型的时候，指定 TestReader.inputs_def.image_shape=[Channel,Height,Width], 详情可见：[PaddleDetection模型导出文档](https://github.com/PaddlePaddle/PaddleDetection/blob/master/docs/advanced_tutorials/deploy/EXPORT_MODEL.md#设置导出模型的输入大小)。 +> 3. 模型来源于自己构建，可在网络构建的`fluid.data(shape=[])`中，指定shape参数来固定模型的输入大小。 diff --git a/caffe_custom_layer.md b/caffe_custom_layer.md index 83234ef062e5d3787868c2985267be80fc5ba742..a7bddf2d7823851bf8c538a8bfe0bb35aff32e8b 100644 --- a/caffe_custom_layer.md +++ b/caffe_custom_layer.md @@ -1,4 +1,4 @@ -目前，代码中已经提供了8个非官方op（不在[官网](http://caffe.berkeleyvision.org/tutorial/layers)上的op）的转换，这些op对应的Caffe实现源码如下： +目前，代码中已经提供了10个非官方op（不在[官网](http://caffe.berkeleyvision.org/tutorial/layers)上的op）的转换，这些op对应的Caffe实现源码如下： | op | 该版本实现源码 | |-------|--------| @@ -10,3 +10,5 @@ | Normalize | [code](https://github.com/weiliu89/caffe/blob/ssd/src/caffe/layers/normalize_layer.cpp) | | ROIPooling | [code](https://github.com/rbgirshick/caffe-fast-rcnn/blob/0dcd397b29507b8314e252e850518c5695efbb83/src/caffe/layers/roi_pooling_layer.cpp) | | Axpy | [code](https://github.com/hujie-frank/SENet/blob/master/src/caffe/layers/axpy_layer.cpp) | +| ReLU6 | [code](https://github.com/chuanqi305/ssd/blob/ssd/src/caffe/layers/relu6_layer.cpp) | +| Upsample | [code](https://github.com/eric612/MobileNet-YOLO/blob/master/src/caffe/layers/upsample_layer.cpp) | diff --git a/op_list.md b/op_list.md index a69a54bef1ad763602a350dd894f816279a7d531..aff73baf9a9f9447e215972c44479709b8a4c2d0 100644 --- a/op_list.md +++ b/op_list.md @@ -34,6 +34,7 @@ | 21 | Axpy | 22 | ROIPolling | 23 | Permute | 24 | DetectionOutput | | 25 | Normalize | 26 | Select | 27 | ShuffleChannel | 28 | ConvolutionDepthwise | | 29 | ReLU | 30 | AbsVal | 31 | Sigmoid | 32 | TanH | +| 33 | ReLU6 | 34 | Upsample | ## ONNX diff --git a/x2paddle/__init__.py b/x2paddle/__init__.py index 55e26bee0b7d5987f5ad784552cc6f8dbed631be..1e963372fdc9e65d75c1abc788b08c179f9965df 100644 --- a/x2paddle/__init__.py +++ b/x2paddle/__init__.py @@ -1,4 +1,4 @@ -__version__ = "0.7.4" +__version__ = "0.8.1" from .core.program import PaddleProgram diff --git a/x2paddle/convert.py b/x2paddle/convert.py index fe0b09e9fdedc7648c97fba316b38b0a6fe69ea1..717bd5fc43138bac1d826b458281f6e97017112c 100644 --- a/x2paddle/convert.py +++ b/x2paddle/convert.py @@ -195,9 +195,14 @@ def onnx2paddle(model_path, save_dir, params_merge=False): def paddle2onnx(model_path, save_dir, opset_version=10): from x2paddle.decoder.paddle_decoder import PaddleDecoder from x2paddle.op_mapper.paddle2onnx.paddle_op_mapper import PaddleOpMapper + import paddle.fluid as fluid model = PaddleDecoder(model_path, '__model__', '__params__') mapper = PaddleOpMapper() - mapper.convert(model.program, save_dir, opset_number=opset_version) + mapper.convert( + model.program, + save_dir, + scope=fluid.global_scope(), + opset_version=opset_version) def main(): @@ -264,7 +269,7 @@ def main(): elif args.framework == "paddle2onnx": assert args.model is not None, "--model should be defined while translating paddle model to onnx" - paddle2onnx(args.model, args.save_dir, args.onnx_opset) + paddle2onnx(args.model, args.save_dir, opset_version=args.onnx_opset) else: raise Exception( diff --git a/x2paddle/decoder/caffe_decoder.py b/x2paddle/decoder/caffe_decoder.py index cb9f59708fa0980b0a3bcb029319a33a3df18f92..4174cc2032b22d335f7bf9a058ac939abb9ad735 100644 --- a/x2paddle/decoder/caffe_decoder.py +++ b/x2paddle/decoder/caffe_decoder.py @@ -88,6 +88,19 @@ class CaffeGraph(Graph): # filter them out here. if (not exclude) and (phase == 'test'): exclude = (type_str == 'Dropout') + if layer.type == 'Dropout': + drop_layer_top = layer.top[0] + drop_layer_bottom = layer.bottom[0] + if drop_layer_top != drop_layer_bottom: + for next_layer in layers: + for next_layer_bottom_idx, next_layer_bottom in enumerate( + next_layer.bottom): + if drop_layer_top == next_layer_bottom: + next_layer.bottom.remove(drop_layer_top) + next_layer.bottom.insert( + next_layer_bottom_idx, + drop_layer_bottom) + if not exclude: filtered_layers.append(layer) # Guard against dupes. diff --git a/x2paddle/decoder/onnx_shape_inference.py b/x2paddle/decoder/onnx_shape_inference.py index 303fc290deb6aef43cd8a9f6fd53078ff77d1564..dae2d0268a00dcac0b1d4a6c796f55431a985ad3 100644 --- a/x2paddle/decoder/onnx_shape_inference.py +++ b/x2paddle/decoder/onnx_shape_inference.py @@ -545,9 +545,6 @@ class SymbolicShapeInference: self.sympy_data_[node.output[0]] = data new_shape = np.array(data).shape vi = self.known_vi_[node.output[0]] - #print(node.output[0]) - #print(new_shape) - #vi.CopyFrom(helper.make_tensor_value_info(node.output[0], self.known_vi_[node.input[0]].type.tensor_type.elem_type, list(new_shape))) def _pass_on_sympy_data(self, node): assert len(node.input) == 1 or node.op_type == 'Reshape' @@ -854,12 +851,7 @@ class SymbolicShapeInference: axis = handle_negative_axis( get_attribute(node, 'axis', 0), len(data_shape)) indices_shape = self._get_shape(node, 1) - #if indices_shape == []: - # value = self._get_initializer_value(node, 1) - # if isinstance(value.tolist(), int): - # indices_shape = [1] new_shape = data_shape[:axis] + indices_shape + data_shape[axis + 1:] - #print(new_shape) vi = self.known_vi_[node.output[0]] vi.CopyFrom( helper.make_tensor_value_info(node.output[ diff --git a/x2paddle/op_mapper/caffe_custom_layer/__init__.py b/x2paddle/op_mapper/caffe_custom_layer/__init__.py index b75ca72c83aba24b0e384779836e986135134c0a..5dc3079cb6dba452d767f20cd2ed29912940c124 100644 --- a/x2paddle/op_mapper/caffe_custom_layer/__init__.py +++ b/x2paddle/op_mapper/caffe_custom_layer/__init__.py @@ -10,6 +10,8 @@ from . import select from . import shufflechannel from . import convolutiondepthwise from . import axpy +from . import upsample +from . import relu6 #custom layer import ends custom_layers = get_registered_layers() diff --git a/x2paddle/op_mapper/caffe_custom_layer/axpy.py b/x2paddle/op_mapper/caffe_custom_layer/axpy.py index c32de40f5c45cd345847c4bc031f969afb34e466..6e0f843398afc877e22b24a454f45c96e5369ceb 100644 --- a/x2paddle/op_mapper/caffe_custom_layer/axpy.py +++ b/x2paddle/op_mapper/caffe_custom_layer/axpy.py @@ -2,7 +2,7 @@ from .register import register from x2paddle.core.util import * -def axpy_shape(input_shape): +def axpy_shape(input_shapes): assert len(input_shapes) == 3, "not valid input shape for axpy layer" assert len(input_shapes[0]) == len(input_shapes[1]), 'should have same dims' output_shape = input_shapes[1] @@ -18,7 +18,7 @@ def axpy_layer(inputs, input_shape=None, name=None): y = inputs[2] out = fluid.layers.elementwise_mul(x, alpha, axis=0) out = fluid.layers.elementwise_add(out, y, name=name) - print(out) + return out def axpy_weights(name, data=None): diff --git a/x2paddle/op_mapper/caffe_custom_layer/relu6.py b/x2paddle/op_mapper/caffe_custom_layer/relu6.py new file mode 100644 index 0000000000000000000000000000000000000000..4e94fcf1a77d730ba8552e30c1c494e966038043 --- /dev/null +++ b/x2paddle/op_mapper/caffe_custom_layer/relu6.py @@ -0,0 +1,21 @@ +from .register import register +from x2paddle.core.util import * + + +def relu6_shape(input_shape): + return input_shape + + +def relu6_layer(inputs, input_shape=None, name=None): + input = inputs[0] + out = fluid.layers.relu6(x=input) + return out + + +def relu6_weights(name, data=None): + weights_name = [] + return weights_name + + +register( + kind='ReLU6', shape=relu6_shape, layer=relu6_layer, weights=relu6_weights) diff --git a/x2paddle/op_mapper/caffe_custom_layer/upsample.py b/x2paddle/op_mapper/caffe_custom_layer/upsample.py new file mode 100644 index 0000000000000000000000000000000000000000..44ce7f516bc9049bd10ea41183e676dae5692ef3 --- /dev/null +++ b/x2paddle/op_mapper/caffe_custom_layer/upsample.py @@ -0,0 +1,66 @@ +# -*- coding: utf-8 -*- +################################################################################ +# +# Copyright (c) 2020 Baidu.com, Inc. All Rights Reserved +# +################################################################################ +""" +Author: Drift +Email: wutuobang@baidu.com +Date: 2020/04/22 18:45 +""" + +from .register import register +from x2paddle.core.util import * + + +def upsample_shape(input_shapes, scale): + """ + + :param input_shapes: + :param scale: + :return: + """ + assert len(input_shapes) == 1, "not valid input shape for upsample layer" + assert type(scale) is int + + input_shape = input_shapes[0] + new_h = scale * input_shape[2] + new_w = scale * input_shape[3] + + output_shape = [input_shape[0], input_shape[1], new_h, new_w] + return [output_shape] + + +def upsample_layer(inputs, scale, input_shape=None, name=None): + """ + + :param inputs: + :param scale: + :param input_shape: + :param name: + :return: + """ + x = inputs[0] + out = fluid.layers.resize_nearest( + x, align_corners=False, scale=scale, name=name) + + return out + + +def upsample_weights(name, data=None): + """ + + :param name: + :param data: + :return: + """ + weights_name = [] + return weights_name + + +register( + kind='Upsample', + shape=upsample_shape, + layer=upsample_layer, + weights=upsample_weights) diff --git a/x2paddle/op_mapper/caffe_op_mapper.py b/x2paddle/op_mapper/caffe_op_mapper.py index f350c50ea707518e296c6e807e7d89cca686bbcf..17797ed01a0783af14a46edcb96a7a6a29f5911e 100644 --- a/x2paddle/op_mapper/caffe_op_mapper.py +++ b/x2paddle/op_mapper/caffe_op_mapper.py @@ -23,7 +23,6 @@ from x2paddle.op_mapper.caffe_custom_layer import * class CaffeOpMapper(OpMapper): directly_map_ops = { - 'ReLU': 'relu', 'AbsVal': 'abs', 'Sigmoid': 'sigmoid', 'TanH': 'tanh', @@ -435,6 +434,26 @@ class CaffeOpMapper(OpMapper): node.fluid_code.add_layer( "concat", inputs=inputs, output=node, param_attr=attr) + def ReLU(self, node): + """ + + :param node: + :return: + """ + assert len( + node.inputs) == 1, 'The count of ReLU node\'s input is not 1.' + input = self.graph.get_bottom_node(node, idx=0, copy=True) + + params = node.layer.relu_param + if params.HasField('negative_slope') and params.negative_slope != 0: + negative_slope = float(params.negative_slope) + + attr = {'alpha': negative_slope} + node.fluid_code.add_layer( + 'leaky_relu', inputs=input, output=node, param_attr=attr) + else: + node.fluid_code.add_layer('relu', inputs=input, output=node) + def PReLU(self, node): assert len( node.inputs) == 1, 'The count of PReLU node\'s input is not 1.' diff --git a/x2paddle/op_mapper/onnx2paddle/opset9/opset.py b/x2paddle/op_mapper/onnx2paddle/opset9/opset.py index e1ebdf23b5506bba3cbb659f598e9eab029a0cd8..3cb60ffa4145a455f0552c092ae31a324b3702ad 100644 --- a/x2paddle/op_mapper/onnx2paddle/opset9/opset.py +++ b/x2paddle/op_mapper/onnx2paddle/opset9/opset.py @@ -332,10 +332,37 @@ class OpSet9(): def _interpolate(self, node): val_x = self.graph.get_input_node(node, idx=0, copy=True) + inputs = {'input': val_x} if node.layer_type == 'Resize': - val_scales = self.graph.get_input_node(node, idx=2, copy=True) + if len(node.layer.input) == 2: + # opset 10 + val_scales = self.graph.get_input_node(node, idx=1, copy=True) + inputs['scale'] = val_scales + elif len(node.layer.input) == 3: + # opset 11 + val_scales = self.graph.get_input_node(node, idx=2, copy=True) + inputs['scale'] = val_scales + elif len(node.layer.input) == 4: + # opset 11 + val_sizes = self.graph.get_input_node(node, idx=3, copy=True) + var_nc, var_hw = val_sizes.layer_name + '_nc', val_sizes.layer_name + '_hw' + node.fluid_code.add_layer( + 'split', + inputs=val_sizes, + output=var_nc + ',' + var_hw, + param_attr={ + 'dim': 0, + 'num_or_sections': [2, 2], + }) + node.fluid_code.add_layer( + "cast", + inputs=var_hw, + output=var_hw, + param_attr={'dtype': string('int32')}) + inputs['out_shape'] = var_hw elif node.layer_type == 'Upsample': val_scales = self.graph.get_input_node(node, idx=1, copy=True) + inputs['scale'] = val_scales attr = {'name': string(node.layer_name)} mode = node.get_attr('mode', 'nearest') @@ -345,13 +372,8 @@ class OpSet9(): 'Warnning: paddle not support op:resize wiht mode: linear, we use bilinear replace linear' ) fluid_op = 'resize_bilinear' - node.fluid_code.add_layer( - fluid_op, - inputs={'input': val_x, - 'scale': val_scales}, - output=node, - param_attr=attr) + fluid_op, inputs=inputs, output=node, param_attr=attr) @print_mapping_info def RoiAlign(self, node): @@ -497,7 +519,6 @@ class OpSet9(): 'attribute "shape" of %s not inferred, ' 'using value as 1-D tensor may lead to fails', val_output.layer_name, val_output.layer_name) - if len(value) == 1: value = value.tolist() shape = [1] @@ -814,11 +835,13 @@ class OpSet9(): inputs=val_shape, output=val_shape_cast, param_attr={'dtype': string('int32')}) - node.fluid_code.add_layer( - 'reshape', - inputs=val_shape_cast, - output=val_shape_cast, - param_attr={'shape': val_shape.out_shapes[0]}) + # shape may be [], come form Gather by scalar indices + if len(val_shape.out_shapes[0]) > 0: + node.fluid_code.add_layer( + 'reshape', + inputs=val_shape_cast, + output=val_shape_cast, + param_attr={'shape': val_shape.out_shapes[0]}) node.fluid_code.add_layer( 'reshape', inputs={'x': val_x, @@ -826,11 +849,13 @@ class OpSet9(): output=node, param_attr=attr) else: - node.fluid_code.add_layer( - 'reshape', - inputs=val_shape, - output=val_shape, - param_attr={'shape': val_shape.out_shapes[0]}) + # shape may be [], come form Gather by scalar indices + if len(val_shape.out_shapes[0]) > 0: + node.fluid_code.add_layer( + 'reshape', + inputs=val_shape, + output=val_shape, + param_attr={'shape': val_shape.out_shapes[0]}) node.fluid_code.add_layer( 'reshape', inputs={'x': val_x, diff --git a/x2paddle/op_mapper/paddle2onnx/opset10/opset.py b/x2paddle/op_mapper/paddle2onnx/opset10/opset.py index 72925ddec33c922f5cc6de9140a926552b02a36b..f3dfc693049361fe9ba189a954b3b2b2d6adbea4 100644 --- a/x2paddle/op_mapper/paddle2onnx/opset10/opset.py +++ b/x2paddle/op_mapper/paddle2onnx/opset10/opset.py @@ -47,15 +47,3 @@ class OpSet10(OpSet9): inputs=[op.input('Input')[0], starts_name, ends_name, axes_name], outputs=op.output('Out'), ) return [starts_node, ends_node, axes_node, node] - - def im2sequence(self, op, block): - from .paddle_custom_layer.im2sequence import im2sequence - return im2sequence(op, block) - - def yolo_box(self, op, block): - from .paddle_custom_layer.yolo_box import yolo_box - return yolo_box(op, block) - - def multiclass_nms(self, op, block): - from .paddle_custom_layer.multiclass_nms import multiclass_nms - return multiclass_nms(op, block) diff --git a/x2paddle/op_mapper/paddle2onnx/opset10/paddle_custom_layer/im2sequence.py b/x2paddle/op_mapper/paddle2onnx/opset10/paddle_custom_layer/im2sequence.py deleted file mode 100644 index 84144ca1fdacbc18cc640370352b0ff726b45f0b..0000000000000000000000000000000000000000 --- a/x2paddle/op_mapper/paddle2onnx/opset10/paddle_custom_layer/im2sequence.py +++ /dev/null @@ -1,22 +0,0 @@ -# Copyright (c) 2020 PaddlePaddle Authors. All Rights Reserved. -# -# Licensed under the Apache License, Version 2.0 (the "License" -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -import onnx -import numpy as np -from onnx import onnx_pb, helper -from x2paddle.op_mapper.paddle2onnx.opset9.paddle_custom_layer.im2sequence import im2sequence as im2sequence9 - - -def im2sequence(op, block): - return im2sequence9(op, block) diff --git a/x2paddle/op_mapper/paddle2onnx/opset10/paddle_custom_layer/multiclass_nms.py b/x2paddle/op_mapper/paddle2onnx/opset10/paddle_custom_layer/multiclass_nms.py deleted file mode 100644 index ef57b76f65d41ad6c62730db84b6873b50afdee7..0000000000000000000000000000000000000000 --- a/x2paddle/op_mapper/paddle2onnx/opset10/paddle_custom_layer/multiclass_nms.py +++ /dev/null @@ -1,32 +0,0 @@ -# Copyright (c) 2019 PaddlePaddle Authors. All Rights Reserved. -# -# Licensed under the Apache License, Version 2.0 (the "License" -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -import math -import sys -import os -import numpy as np -import paddle.fluid.core as core -import paddle.fluid as fluid -import onnx -import warnings -from onnx import helper, onnx_pb -from x2paddle.op_mapper.paddle2onnx.opset9.paddle_custom_layer.multiclass_nms import multiclass_nms as multiclass_nms9 - - -def multiclass_nms(op, block): - """ - Convert the paddle multiclass_nms to onnx op. - This op is get the select boxes from origin boxes. - """ - return multiclass_nms9(op, block) diff --git a/x2paddle/op_mapper/paddle2onnx/opset10/paddle_custom_layer/yolo_box.py b/x2paddle/op_mapper/paddle2onnx/opset10/paddle_custom_layer/yolo_box.py deleted file mode 100644 index 563c5b1db2f57f99eeb1c29efad687bc5a0cfab1..0000000000000000000000000000000000000000 --- a/x2paddle/op_mapper/paddle2onnx/opset10/paddle_custom_layer/yolo_box.py +++ /dev/null @@ -1,22 +0,0 @@ -# Copyright (c) 2020 PaddlePaddle Authors. All Rights Reserved. -# -# Licensed under the Apache License, Version 2.0 (the "License" -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -import onnx -import numpy as np -from onnx import onnx_pb, helper -from x2paddle.op_mapper.paddle2onnx.opset9.paddle_custom_layer.yolo_box import yolo_box as yolo_box9 - - -def yolo_box(op, block): - return yolo_box9(op, block) diff --git a/x2paddle/op_mapper/paddle2onnx/opset11/opset.py b/x2paddle/op_mapper/paddle2onnx/opset11/opset.py index 4ec88b5b7d8544f41f545e6758f0803f47b8ebac..d1a48732152c998c50969201ad5b0cca42f9be4a 100644 --- a/x2paddle/op_mapper/paddle2onnx/opset11/opset.py +++ b/x2paddle/op_mapper/paddle2onnx/opset11/opset.py @@ -68,6 +68,19 @@ class OpSet11(OpSet10): mode=op.attr('mode')) return [pads_node, constant_value_node, node] + def clip(self, op, block): + min_name = self.get_name(op.type, 'min') + max_name = self.get_name(op.type, 'max') + min_node = self.make_constant_node(min_name, onnx_pb.TensorProto.FLOAT, + op.attr('min')) + max_node = self.make_constant_node(max_name, onnx_pb.TensorProto.FLOAT, + op.attr('max')) + node = helper.make_node( + 'Clip', + inputs=[op.input('X')[0], min_name, max_name], + outputs=op.output('Out')) + return [min_node, max_node, node] + def bilinear_interp(self, op, block): input_names = op.input_names coordinate_transformation_mode = '' @@ -263,10 +276,6 @@ class OpSet11(OpSet10): node3 ] - def im2sequence(self, op, block): - from .paddle_custom_layer.im2sequence import im2sequence - return im2sequence(op, block) - def yolo_box(self, op, block): from .paddle_custom_layer.yolo_box import yolo_box return yolo_box(op, block) diff --git a/x2paddle/op_mapper/paddle2onnx/opset11/paddle_custom_layer/im2sequence.py b/x2paddle/op_mapper/paddle2onnx/opset11/paddle_custom_layer/im2sequence.py deleted file mode 100644 index be7aa3423d06c781413a87a9ca37cafb9bda5af4..0000000000000000000000000000000000000000 --- a/x2paddle/op_mapper/paddle2onnx/opset11/paddle_custom_layer/im2sequence.py +++ /dev/null @@ -1,22 +0,0 @@ -# Copyright (c) 2020 PaddlePaddle Authors. All Rights Reserved. -# -# Licensed under the Apache License, Version 2.0 (the "License" -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -import onnx -import numpy as np -from onnx import onnx_pb, helper -from x2paddle.op_mapper.paddle2onnx.opset10.paddle_custom_layer.im2sequence import im2sequence as im2sequence10 - - -def im2sequence(op, block): - return im2sequence10(op, block) diff --git a/x2paddle/op_mapper/paddle2onnx/opset11/paddle_custom_layer/multiclass_nms.py b/x2paddle/op_mapper/paddle2onnx/opset11/paddle_custom_layer/multiclass_nms.py index fa578cc65d5c36aec5e006394c38e4a3f3f469b6..b6bb8cce57fc06bde20ce1c0faa68cb9bd615cb0 100644 --- a/x2paddle/op_mapper/paddle2onnx/opset11/paddle_custom_layer/multiclass_nms.py +++ b/x2paddle/op_mapper/paddle2onnx/opset11/paddle_custom_layer/multiclass_nms.py @@ -19,7 +19,7 @@ import numpy as np import paddle.fluid.core as core import paddle.fluid as fluid import onnx -import warnings +import logging from onnx import helper, onnx_pb @@ -42,9 +42,9 @@ def multiclass_nms(op, block): background = attrs['background_label'] normalized = attrs['normalized'] if normalized == False: - warnings.warn( - 'The parameter normalized of multiclass_nms OP of Paddle is False, which has diff with ONNX. \ - Please set normalized=True in multiclass_nms of Paddle') + logging.warn( + "The parameter normalized of multiclass_nms OP of Paddle is False, which has diff with ONNX." \ + " Please set normalized=True in multiclass_nms of Paddle, see doc Q4 in https://github.com/PaddlePaddle/X2Paddle/blob/develop/FAQ.md") #convert the paddle attribute to onnx tensor name_score_threshold = [outputs['Out'][0] + "@score_threshold"] diff --git a/x2paddle/op_mapper/paddle2onnx/opset11/paddle_custom_layer/yolo_box.py b/x2paddle/op_mapper/paddle2onnx/opset11/paddle_custom_layer/yolo_box.py index 5e674d5217693227d4eec33b58f7d252296947f1..919022167b235899422e9bb8079e03ada683a8c5 100644 --- a/x2paddle/op_mapper/paddle2onnx/opset11/paddle_custom_layer/yolo_box.py +++ b/x2paddle/op_mapper/paddle2onnx/opset11/paddle_custom_layer/yolo_box.py @@ -15,21 +15,9 @@ import onnx import numpy as np from onnx import onnx_pb, helper - -MAX_FLOAT = np.asarray([255, 255, 127, 127], dtype=np.uint8).view(np.float32)[0] - - -def get_old_name(arg, name_prefix=''): - prefix_index = arg.find(name_prefix) - - if prefix_index != -1: - last_prefix = arg[len(name_prefix):] - else: - last_prefix = arg - idx = last_prefix.find('@') - if idx != -1: - last_prefix = last_prefix[:idx] - return name_prefix + last_prefix +from x2paddle.op_mapper.paddle2onnx.opset9.paddle_custom_layer.yolo_box import is_static_shape +from x2paddle.op_mapper.paddle2onnx.opset9.paddle_custom_layer.yolo_box import get_old_name +from x2paddle.op_mapper.paddle2onnx.opset9.paddle_custom_layer.yolo_box import MAX_FLOAT32 def yolo_box(op, block): @@ -44,6 +32,7 @@ def yolo_box(op, block): attrs[name] = op.attr(name) model_name = outputs['Boxes'][0] input_shape = block.vars[get_old_name(inputs['X'][0])].shape + is_static_shape(input_shape) image_size = inputs['ImgSize'] input_height = input_shape[2] input_width = input_shape[3] @@ -785,7 +774,7 @@ def yolo_box(op, block): name=max_const_name, data_type=onnx.TensorProto.FLOAT, dims=(), - vals=[MAX_FLOAT])) + vals=[MAX_FLOAT32])) node_list.append(max_const) node_pred_box_x1_clip = onnx.helper.make_node( diff --git a/x2paddle/op_mapper/paddle2onnx/opset9/opset.py b/x2paddle/op_mapper/paddle2onnx/opset9/opset.py index 850e3d0430d8bcb05c209560765b29d61a022edf..06e12f08e7d93632f35fb0f6f453e074ff4e98b9 100644 --- a/x2paddle/op_mapper/paddle2onnx/opset9/opset.py +++ b/x2paddle/op_mapper/paddle2onnx/opset9/opset.py @@ -59,7 +59,7 @@ class OpSet9(object): 'Constant', inputs=[], outputs=[name], value=tensor) return node - def convert_weights(self, program): + def convert_weights(self, program, scope=None): var_names = program.global_block().vars nodes = list() for name in var_names: @@ -68,7 +68,7 @@ class OpSet9(object): continue if not var.persistable: continue - weight = np.array(fluid.global_scope().find_var(name).get_tensor()) + weight = np.array(scope.find_var(name).get_tensor()) tensor = helper.make_tensor( name=name, dims=var.shape, @@ -110,11 +110,32 @@ class OpSet9(object): 'Relu', inputs=op.input('X'), outputs=op.output('Out')) return node + def tanh(self, op, block): + node = helper.make_node( + 'Tanh', inputs=op.input('X'), outputs=op.output('Out')) + return node + + def log(self, op, block): + node = helper.make_node( + 'Log', inputs=op.input('X'), outputs=op.output('Out')) + return node + def sigmoid(self, op, block): node = helper.make_node( 'Sigmoid', inputs=op.input('X'), outputs=op.output('Out')) return node + def clip(self, op, block): + min_value = op.attr('min') + max_value = op.attr('max') + node = helper.make_node( + 'Clip', + inputs=[op.input('X')[0]], + outputs=op.output('Out'), + max=max_value, + min=min_value) + return node + def exp(self, op, block): node = helper.make_node( 'Exp', inputs=op.input('X'), outputs=op.output('Out')) @@ -436,6 +457,14 @@ class OpSet9(object): perm=op.attr('axis')) return node + def flatten2(self, op, block): + node = helper.make_node( + 'Flatten', + inputs=op.input('X'), + outputs=op.output('Out'), + axis=op.attr('axis')) + return node + def reshape2(self, op, block): input_names = op.input_names if len(op.input('ShapeTensor')) > 1: @@ -460,7 +489,7 @@ class OpSet9(object): inputs=[op.input('X')[0], temp_name], outputs=op.output('Out')) return cast_shape_nodes + [shape_node, node] - else: + elif len(op.input('ShapeTensor')) == 1: temp_name = self.get_name(op.type, 'shape.cast') cast_shape_node = helper.make_node( 'Cast', @@ -472,6 +501,16 @@ class OpSet9(object): inputs=[op.input('X')[0], temp_name], outputs=op.output('Out')) return [cast_shape_node, node] + elif op.attr('shape') is not None and len(op.attr('shape')) > 0: + shape_name = self.get_name(op.type, 'shape') + shape_node = self.make_constant_node(shape_name, + onnx_pb.TensorProto.INT64, + op.attr('shape')) + reshape_node = helper.make_node( + 'Reshape', + inputs=[op.input('X')[0], shape_name], + outputs=op.output('Out')) + return [shape_node, reshape_node] def dropout(self, op, block): dropout_mode = op.attr('dropout_implementation') @@ -506,7 +545,7 @@ class OpSet9(object): input_shape = block.vars[op.input('X')[0]].shape if op.attr('align_corners') or op.attr('align_mode') == 0: raise Exception( - "Resize in onnx(opset<=10) only support coordinate_transformation_mode: 'asymmetric'." + "Resize in onnx(opset<=10) only support coordinate_transformation_mode: 'asymmetric', Try converting with --onnx_opest 11" ) if ('OutSize' in input_names and len(op.input('OutSize')) > 0) or ( 'SizeTensor' in input_names and @@ -612,7 +651,7 @@ class OpSet9(object): input_names = op.input_names if op.attr('align_corners'): raise Exception( - "Resize in onnx(opset<=10) only support coordinate_transformation_mode: 'asymmetric'." + "Resize in onnx(opset<=10) only support coordinate_transformation_mode: 'asymmetric', Try converting with --onnx_opest 11" ) if 'OutSize' in input_names and len(op.input('OutSize')) > 0: node = helper.make_node( @@ -766,3 +805,11 @@ class OpSet9(object): def multiclass_nms(self, op, block): from .paddle_custom_layer.multiclass_nms import multiclass_nms return multiclass_nms(op, block) + + def box_coder(self, op, block): + from .paddle_custom_layer.box_coder import box_coder + return box_coder(op, block) + + def prior_box(self, op, block): + from .paddle_custom_layer.prior_box import prior_box + return prior_box(op, block) diff --git a/x2paddle/op_mapper/paddle2onnx/opset9/paddle_custom_layer/box_coder.py b/x2paddle/op_mapper/paddle2onnx/opset9/paddle_custom_layer/box_coder.py new file mode 100644 index 0000000000000000000000000000000000000000..00111fee526b53a735da60da19495231e808f852 --- /dev/null +++ b/x2paddle/op_mapper/paddle2onnx/opset9/paddle_custom_layer/box_coder.py @@ -0,0 +1,401 @@ +# Copyright (c) 2019 PaddlePaddle Authors. All Rights Reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License" +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +import sys +import math +import onnx +import warnings +import numpy as np +from functools import partial +from onnx import TensorProto +from onnx.helper import make_node, make_tensor +from onnx import onnx_pb +from paddle.fluid.executor import _fetch_var as fetch_var +from onnx import helper +import paddle.fluid as fluid +import paddle.fluid.core as core + + +def box_coder(op, block): + """ + In this function, we will use the decode the prior box to target box, + we just use the decode mode to transform this op. + """ + node_list = [] + input_names = op.input_names + + prior_var = block.var(op.input('PriorBox')[0]) + t_size = block.var(op.input('TargetBox')[0]).shape + p_size = prior_var.shape + + # get the outout_name + result_name = op.output('OutputBox')[0] + # n is size of batch, m is boxes num of targe_boxes + n = t_size[0] + m = t_size[0] + + axis = int(op.attr('axis')) + + #norm + norm = bool(op.attr('box_normalized')) + + name_slice_x1 = op.output('OutputBox')[0] + "@x1" + name_slice_y1 = op.output('OutputBox')[0] + "@y1" + name_slice_x2 = op.output('OutputBox')[0] + "@x2" + name_slice_y2 = op.output('OutputBox')[0] + "@y2" + + #make onnx tensor to save the intermeidate reslut + name_slice_indices = [[op.output('OutputBox')[0] + "@slice_" + str(i)] + for i in range(1, 3)] + node_slice_indices = [None for i in range(1, 3)] + + # create the range(0, 4) const data to slice + for i in range(1, 3): + node = onnx.helper.make_node( + 'Constant', + inputs=[], + outputs=name_slice_indices[i - 1], + value=onnx.helper.make_tensor( + name=name_slice_indices[i - 1][0] + "@const", + data_type=onnx.TensorProto.FLOAT, + dims=(), + vals=[i])) + node_list.append(node) + # make node split data + name_box_split = [ + name_slice_x1, name_slice_y1, name_slice_x2, name_slice_y2 + ] + split_shape = list(p_size) + split_shape[-1] = 1 + + node_split_prior_node = onnx.helper.make_node( + 'Split', inputs=op.input('PriorBox'), outputs=name_box_split, axis=1) + node_list.append(node_split_prior_node) + + # make node get centor node for decode + final_outputs_vars = [] + if not norm: + name_centor_w_tmp = [op.output('OutputBox')[0] + "@centor_w_tmp"] + name_centor_h_tmp = [op.output('OutputBox')[0] + "@centor_h_tmp"] + node_centor_w_tmp = None + node_centor_h_tmp = None + name_centor_tmp_list = [name_centor_w_tmp, name_centor_h_tmp] + node_centor_tmp_list = [node_centor_w_tmp, node_centor_h_tmp] + + count = 2 + for (name, node) in zip(name_centor_tmp_list, node_centor_tmp_list): + node = onnx.helper.make_node('Add', + inputs=[op.output('OutputBox')[0] + "@slice_" + str(1)]\ + + [name_box_split[count]], + outputs=name) + node_list.append(node) + count = count + 1 + if not norm: + inputs_sub = [[name_centor_w_tmp[0], name_box_split[0]], + [name_centor_h_tmp[0], name_box_split[1]]] + else: + inputs_sub = [[name_box_split[2], name_box_split[0]], + [name_box_split[3], name_box_split[1]]] + outputs_sub = [result_name + "@pb_w", result_name + "@pb_h"] + for i in range(0, 2): + node = onnx.helper.make_node( + 'Sub', inputs=inputs_sub[i], outputs=[outputs_sub[i]]) + node_list.append(node) + # according to prior_box height and weight to get centor x, y + name_half_value = [result_name + "@half_value"] + node_half_value = onnx.helper.make_node( + 'Constant', + inputs=[], + outputs=name_half_value, + value=onnx.helper.make_tensor( + name=name_slice_indices[i][0] + "@const", + data_type=onnx.TensorProto.FLOAT, + dims=(), + vals=[0.5])) + node_list.append(node_half_value) + outputs_half_wh = [[result_name + "@pb_w_half"], + [result_name + "@pb_h_half"]] + inputs_half_wh = [[result_name + "@pb_w", name_half_value[0]], + [result_name + "@pb_h", name_half_value[0]]] + + for i in range(0, 2): + node = onnx.helper.make_node( + 'Mul', inputs=inputs_half_wh[i], outputs=outputs_half_wh[i]) + node_list.append(node) + + inputs_centor_xy = [[outputs_half_wh[0][0], name_slice_x1], + [outputs_half_wh[1][0], name_slice_y1]] + + outputs_centor_xy = [[result_name + "@pb_x"], [result_name + "@pb_y"]] + + # final calc the centor x ,y + for i in range(0, 2): + node = onnx.helper.make_node( + 'Add', inputs=inputs_centor_xy[i], outputs=outputs_centor_xy[i]) + node_list.append(node) + # reshape the data + shape = (1, split_shape[0]) if axis == 0 else (split_shape[0], 1) + + # need to reshape the data + inputs_transpose_pb = [ + [result_name + "@pb_w"], + [result_name + "@pb_h"], + [result_name + "@pb_x"], + [result_name + "@pb_y"], + ] + outputs_transpose_pb = [ + [result_name + "@pb_w_transpose"], + [result_name + "@pb_h_transpose"], + [result_name + "@pb_x_transpose"], + [result_name + "@pb_y_transpose"], + ] + if axis == 0: + name_reshape_pb = [result_name + "@pb_transpose"] + # reshape the data + for i in range(0, 4): + node = onnx.helper.make_node( + 'Transpose', + inputs=inputs_transpose_pb[i], + outputs=outputs_transpose_pb[i]) + node_list.append(node) + # decoder the box according to the target_box and variacne + name_variance_raw = [result_name + "@variance_raw"] + name_variance_unsqueeze = [result_name + "@variance_unsqueeze"] + shape = [] + # make node to extend the data + var_split_axis = 0 + var_split_inputs_name = [] + if 'PriorBoxVar' in input_names and len(op.input('PriorBoxVar')) > 0: + if axis == 1: + raise Exception( + "The op box_coder has variable do not support aixs broadcast") + prior_variance_var = block.var(op.input('PriorBoxVar')[0]) + axes = [] + var_split_inputs_name = [result_name + "@variance_split"] + node = onnx.helper.make_node( + 'Transpose', + inputs=op.input('PriorBoxVar'), + outputs=var_split_inputs_name) + node_list.append(node) + var_split_axis = 0 + else: + variances = [1.0, 1.0, 1.0, 1.0] + if 'variance' in op.attr and len(op.attr('variance')) > 0: + variances = [float(var) for var in op.attr('variance')] + node_variance_create = onnx.helper.make_node( + 'Constant', + inputs=[], + outputs=name_variance_raw, + value=onnx.helper.make_tensor( + name=name_variance_raw[0] + "@const", + data_type=onnx.TensorProto.FLOAT, + dims=[len(variances)], + vals=variances)) + node_list.append(node_variance_create) + var_split_axis = 0 + var_split_inputs_name = name_variance_raw + + # decode the result + outputs_split_variance = [ + result_name + "@variance_split" + str(i) for i in range(0, 4) + ] + outputs_split_targebox = [ + result_name + "@targebox_split" + str(i) for i in range(0, 4) + ] + node_split_var = onnx.helper.make_node( + 'Split', + inputs=var_split_inputs_name, + outputs=outputs_split_variance, + axis=var_split_axis) + node_split_target = onnx.helper.make_node( + 'Split', + inputs=op.input('TargetBox'), + outputs=outputs_split_targebox, + axis=2) + node_list.extend([node_split_var, node_split_target]) + + outputs_squeeze_targebox = [ + result_name + "@targebox_squeeze" + str(i) for i in range(0, 4) + ] + for (input_name, output_name) in zip(outputs_split_targebox, + outputs_squeeze_targebox): + node = onnx.helper.make_node( + 'Squeeze', inputs=[input_name], outputs=[output_name], axes=[2]) + node_list.append(node) + + output_shape_step1 = list(t_size)[:-1] + + inputs_tb_step1 = [ + [outputs_squeeze_targebox[0], outputs_split_variance[0]], + [outputs_squeeze_targebox[1], outputs_split_variance[1]], + [outputs_squeeze_targebox[2], outputs_split_variance[2]], + [outputs_squeeze_targebox[3], outputs_split_variance[3]] + ] + outputs_tb_step1 = [[result_name + "@decode_x_step1"], + [result_name + "@decode_y_step1"], + [result_name + "@decode_w_step1"], + [result_name + "@decode_h_step1"]] + + for input_step1, output_step_1 in zip(inputs_tb_step1, outputs_tb_step1): + node = onnx.helper.make_node( + 'Mul', inputs=input_step1, outputs=output_step_1) + node_list.append(node) + if axis == 0: + inputs_tbxy_step2 = [ + [outputs_tb_step1[0][0], outputs_transpose_pb[0][0]], + [outputs_tb_step1[1][0], outputs_transpose_pb[1][0]] + ] + else: + inputs_tbxy_step2 = [ + [outputs_tb_step1[0][0], inputs_transpose_pb[0][0]], + [outputs_tb_step1[1][0], inputs_transpose_pb[1][0]] + ] + + outputs_tbxy_step2 = [[result_name + "@decode_x_step2"], + [result_name + "@decode_y_step2"]] + + for input_step2, output_step_2 in zip(inputs_tbxy_step2, + outputs_tbxy_step2): + node = onnx.helper.make_node( + 'Mul', inputs=input_step2, outputs=output_step_2) + node_list.append(node) + if axis == 0: + inputs_tbxy_step3 = [ + [outputs_tbxy_step2[0][0], outputs_transpose_pb[2][0]], + [outputs_tbxy_step2[1][0], outputs_transpose_pb[3][0]] + ] + else: + inputs_tbxy_step3 = [ + [outputs_tbxy_step2[0][0], inputs_transpose_pb[2][0]], + [outputs_tbxy_step2[1][0], inputs_transpose_pb[3][0]] + ] + + outputs_tbxy_step3 = [[result_name + "@decode_x_step3"], + [result_name + "@decode_y_step3"]] + + for input_step3, output_step_3 in zip(inputs_tbxy_step3, + outputs_tbxy_step3): + node = onnx.helper.make_node( + 'Add', inputs=input_step3, outputs=output_step_3) + node_list.append(node) + + # deal with width & height + inputs_tbwh_step2 = [outputs_tb_step1[2], outputs_tb_step1[3]] + outputs_tbwh_step2 = [[result_name + "@decode_w_step2"], + [result_name + "@decode_h_step2"]] + + for input_name, output_name in zip(inputs_tbwh_step2, outputs_tbwh_step2): + node = onnx.helper.make_node( + 'Exp', inputs=input_name, outputs=output_name) + node_list.append(node) + + if axis == 0: + inputs_tbwh_step3 = [ + [outputs_tbwh_step2[0][0], outputs_transpose_pb[0][0]], + [outputs_tbwh_step2[1][0], outputs_transpose_pb[1][0]] + ] + else: + inputs_tbwh_step3 = [ + [outputs_tbwh_step2[0][0], inputs_transpose_pb[0][0]], + [outputs_tbwh_step2[1][0], inputs_transpose_pb[1][0]] + ] + + outputs_tbwh_step3 = [[result_name + "@decode_w_step3"], + [result_name + "@decode_h_step3"]] + + for input_name, output_name in zip(inputs_tbwh_step3, outputs_tbwh_step3): + node = onnx.helper.make_node( + 'Mul', inputs=input_name, outputs=output_name) + node_list.append(node) + + # final step to calc the result, and concat the result to output + # return the output box, [(x1, y1), (x2, y2)] + + inputs_half_tbwh_step4 = [ + [outputs_tbwh_step3[0][0], result_name + "@slice_2"], + [outputs_tbwh_step3[1][0], result_name + "@slice_2"] + ] + + outputs_half_tbwh_step4 = [[result_name + "@decode_half_w_step4"], + [result_name + "@decode_half_h_step4"]] + for inputs_name, outputs_name in zip(inputs_half_tbwh_step4, + outputs_half_tbwh_step4): + node = onnx.helper.make_node( + 'Div', inputs=inputs_name, outputs=outputs_name) + node_list.append(node) + inputs_output_point1 = [ + [outputs_tbxy_step3[0][0], outputs_half_tbwh_step4[0][0]], + [outputs_tbxy_step3[1][0], outputs_half_tbwh_step4[1][0]] + ] + + outputs_output_point1 = [[result_name + "@ouput_x1"], + [result_name + "@output_y1"]] + for input_name, output_name in zip(inputs_output_point1, + outputs_output_point1): + node = onnx.helper.make_node( + 'Sub', inputs=input_name, outputs=output_name) + node_list.append(node) + + inputs_output_point2 = [ + [outputs_tbxy_step3[0][0], outputs_half_tbwh_step4[0][0]], + [outputs_tbxy_step3[1][0], outputs_half_tbwh_step4[1][0]] + ] + + outputs_output_point2 = [[result_name + "@ouput_x2"], + [result_name + "@output_y2"]] + + for input_name, output_name in zip(inputs_output_point2, + outputs_output_point2): + node = onnx.helper.make_node( + 'Add', inputs=input_name, outputs=output_name) + node_list.append(node) + if not norm: + inputs_unnorm_point2 = [ + [outputs_output_point2[0][0], result_name + "@slice_1"], + [outputs_output_point2[1][0], result_name + "@slice_1"] + ] + outputs_unnorm_point2 = [[result_name + "@ouput_unnorm_x2"], + [result_name + "@ouput_unnorm_y2"]] + + for input_name, output_name in zip(inputs_unnorm_point2, + outputs_unnorm_point2): + node = onnx.helper.make_node( + 'Sub', inputs=input_name, outputs=output_name) + node_list.append(node) + outputs_output_point2 = outputs_unnorm_point2 + + outputs_output_point1.extend(outputs_output_point2) + ouputs_points_unsqueeze = [[result_name + "@points_unsqueeze_x1"], + [result_name + "points_unsqueeze_y1"], + [result_name + "points_unsqueeze_x2"], + [result_name + "points_unsqueeze_y2"]] + + for input_name, output_name in zip(outputs_output_point1, + ouputs_points_unsqueeze): + node = onnx.helper.make_node( + 'Unsqueeze', + inputs=input_name, + outputs=output_name, + axes=[len(output_shape_step1)]) + node_list.append(node) + outputs_points_unsqueeze_list = [ + output[0] for output in ouputs_points_unsqueeze + ] + node_point_final = onnx.helper.make_node( + 'Concat', + inputs=outputs_points_unsqueeze_list, + outputs=op.output('OutputBox'), + axis=len(output_shape_step1)) + node_list.append(node_point_final) + return node_list diff --git a/x2paddle/op_mapper/paddle2onnx/opset9/paddle_custom_layer/multiclass_nms.py b/x2paddle/op_mapper/paddle2onnx/opset9/paddle_custom_layer/multiclass_nms.py index 58d8e0c9d1c9e780b62cfba578973512e1214ba8..65430bb159bb4698b62fa9bda6b572062b49b6fc 100644 --- a/x2paddle/op_mapper/paddle2onnx/opset9/paddle_custom_layer/multiclass_nms.py +++ b/x2paddle/op_mapper/paddle2onnx/opset9/paddle_custom_layer/multiclass_nms.py @@ -19,7 +19,7 @@ import numpy as np import paddle.fluid.core as core import paddle.fluid as fluid import onnx -import warnings +import logging from onnx import helper, onnx_pb @@ -42,9 +42,9 @@ def multiclass_nms(op, block): background = attrs['background_label'] normalized = attrs['normalized'] if normalized == False: - warnings.warn( - 'The parameter normalized of multiclass_nms OP of Paddle is False, which has diff with ONNX. \ - Please set normalized=True in multiclass_nms of Paddle') + logging.warn( + "The parameter normalized of multiclass_nms OP of Paddle is False, which has diff with ONNX." \ + " Please set normalized=True in multiclass_nms of Paddle, see doc Q4 in https://github.com/PaddlePaddle/X2Paddle/blob/develop/FAQ.md") #convert the paddle attribute to onnx tensor name_score_threshold = [outputs['Out'][0] + "@score_threshold"] diff --git a/x2paddle/op_mapper/paddle2onnx/opset9/paddle_custom_layer/prior_box.py b/x2paddle/op_mapper/paddle2onnx/opset9/paddle_custom_layer/prior_box.py new file mode 100644 index 0000000000000000000000000000000000000000..71f6f3f60b6ce82a9cad809aeccef40e08209baf --- /dev/null +++ b/x2paddle/op_mapper/paddle2onnx/opset9/paddle_custom_layer/prior_box.py @@ -0,0 +1,174 @@ +# Copyright (c) 2019 PaddlePaddle Authors. All Rights Reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License" +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +import sys +import math +import onnx +import warnings +import numpy as np +from functools import partial +from onnx import TensorProto +from onnx.helper import make_node, make_tensor +from onnx import onnx_pb +from paddle.fluid.executor import _fetch_var as fetch_var +from onnx import helper +import paddle.fluid as fluid +import paddle.fluid.core as core + + +def ExpandAspectRations(input_aspect_ratior, flip): + expsilon = 1e-6 + output_ratios = [1.0] + for input_ratio in input_aspect_ratior: + already_exis = False + for output_ratio in output_ratios: + if abs(input_ratio - output_ratio) < expsilon: + already_exis = True + break + if already_exis == False: + output_ratios.append(input_ratio) + if flip: + output_ratios.append(1.0 / input_ratio) + return output_ratios + + +def prior_box(op, block): + """ + In this function, use the attribute to get the prior box, because we do not use + the image data and feature map, wo could the python code to create the varaible, + and to create the onnx tensor as output. + """ + flip = bool(op.attr('flip')) + clip = bool(op.attr('clip')) + min_max_aspect_ratios_order = bool(op.attr('min_max_aspect_ratios_order')) + min_sizes = [float(size) for size in op.attr('min_sizes')] + max_sizes = [float(size) for size in op.attr('max_sizes')] + if isinstance(op.attr('aspect_ratios'), list): + aspect_ratios = [float(ratio) for ratio in op.attr('aspect_ratios')] + else: + aspect_ratios = [float(op.attr('aspect_ratios'))] + variances = [float(var) for var in op.attr('variances')] + # set min_max_aspect_ratios_order = false + output_ratios = ExpandAspectRations(aspect_ratios, flip) + + step_w = float(op.attr('step_w')) + step_h = float(op.attr('step_h')) + offset = float(op.attr('offset')) + + input_shape = block.var(op.input('Input')[0]).shape + image_shape = block.var(op.input('Image')[0]).shape + + img_width = image_shape[3] + img_height = image_shape[2] + feature_width = input_shape[3] + feature_height = input_shape[2] + + step_width = 1.0 + step_height = 1.0 + + if step_w == 0.0 or step_h == 0.0: + step_w = float(img_width / feature_width) + step_h = float(img_height / feature_height) + + num_priors = len(output_ratios) * len(min_sizes) + if len(max_sizes) > 0: + num_priors += len(max_sizes) + out_dim = (feature_height, feature_width, num_priors, 4) + out_boxes = np.zeros(out_dim).astype('float32') + out_var = np.zeros(out_dim).astype('float32') + + idx = 0 + for h in range(feature_height): + for w in range(feature_width): + c_x = (w + offset) * step_w + c_y = (h + offset) * step_h + idx = 0 + for s in range(len(min_sizes)): + min_size = min_sizes[s] + if not min_max_aspect_ratios_order: + # rest of priors + for r in range(len(output_ratios)): + ar = output_ratios[r] + c_w = min_size * math.sqrt(ar) / 2 + c_h = (min_size / math.sqrt(ar)) / 2 + out_boxes[h, w, idx, :] = [ + (c_x - c_w) / img_width, (c_y - c_h) / img_height, + (c_x + c_w) / img_width, (c_y + c_h) / img_height + ] + idx += 1 + + if len(max_sizes) > 0: + max_size = max_sizes[s] + # second prior: aspect_ratio = 1, + c_w = c_h = math.sqrt(min_size * max_size) / 2 + out_boxes[h, w, idx, :] = [ + (c_x - c_w) / img_width, (c_y - c_h) / img_height, + (c_x + c_w) / img_width, (c_y + c_h) / img_height + ] + idx += 1 + else: + c_w = c_h = min_size / 2. + out_boxes[h, w, idx, :] = [ + (c_x - c_w) / img_width, (c_y - c_h) / img_height, + (c_x + c_w) / img_width, (c_y + c_h) / img_height + ] + idx += 1 + if len(max_sizes) > 0: + max_size = max_sizes[s] + # second prior: aspect_ratio = 1, + c_w = c_h = math.sqrt(min_size * max_size) / 2 + out_boxes[h, w, idx, :] = [ + (c_x - c_w) / img_width, (c_y - c_h) / img_height, + (c_x + c_w) / img_width, (c_y + c_h) / img_height + ] + idx += 1 + + # rest of priors + for r in range(len(output_ratios)): + ar = output_ratios[r] + if abs(ar - 1.) < 1e-6: + continue + c_w = min_size * math.sqrt(ar) / 2 + c_h = (min_size / math.sqrt(ar)) / 2 + out_boxes[h, w, idx, :] = [ + (c_x - c_w) / img_width, (c_y - c_h) / img_height, + (c_x + c_w) / img_width, (c_y + c_h) / img_height + ] + idx += 1 + + if clip: + out_boxes = np.clip(out_boxes, 0.0, 1.0) + # set the variance. + out_var = np.tile(variances, (feature_height, feature_width, num_priors, 1)) + + #make node that + node_boxes = onnx.helper.make_node( + 'Constant', + inputs=[], + outputs=op.output('Boxes'), + value=onnx.helper.make_tensor( + name=op.output('Boxes')[0] + "@const", + data_type=onnx.TensorProto.FLOAT, + dims=out_boxes.shape, + vals=out_boxes.flatten())) + node_vars = onnx.helper.make_node( + 'Constant', + inputs=[], + outputs=op.output('Variances'), + value=onnx.helper.make_tensor( + name=op.output('Variances')[0] + "@const", + data_type=onnx.TensorProto.FLOAT, + dims=out_var.shape, + vals=out_var.flatten())) + return [node_boxes, node_vars] diff --git a/x2paddle/op_mapper/paddle2onnx/opset9/paddle_custom_layer/yolo_box.py b/x2paddle/op_mapper/paddle2onnx/opset9/paddle_custom_layer/yolo_box.py index c1a37030302b94a111a019c3c8713cdbc1b02364..bee7d8c6ccf7d5aa7e42a2939891dd48f1908ace 100644 --- a/x2paddle/op_mapper/paddle2onnx/opset9/paddle_custom_layer/yolo_box.py +++ b/x2paddle/op_mapper/paddle2onnx/opset9/paddle_custom_layer/yolo_box.py @@ -16,6 +16,9 @@ import onnx import numpy as np from onnx import onnx_pb, helper +MAX_FLOAT32 = np.asarray( + [255, 255, 127, 127], dtype=np.uint8).view(np.float32)[0] + def get_old_name(arg, name_prefix=''): prefix_index = arg.find(name_prefix) @@ -30,6 +33,13 @@ def get_old_name(arg, name_prefix=''): return name_prefix + last_prefix +def is_static_shape(shape): + if len(shape) > 1 and shape.count(-1) > 1: + raise Exception( + "Converting this model to ONNX need with static input shape, please fix input shape of this model, see doc Q5 in https://github.com/PaddlePaddle/X2Paddle/blob/develop/FAQ.md." + ) + + def yolo_box(op, block): inputs = dict() outputs = dict() @@ -42,6 +52,7 @@ def yolo_box(op, block): attrs[name] = op.attr(name) model_name = outputs['Boxes'][0] input_shape = block.vars[get_old_name(inputs['X'][0])].shape + is_static_shape(input_shape) image_size = inputs['ImgSize'] input_height = input_shape[2] input_width = input_shape[3] @@ -766,7 +777,7 @@ def yolo_box(op, block): inputs=outputs_pred_box_x1_decode, outputs=outputs_pred_box_x1_clip, min=0.0, - max=float(np.inf)) + max=float(MAX_FLOAT32)) node_list.append(node_pred_box_x1_clip) node_pred_box_y1_clip = onnx.helper.make_node( @@ -774,7 +785,7 @@ def yolo_box(op, block): inputs=outputs_pred_box_y1_decode, outputs=outputs_pred_box_y1_clip, min=0.0, - max=float(np.inf)) + max=float(MAX_FLOAT32)) node_list.append(node_pred_box_y1_clip) node_pred_box_x2_clip = onnx.helper.make_node( @@ -782,7 +793,7 @@ def yolo_box(op, block): inputs=outputs_pred_box_x2_sub_w, outputs=outputs_pred_box_x2_clip, min=0.0, - max=float(np.inf)) + max=float(MAX_FLOAT32)) node_list.append(node_pred_box_x2_clip) node_pred_box_y2_clip = onnx.helper.make_node( @@ -790,7 +801,7 @@ def yolo_box(op, block): inputs=outputs_pred_box_y2_sub_h, outputs=outputs_pred_box_y2_clip, min=0.0, - max=float(np.inf)) + max=float(MAX_FLOAT32)) node_list.append(node_pred_box_y2_clip) outputs_pred_box_x2_res = [model_name + "@box_x2_res"] diff --git a/x2paddle/op_mapper/paddle2onnx/paddle_op_mapper.py b/x2paddle/op_mapper/paddle2onnx/paddle_op_mapper.py index 1ce2ec5e7093d6d5302e673c4400fe0a87d66583..f167dfdd73b05aae2036c2ab4001c7c6a838d267 100644 --- a/x2paddle/op_mapper/paddle2onnx/paddle_op_mapper.py +++ b/x2paddle/op_mapper/paddle2onnx/paddle_op_mapper.py @@ -33,9 +33,9 @@ class PaddleOpMapper(object): self.name_counter = dict() self.op_set = None - def convert(self, program, save_dir, opset_number=10): - self.op_set = self.create_opset(opset_number) - weight_nodes = self.op_set.convert_weights(program) + def convert(self, program, save_dir, scope=None, opset_version=10): + self.op_set = self.create_opset(opset_version) + weight_nodes = self.op_set.convert_weights(program, scope=scope) op_nodes = list() input_nodes = list() output_nodes = list() @@ -77,7 +77,7 @@ class PaddleOpMapper(object): initializer=[], inputs=input_nodes, outputs=output_nodes) - opset_imports = [helper.make_opsetid("", opset_number)] + opset_imports = [helper.make_opsetid("", opset_version)] model = helper.make_model( graph, producer_name='X2Paddle', opset_imports=opset_imports) onnx.checker.check_model(model) @@ -89,20 +89,20 @@ class PaddleOpMapper(object): print("\nTranslated model saved in {}".format( os.path.join(save_dir, 'x2paddle_model.onnx'))) - def create_opset(self, opset_number): + def create_opset(self, opset_version=10): run_opset = self.default_opset opset = '' - if opset_number in self.support_opsets: - run_opset = opset_number + if opset_version in self.support_opsets: + run_opset = opset_version else: - for support_opset_number in self.support_opsets: - if support_opset_number < opset_number: - run_opset = support_opset_number + for support_opset_version in self.support_opsets: + if support_opset_version < opset_version: + run_opset = support_opset_version else: break print( 'Now, onnx2paddle support convert onnx model opset_verison {},' 'opset_verison of your onnx model is {}, automatically treated as op_set: {}.' - .format(self.support_opsets, opset_number, run_opset)) + .format(self.support_opsets, opset_version, run_opset)) opset = 'OpSet' + str(run_opset) return eval(opset)() diff --git a/x2paddle_model_zoo.md b/x2paddle_model_zoo.md index d2a95c735b1a80a9680e0b9b39958bed1b6702f2..73c65511de064115d958483172a38949db93fe75 100644 --- a/x2paddle_model_zoo.md +++ b/x2paddle_model_zoo.md @@ -33,6 +33,8 @@ | MobileNet_V1 | [code](https://github.com/shicai/MobileNet-Caffe) | | MobileNet_V2 | [code](https://github.com/shicai/MobileNet-Caffe) | | ShuffleNet_v2 | [code](https://github.com/miaow1988/ShuffleNet_V2_pytorch_caffe/releases/tag/v0.1.0) | +| InceptionV3 | [code](https://github.com/soeaver/caffe-model/blob/master/cls/inception/) | +| InceptionV4 | [code](https://github.com/soeaver/caffe-model/blob/master/cls/inception/) | | mNASNet | [code](https://github.com/LiJianfei06/MnasNet-caffe) | | MTCNN | [code](https://github.com/kpzhang93/MTCNN_face_detection_alignment/tree/master/code/codes/MTCNNv1/model) | | Mobilenet_SSD | [code](https://github.com/chuanqi305/MobileNet-SSD) |