Former-commit-id: 34928967

* split BaseObject and EagerBaseObject

* Use BlobObject in FeedOrFetch, rename functions

* ForEachIbnAndEagerBlobObject->ForEachIbnAndBlobObject

* address reviews

* give lazy ref blob a full blob desc

* rename ForEachObnAndEagerBlobObject->ForEachMutBnAndBlobObject, ForEachIbnAndBlobObject->ForEachConstBnAndBlobObject

* fix bug

Former-commit-id: 4131ea97

* use cocoapi release

* add SKIP_WHEEL

* fix lr compile

* add TODO

* fix bias add grad

* change cond to check
Co-authored-by: Ntsai <caishenghang@oneflow.org>
Former-commit-id: 448310a2

Former-commit-id: 64deea08

Former-commit-id: d40a6f81

Former-commit-id: 65069051

* compile tf 2.3.0 with gcc 7.3

* fix oneflow eigen

* minor fix

* fix include

* update protobuf if xla is on

* update path of tf proto generated cpp files

* fix path in script

* add .clangd to git ignore

* update xla ifs

* update scripts

* update path in script for clangd

* add gitignore

* add cmake flag XRT_TF_URL

* rm comment

* check in changes

* bash tricks to enable gcc 7.3

* use arg to control tuna

* bumpversion

* fix build wheel

* use real path

* add dir for cpu

* fix unwanted yum update cublas

* uncomment all

* rm suffix of wheelhouse_dir

* add log info
Co-authored-by: Ntsai <caishenghang@1f-dev.kbaeegfb1x0ubnoznzequyxzve.bx.internal.cloudapp.net>
Co-authored-by: Ntsai <caishenghang@oneflow.org>
Former-commit-id: da12e8db

Former-commit-id: 70db228a

* change rmsprop optimizer mean_square from temp_buffer to normal variable

* add centered parameter to rmsprop & add unit test for it

* add unit test for adam optimizer

* add unit test for sgd

* add unit test for lazy adam optimizer

* add unit test for lars optimizer

* make changes according to the review

* add random mask multiplier at each iteration

Former-commit-id: 474f2fe3

Co-authored-by: Noneflow-bot <69100618+oneflow-bot@users.noreply.github.com>
Former-commit-id: a6175f08

Former-commit-id: 2a0480b3

Co-authored-by: Noneflow-bot <69100618+oneflow-bot@users.noreply.github.com>
Former-commit-id: 7f741ae7

* FuseAddToOutput

* change default value of enable_fuse_add_to_output to false

* consider reachability

* supported_op_type_name2output_bn=>supported_op_type_name2output_arg

* Add necessary comments

* refine

* fix
Co-authored-by: Nguo ran <360112263@qq.com>
Former-commit-id: fd2fe579

* add api docs

* fix docs

* add api doc in math_ops

* Change the theme to read_the_docs

* add equation docs in math_add

* add equation docs in sub, mul, addn

* add mod div tanh docs

* add gelu, relu docs

* add sigmoid unsorted_segment_sum api docs

* add unsorted_segment example docs

* add equal to greate_equal example docs

* add logical and example docs

* add other example docs, include squared difference

* add requirements of sphinx_rtd_theme

* delete the requirement of sphinx-material

* use make of format

* add optimizer docs

* add scheduler docs

* fix multiplier docs in warmup

* add api docs for math binary elementwise ops

* add math unary api docs

* fix docs about round

* add api docs for reduce mean

* add api docs for reduce ops

* fix code block for reduce ops

* fix math equation

* fix math equation erf

* add code example for layers

* fix docs in layers

* fix of format

* add warmup module in rst

* fix focs for sbp author:yaochi

* fix details in api docs

* add nn api docs and add optimizer_grad_clip module in rst

* fix details in api docs

* fix details in nn

* add prelu api docs

* add smooth l1 loss api docs

* add api docs for matmul

* add api docs for sortops

* add direction explanation in sortops

* add regularizer api docs

* fix layers api docs

* fix apidocs # output to # out

* fix land symbol in logical_and

* add initializer api docs

* fix warmup name error in Sphinx

* rename globalnorm to fix error in sphinx docs

* add more examples for initializer

* Fix output to out

* fix detail

* add api docs for categorical_ordinal_encode

* add api docs for watch

* add api docs for one hot

* add api docs for pad

* fix expression in categorical ordinal encode

* fix hash precomputed docs

* fix of_format
Co-authored-by: Ndoombeaker <later@usopp.net>
Co-authored-by: NShenghang Tsai <jackalcooper@gmail.com>
Co-authored-by: Noneflow-bot <69100618+oneflow-bot@users.noreply.github.com>
Former-commit-id: a3952faa

* Migrate L1L2RegularizeGradientOp to UserOp Framework

* ScopeSymbolId

Former-commit-id: 2243f12b

* add tril op

* fix code format

* rename arg names

* optimize tril op

* optimize tril op

* add half compute of scalar_by_tensor_add_op

* optimize tril op & add half unit test

* optimize tril op

* add tril op doc

* rename TrilCalGpu to TrilGpu

* fix tril op doc

* fix C++ format

* add half compute of scalar_by_tensor_op

* optimize tril

Former-commit-id: 95a3e2db

* add masked fill op

* add blank line at end of file

* reimplement masked_fill & optimize masked_fill test scrip

* fix code format

* delete old version masked_fill op impl

* add doc for maksed_fill

* add half compute of a series of op

* add half test of masked_fill op

* fix code format

* fix masked_fill op doc

* fix masked_fill op doc

* Update oneflow/core/kernel/broadcast_to_compatible_with_kernel.cpp

change piar to macro
Co-authored-by: NJuncheng <liujuncheng1022@gmail.com>

* optimize masked_fill

* regist float16 of scalar_by_tensor sub, mul, div
Co-authored-by: NJuncheng <liujuncheng1022@gmail.com>
Former-commit-id: 4511d649

Former-commit-id: 1905fe78

* clean up so

* put cpu script in a function

* fix package name

* dont mount build/lib

Former-commit-id: 1ad88512

* broadcast_div_grad add split sbp

* refine

Former-commit-id: d722b5f5

* Fused BatchNormAddRelu

* skip if cpu

* d_addend=>addend_diff

Former-commit-id: 5f7a1ba7

* upgrad to pybind11 master and pass exe path

* quick fix

* use uft8 in py fmt
Co-authored-by: Ntsai <caishenghang@oneflow.org>
Former-commit-id: 0a647bcf

* flat ErrorProto error_type

* add function to Error

* fix check logic

* add Error function Assign
Co-authored-by: NLi Xinqi <lixinqi2010@gmail.com>
Former-commit-id: f0776ddd

* naive integration

* fix order

* make cxx flags global

* check module load

* fmt

* rm example code

* always add pybind11 in cmake

* mv cxx flags up

* rename internal2 =>  oneflow api

* rm old deprecated code

* update setup.py

* add of_pybind_obj_cc

* mv pybind within cpp rule

* check pathname

* rm unused include header
Co-authored-by: Ntsai <caishenghang@oneflow.org>
Former-commit-id: 0b41436b

* add api docs

* fix docs

* add api doc in math_ops

* Change the theme to read_the_docs

* add equation docs in math_add

* add equation docs in sub, mul, addn

* add mod div tanh docs

* add gelu, relu docs

* add sigmoid unsorted_segment_sum api docs

* add unsorted_segment example docs

* add equal to greate_equal example docs

* add logical and example docs

* add other example docs, include squared difference

* add requirements of sphinx_rtd_theme

* delete the requirement of sphinx-material

* use make of format

* add optimizer docs

* add scheduler docs

* fix multiplier docs in warmup

* add api docs for math binary elementwise ops

* add math unary api docs

* fix docs about round

* add api docs for reduce mean

* add api docs for reduce ops

* fix code block for reduce ops

* fix math equation

* fix math equation erf

* add code example for layers

* fix docs in layers

* fix of format

* add warmup module in rst

* fix focs for sbp author:yaochi

* fix details in api docs

* add nn api docs and add optimizer_grad_clip module in rst

* fix details in api docs
Co-authored-by: Ndoombeaker <later@usopp.net>
Co-authored-by: NShenghang Tsai <jackalcooper@gmail.com>
Co-authored-by: Noneflow-bot <69100618+oneflow-bot@users.noreply.github.com>
Former-commit-id: eb6d3cc3

* fix bug

* add test

* fix consistent_user_op_builder

* fix bug and update test

* more accuracy api define and add test

Former-commit-id: 7480d36b

Former-commit-id: 509a2aa0

Former-commit-id: 7733009b

Former-commit-id: 7a231a4a

* fix ubuntu build

* skip wheel install

* dont rm file

* fmt

* rm cmake from apt
Co-authored-by: Ntsai <caishenghang@1f-dev.kbaeegfb1x0ubnoznzequyxzve.bx.internal.cloudapp.net>
Co-authored-by: Ntsai <caishenghang@oneflow.org>
Former-commit-id: 576c2cb9

Former-commit-id: 1f42431a

* fix load mnist data

* format
Co-authored-by: Nguo ran <360112263@qq.com>
Former-commit-id: 818d50d2

* Fix do_bias_correction for Adam

* refine

Former-commit-id: ad1aac6e

* bump version 0.1.9

* update readme

* update script

* add cuda 11 oneflow

* add cpu install

* add cuda 11 script

* add cpu only instruction
Co-authored-by: Ntsai <caishenghang@1f-dev.kbaeegfb1x0ubnoznzequyxzve.bx.internal.cloudapp.net>
Former-commit-id: 297e636f

* add api docs

* fix docs

* add api doc in math_ops

* Change the theme to read_the_docs

* add equation docs in math_add

* add equation docs in sub, mul, addn

* add mod div tanh docs

* add gelu, relu docs

* add sigmoid unsorted_segment_sum api docs

* add unsorted_segment example docs

* add equal to greate_equal example docs

* add logical and example docs

* add other example docs, include squared difference

* add requirements of sphinx_rtd_theme

* delete the requirement of sphinx-material

* use make of format
Co-authored-by: Ndoombeaker <later@usopp.net>
Co-authored-by: NShenghang Tsai <jackalcooper@gmail.com>
Co-authored-by: Noneflow-bot <69100618+oneflow-bot@users.noreply.github.com>
Former-commit-id: 3a1ef10f

Former-commit-id: 2ec8fc69

* add SubTskGphBuilderUtil::BuildBoxingInfo() interface

* add BuildBoxingInfo to sub_task_graph_builder

* save boxing log

* add not nullptr check

* add TYR

* remove blank line

* change BuildBoxingInfo to BuildBoxingLogInfo

* fix code style

* rename boxing_logging_ to boxing_logging_lines_

* fix c++ code format

* rename boxing_info to boxing_log_line

* rename boxing_info to boxing_log_line

* fix code format

* abstract SubTskGphBuilderStatus struct

* rename boxing_log_line to boxingbuilderstatus

* delete useless code

* fix code format

* remove useless include file

* complete GetBlobInfo4LogicalBlobDesc

* change boxing_log_lines to boxing_log_list_(std::list)

* Optimize log saving function

* remove useless include file

* add colnum filed  Macro

* add CHECK_NOT_NULL

* Overload SetLogStream

* change inappropriate variable name

* remove usleless include file

* add CHECK_JUST & TRY

* fix c++ format

* add new line at end of file

* optimization boxing log

* remove useless include

* fix c++ format

* optimize SubTskGphBuilderStatus Getters

* change macro name

* fix c++ format

* optimize boxing log

* fix c++ format

* rename SerializeParallelDesc to ParallelDescToString

* optimize boxing log

* fix small bug

* optimize boxing log

* fix c++ format

* optimize boxing log

* fix c++ format

* optimize boxing log

* fix c++ format

* optimize boxing log

* optimize boxing log

* fix c++ format

* optimize boxing log

* remove useless include file

* optimize boxing log
Co-authored-by: Noneflow-bot <69100618+oneflow-bot@users.noreply.github.com>
Former-commit-id: e0370ccd

* move generated python files into oneflow/python_gen

* python_gen.framework.sysconfig -> python_gen.sysconfig

* write file in build step instead of configure step

Former-commit-id: 1d35fa1e

* conv2d cpu groups

* address review
Co-authored-by: Noneflow-bot <69100618+oneflow-bot@users.noreply.github.com>
Former-commit-id: 79256ae2

Former-commit-id: 0fb58c4d