* vm::cuda_allocator interface

* Implement of CudaAllocator::Allocate

* Implement of CudaAllocator::Deallocate

* Add cuda allocator test scripts and fix bug of CudaAllocator::pieces_ and pass test

* remove part note

* fix CudaAllocator::total_memory_bytes_ strategy and Pass all test

* CudaAllocator Support dynamic growth and garbage collection

* add interface note

* refine total_memory_bytes when garbage collection

* fix bug of merge

* remove log and strong boundary test

* SingleThreadOnlyAllocator and ThreadSafeAllocator implement and test

* SingleThreadOnlyAllcoator for CudaStreamHanldeDeviceCtx

* ThreadOnlyAllcoator for CudaStreamHanldeDeviceCtx

* fix spell
Co-authored-by: NLi Xinqi <lixinqi2010@gmail.com>
Co-authored-by: Nguo ran <360112263@qq.com>

* use cudnnDeriveBNTensorDescriptor

* CudnnTensorDescHelper

* fix bn min eps
Co-authored-by: Nguo ran <360112263@qq.com>

Co-authored-by: NLi Xinqi <lixinqi2010@gmail.com>

* refactor_placement

* delete useless code

* fix code style

* remove prior device placement call

* fix placement_util

Co-authored-by: NLi Xinqi <lixinqi2010@gmail.com>

* lazy adam model_diff mut_dptr->dptr

* of format
Co-authored-by: NJuncheng <liujuncheng1022@gmail.com>

* add siganture to ops api

* Adjust function signature

* remove python api keras

* remove detection.py

* dynamic_concat

* check dynamic shape less than static shape

* grad and test

* update kernel match to hob

* combine dynamic concat to concat

* minor fix

* backward compatible

* minor fix

* modify by review

* refactor rand input code

* fix bug

* update test

* update test

* changes by suggestions

* check log

* refactor if

* rename

* add import as
Co-authored-by: Ntsai <caishenghang@oneflow.org>

Co-authored-by: NLi Xinqi <lixinqi2010@gmail.com>

* save naive draft

* works but unittest not found

* update api gen

* not working

* push to compare

* restore file

* revert changes

* add rm

* dont' clean

* smarter cleanup
Co-authored-by: Ntsai <caishenghang@oneflow.org>

Co-authored-by: NLi Xinqi <lixinqi2010@gmail.com>

* #3196 refactor user op c++ register draft

* #3196 refactor op register c++ api part2

* #3196 refine interface

* #3196 user new get

* #3196 rename

* #3196 finish Op reg

* #3196 finish Op grad reg

* #3196 refine

* #3196 op kernel reg part 1

* #3196 op kernel reg part 2

* #3196 op kernel reg part 3

* #3196 op kernel reg part 4

* #3196 op kernel reg part 5 fix

* #3196 compile & test pass

* #3196 merge branch develop

* #3196 rm useless code

* add load_mnist.py for document demo

* remove chinese character

* del test code

* del useless code

* del useless code

* modify load_mnist, add tqdm to setup.py

* add tqdm to dev-requirements.txt

* refine load_mnist.py

* add code api doc

* refine docstring

* undo changes of api docs

* add required library 'requests' to setup.py
Co-authored-by: NShenghang Tsai <jackalcooper@gmail.com>

* fix and test

* fix
Co-authored-by: NLi Xinqi <lixinqi2010@gmail.com>

* rm support for legacy code

* rm user op in task name and script name
Co-authored-by: Ntsai <caishenghang@oneflow.org>

Co-authored-by: NShenghang Tsai <jackalcooper@gmail.com>

* add upsample op

* rename

* use NdIndexOffsetHelper

* add test case

* refine

* fix

* refine

* fix

* ndarray()->numpy()

* format

* fix

* refine

* use flow.scope.placement
Co-authored-by: NJuncheng <liujuncheng1022@gmail.com>

* optimize L1L2RegularizeGradientKernel

* use thread pool

* Remove reduce/broadcast system op

* Remove add system op

* fix

* fix op type name

* Remove constant system op

* fix

* fix

* AutoTrainStep use assign user op


boxing util use assign


assign to system.assign


revert

* flow.assign use user op
Co-authored-by: Nguo ran <360112263@qq.com>

* replace FixedTensorDef with Numpy.Def

* oneflow.Numpy.Def demo

* support eager global_function input signature

* replace FixedTensorDef with Numpy.Def

* oneflow.Numpy.Def demo

* remove code in Attr relevant to 'attr_type_name'

* refine Attr call of array_ops.py

* refine Attr call of categorical_ordinal_encode_op.py constant_op.py data_ops.py

* refine Attr call of layers.py loss_ops.py

* refine Attr call in eager_nccl_ops.py  layers.py linalg.py math_ops.py

* refine Attr call in math_ops.py

* refactor Attr call in nn_ops.py

* refactor Attr call in 6 files

one_hot.py
pad.py
random_ops.py
reduce_ops.py
sort_ops.py
two_stage_reduce.py

* refactor Attr call in user_data_ops.py

* refactor Attr call in 9 files

user_op_builder.py
test_reshape.py
test_TestDataTypeAttr.py
test_TestListDataTypeAndListShapeAndListStringAttr.py
test_TestRandomSource.py
test_TestReshape.py
test_TestReshape4KeepHeaderOnly.py
test_TestSourceMultiGpuFixedOutNum.py
test_user_op_attr_auto_type.py

* remove 3rd argument of Attr method

* reformat

* refactor Attr call in nn_ops.py line 710

* add 'attr_type_name=None' for downward compatibility

* add warning message for deprecated Attr call
Co-authored-by: NYao Chi <yaochi@oneflow.org>
Co-authored-by: NLi Xinqi <lixinqi2010@gmail.com>

* remove boxing_v1

* of_format

* Remove MayConsumeModelDiff

* rm RankContext

* remove LossLogicalNode

* add deprecated info

* remove  outdated code

* Revert BoxingOp/Kernel

* always do DiffStaticShapeCast

* fix
Co-authored-by: NLi Xinqi <lixinqi2010@gmail.com>

* AddDiffStaticShapeCast

* PruneCastToStaticShapeOpsPass

* only enable when boxing_v2

* boxing v2
Co-authored-by: NLi Xinqi <lixinqi2010@gmail.com>

* add alias to some python api
1. oneflow.device_prior_placement -> oneflow.scope.placement
2. oneflow.distribute.consistent_strategy -> oneflow.scope.consistent_view
3. oneflow.distribute.mirrored_strategy -> oneflow.scope.mirrored_view
4. oneflow.name_scope -> oneflow.scope.namespace

deprecated api names are NOT removed yet

* updates revelant docstrings
docstrings of functions below updated:
1. scope.consistent_view
2. scope.mirrored_view
3. scope.namespace

* run make of_format

* add deprecated function wrapper

1. deprecated_consistent_strategy for "distribute.consistent_strategy"
2. deprecated_mirrored_strategy for "distribute.mirrored_strategy"
3. deprecated_name_scope for "name_scope", "experimental.name_scope", "deprecated.variable_scope"

* refine fixed_placement/device_prior_placement

1. add "fxied_placment"/"device_prior_placement" wrapper for depreacted functions
2. rename "api_device_prior_placement" to "api_placement"

* using (*args, **kwargs) for depreacted function
1. def deprecated_placement(*args, **kwargs)
2. def deprecated_name_scope(*args, **kwargs)
3. update docstring: consistent/mirroed strategy view -> consistent/mirroed view

* exported twice of 'fixed_placement', solved

* use traceback with warning on deprecated functions
relevant items:
  "device_prior_placement"
  "fixed_placement"
  "distribute.consistent_strategy"
  "distribute.mirrored_strategy"

* reformat code

* index range [-2:-1] -> [-2] when print traceback

* using index instead of index range when print traceback info

* print depreacted function name using literal string instead of _ONEFLOW_API

* name scope -> namespace

* fix docstring typos of scope.namespace

* Dev change api (#3220)

* change consistent_strategy and mirrored_strategy api

* change api (device_prior_placement to scope.placement)

* change name_scope api to scope.namespace

* add middle layer for outdated api
Co-authored-by: NYao Chi <yaochi@oneflow.org>
Co-authored-by: binbinHan <han_binbin@163.com>

* test_lazy_get_job_shared_variable

* minor modification

* new test and fix bug

* update test

* cannot override __eq__
Co-authored-by: NLi Xinqi <lixinqi2010@gmail.com>

Co-authored-by: NShenghang Tsai <jackalcooper@gmail.com>

* add alias to some python api
1. oneflow.device_prior_placement -> oneflow.scope.placement
2. oneflow.distribute.consistent_strategy -> oneflow.scope.consistent_view
3. oneflow.distribute.mirrored_strategy -> oneflow.scope.mirrored_view
4. oneflow.name_scope -> oneflow.scope.namespace

deprecated api names are NOT removed yet

* updates revelant docstrings
docstrings of functions below updated:
1. scope.consistent_view
2. scope.mirrored_view
3. scope.namespace

* run make of_format

* add deprecated function wrapper

1. deprecated_consistent_strategy for "distribute.consistent_strategy"
2. deprecated_mirrored_strategy for "distribute.mirrored_strategy"
3. deprecated_name_scope for "name_scope", "experimental.name_scope", "deprecated.variable_scope"

* refine fixed_placement/device_prior_placement

1. add "fxied_placment"/"device_prior_placement" wrapper for depreacted functions
2. rename "api_device_prior_placement" to "api_placement"

* using (*args, **kwargs) for depreacted function
1. def deprecated_placement(*args, **kwargs)
2. def deprecated_name_scope(*args, **kwargs)
3. update docstring: consistent/mirroed strategy view -> consistent/mirroed view

* exported twice of 'fixed_placement', solved

* use traceback with warning on deprecated functions
relevant items:
  "device_prior_placement"
  "fixed_placement"
  "distribute.consistent_strategy"
  "distribute.mirrored_strategy"

* reformat code

* index range [-2:-1] -> [-2] when print traceback

* using index instead of index range when print traceback info

* print depreacted function name using literal string instead of _ONEFLOW_API

* name scope -> namespace

* fix docstring typos of scope.namespace

* add 'return' in deprecated_name_scope func
Co-authored-by: NYao Chi <yaochi@oneflow.org>

* bump version

* deconv use conv tolenrance

* larger tolerance

* larger tolerance

Co-authored-by: NOuYang Yu <xuanjiuye@gmail.com>