* Init base buffer and storage

* Use ratelimit as middleware

* Pass style check

* Keep the return original return value

* Add buffer.view

* Add replace flag on sample, rewrite middleware processing

* Test slicing

* Add buffer copy middleware

* Add update/delete api in buffer, rename middleware

* Implement update and delete api of buffer

* add naive use time count middleware in buffer

* Rename next to chain

* feature(nyz): add staleness check middleware and polish buffer

* feature(nyz): add naive priority experience replay

* Sample by indices

* Combine buffer and storage layers

* Support indices when deleting items from the queue

* Use dataclass to save buffered data, remove return_index and return_meta

* Add ignore_insufficient

* polish(nyz): add return index in push and copy same data in sample

* Drop useless import

* Fix sample with indices, ensure return size is equal to input size or indices size

* Make sure sampled data in buffer is different from each other

* Support sample by grouped meta key

* Support sample by rolling window

* Add import/export data in buffer

* Padding after sampling from buffer

* Polish use_time_check

* Use buffer as dataset

* Set collate_fn in buffer test

* Init framework

* Remove set_default, add keep

* Move backward_stack to task

* Fix total_step

* Pydash pick is too slow

* Add step records

* Add async mode

* Reuse forward and backward functions in sequence

* Fix sample profile

* demo(nyz): add atari pong runnable demo

* Fix forward bug

* Add task test

* Test pong

* feature(nyz): add deque buffer compatibility wrapper and demo

* polish(nyz): polish code style and add pong dqn new deque buffer demo

* Use sync mode

* Config worker number

* Init parallel mode

* Add prev property on context

* Mesh workers

* First version of parallel mode

* Make send rpc async

* Dont pickle prev

* Support tcp

* More cleanup on system exit

* Test parallel and task

* Enable task copy

* Test attach mode

* Add with statment

* Polish code

* Raise exception when timeout in attach mode

* Add event listeners

* feature(nyz): add pendulum sac new pipeline demo

* Fix main

* Add profiler and step profiler

* Rewrite parallel, cleanup res after task finished

* Add comments

* Remove ctx.prev

* Enable standalone parallel mode

* Remove hooks on ctx

* Add max mean

* demo(nyz): add pong dqn new pipeline demo

* Ensure parallel sock closed before program exit

* Fix parallel test

* Fix pong

* feature(zjow): add feature of profile in ding (#135)

* add profiling feature in ding cli.

* fix ding --profile cli.

* reformat files.

* reformat files again.

* reformat files again.

* Remove flameprof

* Change kept_keys to set

* Use finish as a properity

* Use wrapper

* Reformat step timer output

* Test random seed

* Revert learning rate

* Add topology on parallel

* Use labels on task

* Star in parallel mode

* Don't use daemon process

* Auto sync finish state

* Return logvars

* Fix test wrapper

* Fix test profiler helper

* Pass flake_check

* Lazy launch

* Reporter

* Replace main with main_sac

* Fix parallel ctx

* Fix test

* Fix merge issues
Co-authored-by: Nniuyazhe <niuyazhe@sensetime.com>
Co-authored-by: Nzjowowen <93968541+zjowowen@users.noreply.github.com>