3.1.0 · 标签 · apache / DolphinScheduler

3.1.0

caishunfeng2021@gmail.com

[maven-release-plugin] copy for tag 3.1.0

Feature

Click to expand

[Feature-8252][doc] K8s and namespace manager docs and web page update (#9881)
[feat][task plugin] Add Jupyter task plugin (#9872)
[Feature-9474] [CI] Add cluster test script verify on shell script (#9997)
[Feature-10034][CI] Add postgresql cluster test in CI (#10035)
[feat] Add batch rerun and stop for process instance (#10013)
[Feat][UI] Add resource username for function and resource manage (#10086)
[Feature][plugin] Add k8s task in task plugin (#9425)
[Feature][Task Plugin] Add mlflow task plugin for MLops scenario. (#10036)
[Feature] [CI] Add alert server cluster test #10125
[python] Change distribute name avoid confusion with main pkg (#10134)
[Feature][MLops] Support MLflow Models to deploy model service (MLflow models serve and Docker) (#10150)
[Feature][CI] Schedule Dead Link Checker (#10000)
[Feature][MLops] Add MLflow Projects (custom project) component (#10175)
[Feature][Permission] Project management module authority control refactoring, authority abstract interface setting. (#10261)
[Feature][Datasource]replace DataSourceController API with string JSON and cast to DTO (#10276)
[Feature][API] Modify the permissions of project management, security center, data source center and data quality module. (#10289)
[Feature][Permission] Reconstruction of permissions of resource center and monitoring center. (#10307)
[feat][task] Add OpenMLDB task plugin (#10198)
[Feature][Task Plugin] Add support for dynamic form for zeppelin task plugin (#9977) (#10269)
[Feature-10290] [API] Refactor org.apache.dolphinscheduler.api.controller.AccessTokenController#createToken api (#10301)
[Feature] Should not add refresh logic in the e2e case. (#10331)
[Feature][Monitoring] Added disk available capacity monitoring (#10287)
[Feature-6758][Task] Add limit resource usage for tasks base on cgroup (#10373)
[Feature][metrics] Add master, worker metrics (#10326)
[Feature] [MLOps] support mlflow deploy with docker compose (#10217)
[Feature][Metrics] Enable prometheus to collect metrics in standalone mode demo (#10398)
[Feature][UI] Add manual input for scheduling dates. (#10396)
Add mysql registry plugin (#10406)
[CI] Add OWASP Dependency Check (#10058)
[Feature][Improvement] Support multi cluster environments - k8s type (#10096)
[Feature-10411] Add tenant api test (#10442)
[Feature-10363][server] S3 Resource center supports bucket customization (#10433)
[feature][task] Enable zeppelin schedule a whole zeppelin note (#10434)
[Feature] Enable users to switch and install conda env in jupyter task (#10337)
[Feature][Task Plugin] Add DVC task plugin for MLops scenario (#10372) (#10407)
[Feature] [Permission]Data permission module added and Query class api remove verification. (#10504)
[Feature] TaskGroup remove query func permission check. #10557
[python] Support using full-name definitions for resources (#10551)
[feat] ui build no inherit from maven proxy (#10590)
[Feature] Time function analysis extension. (#10624)
[feature][task-flink] Support Flink application mode (#9577)
[Feature][Master] Failover workflow instance base on priority (#10754)
[Feature] Enable users to create python env from requirements.txt (#10658)
[Feature-10629][Improvement] Support multi cluster environments - namespace use cluster (#10650)
[Feature][dependencies] Bump spring-core from 5.3.12 to 5.3.19 (#10857)
[Feature-10219][EMR] EMR supports use to add steps to an existing cluster (#10657)
[Feature-10871] add workflow executing data query (#10875)
[Feat][UI] Uniform search button size. (#10880)
[Feature][Metrics] Add resource download related metrics for workers (#10749)
[Feat][UI] Add title display to the table in the security center. (#10969)
feat: Added global trim method (#10975)
[Feature][Task Plugin] Enable users to switch endpoints in zeppelin tasks (#10925)
[Feature][Datax Task]Datax task plugin support custom parameters (#10950)
feat: Add trim method to all input (#10984)
[Feat][UI] Added LDAP function. (#11029)
[Feature][Task Plugin] Increase zeppelin task stability in production (#11010)
[Feature][Task Plugin] Add SageMaker Pipeline task plugin for MLOps scenario (#10826) (#10935)
[python] add import SageMaker task (#11121)
Add base loop task execotor and http template parser (#11137)
[Feature-10604] Task name in the copied and imported workflow without copy suffix (#10607)
[Feature][Api] Refactor org.apache.dolphinscheduler.api.controller.ProjectController (#10832)
[Feature-10837][Task Plugin] Sqoop Task Add --split-by Parameter (#10960)
[Feature] Add CURD to the project/tenant/user section of the API and add its tests. (#10913)
[Feature] Add new task type chunjun (#10937)
Write alert result into db (#11221)
[Python] Migrate exists method call in PythonGateway to corresponding service (#11188)
[Feature][style] Add spotless maven plugin for automatic style fix. (#11272)
[Python] Combine gateway.entry_point call in python api side (#11330)
[Feature-9978][UI] add node dependency type feature (#11235)
[Feature][Api] Refactor org.apache.dolphinscheduler.api.controller.QueueController (#11154)
[python] Add local dev mod runing python integate tests (#11362)
[Feature-10117][UI] Hide node execution when start from the workflow definition list page (#11372)
[Feature-11223] support stream task (#11350)
[python] Add multiple version of document (#11391)
[python] Add support interpreter version 310 and 311-dev (#11170)
[Feature-7024] Add waiting strategy to support master/worker can recover from registry lost (#11368)
[Feature][Style] Enable spotless to manage imports (#11458)
Support reading application properties from kubernetes config map (#11402)
[Feature-10182][task-datax] datax access kerberos authenticated bigdata cluster (#11387)
[Feature][Style] Make sure there is one and only one empty line at the end of each java file (#11520)
[Feature] New task plugin Pytorch (#11498)
[feature-11408] add hive option (#11493)
feat: Locations Formats the layout if it does not exist (#11535)
[Feature][UI] Enable highlight and auto-complete for SQL scripts (#11659)
[python] Auto location when create process definition (#11681)
[Feature][Task Plugin] Support hive cli task plugin (#11651)
[feature][API]feature: add task type list and enable user to add task type to fav (#11727)
Add Kubernetes configmap reload to all components (#11730)
[Feature-11530] add state history for process instance (#11757)
[DSIP-11][python] create workflows from YAML configuration (#11988)
[Feature][Registry] Support etcd as registry (#10981) (#11997)
[Feat][python] Add parameter environment to task (#11763) (#11989)
[feat][python] Support Pytorch task in python api (#11975)
[feat][python] Add sagemaker yaml example (#11925)
[feat][python] Support OpenMLDB task in python api (#11944)
[feat][python] Support MLflow task in python api (#11962)
[Feature][PyDolphinScheduler] Support DVC task in pyds #11922 (#11941)
[Feature-10273][Datasource] Add dataSource for AWS Athena (#11157) (#12039)
[Feat][UI] Update package. (#12046)
[python] Support SageMaker task type (#11002)
[Feature][Resource Center] Add support for Alibaba Cloud OSS as storage of resource center (#11708)

Improvement

Click to expand

[refactor][perf] cache compiled regex pattern (#10099)
[improve] Change Mysql Driver (#10220)
[Style][UI] Split multilingual modules. (#10256)
Clean up unused dependencies and packaging issues (#9944)
[Style][UI] Format code. (#10314)
Add dolphinscheduler-task-all in api module (#10350)
Move datasource params to datasouce plugin module (#10353)
issues-10354: upgrade commons-io to fix CVE (#10355)
issue-10356: upgrade logback to fix cve (#10357)
Remove quartz at WorkerServer (#10358)
Add dolphinscheduler-scheduler module (#10360)
[improve] Using create or replace function in sql task (#10170)
Optimize MasterServer, add MasterRPCService (#10371)
change to common lang3 and fix checkstyle (#10380)
Upgrade log4j in E2E (#10359)
Add some warning log in master (#10383)
[Improve] Enhance complement function transformation (#10376)
Update chart.yaml dependencies version (#10420)
[imporve] Add index to table t_ds_relation_process_instance (#10461)
[Improvement][Api] Interface and entity optimization to improve code scalability (#10466). (#10468)
[Improve] Add LDAP user not exsitst action config (#10451)
Modify obsolete swagger parameters in AuditLogController (#10494)
Remove the schedule thread in LowerWeightHostManager (#10310)
[Improvement][Metrics] Apply micrometer naming convention to metrics (#10477)
change import method access to protected (#10536)
[Improve]Remove the code logger.isDebugEnabled() #5918 (#10542)
[Improvement][Metrics] Update some metrics names in grafana-demo dashboards (#10432) (#10552)
Optimize master log, use MDC to inject workflow instance id and task instance id in log (#10516)
add datasource health check to the healthcheck endpoint (#10588)
dependent node log output optimization and fix checkstyle (#10593)
[Improvement][Master] batch remove TaskInstaceId and workflowInstanceId (#10620)
[Improvement][Worker] fixed naming of rpc package (#10614)
Add JDK11 on unit-test (#10364)
Validate master/worker config (#10649)
[Improvement]Delete the getSession in the Registry interface (#10633)
[Refactor][UI] Refactor yarn queue manage using NSpace component. (#10680)
[Refactor][UI] Refactor token manage using NSpace component. (#10681)
[Refactor][UI] Refactor k8s namespace manage using NSpace component. (#10682)
[Improvement-10617] Add comment in slot check (#10618)
[Improvement] heartbeat info improvement (#10699)
Optimize sql script use insert ignore (#10700)
[Optimization] Calculate global parameter and local parameter at master. (#10704)
resource modules query fix. (#10687)
[Refactor][UI] Refactor environment manage using NSpace component. (#10702)
[Refactor][UI] Refactor cluster manage using NSpace component. (#10701)
[Enhancement][API] Support batch copy and export online workflow #10678
[Refactor][UI] Refactor worker group manage using NSpace component. (#10735)
[Task] Add Dinky task to better support the development and execution of FlinkSQL (#10640)
[Optimization]Optimize some details of MLFlow task plugin #10740 (#10739)
[Optimization] Config module resource relation fix. (#10718)
[Improvement] [Alert Server] Add default datasource url in alert server application.yaml for consistency (#10790)
[python] Correct some docs for python api (#10772)
[Task] Added Apache SeaTunnel 2.x task node (#10480)
[Refactor][UI] Refactor alarm instance manage using NSpace component. (#10799)
[Refactor][UI] Refactor alarm group manage using NSpace component. (#10803)
Remove quartz in service (#10748)
[Refactor][UI] Refactor user manage using NSpace component. (#10816)
[Improvement] Resource authority module database table unique index increased. (#10807)
[ui] Data Quality Result Display Optimize (#10835)
[Improvement] [CI] Add label robot to pull request (#10848)
[Refactor][UI] Refactor tenant manage using NSpace component. (#10879)
[Python] Supports creating or editing resources. (#10823)
[Refactor][UI] Refactor audio log using NSpace component. (#10991)
impv: not mark issue stale with security label (#10982)
[Refactor][UI] Refactor datasource using NSpace component. (#11022)
[Refactor][UI] Refactor task result using NSpace component. (#11054)
Add dolphinscheduler-bom to manage the dependency version (#11025)
[Improvement][Task Plugin]Refactor JupyterParameters.java with lombok (#11045)
[Improvement][Task]Improve seatunnel task command log (#11027)
[Improve] Hide Change Pwd Page When Login With LDAP (#11041)
[Refactor][UI] Refactor rule manage using NSpace component. (#11069)
[Refactor][UI] Refactor time display use datatable render function (#11078)
[Refactor][UI] Refactor file manage using NSpace component. (#11073)
[Improvement][Task Log] Task status log print description instead of code (#11009)
[Improvement-11013][dolphinscheduler-common] YarnHAAdminUtils#getActiveRMName function Add HTTPS Hadoop environment support (#11017)
[Refactor][UI] Refactor function manage using NSpace component. (#11124)
[Improvement 11072][common] Added HTTPS request support in kerberos environment (#11127)
[Refactor][UI] Refactor resource manage using NSpace component. (#11098)
[Improvement][Metrics] Switch to use tags to indicate task / workflow execution status for metrics (#11128)
[Enhancement][API] Copy workflow with scheduler info (#10677)
Refactor alert-api with lombok (#11133)
[Improvement][ExecutorServiceImpl] remove some unnecessary (#11122)
Add is success in loop task status (#11169)
[python] Refactor structure to avoid cycle import (#11167)
[Improvement][Metrics] Upgrade plugin versions in grafana demo dashboards (#11192) (#11193)
[Improvement][UI] Unified local parameters UI (#11190)
[Refactor][UI] Refactor task group option using NSpace component. (#11209)
[Improvement-10760][dolphinscheduler-api] Some variables are redundant and fix checkstyle (#11213)
[Improvement][UI] Improve monaco-editor component in ts. (#11274)
[Improvement][Metrics] Add metrics for alert server (#11240)
[Improvement-11293][dolphinscheduler-api] remove some variables that redundant in subclasses of AbstractAuthenticator (#11305)
[Improvement][UI] Scrolling will be applied to parent model on editor scrolled to top/bottom (#11335)
[Refactor][UI] Refactor task group queue using NSpace component. (#11341)
[Refactor][UI] Refactor project list using NSpace component. (#11369)
Split ExecutionStatus to WorkflowExecutionStatus and TaskExecutionStatus (#11340)
[Improvement-11394][Task Plugin] Remove Passed Parameter --queue in Seatunnel Task Plugin (#11395)
Add pause to task instance status (#11390)
[Refactor][UI] Refactor project workflow definition using NSpace component. (#11398)
[Improvement-11380][scp-host.sh] Set StrictHostKeyChecking=no option to ssh (#11382)
[Refactor][UI] Refactor workflow definition timing using NSpace component. (#11415)
[Refactor][UI] Refactor workflow definition version using NSpace component. (#11420)
[Improvement] Avoid using search in for and start using testSaveTaskDefine (#11383)
Clear thread name in BaseTaskProcessor (#11422)
[Refactor][UI] Refactor task instance batch using NSpace component. (#11447)
[Refactor][UI] Refactor workflow instance using NSpace component. (#11446)
[Improvement-11386][UI] Concise the logic available for task action buttons (#11419)
[Refactor][UI] Refactor task instance stream using NSpace component. (#11454)
optimize sorting priority for workflow define (#11462)
[Refactor][UI] Refactor task definition batch stream using NSpace component. (#11485)
[Refactor][UI] Refactor task definition version using NSpace component. (#11495)
[test-11432][HttpTask] Add unit tests to http task (#11453)
[Improvement] File deletion logic optimization (#11519)
[Improvement-11468] Support re running historical version workflow (#11489)
[Improvement]Add a stop function when you are ready to do pause operation (#11543)
[Improvement] [LDAP] Bug fix: Config value return "null" instead of real null currently. (#11325)
Remove logger header in task log file (#11555)
[Improvement-11550] [Doc]Document content update (#11577)
Upgrade spring boot and swagger (#11574)
Add dependent task instance log (#11541)
[improvement] Add two parameters in workergroup, and support the application of description display and other parameters (#11542)
[Improvement][TaskInstance] reduce database queries (#11522)
[Improvement-10942][UI]The workflow or task name is too long and results in incomplete display (#10957)
[Improvement][User] Query all users filter un enabled. (#11504)
[Improvement-#11613] Add spi priority factory (#11614)
[improvement-#11630]fix document about common.properties (#11653)
Refactor worker execute task process (#11540)
add unit test ci timeout (#11655)
[Improvement-11662][dao] fix foreach null items in mapper.xml
Refactor heart beat task, use json to serialize/deserialize (#11702)
optimize the process instance query, change the date time range (#11719)
reduce task dependencies (#11731)
[Improvement] Replace commons-lang 2 function invocations with commons-lang3 (#11810)
Bump up dependencies to fix cves (#11765)
Remove not use web service comment in alert server (#11819)
[Improvement-#11768][Monitor] Support monitor h2 database in monitor page (#11813)
[Improve] Upgrade Hadoop to 3.2.4 (#11832)
Remove htrace dependencies (#11881)
[Improvement-11834] Upgrade docker base image to support python3.9 (#11835)
Optimize date range condition (#11911)
Add debug logs for properties read from common.properties (#11910)
Add validations of possible malicious keys (#11966)
[Improvement] and task instance date range search (#11963)
[Improvement-11291][sql] Improvement the log output while executing multiple sql statements (#11292)
Script cannot contains ''' in params (#12068)
[improvement]Add Set cluster name (#12058)
[Improvement] Add Set cluster name (#12074)
add(dependent node's add unprivileged project list): dependent node's add unprivileged project list (#12073)
[improve] Optimize MLFlow task plugin for easy of use (#12071)
Remove dao in worker (#10994)
Move utils, builder under master (#11000)
[Improvement][Worker] Resource may store in hadoop/S3 (#11206)

Bugfix

Click to expand

[Fix][UI][V1.0.0-Beta] Fix the problem of packing failure. (#9962)
[Bug] [MASTER-9811]fix cmd param to overwrite global param when executing complement (#9952)
[Fix-10002] Fix some bugs in datasource list (#10005)
[Fix][UI] Fix the task name validator error. (#10008)
[Fix-9975] The selected task instance was recreated when the Master service failed over (#9976)
[Fix][UI] Support only one file upload on the file manage page. (#10007)
[Fix][UI] Fix errorOutputPath column in dataquality page (#10015)
[fix-9991][worker]fix statement is closed before resultSet.getMetaData() (#10014)
[dataquality] Fix task commnd null bug (#9974)
[Fix-10039] Flink run command when perfecting Python jobs (#10042)
[Bug] fix run on docker and k8s (#10026)
[python] Fix pre_submit_check error when local params in tasks (#10032)
[Fix][UI]Modify error message (#10054)
Fix flaky filemanage_e2e_test (#10068)
[Fix][UI] Fix in the Chinese state, there is a problem with the multi-language support of the form. (#10066)
[Fix][UI] Fix bug where username is empty in file manage (#10070)
[Bug][Master] fix master task failover (#10065)
[Fix-10080]When the created tenant name is too long, an error message will be prompted (#10081)
Create process definition with task group and task group priority cannot be save into db (#10078)
[Fix][Docker] Support Multi-Architecture Docker Images (#10052)
[Fix-10082]Resource userName is null and udf function manager add a username (#10064)
[Fix-10049] Conditions Task branch flow failed (#10077)
[Fix][UI] fulfill missing fields in DataQuality (#10098)
[fix] Enhance name pre checker in resource center (#10094)
[BUG][TaskGroup] Task group does not take effect (#10093)
[Build][UI] Solve vite config throws terser warning. (#10105)
Fix alert server curator health check not work (#10111)
Fix unit test and remove unused import (#10113)
[Bug][doc] Fix the open-api docs request error (#10100)
[Fix][UI] Add the task status of pending. (#10129)
[Fix][UI] Limit task group resource capacity to only positive integers. (#10158)
[Fix]Change the HTTP or HTTPS regular expression (#10166)
[Fix][UI] Fix the problem that the failure strategy field in the time management form is multilingual. (#10169)
[Fix-10057][api] api parameter name misspelling (#10063)
[Fix][UI] Add seatunnel icon (#10178)
fix spcial week init bug (#10183)
[Fix-10076] queryUserByToken use server timezone (#10177)
[Fix][UI] Fix data-quality component multi-language support. (#10202)
[Fix-10199] [Workflow/Workflow Definition] After deleting messages in batches, the error message is repeated (#10201)
[Bug][Deploy]Fix worker-server path in script scp-hosts.sh (#10208) (#10209)
fix#10214 Delete online workflows in batches, prompt information is accurate (#10215)
[python] Fix process definition attr release state not work (#10151)
[Fix][UI] add assignment of totalPage (#10165)
[Fix-10103][k8s]Fix k8s Change DataSource Error (#10128)
[Fix][UI] Fix the problem of multi-language support of workflow instance host in Chinese state. (#10223)
[fix-10207] When the registered alarm plugin is empty, the error message is not clear (#10224)
[Fix][UI] Fix issue with treemap depth in workflow relationships. (#10229)
[fix] Task group queue query all list error (#10237)
[Fix#9923] The alarm instance management interface is always in the loading state (#9924)
[Fix][UI] Set task group name empty of default value (#10243)
[fix][deploy] File overwrite when specific value set to installPath (#10241)
[fix] Deploy scp-hosts use fullpath avoid dangling (#10249)
[Bug] init docker compose DATABASE env (#10244)
[Fix-10181] Fix the logic of judging that the tenant does not exist (#10185)
[Fix][UI] Fix status statistics card height is inconsistent. (#10268)
[Fix] Fix the timeout alarm button is invalid (#10260)
[Fix] Fix the line.separator problem for alert-plugin' test case #10277 (#10278)
[fix] Fix the SERIAL_DISCARD execution_type not working (#10258)
[Fix][UI] Add mapping_columns filed for the data_quality ruleInputParamter. (#10286)
[Fix][UI] Display the result page when the DB page has no data. (#10298)
[Fix-8828] [Master] Assign tasks to worker optimization (#9919)
Fix database initialize late in standalone. (#10296)
[Fix-10317] [ui] Fix the tips of startTimeout in the jupyter form (#10319)
[Bug][Api-Server]Fix bug when user without tenant uploads file in resource center (#10303)
[Fix] Privilege Reconstruction Data Query Fix & Resource Creation Post Operations (#10313)
[Fix][UI] Fixed the issue that the status statistics on the project overview page were inconsistent. (#10328)
[Fix-10305][common] AWS config Constant definitions are repeated (#10311)
fix bug_10343 (#10344)
[Fix-10318][ui] Fix EMR Task UI custom parameters Is not correct (#10334)
[Fix-10338] Tenant does not belong to resource permission. (#10339)
[Fix] [Worker] Fix worker will hang if fails to start (#10342)
[fix][deploy]fix run on kubernetes bug (#10379)
[Fix-10367] Permission check related key adjustment. (#10368)
[Fix-10412] [Standalone] Failed to start standalone server according to the official document (#10414)
Fixed the problem of not having permission to modify the allocated resource data. (#10410)
[fix][Python] Support same task name in project defferent process definition (#10428)
[DolphinScheduler] Fix wrong class name in doc (#10438)
[Bug][Tools] Add datasource url for mysql profile in tools application.yaml (#10399)
[Fix-10425]Recovery LDAP Config (#10429)
[fix-10386]LDAP user create with status enable (#10426)
[Fix][UI] Add loading before file modification data is returned. (#10450)
[Bug-10394] [Update Possword] Change password shows error (#10437)
[fix][python] Correct variable type of SQL type (#10464)
fix rename form not init bug in file manange (#10471)
Fix TaskProcessorFactory#getTaskProcessor get common processor is not thread safe (#10479)
[Fix] Fix the JSONUtils tool class time zone problem #10282 (#10284)
[fix][master]: fix condition npe (#10478)
Fix PeerTaskInstancePriorityQueue cannot contains method use taskInstanceId to check (#10491)
[Fix]Recovery Traffic Control Config (#10505)
[Fix-10413] Fix Master startup failure the server still hang (#10500)
[FIX] Could not find or load main class org.apache.zookeeper.ZooKeeperMain (#10535)
[Fix-10522] [API] Fix get executable time null pointer exception (#10523)
[Fix-10452] Serial wait for policy recovery (#10453)
[Fix-10274]Close InitialLDAPContext In LDAP (#10532)
fix: Current node settings number of failed retries may not be negative (#10530)
[fix][doc] Add DVC task plugin doc link (#10529)
[Fix] fix the HttpUtilsTest' test case , it is always time out. (#10294)
[fix] Change K8s postgresql password key name (#10545)
[Fix-10513] Fix the doc of pseudo-cluster mysql installation error #10564
[Fix-10547][data-quality] Fix Table or view not found #10549
[Fix-10514] The master frequently print the same info or error logs, leads to insufficient disk space #10568
fix standalone start error (#10577)
fix dependent task re-run params error (#10580)
[Fix-10400]Fix K8s deployment bitnami version error (#10567)
[Fix][dolphinscheduler-spi] json parse support LocalDateTime (#10440)
[Fix-10539] [Bug] [UI] Copy task a to B, change the user-defined parameter (#10611)
[Fix][UI] Fix bytesToSize function calculation error. (#10627)
[fix-10596] Workflow relationship cannot be displayed normally (#10625)
[fix-10632][ui] fix the workflow name with underline cannot be displayed correctly error. (#10636)
[Bug] [Master] Worker failover will cause task cannot be failover (#10631)
[Bug] [Api] The copied workflow create time and update time is still copied from the original workflow
[Fix-10641] [Common] Fix the hdfs NPE (#10642)
[Bug] [Task] SQL task no exception stack trace in task log (#10656)
[Fix-10652] fix docker-compose.yml (#10653)
[Fix][UI] Fix bug where the home page displays no data. (#10697)
fix bug where project overview page display no data (#10720)
[Bug] [Api-Service] Workflow and subworkflow instance does not change after clicking rerun。 (#10727)
[Fix-10666] Workflow submit failed will still in memory and never retry (#10667)
Fix language not matched (#10775)
[Bug][Script] Remove deprecated arg in remove-zk-node.sh (#10789)
[Fix][UI] Fix the environment name display back error. (#10797)
[API] Do not specify rollback for ensure in case of any exception or error. (#10793)
[Fix-10785] Fix state event handle error will not retry (#10786)
[CI] Fix checkstyle & review dog don't work (#10812)
[python] fix doc build error (#10810)
[python] Fix submit and run error (#10792)
[Fix-10829][UI] display real workflow relation in WorkFlow Relation page (#10836)
[Fix-10842] Fix master/worker failover will cause status incorrect (#10839)
Complement numbers will run in a loop under the serial strategy fixed. (#10862)
[Bug][Api] Complete Data workflow remove duplicates. (#10850)
[Fixed-10833] [Bug] [Quartz] timezone display doesn't match the next_fire_time in ds 3.0.0-beta1 version (#10865)
[Bug][Api] Complete Data workflow remove duplicates. (#10868)
[Fix-10863] Command execution result data statistics remove userId condition. (#10864)
[Fix-10854] Fix database restart may lost task instance status (#10866)
[fix] update queue and tenant with same code error (#10877)
[Fix][UI] Fix hive datasource doesn't show principal bug (#10876)
[Fix-10827] Fix network error cause worker cannot send message to master (#10886)
[bug][task-plugins]fix switch class name and add param check to the switch task plugin (#10894)
[fix] mlflow project success status (#10897)
[fix] delete workflow or task used by other tasks (#10873)
display timing record start time (#10915)
fix bug where task name display in gantt (#10900)
[Fix][UI] Add validator to the next node field of the switch task. (#10898)
Catch exception when check state in StateWheelExecuteThread (#10908)
Solve transaction problems under multi-threading (#10917)
[fix][plugin]: fix spark sql use custom parma. (#10821)
[FIX-10784][Bug] [ALERT-SERVER] FEISHU Plugin might block the whole alert process loop (#10888)
[fix] Can not query exists environments (#10884)
[Fix-10909] Returns executionStatus for each process instance status and access token module support. (#10922)
Fix taskInstance's host is not worker nettyServer address (#10926)
Fix kill yarn job error when failover caused by doesn't set ProcessDefinition (#10948)
[Fix][UI] Set the next node to not required. (#10932)
[fix-10954]Querying the task group list returns empty data (#10955)
fix: typo task definition. (#10958)
[Fix-10890] fix some bugs in the Resource page (#10907)
[Fix-10595][api] Fix Dependent Node Only View Projects Created By Self (#10834)
[python] Fix tasks with multiple upstream and workflow query error (#10941)
[fix] [MLflow Task plugin] Sometimes errors are reported using preset algorithms (#10986) (#10987)
Kill task when we do master failover to avoid task lost (#10997)
fix: typo executor service (#10980)
Fix worker cannot shutdown due to resource close failed or heart beat check failed (#10979)
[hotfix][Worker] Remove service dependency from worker module (#11008)
[Fix-11003]Task group queue is not updated to final state (#11004)
[Bug] fix task params pass (#11053)
[fix-10729]The request's url given by the test case looks different from the controller layer. Some usages and parameter format errors (#10734)
[Fix-11031] AccessToken can only be used by the creator. (#11032)
[E2E] [CI] Fix flaky e2e test (#11077)
Try to fix filemanage e2e flaky test (#11097)
add oracle dependency (#11095)
[Fix][UI] Widen the segment separator input box. (#11096)
fix view variable display bug (#11100)
[Fix][UI] Fix the problem of too large spacing in operation. (#11090)
fix space validate for create folder bug (#11106)
[fix] py sagemaker doc (#11109)
[Fix-10918] Close datasource when expire from guava cache (#11120)
[fix][Data Quality] Data Quality Report Display For Admin Role #10919 (#10977)
[Fix][UI] fix bug where The Gantt chart does not show the execution status (#11135)
[Bug] [API] If the single log length is long, it will causes the View Log page to hang (#10612)
[Fix][UI] Add duplicate of file suffix (#11136)
fix k8s master and worker start error (#11156)
[Bug-11101] fix task failover NPE (#11168)
[Fix-10665] [S3] Fix s3 download method (#10675)
[Fix-11140] Added resource parameters for function verification (#11141)
[Fix] Parameter values disappear when editing Zeppelin task (#11115)
[Bug-11071] fix local params (#11184)
fix nodeSize is not a function bug (#11200)
[BUG-11181] [Document] text check (#11183)
[Fix-11007] [Master] fix forced_success bug (#11088)
[Fix][UI] Fix datax task data instance replay error. (#11222)
fix work oom when task logs's size is very large (#11224)
Add alertPluginInstanceId in AlertInfo (#11231)
Fix recovery from failed task will dead loop (#11239)
fix bug where confirm disable error (#11251)
[Fix][UI] fix bug where log modal display object (#11271)
[fix-11233][ui] Remove custom template switch button in chunjun component module (#11234)
Fix TaskGroup cannot work and will cause master dead loop (#11254)
[fix] Process definition length too long when copy or import (#10621)
no-admin user query alert group fix (#11301)
[Fix][UI] Fix the taskPriority options not showing when hover. (#11294)
[Bug][Doc] Fix typos in docs which causes links to upgrade section to fail (#11333)
[Fix][UI] fix bug where icons are displayed in one line. (#11320)
[fix-11295] remove duplicate classes (#11355)
[Fix-11366][UI] Workflow instance should not support right-click running (#11367)
[Fix-11376][task plugin]Seatunnel Task cannot save successfully when select spark engine (#11377)
[fix-11378] fix API-Server release response message typo (#11400)
[Fix-10392][SqlTask]: Fix list param error when use sql task (#11285)
[Fix-11051][Task]Fix the process exitValue exception when the process times out and the process does not end. (#11099)
[Fix-1360]: Fix Description length check (#11371)
[Fix-11413] Cannot set task status to kill if the task is not in running(#11414)
[Fix][UI] fix bug where warn group name not display (#11429)
[fix-10961][alert server]Change the content of alert to an array (#11033)
Fix find last process instance in dependent task is not correct (#11424)
Fix workflow instance may failover many times due to doesn't check the restart time (#11445)
Set master's task running status in runTask to avoid the task group acquire failed, but the task status is in running (#11451)
If the task processor is not found need to throw error rather than exception (#11461)
[Fix-11465] Serial first, the last executing instance will be stopped (#11466)
[Fix][UI] Fix the table header in the workflow instance. (#11487)
[Fix][UI] Fix the preTask options are unavailable when creating a task in the task definition page. (#11501)
[fix] Database change not sync (#11258)
[Fix][UI] Remove unsupported datasource types in the datax task. (#11527)
Set process instance and task priority default value as 2 (#11539)
[fix-11404]: make the common.properties to configurable on values.yaml (#11441)
Fix some resource leak (#11576)
[Fix][Issue-11579] Cannot check the application status. (#11580)
Fix vulnerability in LDAP login (#11586)
Refresh server startup time when recovery from waiting (#11595)
Fix quartz threadPriority config name error (#11596)
Fix kill task failed will cause the taskGroup cannot release and add taskGroup log (#11469)
[Fix-9980] [Server] fix heartBeatTaskCount bug (#11232)
Fix homepage email subscription link (#11622)
[Fix][UI] Fix the port in the datasource edit. (#11624)
[Fix-11616] Workflow Definition Name Change Validation Optimization. (#11617)
[Fix][UI] Fix the problem that the pagination component is not centered in the environment management. (#11646)
[Fix][UI] Fix bug where crontab special of month and year (#11661)
[fix][python] Sql pattern add truncate. (#11666)
[bug][python] Fix task switch error when default branch not defined last (#11606)
[Fix-11633] The data type of the PostgreSQL database column is wrong. (#11633) (#11634)
[Bug] [Worker] Optimize the getAppId method to avoid worker OOM when kill task (#11701)
[Fix][UI] Add the task name entry when the workflow instance detail page goes to the task instance page. (#11761)
[fix][doc] Update the registry related configuration in values.yaml (#11444) (#11981)
[Bug][Workflow Definition] fix ordinary users can not create depend task (#11961) (#11977)
[Bug-11650][worker] #11650 fix SQL type task, stop task cause NPE (#11668) (#11958)
[Bug] [Master] WorkflowExecuteRunnable will face a infinite loop #11838 (#11864) (#11950)
[Fix][db] fix init&upgrade mysql-meta-schema bugs (#11887) (#11935)
[Bug] Fix procedure task param pass #11919 (#11926)
[Fix-11217] [Doc] add postgresql config in doc: datasource-setting (#11326)
[fix-#11689] change replaceAll to replace in method of tryExecuteSqlResolveColumnNames (#11696)
[fix-#11691] Add lz4-java in clickhouse datasource. (#11697)
Example Change the regular expression of the data source password (#11459) (#11990)
Fix insertOrUpdate plugin may failed due to concurrent operation (#11471) (#11996)
Update dolphinscheduler_ddl.sql fix alert t_ds_user add time_zone is not correct (#11795)
[python] Fix permission error when create related objects (#11783)
[Bug][Dependent]: Id also clone due to duplicate when use dependent mode. (#11929)
Fix listing processDefinition the schedulerReleaseState will never be null (#11888)
Add chunjun in dolphinscheduler-task-all/pom.xml (#11375)
[Bug] [dolphinscheduler-api] Data Analysis Service API responds with jackson deserialization exception (#11858)
[Fix][Monitor]Monitor UI not show DisakAvailable and MemoryUsage correctly (#11870)
fix workflow keep running when task fail (#11930)
Fix cannot save processDefinition (#11931)
Bug Dependent downstream trigger error when schedule cycle not day. (#11734)
Fix update TaskDefinition error (#12064)
[fix][python] Task switch branch not show in webui (#12120)
[Fix] Also auto formatter workflow instance if location is null (#12080)
fix python home conflict (#12112)
[Fix][UI] Fix workflow name overlaps bug (#11862) (#12125)
[BUG]Delete Setting ID (#12096)
Fix insert command error due to the id is not null (#12092) (#12098)
[BUG][BE-12113]fix htrace dependency conflicts (#12126)
[bug][k8s] Change the config file keys in K8s to be the same as those in common.properties (#11812)
[k8s] Delete unnecessary configurations. (#11808)
fix upgrade for 310 (#12137)
[fix#12000]Cannot remove the WorkerGroup from the master service (#12050)
fix env config space bug (#12147)

Doc

Click to expand

[Docs] Upgrade license. (#9967)
[Docs] Update README. (#9970)
[doc] Fix docs deadlink in readme (#9972)
[doc] Update the document picture to new UI (#9953)
[Doc] Add example and notice about task type Conditions (#9945)
[Docs] Fix docker link. (#9998)
[doc] Add the description about execute type in SQL task (#9987)
[doc] Add example and notice about task type Dependent (#10001)
[doc] Correct kubernetes (#9985)
[doc] Correct docs of development-environment-setup (#9995)
[Bug][Document] Fix task group management document link error (#10062)
[Fix-10083][Doc]Change Docker cmd && Change WebexTeams Chinese Translation (#10084)
[doc] Show all the functions of the home page through screenshots (#10118)
[doc] change document structure (#10124)
[doc] Remove flink call docs (#10122)
[doc] Update editorial workflow content and images (#10130)
[doc] Update website docs link (#10161)
[doc] Refactor global parameter docment (#10045)
[doc] Update project directory structure in docs (#10107)
[Doc] Add complie supporting system (#10188)
[doc] Update delete workflow description (#10203)
[doc] Update task DataX document (#10218)
[doc] Update kubernetes' FAQ (#10221)
[doc] Refactor local parameter docment (#10119)
[doc] Delete SPRING_DATASOURCE_DRIVER_CLASS_NAME in doc (#10228)
[doc] Add setValue syntax in local param (#10245)
[doc] Correct all setup documents (#10074)
[doc] Upgrade screenshot to latest version in dev document (#10242)
[doc] Refactor context parameter docment (#10227)
[docs] Added local file configuration guide for resource center (#10264)
[doc] Update the development environment to build the software version (#10246)
[doc] Fix api standard error (#10292)
[doc] Add how to use resource in shell task (#10308)
[doc] Use related path for img (#10325)
[Docs]Update Slack Invite Link (#10362)
[Docs] Upgrade license. (#10365)
[doc] Add switch task demo (#10247)
[doc] Correct resource in HDFS HA cluster (#10445)
[Doc] Add how to set the alarm mailbox in faq (#10401)
[Doc] Add api test doc (#10499)
[doc] Correct the complement add docs (#10521)
[doc] Add docs about complement dependent mode (#10587)
[doc] Add doc about how to use healthcheck endpoints (#10592)
[doc] Add run the task alone doc. (#10581)
[docs] Add links to metrics docs (#10583)
[doc] Correct installPath when using related path (#10597)
[doc] Modify Shell task file (#10634)
[doc] Change subprocess dependent (#10637)
[doc] Modified parameter doc files. (#10659)
[doc] Replace the dag screenshot of the README. (#10662)
[doc] Update metadata and design doc (#10600)
[doc] Add doc for pre-sql and post-sql (#10685)
[doc] Change datasource doc (#10660)
[doc] Change tasks doc (#10639)
[doc] Modified alert doc files (#10714)
[doc] Modify quick start. (#10712)
Merging general settings to faq (#10716)
[Docs] Replace the dag screenshot of the README. (#10741)
[doc] Modified project doc. (#10711)
[doc] Change tasks docs (#10751)
[doc] Unify data source switching of the docs during installation (#10688)
[doc] Replace the dag screenshot of the README. (#10798)
[doc] Modified dq, monitor, security, resources (#10715)
[doc] Modify features, hw, intro and homepage docs. (#10713)
[doc] Correct custom jdbc driver for K8s Doc (#10768)
[Docs] Updated license. (#10838)
[Doc][Upgrade] Fix tiny error in upgrade docs (#10847)
[doc] Add missing doc change by snehal (#10892)
[Doc]Chinese translation updated (#11146)
[doc] Fix bugs in doc installation, design and glossary (#11084)
[doc] Add new page incompatible change (#11153)
[doc] Add notice when environment can not select (#11276)
[Doc] update the configuration doc (#11113)
[doc] migrate community and development docs into main repo (#11313)
[Doc] Add tools/libs to the init database steps (#11345)
[doc] Fixed hadoop link to avoid dead link (#11363)
[doc] Improve the document introduction of complement (#11275)
[doc] Refine the deployment documentation (#11361)
[doc] Recover traffic configuration (#11288)
[doc] Add how obtain IP address from network card in FAQ (#11311)
[Doc][Resources] Instruct users to use local storage if they have remote storage mounted to local (#11435)
[Doc][DSIP] Move DSIP docs to the right place (#11442) (#11443)
[doc] Use Rainbond deployment DolphinScheduler Cluster (#10951)
[Doc][Security] Update instructions on worker groups (#11483)
[Doc] Update UI readme. (#11496)
[Doc][Bug] Fix resource center docs for standalone mode (#11600)
[DOC] should notice that need setting the full path when call the resource file #11620 (#11621)
[Docs] Update README. (#11641)
[doc] Correct E2E Doc, fix WorkerGroupPage typo (#11629)
[DOC] improve zk digest doc (#11703)
[Doc][Improvement] Add instructions for process execution type and data complement (#12121)
add Feishu alert doc and configure email alert doc to the docsdev.js (#11173)
[doc] Fix installation typo (#11220)
[Improvement-11481][docs] Add log specification document to contribution guidelines. (#11484)
Enable spotless to fix markdown checkstyle and refactor faq.md as an example (#11384)
Update doc according to recent update of Spring / Swagger (#11755)
[Doc] Put default task parameters in a new file #11776 (#11957)
[Fix-10830][Doc] Change the start or install script command sh in the document to bash (#10831)
[Doc][Style] Fix doc format once for all (#12006)

Chore

Click to expand

[ci] Correct CODEOWNERS and add new pom.xml owner (#9954)
[Build][UI] Upgrade dependencies. (#9956)
[chore] Combine version 206 and 210 sql upgrade to 300 (#9959)
[ci] Dead link check all markdown file (#10004)
[chore] update by release version (#10299)
[ci] Fix mergeable for ddl change and remove label unexpect (#7099)
[CI] bumping actions/cache to v3 to fix ci error (#10345)
[Build][UI] Upgrade dependencies. (#10346)
[CI] Add code owner of .github (#10347)
[CI] Add some codeowner (#10385)
[chore] Remove unnecessary badge total lines (#10405)
add codeowner (#10457)
add_some_code_owners (#10462)
Open checkstyle, change checkstyle severity from info to warning (#10492)
add codeowner (#10537)
[chore] update by release version (#10742)
[ci] mergable do not handle label (#10638)
Add improvement template (#10764)
Add doc issue template (#10804)
[API-Test] [Checkstyle] Reformat api-test code and enable checkstyle (#10805)
add reviewdog token to enable checkstyle (#10819)
[Build][UI] Updated dependencies. (#10813)
[CI] Try to fix label bot (#10852)
[ci] Fix mergeable will get mad add/remove label sometimes (#10856)
fix label bot again (#10859)
[python] Integrate test run strategy change (#10825)
Add EricGao888 to help review docs, tools, etc. (#11047)
[python] Add directly install without clone source code (#11179)
[ci] Make cluster test work on release branch (#11199)
[community] Remove stackoverflow ref link in website (#11342)
Modifying Slack Links (#11416)
update PyDolphinScheduler documentation link. (#11474)
update slack (#11578)
[CI] Fix cluster test abnormal (#11688)
add lombok config (#11212)

项目简介

Apache DolphinScheduler is the modern data workflow orchestration platform with powerful user interface, dedicated to solving complex task dependencies in the data pipeline and providing various types of jobs available out of the box

🚀 Github 镜像仓库 🚀

源项目地址 ⬇ ⬇ ⬇

https://github.com/apache/dolphinscheduler