Skip to content
体验新版
项目
组织
正在加载...
登录
切换导航
打开侧边栏
DolphinScheduler社区
DolphinScheduler
提交
572ee0fc
DolphinScheduler
项目概览
DolphinScheduler社区
/
DolphinScheduler
与 Fork 源项目一致
Fork自
apache / DolphinScheduler
通知
67
Star
1
Fork
0
代码
文件
提交
分支
Tags
贡献者
分支图
Diff
Issue
0
列表
看板
标记
里程碑
合并请求
0
Wiki
0
Wiki
分析
仓库
DevOps
项目成员
Pages
DolphinScheduler
项目概览
项目概览
详情
发布
仓库
仓库
文件
提交
分支
标签
贡献者
分支图
比较
Issue
0
Issue
0
列表
看板
标记
里程碑
合并请求
0
合并请求
0
Pages
分析
分析
仓库分析
DevOps
Wiki
0
Wiki
成员
成员
收起侧边栏
关闭侧边栏
动态
分支图
创建新Issue
提交
Issue看板
前往新版Gitcode,体验更适合开发者的 AI 搜索 >>
提交
572ee0fc
编写于
6月 04, 2020
作者:
S
simon824
浏览文件
操作
浏览文件
下载
电子邮件补丁
差异文件
Using Jackson instead of Fastjson
上级
6205e687
变更
7
隐藏空白更改
内联
并排
Showing
7 changed file
with
116 addition
and
16 deletion
+116
-16
dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/ProcessDefinitionService.java
...olphinscheduler/api/service/ProcessDefinitionService.java
+7
-1
dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/ProcessInstanceService.java
.../dolphinscheduler/api/service/ProcessInstanceService.java
+18
-3
dolphinscheduler-common/src/main/java/org/apache/dolphinscheduler/common/utils/JSONUtils.java
...a/org/apache/dolphinscheduler/common/utils/JSONUtils.java
+6
-1
dolphinscheduler-dao/src/main/java/org/apache/dolphinscheduler/dao/entity/ProcessDefinition.java
...apache/dolphinscheduler/dao/entity/ProcessDefinition.java
+26
-8
dolphinscheduler-dao/src/test/java/org/apache/dolphinscheduler/dao/entity/ProcessDefinitionTest.java
...he/dolphinscheduler/dao/entity/ProcessDefinitionTest.java
+39
-0
dolphinscheduler-server/src/main/java/org/apache/dolphinscheduler/server/worker/runner/TaskExecuteThread.java
...phinscheduler/server/worker/runner/TaskExecuteThread.java
+8
-1
dolphinscheduler-service/src/main/java/org/apache/dolphinscheduler/service/process/ProcessService.java
...ache/dolphinscheduler/service/process/ProcessService.java
+12
-2
未找到文件。
dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/ProcessDefinitionService.java
浏览文件 @
572ee0fc
...
...
@@ -19,6 +19,7 @@ package org.apache.dolphinscheduler.api.service;
import
com.baomidou.mybatisplus.core.metadata.IPage
;
import
com.baomidou.mybatisplus.extension.plugins.pagination.Page
;
import
com.fasterxml.jackson.core.JsonProcessingException
;
import
com.fasterxml.jackson.core.type.TypeReference
;
import
com.fasterxml.jackson.databind.JsonNode
;
import
com.fasterxml.jackson.databind.node.ArrayNode
;
import
com.fasterxml.jackson.databind.node.ObjectNode
;
...
...
@@ -755,8 +756,13 @@ public class ProcessDefinitionService extends BaseDAGService {
public
Map
<
String
,
Object
>
importProcessDefinition
(
User
loginUser
,
MultipartFile
file
,
String
currentProjectName
)
{
Map
<
String
,
Object
>
result
=
new
HashMap
<>(
5
);
String
processMetaJson
=
FileUtils
.
file2String
(
file
);
List
<
ProcessMeta
>
processMetaList
=
JSONUtils
.
toList
(
processMetaJson
,
ProcessMeta
.
class
);
List
<
ProcessMeta
>
processMetaList
=
new
ArrayList
<>(
);
try
{
processMetaList
=
JSONUtils
.
getMapper
().
readValue
(
processMetaJson
,
new
TypeReference
<
List
<
ProcessMeta
>>()
{});
}
catch
(
Exception
e
)
{
logger
.
error
(
"parse list exception!"
,
e
);
}
//check file content
if
(
CollectionUtils
.
isEmpty
(
processMetaList
))
{
putMsg
(
result
,
Status
.
DATA_IS_NULL
,
"fileContent"
);
...
...
dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/ProcessInstanceService.java
浏览文件 @
572ee0fc
...
...
@@ -18,6 +18,7 @@ package org.apache.dolphinscheduler.api.service;
import
com.baomidou.mybatisplus.core.metadata.IPage
;
import
com.baomidou.mybatisplus.extension.plugins.pagination.Page
;
import
com.fasterxml.jackson.core.type.TypeReference
;
import
org.apache.dolphinscheduler.api.dto.gantt.GanttDto
;
import
org.apache.dolphinscheduler.api.dto.gantt.Task
;
import
org.apache.dolphinscheduler.api.enums.Status
;
...
...
@@ -531,7 +532,11 @@ public class ProcessInstanceService extends BaseDAGService {
List
<
Property
>
globalParams
=
new
ArrayList
<>();
if
(
userDefinedParams
!=
null
&&
userDefinedParams
.
length
()
>
0
)
{
globalParams
=
JSONUtils
.
toList
(
userDefinedParams
,
Property
.
class
);
try
{
globalParams
=
JSONUtils
.
getMapper
().
readValue
(
userDefinedParams
,
new
TypeReference
<
List
<
Property
>>()
{});
}
catch
(
Exception
e
)
{
logger
.
error
(
"parse list exception!"
,
e
);
}
}
...
...
@@ -540,7 +545,11 @@ public class ProcessInstanceService extends BaseDAGService {
// global param string
String
globalParamStr
=
JSONUtils
.
toJsonString
(
globalParams
);
globalParamStr
=
ParameterUtils
.
convertParameterPlaceholders
(
globalParamStr
,
timeParams
);
globalParams
=
JSONUtils
.
toList
(
globalParamStr
,
Property
.
class
);
try
{
globalParams
=
JSONUtils
.
getMapper
().
readValue
(
globalParamStr
,
new
TypeReference
<
List
<
Property
>>()
{});
}
catch
(
Exception
e
)
{
logger
.
error
(
"parse list exception!"
,
e
);
}
for
(
Property
property
:
globalParams
)
{
timeParams
.
put
(
property
.
getProp
(),
property
.
getValue
());
}
...
...
@@ -553,7 +562,13 @@ public class ProcessInstanceService extends BaseDAGService {
String
localParams
=
map
.
get
(
LOCAL_PARAMS
);
if
(
localParams
!=
null
&&
!
localParams
.
isEmpty
())
{
localParams
=
ParameterUtils
.
convertParameterPlaceholders
(
localParams
,
timeParams
);
List
<
Property
>
localParamsList
=
JSONUtils
.
toList
(
localParams
,
Property
.
class
);
List
<
Property
>
localParamsList
=
new
ArrayList
<>();
try
{
localParamsList
=
JSONUtils
.
getMapper
().
readValue
(
localParams
,
new
TypeReference
<
List
<
Property
>>()
{});
}
catch
(
Exception
e
)
{
logger
.
error
(
"parse list exception!"
,
e
);
}
Map
<
String
,
Object
>
localParamsMap
=
new
HashMap
<>();
localParamsMap
.
put
(
"taskType"
,
taskNode
.
getType
());
localParamsMap
.
put
(
"localParamsList"
,
localParamsList
);
...
...
dolphinscheduler-common/src/main/java/org/apache/dolphinscheduler/common/utils/JSONUtils.java
浏览文件 @
572ee0fc
...
...
@@ -50,6 +50,11 @@ public class JSONUtils {
objectMapper
.
configure
(
DeserializationFeature
.
ACCEPT_EMPTY_ARRAY_AS_NULL_OBJECT
,
true
).
setTimeZone
(
TimeZone
.
getDefault
());
}
public
static
ObjectMapper
getMapper
()
{
return
objectMapper
;
}
public
static
ArrayNode
createArrayNode
()
{
return
objectMapper
.
createArrayNode
();
}
...
...
@@ -124,7 +129,7 @@ public class JSONUtils {
return
objectMapper
.
readValue
(
json
,
new
TypeReference
<
List
<
T
>>()
{
});
}
catch
(
Exception
e
)
{
logger
.
error
(
"
JSONArray.parseArray
exception!"
,
e
);
logger
.
error
(
"
parse list
exception!"
,
e
);
}
return
new
ArrayList
<>();
...
...
dolphinscheduler-dao/src/main/java/org/apache/dolphinscheduler/dao/entity/ProcessDefinition.java
浏览文件 @
572ee0fc
...
...
@@ -21,15 +21,21 @@ import com.baomidou.mybatisplus.annotation.TableField;
import
com.baomidou.mybatisplus.annotation.TableId
;
import
com.baomidou.mybatisplus.annotation.TableName
;
import
com.baomidou.mybatisplus.core.toolkit.StringUtils
;
import
com.fasterxml.jackson.core.type.TypeReference
;
import
org.apache.dolphinscheduler.common.enums.Flag
;
import
org.apache.dolphinscheduler.common.enums.ReleaseState
;
import
org.apache.dolphinscheduler.common.process.Property
;
import
org.apache.dolphinscheduler.common.utils.*
;
import
org.slf4j.Logger
;
import
org.slf4j.LoggerFactory
;
import
java.io.IOException
;
import
java.util.ArrayList
;
import
java.util.Date
;
import
java.util.List
;
import
java.util.Map
;
import
java.util.stream.Collectors
;
import
java.util.stream.Stream
;
/**
...
...
@@ -37,10 +43,12 @@ import java.util.stream.Collectors;
*/
@TableName
(
"t_ds_process_definition"
)
public
class
ProcessDefinition
{
private
static
final
Logger
logger
=
LoggerFactory
.
getLogger
(
ProcessDefinition
.
class
);
/**
* id
*/
@TableId
(
value
=
"id"
,
type
=
IdType
.
AUTO
)
@TableId
(
value
=
"id"
,
type
=
IdType
.
AUTO
)
private
int
id
;
/**
...
...
@@ -81,14 +89,14 @@ public class ProcessDefinition {
/**
* user defined parameter list
*/
@TableField
(
exist
=
false
)
@TableField
(
exist
=
false
)
private
List
<
Property
>
globalParamList
;
/**
* user define parameter map
*/
@TableField
(
exist
=
false
)
private
Map
<
String
,
String
>
globalParamMap
;
@TableField
(
exist
=
false
)
private
Map
<
String
,
String
>
globalParamMap
;
/**
* create time
...
...
@@ -145,7 +153,7 @@ public class ProcessDefinition {
/**
* schedule release state : online/offline
*/
@TableField
(
exist
=
false
)
@TableField
(
exist
=
false
)
private
ReleaseState
scheduleReleaseState
;
/**
...
...
@@ -271,7 +279,11 @@ public class ProcessDefinition {
}
public
void
setGlobalParams
(
String
globalParams
)
{
this
.
globalParamList
=
JSONUtils
.
toList
(
globalParams
,
Property
.
class
);
try
{
this
.
globalParamList
=
JSONUtils
.
getMapper
().
readValue
(
globalParams
,
new
TypeReference
<
List
<
Property
>>()
{});
}
catch
(
IOException
e
)
{
logger
.
error
(
"json parse exception!"
,
e
);
}
this
.
globalParams
=
globalParams
;
}
...
...
@@ -285,10 +297,16 @@ public class ProcessDefinition {
}
public
Map
<
String
,
String
>
getGlobalParamMap
()
{
List
<
Property
>
propList
;
List
<
Property
>
propList
=
new
ArrayList
<>
()
;
if
(
globalParamMap
==
null
&&
StringUtils
.
isNotEmpty
(
globalParams
))
{
propList
=
JSONUtils
.
toList
(
globalParams
,
Property
.
class
);
try
{
propList
=
JSONUtils
.
getMapper
().
readValue
(
globalParams
,
new
TypeReference
<
List
<
Property
>>()
{
});
}
catch
(
IOException
e
)
{
logger
.
error
(
"json parse exception!"
,
e
);
}
globalParamMap
=
propList
.
stream
().
collect
(
Collectors
.
toMap
(
Property:
:
getProp
,
Property:
:
getValue
));
}
...
...
dolphinscheduler-dao/src/test/java/org/apache/dolphinscheduler/dao/entity/ProcessDefinitionTest.java
0 → 100644
浏览文件 @
572ee0fc
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package
org.apache.dolphinscheduler.dao.entity
;
import
org.junit.Assert
;
import
org.junit.Test
;
public
class
ProcessDefinitionTest
{
/**
* task instance sub process
*/
@Test
public
void
getGlobalParamMapTest
()
{
ProcessDefinition
taskInstance
=
new
ProcessDefinition
();
//sub process
taskInstance
.
setGlobalParams
(
"[{\"prop\":\"selenium_global_parameters_1\",\"direct\":\"IN\",\"type\":\"VARCHAR\",\"value\":\"selenium_global_parameters_value_1\"}]"
);
Assert
.
assertEquals
(
taskInstance
.
getGlobalParamMap
().
toString
(),
"{selenium_global_parameters_1=selenium_global_parameters_value_1}"
);
}
}
dolphinscheduler-server/src/main/java/org/apache/dolphinscheduler/server/worker/runner/TaskExecuteThread.java
浏览文件 @
572ee0fc
...
...
@@ -16,6 +16,7 @@
*/
package
org.apache.dolphinscheduler.server.worker.runner
;
import
com.fasterxml.jackson.core.type.TypeReference
;
import
org.apache.dolphinscheduler.common.utils.*
;
import
org.apache.dolphinscheduler.common.Constants
;
...
...
@@ -152,7 +153,13 @@ public class TaskExecuteThread implements Runnable {
// global params string
String
globalParamsStr
=
taskExecutionContext
.
getGlobalParams
();
if
(
globalParamsStr
!=
null
)
{
List
<
Property
>
globalParamsList
=
JSONUtils
.
toList
(
globalParamsStr
,
Property
.
class
);
List
<
Property
>
globalParamsList
=
new
ArrayList
<>();
try
{
globalParamsList
=
JSONUtils
.
getMapper
().
readValue
(
globalParamsStr
,
new
TypeReference
<
List
<
Property
>>()
{});
}
catch
(
Exception
e
)
{
logger
.
error
(
"parse list exception!"
,
e
);
}
globalParamsMap
.
putAll
(
globalParamsList
.
stream
().
collect
(
Collectors
.
toMap
(
Property:
:
getProp
,
Property:
:
getValue
)));
}
return
globalParamsMap
;
...
...
dolphinscheduler-service/src/main/java/org/apache/dolphinscheduler/service/process/ProcessService.java
浏览文件 @
572ee0fc
...
...
@@ -17,6 +17,7 @@
package
org.apache.dolphinscheduler.service.process
;
import
com.cronutils.model.Cron
;
import
com.fasterxml.jackson.core.type.TypeReference
;
import
com.fasterxml.jackson.databind.node.ObjectNode
;
import
org.apache.commons.lang.ArrayUtils
;
import
org.apache.dolphinscheduler.common.Constants
;
...
...
@@ -36,6 +37,7 @@ import org.springframework.beans.factory.annotation.Autowired;
import
org.springframework.stereotype.Component
;
import
org.springframework.transaction.annotation.Transactional
;
import
java.io.IOException
;
import
java.util.*
;
import
java.util.stream.Collectors
;
...
...
@@ -751,8 +753,16 @@ public class ProcessService {
* @return global params join
*/
private
String
joinGlobalParams
(
String
parentGlobalParams
,
String
subGlobalParams
){
List
<
Property
>
parentPropertyList
=
JSONUtils
.
toList
(
parentGlobalParams
,
Property
.
class
);
List
<
Property
>
subPropertyList
=
JSONUtils
.
toList
(
subGlobalParams
,
Property
.
class
);
List
<
Property
>
parentPropertyList
=
new
ArrayList
<>();
List
<
Property
>
subPropertyList
=
new
ArrayList
<>();
try
{
parentPropertyList
=
JSONUtils
.
getMapper
().
readValue
(
parentGlobalParams
,
new
TypeReference
<
List
<
Property
>>()
{});
subPropertyList
=
JSONUtils
.
getMapper
().
readValue
(
subGlobalParams
,
new
TypeReference
<
List
<
Property
>>()
{});
}
catch
(
IOException
e
)
{
logger
.
error
(
"json parse exception!"
,
e
);
}
Map
<
String
,
String
>
subMap
=
subPropertyList
.
stream
().
collect
(
Collectors
.
toMap
(
Property:
:
getProp
,
Property:
:
getValue
));
for
(
Property
parent
:
parentPropertyList
){
...
...
编辑
预览
Markdown
is supported
0%
请重试
或
添加新附件
.
添加附件
取消
You are about to add
0
people
to the discussion. Proceed with caution.
先完成此消息的编辑!
取消
想要评论请
注册
或
登录