未验证 提交 a5912446 编写于 作者: J Jiaxiao Zheng 提交者: GitHub

Clean up 'out of date' banner for few more KFP pages (#2174)

* Squashed commit of the following:

commit a601bc00
Author: Oleg Butuzov <butuzov@users.noreply.github.com>
Date:   Wed Aug 21 02:25:17 2019 +0300

    markdown formatting fix. (#1088)

    this fix changes markdown formating for link to `/docs/started/k8s/overview` page,
    that currently broken on the website.

    see - https://www.kubeflow.org/docs/other-guides/virtual-dev/getting-started-minikube/

commit 48955d4b
Author: Yuan (Bob) Gong <gongyuan94@gmail.com>
Date:   Tue Aug 20 09:49:32 2019 +0800

    Update kubeflow pipelines upgrade guide for kubeflow 0.6 (#1055)

    * Fix kfp upgrade/reinstall preparation guide.

    * Update upgrade instructions for kubeflow deployment.

    * Remove kfp upgrade related instructions because they don't work

    * Rephrase PV to PD to be more accurate

commit 384368db
Author: Sarah Maddox <sarahmaddox@users.noreply.github.com>
Date:   Tue Aug 20 10:23:33 2019 +1000

    Clarified the usage reporting guide (#1087)

    * Clarified usage reporting guide.

    * Clarified and added section formatting.

    * Changed word component to application

    * Changed another "component" to "application"

commit 564e4606
Author: Yuval Yaari <7331634+yuval-yaari@users.noreply.github.com>
Date:   Sun Aug 18 03:46:04 2019 +0000

    Fixing command to retrieve the existing Dex configmap (#1086)

commit 1a863f43
Author: Leilei Hu <leileihu@cn.ibm.com>
Date:   Sat Aug 17 15:58:06 2019 -0400

    Modify the out of date guide docs. (#1085)

commit 5248a15e
Author: Yannis Zarkadas <yanniszark@arrikto.com>
Date:   Sat Aug 17 06:16:08 2019 +0300

    pinconfig links for v0.6 to v0.6-branch (#1071)
Signed-off-by: NYannis Zarkadas <yanniszark@arrikto.com>

commit 0f480d3a
Author: Adhita Selvaraj <adhita94@gmail.com>
Date:   Fri Aug 16 14:14:09 2019 -0700

    Update kfctl-k8s-istio.md (#1044)

    * Update kfctl-k8s-istio.md

    Update script for quickly curling the latest release.
    Similar to kustomize: https://github.com/kubernetes-sigs/kustomize/blob/master/docs/INSTALL.md#quickly-curl-the-latest

    * Keep both versions for install instructions

commit 09e44c21
Author: Sarah Maddox <sarahmaddox@users.noreply.github.com>
Date:   Sat Aug 17 06:38:09 2019 +1000

    Clarified port forwarding and auth for GCP (#1083)

    * Clarifies port forwarding and auth for GCP.

    * Fixed formatting.

    * Addressed review comments.

commit a8d98874
Author: Yuan (Bob) Gong <gongyuan94@gmail.com>
Date:   Fri Aug 16 08:12:34 2019 +0800

    Update pipeline quickstart for kf 0.6 (#1080)

commit 1654842f
Author: cliveseldon <cc@seldon.io>
Date:   Fri Aug 16 00:54:34 2019 +0100

    Update Seldon docs (#1073)

    * Update Seldon docs

    * Update docs based on review

commit 1b9e0d6c
Author: Jiaxin Shan <seedjeffwan@gmail.com>
Date:   Wed Aug 14 19:44:33 2019 -0700

    Merge installation guidance for new and existing cluster (#1072)

    * Merge installation guidance for new and existing cluster

    * Address code review feedbacks

commit 2b5c19a0
Author: Sarah Maddox <sarahmaddox@users.noreply.github.com>
Date:   Thu Aug 15 12:30:33 2019 +1000

    Change will to may in metadata warning. (#1078)

commit 8177767e
Author: Sarah Maddox <sarahmaddox@users.noreply.github.com>
Date:   Tue Aug 13 17:03:39 2019 +1000

    Expanded docs for the Metadata component (#1061)

    * WIP Expanded docs for Metadata component.

    * Added more about metadata.

    * Addressed review comments.

commit 5b6caa19
Author: Jeremy Lewi <jeremy+github@lewi.us>
Date:   Mon Aug 12 18:33:37 2019 -0700

    Fix port-forarding instructions; port-forward to istio gateway (#1067)

    * Fix port-forarding instructions; port-forward to istio gateway

    See #1041 and kubeflow/kubeflow#3618

    * With 0.6 if no auth is setup and you want to connect via port-forwarding
      then you need to port-forward to the ISTIO gateway; not ambassador

    * I suspect the original suggestion in #1041 to port-forward to Ambassador
      only worked in the basic auth setting in which case we are still setting
      up Ambassador.

    Fix: #1041

    * Cleanup the text.

commit 2d9564cb
Author: Zhenghui Wang <zhenghuiwang@users.noreply.github.com>
Date:   Mon Aug 12 17:41:38 2019 -0700

    Add metadata API reference doc (#1058)

    * add metadata API spec

    * add link to metadata doc page

    * address comments

    * remove metadata.md

commit d01f2cb3
Author: Christos S. Pavlou <cspavlou@arrikto.com>
Date:   Tue Aug 13 00:25:38 2019 +0300

    Amend the kfctl_existing_arrikto doc (#1056)

    Add instructions for installing MetalLB
    Add `Troubleshooting` section
    Add architecture figure
    Change the kfctl_existing_arrikto description in the Overview page
Signed-off-by: NChristos S. Pavlou <cspavlou@arrikto.com>

commit 85ba5bbf
Author: Sarah Maddox <sarahmaddox@users.noreply.github.com>
Date:   Tue Aug 13 02:15:13 2019 +1000

    Refactored 'Other Guides' and applied redirects. (#1064)

commit cf7b9f3a
Author: Jiaxiao Zheng <jxzheng@google.com>
Date:   Sun Aug 11 18:21:10 2019 -0700

    Fix broken links. (#1060)

commit 40a3f49f
Author: Carmine Rimi <carmine.rimi@canonical.com>
Date:   Sun Aug 11 19:43:10 2019 -0400

    Refactor getting started - streamlined hierarchy (#1051)

    * Refactor getting started - streamlined hierarchy

    A number of changes / benefits:
    1) eliminated some duplicate information
    2) added three main top level areas for installation:
    - cloud
    - workstation / server
    - existing kubernetes
    3) Adding workstation operating systems - there are some slight
       variances in experience. For instance, for MacOS, we can emphasize
       brew to install some of the assets. For Windows, we'll want to add
       WSL2, and potentially chocolatey.

    * Updated PR #1051 with recommended changes

    This commit holds a set of minor changes that were requested in #1051

    * Move VM based local workstation to reference

    Moving the VM instructions out of getting started directory tree
    so that the navigation bar remains clean / less busy.

    * Add deleted assets files, reword k8s _index

    A previous commit deleted the resources/_gen/assets/*/search*.content
    file. This commit adds it back in.

    Additionally, fixed the grammar in k8s/_index.md

commit 05d3d923
Author: Ning <ngao@google.com>
Date:   Fri Aug 9 21:51:13 2019 -0700

    update sample link after kfp sample restructure (#1057)

commit 9c7fcd8b
Author: IronPan <yangpa@google.com>
Date:   Fri Aug 9 20:33:13 2019 -0700

    Update pipeline API version (#1054)

    * update pipeline deployment doc

    * update pipeline deployment doc

    * update pipeline deployment doc

    * update pipeline deployment doc

    * Update deploy-pipelines-service.md

    * update pipeline deployment doc

    * update pipeline deployment doc

    * update pipeline deployment doc

    * update pipeline deployment doc

    * comment

    * format

    * update kf version

    * update deployment wording

    * update deployment wording

    * update deployment wording

    * update

commit 61de916e
Author: Lun-Kai Hsu <lunkai@google.com>
Date:   Fri Aug 9 16:35:14 2019 -0700

    Fix tfserving doc (#1025)

    * fix tfserving doc

    * fix

    * fix

commit 0823f78d
Author: Sarah Maddox <sarahmaddox@users.noreply.github.com>
Date:   Sat Aug 10 06:51:13 2019 +1000

    Updated screenshots and UI info for Kubeflow v0.6 (#1029)

    * WIP Updating screenshots and UI info for v0.6

    * Fixed UI text and URLs in pipelines quickstart.

    * Fixed UI text and clarified Cloud Shell in Pipelines tutorial.

    * Removed screenshot of Kubeflow central UI as it doesn't show TFJob.

    * Started updates to notebooks.

    * Updated port-forwarding instructions for UI.

    * Updated UI for notebooks.

    * Finished UI updates for notebooks.

    * Fixed caps in URL variable.

commit cc50aca1
Author: Ed Henry <edward.m.henry@gmail.com>
Date:   Thu Aug 8 21:41:24 2019 -0700

    xgboost url update (#1052)

    * xgboost url update

    Pipeline overview doc reflected a stale URL. This commit updates to the correct URL of the xgboost-spark directory of the Kubeflow repo.

    * Remove docsy submodule

    Removing the docsy submodule

commit ca09f1f0
Author: Jiaxin Shan <seedjeffwan@gmail.com>
Date:   Thu Aug 8 20:33:25 2019 -0700

    Update aws docs for 0.6 release (#1030)

    * Update aws docs for 0.6 release

    * Address code review feedbacks

    * Update indents to render markdown

    * Remove shell in code blocks

    * Fix indent issue

    * Make existing cluster solution more clear

commit d09cecc1
Author: Krishna Durai <krishnadurai20@gmail.com>
Date:   Fri Aug 9 08:55:25 2019 +0530

    Instructions on setting up dynamic volume provisioner (#1040)

    * Instructions on setting up dynamic volume provisioner
    Resolves kubeflow/kubeflow#3774

    * Clarifies what to delete

    * Rearranges and clarifies descriptions

    * Re-arranges and rephrases content

    * Fixes link
    Corrects description

    * Addresses review comments

commit f557a0e6
Author: Hougang Liu <liuhoug@cn.ibm.com>
Date:   Thu Aug 8 12:27:16 2019 +0800

    Update katib Dockerfile locations link (#1049)

commit 0b03ae0c
Author: Abhishek Gupta <abhishek@google.com>
Date:   Wed Aug 7 21:19:17 2019 -0700

    Update to include limitations and integrations (#1047)

commit beefae93
Author: Michael Benavidez <MBenavidez@gmail.com>
Date:   Wed Aug 7 09:02:51 2019 -0700

    PR for Issue 795 (outdated Pipelines SDK guide) (#971)

    * Update pipelines-tutorial.md

    * Add files via upload

    * Update kubeflow-current-version.html

    * Delete kubeflow-current-version.html

    Unnecessary, kf-stable-tag already serves this purpose.

    * Update pipelines-tutorial.md

    * Update pipelines-tutorial.md

    * Update pipelines-tutorial.md

    * Update build-component.md

    * Update build-component.md

    * Update build-component.md

    * Update build-component.md

    * Update build-component.md

    * Update build-component.md

commit 2f5356c3
Author: Yuan (Bob) Gong <gongyuan94@gmail.com>
Date:   Wed Aug 7 10:32:53 2019 +0800

    Fix other js features broken by heading anchor link. (#1039)

commit c36ace3f
Author: Jin Chi He <hejinchi@cn.ibm.com>
Date:   Wed Aug 7 09:24:53 2019 +0800

    fix mini problem in fairing doc (#1031)

commit 670f8b4a
Author: Leilei Hu <leileihu@cn.ibm.com>
Date:   Tue Aug 6 21:18:51 2019 -0400

    Fix docs link 404 errors. (#1032)

commit ef6374a5
Author: Sarah Maddox <sarahmaddox@users.noreply.github.com>
Date:   Wed Aug 7 10:28:52 2019 +1000

    Added redirect and fixed links to getting-started guide for existing clusters (#1035)

    * Added redirect and fixed links to existing-custer getting-started guide.

    * Added empty line at end of file.

commit dda2f0ce
Author: Sarah Maddox <sarahmaddox@google.com>
Date:   Sun Aug 4 06:23:28 2019 +1000

    Added a warning.

commit d27e54b1
Author: Sarah Maddox <sarahmaddox@google.com>
Date:   Fri Aug 2 10:53:22 2019 +1000

    Fixed a typo.

commit 1a5bd1ba
Author: Sarah Maddox <sarahmaddox@google.com>
Date:   Fri Aug 2 10:48:50 2019 +1000

    Fixed a couple more links.

commit 065a1330
Author: Sarah Maddox <sarahmaddox@google.com>
Date:   Fri Aug 2 10:35:45 2019 +1000

    Added page for shared components and resources

commit 70bda37a
Author: Johnu George <johnugeorge109@gmail.com>
Date:   Tue Aug 6 22:57:56 2019 +0530

    Fixing doc for KF installation on existing clusters (#977)

    * Fixing doc for existing clusters

    * Update authentication.md

    * Update getting-started-k8s.md

    * Update getting-started-k8s.md

    * Address review comments

    * Minor changes

    * Address review comments

    * Address review comments

    * Minor title change

    * Fix path

commit 63b34cb1
Author: Sarah Maddox <sarahmaddox@users.noreply.github.com>
Date:   Tue Aug 6 13:55:53 2019 +1000

    Fixed up the GCP deployment guide for CLI v0.6 (#1021)

    * WIP Fixing up the GCP deployment guide for CLI.

    * Further updates during CLI deployment testing.

    * More tweaks to wording for clarity.

commit 39bf96eb
Author: Scott Lee <scottilee@users.noreply.github.com>
Date:   Mon Aug 5 19:39:53 2019 -0700

    Add references to kustomize issues (#1015)

commit f1f3a590
Author: Eric Lilienstein <5490671+elilienstein@users.noreply.github.com>
Date:   Mon Aug 5 18:53:54 2019 -0700

    Deployment UI requires project id, not name (#1028)

    Following the instructions above:
    - go to https://deploy.kubeflow.cloud
    - enter previously created GCP project name and other required information
    - click 'Create Deployment'
    Result:
    An error message indicates that the project can't be located. The required information was the project ID, not the project name

commit f3d9c3a0
Author: Amy <amyu@google.com>
Date:   Mon Aug 5 17:43:54 2019 -0700

    typo fix (#1027)

commit b3fb2385
Author: Amy <amyu@google.com>
Date:   Mon Aug 5 15:19:49 2019 -0700

    added a few clarifications to the GKE CLI installation instructions (#1026)

commit ecc9c5ba
Author: Yuan (Bob) Gong <gongyuan94@gmail.com>
Date:   Mon Aug 5 12:55:48 2019 +0800

    Fix anchor icon color + formatting (#1022)

commit db5ad7e7
Author: Matt Callanan <mattcallanan@users.noreply.github.com>
Date:   Mon Aug 5 06:33:48 2019 +1000

    Fixed link to xgboost-training-cm.py (#1018)

commit 89fd3f89
Author: Yuan (Bob) Gong <gongyuan94@gmail.com>
Date:   Mon Aug 5 04:29:49 2019 +0800

    Add anchor links to markdown headings. (#1009)

    * Add anchor links automatically.

    * Improve anchor link UX

    * improve corner-case layout stability

    * improve a11y

    * Change link icon to material icon

commit 1634ea32
Author: Abhishek Gupta <abhishek@google.com>
Date:   Fri Aug 2 15:43:54 2019 -0400

    add owner role requirement in GCP project setup (#1016)

commit 1dd668e7
Author: Iman Tabrizian <tabrizian@outlook.com>
Date:   Fri Aug 2 01:15:52 2019 -0400

    Add troubleshooting guide for notebooks (#1008)

    * feat: add template of troubleshooting guide for notebooks

    * feat: improve the notebooks troubleshooting guide

    * fix: improve the note for GCP users

    * fix: minor bug in the link

    * fix: improve the troubleshooting guide

    * fix: fix troubleshooting guides

commit 3bc674e9
Author: Sarah Maddox <sarahmaddox@users.noreply.github.com>
Date:   Thu Aug 1 23:26:15 2019 +1000

    Moved GCP Pipelines tutorial to GCP section of docs (#1004)

    * Moved GCP Pipelines tutorial to GCP section of docs.

    * Addressed review comments.

commit 38ab9cc2
Author: Ce Gao <gaoce@caicloud.io>
Date:   Thu Aug 1 09:42:14 2019 +0800

    fix: Fix the repo (#1005)

    * fix: Fix the repo
Signed-off-by: NCe Gao <gaoce@caicloud.io>

    * fix: Fix the statement
Signed-off-by: NCe Gao <gaoce@caicloud.io>

commit 070a0ead
Author: Ning <ngao@google.com>
Date:   Wed Jul 31 16:54:13 2019 -0700

    minor fix (#1007)

commit be10697e
Author: Sarah Maddox <sarahmaddox@users.noreply.github.com>
Date:   Thu Aug 1 03:18:14 2019 +1000

    Added link from notebooks setup to multi-user isolation doc. (#1001)

commit 7b086d70
Author: Sarah Maddox <sarahmaddox@users.noreply.github.com>
Date:   Wed Jul 31 15:22:20 2019 +1000

    Fixed links to tutorials and codelabs. (#1003)

commit e6eb2914
Author: Jiaxiao Zheng <jxzheng@google.com>
Date:   Tue Jul 30 16:45:10 2019 -0700

    Update customizing-gke.md (#987)

    * Update customizing-gke.md

    Fix stale links.

    * Update customizing-gke.md

    Fix the links to cluster.jinja also

commit 67f6edb1
Author: Jiaxiao Zheng <jxzheng@google.com>
Date:   Tue Jul 30 16:31:09 2019 -0700

    Update customizing-gke.md (#999)

    Correct a typo

commit 395c848a
Author: Zhenghui Wang <zhenghuiwang@users.noreply.github.com>
Date:   Mon Jul 29 17:21:38 2019 -0700

    Update metadata.md

    change the minimum requirements for using Metadata components.

commit 7f4beb78
Author: Sarah Maddox <sarahmaddox@users.noreply.github.com>
Date:   Tue Jul 30 09:18:49 2019 +1000

    Clarified versions available in deployment UI and CLI (#981)

    * Clarified versions available in deployment UI and CLI.

    * Removed empty line at EOF because it causes a space in rendering on page.

    * Review comment
Co-Authored-By: NDwight J Lyle <dwightjl@users.noreply.github.com>

commit fae0bf98
Author: Karen Bradshaw <kbhawkey@gmail.com>
Date:   Mon Jul 29 17:58:49 2019 -0400

    fix a few typos (#985)

    - See #936
    - See #982

* minor typo and update a screenshot.

* Update central-ui.png and correct wording

* Correct stale naming

* Update pipeline ui screenshot

* Update pipeline upload screenshot

* Update three pictures.

* SDK install page

* pipeline param

* lint and remove banner for type check in DSL

* remove banner for real

* gcp

* recover image

* fix typo

* Apply PR comments
上级 2956504c
......@@ -4,10 +4,6 @@ description = "SDK features that are available on Google Cloud Platform (GCP) on
weight = 130
+++
{{% alert title="Out of date" color="warning" %}}
This guide contains outdated information pertaining to Kubeflow 1.0. This guide
needs to be updated for Kubeflow 1.1.
{{% /alert %}}
For pipeline features that are specific to GCP, including SDK features, see the
[GCP section of the docs](/docs/gke/pipelines/).
......@@ -4,10 +4,6 @@ description = "Setting up your Kubeflow Pipelines development environment"
weight = 20
+++
{{% alert title="Out of date" color="warning" %}}
This guide contains outdated information pertaining to Kubeflow 1.0. This guide
needs to be updated for Kubeflow 1.1.
{{% /alert %}}
This guide tells you how to install the
[Kubeflow Pipelines SDK](https://github.com/kubeflow/pipelines/tree/master/sdk)
......@@ -80,16 +76,20 @@ Run the following command to install the Kubeflow Pipelines SDK:
```bash
pip3 install kfp --upgrade
```
**Note:** If you are not using a virtual environment, such as `conda`, when installing the Kubeflow Pipelines SDK, you may receive the following error:
```bash
ERROR: Could not install packages due to an EnvironmentError: [Errno 13] Permission denied: '/usr/local/lib/python3.5/dist-packages/kfp-0.2.0.dist-info'
ERROR: Could not install packages due to an EnvironmentError: [Errno 13] Permission denied: '/usr/local/lib/python3.5/dist-packages/kfp-<version>.dist-info'
Consider using the `--user` option or check the permissions.
```
If you get this error, install `kfp` with the `--user` option:
```bash
pip3 install kfp --upgrade --user
```
This command installs the `dsl-compile` and `kfp` binaries under `~/.local/bin`, which is not part of the PATH in some Linux distributions, such as Ubuntu. You can add `~/.local/bin` to your PATH by appending the following to a new line at the end of your `.bashrc` file:
```bash
......
......@@ -4,10 +4,6 @@ description = "Passing data between pipeline components"
weight = 70
+++
{{% alert title="Out of date" color="warning" %}}
This guide contains outdated information pertaining to Kubeflow 1.0. This guide
needs to be updated for Kubeflow 1.1.
{{% /alert %}}
The [`kfp.dsl.PipelineParam`
class](https://kubeflow-pipelines.readthedocs.io/en/latest/source/kfp.dsl.html#kfp.dsl.PipelineParam)
......@@ -23,7 +19,7 @@ The task output references can again be passed to other components as arguments.
In most cases you do not need to construct `PipelineParam` objects manually.
The following code sample shows how to define pipeline with parameters:
The following code sample shows how to define a pipeline with parameters:
```python
@kfp.dsl.pipeline(
......@@ -36,7 +32,9 @@ def my_pipeline(
my_url: str = 'http://example.com'
):
...
# In the pipeline function body you can use the `my_num`, `my_name`,
# `my_url` arguments as PipelineParam objects.
```
See more in the guide to [building a
component](/docs/pipelines/sdk/build-component/#create-a-python-class-for-your-component).
For more information, you can refer to the guide on
[building components and pipelines](/docs/pipelines/sdk/build-component/#create-a-python-class-for-your-component).
\ No newline at end of file
......@@ -4,10 +4,6 @@ description = "Statically check the component I/O types"
weight = 100
+++
{{% alert title="Out of date" color="warning" %}}
This guide contains outdated information pertaining to Kubeflow 1.0. This guide
needs to be updated for Kubeflow 1.1.
{{% /alert %}}
This page describes how to integrate the type information in the pipeline and utilize the
static type checking for fast development iterations.
......@@ -37,6 +33,7 @@ In the component YAML, types are specified as a string or a dictionary with the
"*component a*" expects an input with Integer type and emits three outputs with the type GCSPath, customized_type and GCRPath.
Among these types, Integer, GCSPath, and GCRPath are core types that are predefined in the SDK while customized_type is a user-defined
type.
```yaml
name: component a
description: component desc
......@@ -58,55 +55,80 @@ implementation:
field_n: /feature.txt
field_o: /output.txt
```
Similarly, when you write a component with the decorator, you can annotate I/O with types in the function signature, as shown below.
```python
from kfp.dsl import component
from kfp.dsl.types import Integer, GCRPath
@component
def task_factory_a(field_l: Integer()) -> {'field_m': {'GCSPath': {'openapi_schema_validator': '{"type": "string", "pattern": "^gs://.*$"}'}},
'field_n': 'customized_type',
'field_o': GCRPath()
}:
return ContainerOp(
name = 'operator a',
image = 'gcr.io/ml-pipeline/component-a',
command = [
'python3', '/pipelines/component/src/train.py'
],
arguments = [
'--field-l', field_l,
],
file_outputs = {
'field_m': '/schema.txt',
'field_n': '/feature.txt',
'field_o': '/output.txt'
def task_factory_a(field_l: Integer()) -> {
'field_m': {
'GCSPath': {
'openapi_schema_validator':
'{"type": "string", "pattern": "^gs://.*$"}'
}
)
},
'field_n': 'customized_type',
'field_o': GCRPath()
}:
return ContainerOp(
name='operator a',
image='gcr.io/ml-pipeline/component-a',
command=['python3', '/pipelines/component/src/train.py'],
arguments=[
'--field-l',
field_l,
],
file_outputs={
'field_m': '/schema.txt',
'field_n': '/feature.txt',
'field_o': '/output.txt'
})
```
You can also annotate pipeline inputs with types and the input are checked against the component I/O types as well. For example,
```python
@component
def task_factory_a(field_m: {'GCSPath': {'openapi_schema_validator': '{"type": "string", "pattern": "^gs://.*$"}'}}, field_o: 'Integer'):
return ContainerOp(
name = 'operator a',
image = 'gcr.io/ml-pipeline/component-a',
arguments = [
'--field-l', field_m,
'--field-o', field_o,
],
)
def task_factory_a(
field_m: {
'GCSPath': {
'openapi_schema_validator':
'{"type": "string", "pattern": "^gs://.*$"}'
}
}, field_o: 'Integer'):
return ContainerOp(
name='operator a',
image='gcr.io/ml-pipeline/component-a',
arguments=[
'--field-l',
field_m,
'--field-o',
field_o,
],
)
# Pipeline input types are also checked against the component I/O types.
@dsl.pipeline(name='type_check',
description='')
def pipeline(a: {'GCSPath': {'openapi_schema_validator': '{"type": "string", "pattern": "^gs://.*$"}'}}='good', b: Integer()=12):
task_factory_a(field_m=a, field_o=b)
@dsl.pipeline(name='type_check', description='')
def pipeline(
a: {
'GCSPath': {
'openapi_schema_validator':
'{"type": "string", "pattern": "^gs://.*$"}'
}
} = 'good',
b: Integer() = 12):
task_factory_a(field_m=a, field_o=b)
try:
compiler.Compiler().compile(pipeline, 'pipeline.tar.gz', type_check=True)
compiler.Compiler().compile(pipeline, 'pipeline.tar.gz', type_check=True)
except InconsistentTypeException as e:
print(e)
print(e)
```
## How does the type checking work?
......@@ -127,26 +149,35 @@ If inconsistent types are detected, it throws an [InconsistentTypeException](htt
Type checking is enabled by default and it can be disabled in two ways:
If you compile the pipeline programmably:
```python
compiler.Compiler().compile(pipeline_a, 'pipeline_a.tar.gz', type_check=False)
```
If you compile the pipeline using the dsl-compiler tool:
```bash
dsl-compiler --py pipeline.py --output pipeline.zip --disable-type-check
```
### Fine-grained configuration
Sometimes, you might want to enable the type checking but disable certain arguments. For example,
when the upstream component generates an output with type "*Float*" and the downstream can ingest either
"*Float*" or "*Integer*", it might fail if you define the type as "*Float_or_Integer*".
Disabling the type checking per-argument is also supported as shown below.
```python
@dsl.pipeline(name='type_check_a',
description='')
@dsl.pipeline(name='type_check_a', description='')
def pipeline():
a = task_factory_a(field_l=12)
# For each of the arguments, you can also ignore the types by calling ignore_type function.
b = task_factory_b(field_x=a.outputs['field_n'], field_y=a.outputs['field_o'], field_z=a.outputs['field_m'].ignore_type())
a = task_factory_a(field_l=12)
# For each of the arguments, you can also ignore the types by calling
# ignore_type function.
b = task_factory_b(
field_x=a.outputs['field_n'],
field_y=a.outputs['field_o'],
field_z=a.outputs['field_m'].ignore_type())
compiler.Compiler().compile(pipeline, 'pipeline.tar.gz', type_check=True)
```
......@@ -158,4 +189,6 @@ type checking would still fail if some I/Os lack the type information and some I
## Next steps
* See [type checking sample](https://github.com/kubeflow/pipelines/blob/master/samples/core/dsl_static_type_checking/dsl_static_type_checking.ipynb).
Learn how to define a KubeFlow pipeline with Python DSL and compile the
pipeline with type checking: a
[Jupyter notebook demo](https://github.com/kubeflow/pipelines/blob/master/samples/core/dsl_static_type_checking/dsl_static_type_checking.ipynb).
Markdown is supported
0% .
You are about to add 0 people to the discussion. Proceed with caution.
先完成此消息的编辑!
想要评论请 注册