After using namespaced kubernetes client, we will not need to always set the namespace when creating kubernetes resources(e.g. deployment, pods, configmap, watch, etc.).

Address comments: Update the unit test to verify the configmap watch is created in appropriate namespace

This closes #14570.

[FLINK-20650][k8s] Remove "native-k8s" command and add bash wrapper for JobManager and TaskManager container args

The "native-k8s" command has been deprecated. Flink should use the pass-through mode. Note that Flink needs to add a "bash -c" wrapper so that the environment variables could be expanded when executing in the "docker-entrypoint.sh".

The corresponding PR in flink-docker is https://github.com/apache/flink-docker/pull/49.

This closes #14419

This closes #14132.

[FLINK-19662][runtime][kubernetes][yarn][dist] Added dynamic parameters to BashJavaUtils' JobManager argument extraction. Additionally, the Kubernetes and Yarn deployment were extended accordingly.

The parameters are returned using the newly introduced global environment variable DYNAMIC_PARAMETERS.

This closes #13755.

[FLINK-18971][kubernetes] Support to mount kerberos conf as ConfigMap and Keytab as Secrete in Kubernetes

This closes #13263.

This closes #12899.

This closes #12054.

The JVM Direct Memory leak is unlikely in JM. Therefore, we could
disable its limit by default. This way it could span into e.g. JVM
Overhead w/o failure to improve the user experience as before FLIP-116.
If user needs the limit, e.g. to investigate container OOMs, the limit can be enabled by
setting the 'jobmanager.memory.enable-jvm-direct-memory-limit' option.

This closes #12745.

DeploymentOptionsInternal#CONF_DIR is an internal option and stores the client config path. It should not be added to config map and used by JobManager pod. Instead, KubernetesConfigOptions#FLINK_CONF_DIR will be used.

This closes #12501.

This closes #12455.

Revert "[FLINK-17935] Move set yarn.log-config-file to YarnClusterClientFactory.createClusterDescriptor()"

This reverts commit 2eb73771

This closes #12455.

This closes #12277 .

[FLINK-15836][k8s] Throw fatal error in KubernetesResourceManager when the pods watcher is closed with exception

By default the watcher will always reconnect in Kubernetes client internally. However, if the watchReconnectLimit is configured by users via java properties or environment, the watcher may be stopped. Then all the changes of pods will not be processed properly. The reason why the the watcher closed exceptionally is usually because of network problems or port abuse. The correct way is to fail the jobmanager pod and retry in a new one.

This closes #11010.

This closes #12105 .

[hotfix] Rename JobManagerProcessUtils#processSpecFromConfigWithFallbackForLegacyHeap to processSpecFromConfigAndFallbackToNewOptIfLegacyHeapSet

This closes #11949 .

This closes #11675.

This closes #11708.

This closes #11705 .

[FLINK-15790][k8s] Make some interfaces in FlinkKubeClient asynchronous which potentially blocks the execution of RpcEndpoint's main thread

The interfaces in FlinkKubeClient will be called both in Client and ResourceManager. To avoid potentially blocking the execution of RpcEndpoint's main thread, these interfaces #createTaskManagerPod, #stopPod should be implemented asynchronously.

This closes #11427.

[FLINK-16598][k8s] Respect the rest port exposed by Service in Fabric8FlinkKubeClient#getRestEndpoint

This closes #11715 .

Taints and tolerations work together to ensure that pods are not scheduled onto inappropriate nodes. One or more taints are applied to a node; this marks that the node should not accept any pods that do not tolerate the taints. Tolerations are applied to pods, and allow (but do not require) the pods to schedule onto nodes with matching taints.

This closes #11606 .

1.The REST service serves REST traffic while the internal service serves internal requests from TMs to JM.
2.The REST service is always created but the internal service is only created in non-high availability setup.
Co-authored-by: Nfelixzheng <felixzheng@tencent.com>

This closes #11456 .

The node-selector is a collection of key/value pairs to constrain a pod to only be able to run on particular node(s). Since affinity and anti-affinity are uncommon use case for Flink, so we leave the support in pod template.

This closes #11500 .

This closes  #11569 .