After using namespaced kubernetes client, we will not need to always set the namespace when creating kubernetes resources(e.g. deployment, pods, configmap, watch, etc.).

Address comments: Update the unit test to verify the configmap watch is created in appropriate namespace

This closes #14570.

[FLINK-20650][k8s] Remove "native-k8s" command and add bash wrapper for JobManager and TaskManager container args

The "native-k8s" command has been deprecated. Flink should use the pass-through mode. Note that Flink needs to add a "bash -c" wrapper so that the environment variables could be expanded when executing in the "docker-entrypoint.sh".

The corresponding PR in flink-docker is https://github.com/apache/flink-docker/pull/49.

This closes #14419

This closes #14132.

[FLINK-19662][runtime][kubernetes][yarn][dist] Added dynamic parameters to BashJavaUtils' JobManager argument extraction. Additionally, the Kubernetes and Yarn deployment were extended accordingly.

The parameters are returned using the newly introduced global environment variable DYNAMIC_PARAMETERS.

[FLINK-19662][runtime][core] Moved assembleDynamicConfigsStr into ConfigurationUtils to be used by both TaskExecutorProcessUtils and JobManagerProcessUtils.

This changed the way how the parameter list is generated for the TaskExecutor removing the final space. This made it necessary to update the JavaCmdTaskManagerDecoratorTest.

This closes #13755.

[FLINK-18971][kubernetes] Support to mount kerberos conf as ConfigMap and Keytab as Secrete in Kubernetes

This closes #13263.

This closes #12899.

This closes #12054.

The JVM Direct Memory leak is unlikely in JM. Therefore, we could
disable its limit by default. This way it could span into e.g. JVM
Overhead w/o failure to improve the user experience as before FLIP-116.
If user needs the limit, e.g. to investigate container OOMs, the limit can be enabled by
setting the 'jobmanager.memory.enable-jvm-direct-memory-limit' option.

This closes #12745.

DeploymentOptionsInternal#CONF_DIR is an internal option and stores the client config path. It should not be added to config map and used by JobManager pod. Instead, KubernetesConfigOptions#FLINK_CONF_DIR will be used.

This closes #12501.

This closes #12215 .

This closes #12277 .

[hotfix] Add setupFlinkConfig to KubernetesTestBase to make sure flinkConfig is always setup before used

[FLINK-15836][k8s] Throw fatal error in KubernetesResourceManager when the pods watcher is closed with exception

By default the watcher will always reconnect in Kubernetes client internally. However, if the watchReconnectLimit is configured by users via java properties or environment, the watcher may be stopped. Then all the changes of pods will not be processed properly. The reason why the the watcher closed exceptionally is usually because of network problems or port abuse. The correct way is to fail the jobmanager pod and retry in a new one.

This closes #11010.

This closes #12105 .

This closes #11949 .

This closes #11675.

This closes #11708.

This closes #11705 .

[FLINK-16437][runtime] Remove numSlotsPerTaskManager from ActiveResourceManager and ContaineredTaskManagerParameters.

Since now we compute pending slot profiles inside SlotManager, ResourceManagers no longer need to be aware of number of slots per worker.

[FLINK-15790][k8s] Make some interfaces in FlinkKubeClient asynchronous which potentially blocks the execution of RpcEndpoint's main thread

The interfaces in FlinkKubeClient will be called both in Client and ResourceManager. To avoid potentially blocking the execution of RpcEndpoint's main thread, these interfaces #createTaskManagerPod, #stopPod should be implemented asynchronously.

This closes #11427.