* Make unit tests to only use dynamically allocated ports

* Fixed proxy tests

* Fixed zk test utils after merge

* Fixed multi host client test

* Fix for testConcurrentConsumerReceiveWhileReconnect

* Increased timeout on testPulsarSourceLocalRunWithFile

* Fixed newer test to also avoid port manager

* Fixed test race condition with thread starting in PulsarFunctionLocalRunTest

* Fixed ProxyWithAuthorizationNegTest

* Fixed ProxySaslAuthenticationTest

Allow volume override for docker commands
with env variable, this allows the scripts to
work in dind environments.

### Motivation

The standalone service is failing to startup when the hostname is not resolvable to an IP. This is common in laptops. 

The reason is that the function worker in standalone is trying to connect to the advertised address rather than "localhost". 

To fix this, we should make the standalone to always advertise "localhost" as the the broker address. This is ok because in any case the standalone treats the advertisement as the pulsar proxy, in that it forces the clients to always go through the same service url, so a client connecting from a different host won't have problems.

Motivation
In reader builder, clone() method does not clone the config, and will cause error once we call a builder.clone() concurrently to create readers. This PR mainly try to fix this issue.

Modifications
use conf.clone(), to make sure conf is cloned.
add test to verify it.
fix other small issues, like get() methods, and access level, make it align with consumer/producer builder.
Verifying this change
ut passed.

Motivation
Fixes #5597

Add backoff retries when getting partitioned metadata from brokers.

The change in #5734 (copy from #5603) used the wrong time unit when inited Backoff which failed to trigger the retry logic as expected.

Modifications
Correct the time unit and add some useful log.

* Cache Pulsar client to make it shared among tasks in a process

* code format & add tests

* fix style
Co-authored-by: NSijie Guo <guosijie@gmail.com>

fix  2 copies of 3bookie and stop 2 booksie after production. After the entire bookie is restored, it can be produced at this time, and the produced data cannot be consumed normally. Restarting the broker can resume normal (#5965)

After stopping the bookie and recovering, it can be produced at this time, but cannot be consumed normally. Restarting the broker can resume normal

Fixes #5962

Motivation
problem :
Topic(E=3,W=3,A=2), stopped 2 bookie nodes, then recover the 2 bookie,
The message can be produced normally, but the consumer cannot pull the message unless the broker is restarted.
We hope that the bookie summary will resume normal consumption

Modifications
The main cause of this problem is that readHandle.readAsync does not catch the exception,
causing some methods to not trigger.
therefore, added the exception capture module , and the exceptions involved in the bookeeper project are also fixed.

Fixes #4075

Motivation
Debezium Postgres connector will first create a snap for the current table. And this step will hang if the table is too big.

Modifications
Add an introduction to parameter max.queue.size
Verifying this change
Integration tests
yarn build

*Motivation*

Currently proxy uses the proxy ca certicate to connect to brokers.
It is fine if proxy and broker are using th same CA. However if the broker
is using a different CA than proxy, "HTTP 502 Bad Gateway" is returned from proxy
when tlsEnabledWithBroker is set to true.

*Modifications*

Change to use the right CA

*Verify this change*

Verify it is working in a production environment.

Signed-off-by: Nxiaolong.ran <rxl@apache.org>

Set the startup order of broker and bookie

### Motivation

Most of the configuration classes of the connectors had the following annotations:

```
@data
@Setter	
@Getter	
@EqualsAndHashCode	
@ToString
```

The [@data](https://projectlombok.org/features/Data) annotation includes all other annotations:

> All together now: A shortcut for @ToString, @EqualsAndHashCode, @Getter on all fields, @Setter on all non-final fields, and @RequiredArgsConstructor!


### Modifications

Removed `@Setter`, `@Getter`, `@EqualsAndHashCode`, and '@ToString' if the `@data` annotation was also present

Most namespace level configurations have corresponding cluster
configuration that set a namespace default.

The offload threshold does not, which makes it more difficult to ensure
that namespaces have the cluster wide namespace defaults.

There is one small wrinkle with this commit in that `-1` is used as a
sentinel value to indicate to use the cluster default, this means that
if the cluster default is to have offloading on and it is desired to
  disable a specific namespace, the namespace needs to set this value to
  some negative number other than `-1`!

Fixes #5454 


### Motivation

The current CPP client cannot correctly obtain the schema version, resulting in an error in parsing with java client when sending data with schema using python client.

Test code:

```
import pulsar
import json

from pulsar.schema import *

class Test(Record):
    name = String()
    id = Integer()

client = pulsar.Client('pulsar://localhost:6650');
producer = client.create_producer('test-producer-schema', schema=AvroSchema(Test))
producer.send(Test(name='Hello', id=1))
client.close()
```



### Modifications

* Add set schema version in msgmetadata

### Verifying this change

Add check schema version in unit test SchemaTest

Fixes #5841

*Motivation*

when enabling authentication and authorization, if a user using
the wrong key to send to the server, the server will return an
`Error` message. There is no `Connected` message return to the
client so the client is staying in `SentConnectFrame` and it can
receive the server `Error` message.
We need to check the `SentConnectFrame` state when receiving `Error` message.

The client will throw error:

```
java.lang.IllegalArgumentException: null
	at com.google.common.base.Preconditions.checkArgument(Preconditions.java:108) ~[guava-21.0.jar:?]
	at org.apache.pulsar.client.impl.ClientCnx.handleError(ClientCnx.java:588) ~[pulsar-client-original-2.4.1.jar:2.4.1]
	at org.apache.pulsar.common.protocol.PulsarDecoder.channelRead(PulsarDecoder.java:154) ~[pulsar-common-2.4.1.jar:2.4.1]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-all-4.1.32.Final.jar:4.1.32.Final]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-all-4.1.32.Final.jar:4.1.32.Final]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-all-4.1.32.Final.jar:4.1.32.Final]
	at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:323) [netty-all-4.1.32.Final.jar:4.1.32.Final]
	at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:297) [netty-all-4.1.32.Final.jar:4.1.32.Final]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-all-4.1.32.Final.jar:4.1.32.Final]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-all-4.1.32.Final.jar:4.1.32.Final]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-all-4.1.32.Final.jar:4.1.32.Final]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1434) [netty-all-4.1.32.Final.jar:4.1.32.Final]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-all-4.1.32.Final.jar:4.1.32.Final]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-all-4.1.32.Final.jar:4.1.32.Final]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:965) [netty-all-4.1.32.Final.jar:4.1.32.Final]
	at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:799) [netty-all-4.1.32.Final.jar:4.1.32.Final]
	at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:433) [netty-all-4.1.32.Final.jar:4.1.32.Final]
	at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:330) [netty-all-4.1.32.Final.jar:4.1.32.Final]
	at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:909) [netty-all-4.1.32.Final.jar:4.1.32.Final]
	at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) [netty-all-4.1.32.Final.jar:4.1.32.Final]
	at java.lang.Thread.run(Thread.java:748) [?:1.8.0_201]
```

*Modifications*

- Add `SentConnectFrame` check

Fixes #5967

### Motivation
Cannot build the project successfully.

### Modifications
Add install commands to install the unzip zip tools when vm sets up

### Motivation

The current GitHub action test will show that the disk space exceeds 95%, and then the bookie will detect that bookie exits, so fix this problem.

![image](https://user-images.githubusercontent.com/1907867/71616693-8748da00-2bf2-11ea-82f2-e54eda012136.png)


### Modifications

* Configuration parameter disUsageThreshold is 0.99 to prevent bookie exit.

### Verifying this change

Integration test pass

Signed-off-by: Nxiaolong.ran <rxl@apache.org>

### Motivation
Generally, users rarely change the configuration of downloadDirectory. If we store it in the /tmp directory, the file will be invalid after a period of time.

### Modifications

Replace /tmp/pulsar_functions with download/pulsar_functions

The build fails with license check with maven 3.6.2 which is part of ubuntu-latest in gitlab actions, here we downgrade it get it to pass.

Move additional jobs to maven 3.6.1 to improve stability.

Currently functions do not report the default Prometheus metrics
like the broker does, or any JMX metrics that normally get
reported by kafka clients.

This add the default Prometheus exports used by the broker as
well as JMX exports that are reported by kafka clients when
using the kafka wrapper.

This change is a trivial rework / code cleanup without any test coverage.

Fixes #5881

### Motivation

Currently implementation only increase redelivery count when consumer call redeliver un-ack messages, since a consumer disconnect also let messages redeliver to other consumers in Shared subscription mode, but the redelivery count does not increase. 

### Modifications

1. When consumer send redelivery un-ack message request or consumer disconnect, add the position to redelivery message tracker.
2. When send messages to consumer will check if the tracker contains this message, if yes, increase redelivery count, otherwise use 0 as redelivery count.

### Verifying this change

Added unit test for this change

### Motivation

Fix topic stop dispatch messages to consumers after read or replay 0 entries from bookie
fter check the broker log, i find following logs:
```
23:18:27.125 [bookkeeper-ml-workers-OrderedExecutor-0-0] DEBUG org.apache.pulsar.broker.service.persistent.PersistentDispatcherMultipleConsumers - [persistent://message-platform/client-metrics/monitor-metrics / metrics-report] Distributing 0 messages to 4 consumers
```
This will cause do not continue reading more entries

https://github.com/apache/pulsar/blob/master/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/persistent/PersistentDispatcherMultipleConsumers.java#L460

Because readMoreEntries is called at the end of sendMessagesToConsumers, if the consumer no longer sends a permit request because it has not received any messages, dispatcher will stop dispatch any messages.

### Modifications

If read 0 entries from bookie, trigger readMoreEntries

### Motivation

According to https://issues.apache.org/jira/browse/ZOOKEEPER-3056 we need to pass a flag to ZK to make sure we can start it without a snapshot file, as part of the upgrade from zk-3.4.x. 

From Pulsar 2.6 on, we'll then be able to remove this flag.

## Motivation
when ledgers.ceilingKey return a NULL value, we need to process the ledgerId is NUll case.

## Modifications
change long ledgerId to Long ledgerId and check if ledgerId is NULL.

Use BOOKIE_MEM and BOOKIE_GC if they have already been set. This removes the need to overwrite the bkenv.sh config file and this is useful when running in a container environment.

Fixes https://github.com/apache/pulsar/issues/5832

Master Issue: https://github.com/apache/pulsar/issues/5832

### Motivation

Currently, in debezium, for delete events, in order to be compatible with Kafka's compaction, each delete event will be followed by a record with a null value. For records with a null value, the null value is not currently processed in pulsar, resulting in an exception of throwing a null pointer. Now, we considered to ignore the null value or use it in combination with pulsar's compaction feature in the future.

![image](https://user-images.githubusercontent.com/1907867/70976050-21cef680-20e6-11ea-963f-56471c915c98.png)
https://debezium.io/documentation/reference/0.10/connectors/mysql.html

### Modifications

* Ignore the record of the null value.
* Add more integrations, such as insert, delete, update.

Fixes #5612 


### Motivation
for MongoSink 

### Modifications
 replace ”com.mongodb.async.client” package with ”com.mongodb.reactivestreams.client.”

### Motivation
Pulsar doesn't support deploy on IPv6 network environment.
This PR just makes an effort to make it.
The error happens if the client connect to brokers by IPv6 address, like fec0:0:0:ffff::1.
- Wrong format: fec0:0:0:ffff::1:6650
- Correct format: [fec0:0:0:ffff::1]:6650

Cause the split regex is ':', brackets are needed and the ip:port can't split by ':' directly.

### Modifications

validateHostName in ServiceURI

### Motivation

Currently, the commands on the page http://pulsar.apache.org/docs/en/pulsar-admin/ are all manually added. when the document changes, there are often wrong contents. in order to solve this problem, we expect the commands on this page to be automatically generated. this pr is the first step. If it passes, we will add a new page on the website to show the automatically generated commands later.

Fixes #5726 

### Motivation

When running pulsar-io connectors and functions from the Intellij IDE some actions fail
due to uncaught class-not-found throwables.
The expectation being that the class is being dynamically loaded and only the ClassNotFoundException will occur if the class is not found.
When the function is created or run with https://pulsar.apache.org/docs/en/functions-deploying/#local-run-mode this is indeed the case.
When running under the control https://pulsar.apache.org/docs/en/functions-debug/#debug-with-localrun-mode as a gradle plugin the class may already be known and throw a NoClassDefFoundError.
It seems to me that any time ClassNotFoundException is handled then NoClassDefFoundError should also be caught.

### Modifications

Wherever there was a `catch (ClassNotFoundException ` I replaced it with 
`catch (ClassNotFoundException | NoClassDefFoundError ` .
There were multiple cases where the ClassNotFoundException were handled e.g. the jar loader failed so the nar loader was used to handle the jar loader's failure.

fix some typos in the comments

Fixes #5857 

### Motivation

With current aproach for specifying storage class in persistent volume claim it's not possible to customize the provisioner parameters. If the property 'storageClass' is declared the chart always create a new storage class with hardcoded parameters.

### Modifications

A property 'storageClassName' was added to support an existent storage class.

### Verifying this change

This change is a trivial rework / code cleanup without any test coverage.

Fixes #5825

### Motivation

PulsarClusterMetadataSetup runs on the second time will failed because some zk nodes already exists. This change will fix this issue.

### Modifications

- catch and ignore exception for exists zk nodes.
- add test for it.

### Motivation
Pulsar Summit CFP is open, we need a blog to announce it.

### Modifications
Add a blog to announce Pulsar Summit CFP.

* Update a format issue

* Update the link issue