fix #4707

(cherry picked from commit 0362944f)

Merge Request for #4808: TYPO in C++ client producer method for processing failure case, and add corresponding unit test case. (#4873)

Definitely, this is a typo. This method is dealing with the Failed Message with the GIVEN result, but not a CERTAIN result.

Contribution Checklist
#4808 : TYPO in C++ client producer method for processing failure case
Add c++ client producer failure message unit test case.

UT passed:

BatchMessageTest
(cherry picked from commit b90b4ea1)

(cherry picked from commit 40d6248e)

* Use classloading to load use code for functions
(cherry picked from commit 6ff1bbae)

Adding back Prometheus TYPE definitions; fixed duplicate TYPE errors; fixed format issue in metricWithRemoteCluster; added test for Prometheus types (#4183)

Fixes #3112

In this pull request, I set out to fix #3112, which is caused by duplicate TYPE statements in the metrics output which leads to parsing of Prometheus metrics to fail in recent versions of Prometheus. Because of this, Prometheus will report the broker target as down.

Since I started looking at this, the type definitions have been removed (#4136) from topics metric output. I think these types are useful in Prometheus and have added them back in.

While testing this fix in my geo-replicated setup, I found a format in error (missing quote and comma) in the TopicStats.metricWithRemoteCluster method. This pull request includes a fix for that issue.

I have also added a new test to PrometheusMetricsTest.java that fails without these changes but passed with them.

I added a static HashMap to TopicStats to keep track of the TYPEs that have been output. All writing of the TYPE for topics and namespaces is done with the TopicStats.metricType method. I modified that method to update the HashMap and only print the TYPE out for the first occurrence of the metric name.  I also added a method reset the HashMap, which gets called in NamespaceStatsAggregator.generate.

This change added tests and can be verified as follows:
  - Added testDuplicateMetricTypeDefinitions which checks for:
       - duplicate TYPE definitions in the Prometheus output
       - validates that no TYPE definition appears after the first metric sample
       - ensures that all metrics have a defined type

I execute the test twice to make sure the resetting of the HashMap of the already seen metric type definitions works correctly. This test passes for me reliably (both occurrences).

I have confirmed using promtool that the metrics output will now parse without error using versions 2.7.1 and 2.9.2 (which is the latest). There are many warnings around missing HELP definitions and metrics using reserved suffixes (ex. _count), but no errors.

In addition, I have patched 2.3.1 with this fix and am currently running it in my cluster. Prometheus (2.7.1) successfully parses the metrics and I am able to see namespace and topic-level metrics.

(cherry picked from commit 8d32b588)

### Motivation

IN PR #4883, we support 1 partitioned topic producer/consumer in java client. this is for Cpp client support.

### Modifications
- change cpp client
- add unit test

### Verifying this change
New added Ut Passed

(cherry picked from commit bf948909)

Fixes https://github.com/apache/pulsar/issues/4957

Master Issue: https://github.com/apache/pulsar/issues/4957

### Motivation

Currently, when packaging the docker image, the dependency less is lost, resulting in the following exception being thrown when presto SQL is running
```
Cannot run program "less": error=2, No such file or directory
```

### Modifications

* Add dependency less

(cherry picked from commit dd5b4d08)

(cherry picked from commit ee158d9f)

Need to reinitialize certain components for externally managed runtimes when moving functions (#5007)

(cherry picked from commit 27a9f628)

Fixes #4720
### Motivation

for now, the snappy compressor can not be compiled successfully,
which raise undeclared identifier and no matching function errors.

### Modifications

fix compile errors, use snappy c++ apis.
(cherry picked from commit 385a0368)

Fixes https://github.com/apache/pulsar/issues/4956

Master Issue: https://github.com/apache/pulsar/issues/4956

### Motivation

In the latest version of pyyaml, using load will issue a warning and explain some reasons here https://github.com/yaml/pyyaml/wiki/PyYAML-yaml.load(input)-Deprecation.

### Modifications

* Use yaml.FullLoader as default loader

(cherry picked from commit d49e61e0)

*Motivation*

The getSchema logic in Pulsar proxy handler doesn't handle the case that
the request doesn't have schema version.

*Modification*

- Fix the logic to handle the case that a GetSchema request doesn't have schema version.
- Forward the GetSchemaResponse back to the client

*Tests*

The GetSchema tests in ProxyTest was disabled by mistake. Turn it on.
(cherry picked from commit 676fd71a)

(cherry picked from commit 41bc407e)

(cherry picked from commit 33441ea0)

Fixes #4969

### Motivation

close `FileOutputStream` after `WorkerUtils.downloadFromBookkeeper`
(cherry picked from commit 2c215001)

### Motivation
StructSchema LoadingCache for cache reader, but key is byte[], it will compare with the address's  hashcode. So every decode will generate a new reader so we should change the type of the key for LoadingCache

(cherry picked from commit 0367f5f0)

*Motivation*

Currently KeyValue schema doesn't support using AUTO_CONSUME.

This PR is to add this support.

This PR is based on #4836

*Changes*

- refactor a bit on Schema interface to support fetching schema info for both AutoConsumeSchema and KeyValueSchema before subscribing
- add AUTO_CONSUME support to KeyValueSchema
- add tests
(cherry picked from commit dd7cc890)

This commit adds a new option optionally disable authentication for the
`/metrics` endpoint in the pulsar-proxy.

Currently, authentication is required for the metrics endpoint when
authentication is enabled, which makes monitoring more difficult.
However, rather than just disable it completely and allow for metrics to
be exposed to any unknown user, this makes it opt in.

It could be argued that it should default to false, but as it is likely
that the proxy is the only component potentially exposed to the public internet, we
default to not exposing data.

Fixes #4920
(cherry picked from commit be7b24f9)

(cherry picked from commit 02f9fd3e)

If you want to specify a no_proxy variable in your dockerfile or docker-compose.yml, then this little fix respects the variable in get-requests. Otherwise the proxies could break your docker namespace lookup.
(cherry picked from commit 648f0c82)

* Handle get schema 404 in pulsar sql(table meta and get splits)

* Fix unit test.

* add defaultSchema()

* use Schema.BYTES.getSchemaInfo()

* add unit test

* rebase and fix unit tests

(cherry picked from commit 2069f761)

Fixes #4785

### Motivation

1. Stop return partition name in table list, just return the partitioned topic name in table list. This will avoid huge tables while user create large number of partition.
2. Make partition as internal column, provide users with the ability to get which partition data in and filtration based on partition. For example:
```
SELECT * FROM "my-table" WHERE "__partition__" = 0;
SELECT * FROM "my-table" WHERE "__partition__" in (2,3);
SELECT * FROM "my-table" WHERE "__partition__" < 1;
```
### Modifications

1. Add "__partition__" internal column.
2. Add domain handle for "__partition__".

### Verifying this change

Added new unit test to verify this change

### Does this pull request potentially affect one of the following parts:

*If `yes` was chosen, please highlight the changes*

  - Dependencies (does it add or upgrade a dependency): (no)
  - The public API: (yes)
  - The schema: (no)
  - The default values of configurations: (no)
  - The wire protocol: ( no)
  - The rest endpoints: (no)
  - The admin cli options: (no)
  - Anything that affects deployment: (no)

### Documentation

  - Does this pull request introduce a new feature? (yes)

(cherry picked from commit 5adc522e)

The checksyle plugin was added to the pulsar sql module to enforce the defined style. All violations were fixed:

- Ordering of imports.
- Formatting of the code.
- Absent Javadoc comments.
- Other small issues.
(cherry picked from commit f6fee1c6)

(cherry picked from commit f88ea9df)

(cherry picked from commit 075f28b7)

### Motivation

Fix #4732

### Modifications

Add options to rewrite the namespace delimiter, disable by default

Enable rewrite namespace delimiter can work well with superset:
<img width="1279" alt="superset" src="https://user-images.githubusercontent.com/12592133/61385412-f0f35700-a8e4-11e9-87b2-a31b62128b58.png">

### Does this pull request potentially affect one of the following parts:

*If `yes` was chosen, please highlight the changes*

  - Dependencies (does it add or upgrade a dependency): (no)
  - The public API: (no)
  - The schema: (no)
  - The default values of configurations: (no)
  - The wire protocol: (no)
  - The rest endpoints: (no)
  - The admin cli options: (no)
  - Anything that affects deployment: (no)

### Documentation

  - Does this pull request introduce a new feature? (no)

(cherry picked from commit 6ddd51ff)

Motivation
In PR #4764, we allow to create partitioned topic with 1 partition, But in Pulsar Client, user still not able to do it.
This fix try to make sure user could create consumer/producer for 1 partitioned topic .

Modifications
- old and new added unit test passed.
(cherry picked from commit 128287e3)

### Motivation
Fix concurrent access of `uninitializedCursors` in `ManagedLedgerImpl.asyncOpenCursor`.

### Modifications
* Adds test to expose concurrent access of `uninitializedCursors` in `ManagedLedgerImpl.asyncOpenCursor`.
* Fixes concurrent access of `uninitializedCursors` in `ManagedLedgerImpl.asyncOpenCursor`.
(cherry picked from commit 5bf319e4)

### Motivation
Incorrect value is being used in Pulsar Helm template `autorecovery-deployment.yaml`

### Modifications
Proper variable name set.

### Verifying this change
Fixed variable name is already set in `values.yaml` and `values-mini.yaml`.
This change is a trivial rework / code cleanup without any test coverage.

### Documentation
None needed.

(cherry picked from commit 2b52ccb2)

Fixes #4852

Bug:

After create an empty ledger by sub/unsub and trim ledgers, the first position in managedLedger is ahead of the last when create a consumer without reopen the managed ledger.

Reason：

Last position will be update when opening managedLedger.
Because when creating an empty Ledger without reopening managedLedger, the last position is not be updated. Since the ledger pointed of the last position has been deleted, the first position point to the new ledger.

Fix:

The first position should fall back to last position.
(cherry picked from commit 46bfb2c2)

### Motivation

jdbc sink treat all record as INSERT before #4358 , now it requires an indispensable action property which seems to be a break change, and we can deal records without any action property as INSERT.

### Modifications

treat action not set as INSERT action like before.
(cherry picked from commit d8356d8c)

(cherry picked from commit 00ce7815)

the 0 size limit can't means "infinite" size quota
(cherry picked from commit 77f63709)

(cherry picked from commit 3c9136b2)

*Motivation*

Currently fetching schema information is done synchronously.
It is called in netty callback threads and will potentially block
async operations.

*Modifications*

Make most of the operations asynchronously in SchemaInfoProvider.
(cherry picked from commit 91c4254c)

fix #4755
Retention policy is used in Ledger GC and Topic GC.
Default retention policy is not uploaded to zookeeper, but it is getted from zookeeper when it is used.
So, if zookeeper doesn't have retention policy, we should load it from default config file.
In Ledger GC configuration, it's OK.
https://github.com/apache/pulsar/blob/075f28b71c8fd9259ce8e136dbb81c0629c3f271/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/BrokerService.java#L739-L742
But in Topic GC, do nothing.
(cherry picked from commit 1dcac8cc)

(cherry picked from commit abe27e63)

### Modifications

When submitting a function via REST interface, if user doesn't provide the function package or function runtime as part of function config, the function will get submitted and deployed to run but the typeClassName for the source will not be set causing a weird error in the function instance code
(cherry picked from commit e1547e12)

### Motivation

A message doesn't get automatically ACKed if there is the output topic is not set
(cherry picked from commit 19b883b7)

(cherry picked from commit 88c85447)