Update logic for picking active consumer for failover subscription on non-partitioned topic. (#4604)

Instead of sorting the consumers based on priority level and consumer name then pick a active consumer, which could cause subscription getting into a flaky state, where the "active" consumer joins and leaves, no consumer is actually elected as "active" and consuming the messages.
Fix logic to always pick the first consumer in the consumer list without sorting consumers. So consumers will be picked as acive consumer based on the order of their subscription.

(cherry picked from commit 22f1d7d1)

Fixes #4909
### Motivation

It would be good to expose method `getLastMessageId` in `ConsumerImpl` to a public method. 
eg. some times user would like to know the lag messages; or only consume messages before current time.

### Modifications

- expose method `getLastMessageId` in consumer api.
- add unit test.

### Verifying this change
Ut passed

(cherry picked from commit 93d95c74)

Signed-off-by: Nxiaolong.ran <ranxiaolong716@gmail.com>

* Data is not deleted after expiration due to connected readers

*Problem*

A problem is observed when stress testing pulsar using [pulsar-flink](https://github.com/streamnative/pulsar-flink) -
No matter what TTL or retention setting is used, the data is never cleaned up. So the stress test ends up failing due
to disk filled up.

The root cause of the problem is described as below.

when a reader is opened using `MessageId.earliest`, a non-durable cursor with position (-1, -2) is added to the cursor heap.
The position `(-1, -2)` in the heap is never updated because non-durable cursors are never advanced when mark-deletions
happen. So the slowest cursor position is always `(-1, -2)`, thus causing no ledger can be deleted even they are expired
or over quota.

*Motivation*

Fix the problem to make sure Pulsar honor to TTL and retention settings.

*Modifications*

- Fix the `startPosition` when PersistentTopic opens a non-durable cursor on `MessageId.earliest`.
  So the `startPosition` is (-1, -1) not (-1, -2).

- Fix the `NonDurableCursorImpl` constructor to check if the position in the ledger of `MessageId.earliest`.
  If the provided position is in the `earliest` ledger, the mark-deleted position will be set to the previous
  position of first position.

- Fix the `NonDurableCursorImpl` to advance ledger cursor when mark-deletion happens on a non-durable cursor.

*Verify this change*

Unit tests are coming.

(cherry picked from commit 3e7cb68b)

Signed-off-by: Nxiaolong.ran <ranxiaolong716@gmail.com>

Fixes: #2719 

### Motivation

In website, the go client docs missing.

### Modifications

Add missing docs for go client.

(cherry picked from commit 006f291b)

(cherry picked from commit cfef0fb4)

(cherry picked from commit b7b9926b)

* support mongodb connector

* fix typo

* revert

(cherry picked from commit 96a94188)

Signed-off-by: Nxiaolong.ran <ranxiaolong716@gmail.com>
(cherry picked from commit 5c771d20)

* Fix docs about reset cursor
Signed-off-by: Nxiaolong.ran <ranxiaolong716@gmail.com>

* fix comments
Signed-off-by: Nxiaolong.ran <ranxiaolong716@gmail.com>
(cherry picked from commit 80e4b5f6)

### Motivation

- The connectorsManager is initialized twice in WorkerService, it seems to be a misoperation when merging codes. 

### Modifications

- Delete the redundant codes.

(cherry picked from commit 7275d53a)

* Add subscribe position param for consumer of sink
Signed-off-by: Nxiaolong.ran <ranxiaolong716@gmail.com>

(cherry picked from commit 39af4777)

* fix: Functions unnecessarily restart during FunctionRuntimeManager init phase

(cherry picked from commit dd58baf8)

* Fix list non-persistent topics shows the persistent topics
---

Fixes #5414

*Motivation*

When using the REST API to request to list all the non-persistent
topics, it will show the persistent topics.

*Modifications*

- Add a filter when before sending the response

(cherry picked from commit 79f47106)

### Motivation

While this doesn't appear to affect the correctness, we should just return empty after we trigger the callback.

(cherry picked from commit 734e7484)

There is a race condition when replaying messages on the `PersistentDispatcherMultipleConsumers`. This is apparently only happening in the context of delivering messages after a scheduled delay. 

The `Dispatcher.readMoreEntries()` can get invoked in multiple occasions: 
 * Piggybacking on the completion of the previous read operation
 * When the delaye tracker gets triggered

If there is already a replay read operation pending, we're currently ignoring the message ids returned by the delay tracker. These messages will not be retried on any other occasion, leading to backlog building up (until a consumer disconnects/reconnects).

(cherry picked from commit 01ca24b3)

* Trim deleted entries after recover cursor.

* Fix errors

* Add managed cursor unit tests.

* Fix tests and handle cursor reset.

* fix unit tests

* Fix tests

* Fix check style

(cherry picked from commit dc7d01ef)

Efficiency improvements for delay delivery tracker

(cherry picked from commit 467ffab1)

There are some link issues on Pulsar website, check and fix them.

(cherry picked from commit f538c522)

### Motivation

The code snippets in functions documentation are wrong: `Function` class of java native is from package `java.util.function` instead of `java.util`.

### Modifications

in 2 markdown file, corrected 2 java source snippets.

(cherry picked from commit bdfefc72)

* add thread/process modes

* update the differences between thread and process modes

* update

* update

(cherry picked from commit 8c163eb2)

Fixes #5484

(cherry picked from commit 6df7cec6)

Fixes #5365

(cherry picked from commit 49a8658c)

* [Issue 5309][docs] Clarify wording around retention policies

Fixes #5309

Docs are confusing.

Replaced retention wording to make distinction between acked and
non-acked messages clearer.

* [Issue 5309][docs] Remove TODO

(cherry picked from commit 418362e1)

- Make sure terminology is precise
- Clarify that each subscription has a backlog
- Clarify how retention interacts with readers

(cherry picked from commit a4c14cdf)

related issue #5349

(cherry picked from commit 09360682)

Some of the ManagedLedger methods, such as hasMoreEntries/getPreviousPosition/getNextValidPosition, are protected. This change try to make them public.
These methods are useful when user want to do some advanced operation in Data Processing.

(cherry picked from commit 89e3f30d)

Fix broken custom auth-provider that uses authenticationData

(cherry picked from commit ebaf97cc)

(cherry picked from commit ddf5429e)

(cherry picked from commit 0c7319fc)

* [pulsar-client] Fix message corruption on OOM for batch messages

* remove comments

* Address comments: index in local-var + remove lastSerializedMessageIndex var

(cherry picked from commit bf9a9019)

(cherry picked from commit b466b085)

* [pulsar-perf] record message failure and avoid exiting from process on publish failure

* add flag to exit on publish failure (default: disable and continue)

(cherry picked from commit c59a37b2)

*Motivation*

Allow spinning up multiple threads for running the perf-producer.

*Modifications*

- Introduce a new option for configuring multiple threads
- Each thread has one separate pulsar client

(cherry picked from commit 0b1e3b37)

Add support for partitioned topic consumer seek by time.

Call each partition consumer seekAsync() while call partitioned consumer seekAsync()

Update unit tests for consumer.seek().

(cherry picked from commit a95bea60)

(cherry picked from commit 8a022064)

This reverts commit d49e61e0.

(cherry picked from commit c4035d3d)

* Fix bug that namespace policies does not take effect due to NPE

* Prevent NPE if Dispatcher and DispatchRateLimiter return to null

(cherry picked from commit 84a519fa)

* [pulsar-broker] Fix: race condition : Failed to read-more entries on dispatcher

* clean up non-used method

(cherry picked from commit 13ea25a3)