- 19 11月, 2019 40 次提交
-
-
由 Matteo Merli 提交于
There is a race condition when replaying messages on the `PersistentDispatcherMultipleConsumers`. This is apparently only happening in the context of delivering messages after a scheduled delay. The `Dispatcher.readMoreEntries()` can get invoked in multiple occasions: * Piggybacking on the completion of the previous read operation * When the delaye tracker gets triggered If there is already a replay read operation pending, we're currently ignoring the message ids returned by the delay tracker. These messages will not be retried on any other occasion, leading to backlog building up (until a consumer disconnects/reconnects). (cherry picked from commit 01ca24b3)
-
由 Matteo Merli 提交于
Efficiency improvements for delay delivery tracker (cherry picked from commit 467ffab1)
-
由 Jennifer Huang 提交于
There are some link issues on Pulsar website, check and fix them. (cherry picked from commit f538c522)
-
由 Jennifer Huang 提交于
* add thread/process modes * update the differences between thread and process modes * update * update (cherry picked from commit 8c163eb2)
-
由 Brian Candler 提交于
Fixes #5484 (cherry picked from commit 6df7cec6)
-
由 Brian Candler 提交于
Fixes #5365 (cherry picked from commit 49a8658c)
-
由 Raman Gupta 提交于
* [Issue 5309][docs] Clarify wording around retention policies Fixes #5309 Docs are confusing. Replaced retention wording to make distinction between acked and non-acked messages clearer. * [Issue 5309][docs] Remove TODO (cherry picked from commit 418362e1)
-
由 Rajan Dhabalia 提交于
Fix broken custom auth-provider that uses authenticationData (cherry picked from commit ebaf97cc)
-
由 Matteo Merli 提交于
(cherry picked from commit ddf5429e)
-
由 Rajan Dhabalia 提交于
(cherry picked from commit 0c7319fc)
-
由 Rajan Dhabalia 提交于
* [pulsar-client] Fix message corruption on OOM for batch messages * remove comments * Address comments: index in local-var + remove lastSerializedMessageIndex var (cherry picked from commit bf9a9019)
-
由 Rajan Dhabalia 提交于
(cherry picked from commit b466b085)
-
由 Rajan Dhabalia 提交于
* [pulsar-perf] record message failure and avoid exiting from process on publish failure * add flag to exit on publish failure (default: disable and continue) (cherry picked from commit c59a37b2)
-
由 Boyang Jerry Peng 提交于
(cherry picked from commit 8a022064)
-
由 Boyang Jerry Peng 提交于
This reverts commit d49e61e0. (cherry picked from commit c4035d3d)
-
由 Masahiro Sakamoto 提交于
* Fix bug that namespace policies does not take effect due to NPE * Prevent NPE if Dispatcher and DispatchRateLimiter return to null (cherry picked from commit 84a519fa)
-
由 Rajan Dhabalia 提交于
* [pulsar-broker] Fix: race condition : Failed to read-more entries on dispatcher * clean up non-used method (cherry picked from commit 13ea25a3)
-
由 Boyang Jerry Peng 提交于
* fix potential deadlock that can occur in addConsumer * add fix to PersistentDispatcherSingleActiveConsumer * fixing tests (cherry picked from commit 590b0684)
-
由 Addison Higham 提交于
Fixes #5360 This adds a small cache of the request body to ensure that it can be re-sent. TODO: still needs tested (cherry picked from commit 978efaf1)
-
由 Matteo Merli 提交于
* Set default ensemble size to 2 in service conf, to match broker.conf * Fixed test (cherry picked from commit be5a7cf9)
-
由 Ivan Kelly 提交于
There's a bug in how user metadata is attached to a block that if the user doesn't specify both the region and the endpoint, offloading will throw an exception, as you can't add a null value to an immutable map. This change elides null to the empty string in these cases, so that offloading can continue. (cherry picked from commit 3e70d365)
-
由 Masahiro Sakamoto 提交于
### Motivation If cursor is not durable, the subscription is closed and removed from the topic when all consumers are removed. https://github.com/apache/pulsar/blob/413ba03050036a620fa346456ef6c3ff6071e9ab/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/persistent/PersistentSubscription.java#L237-L254 As mentioned in #5288, the dispatcher also needs to be closed at this time. Otherwise, `RateLimiter` instances will not be garbage collected, causing a memory leak. (cherry picked from commit 136048c0)
-
由 Boyang Jerry Peng 提交于
* fix instability with tumbling window test * cleaning up * cleaning up (cherry picked from commit 094ebf7a)
-
由 Boyang Jerry Peng 提交于
* Bug in Message Deduplication that may cause incorrect behavior * add tests * fix error message * fix client backoff * fix tests * cleaning up * Fix handling of BK write failures for message dedup * tests and clean up * refactoring code * fixing bugs * addressing comments * add missing license header * Improve error handling of BK write failures * fixing tests * fixing bugs * cleaning up * addressing comments * fixing tests (cherry picked from commit 60abcaa9)
-
由 Rajan Dhabalia 提交于
(cherry picked from commit 87aa78ba)
-
由 Matteo Merli 提交于
* Allow to specify delivery delay in C++ client * Fixed symbols visibility to make it available for unit tests * Fixed format (cherry picked from commit 0984f42f)
-
由 Matteo Merli 提交于
* [Cpp] Fixed negative ack tracker constructor sequence * Fixed formatting * Fixed default for neg ack delay on partitioned topics * Fixed format (cherry picked from commit 6f113f73)
-
由 Easyfan Zheng 提交于
Merge Request for #4809: provide a convenient method for C++ client producer batch container (#4885) provide a convenient method for C++ client producer batch container https://github.com/apache/pulsar/issues/4809 Formally the container element must be used like the following way: `MessageContainerList::iterator iter = messagesContainerListPtr->begin();` `iter->sendCallback_(r, iter->message_);` There are totally 3 reference operation steps inside: 1. Reference an exact element of MessageContainer explicitly from outside by iteration operator; 2. Reference the `sendCallback_` function pointer of said MessageContainer element explicitly from outside; 3. Reference the `message_` member of said MessageContainer element explicitly from outside; Besides, there is only one incoming variable say `Result r` is given from outside; In an ideal design, a user of `MessageContainer` should not have to know exactly the existence of `MessageContainer::sendCallback_` or `MessageContainer::message_`, what they exactly are, either how to use them. Organize those stuff in a right way, should be the responsibility of struct `MessageContainer` itself. So a reasonable convenient invoking method should be like: `MessageContainerList::iterator iter = messagesContainerListPtr->begin();` `iter->callBack(r);` And said `MessageContainer::callBack` function shall be implemented like below: `void callBack(const pulsar::Result& r) { sendCallback_(r, message_); }` Obviously, said convenient method is also an efficient one. Moreover, use a more efficient iteration method while going through the MessageContainerList; From some benchmark test result in my local environment, such "for iteration based a fixed-length", will be 5 times faster than the STL::iterator operator way. Refer to the change on BatchMessageContainer.cc, please. (cherry picked from commit 8058775c)
-
由 Masahiro Sakamoto 提交于
Master Issue: #5234 ### Motivation The other day, I fixed a memory leak caused by not being executed the destructor of C++ producer in #5246. However, when running a producer application written in Go in an environment with the modified C++ client library installed, the program occasionally crashes due to a "bad_weak_ptr" error. ``` 2019/10/01 16:34:30.210 c_client.go:68: [info] INFO | ProducerImpl:481 | [persistent://massakam/global/test/t1, dc1-904-1012912] Closing producer for topic persistent://massakam/global/test/t1 2019/10/01 16:34:30.211 c_client.go:68: [info] INFO | ProducerImpl:463 | Producer - [persistent://massakam/global/test/t1, dc1-904-1012912] , [batchMessageContainer = { BatchContainer [size = 0] [batchSizeInBytes_ = 0] [maxAllowedMessageBatchSizeInBytes_ = 131072] [maxAllowedNumMessagesInBatch_ = 1000] [topicName = persistent://massakam/global/test/t1] [producerName_ = dc1-904-1012912] [batchSizeInBytes_ = 0] [numberOfBatchesSent = 1] [averageBatchSize = 1]}] 2019/10/01 16:34:30.211 c_client.go:68: [info] INFO | BatchMessageContainer:171 | [numberOfBatchesSent = 1] [averageBatchSize = 1] terminate called after throwing an instance of 'std::bad_weak_ptr' what(): 2019/10/01 16:34:30.211 c_client.go:68: [info] INFO | ProducerImpl:463 | Producer - [persistent://massakam/global/test/t1, dc1-904-1012911] , [batchMessageContainer = { BatchContainer [size = 0] [batchSizeInBytes_ = 0] [maxAllowedMessageBatchSizeInBytes_ = 131072] [maxAllowedNumMessagesInBatch_ = 1000] [topicName = persistent://massakam/global/test/t1] [producerName_ = dc1-904-1012911] [batchSizeInBytes_ = 0] [numberOfBatchesSent = 1] [averageBatchSize = 1]}] bad_weak_ptr 2019/10/01 16:34:30.211 c_client.go:68: [info] INFO | BatchMessageContainer:171 | [numberOfBatchesSent = 1] [averageBatchSize = 1] 2019/10/01 16:34:30.211 c_client.go:68: [info] INFO | ProducerImpl:463 | Producer - [persistent://massakam/global/test/t1, dc1-904-1012910] , [batchMessageContainer = { BatchContainer [size = 0] [batchSizeInBytes_ = 0] [maxAllowedMessageBatchSizeInBytes_ = 131072] [maxAllowedNumMessagesInBatch_ = 1000] [topicName = persistent://massakam/global/test/t1] [producerName_ = dc1-904-1012910] [batchSizeInBytes_ = 0] [numberOfBatchesSent = 1] [averageBatchSize = 1]}] SIGABRT: abort PC=0x7fc78d39d2c7 m=0 sigcode=18446744073709551610 ``` As a result of the investigation, I found that the destructor was called before the process of closing `ProducerImpl` was completed, and the object was destroyed. ### Modifications To keep the `ProducerImpl` object alive, get its own shared pointer at the beginning of `ProducerImpl::closeAsync()`. And the pointer must be passed to `ProducerImpl::handleClose()`. Otherwise, the object will be destroyed before `handleClose()` is called. So far, this issue has not been reproduced in `ConsumerImpl`, but I fixed it in the same way as `ProducerImpl`. (cherry picked from commit dbd48ab1)
-