pulsar.sink.deliveryGuarantee |
none |
Enum |
Optional delivery guarantee when committing.
Possible values:- "exactly-once": Records are only delivered exactly-once also under failover scenarios. To build a complete exactly-once pipeline is required that the source and sink support exactly-once and are properly configured.
- "at-least-once": Records are ensured to be delivered but it may happen that the same record is delivered multiple times. Usually, this guarantee is faster than the exactly-once delivery.
- "none": Records are delivered on a best effort basis. It is often the fastest way to process records but it may happen that records are lost or duplicated.
|
pulsar.sink.enableMetrics |
true |
Boolean |
The metrics from Pulsar Producer are only exposed if you enable this option. You should set the pulsar.client.statsIntervalSeconds to a positive value if you enable this option. |
pulsar.sink.enableSchemaEvolution |
false |
Boolean |
If you enable this option and use PulsarSinkBuilder.setSerializationSchema(Schema) , we would produce and serialize the message by using Pulsar's Schema . |
pulsar.sink.maxRecommitTimes |
5 |
Integer |
The allowed transaction recommit times if we meet some retryable exception. This is used in Pulsar Transaction. |
pulsar.sink.messageKeyHash |
murmur-3-32-hash |
Enum |
The hash policy for routing message by calculating the hash code of message key.
Possible values:- "java-hash": This hash would use
String.hashCode() to calculate the message key string's hash code. - "murmur-3-32-hash": This hash would calculate message key's hash code by using Murmur3 algorithm.
|
pulsar.sink.topicMetadataRefreshInterval |
1800000 |
Long |
Auto update the topic metadata in a fixed interval (in ms). The default value is 30 minutes. |
pulsar.sink.transactionTimeoutMillis |
10800000 |
Long |
This option is used when the user require the DeliveryGuarantee.EXACTLY_ONCE semantic. We would use transaction for making sure the message could be write only once. |
pulsar.sink.validateSinkMessageBytes |
false |
Boolean |
Pulsar client can validate the raw message bytes with the latest topic schema. This can make sure your serialized messages bytes is valid for consumer. |