C# Class KafkaNet.BrokerRouter

This class provides an abstraction from querying multiple Kafka servers for Metadata details and caching this data. All metadata queries are cached lazily. If metadata from a topic does not exist in cache it will be queried for using the default brokers provided in the constructor. Each Uri will be queried to get metadata information in turn until a response is received. It is recommended therefore to provide more than one Kafka Uri as this API will be able to to get metadata information even if one of the Kafka servers goes down. The metadata will stay in cache until an error condition is received indicating the metadata is out of data. This error can be in the form of a socket disconnect or an error code from a response indicating a broker no longer hosts a partition.

Inheritance: IBrokerRouter

Show file Open project: gigya/KafkaNetClient Class Usage Examples

Public Methods

Method	Description
BrokerRouter ( KafkaOptions kafkaOptions ) : KafkaNet.Common
Dispose ( ) : void
GetAllTopicMetadataFromLocalCache ( ) : List	Returns Topic metadata for each topic. The topic metadata will by default check the cache. To force querying the metadata from the server use RefreshAllTopicMetadata
GetTopicMetadataFromLocalCache ( ) : List	Returns Topic metadata for each topic requested. The topic metadata will by default check the cache first and then if it does not exist it will then request metadata from the server. To force querying the metadata from the server use RefreshTopicMetadata
GetTopicMetadataRefreshTime ( string topic ) : System.DateTime
RefreshAllTopicMetadata ( ) : Task	Force a call to the kafka servers to refresh metadata for all topics. This method will ignore the cache and initiate a call to the kafka servers for all topics, updating the cache with the resulting metadata. Only call this method to force a metadata update. For all other queries use GetAllTopicMetadataFromLocalCache which uses cached values.
RefreshMissingTopicMetadata ( ) : Task
RefreshTopicMetadata ( ) : Task	Force a call to the kafka servers to refresh metadata for the given topics. This method will ignore the cache and initiate a call to the kafka servers for all given topics, updating the cache with the resulting metadata. Only call this method to force a metadata update. For all other queries use GetTopicMetadataFromLocalCache which uses cached values.
SelectBrokerRouteFromLocalCache ( string topic, byte key = null ) : KafkaNet.BrokerRoute	Select a broker for a given topic using the IPartitionSelector function.
SelectBrokerRouteFromLocalCache ( string topic, int partitionId ) : KafkaNet.BrokerRoute	Select a broker for a specific topic and partitionId. This function does not use any selector criteria. If the given partitionId does not exist an exception will be thrown.

Private Methods

Method	Description
GetCachedRoute ( string topic, KafkaNet.Protocol.Partition partition ) : KafkaNet.BrokerRoute
GetCachedTopic ( string topic, System.TimeSpan expiration = null ) : KafkaNet.Protocol.Topic
GetConnections ( ) : IKafkaConnection[]
RefreshAllTopicMetadata ( System.TimeSpan timeout ) : Task	Force refresh each topic metadata that exists on kafka servers.
RefreshTopicMetadata ( System.TimeSpan cacheExpiration, System.TimeSpan timeout ) : Task	Refresh metadata Request will try to refresh only the topics that were expired in the cache. If cacheExpiration is null: refresh metadata Request will try to refresh only topics that are not in the cache.
SearchCacheForTopics ( IEnumerable topics, System.TimeSpan expiration ) : TopicSearchResult
TryGetRouteFromCache ( string topic, KafkaNet.Protocol.Partition partition ) : KafkaNet.BrokerRoute
UpdateInternalMetadataCache ( MetadataResponse metadata ) : void
UpsertConnectionToBrokerConnectionIndex ( int brokerId, KafkaNet.Model.KafkaEndpoint brokerEndpoint, IKafkaConnection>.Func connectionFactory ) : void

Method Details

BrokerRouter() public method

None of the provided Kafka servers are resolvable.

public BrokerRouter ( KafkaOptions kafkaOptions ) : KafkaNet.Common
kafkaOptions	KafkaNet.Model.KafkaOptions
return	KafkaNet.Common

Dispose() public method

public Dispose ( ) : void
return	void

GetAllTopicMetadataFromLocalCache() public method

Returns Topic metadata for each topic.

The topic metadata will by default check the cache. To force querying the metadata from the server use RefreshAllTopicMetadata

public GetAllTopicMetadataFromLocalCache ( ) : List
return	List

GetTopicMetadataFromLocalCache() public method

Returns Topic metadata for each topic requested.

The topic metadata will by default check the cache first and then if it does not exist it will then request metadata from the server. To force querying the metadata from the server use RefreshTopicMetadata

Thrown if the topic metadata does not exist in the cache.

public GetTopicMetadataFromLocalCache ( ) : List
return	List

GetTopicMetadataRefreshTime() public method

public GetTopicMetadataRefreshTime ( string topic ) : System.DateTime
topic	string
return	System.DateTime

RefreshAllTopicMetadata() public method

Force a call to the kafka servers to refresh metadata for all topics.

This method will ignore the cache and initiate a call to the kafka servers for all topics, updating the cache with the resulting metadata. Only call this method to force a metadata update. For all other queries use GetAllTopicMetadataFromLocalCache which uses cached values.

public RefreshAllTopicMetadata ( ) : Task
return	Task

RefreshMissingTopicMetadata() public method

public RefreshMissingTopicMetadata ( ) : Task
return	Task

RefreshTopicMetadata() public method

Force a call to the kafka servers to refresh metadata for the given topics.

This method will ignore the cache and initiate a call to the kafka servers for all given topics, updating the cache with the resulting metadata. Only call this method to force a metadata update. For all other queries use GetTopicMetadataFromLocalCache which uses cached values.

public RefreshTopicMetadata ( ) : Task
return	Task

SelectBrokerRouteFromLocalCache() public method

Select a broker for a given topic using the IPartitionSelector function.

Thrown if the topic metadata does not exist in the cache.

public SelectBrokerRouteFromLocalCache ( string topic, byte key = null ) : KafkaNet.BrokerRoute
topic	string	The topic to retreive a broker route for.
key	byte	The key used by the IPartitionSelector to collate to a consistent partition. Null value means key will be ignored in selection process.
return	KafkaNet.BrokerRoute

SelectBrokerRouteFromLocalCache() public method

Select a broker for a specific topic and partitionId.

This function does not use any selector criteria. If the given partitionId does not exist an exception will be thrown.

Thrown if the give partitionId does not exist for the given topic. Thrown if the topic metadata does not exist in the cache.

public SelectBrokerRouteFromLocalCache ( string topic, int partitionId ) : KafkaNet.BrokerRoute
topic	string	The topic name to select a broker for.
partitionId	int	The exact partition to select a broker for.
return	KafkaNet.BrokerRoute