C# Class StopGuessing.Clients.DistributedBinomialLadderFilterClient

The client side of a distributed binomial ladder frequency filter. Binomial ladder filters are used to identify frequently-occuring elements in streams while minimizing the data revealed/stored about infrequently-occuring elements. This client is utilized both by applications using a distributed binomial ladder filter and is also by the servers implementing the server to communicate amongst themselves. For more information about the binomial ladder filter, search its name and "Microsoft Research" to find detailed publications/tech reports.

Inheritance: IBinomialLadderFilter

显示文件 Open project: Microsoft/StopGuessing Class Usage Examples

Public Properties

Property	Type	Description
MaxLadderHeight	int
MinimumCacheFreshnessRequired	System.TimeSpan
NumberOfShards	int
ShardToHostMapping	IDistributedResponsibilitySet

Protected Properties

Property	Type	Description
CacheOfElementsAtTopOfLadder	DateTime>.FixedSizeLruCache
ShardHashFunction	StopGuessing.EncryptionPrimitives.UniversalHashFunction

Public Methods

Method	Description
AssignRandomBit ( int valueToAssign, int shardNumber = null ) : void	Assign a random element with the binomial ladder filter to either 0 or 1.
DistributedBinomialLadderFilterClient ( int numberOfShards, int defaultHeightOfLadder, IDistributedResponsibilitySet shardToHostMapping, string configurationKey, System.TimeSpan mininmumCacheFreshnessRequired = null ) : System	Create a client for a distributed binomial ladder filter
GetHeightAsync ( string element, int heightOfLadderInRungs = null, System.TimeSpan timeout = null, CancellationToken cancellationToken = newCancellationToken() ) : Task	Get the height of an element on its binomial ladder. The height of an element is the number of the H array elements that are associated with it that have value 1.
GetRandomShardIndex ( ) : int	Get a random shard index from the set of shards that make up the array elements of the binomial ladder filter.
GetShardIndex ( string key ) : int	Get the shard index associated with an element.
StepAsync ( string key, int heightOfLadderInRungs = null, System.TimeSpan timeout = null, CancellationToken cancellationToken = newCancellationToken() ) : Task	The Step operation records the occurrence of an element by increasing the element's height on the binomial ladder. It does this by identify the shard of the filter array associated with the element, fetching the H elements within that shard associated with the element (the element's rungs), and setting one of the elements with value 0 (a rung above the element) to have value 1 (to put it below the element). It then clears two random elements from the entire array (not just the shard). If none of the H rungs associated with an element have value 0, Step will instead set two random elements from within the full array to have value 1.

Method Details

AssignRandomBit() public method

Assign a random element with the binomial ladder filter to either 0 or 1.

public AssignRandomBit ( int valueToAssign, int shardNumber = null ) : void
valueToAssign	int	The random element will be set to 1 if this parameter is nonzero, and to 0 otherwise.
shardNumber	int	Optioanlly set this optional value to identify a shard number within to select a random element. /// If not set, this method will choose a shard at random.
return	void

DistributedBinomialLadderFilterClient() public method

Create a client for a distributed binomial ladder filter

public DistributedBinomialLadderFilterClient ( int numberOfShards, int defaultHeightOfLadder, IDistributedResponsibilitySet shardToHostMapping, string configurationKey, System.TimeSpan mininmumCacheFreshnessRequired = null ) : System
numberOfShards	int	The number of shards that the bit array of the binomial ladder filter will be divided into. /// The greater the number of shards, the more evently it can be distributed. However, the number of shards should still /// be a few orders of magnitude smaller than the ladder height.
defaultHeightOfLadder	int	The default ladder height for elements on the ladder.
shardToHostMapping	IDistributedResponsibilitySet	An object that maps each shard number to the host responsible for that shard.
configurationKey	string	A key used to protect the hashing from algorithmic complexity attacks. /// This key should not be unique to the application using the filter and should not be known to any untrusted /// systems that might control which elements get sent to the filter. If an attacker could submit elements to the filter /// and knew this key, the attacker could arrange for all elements to go to the same shard and in so doing overload that shard.
mininmumCacheFreshnessRequired	System.TimeSpan	The maximum time that an element should be kept in the cache of elements at the top of their ladder. /// In other words, how long to bound the possible time that an element may still appear to be at the top of its ladder in the cache /// when it is no longer at the top of the ladder based on the filter array. Defaults to one minute.
return	System

GetHeightAsync() public method

Get the height of an element on its binomial ladder. The height of an element is the number of the H array elements that are associated with it that have value 1.

public GetHeightAsync ( string element, int heightOfLadderInRungs = null, System.TimeSpan timeout = null, CancellationToken cancellationToken = newCancellationToken() ) : Task
element	string	The element to be measured to determine its height on its binomial ladder.
heightOfLadderInRungs	int	>If set, use a binomial ladder of this height rather than the default height. /// (Must not be greater than the default height that the binmomial ladder was initialized with.
timeout	System.TimeSpan
cancellationToken	System.Threading.CancellationToken
return	Task

GetRandomShardIndex() public method

Get a random shard index from the set of shards that make up the array elements of the binomial ladder filter.

public GetRandomShardIndex ( ) : int
return	int

GetShardIndex() public method

Get the shard index associated with an element.

public GetShardIndex ( string key ) : int
key	string	The element to match to a shard index.
return	int

StepAsync() public method

The Step operation records the occurrence of an element by increasing the element's height on the binomial ladder. It does this by identify the shard of the filter array associated with the element, fetching the H elements within that shard associated with the element (the element's rungs), and setting one of the elements with value 0 (a rung above the element) to have value 1 (to put it below the element). It then clears two random elements from the entire array (not just the shard). If none of the H rungs associated with an element have value 0, Step will instead set two random elements from within the full array to have value 1.

public StepAsync ( string key, int heightOfLadderInRungs = null, System.TimeSpan timeout = null, CancellationToken cancellationToken = newCancellationToken() ) : Task
key	string	The element to take a step for.
heightOfLadderInRungs	int	If set, use a binomial ladder of this height rather than the default height. /// (Must not be greater than the default height that the binmomial ladder was initialized with.)
timeout	System.TimeSpan
cancellationToken	System.Threading.CancellationToken
return	Task

Property Details

CacheOfElementsAtTopOfLadder protected_oe property

A LRU cache of elements that are at the top of their ladder, used less to improve performance and more to prevent a single node from being overloaded if all requests target a single element.

protected FixedSizeLruCache CacheOfElementsAtTopOfLadder
return	DateTime>.FixedSizeLruCache

MaxLadderHeight public_oe property

The maximum (and default) height of elements' ladders.

public int MaxLadderHeight
return	int

MinimumCacheFreshnessRequired public_oe property

The maximum amount of time an element should allowed to be treated as at the top of its ladder due to caching when it may actually have fallen down the ladder since the value was cached.

public TimeSpan,System MinimumCacheFreshnessRequired
return	System.TimeSpan

NumberOfShards public_oe property

The number of shards that the filter array is evenly divided into.

public int NumberOfShards
return	int

ShardHashFunction protected_oe property

The hash function used to map elements to shards.

protected UniversalHashFunction,StopGuessing.EncryptionPrimitives ShardHashFunction
return	StopGuessing.EncryptionPrimitives.UniversalHashFunction

ShardToHostMapping public_oe property

Records for each shard that is (or at one time has been) stored on this server.

public IDistributedResponsibilitySet ShardToHostMapping
return	IDistributedResponsibilitySet