C# Class StopGuessing.Clients.DistributedBinomialLadderFilterClient

The client side of a distributed binomial ladder frequency filter. Binomial ladder filters are used to identify frequently-occuring elements in streams while minimizing the data revealed/stored about infrequently-occuring elements. This client is utilized both by applications using a distributed binomial ladder filter and is also by the servers implementing the server to communicate amongst themselves. For more information about the binomial ladder filter, search its name and "Microsoft Research" to find detailed publications/tech reports.
Inheritance: IBinomialLadderFilter
Show file Open project: Microsoft/StopGuessing Class Usage Examples

Public Properties

Property Type Description
MaxLadderHeight int
MinimumCacheFreshnessRequired System.TimeSpan
NumberOfShards int
ShardToHostMapping IDistributedResponsibilitySet

Protected Properties

Property Type Description
CacheOfElementsAtTopOfLadder DateTime>.FixedSizeLruCache
ShardHashFunction StopGuessing.EncryptionPrimitives.UniversalHashFunction

Public Methods

Method Description
AssignRandomBit ( int valueToAssign, int shardNumber = null ) : void

Assign a random element with the binomial ladder filter to either 0 or 1.
DistributedBinomialLadderFilterClient ( int numberOfShards, int defaultHeightOfLadder, IDistributedResponsibilitySet shardToHostMapping, string configurationKey, System.TimeSpan mininmumCacheFreshnessRequired = null ) : System

Create a client for a distributed binomial ladder filter
GetHeightAsync ( string element, int heightOfLadderInRungs = null, System.TimeSpan timeout = null, CancellationToken cancellationToken = newCancellationToken() ) : Task

Get the height of an element on its binomial ladder. The height of an element is the number of the H array elements that are associated with it that have value 1.
GetRandomShardIndex ( ) : int

Get a random shard index from the set of shards that make up the array elements of the binomial ladder filter.
GetShardIndex ( string key ) : int

Get the shard index associated with an element.
StepAsync ( string key, int heightOfLadderInRungs = null, System.TimeSpan timeout = null, CancellationToken cancellationToken = newCancellationToken() ) : Task

The Step operation records the occurrence of an element by increasing the element's height on the binomial ladder. It does this by identify the shard of the filter array associated with the element, fetching the H elements within that shard associated with the element (the element's rungs), and setting one of the elements with value 0 (a rung above the element) to have value 1 (to put it below the element). It then clears two random elements from the entire array (not just the shard). If none of the H rungs associated with an element have value 0, Step will instead set two random elements from within the full array to have value 1.

Method Details

AssignRandomBit() public method

Assign a random element with the binomial ladder filter to either 0 or 1.
public AssignRandomBit ( int valueToAssign, int shardNumber = null ) : void
valueToAssign int The random element will be set to 1 if this parameter is nonzero, and to 0 otherwise.
shardNumber int Optioanlly set this optional value to identify a shard number within to select a random element. /// If not set, this method will choose a shard at random.
return void

DistributedBinomialLadderFilterClient() public method

Create a client for a distributed binomial ladder filter
public DistributedBinomialLadderFilterClient ( int numberOfShards, int defaultHeightOfLadder, IDistributedResponsibilitySet shardToHostMapping, string configurationKey, System.TimeSpan mininmumCacheFreshnessRequired = null ) : System
numberOfShards int The number of shards that the bit array of the binomial ladder filter will be divided into. /// The greater the number of shards, the more evently it can be distributed. However, the number of shards should still /// be a few orders of magnitude smaller than the ladder height.
defaultHeightOfLadder int The default ladder height for elements on the ladder.
shardToHostMapping IDistributedResponsibilitySet An object that maps each shard number to the host responsible for that shard.
configurationKey string A key used to protect the hashing from algorithmic complexity attacks. /// This key should not be unique to the application using the filter and should not be known to any untrusted /// systems that might control which elements get sent to the filter. If an attacker could submit elements to the filter /// and knew this key, the attacker could arrange for all elements to go to the same shard and in so doing overload that shard.
mininmumCacheFreshnessRequired System.TimeSpan The maximum time that an element should be kept in the cache of elements at the top of their ladder. /// In other words, how long to bound the possible time that an element may still appear to be at the top of its ladder in the cache /// when it is no longer at the top of the ladder based on the filter array. Defaults to one minute.
return System

GetHeightAsync() public method

Get the height of an element on its binomial ladder. The height of an element is the number of the H array elements that are associated with it that have value 1.
public GetHeightAsync ( string element, int heightOfLadderInRungs = null, System.TimeSpan timeout = null, CancellationToken cancellationToken = newCancellationToken() ) : Task
element string The element to be measured to determine its height on its binomial ladder.
heightOfLadderInRungs int >If set, use a binomial ladder of this height rather than the default height. /// (Must not be greater than the default height that the binmomial ladder was initialized with.
timeout System.TimeSpan
cancellationToken System.Threading.CancellationToken
return Task

GetRandomShardIndex() public method

Get a random shard index from the set of shards that make up the array elements of the binomial ladder filter.
public GetRandomShardIndex ( ) : int
return int

GetShardIndex() public method

Get the shard index associated with an element.
public GetShardIndex ( string key ) : int
key string The element to match to a shard index.
return int

StepAsync() public method

The Step operation records the occurrence of an element by increasing the element's height on the binomial ladder. It does this by identify the shard of the filter array associated with the element, fetching the H elements within that shard associated with the element (the element's rungs), and setting one of the elements with value 0 (a rung above the element) to have value 1 (to put it below the element). It then clears two random elements from the entire array (not just the shard). If none of the H rungs associated with an element have value 0, Step will instead set two random elements from within the full array to have value 1.
public StepAsync ( string key, int heightOfLadderInRungs = null, System.TimeSpan timeout = null, CancellationToken cancellationToken = newCancellationToken() ) : Task
key string The element to take a step for.
heightOfLadderInRungs int If set, use a binomial ladder of this height rather than the default height. /// (Must not be greater than the default height that the binmomial ladder was initialized with.)
timeout System.TimeSpan
cancellationToken System.Threading.CancellationToken
return Task

Property Details

CacheOfElementsAtTopOfLadder protected property

A LRU cache of elements that are at the top of their ladder, used less to improve performance and more to prevent a single node from being overloaded if all requests target a single element.
protected FixedSizeLruCache CacheOfElementsAtTopOfLadder
return DateTime>.FixedSizeLruCache

MaxLadderHeight public property

The maximum (and default) height of elements' ladders.
public int MaxLadderHeight
return int

MinimumCacheFreshnessRequired public property

The maximum amount of time an element should allowed to be treated as at the top of its ladder due to caching when it may actually have fallen down the ladder since the value was cached.
public TimeSpan,System MinimumCacheFreshnessRequired
return System.TimeSpan

NumberOfShards public property

The number of shards that the filter array is evenly divided into.
public int NumberOfShards
return int

ShardHashFunction protected property

The hash function used to map elements to shards.
protected UniversalHashFunction,StopGuessing.EncryptionPrimitives ShardHashFunction
return StopGuessing.EncryptionPrimitives.UniversalHashFunction

ShardToHostMapping public property

Records for each shard that is (or at one time has been) stored on this server.
public IDistributedResponsibilitySet ShardToHostMapping
return IDistributedResponsibilitySet