C# Class StopGuessing.Clients.DistributedBinomialLadderFilterClient

The client side of a distributed binomial ladder frequency filter. Binomial ladder filters are used to identify frequently-occuring elements in streams while minimizing the data revealed/stored about infrequently-occuring elements. This client is utilized both by applications using a distributed binomial ladder filter and is also by the servers implementing the server to communicate amongst themselves. For more information about the binomial ladder filter, search its name and "Microsoft Research" to find detailed publications/tech reports.
Inheritance: IBinomialLadderFilter
Afficher le fichier Open project: Microsoft/StopGuessing Class Usage Examples

Méthodes publiques

Свойство Type Description
MaxLadderHeight int
MinimumCacheFreshnessRequired System.TimeSpan
NumberOfShards int
ShardToHostMapping IDistributedResponsibilitySet

Protected Properties

Свойство Type Description
CacheOfElementsAtTopOfLadder DateTime>.FixedSizeLruCache
ShardHashFunction StopGuessing.EncryptionPrimitives.UniversalHashFunction

Méthodes publiques

Méthode Description
AssignRandomBit ( int valueToAssign, int shardNumber = null ) : void

Assign a random element with the binomial ladder filter to either 0 or 1.
DistributedBinomialLadderFilterClient ( int numberOfShards, int defaultHeightOfLadder, IDistributedResponsibilitySet shardToHostMapping, string configurationKey, System.TimeSpan mininmumCacheFreshnessRequired = null ) : System

Create a client for a distributed binomial ladder filter
GetHeightAsync ( string element, int heightOfLadderInRungs = null, System.TimeSpan timeout = null, CancellationToken cancellationToken = newCancellationToken() ) : Task

Get the height of an element on its binomial ladder. The height of an element is the number of the H array elements that are associated with it that have value 1.
GetRandomShardIndex ( ) : int

Get a random shard index from the set of shards that make up the array elements of the binomial ladder filter.
GetShardIndex ( string key ) : int

Get the shard index associated with an element.
StepAsync ( string key, int heightOfLadderInRungs = null, System.TimeSpan timeout = null, CancellationToken cancellationToken = newCancellationToken() ) : Task

The Step operation records the occurrence of an element by increasing the element's height on the binomial ladder. It does this by identify the shard of the filter array associated with the element, fetching the H elements within that shard associated with the element (the element's rungs), and setting one of the elements with value 0 (a rung above the element) to have value 1 (to put it below the element). It then clears two random elements from the entire array (not just the shard). If none of the H rungs associated with an element have value 0, Step will instead set two random elements from within the full array to have value 1.

Method Details

AssignRandomBit() public méthode

Assign a random element with the binomial ladder filter to either 0 or 1.
public AssignRandomBit ( int valueToAssign, int shardNumber = null ) : void
valueToAssign int The random element will be set to 1 if this parameter is nonzero, and to 0 otherwise.
shardNumber int Optioanlly set this optional value to identify a shard number within to select a random element. /// If not set, this method will choose a shard at random.
Résultat void

DistributedBinomialLadderFilterClient() public méthode

Create a client for a distributed binomial ladder filter
public DistributedBinomialLadderFilterClient ( int numberOfShards, int defaultHeightOfLadder, IDistributedResponsibilitySet shardToHostMapping, string configurationKey, System.TimeSpan mininmumCacheFreshnessRequired = null ) : System
numberOfShards int The number of shards that the bit array of the binomial ladder filter will be divided into. /// The greater the number of shards, the more evently it can be distributed. However, the number of shards should still /// be a few orders of magnitude smaller than the ladder height.
defaultHeightOfLadder int The default ladder height for elements on the ladder.
shardToHostMapping IDistributedResponsibilitySet An object that maps each shard number to the host responsible for that shard.
configurationKey string A key used to protect the hashing from algorithmic complexity attacks. /// This key should not be unique to the application using the filter and should not be known to any untrusted /// systems that might control which elements get sent to the filter. If an attacker could submit elements to the filter /// and knew this key, the attacker could arrange for all elements to go to the same shard and in so doing overload that shard.
mininmumCacheFreshnessRequired System.TimeSpan The maximum time that an element should be kept in the cache of elements at the top of their ladder. /// In other words, how long to bound the possible time that an element may still appear to be at the top of its ladder in the cache /// when it is no longer at the top of the ladder based on the filter array. Defaults to one minute.
Résultat System

GetHeightAsync() public méthode

Get the height of an element on its binomial ladder. The height of an element is the number of the H array elements that are associated with it that have value 1.
public GetHeightAsync ( string element, int heightOfLadderInRungs = null, System.TimeSpan timeout = null, CancellationToken cancellationToken = newCancellationToken() ) : Task
element string The element to be measured to determine its height on its binomial ladder.
heightOfLadderInRungs int >If set, use a binomial ladder of this height rather than the default height. /// (Must not be greater than the default height that the binmomial ladder was initialized with.
timeout System.TimeSpan
cancellationToken System.Threading.CancellationToken
Résultat Task

GetRandomShardIndex() public méthode

Get a random shard index from the set of shards that make up the array elements of the binomial ladder filter.
public GetRandomShardIndex ( ) : int
Résultat int

GetShardIndex() public méthode

Get the shard index associated with an element.
public GetShardIndex ( string key ) : int
key string The element to match to a shard index.
Résultat int

StepAsync() public méthode

The Step operation records the occurrence of an element by increasing the element's height on the binomial ladder. It does this by identify the shard of the filter array associated with the element, fetching the H elements within that shard associated with the element (the element's rungs), and setting one of the elements with value 0 (a rung above the element) to have value 1 (to put it below the element). It then clears two random elements from the entire array (not just the shard). If none of the H rungs associated with an element have value 0, Step will instead set two random elements from within the full array to have value 1.
public StepAsync ( string key, int heightOfLadderInRungs = null, System.TimeSpan timeout = null, CancellationToken cancellationToken = newCancellationToken() ) : Task
key string The element to take a step for.
heightOfLadderInRungs int If set, use a binomial ladder of this height rather than the default height. /// (Must not be greater than the default height that the binmomial ladder was initialized with.)
timeout System.TimeSpan
cancellationToken System.Threading.CancellationToken
Résultat Task

Property Details

CacheOfElementsAtTopOfLadder protected_oe property

A LRU cache of elements that are at the top of their ladder, used less to improve performance and more to prevent a single node from being overloaded if all requests target a single element.
protected FixedSizeLruCache CacheOfElementsAtTopOfLadder
Résultat DateTime>.FixedSizeLruCache

MaxLadderHeight public_oe property

The maximum (and default) height of elements' ladders.
public int MaxLadderHeight
Résultat int

MinimumCacheFreshnessRequired public_oe property

The maximum amount of time an element should allowed to be treated as at the top of its ladder due to caching when it may actually have fallen down the ladder since the value was cached.
public TimeSpan,System MinimumCacheFreshnessRequired
Résultat System.TimeSpan

NumberOfShards public_oe property

The number of shards that the filter array is evenly divided into.
public int NumberOfShards
Résultat int

ShardHashFunction protected_oe property

The hash function used to map elements to shards.
protected UniversalHashFunction,StopGuessing.EncryptionPrimitives ShardHashFunction
Résultat StopGuessing.EncryptionPrimitives.UniversalHashFunction

ShardToHostMapping public_oe property

Records for each shard that is (or at one time has been) stored on this server.
public IDistributedResponsibilitySet ShardToHostMapping
Résultat IDistributedResponsibilitySet