org.apache.spark.util

Class TimeStampedHashMap<A,B>

Object

org.apache.spark.util.TimeStampedHashMap<A,B>

All Implemented Interfaces:

Cloneable, Logging, scala.collection.generic.Clearable, scala.collection.generic.FilterMonadic<scala.Tuple2<A,B>,scala.collection.mutable.Map<A,B>>, scala.collection.generic.GenericTraversableTemplate<scala.Tuple2<A,B>,scala.collection.GenIterable>, scala.collection.generic.Growable<scala.Tuple2<A,B>>, scala.collection.generic.HasNewBuilder<scala.Tuple2<A,B>,scala.collection.mutable.Map<A,B>>, scala.collection.generic.Shrinkable<A>, scala.collection.generic.Subtractable<A,scala.collection.mutable.Map<A,B>>, scala.collection.GenIterable<scala.Tuple2<A,B>>, scala.collection.GenIterableLike<scala.Tuple2<A,B>,scala.collection.mutable.Map<A,B>>, scala.collection.GenMap<A,B>, scala.collection.GenMapLike<A,B,scala.collection.mutable.Map<A,B>>, scala.collection.GenTraversable<scala.Tuple2<A,B>>, scala.collection.GenTraversableLike<scala.Tuple2<A,B>,scala.collection.mutable.Map<A,B>>, scala.collection.GenTraversableOnce<scala.Tuple2<A,B>>, scala.collection.Iterable<scala.Tuple2<A,B>>, scala.collection.IterableLike<scala.Tuple2<A,B>,scala.collection.mutable.Map<A,B>>, scala.collection.Map<A,B>, scala.collection.MapLike<A,B,scala.collection.mutable.Map<A,B>>, scala.collection.mutable.Builder<scala.Tuple2<A,B>,scala.collection.mutable.Map<A,B>>, scala.collection.Parallelizable<scala.Tuple2<A,B>,scala.collection.parallel.mutable.ParMap<A,B>>, scala.collection.Traversable<scala.Tuple2<A,B>>, scala.collection.TraversableLike<scala.Tuple2<A,B>,scala.collection.mutable.Map<A,B>>, scala.collection.TraversableOnce<scala.Tuple2<A,B>>, scala.Equals, scala.Function1<A,B>, scala.Mutable, scala.PartialFunction<A,B>

public class TimeStampedHashMap<A,B>
extends Object
implements scala.collection.mutable.Map<A,B>, Logging

This is a custom implementation of scala.collection.mutable.Map which stores the insertion timestamp along with each key-value pair. If specified, the timestamp of each pair can be updated every time it is accessed. Key-value pairs whose timestamp are older than a particular threshold time can then be removed using the clearOldValues method. This is intended to be a drop-in replacement of scala.collection.mutable.HashMap.

Nested Class Summary

Nested classes/interfaces inherited from interface scala.collection.mutable.Map

scala.collection.mutable.Map.WithDefault<A,B>

Nested classes/interfaces inherited from interface scala.collection.MapLike

scala.collection.MapLike.DefaultKeySet, scala.collection.MapLike.DefaultValuesIterable, scala.collection.MapLike.FilteredKeys, scala.collection.MapLike.MappedValues<C>

Nested classes/interfaces inherited from interface scala.PartialFunction

scala.PartialFunction.AndThen<A,B,C>, scala.PartialFunction.Lifted<A,B>, scala.PartialFunction.OrElse<A,B>, scala.PartialFunction.Unlifted<A,B>

Nested classes/interfaces inherited from interface scala.collection.TraversableLike

scala.collection.TraversableLike.WithFilter

Nested classes/interfaces inherited from interface scala.collection.TraversableOnce

scala.collection.TraversableOnce.BufferedCanBuildFrom<A,Coll extends scala.collection.TraversableOnce<Object>>, scala.collection.TraversableOnce.FlattenOps<A>, scala.collection.TraversableOnce.ForceImplicitAmbiguity, scala.collection.TraversableOnce.MonadOps<A>, scala.collection.TraversableOnce.OnceCanBuildFrom<A>

Constructor Summary

Constructors

Constructor and Description
TimeStampedHashMap(boolean updateTimeStampOnGet)

Method Summary

Methods

Modifier and Type	Method and Description
B	apply(A key)
void	clearOldValues(long threshTime) Removes old key-value pairs that have timestamp earlier than `threshTime`.
void	clearOldValues(long threshTime, scala.Function2<A,B,scala.runtime.BoxedUnit> f)
scala.collection.mutable.Map<A,B>	empty()
scala.collection.mutable.Map<A,B>	filter(scala.Function1<scala.Tuple2<A,B>,Object> p)
<U> void	foreach(scala.Function1<scala.Tuple2<A,B>,U> f)
scala.Option<B>	get(A key)
java.util.Set<java.util.Map.Entry<A,TimeStampedValue<B>>>	getEntrySet()
scala.Option<Object>	getTimestamp(A key)
scala.Option<TimeStampedValue<B>>	getTimeStampedValue(A key)
scala.collection.Iterator<scala.Tuple2<A,B>>	iterator()
void	putAll(scala.collection.immutable.Map<A,B> map)
scala.Option<B>	putIfAbsent(A key, B value)
int	size()
scala.collection.immutable.Map<A,B>	toMap()
void	update(A key, B value)

Methods inherited from class Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface scala.collection.mutable.Map

seq, withDefault, withDefaultValue

Methods inherited from interface scala.collection.mutable.Iterable

companion, parCombiner

Methods inherited from interface scala.collection.GenMap

updated

Methods inherited from interface scala.collection.mutable.MapLike

$minus, $minus, $minus$eq, $minus$minus, $plus, $plus, $plus$eq, $plus$plus, clear, clone, getOrElseUpdate, newBuilder, parCombiner, put, remove, result, retain, transform, updated

Methods inherited from interface scala.collection.MapLike

addString, contains, default, filterKeys, filterNot, getOrElse, isDefinedAt, isEmpty, keys, keySet, keysIterator, mapValues, stringPrefix, toBuffer, toSeq, toString, values, valuesIterator

Methods inherited from interface scala.PartialFunction

andThen, applyOrElse, lift, orElse, runWith

Methods inherited from interface scala.Function1

andThen$mcDD$sp, andThen$mcDF$sp, andThen$mcDI$sp, andThen$mcDJ$sp, andThen$mcFD$sp, andThen$mcFF$sp, andThen$mcFI$sp, andThen$mcFJ$sp, andThen$mcID$sp, andThen$mcIF$sp, andThen$mcII$sp, andThen$mcIJ$sp, andThen$mcJD$sp, andThen$mcJF$sp, andThen$mcJI$sp, andThen$mcJJ$sp, andThen$mcVD$sp, andThen$mcVF$sp, andThen$mcVI$sp, andThen$mcVJ$sp, andThen$mcZD$sp, andThen$mcZF$sp, andThen$mcZI$sp, andThen$mcZJ$sp, apply$mcDD$sp, apply$mcDF$sp, apply$mcDI$sp, apply$mcDJ$sp, apply$mcFD$sp, apply$mcFF$sp, apply$mcFI$sp, apply$mcFJ$sp, apply$mcID$sp, apply$mcIF$sp, apply$mcII$sp, apply$mcIJ$sp, apply$mcJD$sp, apply$mcJF$sp, apply$mcJI$sp, apply$mcJJ$sp, apply$mcVD$sp, apply$mcVF$sp, apply$mcVI$sp, apply$mcVJ$sp, apply$mcZD$sp, apply$mcZF$sp, apply$mcZI$sp, apply$mcZJ$sp, compose, compose$mcDD$sp, compose$mcDF$sp, compose$mcDI$sp, compose$mcDJ$sp, compose$mcFD$sp, compose$mcFF$sp, compose$mcFI$sp, compose$mcFJ$sp, compose$mcID$sp, compose$mcIF$sp, compose$mcII$sp, compose$mcIJ$sp, compose$mcJD$sp, compose$mcJF$sp, compose$mcJI$sp, compose$mcJJ$sp, compose$mcVD$sp, compose$mcVF$sp, compose$mcVI$sp, compose$mcVJ$sp, compose$mcZD$sp, compose$mcZF$sp, compose$mcZI$sp, compose$mcZJ$sp

Methods inherited from interface scala.collection.IterableLike

canEqual, copyToArray, drop, dropRight, exists, find, foldRight, forall, grouped, head, reduceRight, sameElements, slice, sliding, sliding, take, takeRight, takeWhile, thisCollection, toCollection, toIterable, toIterator, toStream, view, view, zip, zipAll, zipWithIndex

Methods inherited from interface scala.collection.TraversableLike

$plus$plus, $plus$plus$colon, $plus$plus$colon, collect, dropWhile, flatMap, groupBy, hasDefiniteSize, headOption, init, inits, isTraversableAgain, last, lastOption, map, partition, repr, scan, scanLeft, scanRight, sliceWithKnownBound, sliceWithKnownDelta, span, splitAt, tail, tails, to, toTraversable, withFilter

Methods inherited from interface scala.collection.TraversableOnce

$colon$bslash, $div$colon, addString, addString, aggregate, collectFirst, copyToArray, copyToArray, copyToBuffer, count, fold, foldLeft, max, maxBy, min, minBy, mkString, mkString, mkString, nonEmpty, product, reduce, reduceLeft, reduceLeftOption, reduceOption, reduceRightOption, reversed, sum, toArray, toIndexedSeq, toList, toMap, toSet, toVector

Methods inherited from interface scala.collection.GenMapLike

equals, hashCode

Methods inherited from interface scala.collection.GenTraversableOnce

$div$colon$bslash

Methods inherited from interface scala.collection.Parallelizable

par

Methods inherited from interface scala.collection.generic.Subtractable

repr

Methods inherited from interface scala.collection.mutable.Builder

mapResult, sizeHint, sizeHint, sizeHint, sizeHintBounded

Methods inherited from interface scala.collection.generic.Growable

$plus$eq, $plus$plus$eq

Methods inherited from interface scala.collection.generic.Shrinkable

$minus$eq, $minus$minus$eq

Methods inherited from interface scala.collection.mutable.Cloneable

scala$collection$mutable$Cloneable$$super$clone

Methods inherited from interface org.apache.spark.Logging

initializeIfNecessary, initializeLogging, isTraceEnabled, log_, log, logDebug, logDebug, logError, logError, logInfo, logInfo, logName, logTrace, logTrace, logWarning, logWarning

Constructor Detail

TimeStampedHashMap

public TimeStampedHashMap(boolean updateTimeStampOnGet)

Method Detail

get

public scala.Option<B> get(A key)

Specified by:

get in interface scala.collection.GenMapLike<A,B,scala.collection.mutable.Map<A,B>>

Specified by:

get in interface scala.collection.MapLike<A,B,scala.collection.mutable.Map<A,B>>

iterator

public scala.collection.Iterator<scala.Tuple2<A,B>> iterator()

Specified by:

iterator in interface scala.collection.GenIterableLike<scala.Tuple2<A,B>,scala.collection.mutable.Map<A,B>>

Specified by:

iterator in interface scala.collection.IterableLike<scala.Tuple2<A,B>,scala.collection.mutable.Map<A,B>>

Specified by:

iterator in interface scala.collection.MapLike<A,B,scala.collection.mutable.Map<A,B>>

getEntrySet

public java.util.Set<java.util.Map.Entry<A,TimeStampedValue<B>>> getEntrySet()

update

public void update(A key,
          B value)

apply

public B apply(A key)

Specified by:

apply in interface scala.collection.GenMapLike<A,B,scala.collection.mutable.Map<A,B>>

Specified by:

apply in interface scala.collection.MapLike<A,B,scala.collection.mutable.Map<A,B>>

Specified by:

apply in interface scala.Function1<A,B>

filter

public scala.collection.mutable.Map<A,B> filter(scala.Function1<scala.Tuple2<A,B>,Object> p)

Specified by:

filter in interface scala.collection.GenTraversableLike<scala.Tuple2<A,B>,scala.collection.mutable.Map<A,B>>

Specified by:

filter in interface scala.collection.TraversableLike<scala.Tuple2<A,B>,scala.collection.mutable.Map<A,B>>

empty

public scala.collection.mutable.Map<A,B> empty()

Specified by:

empty in interface scala.collection.Map<A,B>

Specified by:

empty in interface scala.collection.MapLike<A,B,scala.collection.mutable.Map<A,B>>

size

public int size()

Specified by:

size in interface scala.collection.GenTraversableLike<scala.Tuple2<A,B>,scala.collection.mutable.Map<A,B>>

Specified by:

size in interface scala.collection.GenTraversableOnce<scala.Tuple2<A,B>>

Specified by:

size in interface scala.collection.TraversableOnce<scala.Tuple2<A,B>>

foreach

public <U> void foreach(scala.Function1<scala.Tuple2<A,B>,U> f)

Specified by:

foreach in interface scala.collection.generic.FilterMonadic<scala.Tuple2<A,B>,scala.collection.mutable.Map<A,B>>

Specified by:

foreach in interface scala.collection.generic.GenericTraversableTemplate<scala.Tuple2<A,B>,scala.collection.GenIterable>

Specified by:

foreach in interface scala.collection.GenTraversableLike<scala.Tuple2<A,B>,scala.collection.mutable.Map<A,B>>

Specified by:

foreach in interface scala.collection.GenTraversableOnce<scala.Tuple2<A,B>>

Specified by:

foreach in interface scala.collection.IterableLike<scala.Tuple2<A,B>,scala.collection.mutable.Map<A,B>>

Specified by:

foreach in interface scala.collection.TraversableLike<scala.Tuple2<A,B>,scala.collection.mutable.Map<A,B>>

Specified by:

foreach in interface scala.collection.TraversableOnce<scala.Tuple2<A,B>>

putIfAbsent

public scala.Option<B> putIfAbsent(A key,
                          B value)

putAll

public void putAll(scala.collection.immutable.Map<A,B> map)

toMap

public scala.collection.immutable.Map<A,B> toMap()

clearOldValues

public void clearOldValues(long threshTime,
                  scala.Function2<A,B,scala.runtime.BoxedUnit> f)

clearOldValues

public void clearOldValues(long threshTime)

Removes old key-value pairs that have timestamp earlier than `threshTime`.

getTimeStampedValue

public scala.Option<TimeStampedValue<B>> getTimeStampedValue(A key)

getTimestamp

public scala.Option<Object> getTimestamp(A key)