org.apache.spark.util

Class TimeStampedWeakValueHashMap<A,B>

Object

org.apache.spark.util.TimeStampedWeakValueHashMap<A,B>

All Implemented Interfaces:

Cloneable, Logging, scala.collection.generic.Clearable, scala.collection.generic.FilterMonadic<scala.Tuple2<A,B>,scala.collection.mutable.Map<A,B>>, scala.collection.generic.GenericTraversableTemplate<scala.Tuple2<A,B>,scala.collection.GenIterable>, scala.collection.generic.Growable<scala.Tuple2<A,B>>, scala.collection.generic.HasNewBuilder<scala.Tuple2<A,B>,scala.collection.mutable.Map<A,B>>, scala.collection.generic.Shrinkable<A>, scala.collection.generic.Subtractable<A,scala.collection.mutable.Map<A,B>>, scala.collection.GenIterable<scala.Tuple2<A,B>>, scala.collection.GenIterableLike<scala.Tuple2<A,B>,scala.collection.mutable.Map<A,B>>, scala.collection.GenMap<A,B>, scala.collection.GenMapLike<A,B,scala.collection.mutable.Map<A,B>>, scala.collection.GenTraversable<scala.Tuple2<A,B>>, scala.collection.GenTraversableLike<scala.Tuple2<A,B>,scala.collection.mutable.Map<A,B>>, scala.collection.GenTraversableOnce<scala.Tuple2<A,B>>, scala.collection.Iterable<scala.Tuple2<A,B>>, scala.collection.IterableLike<scala.Tuple2<A,B>,scala.collection.mutable.Map<A,B>>, scala.collection.Map<A,B>, scala.collection.MapLike<A,B,scala.collection.mutable.Map<A,B>>, scala.collection.mutable.Builder<scala.Tuple2<A,B>,scala.collection.mutable.Map<A,B>>, scala.collection.Parallelizable<scala.Tuple2<A,B>,scala.collection.parallel.mutable.ParMap<A,B>>, scala.collection.Traversable<scala.Tuple2<A,B>>, scala.collection.TraversableLike<scala.Tuple2<A,B>,scala.collection.mutable.Map<A,B>>, scala.collection.TraversableOnce<scala.Tuple2<A,B>>, scala.Equals, scala.Function1<A,B>, scala.Mutable, scala.PartialFunction<A,B>

public class TimeStampedWeakValueHashMap<A,B>
extends Object
implements scala.collection.mutable.Map<A,B>, Logging

A wrapper of TimeStampedHashMap that ensures the values are weakly referenced and timestamped.

If the value is garbage collected and the weak reference is null, get() will return a non-existent value. These entries are removed from the map periodically (every N inserts), as their values are no longer strongly reachable. Further, key-value pairs whose timestamps are older than a particular threshold can be removed using the clearOldValues method.

TimeStampedWeakValueHashMap exposes a scala.collection.mutable.Map interface, which allows it to be a drop-in replacement for Scala HashMaps. Internally, it uses a Java ConcurrentHashMap, so all operations on this HashMap are thread-safe.

Nested Class Summary

Nested classes/interfaces inherited from interface scala.collection.mutable.Map

scala.collection.mutable.Map.WithDefault<A,B>

Nested classes/interfaces inherited from interface scala.collection.MapLike

scala.collection.MapLike.DefaultKeySet, scala.collection.MapLike.DefaultValuesIterable, scala.collection.MapLike.FilteredKeys, scala.collection.MapLike.MappedValues<C>

Nested classes/interfaces inherited from interface scala.PartialFunction

scala.PartialFunction.AndThen<A,B,C>, scala.PartialFunction.Lifted<A,B>, scala.PartialFunction.OrElse<A,B>, scala.PartialFunction.Unlifted<A,B>

Nested classes/interfaces inherited from interface scala.collection.TraversableLike

scala.collection.TraversableLike.WithFilter

Nested classes/interfaces inherited from interface scala.collection.TraversableOnce

scala.collection.TraversableOnce.BufferedCanBuildFrom<A,Coll extends scala.collection.TraversableOnce<Object>>, scala.collection.TraversableOnce.FlattenOps<A>, scala.collection.TraversableOnce.ForceImplicitAmbiguity, scala.collection.TraversableOnce.MonadOps<A>, scala.collection.TraversableOnce.OnceCanBuildFrom<A>

Constructor Summary

Constructors

Constructor and Description
TimeStampedWeakValueHashMap(boolean updateTimeStampOnGet)

Method Summary

Methods

Modifier and Type	Method and Description
B	apply(A key)
static int	CLEAR_NULL_VALUES_INTERVAL()
void	clearNullValues() Remove entries with values that are no longer strongly reachable.
void	clearOldValues(long threshTime) Remove old key-value pairs with timestamps earlier than `threshTime`.
scala.collection.mutable.Map<A,B>	empty()
scala.collection.mutable.Map<A,B>	filter(scala.Function1<scala.Tuple2<A,B>,Object> p)
<U> void	foreach(scala.Function1<scala.Tuple2<A,B>,U> f)
static <V> V	fromWeakReference(java.lang.ref.WeakReference<V> ref)
static <K,V> scala.collection.Iterator<scala.Tuple2<K,V>>	fromWeakReferenceIterator(scala.collection.Iterator<scala.Tuple2<K,java.lang.ref.WeakReference<V>>> it)
static <K,V> scala.collection.mutable.Map<K,V>	fromWeakReferenceMap(scala.collection.mutable.Map<K,java.lang.ref.WeakReference<V>> map)
static <V> scala.Option<V>	fromWeakReferenceOption(scala.Option<java.lang.ref.WeakReference<V>> v)
static <K,V> scala.Tuple2<K,V>	fromWeakReferenceTuple(scala.Tuple2<K,java.lang.ref.WeakReference<V>> kv)
scala.Option<B>	get(A key)
scala.Option<java.lang.ref.WeakReference<B>>	getReference(A key)
scala.Option<Object>	getTimestamp(A key)
scala.collection.Iterator<scala.Tuple2<A,B>>	iterator()
scala.Option<B>	putIfAbsent(A key, B value)
int	size()
scala.collection.immutable.Map<A,B>	toMap()
static <V> java.lang.ref.WeakReference<V>	toWeakReference(V v)
static <K,V,R> scala.Function1<scala.Tuple2<K,java.lang.ref.WeakReference<V>>,R>	toWeakReferenceFunction(scala.Function1<scala.Tuple2<K,V>,R> p)
static <K,V> scala.Tuple2<K,java.lang.ref.WeakReference<V>>	toWeakReferenceTuple(scala.Tuple2<K,V> kv)
void	update(A key, B value)

Methods inherited from class Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface scala.collection.mutable.Map

seq, withDefault, withDefaultValue

Methods inherited from interface scala.collection.mutable.Iterable

companion, parCombiner

Methods inherited from interface scala.collection.GenMap

updated

Methods inherited from interface scala.collection.mutable.MapLike

$minus, $minus, $minus$eq, $minus$minus, $plus, $plus, $plus$eq, $plus$plus, clear, clone, getOrElseUpdate, newBuilder, parCombiner, put, remove, result, retain, transform, updated

Methods inherited from interface scala.collection.MapLike

addString, contains, default, filterKeys, filterNot, getOrElse, isDefinedAt, isEmpty, keys, keySet, keysIterator, mapValues, stringPrefix, toBuffer, toSeq, toString, values, valuesIterator

Methods inherited from interface scala.PartialFunction

andThen, applyOrElse, lift, orElse, runWith

Methods inherited from interface scala.Function1

andThen$mcDD$sp, andThen$mcDF$sp, andThen$mcDI$sp, andThen$mcDJ$sp, andThen$mcFD$sp, andThen$mcFF$sp, andThen$mcFI$sp, andThen$mcFJ$sp, andThen$mcID$sp, andThen$mcIF$sp, andThen$mcII$sp, andThen$mcIJ$sp, andThen$mcJD$sp, andThen$mcJF$sp, andThen$mcJI$sp, andThen$mcJJ$sp, andThen$mcVD$sp, andThen$mcVF$sp, andThen$mcVI$sp, andThen$mcVJ$sp, andThen$mcZD$sp, andThen$mcZF$sp, andThen$mcZI$sp, andThen$mcZJ$sp, apply$mcDD$sp, apply$mcDF$sp, apply$mcDI$sp, apply$mcDJ$sp, apply$mcFD$sp, apply$mcFF$sp, apply$mcFI$sp, apply$mcFJ$sp, apply$mcID$sp, apply$mcIF$sp, apply$mcII$sp, apply$mcIJ$sp, apply$mcJD$sp, apply$mcJF$sp, apply$mcJI$sp, apply$mcJJ$sp, apply$mcVD$sp, apply$mcVF$sp, apply$mcVI$sp, apply$mcVJ$sp, apply$mcZD$sp, apply$mcZF$sp, apply$mcZI$sp, apply$mcZJ$sp, compose, compose$mcDD$sp, compose$mcDF$sp, compose$mcDI$sp, compose$mcDJ$sp, compose$mcFD$sp, compose$mcFF$sp, compose$mcFI$sp, compose$mcFJ$sp, compose$mcID$sp, compose$mcIF$sp, compose$mcII$sp, compose$mcIJ$sp, compose$mcJD$sp, compose$mcJF$sp, compose$mcJI$sp, compose$mcJJ$sp, compose$mcVD$sp, compose$mcVF$sp, compose$mcVI$sp, compose$mcVJ$sp, compose$mcZD$sp, compose$mcZF$sp, compose$mcZI$sp, compose$mcZJ$sp

Methods inherited from interface scala.collection.IterableLike

canEqual, copyToArray, drop, dropRight, exists, find, foldRight, forall, grouped, head, reduceRight, sameElements, slice, sliding, sliding, take, takeRight, takeWhile, thisCollection, toCollection, toIterable, toIterator, toStream, view, view, zip, zipAll, zipWithIndex

Methods inherited from interface scala.collection.TraversableLike

$plus$plus, $plus$plus$colon, $plus$plus$colon, collect, dropWhile, flatMap, groupBy, hasDefiniteSize, headOption, init, inits, isTraversableAgain, last, lastOption, map, partition, repr, scan, scanLeft, scanRight, sliceWithKnownBound, sliceWithKnownDelta, span, splitAt, tail, tails, to, toTraversable, withFilter

Methods inherited from interface scala.collection.TraversableOnce

$colon$bslash, $div$colon, addString, addString, aggregate, collectFirst, copyToArray, copyToArray, copyToBuffer, count, fold, foldLeft, max, maxBy, min, minBy, mkString, mkString, mkString, nonEmpty, product, reduce, reduceLeft, reduceLeftOption, reduceOption, reduceRightOption, reversed, sum, toArray, toIndexedSeq, toList, toMap, toSet, toVector

Methods inherited from interface scala.collection.GenMapLike

equals, hashCode

Methods inherited from interface scala.collection.GenTraversableOnce

$div$colon$bslash

Methods inherited from interface scala.collection.Parallelizable

par

Methods inherited from interface scala.collection.generic.Subtractable

repr

Methods inherited from interface scala.collection.mutable.Builder

mapResult, sizeHint, sizeHint, sizeHint, sizeHintBounded

Methods inherited from interface scala.collection.generic.Growable

$plus$eq, $plus$plus$eq

Methods inherited from interface scala.collection.generic.Shrinkable

$minus$eq, $minus$minus$eq

Methods inherited from interface scala.collection.mutable.Cloneable

scala$collection$mutable$Cloneable$$super$clone

Methods inherited from interface org.apache.spark.Logging

initializeIfNecessary, initializeLogging, isTraceEnabled, log_, log, logDebug, logDebug, logError, logError, logInfo, logInfo, logName, logTrace, logTrace, logWarning, logWarning

Constructor Detail

TimeStampedWeakValueHashMap

public TimeStampedWeakValueHashMap(boolean updateTimeStampOnGet)

Method Detail

CLEAR_NULL_VALUES_INTERVAL

public static int CLEAR_NULL_VALUES_INTERVAL()

toWeakReference

public static <V> java.lang.ref.WeakReference<V> toWeakReference(V v)

toWeakReferenceTuple

public static <K,V> scala.Tuple2<K,java.lang.ref.WeakReference<V>> toWeakReferenceTuple(scala.Tuple2<K,V> kv)

toWeakReferenceFunction

public static <K,V,R> scala.Function1<scala.Tuple2<K,java.lang.ref.WeakReference<V>>,R> toWeakReferenceFunction(scala.Function1<scala.Tuple2<K,V>,R> p)

fromWeakReference

public static <V> V fromWeakReference(java.lang.ref.WeakReference<V> ref)

fromWeakReferenceOption

public static <V> scala.Option<V> fromWeakReferenceOption(scala.Option<java.lang.ref.WeakReference<V>> v)

fromWeakReferenceTuple

public static <K,V> scala.Tuple2<K,V> fromWeakReferenceTuple(scala.Tuple2<K,java.lang.ref.WeakReference<V>> kv)

fromWeakReferenceIterator

public static <K,V> scala.collection.Iterator<scala.Tuple2<K,V>> fromWeakReferenceIterator(scala.collection.Iterator<scala.Tuple2<K,java.lang.ref.WeakReference<V>>> it)

fromWeakReferenceMap

public static <K,V> scala.collection.mutable.Map<K,V> fromWeakReferenceMap(scala.collection.mutable.Map<K,java.lang.ref.WeakReference<V>> map)

get

public scala.Option<B> get(A key)

Specified by:

get in interface scala.collection.GenMapLike<A,B,scala.collection.mutable.Map<A,B>>

Specified by:

get in interface scala.collection.MapLike<A,B,scala.collection.mutable.Map<A,B>>

iterator

public scala.collection.Iterator<scala.Tuple2<A,B>> iterator()

Specified by:

iterator in interface scala.collection.GenIterableLike<scala.Tuple2<A,B>,scala.collection.mutable.Map<A,B>>

Specified by:

iterator in interface scala.collection.IterableLike<scala.Tuple2<A,B>,scala.collection.mutable.Map<A,B>>

Specified by:

iterator in interface scala.collection.MapLike<A,B,scala.collection.mutable.Map<A,B>>

update

public void update(A key,
          B value)

apply

public B apply(A key)

Specified by:

apply in interface scala.collection.GenMapLike<A,B,scala.collection.mutable.Map<A,B>>

Specified by:

apply in interface scala.collection.MapLike<A,B,scala.collection.mutable.Map<A,B>>

Specified by:

apply in interface scala.Function1<A,B>

filter

public scala.collection.mutable.Map<A,B> filter(scala.Function1<scala.Tuple2<A,B>,Object> p)

Specified by:

filter in interface scala.collection.GenTraversableLike<scala.Tuple2<A,B>,scala.collection.mutable.Map<A,B>>

Specified by:

filter in interface scala.collection.TraversableLike<scala.Tuple2<A,B>,scala.collection.mutable.Map<A,B>>

empty

public scala.collection.mutable.Map<A,B> empty()

Specified by:

empty in interface scala.collection.Map<A,B>

Specified by:

empty in interface scala.collection.MapLike<A,B,scala.collection.mutable.Map<A,B>>

size

public int size()

Specified by:

size in interface scala.collection.GenTraversableLike<scala.Tuple2<A,B>,scala.collection.mutable.Map<A,B>>

Specified by:

size in interface scala.collection.GenTraversableOnce<scala.Tuple2<A,B>>

Specified by:

size in interface scala.collection.TraversableOnce<scala.Tuple2<A,B>>

foreach

public <U> void foreach(scala.Function1<scala.Tuple2<A,B>,U> f)

Specified by:

foreach in interface scala.collection.generic.FilterMonadic<scala.Tuple2<A,B>,scala.collection.mutable.Map<A,B>>

Specified by:

foreach in interface scala.collection.generic.GenericTraversableTemplate<scala.Tuple2<A,B>,scala.collection.GenIterable>

Specified by:

foreach in interface scala.collection.GenTraversableLike<scala.Tuple2<A,B>,scala.collection.mutable.Map<A,B>>

Specified by:

foreach in interface scala.collection.GenTraversableOnce<scala.Tuple2<A,B>>

Specified by:

foreach in interface scala.collection.IterableLike<scala.Tuple2<A,B>,scala.collection.mutable.Map<A,B>>

Specified by:

foreach in interface scala.collection.TraversableLike<scala.Tuple2<A,B>,scala.collection.mutable.Map<A,B>>

Specified by:

foreach in interface scala.collection.TraversableOnce<scala.Tuple2<A,B>>

putIfAbsent

public scala.Option<B> putIfAbsent(A key,
                          B value)

toMap

public scala.collection.immutable.Map<A,B> toMap()

clearOldValues

public void clearOldValues(long threshTime)

Remove old key-value pairs with timestamps earlier than `threshTime`.

clearNullValues

public void clearNullValues()

Remove entries with values that are no longer strongly reachable.

getTimestamp

public scala.Option<Object> getTimestamp(A key)

getReference

public scala.Option<java.lang.ref.WeakReference<B>> getReference(A key)