org.apache.spark

Class Accumulable<R,T>

Object

org.apache.spark.Accumulable<R,T>

Type Parameters:

R - the full accumulated data (result type)

T - partial data that can be added in

All Implemented Interfaces:

java.io.Serializable

Direct Known Subclasses:

Accumulator

Deprecated.

use AccumulatorV2. Since 2.0.0.

public class Accumulable<R,T>
extends Object
implements java.io.Serializable

A data type that can be accumulated, i.e. has a commutative and associative "add" operation, but where the result type, R, may be different from the element type being added, T.

You must define how to add data, and how to merge two of these together. For some data types, such as a counter, these might be the same operation. In that case, you can use the simpler Accumulator. They won't always be the same, though -- e.g., imagine you are accumulating a set. You will add items to the set, and you will union two sets together.

Operations are not thread-safe.

param: id ID of this accumulator; for internal use only. param: initialValue initial value of accumulator param: param helper object defining how to add elements of type R and T param: name human-readable name for use in Spark's web UI param: countFailedValues whether to accumulate values from failed tasks. This is set to true for system and time metrics like serialization time or bytes spilled, and false for things with absolute values like number of input rows. This should be used for internal metrics only.

See Also:

Serialized Form

Constructor Summary

Constructors

Constructor and Description
Accumulable(R initialValue, AccumulableParam<R,T> param) Deprecated.

Method Summary

Modifier and Type	Method and Description
void	add(T term) Deprecated. Add more data to this accumulator / accumulable
long	id() Deprecated.
R	localValue() Deprecated. Get the current value of this accumulator from within a task.
void	merge(R term) Deprecated. Merge two accumulable objects together
scala.Option<String>	name() Deprecated.
void	setValue(R newValue) Deprecated. Set the accumulator's value.
String	toString() Deprecated.
R	value() Deprecated. Access the accumulator's current value; only allowed on driver.
R	zero() Deprecated.

Methods inherited from class Object

equals, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

Accumulable

public Accumulable(R initialValue,
                   AccumulableParam<R,T> param)

Deprecated.

Method Detail

add

public void add(T term)

Deprecated.

Add more data to this accumulator / accumulable

Parameters:

term - the data to add

id

public long id()

Deprecated.

localValue

public R localValue()

Deprecated.

Get the current value of this accumulator from within a task.

This is NOT the global value of the accumulator. To get the global value after a completed operation on the dataset, call value.

The typical use of this method is to directly mutate the local value, eg., to add an element to a Set.

Returns:

(undocumented)

merge

public void merge(R term)

Deprecated.

Merge two accumulable objects together

Normally, a user will not want to use this version, but will instead call add.

Parameters:

term - the other R that will get merged with this

name

public scala.Option<String> name()

Deprecated.

setValue

public void setValue(R newValue)

Deprecated.

Set the accumulator's value. For internal use only.

Parameters:

newValue - (undocumented)

toString

public String toString()

Deprecated.

Overrides:

toString in class Object

value

public R value()

Deprecated.

Access the accumulator's current value; only allowed on driver.

Returns:

(undocumented)

zero

public R zero()

Deprecated.