org.apache.spark.graphx

Class EdgeRDD<ED>

java.lang.Object

org.apache.spark.rdd.RDD<Edge<ED>>

org.apache.spark.graphx.EdgeRDD<ED>

All Implemented Interfaces:

java.io.Serializable, Logging

Direct Known Subclasses:

EdgeRDDImpl

public abstract class EdgeRDD<ED>
extends RDD<Edge<ED>>

EdgeRDD[ED, VD] extends RDD[Edge[ED} by storing the edges in columnar format on each partition for performance. It may additionally store the vertex attributes associated with each edge to provide the triplet view. Shipping of the vertex attributes is managed by impl.ReplicatedVertexView.

See Also:

Serialized Form

Constructor Summary

Constructors

Constructor and Description
EdgeRDD(SparkContext sc, scala.collection.Seq<Dependency<?>> deps)

Method Summary

Methods

Modifier and Type	Method and Description
scala.collection.Iterator<Edge<ED>>	compute(Partition part, TaskContext context) :: DeveloperApi :: Implemented by subclasses to compute a given partition.
static <ED,VD> EdgeRDDImpl<ED,VD>	fromEdges(RDD<Edge<ED>> edges, scala.reflect.ClassTag<ED> evidence$4, scala.reflect.ClassTag<VD> evidence$5) Creates an EdgeRDD from a set of edges.
protected Partition[]	getPartitions() Implemented by subclasses to return the set of partitions in this RDD.
abstract <ED2,ED3> EdgeRDD<ED3>	innerJoin(EdgeRDD<ED2> other, scala.Function4<java.lang.Object,java.lang.Object,ED,ED2,ED3> f, scala.reflect.ClassTag<ED2> evidence$2, scala.reflect.ClassTag<ED3> evidence$3) Inner joins this EdgeRDD with another EdgeRDD, assuming both are partitioned using the same PartitionStrategy.
abstract <ED2> EdgeRDD<ED2>	mapValues(scala.Function1<Edge<ED>,ED2> f, scala.reflect.ClassTag<ED2> evidence$1) Map the values in an edge partitioning preserving the structure but changing the values.
abstract EdgeRDD<ED>	reverse() Reverse all the edges in this RDD.

Methods inherited from class org.apache.spark.rdd.RDD

aggregate, cache, cartesian, checkpoint, checkpointData, clearDependencies, coalesce, collect, collect, context, count, countApprox, countApproxDistinct, countApproxDistinct, countByValue, countByValueApprox, creationSite, dependencies, distinct, distinct, doubleRDDToDoubleRDDFunctions, filter, filterWith, first, firstParent, flatMap, flatMapWith, fold, foreach, foreachPartition, foreachWith, getCheckpointFile, getDependencies, getPreferredLocations, getStorageLevel, glom, groupBy, groupBy, groupBy, id, intersection, intersection, intersection, isCheckpointed, isEmpty, iterator, keyBy, localCheckpoint, map, mapPartitions, mapPartitionsWithContext, mapPartitionsWithIndex, mapPartitionsWithSplit, mapWith, max, min, name, numericRDDToDoubleRDDFunctions, parent, partitioner, partitions, persist, persist, pipe, pipe, pipe, preferredLocations, randomSplit, rddToAsyncRDDActions, rddToOrderedRDDFunctions, rddToPairRDDFunctions, rddToSequenceFileRDDFunctions, reduce, repartition, sample, saveAsObjectFile, saveAsTextFile, saveAsTextFile, scope, setName, sortBy, sparkContext, subtract, subtract, subtract, take, takeOrdered, takeSample, toArray, toDebugString, toJavaRDD, toLocalIterator, top, toString, treeAggregate, treeReduce, union, unpersist, zip, zipPartitions, zipPartitions, zipPartitions, zipPartitions, zipPartitions, zipPartitions, zipWithIndex, zipWithUniqueId

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Methods inherited from interface org.apache.spark.Logging

initializeIfNecessary, initializeLogging, isTraceEnabled, log_, log, logDebug, logDebug, logError, logError, logInfo, logInfo, logName, logTrace, logTrace, logWarning, logWarning

Constructor Detail

EdgeRDD

public EdgeRDD(SparkContext sc,
       scala.collection.Seq<Dependency<?>> deps)

Method Detail

fromEdges

public static <ED,VD> EdgeRDDImpl<ED,VD> fromEdges(RDD<Edge<ED>> edges,
                                   scala.reflect.ClassTag<ED> evidence$4,
                                   scala.reflect.ClassTag<VD> evidence$5)

Creates an EdgeRDD from a set of edges.

Parameters:

edges - (undocumented)

evidence$4 - (undocumented)

evidence$5 - (undocumented)

Returns:

(undocumented)

getPartitions

protected Partition[] getPartitions()

Description copied from class: RDD

Implemented by subclasses to return the set of partitions in this RDD. This method will only be called once, so it is safe to implement a time-consuming computation in it.

Specified by:

getPartitions in class RDD<Edge<ED>>

Returns:

(undocumented)

compute

public scala.collection.Iterator<Edge<ED>> compute(Partition part,
                                          TaskContext context)

Description copied from class: RDD

:: DeveloperApi :: Implemented by subclasses to compute a given partition.

Specified by:

compute in class RDD<Edge<ED>>

Parameters:

part - (undocumented)

context - (undocumented)

Returns:

(undocumented)

mapValues

public abstract <ED2> EdgeRDD<ED2> mapValues(scala.Function1<Edge<ED>,ED2> f,
                           scala.reflect.ClassTag<ED2> evidence$1)

Map the values in an edge partitioning preserving the structure but changing the values.

Parameters:

f - the function from an edge to a new edge value

evidence$1 - (undocumented)

Returns:

a new EdgeRDD containing the new edge values

reverse

public abstract EdgeRDD<ED> reverse()

Reverse all the edges in this RDD.

Returns:

a new EdgeRDD containing all the edges reversed

innerJoin

public abstract <ED2,ED3> EdgeRDD<ED3> innerJoin(EdgeRDD<ED2> other,
                               scala.Function4<java.lang.Object,java.lang.Object,ED,ED2,ED3> f,
                               scala.reflect.ClassTag<ED2> evidence$2,
                               scala.reflect.ClassTag<ED3> evidence$3)

Inner joins this EdgeRDD with another EdgeRDD, assuming both are partitioned using the same PartitionStrategy.

Parameters:

other - the EdgeRDD to join with

f - the join function applied to corresponding values of this and other

evidence$2 - (undocumented)

evidence$3 - (undocumented)

Returns:

a new EdgeRDD containing only edges that appear in both this and other, with values supplied by f