org.apache.spark.serializer

Class KryoSerializer

Object

org.apache.spark.serializer.Serializer

org.apache.spark.serializer.KryoSerializer

All Implemented Interfaces:

java.io.Serializable, Logging

public class KryoSerializer
extends Serializer
implements Logging, java.io.Serializable

A Spark serializer that uses the Kryo serialization library.

Note that this serializer is not guaranteed to be wire-compatible across different versions of Spark. It is intended to be used to serialize/de-serialize data within a single Spark application.

See Also:

Serialized Form

Constructor Summary

Constructors

Constructor and Description
KryoSerializer(SparkConf conf)

Method Summary

Methods

Modifier and Type	Method and Description
int	maxBufferSizeMb()
SerializerInstance	newInstance() Creates a new SerializerInstance.
com.esotericsoftware.kryo.Kryo	newKryo()
com.esotericsoftware.kryo.io.Output	newKryoOutput()
boolean	supportsRelocationOfSerializedObjects() :: Private :: Returns true if this serializer supports relocation of its serialized objects and false otherwise.

Methods inherited from class org.apache.spark.serializer.Serializer

getSerializer, getSerializer, setDefaultClassLoader

Methods inherited from class Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.spark.Logging

initializeIfNecessary, initializeLogging, isTraceEnabled, log_, log, logDebug, logDebug, logError, logError, logInfo, logInfo, logName, logTrace, logTrace, logWarning, logWarning

Constructor Detail

KryoSerializer

public KryoSerializer(SparkConf conf)

Method Detail

maxBufferSizeMb

public int maxBufferSizeMb()

newKryoOutput

public com.esotericsoftware.kryo.io.Output newKryoOutput()

newKryo

public com.esotericsoftware.kryo.Kryo newKryo()

newInstance

public SerializerInstance newInstance()

Description copied from class: Serializer

Creates a new SerializerInstance.

Specified by:

newInstance in class Serializer

supportsRelocationOfSerializedObjects

public boolean supportsRelocationOfSerializedObjects()

Description copied from class: Serializer

:: Private :: Returns true if this serializer supports relocation of its serialized objects and false otherwise. This should return true if and only if reordering the bytes of serialized objects in serialization stream output is equivalent to having re-ordered those elements prior to serializing them. More specifically, the following should hold if a serializer supports relocation:

 serOut.open()
 position = 0
 serOut.write(obj1)
 serOut.flush()
 position = # of bytes writen to stream so far
 obj1Bytes = output[0:position-1]
 serOut.write(obj2)
 serOut.flush()
 position2 = # of bytes written to stream so far
 obj2Bytes = output[position:position2-1]
 serIn.open([obj2bytes] concatenate [obj1bytes]) should return (obj2, obj1)
 

In general, this property should hold for serializers that are stateless and that do not write special metadata at the beginning or end of the serialization stream.

This API is private to Spark; this method should not be overridden in third-party subclasses or called in user code and is subject to removal in future Spark releases.

See SPARK-7311 for more details.

Returns:

(undocumented)