org.apache.spark.sql.execution
UnsafeRowSerializer
Related Doc:
package execution
the number of fields in the row being serialized.
Default ClassLoader to use in deserialization.
Default ClassLoader to use in deserialization. Implementations of Serializer should make sure it is using this when set.
Creates a new SerializerInstance.
Creates a new SerializerInstance.
Sets a class loader for the serializer to use in deserialization.
Sets a class loader for the serializer to use in deserialization.
this Serializer object
:: Private :: Returns true if this serializer supports relocation of its serialized objects and false otherwise.
:: Private :: Returns true if this serializer supports relocation of its serialized objects and false otherwise. This should return true if and only if reordering the bytes of serialized objects in serialization stream output is equivalent to having re-ordered those elements prior to serializing them. More specifically, the following should hold if a serializer supports relocation:
serOut.open() position = 0 serOut.write(obj1) serOut.flush() position = # of bytes writen to stream so far obj1Bytes = output[0:position-1] serOut.write(obj2) serOut.flush() position2 = # of bytes written to stream so far obj2Bytes = output[position:position2-1] serIn.open([obj2bytes] concatenate [obj1bytes]) should return (obj2, obj1)
In general, this property should hold for serializers that are stateless and that do not write special metadata at the beginning or end of the serialization stream.
This API is private to Spark; this method should not be overridden in third-party subclasses or called in user code and is subject to removal in future Spark releases.
See SPARK-7311 for more details.
Serializer for serializing UnsafeRows during shuffle. Since UnsafeRows are already stored as bytes, this serializer simply copies those bytes to the underlying output stream. When deserializing a stream of rows, instances of this serializer mutate and return a single UnsafeRow instance that is backed by an on-heap byte array.
Note that this serializer implements only the Serializer methods that are used during shuffle, so certain SerializerInstance methods will throw UnsupportedOperationException.