CustomReceiver

Instance Constructors

new CustomReceiver(host: String, port: Int)

Value Members

final def !=(arg0: Any): Boolean

Definition Classes
AnyRef → Any
final def ##(): Int

Definition Classes
AnyRef → Any
final def ==(arg0: Any): Boolean

Definition Classes
AnyRef → Any
final def asInstanceOf[T0]: T0

Definition Classes
Any
def clone(): AnyRef

Attributes
protected[java.lang]
Definition Classes
AnyRef
Annotations
@throws( ... )
final def eq(arg0: AnyRef): Boolean

Definition Classes
AnyRef
def equals(arg0: Any): Boolean

Definition Classes
AnyRef → Any
def finalize(): Unit

Attributes
protected[java.lang]
Definition Classes
AnyRef
Annotations
@throws( classOf[java.lang.Throwable] )
final def getClass(): Class[_]

Definition Classes
AnyRef → Any
def hashCode(): Int

Definition Classes
AnyRef → Any
def initializeLogIfNecessary(isInterpreter: Boolean, silent: Boolean = false): Boolean

Attributes
protected
Definition Classes
Logging
def initializeLogIfNecessary(isInterpreter: Boolean): Unit

Attributes
protected
Definition Classes
Logging
final def isInstanceOf[T0]: Boolean

Definition Classes
Any
def isStarted(): Boolean

Check if the receiver has started or not.
Check if the receiver has started or not.

Definition Classes
Receiver
def isStopped(): Boolean

Check if receiver has been marked for stopping.
Check if receiver has been marked for stopping. Use this to identify when the receiving of data should be stopped.

Definition Classes
Receiver
def isTraceEnabled(): Boolean

Attributes
protected
Definition Classes
Logging
def log: Logger

Attributes
protected
Definition Classes
Logging
def logDebug(msg: ⇒ String, throwable: Throwable): Unit

Attributes
protected
Definition Classes
Logging
def logDebug(msg: ⇒ String): Unit

Attributes
protected
Definition Classes
Logging
def logError(msg: ⇒ String, throwable: Throwable): Unit

Attributes
protected
Definition Classes
Logging
def logError(msg: ⇒ String): Unit

Attributes
protected
Definition Classes
Logging
def logInfo(msg: ⇒ String, throwable: Throwable): Unit

Attributes
protected
Definition Classes
Logging
def logInfo(msg: ⇒ String): Unit

Attributes
protected
Definition Classes
Logging
def logName: String

Attributes
protected
Definition Classes
Logging
def logTrace(msg: ⇒ String, throwable: Throwable): Unit

Attributes
protected
Definition Classes
Logging
def logTrace(msg: ⇒ String): Unit

Attributes
protected
Definition Classes
Logging
def logWarning(msg: ⇒ String, throwable: Throwable): Unit

Attributes
protected
Definition Classes
Logging
def logWarning(msg: ⇒ String): Unit

Attributes
protected
Definition Classes
Logging
final def ne(arg0: AnyRef): Boolean

Definition Classes
AnyRef
final def notify(): Unit

Definition Classes
AnyRef
final def notifyAll(): Unit

Definition Classes
AnyRef
def onStart(): Unit

This method is called by the system when the receiver is started.
This method is called by the system when the receiver is started. This function must initialize all resources (threads, buffers, etc.) necessary for receiving data. This function must be non-blocking, so receiving the data must occur on a different thread. Received data can be stored with Spark by calling store(data).
If there are errors in threads started here, then following options can be done (i) reportError(...) can be called to report the error to the driver. The receiving of data will continue uninterrupted. (ii) stop(...) can be called to stop receiving data. This will call onStop() to clear up all resources allocated (threads, buffers, etc.) during onStart(). (iii) restart(...) can be called to restart the receiver. This will call onStop() immediately, and then onStart() after a delay.

Definition Classes
CustomReceiver → Receiver
def onStop(): Unit

This method is called by the system when the receiver is stopped.
This method is called by the system when the receiver is stopped. All resources (threads, buffers, etc.) set up in onStart() must be cleaned up in this method.

Definition Classes
CustomReceiver → Receiver
def preferredLocation: Option[String]

Override this to specify a preferred location (hostname).
Override this to specify a preferred location (hostname).

Definition Classes
Receiver
def reportError(message: String, throwable: Throwable): Unit

Report exceptions in receiving data.
Report exceptions in receiving data.

Definition Classes
Receiver
def restart(message: String, error: Throwable, millisecond: Int): Unit

Restart the receiver.
Restart the receiver. This method schedules the restart and returns immediately. The stopping and subsequent starting of the receiver (by calling onStop() and onStart()) is performed asynchronously in a background thread.

Definition Classes
Receiver
def restart(message: String, error: Throwable): Unit

Restart the receiver.
Restart the receiver. This method schedules the restart and returns immediately. The stopping and subsequent starting of the receiver (by calling onStop() and onStart()) is performed asynchronously in a background thread. The delay between the stopping and the starting is defined by the Spark configuration spark.streaming.receiverRestartDelay. The message and exception will be reported to the driver.

Definition Classes
Receiver
def restart(message: String): Unit

Restart the receiver.
Restart the receiver. This method schedules the restart and returns immediately. The stopping and subsequent starting of the receiver (by calling onStop() and onStart()) is performed asynchronously in a background thread. The delay between the stopping and the starting is defined by the Spark configuration spark.streaming.receiverRestartDelay. The message will be reported to the driver.

Definition Classes
Receiver
def stop(message: String, error: Throwable): Unit

Stop the receiver completely due to an exception
Stop the receiver completely due to an exception

Definition Classes
Receiver
def stop(message: String): Unit

Stop the receiver completely.
Stop the receiver completely.

Definition Classes
Receiver
val storageLevel: StorageLevel

Definition Classes
Receiver
def store(bytes: ByteBuffer, metadata: Any): Unit

Store the bytes of received data as a data block into Spark's memory.
Store the bytes of received data as a data block into Spark's memory. The metadata will be associated with this block of data for being used in the corresponding InputDStream.

Definition Classes
Receiver
def store(bytes: ByteBuffer): Unit

Store the bytes of received data as a data block into Spark's memory.
Store the bytes of received data as a data block into Spark's memory. Note that the data in the ByteBuffer must be serialized using the same serializer that Spark is configured to use.

Definition Classes
Receiver
def store(dataIterator: Iterator[String], metadata: Any): Unit

Store an iterator of received data as a data block into Spark's memory.
Store an iterator of received data as a data block into Spark's memory. The metadata will be associated with this block of data for being used in the corresponding InputDStream.

Definition Classes
Receiver
def store(dataIterator: Iterator[String]): Unit

Store an iterator of received data as a data block into Spark's memory.
Store an iterator of received data as a data block into Spark's memory.

Definition Classes
Receiver
def store(dataIterator: Iterator[String], metadata: Any): Unit

Store an iterator of received data as a data block into Spark's memory.
Store an iterator of received data as a data block into Spark's memory. The metadata will be associated with this block of data for being used in the corresponding InputDStream.

Definition Classes
Receiver
def store(dataIterator: Iterator[String]): Unit

Store an iterator of received data as a data block into Spark's memory.
Store an iterator of received data as a data block into Spark's memory.

Definition Classes
Receiver
def store(dataBuffer: ArrayBuffer[String], metadata: Any): Unit

Store an ArrayBuffer of received data as a data block into Spark's memory.
Store an ArrayBuffer of received data as a data block into Spark's memory. The metadata will be associated with this block of data for being used in the corresponding InputDStream.

Definition Classes
Receiver
def store(dataBuffer: ArrayBuffer[String]): Unit

Store an ArrayBuffer of received data as a data block into Spark's memory.
Store an ArrayBuffer of received data as a data block into Spark's memory.

Definition Classes
Receiver
def store(dataItem: String): Unit

Store a single item of received data to Spark's memory.
Store a single item of received data to Spark's memory. These single items will be aggregated together into data blocks before being pushed into Spark's memory.

Definition Classes
Receiver
def streamId: Int

Get the unique identifier the receiver input stream that this receiver is associated with.
Get the unique identifier the receiver input stream that this receiver is associated with.

Definition Classes
Receiver
final def synchronized[T0](arg0: ⇒ T0): T0

Definition Classes
AnyRef
def toString(): String

Definition Classes
AnyRef → Any
final def wait(): Unit

Definition Classes
AnyRef
Annotations
@throws( ... )
final def wait(arg0: Long, arg1: Int): Unit

Definition Classes
AnyRef
Annotations
@throws( ... )
final def wait(arg0: Long): Unit

Definition Classes
AnyRef
Annotations
@throws( ... )

Related Docs: object CustomReceiver | package streaming

class CustomReceiver extends Receiver[String] with internal.Logging

Instance Constructors

new CustomReceiver(host: String, port: Int)

Value Members

final def !=(arg0: Any): Boolean

final def ##(): Int

final def ==(arg0: Any): Boolean

final def asInstanceOf[T0]: T0

def clone(): AnyRef

final def eq(arg0: AnyRef): Boolean

def equals(arg0: Any): Boolean

def finalize(): Unit

final def getClass(): Class[_]

def hashCode(): Int

def initializeLogIfNecessary(isInterpreter: Boolean, silent: Boolean = false): Boolean

def initializeLogIfNecessary(isInterpreter: Boolean): Unit

final def isInstanceOf[T0]: Boolean

def isStarted(): Boolean

def isStopped(): Boolean

def isTraceEnabled(): Boolean

def log: Logger

def logDebug(msg: ⇒ String, throwable: Throwable): Unit

def logDebug(msg: ⇒ String): Unit

def logError(msg: ⇒ String, throwable: Throwable): Unit

def logError(msg: ⇒ String): Unit

def logInfo(msg: ⇒ String, throwable: Throwable): Unit

def logInfo(msg: ⇒ String): Unit

def logName: String

def logTrace(msg: ⇒ String, throwable: Throwable): Unit

def logTrace(msg: ⇒ String): Unit

def logWarning(msg: ⇒ String, throwable: Throwable): Unit

def logWarning(msg: ⇒ String): Unit

final def ne(arg0: AnyRef): Boolean

final def notify(): Unit

final def notifyAll(): Unit

def onStart(): Unit

def onStop(): Unit

def preferredLocation: Option[String]

def reportError(message: String, throwable: Throwable): Unit

def restart(message: String, error: Throwable, millisecond: Int): Unit

def restart(message: String, error: Throwable): Unit

def restart(message: String): Unit

def stop(message: String, error: Throwable): Unit

def stop(message: String): Unit

val storageLevel: StorageLevel

def store(bytes: ByteBuffer, metadata: Any): Unit

def store(bytes: ByteBuffer): Unit

def store(dataIterator: Iterator[String], metadata: Any): Unit

def store(dataIterator: Iterator[String]): Unit

def store(dataIterator: Iterator[String], metadata: Any): Unit

def store(dataIterator: Iterator[String]): Unit

def store(dataBuffer: ArrayBuffer[String], metadata: Any): Unit

def store(dataBuffer: ArrayBuffer[String]): Unit

def store(dataItem: String): Unit

def streamId: Int

final def synchronized[T0](arg0: ⇒ T0): T0

def toString(): String

final def wait(): Unit

final def wait(arg0: Long, arg1: Int): Unit

final def wait(arg0: Long): Unit

Inherited from internal.Logging

Inherited from Receiver[String]

Inherited from Serializable

Inherited from Serializable

Inherited from AnyRef

Inherited from Any

Ungrouped