org.apache.spark.sql.execution.streaming
StreamingQueryListenerBus
Related Doc:
package streaming
Add a listener to listen events.
Add a listener to listen events. This method is thread-safe and can be called in any thread.
Dispatch events to registered StreamingQueryListeners.
Dispatch events to registered StreamingQueryListeners. Only the events associated queries started in the same SparkSession as this ListenerBus will be dispatched to the listeners.
Called when the application ends
Called when the application ends
Called when the application starts
Called when the application starts
Called when a new block manager has joined
Called when a new block manager has joined
Called when an existing block manager has been removed
Called when an existing block manager has been removed
Called when the driver receives a block update info.
Called when the driver receives a block update info.
Called when environment properties have been updated
Called when environment properties have been updated
Called when the driver registers a new executor.
Called when the driver registers a new executor.
Called when the driver receives task metrics from an executor in a heartbeat.
Called when the driver receives task metrics from an executor in a heartbeat.
Called when the driver removes an executor.
Called when the driver removes an executor.
Called when a job ends
Called when a job ends
Called when a job starts
Called when a job starts
Called when other events like SQL-specific events are posted.
Called when other events like SQL-specific events are posted.
Called when a stage completes successfully or fails, with information on the completed stage.
Called when a stage completes successfully or fails, with information on the completed stage.
Called when a stage is submitted
Called when a stage is submitted
Called when a task ends
Called when a task ends
Called when a task begins remotely fetching its result (will not be called for tasks that do not need to fetch the result remotely).
Called when a task begins remotely fetching its result (will not be called for tasks that do not need to fetch the result remotely).
Called when a task starts
Called when a task starts
Called when an RDD is manually unpersisted by the application
Called when an RDD is manually unpersisted by the application
Post a StreamingQueryListener event to the added StreamingQueryListeners.
Post a StreamingQueryListener event to the added StreamingQueryListeners. Note that only the QueryStarted event is posted to the listener synchronously. Other events are dispatched to Spark listener bus. This method is guaranteed to be called by queries in the same SparkSession as this listener.
Override the parent postToAll to remove the query id from activeQueryRunIds after all
the listeners process QueryTerminatedEvent.
Override the parent postToAll to remove the query id from activeQueryRunIds after all
the listeners process QueryTerminatedEvent. (SPARK-19594)
Remove a listener and it won't receive any events.
Remove a listener and it won't receive any events. This method is thread-safe and can be called in any thread.
A bus to forward events to StreamingQueryListeners. This one will send received StreamingQueryListener.Events to the Spark listener bus. It also registers itself with Spark listener bus, so that it can receive StreamingQueryListener.Events and dispatch them to StreamingQueryListeners.
Note that each bus and its registered listeners are associated with a single SparkSession and StreamingQueryManager. So this bus will dispatch events to registered listeners for only those queries that were started in the associated SparkSession.