The recommended way of starting and stopping a HistoryServer is through the scripts
start-history-server.sh and stop-history-server.sh. The path to a base log directory,
as well as any other relevant history server configuration, should be specified via
the $SPARK_HISTORY_OPTS environment variable. For example:
The recommended way of starting and stopping a HistoryServer is through the scripts start-history-server.sh and stop-history-server.sh. The path to a base log directory, as well as any other relevant history server configuration, should be specified via the $SPARK_HISTORY_OPTS environment variable. For example:
export SPARK_HISTORY_OPTS="-Dspark.history.fs.logDirectory=/tmp/spark-events" ./sbin/start-history-server.sh
This launches the HistoryServer as a Spark daemon.