How to set a custom Spark log level¶
By default, Spark logs messages at the INFO level. This can be very verbose when running jobs
through spark-submit.
Here’s how to reduce the verbosity:
$ sparkctl configure --spark-log-level warn
By default, Spark logs messages at the INFO level. This can be very verbose when running jobs
through spark-submit.
Here’s how to reduce the verbosity:
$ sparkctl configure --spark-log-level warn