How to set a custom Spark log level¶
By default, Spark logs messages at the INFO
level. This can be very verbose when running jobs
through spark-submit
.
Here’s how to reduce the verbosity:
$ sparkctl configure --spark-log-level warn
By default, Spark logs messages at the INFO
level. This can be very verbose when running jobs
through spark-submit
.
Here’s how to reduce the verbosity:
$ sparkctl configure --spark-log-level warn