Contents Menu Expand Light mode Dark mode Auto light/dark, in light mode Auto light/dark, in dark mode Skip to content
sparkctl documentation
sparkctl documentation

Contents:

  • How Tos
    • Getting Started
      • Installation
      • Deploy sparkctl in an HPC environment
    • Configuration
      • How to select compute nodes
      • Heterogeneous Slurm jobs
      • How to handle compute node failures
      • How to use a custom spark-default.conf file
      • How to set a custom Spark log level
    • Execution
      • Start a Spark Cluster
      • How to run Spark jobs in Python
      • How to to monitor Spark resource utilization
    • Applications
      • How to configure a Hive metastore
      • Visualize Data with Tableau
    • Debugging
  • Tutorials
    • Run Python jobs on a Spark Cluster in a script
    • Run Python jobs interactively on a Spark Cluster
    • Run Python jobs on a Spark Cluster with Spark Connect
    • Run jobs on a Spark Cluster with spark-submit or pyspark
  • Reference
    • sparkctl API
    • HPC
      • NREL Kestrel
    • CLI Reference
  • Explanation
Back to top
View this page

Getting StartedΒΆ

Contents:

  • Installation
  • Deploy sparkctl in an HPC environment
    • URLs
    • sparkctl configuration file
Next
Installation
Previous
How Tos
Copyright © 2025, Alliance for Sustainable Energy
Made with Sphinx and @pradyunsg's Furo