Contents Menu Expand Light mode Dark mode Auto light/dark, in light mode Auto light/dark, in dark mode Skip to content
sparkctl documentation
sparkctl documentation

Contents:

  • How Tos
    • Getting Started
      • Installation
      • Deploy sparkctl in an HPC environment
    • Configuration
      • How to select compute nodes
      • Heterogeneous Slurm jobs
      • How to handle compute node failures
      • How to use a custom spark-default.conf file
      • How to set a custom Spark log level
    • Execution
      • Start a Spark Cluster
      • How to run Spark jobs in Python
      • How to to monitor Spark resource utilization
    • Applications
      • How to configure a Hive metastore
      • Visualize Data with Tableau
    • Debugging
  • Tutorials
    • Run Python jobs on a Spark Cluster in a script
    • Run Python jobs interactively on a Spark Cluster
    • Run Python jobs on a Spark Cluster with Spark Connect
    • Run jobs on a Spark Cluster with spark-submit or pyspark
  • Reference
    • sparkctl API
    • HPC
      • NREL Kestrel
    • CLI Reference
  • Explanation
Back to top
View this page

HPCΒΆ

This section contains how-tos for specific HPC environments.

Contents:

  • NREL Kestrel
    • Spark software
    • Compute nodes
Next
NREL Kestrel
Previous
sparkctl API
Copyright © 2025, Alliance for Sustainable Energy
Made with Sphinx and @pradyunsg's Furo