Tutorials

These tutorials guide you through running Spark jobs on HPC clusters using sparkctl. Each tutorial covers a different workflow - choose the one that best fits your needs.

Which Tutorial Should I Use?

Tutorial	Best For	Client Install	Interface
spark-submit / pyspark	Traditional Spark users, production jobs	sparkctl[pyspark] (full)	CLI
Spark Connect CLI	Lightweight client, remote connectivity	sparkctl	CLI
Python Library	Programmatic control, automation scripts	sparkctl	Python
Interactive Development	Exploratory analysis, debugging	sparkctl	Python REPL

Decision Guide

Start here if you’re new to sparkctl: spark-submit / pyspark - this is the most familiar workflow for existing Spark users.

Choose by use case:

• “I want to submit batch jobs” → spark-submit / pyspark

• “I want a minimal client installation” → Spark Connect CLI

• “I want to control the cluster from Python code” → Python Library

• “I want to explore data interactively” → Interactive Development

Contents:

• Run jobs with spark-submit or pyspark
• Run jobs with Spark Connect
• Run jobs in a Python script
• Run jobs interactively with sparkctl