Scheduler Commands Cheat Sheet: Difference between revisions

From HPCWIKI
Jump to navigation Jump to search
No edit summary
No edit summary
 
(2 intermediate revisions by the same user not shown)
Line 1: Line 1:
=== Scheduler Command Cheat Sheet ===
=== Scheduler Command Cheat Sheet<ref>https://wiki.anunna.wur.nl/index.php/SLURM_Compare</ref> ===
PBS/Torque, Slurm, LSF, SGE and LoadLeveler Rosetta Stone
 
This table lists the most common command, environment variables, and job specification options used by the major workload management systems: PBS/Torque, Slurm, LSF, SGE and LoadLeveler. Each of these workload managers has unique features, but the most commonly used functionality is available in all of these environments as listed in the table. This should be considered a work in progress and contributions to improve the document are welcome.
{| class="wikitable"
{| class="wikitable"
|+
!User Commands
|User Commands
!'''PBS/Torque'''
|PBS/Torque
!'''Slurm'''
|Slurm
!LSF
|
!'''SGE'''
|LSF
!LoadLeveler
|SGE
|
|
|Loadleveler
|-
|-
|Job submission
|Job submission
|qsub [script file)
|qsub [script_file]
|sbatch [script file)
|sbatch [script_file]
|
|bsub [script_file]
|bsub [script file)
|qsub [script_file]
| colspan="3" |qsub [script file)
|llsubmit [script_file]
|llsubmit [script file)
|-
|-
|Job deletion
|Job deletion
|qdel uob idl
|qdel [job_id]
|scancel fiob idl
|scancel [job_id]
|
|bkill [job_id]
|bkill Uob idl
|qdel [job_id]
| colspan="2" |qdel uob idl
|llcancel [job_id]
|
|llcancel fjob idl
|-
|-
|Job status (b)'job)
|Job status (by job)
|qstat uob id]
|qstat [job_id]
|squeue uob id]
|squeue --job [job_id]
|
|bjobs [job_id]
|bjobs uob id]
|qstat -u \* [-j job_id]
| colspan="3" |qstat -u \ * [-j job id]
|llq -u [username]
|llq -u [username]
|-
|-
|Job status (b}' user}
|Job status (by user)
|qstat -u [user name]
|qstat -u [user_name]
|squeue -u [user name]
|squeue -u [user_name]
|
|bjobs -u [user_name]
|bjobs -u [user name)
|qstat [-u user_name]
| colspan="3" |qstat [-u user name)
|llq -u [user_name]
|llq -u [user name)
|-
|-
|Job hold
|Job hold
|qhold uob id]
|qhold [job_id]
|scontrol hold uob id]
|scontrol hold [job_id]
|
|bstop [job_id]
|bstop uob id]
|qhold [job_id]
| colspan="3" |qhold uob id]
|llhold -r [job_id]
|llhold -r uob id]
|-
|-
|Job release
|Job release
|qrls uob id]
|qrls [job_id]
| colspan="2" |scontrol release uob id]
|scontrol release [job_id]
|bresume uob id]
|bresume [job_id]
| colspan="2" |qrls uob id]
|qrls [job_id]
|
|llhold -r [job_id]
|llhold -r uob id]
|-
|-
|Queue list
|Queue list
|qstat -Q
|qstat -Q
|squeue
|squeue
|
|bqueues
|bqueues
| colspan="2" |qconf -sql
|qconf -sql
|
|llclass
|II class
|-
|-
|Node list
|Node list
|pbsnodes -I
|pbsnodes -l
| colspan="2" |sinfo -N OR scontrol show nodes
|sinfo -N OR scontrol show nodes
|bhosts
|bhosts
|qhost
|qhost
|
|
|llstatus -L machine
|llstatus -L machine
|-
|-
|Clusterstatus
|Cluster status
|qstat -a
|qstat -a
|sinfo
|sinfo
|
|bqueues
|bqueues
| colspan="2" |qhost -q
|qhost -q
|
|llstatus -L cluster
|llstatus -L cluster
|-
|-
Line 91: Line 77:
|xpbsmon
|xpbsmon
|sview
|sview
|
|xlsf OR xlsbatch
|xlsf OR xlsbatch
|qmon
|qmon
|
|
|xload
|xload
|-
|-
|Environment
|'''Environment'''
|PBS/Torque
|'''PBS/Torque'''
|Slurm
|'''Slurm'''
|
|'''LSF'''
|LSF
|'''SGE'''
|SGE
|'''LoadLeveler'''
|
|
|Load le veler
|-
|-
|Job ID
|Job ID
|$PBS JOBID
|$PBS_JOBID
|$SLURM JOBID
|$SLURM_JOBID
|
|$LSB_JOBID
|$LSB JOBID
|$JOB_ID
| colspan="3" |$JOB id
|$LOAD_STEP_ID
|$LOAD STEP ID
|-
|-
|Submit Directory
|Submit Directory
|$PBS O WORKDIR
|$PBS_O_WORKDIR
|$SLURM SUBMIT DIR
|$SLURM_SUBMIT_DIR
|
|$LSB_SUBCWD
|$LSB SUBCWD
|$SGE_O_WORKDIR
|$SGE
|$LOADL_STEP_INITDIR
|0
|WORKDIR
|$LOADL STEP INITDIR
|-
|-
|Submit Host
|Submit Host
|$PBS O HOST
|$PBS_O_HOST
| colspan="2" |$SLURM SUBMIT HOST
|$SLURM_SUBMIT_HOST
|$LSB SUB HOST
|$LSB_SUB_HOST
| colspan="3" |$SGE O HOST
|$SGE_O_HOST
|N/A
|
|-
|-
|Node List
|Node List
|$PBS NODEFILE
|$PBS_NODEFILE
| colspan="2" |$SLURM JOB NODELIST
|$SLURM_JOB_NODELIST
|$LSB HOSTS/LSB MCPU HOST
|$LSB_HOSTS/LSB_MCPU_HOST
| colspan="3" |$PE HOSTFILE
|$PE_HOSTFILE
|$LOADL PROCESSOR LIST
|$LOADL_PROCESSOR_LIST
|-
|-
|Job Array Index
|Job Array Index
|$PBS ARRA YID
|$PBS_ARRAYID
|$SLURM ARRAY TASK
|$SLURM_ARRAY_TASK_ID
|ID
|$LSB_JOBINDEX
|$LSB JOBINDEX
|$SGE_TASK_ID
| colspan="2" |$SGE TASK
|ID
|
|-
|Job Specification
|PBS/Torque
|Slurm
|
|LSF
|SGE
|
|
|Load leveler
|-
|Script directive
|#PBS
|#SBATCH
|
|
|#BSUB
|#$
|
|
|#@
|-
|Queue
| -q [queue]
| -p [queue)
|
| -q [queue]
| colspan="2" | -q [queue)
|
|class = [queue)
|-
|Node Count
| -I nodes = [count]
| -N [min[-max])
|
| -n [count]
|N/A
|
|
|node=[count]
|-
|CPU count
| -I ppn = [count] OR
| -n [count]
|
| -n [count]
| colspan="3" | -pe [PE) [count]
|N/A
|-
|
| -I mppwidth = [PE count]
|
|
|
|
|
|
|
|-
|Wall Clock Limit
| -I walltime = [hh:mm:ss)
| colspan="2" | -t [min] OR -t [days-hh:mm:ss)
| -W [hh:mm:ss)
| colspan="3" | -I h_rt = [seconds)
|wall clock limit = [hh:mm:ss]
|-
|Standard Output File
| -o [file name)
| -o [file name]
|
| -o [file name]
| colspan="3" | -o [file name)
|output = [file name)
|-
|Standard Error File
| -e [file name]
| -e [file name]
|
| -e [file name]
| colspan="3" | -e [file name)
|error = [file name)
|-
|Combine stdout/err
| -j oe (both to stdout) OR
|(use -o without -e)
|
|(use -o without -e)
| -j yes
|
|
|N/A
|-
|
| -j eo (both to stderr)
|
|
|
|
|
|
|
|-
|Copy Environment
| -V
| colspan="2" | --export = [ALL I NONE !variables)
|N/A
| -V
|
|
|Environment=COPY ALL
|-
|Event Notification
| -m abe
| --mail-type = [events]
|
| -B or -N
| colspan="2" | -m abe
|
|notification = startI errorI complete I never I always
|-
|EmailAddress
| -M [address)
| --mail-user = [address)
|
| -u [address]
| colspan="3" | -M [address]
|Notify_ user=[ address I
|-
|Job Name
| -N [name)
| --job-name = [name)
|
| -J [name)
| colspan="2" | -N [name)
|
|job name = [name)
|-
|Job Restart
| -r [vi n]
| colspan="2" | --requeue OR --no-requeue
| -r
| colspan="2" | -r [yes I no]
|
|Restart = [yes I no]
|-
|
|
| colspan="2" |(NOTE: configurable default)
|
|
|
|
|
|-
|Working Directory
|N/A
| --workdir = [dir _name)
|
|(submission directory)
| colspan="3" | -wd [directory]
|initiald ir=[ directory]
|-
|Resource Sharing
| -I naccesspolicy = singlejob
| --exclusive OR --shared
|
| -x
| colspan="2" | -I exclusive
|
|node_usage=not_shared
|-
|Memory Size
| -I mem = [MB]
| colspan="2" | --mem = [mem)[M I GIT] OR
| -M [MB)
| colspan="3" | -I mem_free = [memory][K IMIG)
|requirements = (Memory> = [MB])
|-
|
|
| colspan="2" | --mem-per-cpu = [mem] [MIG IT]
|
|
|
|
|
|-
|Accou ntto charge
| -W group_list = [account]
| --account = [account]
|
| -P [account]
| colspan="2" | -A [account]
|
|n/a
|-
|Tasks Per Node
| -I mppnppn [PEs_per node]
| colspan="2" | --tasks-per-node = [count]
|N/A
| colspan="3" |(Fixed allocation_rule in PE)
|tasks per node = [count]
|-
|CPUs Per Task
|n/a
| colspan="2" | --cp us-per-task = [count)
|n/a
|n/a
|
|
|n/a
|-
|Job Dependency
| -d [job id]
| colspan="2" | --depend = [state:job id]
| -w [donelexitlfinish)
| colspan="3" | -holdjid [job id I job name]
|n/a
|-
|Job Project
|n/a
| --wckey = [name]
|
| -P [name)
| colspan="2" | - P [name]
|
|n/a
|-
|Job host preference
|n/a
| colspan="2" | --nodelist = [nodes)AND/OR -exclude = [nodes)
| -m [nodes)
| colspan="3" | -q[queue)@[node) OR -q [queue)@@[hostgroup]
|n/a
|-
|Quality of Service
| -I qos = [name)
| --qos = [name]
|
|n/a
|n/a
|
|
|n/a
|-
|Job Arrays
| -t [array_spec)
| colspan="2" | --array = [array _spec) (Siu rm version 2.6+)
|J "name[array_ spec]"
| colspan="3" | -t [array_ spec)
|n/a
|-
|Generic Resources
| -I other = [resource_spec)
| --gres = [resource_ spec I
|
|n/a
| colspan="3" | -I [resource] = [value)
|n/a
|-
|Licenses
|n/a
| colspan="2" | --licenses = [license_spec)
| -R "rusage[license_spec)"
| colspan="3" | -I [license] = [count]
|n/a
|-
|Begin Time
| -A "YYYY-MM-DD HH:MM:SS"
| colspan="2" | --begin = vvvY-MM-DD[THH:MM[:SS))
| -b[[year:] [month:] day:]hour:minute
| colspan="3" | -a [YYMMDDhhmm)
|n/a
|}
|}

Latest revision as of 09:24, 21 March 2023

Scheduler Command Cheat Sheet[1]

PBS/Torque, Slurm, LSF, SGE and LoadLeveler Rosetta Stone

This table lists the most common command, environment variables, and job specification options used by the major workload management systems: PBS/Torque, Slurm, LSF, SGE and LoadLeveler. Each of these workload managers has unique features, but the most commonly used functionality is available in all of these environments as listed in the table. This should be considered a work in progress and contributions to improve the document are welcome.

User Commands PBS/Torque Slurm LSF SGE LoadLeveler
Job submission qsub [script_file] sbatch [script_file] bsub [script_file] qsub [script_file] llsubmit [script_file]
Job deletion qdel [job_id] scancel [job_id] bkill [job_id] qdel [job_id] llcancel [job_id]
Job status (by job) qstat [job_id] squeue --job [job_id] bjobs [job_id] qstat -u \* [-j job_id] llq -u [username]
Job status (by user) qstat -u [user_name] squeue -u [user_name] bjobs -u [user_name] qstat [-u user_name] llq -u [user_name]
Job hold qhold [job_id] scontrol hold [job_id] bstop [job_id] qhold [job_id] llhold -r [job_id]
Job release qrls [job_id] scontrol release [job_id] bresume [job_id] qrls [job_id] llhold -r [job_id]
Queue list qstat -Q squeue bqueues qconf -sql llclass
Node list pbsnodes -l sinfo -N OR scontrol show nodes bhosts qhost llstatus -L machine
Cluster status qstat -a sinfo bqueues qhost -q llstatus -L cluster
GUI xpbsmon sview xlsf OR xlsbatch qmon xload
Environment PBS/Torque Slurm LSF SGE LoadLeveler
Job ID $PBS_JOBID $SLURM_JOBID $LSB_JOBID $JOB_ID $LOAD_STEP_ID
Submit Directory $PBS_O_WORKDIR $SLURM_SUBMIT_DIR $LSB_SUBCWD $SGE_O_WORKDIR $LOADL_STEP_INITDIR
Submit Host $PBS_O_HOST $SLURM_SUBMIT_HOST $LSB_SUB_HOST $SGE_O_HOST
Node List $PBS_NODEFILE $SLURM_JOB_NODELIST $LSB_HOSTS/LSB_MCPU_HOST $PE_HOSTFILE $LOADL_PROCESSOR_LIST
Job Array Index $PBS_ARRAYID $SLURM_ARRAY_TASK_ID $LSB_JOBINDEX $SGE_TASK_ID
  1. ↑ https://wiki.anunna.wur.nl/index.php/SLURM_Compare
Retrieved from "http://wiki.hpcmate.com/index.php?title=Scheduler_Commands_Cheat_Sheet&oldid=261"