Submitting New Jobs Using Job Classes

Univa Grid Engine 8.1 provides a number of new submit switches that can also be used in combination with the qalter command to adjust jobs derived from a job class. These new switches can also be used to adjust conventionally submitted jobs and allow therefore additional operations that were not possible in the past.

Using Job Classes to Submit New Jobs

Job Classes that are in the enabled and no conflict state can be used to create new jobs. To do this a user has to pass the -jc switch in combination with the name of a job class to a submit command like qsub. If the user has access to this job class then a new job will be created and all job template attributes that are defined in the job class will be used to initialize the corresponding parameters in the submitted job.

Depending on the access specifiers that are used in the job class it might be allowed to adjust certain parameters during the submission of the job. In this case additional switches and parameters might be passed to the submit command. All these additionally passed parameters will be used to adjust job parameters that where derived from the job class.

Additionally to the typical switches that are used to define job parameters there is a set of switches available that allow to remove parameters or to adjust parts of list based parameters in a job specification. The same set of switches can also be used with the modification command qalter to adjust job parameters after a job has already been created.

qsub/qalter -clearp <attr_name>

The -clearp switch allows to remove a job parameter from the specification of a job as if it was never specified. What this means depends on the job parameter that is specified by attr_name. For all those attributes that would normally have a default value this default value will be set for all others the corresponding attribute will be empty. Parameter names that can be specified for attr_name are all job class template attribute names.

qsub/qalter -clears <list_attr_name> <key>

This switch allows to remove a list entry in a list based attribute of a job specification. list_attr_name might be any name of a list-based job template attribute. key has to be the name of the key of the sublist entry for key/value pairs or the value itself that should be removed when the list contains only values

qsub/qalter -adds <list_attr_name> <key> <value>

-adds adds a new entry to a list based parameter.

qsub/qalter -mods <list_attr_name> <key> <value>

The -mods switch allows to modify the value of a key/value pair within a list based job parameter.

Example: Submit a Job Class Job and Adjust Some Parameters

Assume that the following job class is defined in you cluster:

   jcname          sleeper
   variant_list    default,short,long
   owner_list      NONE
   user_list       NONE
   xuser_list      NONE
   A               UNSPECIFIED
   a               UNSPECIFIED
   ar              UNSPECIFIED
   b               yes
   binding         UNSPECIFIED
   c_interval      UNSPECIFIED
   c_occasion      UNSPECIFIED
   CMDNAME         /bin/sleep
   CMDARG          60,[short=5],[long=3600]
   ckpt            UNSPECIFIED 
   ac              UNSPECIFIED
   cwd             UNSPECIFIED 
   display         UNSPECIFIED
   dl              UNSPECIFIED
   e               UNSPECIFIED
   h               UNSPECIFIED 
   hold_jid        UNSPECIFIED
   i               UNSPECIFIED
   j               UNSPECIFIED
   js              UNSPECIFIED
   l_hard          {~+}{~}a=true,b=true,{-}c=true
   l_soft          {+}UNSPECIFIED,[{~+}long={~}d=true]
   m               UNSPECIFIED
   M               UNSPECIFIED 
   masterq         UNSPECIFIED
   mbind           UNSPECIFIED
   N               {~-}Sleeper,[{~-}short=ShortSleeper],[{~-}long=LongSleeper]
   notify          UNSPECIFIED
   now             UNSPECIFIED
   o               UNSPECIFIED
   P               UNSPECIFIED
   p               UNSPECIFIED
   pe_name         UNSPECIFIED
   q_hard          UNSPECIFIED
   q_soft          UNSPECIFIED
   R               UNSPECIFIED
   r               UNSPECIFIED
   S               /bin/sh
   shell           UNSPECIFIED 
   V               UNSPECIFIED
   v               UNSPECIFIED

Now it is possible to submit jobs and to adjust the parameters of those jobs during the submission to fit specific needs:

qsub -jc sleeper -N MySleeper
qsub -jc sleeper.short -clearp N
qsub -jc sleeper.short -clears l_hard c -adds l_hard h_vmem 5G
qsub -jc sleeper.long -soft -l res_x=3

The first job that is submitted (1) will be derived from the sleeper.default job class variant but this job will get the name MySleeper.

Job (2) uses the sleeper.short job class but the job name is adjusted. The -clearp switch will remove the job name that is specified in the job class. Instead it will get the default job name that would have been assigned without specifying the name in any explicit way. This will be derived from the last part of the script command that will be executed. This script is /bin/sleep. So the job name of the new job will be sleep.

When job (3) is created the list of hard resource requirements is adjusted. The resource request c is removed and the h_vmem=5G resource request is added.

During the submission of job (4) The list of soft resource request is completely redefined. The use of the -l will completely replace already defined soft resource requests if any have been defined.

Please note that it is not allowed to trigger operations that would violate any access specifiers. In consequence, the following commands would be rejected:

qsub -jc sleeper -hard -l res_x 3
qsub -jc sleeper /bin/my_sleeper 61

Job (5) would remove the a and b resource requests. This is not allowed. Job (6) will be rejected because neither CMDNAME nor the CMDARGs are modifiable.

blog comments powered by Disqus