Exercise 5: Writing a Firetask

This exercise bases on an extended version of the example from Exercise 2. It is implemented as a python script exercises/problems/5_author_firetask/recruiting-script.py. It has two new aspects:

  1. All parameters are loaded at the beginning from a JSON document parameters.json.
  2. The steps candidates_apply() and screen_candidates() are repeated as long as the size of the candidate list is smaller than number_to_invite.

Anatomy of Firetasks[edit | edit source]

All Firetasks are classes derived from the FiretaskBase class. Many built-in Firetasks are available in the upstream Fireworks, such as the ScriptTask, PyTask, FileTransferTask. Additional Firetasks can be written for both generic and specific purposes.

The skeleton of a Firetask looks like this:

from fireworks.core.firework import FiretaskBase, FWAction
from fireworks.utilities.fw_utilities import explicit_serialize

@explicit_serialize
class MyFiretask(FiretaskBase):
    """ My new Firetask """

    _fw_name = 'MyFiretask'
    required_params = ['par1', 'par2']
    optional_params = ['optional par']

    def run_task(self, fw_spec):
        """
        This is the method called upon Firetask execution. It has access
        to the spec through the fw_spec parameter. The required and the 
        optional parameters are accessed through self, i.e. they are class 
        attributes. Optionally this function returns a FWAction object to pass
        data to next Firetasks and Fireworks and/or to dynamically modify the
        workflow.
        """

        print(self['par1'], self['par2']) # print the values of required parameters
        print(self.fw_spec['name']) # print the name of the Firework

        if self.get('optional par'):
            actions = [
                'update_spec': {
                    'par1': self['par1'],
                    'optional par': self['optional par']
                }
            ]
        else:
            actions = []

        return FWAction(*actions)

Here a Firework using this Firetask:

- fw_id: 1
  name: Sample firework
  spec:
    _tasks:
    - _fw_name: {{custom_tasks.MyFiretask}}
      par1: data.json
      par2: data.yaml
      optional par: {}
NOTE: the python module custom_tasks must be in $PYTHONPATH in order to be correctly registered. For this reason keep the file custom_tasks.py either in lib or in exercises/work/5_author_firetask where you start the lpad and rlaunch commands. In the latter case you should add that directory to $PYTHONPATH with:
export PYTHONPATH=`pwd`:$PYTHONPATH

Problem 5.1: Data Loader Firetask[edit | edit source]

Write a DataLoaderTask to load the necessary parameters from the JSON document. If necessary the input data structure can be split into more than one file. The resulting workflow, which is given in exercises/problems/5_author_firetask/dataloader.json, must be successfully running with no further modifications.

Hint: You can use the load() method from the json package to load JSON documents as list or dictionaries and then return a FWAction object with update_spec and the structure (see above example).

Problem 5.2: Conditional Repeater Firetask[edit | edit source]

Write a RepeatIfLengthLesser Firetask that implements the while loop in the script. The Firetask should integrate into the workflow available in dataloader+repeater.json without further adaptations.

Hint: You can use the load_object function from fireworks.utilities.fw_serializers to construct a Firework object and the detours keyword argument of FWAction to insert the Firework dynamically:

firework = Firework(
    tasks=[load_object(task) for task in fw_spec['_tasks']],
    spec=fw_spec,
    name='repeat '+self['measure']
)
return FWAction(detours=firework)
 Previous