Exercise 2: Managing Data Flow

The purpose of this exercise is to learn how to pass data between Fireworks and describe data dependencies using the custom Firetask PythonFunctionTask:

- _fw_name: PythonFunctionTask
  function: any_module.any_function
  - first argument
  - second argument
  - data to forward

The keys specified in the inputs list must be available in the Firework spec at task run time. These are passed as positional arguments to the function. The returned outputs will be stored in the specs of the current Firework and of the child Fireworks under the keys specified in the outputs list.

Problem 2.1[edit | edit source]

Change to folder exercises/work/2_data_flow. Copy the provided template exercises/inputs/2_data_flow/template.[json|yaml] and complete it so that the script exercises/problems/2_data_flow/recruiting-script.py is implemented as a workflow. Check the workflow, add it to LaunchPad and run it in singleshot mode watching the changes in the Fireworks with the workflow run.

Problem 2.2[edit | edit source]

Copy the different solutions recruiting-[012].[json|yaml] from the folder exercises/problems/2_data_flow, and detect and correct the errors and run the workflow in rapidfire mode. Compare the corrected versions to each other and to your solution for Problem 2.1. Compare the results of two instances of the same workflow. Why do they differ?