Playing with the Paths of Step functions

AWS Step Functions work by moving from one state to another state. States need a way to communicate with each other - DATA needs to be passed between them
All such data needs to be JSON. From the docs

When a state machine is started, the caller can provide an initial JSON text as input,
 which is passed to the machine's start state as input. If no input is provided, the 
default is an empty JSON object, {}. As each state is executed, it receives a JSON text 
as input and can produce arbitrary output, which MUST be a JSON text. When two states 
are linked by a transition, the output from the first state is passed as input to the 
second state. The output from the machine's terminal state is treated as its output.

For a state to refer to Json data it uses a syntax called JsonPath
However there is an additional constrain on JsonPath used in Step Functions:

When a state machine is started, the caller can provide an initial JSON text as input,
which is passed to the machine's start state as input. If no input is provided, the 
default is an empty JSON object, {}. As each state is executed, it receives a JSON 
text as input and can produce arbitrary output, which MUST be a JSON text. When two 
states are linked by a transition, the output from the first state is passed as input 
to the second state. The output from the machine's terminal state is treated as its output.

I decided to play around with the data flow in this post.
I wrote a Car assembly Step function which takes as input customer selection on color and car series and generates a spec chart based on input:
Sample Input :

{
  "color": "silver",
  "class" :"LX"
}

My Step Function:

My Step Function basically generates the below car configuration based on customer input.

{
  "car":{
    "make":"Honda",
    "model":"Accord"
  },
  "customerChoice":{
    "color":"silver",
    "trim":"LX"
  },
  "structure":{
    "door":"4",
    "engine":"front",
    "body":"aluminium"
  },
  "engine":{
    "horsepower ":"192 to 252 hp",
    "type ":"gas",
    "brand":"turbo"
  },
  "cylinders":{
    "transmission ":"6-speed manual",
    "type ":"Inline",
    "count":4
  },
  "tyres":{
    "wheelDimension":19,
    "type ":"All Season",
    "MountedSpareAvailable":true
  }
}

However this did not go as imagined.
Step Function xml is as below:

Notice that there are no Lambdas or Activities used here. The Step Function inly uses Pass states. Each state generates a configuration for the car. The step function includes a wait state before it reaches completion.

Learning No 1: Result field can only refer to static values.
For the PaintConfigurations state, I was attempting to generate the json based on the input 'color' attribute. 

  {
    "color":"silver",
    "trim":"LX"
  }

My Pass state looked something like this:

    "PaintConfigurations": {
      "Type": "Pass",
      "Result": {
        "color ": "$.color",
        "style ": "lustre"
      },
      "ResultPath": "$.specs.paint",
      "Next": "SpecialConfigurationsCheck"
    },

However on running this the output comes out as:

    "paint": {
        "color ": "$.color",
        "style ": "lustre"
      },

I even tried the color.$ syntax - something I used for AWS Service Integrations, but that only changed my output to be

"paint": {
        "color.$": "$.color",
        "style ": "lustre"
      },

As can be seen Result in Pass State cannot include any evaluation logic. This fits with AWS Docs definition

The Pass State (identified by "Type":"Pass") simply passes its input to its

 output, performing no work.

I had a similar requirement with ModelConfigurations where I wanted to include the "trim" information from user input into the result generated by the ModelConfigurations state. However being a Pass state no dynamic work was possible.
Learning 2: Parallel states executions leads to multiple outputs being sent to downstream
This is something that I did not see coming. Consider the parallel state we added in our workflow.

The step that follows the parallel state receives the input after all branches complete.

A Parallel state causes the interpreter to execute each branch starting with 
the state named in its “StartAt” field, as concurrently as possible, and wait 
until each branch terminates (reaches a terminal state) before processing
 the Parallel state's “Next” field.

The input received by the following state is :

This broke my Step Function. To solve this I decided to add a combine step here - a Lambda that took the input to generate the output. Similarly to solve the Paint configuration problem, I decided the same Lambda which will create the Result that I need.
I also wanted to add some data to a previous pass state result - ModelConfigurations.
Here is the Lambda:

public class PaintProcessor implements RequestHandler<List, Map> {

    private ObjectMapper mapper = new ObjectMapper();
    private final String SPECS_KEY = "specs";

    @Override
    public Map handleRequest(List input, Context context) {
        LambdaLogger logger = context.getLogger();
        logger.log("In Handler: Executing " + context.getFunctionName() 

               + ", " + context.getFunctionVersion());
        logger.log(input.toString());
        Map<String, Object> flatMap = flattenMap(input);
        addPaintConfigurations(flatMap);
        updateTrimConfigurations(flatMap);
        return flatMap;
    }

    private void addPaintConfigurations(Map<String, Object> flatMap) {
        Map<String, String> paintConfigurations = new HashMap<>();
        paintConfigurations.put("color", (String) flatMap.get("color"));
        paintConfigurations.put("style", "lustre");
        ((Map<String, Object>) flatMap.get(SPECS_KEY)).put("paint",

                 paintConfigurations);
    }


    private void updateTrimConfigurations(Map<String, Object> flatMap) {
        Map<String, String> carConfigurations = (Map<String, String>) 

                ((Map<String, Object>) flatMap.get(SPECS_KEY)).get("car");
        carConfigurations.put("trim", (String) flatMap.get("class"));
    }


    private Map<String, Object> flattenMap(List<Map<String, Object>> 

                 inputMessages) {
        Map<String, Object> result = new HashMap<>();
        for (Map inputKey : inputMessages) {
            for (Object key : inputKey.keySet()) {
                if (key.equals(SPECS_KEY)) {
                    Map specMap = (Map) inputKey.get(SPECS_KEY);
                    Map existingSpecMap = (Map) result.get(SPECS_KEY);
                    if (existingSpecMap == null) {
                        result.put(SPECS_KEY, specMap);
                    } else {
                        existingSpecMap.putAll(specMap);
                    }
                } else {
                    result.put(key.toString(), inputKey.get(key));
                }
            }
        }
        return result;
    }
}

The Lambda gets the List of specs from the parallel execution which it merges and then adds the paint configuration. It then pulls in the trim information into car configuration.
The output from My Lambda is:

Learning 3: Wait States can be dynamic.
I initially setup my wait state to be simple 2 second wait. But there is also a configuration to make the wait state wait depending on the input to the state. I updated my State machine input to include a wait field

"WaitForPublish": {
      "Type": "Wait",
      "SecondsPath": "$.pubDelay",
      "Next": "FinishSpecs"
    }

Learning 4: Output Path versus State Path
There is a good pictorial representation on the various Path variables available to a State here.

The output of a state can be a copy of its input, the result it produces (for 

example, output from a Task state’s Lambda function), or a combination of its 

input and result. Use ResultPath to control which combination of these is passed 

to the state output.

For most of my states I used the Result Path to include my State's input with the output (See here for ResultPath capabilities).
For my last state however, I did not want to output the whole Result till now, but only a selection of the result. This was achieved using the OutputPath

OutputPath enables you to select a portion of the state output to pass to the 

next state. This enables you to filter out unwanted information, and pass only 

the portion of JSON that you care about.

This is how the final state looks.

"FinishSpecs": {
      "Type": "Pass",
      "OutputPath": "$.specs",
      "End": true
    }

It takes the existing input to the state and only retains the value represented by my specs path.
My final State Machine is as below:

{
  "Comment": "Honda Accord Assembly",
  "StartAt": "ModelConfigurations",
  "States": {
    "ModelConfigurations": {
      "Type": "Pass",
      "Result": {
        "make": "Honda",
        "model": "Accord"
      },
      "ResultPath": "$.specs.car",
      "Next": "StructureConfigurations"
    },
    "StructureConfigurations": {
      "Type": "Pass",
      "Result": {
        "door": "4",
        "engine": "front",
        "body": "aluminium"
      },
      "ResultPath": "$.specs.structure",
      "Next": "InternalsCombination"
    },
    "InternalsCombination": {
      "Type": "Parallel",
      "Branches": [
        {
          "StartAt": "EngineConfigurations",
          "States": {
            "EngineConfigurations": {
              "Type": "Pass",
              "Result": {
                "horsepower": "192 to 252 hp",
                "type ": "gas",
                "brand": "turbo"
              },
              "ResultPath": "$.specs.engine",
              "Next": "CylinderConfiguration"
            },
            "CylinderConfiguration": {
              "Type": "Pass",
              "Result": {
                "transmission ": "6-speed manual",
                "type ": "Inline",
                "count": 4
              },
              "ResultPath": "$.specs.cylinders",
              "End": true
            }
          }
        },
        {
          "StartAt": "TyreConfiguration",
          "States": {
            "TyreConfiguration": {
              "Type": "Pass",
              "Result": {
                "wheelDimension": 19,
                "type ": "All Season",
                "MountedSpareAvailable": true
              },
              "ResultPath": "$.specs.tyres",
              "End": true
            }
          }
        }
      ],
      "Next": "PaintConfigurations"
    },
    "PaintConfigurations": {
      "Type": "Task",
      "Resource": "arn:aws:lambda:us-east-1:ActNo:function:ConfigureAndPaint:$LATEST",
      "Next": "SpecialConfigurationsCheck"
    },
    "SpecialConfigurationsCheck": {
      "Type": "Choice",
      "Choices": [
        {
          "Not": {
            "And": [
              {
                "Variable": "$.class",
                "StringEquals": "EX"
              },
              {
                "Variable": "$.class",
                "StringEquals": "VX"
              }
            ]
          },
          "Next": "StandardConfigurations"
        },
        {
          "Variable": "$.class",
          "StringEquals": "EX",
          "Next": "EXConfigurations"
        },
        {
          "Variable": "$.class",
          "StringEquals": "VX",
          "Next": "VXConfigurations"
        }
      ],
      "Default": "SpecialConfigCheckErrorState"
    },
    "StandardConfigurations": {
      "Type": "Pass",
      "Next": "WaitForPublish",
      "Result": {
        "Alloy Wheels": false,
        "WarrantyYears": 2,
        "seatCover": "cloth"
      },
      "ResultPath": "$.specs.extra"
    },
    "EXConfigurations": {
      "Type": "Pass",
      "Next": "WaitForPublish",
      "Result": {
        "Alloy Wheels": true,
        "WarrantyYears": 4,
        "seatCover": "leather",
        "Driver Seat": "6-Way Power"
      },
      "ResultPath": "$.specs.extra"
    },
    "VXConfigurations": {
      "Type": "Pass",
      "Next": "WaitForPublish",
      "Result": {
        "Alloy Wheels": true,
        "WarrantyYears": 6,
        "seatCover": "leather",
        "Driver Seat": "8-Way Power"
      },
      "ResultPath": "$.specs.extra"
    },
    "WaitForPublish": {
      "Type": "Wait",
      "SecondsPath": "$.pubDelay",
      "Next": "FinishSpecs"
    },
    "SpecialConfigCheckErrorState": {
      "Type": "Fail"
    },
    "FinishSpecs": {
      "Type": "Pass",
      "OutputPath": "$.specs",
      "End": true
    }
  }
}