Dynamic Operator Examples

Example 1: Configure Aggregate using a loop over the input schema

Instead of calling each field of an Aggregate operator individually, you can replace the fields array with a loop over the input schema. In this example, you skip the standard field Serial because you need to rename it to serial. While you could accomplish this in another operator, such as Parse JSON or Rename, this is a good example of how to skip fields.

"fields": [
    {% for field in inputs['in'] %}
      {% if field['name'] != 'Serial' %}
      {
        "from_field": "{{ field['name'] }}",
        "to_field": "{{ field['name'] }}",
        "function": {% if field['type'] == 'double' %} "mean" 
                    {% else %} "last"
                    {% endif %}
      },
      {% endif %}
    {% endfor %}
]

However, this may create errors downstream if you are missing a few fields that you had previously created by modifying the output of Autodraft. You can add these fields either before or after the loop (here, shown after):

"fields": [
    {% for field in inputs['in'] %}
      {% if field['name'] != 'Serial' %}
      {
        "from_field": "{{ field['name'] }}",
        "to_field": "{{ field['name'] }}",
        "function": {% if field['type'] == 'double' %} "mean" 
                    {% else %} "last"
                    {% endif %}
      },
      {% endif %}
    {% endfor %}
    {
      "from_field": "timestamp",
      "to_field": "endtime",
      "function": "max"
    },
    {
      "from_field": "timestamp",
      "to_field": "starttime",
      "function": "min"
    },
    {
      "function": "unique",
      "from_field": "machine",
      "to_field": "machines"
    },
    {
      "function": "last",
      "from_field": "Serial",
      "to_field": "serial"
    }
]

Example 2: Configure Aggregate using a loop over a Data Dictionary

Using a Data Dictionary table uploaded through the Manage Tables tool, you can configure an Aggregate operator to pull cycle functions from the table. Here, you use a table called mixing_tank with columns including field_name and cycle_func.

We’ll also add a secondary method of determining the correct function, which will be applied to any fields not in our Data Dictionary.

As in the previous example, you can add extra fields before or after the loop.

{
  "partition_by": [
    "machine"
  ],
  "boundary_field": "cycle_end",
  "emit_window": "when_complete",
  "fields": [
    {% for field in inputs['in'] %}
      {% set row = try_lookup(tables['mixing_tank'], 'field-name', field['name']) %}
      {% if row != null %}
        {
          "function": "{{ row['cycle_func'] }}",
          "from_field": "{{ row['field_name'] }}",
          "to_field": "{{ row['field_name'] }}"
        },
      {% else %}
        {
          "from_field": "{{ field['name'] }}",
          "to_field": "{{ field['name'] }}",
          "function": {% if field['type'] == 'double' %} "mean" 
                      {% else %} "last"
                      {% endif %}
        },
      {% endif %}
    {% endfor %}
    {
      "from_field": "timestamp",
      "to_field": "starttime",
      "function": "min"
    },
  ]
}

Example 3: Configure Annotate by Field using a loop over a Data Dictionary

Using a Data Dictionary table uploaded through the Manage Tables tool, you can configure an Annotate by Field operator to pull cycle functions from the table. Here, you use a table called mixing_tank with columns including field_name and display_name. This can be the same table used for the previous example if it contains all three columns needed.You can apply the display-name annotation using a loop over the table.

Note: Empty cells in a table are treated as empty strings unless you specifically tell the uploader that these cells are null (see the Manage Tables page for instructions). Since for this example you did not do that, you check both whether the cell row['display_name'] exists at all and whether it is an empty string.

{
  "comment": "todo",
  "annotations": [
    {% for row in tables['mixing_tank'] %}
      {% if row['display_name'] != null and row['display_name'] != '' %}
        {
          "field": "{{ row['field_name'] }}",
          "name": "display-name",
          "value": "{{ row['display_name'] }}"
        },
      {% endif %}
    {% endfor %}
  ]
}

Caution: When determining if a variable contains a null value, you must explicitly compare it to null, as shown in line 5 above. Otherwise, Jinja thinks that the variable does not exist and raises an error that looks like the following: “Unknown token found: row['display_name']"

Example 4: Configure Periodic Event using a Table of Constants

You can replace the interval in a Periodic Event operator with a number from a table of constants uploaded through the Manage Tables tool.

You read the interval from the table using lookup. Since you expect the value to be present, you want an error if it cannot be found; otherwise, you could use try_lookup instead. Note how you can intersperse Jinja with JSON.

{
  "initial_timestamp": "2022-08-01T00:00:00Z",
  {% set row = lookup(tables['constants'], 'constant_name', 'periodic_interval') %}
  "interval": "{{ row['constant_value'] }}s"
}