3. Inspecting and Visualizing Raw Data

This article contains the following sections:

Introduction

The AI Data Pipeline includes several visualization tools that allow you to inspect your raw data before and during the contextualization process, namely:

  • Raw Data Monitoring
  • Raw Data Streams
  • Raw Data Table
  • Raw Data Visualization


Viewing Raw Data Monitoring

You can view raw data in real time as it streams into the AI Data Pipeline.

To view Raw Data Monitoring:

  1. On the Raw Data tab, click Raw Data Monitoring.
  2. On the main Raw Data Monitoring screen, on the left, click Data Source to select the assets that you want to stream.
  3. In the Data Source window, select one of the assets.
  4. On the left, click Data Fields. Data Fields are all of the associated fields being captured for each data source.
  5. In the Data Fields window, select one or more data fields, and then click Apply.NOTE: Deselect individual items on the left. To remove all data fields from the selected items list, in the upper-right, click the trashcan icon.
  6. On the left, click the date range selector.
  7. In the calendar window, click Live Data to stream in real time, or select the range that you want to view. Then click Apply.
  8. On the left, click Frequency.

  1. In the Frequency window, select the streaming frequency of the visualization. This setting lets you control the pace at which data is visualized, irrespective of the pace at which it is collected. For example, Fast means you will see less data on-screen, but it will update more frequently. Slow means you will see more data on-screen, but it will update less frequently.
  2. When you are finished making your selections, on the left, click Update.

Viewing the Raw Data Streams

You can view all of your streams that have come in from FactoryTX, as well as their associated stream types.

To view the Raw Data Streams:

  1. On the Raw Data tab, click Raw Data Streams.
  2. On the Raw Data Streams screen, check your FactoryTX streams/assets.

  3. To delete a stream, select a row, and then in the upper-right, click the trashcan icon.
  4. In the Delete Streams window, click Delete to confirm the deletion.

    NOTE: The delete function is only available to internal Sight Machine users at this time, but will be rolled out to all users in a future release.

Using a Raw Data Table

You can generate your raw data into a table and download the results into a CSV file to inspect it or work with it in another application, such as Microsoft Excel.

To use a Raw Data Table:

  1. On the Raw Data tab, click Raw Data Table.
  2. On the main Raw Data Table screen, on the left, click Data Source to select the asset that you want to inspect.
  3. In the Data Source window, select one of the assets.
  4. On the left, click Data Streams.
  5. In the Data Streams window, select a stream, and then click Apply. If you have multiple streams, you can filter to show only a subset.
  6. On the left, click the date range selector.
  7. In the calendar window, select the range that you want to inspect, and then click Apply.
  8. When you are finished making your selections, on the left, click Update to generate a flattened table of your raw data.
    NOTE:
    If you query for more than 10,000 records, the AI Data Pipeline will only display the first 10,000.
  9. In the upper-right, click the download button to generate a CSV file of the results.

Using a Raw Data Visualization 

You can generate charts of your raw data to visualize it. 

To use a Raw Data Visualization:

  1. On the Raw Data tab, click Raw Data Visualization.

  2. On the main Raw Data Visualization screen, on the left, click Data Source to select the asset that you want to inspect.
  3. In the Data Source window, select one of the assets.
  4. On the left, click Data Streams.
  5. In the Data Streams window, select a stream, and then click Apply. If you have multiple streams, you can filter to show only a subset.
  6. On the left, click Raw Field.
  7. In the Raw Field window, select one field to visualize over time.
    NOTE: You can later select a second field, but are limited to one in this window.
  8. On the left, click the date range selector.
  9. In the calendar window, select the range that you want to inspect, and then click Apply.
  10. Click Update to generate your chart.  
    NOTE: If you query for more than 10,000 records, the AI Data Pipeline will only display the first 10,000.
  11. To show a second field, in the upper-right, click the Y axis button.
  12. In the Secondary Y Axes window, make a field selection.
    The chart now displays the two color-coded fields. 
  13. To change the chart display, in the upper-right, click the chart button, and then select another option.

  1. To see the chart as a table instead, in the upper-right, click the table button. You will only see the columns that you have plotted, rather than all of the columns in the raw data table.NOTE: You can view the results as a chart again by clicking the chart button in the upper-right.
  2. On the left, click Show by to change the time display into a distribution (i.e., how often the variable you selected occurs at certain values). The current selection will appear under Show by on the left.