Data Flow Pipelines

The pipeline is a series of processes or stages through which data flow systematically and efficiently. It helps to visualize the flow between nodes, rules, and filters applied for the data flow.

Create Pipeline View

Navigate to explore view, and click on the New Pipeline button to create a new pipeline view
On click of the new pipeline button, a new dialog opens, select the desired date-time range, select the desired namespace, and click Create to create a new pipeline view.
On click of create button a new pipeline tab will get added with the pipeline view as shown bellow
Pipeline View: The pipeline view shows the data flow between the nodes. each node is connected with the edges to show the data flow between the nodes. The node will have the following details.
1. Name of the node 2. Total number of Events 3. Total volume size of the events 4. Total savings for application and target node Minimap: Minimap is a tool for quick preview and exploration of large graphs. Time Range Slider: Helps to select the time range between the selected date and time Zoom In/Out: Tool to zoom in and zoom out the graph Side Bar: Helps to make the following changes 1. Change the theme 2. Change the default title of the pipeline tab 3. Select a different date time range 4. Select a different namespace, can do multi-select 5. Shows the overall EPS and Valume savings.

Add/Choose the existing rule(FILTER, EXTRACT, SIEM, TAG, REWRITE), for namespace and application.

Click the + icon to create or choose the existing rule
Add/Choose existing rule
On click of add rule button, add rule dialog will show, with the options 1. Choose Existing Rule 2. Create New Rule
Select the existing rule to choose an existing rule for the namespace application
Select create rule button to create new rules for the namespace and application
Choose Existing Rule
Create New Rule
The selected/created rule will be displayed under the respective group nodes
Pipeline view with rule groups in the oder which rules will be applied

Add Rule(Forwarder)

Before adding the forwarder rule. create a forwarder and map the forwarder in the data flow tab.
Map a forwarder
After adding the forwarder the new node will be added as bellow
Pipeline with the forwarder
Click on the + icon to choose an existing or create a new forwarder rule for the application and target. After adding the forwarder rule, a new node will be shown between the application and the forwarder.
With the forwarder rule

Rule group node

Rule Group
Each rule type will have separate rule groups(FILTER, EXTRACT, SIEM, TAG, REWRITE, FORWARDER). Each rule node will have the following details. 1. Rule active status within the time range 2. Info icon, on click will show the rule details 3. Name of the rule 4. Savings done by the rule if any

Rule details

To view rule details, click on the info icon which is present in the top right corner, new popup shows with rule details on click of the icon.
Rule details
Activate/Deactivate: On checking/unchecking the Active check box, the rule will get activated/deactivated.