Skip to content

Instantly share code, notes, and snippets.

@neallee2012

neallee2012/sas_esp_apache_nifi.md

Created July 25, 2018 11:57
Show Gist options
  • Star 1 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save neallee2012/3eb625094a46f8e3f0cc9028e04a5093 to your computer and use it in GitHub Desktop.
Save neallee2012/3eb625094a46f8e3f0cc9028e04a5093 to your computer and use it in GitHub Desktop.
Download ZIP
Raw
sas_esp_apache_nifi.md

What is Apache Nifi?

Apache NiFi is a data flow tool that is focused on moving data between systems.
NiFi's focus is on capabilities like visual command and control, filtering of data, enrichment of data, data provenance, and security, just to name a few. With NiFi, you aren't writing code and deploying it as a job, you are building a living data flow through the UI that is taking effect with each action.
Data flow tool is often complimentary and used to manage the flow of data from the sources to the streaming processing platforms, such as SAS ESP.

What is SAS Event Stream Processing (a.k.a. SAS ESP)?

Event stream processing applications typically perform real-time analytics on streams of events. These streams are continuously published into an event stream processing engine. Typical use cases for event stream processing include but are not limited to the following:

  • sensor data monitoring and management
  • operational systems monitoring and management
  • cyber security analytics
  • capital markets trading systems
  • fraud detection and prevention

Comparing with Apache NiFi, SAS ESP can also handle data integration since SAS provided over 300+ predefined adapters and connectors and a documented method for developing new data connection adapters. SAS ESP can also perform transformations and subscription of outputs to any number of formats.
Besides, SAS ESP provide two NiFi processors which can be used to integrate Apache NiFi to Run with SAS ESP.
By doing so, we can take advantage of everything NiFi can do plus use the inherent capabilities of SAS ESP and broader portfolio of analytics.

Furthermore, SAS has a IoT reference architecture specifically for IoT as below.

Functional Comparison

Function SAS ESP Apache NiFi
Filtering Yes (SAS function, regular expression) Yes(regular expression)
Pattern Matching Yes No
Join Yes No
User Define Function Yes No
Invoke external library Yes (C++, DS2, DS, MAS) Yes (Still Experimental)
Training and Scoring online models Yes No
Scoring offline models Yes No

References

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment