Skip to content

Instantly share code, notes, and snippets.

@gkousiouris
gkousiouris / README.md
Last active November 16, 2023 02:04
Dynamic OW Orchestrator Action

The goal of this subflow is to enable the use of dynamically located Openwhisk action invocations in hybrid cloud/edge or multi-cloud scenarios, in which the location of the target function is not known at design time.

Screenshot 2023-09-15 193648

Its main operation is rather simple, it exploits incoming information from the message in order to adapt the url to which the function invocation will be performed. The incoming message to the Dynamic OW Action needs to include the action name in the node UI as well as the following fields in the msg.payload.value:

-__<action_name>_HOST

-___NAMESPACE

@gkousiouris
gkousiouris / README.md
Last active June 19, 2023 13:29
High Availability Router for Openwhisk

This is a pattern subflow in order to implement a routing logic. The subflow is used to divert traffic among locations based on a condition set. The default location is the first output whereas the fallback location is the second output.

router

The user needs to select through the subflow UI the following parameters:

  • redirection metric: one of the monitored parameters of the Openwhisk monitoring subflow, i.e. wait time, init time, duration or success percentage
  • redirection value is the target value above or under which the redirection should occur
  • comparison logic through the boolean redirectIfLarger parameter. This is used to define the comparison between the target value and the current value. So if redirectIfLargeris set to true, the redirection will occur if the redirection metric is larger than the redirection value
  • primary ratio may be used to define whether the redi
@gkousiouris
gkousiouris / README.md
Last active June 16, 2023 08:56
Openwhisk Sliding Window Action Monitor

This is a subflow node in order to monitor the latest performance (sliding window) of Openwhisk actions. The flow pings periodically the target Openwhisk installation in order to retrieve the last executed actions and extract statistics from their execution. The flow has been extended in order to create a third output which includes the detailed logs of failed actions during the observed window.

owmonitor_new

Configuration includes:

  • the polling time of the monitor (default 30 seconds, msg.pollingPeriod)
  • the target Openwhisk endpoint (msg.targetEndpoint)
  • credentials for that endpoint (msg.creds) in the form of user:key
  • the window of time (in minutes) in the past for which you want to retrieve results (msg.window)
  • an optional action name (msg.action), if one wants to filter specific action activations
@gkousiouris
gkousiouris / README.md
Last active May 31, 2023 10:22 — forked from vasKatevas/README.md
K-means clustering flow

Test extension

This is a flow that implements a k-means clustering operation as a service. As such it can be executed inside any Node-RED environment in a service manner. It also implements the Openwhisk API specification so that it can be executed directly as a custom docker action of Openwhisk. The inputs include arrays of objects and their values and the output returns clusters with three centroids for any given input, using the k-means implementation provided by the clusters npm library. clustering_flow

Example Input:

{
   "mode": "multiple",
@gkousiouris
gkousiouris / README.md
Last active May 3, 2023 16:59
Poll2PushConverter With Call Chain

This helper node aims at performing synchronous calls for polling in case of an async API (calls that return prior to completion, e.g. in the case of non blocking calls in OW, the initial request is returned with a submission success and an activation id in order to follow up on the result).

Generic_T4 2-Copy of PEF CHAIN drawio

The main difference from the previous PollTOPushConverter (https://flows.nodered.org/flow/fe8a1ff325abebcf99ffc63744a3c231/in/VOf-0UrN5e2j) is that it supports also function chaining (through a msg.functionChain or UI set boolean parameter). This means that if the activated function can chain itself, to avoid OW maximum runtime timeout, the node can follow-up on the new activation ID and push a notification until all the chain has finished.

poll2pushFC_simple

The accordin

@gkousiouris
gkousiouris / README.md
Last active April 25, 2023 17:26
Minio Webhook Automated Annotator

The specific subflow can be used as a webhook endpoint (/events) for getting notifications from a Minio installation and automating the enrichment of a newly inserted object with metadata.

It filters notification messages based on new objects put but also if they already have the specific metadatum needed. This is because the flow puts the object again with the new metadata, so it creates a new notification and thus an infinite loop if this comparison is not made.

The flow requires the node-red collection @reggae_ulli/node-red-contrib-minio-all-fix for the interface with the Minio server.

Pattern Images-MINIO WEBHOOK ANNOTATOR drawio

The user can configure which metadata field this skip comparison can be performed in the relevant switch node 'if already has metadata skip'.

@gkousiouris
gkousiouris / README.md
Last active February 8, 2023 07:30
Semaphore Service Subflow

This is a subflow to create a semaphore service to act as a distributed lock/synchronization mechanism. It uses flow variables in order to store the state of a semaphore. Changes in the semaphore value are uninterruptable at the level of the node.js process. This is due to the fact that the latter executes them in the single threaded eventloop and does not interrupt their execution unless an async method or worker thread is used in the function's implementation.

main

The subflow defines 5 endpoints:

  • POST /semaphore for new semaphore creation. The body should contain name and initialization value of the created semaphore: {"name":"<semname>","value":1}. The value needs to be a positive integer. If not positive, a 400 HTTP error code is returned. A positive float will be converted to integer. If the semaphore already exists, a 303 HTTP error code is returned.
  • DELETE /semaphore/:name nee
@gkousiouris
gkousiouris / README.md
Last active February 4, 2023 13:29
OpenWhisk Skeleton Interface for Node-RED flows as functions

This flow implements the necessary interface needed by a Node-RED flow, in order to be executed as a function inside Openwhisk . Openwhisk can execute in a serverless manner any type of Docker container, as long as it implements two methods on port 8080:

owskeletonflow_witherror

-A POST /init , used in the typical OW runtime images (such as node.js etc) to retrieve the function code from the OW function DB. In the Node-RED function runtime this is not needed for functional purposes, since the code is already embedded in the flow, but it is needed in order to abide by the OW interface that expects this method. It needs to be stressed that in a warm execution (reuse of existing container from previous same function execution), the /init method is not re-executed. Hence any needed initialization per invocation should not be performed inside this method.

-A POST /run method, which is called upon inv

@gkousiouris
gkousiouris / README.md
Last active February 4, 2023 13:15
Request Aggregator Service Flow

This is a flow to be run as a service, in order to perform request aggregation towards a target endpoint. The purpose is to reduce the stress towards the back-end system, by minimizing the arrival rate of requests and the number of needed serving threads or containers (in the case of FaaS systems). Thus it also reduces costs and improves performance. It is mostly beneficial in cases where the request needs to set up a rather heavyweight environment (thread or container) for performing a relatively small computation (e.g. in model inference, simulation etc.).

initial

The RA gets as input the target endpoint, as well as the method to be invoked and the input data to be the payload of the aggregated call. It holds the requests until a threshold of incoming messages is reached (set in the setBatchSize endpoint), at which point it collates the inputs of the calls into one array request towards the target e

@gkousiouris
gkousiouris / README.md
Last active February 4, 2023 13:10
Producer-Consumer Pattern Implementation using the Semaphore Service Subflow

This is an example distributed producer/consumer problem implementation using the Semaphore Service Subflow.

This is based on a typical implementation that includes the use of a mutex, a full counter (initialized at 0) and an empty counter (initialized at the MAX needed slots, in this case 3). Pattern Images-SEMAPHORE SERVICE drawio

The logic of the producer/consumer appears below:

producer() {
	while (TRUE) {