ASCII Doc for NIO File Ingress Connector

file_nio_ingress_connector.svg

nio_file_ingress_connector.adoc

NIO File Ingress Connector

Version: {product-version}

Supported Since: 2017.01

What is NIO File Ingress Connector?

File transport allows files in the local file system to be read from and written to. A polling transport scans a directory or set of directories repetitively with a given period of interval. This is usually an overhead and leads to inefficient use of system resources since it scans the entire set or directories and files periodically even when there are no modifications. As a solution, NIO file transport acts as a non-polling transport which will trigger an event if and only if a file or a directory is created or modified within its monitoring scope.

How this Works?

The JDK 7 provided a special package called java.nio. Java NIO (New I/O) is an alternative I/O API for Java (from Java 1.4), serving as an alternative to the standard Java I/O and Java Networking API’s. Java NIO offers a different way of working with IO than the standard I/O API’s. This package provided a sub-package java.nio.file containing a file system change notification API, called the WatchService API. This API enables to register a directory (or set of directories) with a watch service which, when started, captures all the events of creation or modification of files and directories and makes them available via an event queue, similar to the Linux inotify API.

Sample Use Case

Prerequisite

In order to use the NIO File Ingress Connector you must first select the “NIO File Connector” dependency from the connector list when you are creating an empty Ultra project. If you have already created a project, you can add the “NIO File Connector” dependency via Component Registry. From Tools menu, select Ultra Studio → Component Registry and from the Connectors list, select the “NIO File Connector” dependency.

Implementation

First let’s create our integration flow named “SampleFlow” and add a NIO File Ingress Connector to that, and as for the root path specify the file path which you are going to scan the files for. Then select the pattern syntax which is glob or regex and specify the file path pattern according to the pattern syntax and save the configuration.

Note	The set of file paths which should be scanned for the files are generated using the above specified root path and file path pattern. See the example below.

Root Path	Pattern Syntax	Path Pattern	Description
/tmp	glob	*.xml	all the xml files inside the /tmp director
/tmp	glob	/*/.xml	all the xml files inside any level of directories in /tmp directory
/tmp	glob	/a/.xml	all the xml files inside any directory inside /tmp, whose name ends with a
/tmp	regex	\d.xml	all the xml files inside /tmp whose names are single digits (0-9)

Next include a Successful Flow End element and connect it to the NIO File Ingress Connector to complete the flow as shown below.

The following is the XML flow configuration created when we design the above diagram in the Ultra Studio UI. Each and every property set using the UI is added to the XML automatically as an externalized property, and vice versa.

<beans xmlns="http://www.springframework.org/schema/beans"
   xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
   xsi:schemaLocation="http://www.springframework.org/schema/beans
   http://www.springframework.org/schema/beans/spring-beans-4.1.xsd">

   <bean id="58ac26d3-52c2-5772-6e1c-a163172c7e90"
         class="org.adroitlogic.x.connector.nio.NIOFileIngressConnector">
       <property name="rootPath" value="/tmp/ram"/>
       <property name="patternSyntax" value="glob"/>
       <property name="filePathPattern" value="/**/*.xml"/>
       <property name="fetchAfterModification" value="1000"/>
       <property name="removeOriginalFile" value="true"/>
       <property name="moveAfterProcess" value="/tmp/nio/passed/xml"/>
       <property name="moveAfterFailure" value="/tmp/nio/failed/xml"/>
       <property name="moveTimeStampFormat" value="yyyy-MM-dd HH:MM:ss.S"/>
       <property name="processingElement" ref="f55c989e-5ac9-03e1-497f-e8268567c9cb"/>
   </bean>

   <bean id="f55c989e-5ac9-03e1-497f-e8268567c9cb" class="org.adroitlogic.x.components.flowend.SuccessfulFlowEnd"/>

</beans>

Now after running the project, you will be able to see that all the files which match the path pattern have been successfully picked up from the any directory which matches the root directory and the path pattern you specified.

Out Ports

Processor	The message will be sent to this outport for the next processing element
On Exception	The message will be sent to this outport if the processing element failed to transform the payload due to some reason

Parameters

* marked fields are mandatory

Root path *	Basic	This is the base path of the file system which you going to scan the files for. You can specify any file path pattern later at the 'Path pattern' field and that will be checked beyond this root path.
Pattern syntax *	Basic	There are 2 types of pattern syntax which we can match with the file paths of the local file system. This will be related to the 'Path pattern' field which you going to fill next. GLOB REGEX
Path pattern *	Basic	File path pattern which we need to scan for the files. This can be just a file pattern or file pattern with the directory pattern. Note The path pattern should match with the pattern syntax mentioned above
Remove original file	Advanced	This says whether to delete the original file after it is fetched by the transport.
Wait after modification	Advanced	This says not to fetch files until this amount of time (IN MILLISECONDS) has passed since the last modified file timestamp.
Move after process	Advanced	Directory path to move the file after processing and if it is success. Note If the move after process directory matches the path pattern mentioned above this file will be fetched again by the watch service. Then this will be happen circularly without stopping.
Move after failure	Advanced	Directory path to move the file after processing and if it is failed. Note If the move after failure directory matches the path pattern mentioned above this file will be fetched again by the watch service. Then this will be happen circularly without stopping.
Move timestamp format	Advanced	This timestamp format is to be appended while moving original file after processing. File name of the file which is going to be moved will be renamed by appending this timestamp format at the beginning of file name. yyyy-MM-dd HH:mm:ss.S AM/PM Z

nio_file_ingress_connector_1.png

nio_file_ingress_connector_2.png