Spotfire

EventFlow fragment examples

Directory structure

The recommended EventFlow directory structure is :

ep-eventflow-fragment
|------  pom.xml
|------  src
|     |------  main
|     |     |------  eventflow        - sbapp, ssql and sbint files
|     |     |------  java             - java functions, operators (if any)
|     |     |------  configurations   - HOCON configuration files
|     |     |------  resources        - sbfs, sbref, upgrade plan and other resources
|     |
|     |------  test
|     |     |------  eventflow        - sbapp, ssql and sbint files for tests
|     |     |------  java             - java functions, operators for tests (if any) and sbunit test case sources
|     |     |------  configurations   - HOCON configuration files for test
|     |     |------  resources        - sbfs, sbref, upgrade plan and other test resources
|     |
|     |------  site                   - documentation
|
|------  target                       - built artifacts

Note that the default EventFlow source directory is set by the plugin to src/main/eventflow, and the default EventFlow test source directory is set to src/test/eventflow.

Test sources (EventFlow, java, etc) are not included in the EventFlow fragment archive.

Basic build, sbunit test and install

The following pom.xml will build, unit test and install to the local maven repository, an eventflow fragment.

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">

    <!-- vim: set tabstop=4 softtabstop=0 expandtab shiftwidth=4 smarttab : -->

    <modelVersion>4.0.0</modelVersion>

    <groupId>com.tibco.ep</groupId>
    <artifactId>effrag</artifactId>
    <packaging>ep-eventflow-fragment</packaging>
    <version>1.0.0</version>
    <name>hello world</name>

    <!-- common definitions for this version of StreamBase -->
    <parent>
        <groupId>com.tibco.ep.sb.parent</groupId>
        <artifactId>ep-eventflow-fragment</artifactId>
        <version>11.0.0</version>
    </parent>

</project>

When the maven install goal is called (mvn install), this pom.xml instructs maven to perform the following steps :

  1. Uses install-product to check if the dependent product ( in this case com.tibco.ep.sb.rt:platform_osxx86_64 ) is installed. If its not, maven will download the archive and the plugin will extract into $TIBCO_EP_HOME.

  2. Uses set-resources to add configuration and eventflow directories to the Maven resources.

  3. Uses the standard maven plugin maven-resources-plugin:resources to copy resources to the output directory.

  4. Uses unpack-nar to unpack Native ARchives to the output directory.

  5. Uses unpack-fragment to unpack fragments to the output directory.

  6. Uses the standard maven plugin maven-compiler-plugin:testCompile to compile any java main sources to class files.

  7. Uses generate-eventflow-fragment to generate java classes for eventflows from the eventflowdirectories into the main output directory.

  8. Uses compile-eventflow-fragment to copy eventflow resources to the main output directory.

  9. Uses the standard maven plugin maven-compiler-plugin:compile to compile any generated java classes.

  10. Uses the standard maven plugin maven-resources-plugin:testResources to copy test resources to the test output directory.

  11. Uses the standard maven plugin maven-compiler-plugin:testCompile to compile any test java classes.

  12. Uses generate-test-eventflow-fragment to generate java classes for eventflows from the testEventflowdirectories into the test output directory.

  13. Uses the standard maven plugin maven-compiler-plugin:testCompile to compile any generated test java classes.

  14. Uses check-testcases to check for test cases.

  15. Uses start-nodes to start a test cluster. Since this pom.xml has no configuration, a single node is started A.${artifactId} (ie A.helloworld in this example) with a random but unused discovery port.

  16. Uses test-eventflow-fragment to launch sbunit on the cluster and report the test results. Should the test cases fail then the node will be stopped.

  17. Uses stop-nodes to stop and remove the test nodes

  18. Uses package-eventflow-fragment to create a EventFlow fragment zip file in the build directory (by default, set to target) and attaches it to the build.

  19. Uses the standard maven plugins to generate sources and javadoc and attach them to the source jar.

  20. Uses the standard maven plugin maven-install-plugin:install to install the built and tested artifacts to the local maven repository.

The EventFlow fragment application code, test cases and sbunit are deployed to the test node :

One node sbunit test

An example run is shown below :

$ mvn install
[INFO] Scanning for projects...
[INFO]
[INFO] -----------------------< com.example:helloworld >-----------------------
[INFO] Building Java Fragment - hello world 3.0.0
[INFO] -----------------------[ ep-eventflow-fragment ]------------------------
[INFO]
[INFO] --- ep-maven-plugin:2.1.0-SNAPSHOT:install-product (default-install-product-1) @ helloworld ---
[INFO] com.tibco.ep.sb.rt:platform_osxx86_64:zip:11.0.0-SNAPSHOT:test already installed manually to /Users/flionell/tibco/dev/core
[INFO]
[INFO] --- ep-maven-plugin:2.1.0-SNAPSHOT:set-resources (default-set-resources) @ helloworld ---
[INFO]
[INFO] --- maven-enforcer-plugin:3.0.0:enforce (hard-build-requirements) @ helloworld ---
[INFO]
[INFO] --- maven-resources-plugin:3.1.0:resources (default-resources) @ helloworld ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 0 resource
[INFO] Copying 0 resource
[INFO] Copying 2 resources
[INFO]
[INFO] --- ep-maven-plugin:2.1.0-SNAPSHOT:unpack-nar (default-unpack-nar) @ helloworld ---
[INFO]
[INFO] --- ep-maven-plugin:2.1.0-SNAPSHOT:unpack-fragment (default-unpack-fragment) @ helloworld ---
[INFO]
[INFO] --- maven-compiler-plugin:3.8.1:compile (default-compile-1) @ helloworld ---
[INFO] Nothing to compile - all classes are up to date
[INFO]
[INFO] --- ep-maven-plugin:2.1.0-SNAPSHOT:generate-main-eventflow (default-generate-main-eventflow) @ helloworld ---
[INFO] Found 1 module
[INFO]
[INFO] --- ep-maven-plugin:2.1.0-SNAPSHOT:compile-eventflow-fragment (default-compile-eventflow-fragment) @ helloworld ---
[INFO]
[INFO] --- maven-compiler-plugin:3.8.1:compile (default-compile) @ helloworld ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 1 source file to /Users/flionell/Documents/StreamBase Studio 11.0 Workspace/helloworld/target/classes
[INFO]
[INFO] --- maven-resources-plugin:3.1.0:testResources (default-testResources) @ helloworld ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 0 resource
[INFO] Copying 0 resource
[INFO]
[INFO] --- maven-compiler-plugin:3.8.1:testCompile (default-testCompile) @ helloworld ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 1 source file to /Users/flionell/Documents/StreamBase Studio 11.0 Workspace/helloworld/target/test-classes
[INFO]
[INFO] --- ep-maven-plugin:2.1.0-SNAPSHOT:generate-test-eventflow (default-generate-test-eventflow) @ helloworld ---
[INFO] Found 0 module
[INFO]
[INFO] --- maven-compiler-plugin:3.8.1:testCompile (default-testCompile-1) @ helloworld ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 1 source file to /Users/flionell/Documents/StreamBase Studio 11.0 Workspace/helloworld/target/test-classes
[INFO]
[INFO] --- ep-maven-plugin:2.1.0-SNAPSHOT:check-testcases (default-check-testcases) @ helloworld ---
[INFO]
[INFO] --- ep-maven-plugin:2.1.0-SNAPSHOT:start-nodes (default-start-nodes) @ helloworld ---
[INFO] UDP port 59781 selected for discovery
[INFO] [A.helloworld] Running "install node"
[INFO] [A.helloworld] 	Installing node
[INFO] [A.helloworld] 		DEVELOPMENT executables
[INFO] [A.helloworld] 		File shared memory
[INFO] [A.helloworld] 		7 concurrent allocation segments
[INFO] [A.helloworld] 		Host name MacBookPro-of-flionell.local
[INFO] [A.helloworld] 		Starting node services
[INFO] [A.helloworld] 		Loading node configuration
[INFO] [A.helloworld] 		Auditing node security
[INFO] [A.helloworld] 		Administration port is 9831
[INFO] [A.helloworld] 		Service name is A.helloworld
[INFO] [A.helloworld] 	Node installed
[INFO] [A.helloworld] Finished "install node"
[INFO] [helloworld] Running "start node"
[INFO] [A.helloworld] 	Starting node
[INFO] [A.helloworld] 		Loading node configuration
[INFO] [A.helloworld] 		Auditing node security
[INFO] [A.helloworld] 		Host name MacBookPro-of-flionell.local
[INFO] [A.helloworld] 		Administration port is 9831
[INFO] [A.helloworld] 		Service name is A.helloworld
[INFO] [A.helloworld] 	Node started
[INFO] [helloworld] Finished "start node"
[INFO]
[INFO] --- ep-maven-plugin:2.1.0-SNAPSHOT:test-eventflow-fragment (default-test-eventflow-fragment) @ helloworld ---
[INFO] [dtm] INFO: Deploy tool version: [TIBCO Distributed Transactional Memory Platform 11.0.0-SNAPSHOT (build 2202210815)] starting at [Mon Feb 28 11:12:20 CET 2022]
[INFO] [dtm] INFO: Node version: [TIBCO Streaming Runtime 11.0.0-SNAPSHOT (build 2202210837)]
[INFO] [dtm] INFO: Starting com.tibco.ep.buildmavenplugin.surefire.Runner on node A.helloworld ...
[INFO] [dtm] INFO: com.tibco.ep.buildmavenplugin.surefire.Runner started on JVM com_tibco_ep_buildmavenplugin_surefire_Runner0 on node A.helloworld.
[INFO] [A.helloworld] No user-defined Logback configuration, using product default configuration
[INFO] [A.helloworld] 2022-02-28 11:12:23.986000+0100 [516:main] INFO  com.tibco.ep.dtm.stdout:
[INFO] [A.helloworld] -------------------------------------------------------
[INFO] [A.helloworld]  T E S T S
[INFO] [A.helloworld] -------------------------------------------------------
[INFO] [A.helloworld] Running com.example.helloworld.TestCase
[INFO] [A.helloworld] Test Case 1
[INFO] [A.helloworld] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 3.853 sec
[INFO] [A.helloworld] 2022-02-28 11:12:27.850000+0100 [516:main] INFO  com.tibco.ep.dtm.stdout:
[INFO] [A.helloworld] Results :
[INFO] [A.helloworld] 2022-02-28 11:12:27.851000+0100 [516:main] INFO  com.tibco.ep.dtm.stdout:
[INFO] [A.helloworld] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
[INFO] [A.helloworld] 2022-02-28 11:12:27.851000+0100 [516:main] INFO  com.tibco.ep.dtm.stdout:
[INFO] [dtm] INFO: Engine com_tibco_ep_buildmavenplugin_surefire_Runner0 on node [A.helloworld] exited with status [0]
[INFO] [helloworld] Finished "deploy com.tibco.ep.buildmavenplugin.surefire.Runner"
[INFO]
[INFO] --- ep-maven-plugin:2.1.0-SNAPSHOT:stop-nodes (default-stop-nodes-1) @ helloworld ---
[INFO] [A.helloworld] Running "stop node"
[INFO] [A.helloworld] 	Stopping node
[INFO] [A.helloworld] 		Engine application::com_tibco_ep_buildmavenplugin_surefire_Runner0 stopped
[INFO] [A.helloworld] 	Node stopped
[INFO] [A.helloworld] Finished "stop node"
[INFO] [A.helloworld] Running "remove node"
[INFO] [A.helloworld] 	Removing node
[INFO] [A.helloworld] 		Shutting down node services
[INFO] [A.helloworld] 		Engine System::administration stopped
[INFO] [A.helloworld] 	Node removed
[INFO] [A.helloworld] Finished "remove node"
[INFO]
[INFO] --- ep-maven-plugin:2.1.0-SNAPSHOT:package-eventflow-fragment (default-package-eventflow-fragment) @ helloworld ---
[INFO] Building zip: /Users/flionell/Documents/StreamBase Studio 11.0 Workspace/helloworld/target/helloworld-3.0.0-ep-eventflow-fragment.zip
[INFO]
[INFO] >>> maven-source-plugin:3.2.1:jar (attach-sources) > generate-sources @ helloworld >>>
[INFO]
[INFO] --- ep-maven-plugin:2.1.0-SNAPSHOT:install-product (default-install-product-1) @ helloworld ---
[INFO] com.tibco.ep.sb.rt:platform_osxx86_64:zip:11.0.0-SNAPSHOT:test already installed manually to /Users/flionell/tibco/dev/core
[INFO]
[INFO] --- ep-maven-plugin:2.1.0-SNAPSHOT:set-resources (default-set-resources) @ helloworld ---
[INFO]
[INFO] --- maven-enforcer-plugin:3.0.0:enforce (hard-build-requirements) @ helloworld ---
[INFO]
[INFO] <<< maven-source-plugin:3.2.1:jar (attach-sources) < generate-sources @ helloworld <<<
[INFO]
[INFO]
[INFO] --- maven-source-plugin:3.2.1:jar (attach-sources) @ helloworld ---
[INFO] Building jar: /Users/flionell/Documents/StreamBase Studio 11.0 Workspace/helloworld/target/helloworld-3.0.0-sources.jar
[INFO]
[INFO] --- maven-javadoc-plugin:3.3.1:jar (attach-javadocs) @ helloworld ---
[INFO]
[INFO] --- maven-install-plugin:2.5.2:install (default-install) @ helloworld ---
[INFO] Installing /Users/flionell/Documents/StreamBase Studio 11.0 Workspace/helloworld/target/helloworld-3.0.0-ep-eventflow-fragment.zip to /Users/flionell/tibco/dev/core/BUILD/repository/com/example/helloworld/3.0.0/helloworld-3.0.0.zip
[INFO] Installing /Users/flionell/Documents/StreamBase Studio 11.0 Workspace/helloworld/pom.xml to /Users/flionell/tibco/dev/core/BUILD/repository/com/example/helloworld/3.0.0/helloworld-3.0.0.pom
[INFO] Installing /Users/flionell/Documents/StreamBase Studio 11.0 Workspace/helloworld/target/helloworld-3.0.0-sources.jar to /Users/flionell/tibco/dev/core/BUILD/repository/com/example/helloworld/3.0.0/helloworld-3.0.0-sources.jar
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  37.467 s
[INFO] Finished at: 2022-02-28T11:12:34+01:00
[INFO] ------------------------------------------------------------------------

Calling out to java

Java code that needs to be included with the application can be placed in java source directory ( src/main/java ). This code will be added to the fragment and included with any test cases.

External dependency

External java (jar) dependencies can be added to the maven pom.xml - these are then included in build, test and package steps.

For example :

...
    <dependencies>

        <!-- compile, test and deploy dependencies -->
        <dependency>
            <groupId>org.jpos</groupId>
            <artifactId>jpos</artifactId>
            <version>1.6.8</version>
        </dependency>

        ...

    </dependencies>
...

Building native libraries

Whilst a maven build could use exec-maven-plugin to call the native toolchain ( make, g++ etc ), the maven plugin nar-maven-plugin is recommended - this plugin performs the native build and creates a nar archive. The nar archive can be treated like any other maven dependency.

An example of creating a nar archive is shown below :

<project>

    <groupId>com.tibco.ep.dtmexamples.eventflowfragment</groupId>
    <artifactId>cpp-lib</artifactId>
    <packaging>nar</packaging>
    <version>3.0.0</version>
    
    ....
   
    <build>

        <plugins>

            <plugin>
                <groupId>com.github.maven-nar</groupId>
                <artifactId>nar-maven-plugin</artifactId>
                <version>3.2.0</version>
                <extensions>true</extensions>
                <configuration>
                    <libraries>
                        <library>
                            <type>jni</type>
                            <narSystemPackage>com.tibco.ep.dtmexamples.eventflowfragment.sbappcallscpp</narSystemPackage>
                        </library>
                    </libraries>
                </configuration>
            </plugin>

        </plugins>

    </build>

</project>

A EventFlow fragment can then use this nar archive :

...
    <!-- common definitions for this version of StreamBase -->
    <parent>
        <groupId>com.tibco.ep.sb.parent</groupId>
        <artifactId>ep-eventflow-fragment</artifactId>
        <version>10.4.0</version>
    </parent>
...
    <dependencies>

        <dependency>
            <groupId>com.tibco.ep.dtmexamples.eventflowfragment</groupId>
            <artifactId>cpp-lib</artifactId>
            <version>3.0.0</version>
            <type>nar</type>
            <classifier>amd64-Linux-gpp-jni</classifier>
        </dependency>
        <dependency>
            <groupId>com.tibco.ep.dtmexamples.eventflowfragment</groupId>
            <artifactId>cpp-lib</artifactId>
            <version>3.0.0</version>
            <type>nar</type>
            <classifier>x86_64-MacOSX-gpp-jni</classifier>
        </dependency>
        
        ....
        
    </dependencies>
...

Note that if the nar module is built on different platforms, the EventFlow fragment module will add all architectures to the fragment zip.

See aol for the list of Architecture-OperatingSystem-Linker (AOL) values references in the classifer above. The maven plugin will map these to the internal paths when the fragment is built.

Third party native libraries

Its also possible to build a nar archive containing shared libraries from a third party - ie when the source isn't available. The directory structure for such a nar project is :

Nar directory structure

The pom.xml for this project will use the nar-maven-plugin and specify shared libraries :

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    
    <modelVersion>4.0.0</modelVersion>

    <groupId>com.example</groupId>
    <artifactId>thirdpartylibs</artifactId>
    <packaging>nar</packaging>
    <name>ThirdParty Shared Libraries</name>
    <version>1.0.0</version>

    <parent>
        <groupId>com.tibco.ep.sb.parent</groupId>
        <artifactId>common</artifactId>
        <version>10.4.0</version>
        <relativePath/>
    </parent>  

    <build>
        <plugins>
            <plugin>
                <groupId>com.github.maven-nar</groupId>
                <artifactId>nar-maven-plugin</artifactId>
                <extensions>true</extensions>
                <configuration>
                    <libraries>
                        <library>
                            <type>shared</type>
                        </library>
                    </libraries>
                </configuration>
            </plugin>
        </plugins>
    </build>

</project>

The built artifact can then be added to a project as a normal maven dependency and the shared libraries will be included in the runtime library path.

    <!-- Bring in right shared library for this platform -->
    <profiles>
        <profile>
            <id>linux profile</id>
            <activation>
                <os><name>linux</name></os>
            </activation>
            <dependencies>
                <dependency>
                    <groupId>com.example</groupId>
                    <artifactId>thirdpartylibs</artifactId>
                    <version>1.0.0</version>
                    <type>nar</type>
                    <classifier>amd64-Linux-gpp-shared</classifier>
                </dependency>
            </dependencies>
        </profile>
        <profile>
            <id>mac profile</id>
            <activation>
                <os><name>mac os x</name></os>
            </activation>
            <dependencies>
                <dependency>
                    <groupId>com.example</groupId>
                    <artifactId>thirdpartylibs</artifactId>
                    <version>1.0.0</version>
                    <type>nar</type>
                    <classifier>x86_64-MacOSX-gpp-shared</classifier>
                </dependency>
            </dependencies>
        </profile>
        <profile>
            <id>windows profile</id>
            <activation>
                <property>
                    <name>path.separator</name>
                    <value>;</value>
                </property>
            </activation>
            <dependencies>
                <dependency>
                    <groupId>com.example</groupId>
                    <artifactId>thirdpartylibs</artifactId>
                    <version>1.0.0</version>
                    <type>nar</type>
                    <classifier>amd64-Windows-msvc-shared</classifier>
                </dependency>
            </dependencies>
        </profile>
    </profiles>

Multiple sbapps

There is no difference between a fragment consisting of one sbapp file and a fragment consisting of multiple sbapp files. Multiple sbapp and sbint files are added to the EventFlow fragment.

Java adapters

EventFlow adapters written in java can be built in a maven module with a packaging type of jar and use a EventFlow application with sbunit to test.

An example pom.xml file for an adapter is shown below :

...
    <groupId>com.tibco.ep.dtmexamples.eventflowfragment</groupId>
    <artifactId>adapter</artifactId>
    <packaging>jar</packaging>
    <version>3.0.0</version>
...
    <!-- common definitions for this version of StreamBase -->
    <parent>
        <groupId>com.tibco.ep.sb.parent</groupId>
        <artifactId>ep-java-fragment</artifactId>
        <version>10.4.0</version>
    </parent>
...

To use this adapter in a EventFlow application just set as a dependency :

    <dependencies>

        <dependency>
            <groupId>com.tibco.ep.dtmexamples.eventflowfragment</groupId>
            <artifactId>adapter</artifactId>
            <version>3.0.0</version>
        </dependency>

        ....

EventFlow fragment referencing another EventFlow fragment

To import a EventFlow application add the fragment to the pom.xml dependencies :-

    <dependencies>

        <dependency>
            <groupId>com.tibco.ep.dtmexamples.eventflowfragment</groupId>
            <artifactId>sbapp-calls-adapter</artifactId>
            <version>3.0.0</version>
            <type>ep-eventflow-fragment</type>
        </dependency>

        ....

The dependency is a binary dependency: the typecheck/build process reads information about the dependency from the EventFlow archive. This archive is read from the maven repository during the maven build, in the same way java dependencies are read. Building the current EventFlow does not require the sources of the dependency.

This is also true in Studio: the EventFlow dependency project does not have to be opened in the current workspace for the Studio build/typecheck to work. In this case, the information are also read from the maven repository. If a dependency is opened in the current workspace, then the information are read from the files in the workspace, not from the repository.