snappy-scala CLI¶

The snappy-scala CLI that was introduced as an experimental feature in the SnappyData 1.2.0 release, is considered a stable feature since the 1.3.0 release. This is similar to the Spark shell in its capabilities. The Spark documentation defines the Spark shell as follows:

Spark’s shell provides a simple way to learn the API, as well as a powerful tool to analyze data interactively. It is available in either Scala (which runs on the Java VM and is thus a good way to use existing Java libraries) or Python.

A developer who is learning the Spark APIs of SparkContext, SparkSession, RDD, DataSet, DataSources, ML, etc. , can use the following utilities:

• Spark shell to quickly bring up an interactive shell and start learning and experimenting with the APIs.
• PySpark, provided by spark, for interactive Python where you can interactively learn the Python APIs that are provided by Spark.

Spark shell is a spark application that is built on Scala’s REPL (Read-Evaluate-Print loop). It accepts Scala code as input, executes the instructions as per the code, and returns the output of those instructions. After this utility is invoked, the spark driver comes into life, in which a SparkContext, a SparkSession, and a REPL object are initialized, and an interactive shell is provided to the users on the driver VM itself for interactive learning. The high cost, in terms of time and efforts, for each change warranting code compilation and then adding into a jar can be entirely avoided by using the Spark shell. A developer who wants to develop a Spark job can start by writing the code in a Spark shell and improve the job in interactive iterations. After the developer is satisfied with the quality of the code, it can be converted into a Spark Job.

The idea of snappy-scala CLI is similar to that of Spark shell with few differences. With the snappy-scala CLI, you can interactively learn all the Spark APIs, the additional SnappyData APIs and also have the same experience of a Scala REPL or a Spark shell REPL app. You can still use the Spark shell and connect to a SnappyData cluster in Smart Connector mode, but you will require a separate Spark cluster for that. Moreover, that will be a different experience wherein the SnappyData cluster is used mostly as a DataSource by the Spark application.

snappy-scala CLI is built on top of the exec scala feature and the already existing snappy CLI utility, which connects to the SnappyData cluster using the JDBC driver. snappy-scala CLI is not a true scala interpreter but mimics a scala or a Spark shell type of interpreter. Each line of the code is shipped to the Lead node of the SnappyData cluster. The code is interpreted there, and the result of the interpretation is then brought back to the snappy-scala CLI. It is displayed in the same fashion as that of a Scala REPL.

Although the experience of the snappy-scala CLI is similar to that of a Scala or a Spark shell, yet a couple of important features are either missing or are thinly supported. This is because it is currently an experimental feature. The following are a couple of notable differences between the Spark shell and snappy-scala CLI:

• The auto completion feature, which is rich in a true Scala or Scala based interpreter. It is almost as rich as an IDE, where it can prompt possible completions, method signature, word completion, syntaxes, etc.
• Support for the list of commands which can be executed on the shell.

The following image shows a simple SnappyData cluster, which is started, and then the snappy-scala is launched to connect.

Command-line options¶

$ ./bin/snappy-scala -h

Usage:

snappy-scala [OPTIONS]

OPTIONS and Default values

   -c LOCATOR_OR_SERVER_ENDPOINT  (default value is localhost:1527)

   -u USERNAME                    (default value is APP)

   -p PASSWORD                    (default value is APP)

   -r SCALA_FILE_PATHS            (comma separated paths if multiple)

   -h, --help                     (prints script usage)

The following options are only available currently:

Securing the Usage of snappy-scala CLI¶

This is the same as securing the usage of exec scala. In a secured system, by default, only the DB owner has the permission to secure the usage and can grant this privilege to other users as well. Apart from running the Scala code, the CLI also takes predefined commands. Each of these commands has a specific purpose. All these commands start with a colon :. With the help of this, the interpreter differentiates between the code and command.

On the snappy-scala prompt, type :help for the list of supported commands. The following image shows the list of the available commands that are displayed when you type :help command:

All the commands are listed with their corresponding description when :help is typed on the snappy-scala prompt. This is mostly a subset of what Spark shell provides. Some of the missing ones are :paste, :edit, :load etc. There are few commands which are specific to snappy-scala such as :maximumdisplaywidth, :elapsedtime on etc. The following table lists the commands that are supported by snappy-scala:

The following are additional commands provided in the snappy-scala CLI over the Spark shell commands.