Monitoring with Metrics¶

Metrics constitutes of the measurements of resource usage or behavior that can be observed and collected all over SnappyData clusters. Using the Metrics, you can monitor the cluster health and statistics. Monitoring the clusters allows you to do the following:

• Increase availability by quickly detecting downtime/degradation.
• Facilitate performance monitoring by external tools. These external tools can handle functions such as metrics aggregation, alerting, and visualization.

SnappyData uses the Spark’s Metrics Subsystem for metrics collection. This system allows you to publish the metrics to a variety of sinks that you can enable for metrics collection. Spark supports the following sinks for Metrics. SnappyData can send metrics to all these sinks. However, MetricsServlet sink is enabled by default and all the metrics get published here.

Whenever you start the cluster, the metrics are published and made available through commonly used sinks, which can be consumed by monitoring tools.

The following type of metrics are made available for SnappyData:

• Availability Metrics This type of metrics alerts about the status and availability of the cluster.

  Examples:

  • Cluster and node status
  • Cluster and node uptime/downtime
  • Member Statistics
  • Table Statistics

• Performance Metrics This type of metric provides insights into system performance.

  Examples:

  • Resource usage for system-level resources such as CPU, memory, disk, and network.
  • Resource usage for SnappyData application/component resources such as heap or off-heap memory and their percentage utilization etc.

Enabling the Sinks for Metrics collection¶

If you want to publish the SnappyData metrics using any of the sinks, you must enable these sinks in the metrics.properties file, which is located in the conf folder of the SnappyData installation directory.

To enable a sink, do the following:

1. Open the conf folder and change the name of metrics.properties.template to metrics.properties file.

   cp metrics.properties.template metrics.properties
   

2. Uncomment the properties (remove the #) of the sink that you want to enable and provide the necessary property values. For example, for enabling CsvSink, you must uncomment the following properties:

   # Enable CsvSink for all instances by class name
   *.sink.csv.class=org.apache.spark.metrics.sink.CsvSink
   
   # Polling period for the CsvSink
   *.sink.csv.period=1
   
   # Unit of the polling period for the CsvSink
   *.sink.csv.unit=minutes
   
   # Polling directory for CsvSink
   *.sink.csv.directory=/tmp/
   
   # Polling period for the CsvSink specific for the worker instance
   worker.sink.csv.period=10
   
   # Unit of the polling period for the CsvSink specific for the worker instance
   worker.sink.csv.unit=minutes
   

3. Start the cluster for the configurations to take effect by executing this command from the SnappyData installation folder:

       ./sbin/snappy-start-all.sh
   

Accessing Metrics¶

You can check the SnappyData metrics collected through MetricsServlet, which is enabled by default, at the following URL: <LeadNode-hostname>:<5050>/metrics/json

Accessing Metrics from JmxSink¶

Do the following to access Metrics from JmxSink:

1. If org.apache.spark.metrics.sink.JmxSink is enabled in the metrics.properties file, you can use JConsole to access Metrics captured through JmxSink. To enable JmxSink, uncomment the properties for the JmxSink in the metrics.properties file. For example:

       *.sink.jmx.class=org.apache.spark.metrics.sink.JmxSink
   

2. Launch JConsole and connect to the host on which primary lead node is running, select the process ID of SnappyData primary lead process.

3. Go to MBeans > metrics, to access the SnappyData Metrics.

Accessing Metrics from CsvSink¶

Do the following to access Metrics from CsvSink:

1. If org.apache.spark.metrics.sink.CsvSink is enabled in the metrics.properties file, you can access the metrics captured in the CSV files at the location specified in the *.sink.csv.directory property. To enable CsvSink, uncomment the properties for the CsvSink in the metrics.properties file. For example,

   *.sink.csv.class=org.apache.spark.metrics.sink.CsvSink
   

Disabling Metrics Collection¶

To disable the metrics collection for a specific Sink, edit the metrics.properties file and comment the corresponding entries by adding the prefix # to the required lines. Save the file and then restart the cluster.

Statistics¶

The following statistics in SnappyData are collected when you Metrics monitoring is enabled.

TableCountStatistics¶

Table Statistics¶

ExternalTableStatistics¶

MemberStatistics¶

MemberStatistics¶

ClusterStatistics¶