Driven Plugin FAQ

Yes, you can download Driven here: http://cascading.io/trial/

OOM messages by Driven plugin are a symptom of an error rather than an error by itself. When the Driven plugin is unable to send data to the Driven server, or if the application is publishing data to Driven plugin at a rate faster than what the Driven plugin can send the data to the server. The Driven plugin maintains an internal backlog queue which it regularly purges to manage its memory consumptions.

There can be many possible reasons:

If your Cascading application executes successfully without the Driven plugin, it will execute successfully whether or not the Driven plugin completes publishing successfully. The only caveat is that if your application uses Yarn, and if Yarn crashes, then the Cascading app will (obviously) be affected. The publishing of the data to the Driven server is decoupled from the execution of the Cascading MapReduce application.

Driven works with any version of Hadoop that’s compatible with Cascading. See http://www.cascading.org/support/compatibility/ for the full list.

When you signup at http://cascading.io/register/ for the cloud version of Driven, or use the self-hosted version for Driven and configure the Driven API key, you get access to additional features such as having the ability to organize applications into Team. This becomes useful when you want to categorize and organize applications around lines of business, different classes of customers, or compare historical execution behavior across the same class of applications to identify outliers and trends.

The matrix of supported JDKs is at http://www.cascading.org/support/compatibility/

The plug-in can be run in 'archive mode.'

If archive mode is set, all records that are sent to the Driven Server are written to disk even if the server is unreachable. This can be useful if the Driven server is not available or unreachable as the archive can later be 'replayed' when the server is reachable. Sending data to the server is idempotent, so re-running data again does not corrupt already recorded data. To use archive mode, pass a JVM property named 'driven.archive.path' when Cascading is invoked.

In case of Apache Hadoop, you can use the HADOOP_OPTS environment variable.

The compressed archive files will be located at the path set with a timestamp when the cascading job started. Do not use ~ expansion for the path.

Copyright (c) 2007-2014 Concurrent, Inc. All Rights Reserved.

Project and contact information: http://www.concurrentinc.com/