cascading.tap.hadoop.util
Class Hadoop18TapUtil

java.lang.Object
  cascading.tap.hadoop.util.Hadoop18TapUtil

public class Hadoop18TapUtil
extends Object

Field Summary

static String

TEMPORARY_PATH The Hadoop temporary path used to prevent collisions

Constructor Summary

Hadoop18TapUtil()

Method Summary

static void	cleanupJob(JobConf conf) May only be called once.
static void	cleanupTapMetaData(JobConf conf, Tap tap) Called from flow step to remove temp dirs
static void	commitTask(JobConf conf) copies all files from the taskoutputpath to the outputpath
static void	makeTempPath(JobConf conf)
static boolean	needsTaskCommit(JobConf conf)
static void	setupJob(JobConf conf) should only be called if not in a Flow
static void	setupTask(JobConf conf)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

TEMPORARY_PATH

public static final String TEMPORARY_PATH

The Hadoop temporary path used to prevent collisions

See Also:

Constant Field Values

Constructor Detail

Hadoop18TapUtil

public Hadoop18TapUtil()

Method Detail

setupJob

public static void setupJob(JobConf conf)
                     throws IOException

should only be called if not in a Flow

Parameters:

conf -

Throws:

IOException

setupTask

public static void setupTask(JobConf conf)
                      throws IOException

Throws:

IOException

needsTaskCommit

public static boolean needsTaskCommit(JobConf conf)
                               throws IOException

Throws:

IOException

commitTask

public static void commitTask(JobConf conf)
                       throws IOException

copies all files from the taskoutputpath to the outputpath

Parameters:

conf -

Throws:

IOException

cleanupTapMetaData

public static void cleanupTapMetaData(JobConf conf,
                                      Tap tap)
                               throws IOException

Called from flow step to remove temp dirs

Parameters:

conf -

Throws:

IOException

cleanupJob

public static void cleanupJob(JobConf conf)
                       throws IOException

May only be called once. should only be called if not in a flow

Parameters:

conf -

Throws:

IOException

makeTempPath

public static void makeTempPath(JobConf conf)
                         throws IOException

Throws:

IOException