cascading.tap.hadoop.util
Class Hadoop18TapUtil

java.lang.Object
  extended by cascading.tap.hadoop.util.Hadoop18TapUtil

public class Hadoop18TapUtil
extends Object


Field Summary
static String TEMPORARY_PATH
          The Hadoop temporary path used to prevent collisions
 
Constructor Summary
Hadoop18TapUtil()
           
 
Method Summary
static void cleanupJob(JobConf conf)
          May only be called once.
static void cleanupTapMetaData(JobConf conf, Tap tap)
          Called from flow step to remove temp dirs
static void commitTask(JobConf conf)
          copies all files from the taskoutputpath to the outputpath
static void makeTempPath(JobConf conf)
           
static boolean needsTaskCommit(JobConf conf)
           
static void setupJob(JobConf conf)
          should only be called if not in a Flow
static void setupTask(JobConf conf)
           
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Field Detail

TEMPORARY_PATH

public static final String TEMPORARY_PATH
The Hadoop temporary path used to prevent collisions

See Also:
Constant Field Values
Constructor Detail

Hadoop18TapUtil

public Hadoop18TapUtil()
Method Detail

setupJob

public static void setupJob(JobConf conf)
                     throws IOException
should only be called if not in a Flow

Parameters:
conf -
Throws:
IOException

setupTask

public static void setupTask(JobConf conf)
                      throws IOException
Throws:
IOException

needsTaskCommit

public static boolean needsTaskCommit(JobConf conf)
                               throws IOException
Throws:
IOException

commitTask

public static void commitTask(JobConf conf)
                       throws IOException
copies all files from the taskoutputpath to the outputpath

Parameters:
conf -
Throws:
IOException

cleanupTapMetaData

public static void cleanupTapMetaData(JobConf conf,
                                      Tap tap)
                               throws IOException
Called from flow step to remove temp dirs

Parameters:
conf -
Throws:
IOException

cleanupJob

public static void cleanupJob(JobConf conf)
                       throws IOException
May only be called once. should only be called if not in a flow

Parameters:
conf -
Throws:
IOException

makeTempPath

public static void makeTempPath(JobConf conf)
                         throws IOException
Throws:
IOException


Copyright © 2007-2013 Concurrent, Inc. All Rights Reserved.