public class JoinFieldedPipesPlatformTest extends PlatformTestCase
Modifier and Type | Class and Description |
---|---|
static class |
JoinFieldedPipesPlatformTest.AllComparator |
name, ROOT_OUTPUT_PATH, ROOT_PLAN_PATH, TEST_TRACEPLAN_ENABLED
Constructor and Description |
---|
JoinFieldedPipesPlatformTest() |
Modifier and Type | Method and Description |
---|---|
void |
testCross() |
void |
testForkCoGroupThenHashJoin()
This test checks for a deadlock when the same input is forked, adapted on one edge, then hashjoined back together.
|
void |
testForkCoGroupThenHashJoinCoGroupAgain()
This test checks for a deadlock when the same input is forked, adapted on one edge, cogroup with something,
then hashjoined back together.
|
void |
testForkThenJoin()
This test checks for a deadlock when the same input is forked, adapted on one edge, then hashjoined back together.
|
void |
testGroupBySplitAroundSplitGroupByJoin() |
void |
testGroupBySplitGroupByJoin() |
void |
testGroupBySplitJoins() |
void |
testGroupBySplitSplitGroupByJoin() |
void |
testJoin() |
void |
testJoinAfterEvery()
Method testCoGroupAfterEvery tests that a tmp tap is inserted after the Every in the cogroup join
|
void |
testJoinAroundJoinLeftMost()
Same source as leftmost
|
void |
testJoinAroundJoinRightMost()
Same source as rightmost
should be a single job as the same file accumulates into the joins
|
void |
testJoinAroundJoinRightMostSwapped()
Upper as leftmost and rightmost forcing two jobs
|
void |
testJoinDiffFields() |
void |
testJoinFilteredBranch()
this test intentionally filters out all values so the intermediate tap is empty.
|
void |
testJoinGroupBy() |
void |
testJoinGroupByJoin() |
void |
testJoinInner()
1 a1
1 a2
1 a3
2 b1
3 c1
4 d1
4 d2
4 d3
5 e1
5 e2
5 e3
7 g1
7 g2
7 g3
7 g4
7 g5
null h1
1 A1
1 A2
1 A3
2 B1
2 B2
2 B3
4 D1
6 F1
6 F2
null H1
1 a1 1 A1
1 a1 1 A2
1 a1 1 A3
1 a2 1 A1
1 a2 1 A2
1 a2 1 A3
1 a3 1 A1
1 a3 1 A2
1 a3 1 A3
2 b1 2 B1
2 b1 2 B2
2 b1 2 B3
4 d1 4 D1
4 d2 4 D1
4 d3 4 D1
null h1 null H1
|
void |
testJoinInnerOuter()
1 a1
1 a2
1 a3
2 b1
3 c1
4 d1
4 d2
4 d3
5 e1
5 e2
5 e3
7 g1
7 g2
7 g3
7 g4
7 g5
null h1
1 A1
1 A2
1 A3
2 B1
2 B2
2 B3
4 D1
6 F1
6 F2
null H1
1 a1 1 A1
1 a1 1 A2
1 a1 1 A3
1 a2 1 A1
1 a2 1 A2
1 a2 1 A3
1 a3 1 A1
1 a3 1 A2
1 a3 1 A3
2 b1 2 B1
2 b1 2 B2
2 b1 2 B3
3 c1 null null
4 d1 4 D1
4 d2 4 D1
4 d3 4 D1
5 e1 null null
5 e2 null null
5 e3 null null
7 g1 null null
7 g2 null null
7 g3 null null
7 g4 null null
7 g5 null null
null h1 null H1
|
void |
testJoinInnerSingleField() |
void |
testJoinMergeGroupBy()
When run against a cluster a Merge before a GroupBy can hide the streamed/accumulated nature of a branch.
|
void |
testJoinMixed()
1 a
5 b
6 c
5 b
5 e
1 A
2 B
3 C
4 D
5 E
1 a
2 b
3 c
4 d
5 e
1 a 1 A 1 a
- - 2 B 2 b
- - 3 C 3 c
- - 4 D 4 d
5 b 5 E 5 e
5 e 5 E 5 e
|
void |
testJoinNone() |
void |
testJoinOuter()
/**
1 a1
1 a2
1 a3
2 b1
3 c1
4 d1
4 d2
4 d3
5 e1
5 e2
5 e3
7 g1
7 g2
7 g3
7 g4
7 g5
null h1
1 A1
1 A2
1 A3
2 B1
2 B2
2 B3
4 D1
6 F1
6 F2
null H1
1 a1 1 A1
1 a1 1 A2
1 a1 1 A3
1 a2 1 A1
1 a2 1 A2
1 a2 1 A3
1 a3 1 A1
1 a3 1 A2
1 a3 1 A3
2 b1 2 B1
2 b1 2 B2
2 b1 2 B3
3 c1 null null
4 d1 4 D1
4 d2 4 D1
4 d3 4 D1
5 e1 null null
5 e2 null null
5 e3 null null
null null 6 F1
null null 6 F2
7 g1 null null
7 g2 null null
7 g3 null null
7 g4 null null
7 g5 null null
null h1 null H1
|
void |
testJoinOuterInner()
1 a1
1 a2
1 a3
2 b1
3 c1
4 d1
4 d2
4 d3
5 e1
5 e2
5 e3
7 g1
7 g2
7 g3
7 g4
7 g5
null h1
1 A1
1 A2
1 A3
2 B1
2 B2
2 B3
4 D1
6 F1
6 F2
null H1
1 a1 1 A1
1 a1 1 A2
1 a1 1 A3
1 a2 1 A1
1 a2 1 A2
1 a2 1 A3
1 a3 1 A1
1 a3 1 A2
1 a3 1 A3
2 b1 2 B1
2 b1 2 B2
2 b1 2 B3
4 d1 4 D1
4 d2 4 D1
4 d3 4 D1
null null 6 F1
null null 6 F2
null h1 null H1
|
void |
testJoinSamePipe() |
void |
testJoinSamePipe2() |
void |
testJoinSamePipe3() |
void |
testJoinSamePipeAroundGroupBy() |
void |
testJoinSamePipeName() |
void |
testJoinSameSourceIntoJoin()
here the same file is fed into the same HashJoin.
|
void |
testJoinSameSourceIntoJoinSimple() |
void |
testJoinSameSourceOverGroupByIntoJoinSimple()
Loosely tests for a deadlock when BlockingHashJoinAnnotator rule doesn't excluce the GroupBy from the blocking
annotation.
|
void |
testJoinSelf() |
void |
testJoinsIntoCoGroup() |
void |
testJoinsIntoCoGroupLhs()
This test results in two MR jobs because one join feeds into the accumulated side of the second.
|
void |
testJoinsIntoCoGroupLhsSwappedJoin()
This test results in one MR jobs because one join feeds into the streamed side of the second.
|
void |
testJoinsIntoCoGroupRhs() |
void |
testJoinsIntoGroupBy()
Tests that two independent streamed sources with loadable tributaries properly plan into a GroupBy
without loading unused sources
|
void |
testJoinSplit()
Under tez, this can result in the HashJoin being duplicated across nodes for each split after the HashJoin
BoundaryBalanceJoinSplitTransformer inserts a Boundary at the split, preventing duplication of the path
|
void |
testJoinSplitBeforeJoin()
checks that a split after a HashJoin does not result in the HashJoin execution being duplicated across
multiple nodes, one for each branch in the split.
|
void |
testJoinWithHasher()
Tests Hasher being honored even if default comparator is null.
|
void |
testJoinWithUnknowns() |
void |
testSameSourceJoin() |
void |
testSameSourceJoinSplitIntoJoin()
catches a situation where BottomUpJoinedBoundariesNodePartitioner may capture an invalid HashJoin sub-graph
if the in-bound Boundary is split upon.
|
copyFromLocal, disableDebug, getOutputPath, getOutputPathElements, getPlanPathElements, getPlatform, getPlatformName, getProperties, installPlatform, makeOutputPath, setUp, tearDown
asCollection, asCollection, asCollection, asCollection, asList, asList, asSet, asSet, getOutputPath, getPlanPath, getSinkAsList, getSourceAsList, getTestCaseName, getTestName, getTestOutputRoot, getTestPlanRoot, invokeAggregator, invokeAggregator, invokeAggregator, invokeAggregator, invokeAggregator, invokeAggregator, invokeBuffer, invokeBuffer, invokeBuffer, invokeBuffer, invokeBuffer, invokeBuffer, invokeFilter, invokeFilter, invokeFilter, invokeFilter, invokeFilter, invokeFilter, invokeFilter, invokeFilter, invokeFunction, invokeFunction, invokeFunction, invokeFunction, invokeFunction, invokeFunction, invokeFunction, invokeFunction, validateLength, validateLength, validateLength, validateLength, validateLength, validateLength, validateLength, validateLength, validateLength, validateLength
assertEquals, assertEquals, assertEquals, assertEquals, assertEquals, assertEquals, assertEquals, assertEquals, assertEquals, assertEquals, assertEquals, assertEquals, assertEquals, assertEquals, assertEquals, assertEquals, assertEquals, assertEquals, assertEquals, assertEquals, assertFalse, assertFalse, assertNotNull, assertNotNull, assertNotSame, assertNotSame, assertNull, assertNull, assertSame, assertSame, assertTrue, assertTrue, countTestCases, createResult, fail, fail, failNotEquals, failNotSame, failSame, format, getName, run, run, runBare, runTest, setName, toString
public JoinFieldedPipesPlatformTest()
public void testCross() throws java.lang.Exception
java.lang.Exception
public void testJoin() throws java.lang.Exception
java.lang.Exception
public void testJoinSamePipeName() throws java.lang.Exception
java.lang.Exception
public void testJoinWithUnknowns() throws java.lang.Exception
java.lang.Exception
public void testJoinFilteredBranch() throws java.lang.Exception
java.lang.Exception
public void testJoinSelf() throws java.lang.Exception
java.lang.Exception
public void testSameSourceJoin() throws java.lang.Exception
java.lang.Exception
public void testJoinAfterEvery() throws java.lang.Exception
java.lang.Exception
- whenpublic void testJoinInnerSingleField() throws java.lang.Exception
java.lang.Exception
public void testJoinInner() throws java.lang.Exception
java.lang.Exception
public void testJoinOuter() throws java.lang.Exception
java.lang.Exception
public void testJoinInnerOuter() throws java.lang.Exception
java.lang.Exception
public void testJoinOuterInner() throws java.lang.Exception
java.lang.Exception
public void testJoinMixed() throws java.lang.Exception
java.lang.Exception
public void testJoinDiffFields() throws java.lang.Exception
java.lang.Exception
public void testJoinGroupBy() throws java.lang.Exception
java.lang.Exception
public void testJoinSamePipe() throws java.lang.Exception
java.lang.Exception
public void testJoinSamePipe2() throws java.lang.Exception
java.lang.Exception
public void testJoinSamePipe3() throws java.lang.Exception
java.lang.Exception
public void testJoinAroundJoinRightMost() throws java.lang.Exception
java.lang.Exception
public void testJoinAroundJoinLeftMost() throws java.lang.Exception
java.lang.Exception
public void testJoinAroundJoinRightMostSwapped() throws java.lang.Exception
java.lang.Exception
public void testJoinGroupByJoin() throws java.lang.Exception
java.lang.Exception
public void testJoinSameSourceIntoJoin() throws java.lang.Exception
java.lang.Exception
public void testJoinSameSourceIntoJoinSimple() throws java.lang.Exception
java.lang.Exception
public void testJoinSameSourceOverGroupByIntoJoinSimple() throws java.lang.Exception
java.lang.Exception
public void testJoinsIntoGroupBy() throws java.lang.Exception
java.lang.Exception
public void testJoinSamePipeAroundGroupBy() throws java.lang.Exception
java.lang.Exception
public void testJoinsIntoCoGroupLhs() throws java.lang.Exception
java.lang.Exception
public void testJoinsIntoCoGroupLhsSwappedJoin() throws java.lang.Exception
java.lang.Exception
public void testJoinsIntoCoGroupRhs() throws java.lang.Exception
java.lang.Exception
public void testJoinsIntoCoGroup() throws java.lang.Exception
java.lang.Exception
public void testJoinWithHasher() throws java.lang.Exception
java.lang.Exception
public void testJoinNone() throws java.lang.Exception
java.lang.Exception
public void testGroupBySplitJoins() throws java.lang.Exception
java.lang.Exception
public void testJoinMergeGroupBy() throws java.lang.Exception
The planner nw
commented code is for troubleshooting.
java.lang.Exception
public void testJoinSplit() throws java.lang.Exception
java.lang.Exception
public void testSameSourceJoinSplitIntoJoin() throws java.lang.Exception
java.lang.Exception
public void testJoinSplitBeforeJoin() throws java.lang.Exception
java.lang.Exception
public void testGroupBySplitGroupByJoin() throws java.lang.Exception
java.lang.Exception
public void testGroupBySplitSplitGroupByJoin() throws java.lang.Exception
java.lang.Exception
public void testGroupBySplitAroundSplitGroupByJoin() throws java.lang.Exception
java.lang.Exception
public void testForkThenJoin() throws java.lang.Exception
java.lang.Exception
public void testForkCoGroupThenHashJoin() throws java.lang.Exception
java.lang.Exception
public void testForkCoGroupThenHashJoinCoGroupAgain() throws java.lang.Exception
java.lang.Exception
Copyright © 2007-2015 Xplenty, Inc. All Rights Reserved.