org.apache.spark.sql.execution.aggregate
TungstenAggregationIterator
Related Doc:
package aggregate
expressions for grouping keys
AggregateExpression containing AggregateFunctions with mode Partial, PartialMerge, or Final.
the attributes of the aggregateExpressions' outputs when they are stored in the final aggregation buffer.
expressions for generating output rows.
the function used to create mutable projections.
attributes of representing input rows from inputIter.
the iterator containing input UnsafeRows.
(Changed in version 2.8.0) collect has changed. The previous behavior can be reproduced with toSeq.
Initializes buffer values for all aggregate functions.
Initializes buffer values for all aggregate functions.
Generate an output row when there is no input and there is no grouping expression.
An iterator used to evaluate aggregate functions. It operates on UnsafeRows.
This iterator first uses hash-based aggregation to process input rows. It uses a hash map to store groups and their corresponding aggregation buffers. If this map cannot allocate memory from memory manager, it spills the map into disk and creates a new one. After processed all the input, then merge all the spills together using external sorter, and do sort-based aggregation.
The process has the following step:
The code of this class is organized as follows: