Why is Spark faster than Hadoop Map Reduce
bafna’s answer provides the memory-side of the story, but I want to add other two important facts:DAG and ecosystem Spark uses “lazy evaluation” to form a directed acyclic graph (DAG) of consecutive computation stages. In this way, the execution plan can be optimized, e.g. to minimize shuffling data around. In contrast, this should be done … Read more