Does Big Data Still Need Stacks?
datanami, Tuesday, February 27th, 2024
The IT industry loves its stacks. First there was the LAMP stack, then the Hadoop stack became popular. Over the past five years, something called the Modern Data Stack has taken hold in our collective data psyche, and now there are rumblings of something called the Compsable Data Stack. But is the stack concept still useful for big data and analytics?
IT stacks grew out of the desire to do as little integration work as possible in assembling production systems, usually from open source parts. You could download the pieces in the original LAMP stack-which included an operating system (Linux), a Web server (Apache), a database (MySQL), and a programming language (PHP, or even Python or Perl)-and hook them together to serve Web apps in 2005 without doling out a seven-figure contract to Accenture or another SI.
By 2010, the Hadoop age was ushering in another exercise in stacks. Initially built on the combination of a distributed file system (HDFS) and a computing framework (MapReduce), the Hadoop stack grew and grew, eventually morphing into a collection of about two dozen different projects (Hive, Spark, HBase, etc. etc. etc.).