The appearance of Hadoop and its related ecosystem was like a Cambrian explosion of open source tools and frameworks to process big amounts of data. But companies who invested early in big data found some challenges. For example, they needed engineers with expert knowledge not only on distributed systems and data processing but also on Java and the related JVM-based languages and tools. Another issue was that the system constraints at the time were constantly evolving as new systems appeared to support in-memory processing and continuous data processing (streaming).
