Big Data Tools Cheat Sheet

Datamania has a good technology landscape cheat sheet. Attached below is the cheat sheet.

Big-Data-Technology-Landscape-CheatSheet
Big-Data-Technology-Landscape-CheatSheet

What surprised me is that Looker is not on here. “Looker is a business intelligence software and big data analytics platform that helps you explore, analyze and share real-time business analytics easily.”

Also, ELT is also not represented in the graph. ELT stands for Extract, Load, Transform. Instead of having a data engineering team extract the data, transform it and load it into a data warehouse, the team extracts and loads the raw data into a data warehouse. Then, analytics tool such as Looker can transform the data on the fly and analyze it straight from the data warehouse. I deal with complex data transformations so I am most curious to hear how Looker or other tools can handle complex transformations.

I hear a lot of ELT and how ETL is dead. However, I did a google trend comparison between ETL and ELT and they both seem to have been stable for the past five years.

Finally, with so many tools and terminologies available, it is crucial to know when to use what. Factors include the data size, the skills, security, infrastructure, etc. There is so many exciting developments in this field!