New Storage Software Company Leil Storage Developing Saunafs, Distributed File System Inspired By Google
StorageNewsletter.com, Monday, April 8th, 2024
For their own usage, hyperscalers developed their own file system because they didn't find commercial or other solutions on the market that can address their needs both in terms of scalability, throughput and resiliency with a very low cost.
After developing and using for several years multiple giant Google File System (GFS) clusters, Google created its successor, Colossus File System, with essentially distributed metadata servers storing metadata in BigTable with small chunk size of 1MB instead of 64 and able to scale 100x over the largest GFS clusters. You probably read the original GFS paper published in 2003 and also the page dedicated to Colossus.
Facebook unveiled their Tectonic File System beyond f4 and Haystack.
HDFS aka Hadoop Distributed File System also has been inspired by Google File System according to Doug Cutting, at Yahoo! at that time, and moved from the Apache Nutch project to the Hadoop one in 2006.