The World’s Fastest Parallel File System with Weka!
I’ve always had an affinity with storage systems… that all started back at my first role with a three rack HPE Storage Area Network system that was cream coloured, big and slow… and when it crashed and burned it really crashed and burned. Since then I’ve dealt with all sorts of storage design and implementations… and continued to deal with performance and scalability pain. Weka promises to resolve a lot of the pain points through a software defined approach that leverages commodity hardware and fast NVMe/SSD disk technology.
There is no doubting that the world is looking at new technology and platforms to help generate and store data on, so a more modern approach to file systems is something that should resonate with a lot of people. By leveraging modern technology like NVMe and Object Storage can lead to a clear differentiating value proposition in the space. This is where Weka can succeed over the next five to ten years… they haven’t set themselves up for the now… but what is to come.
Watch and listen (episode segments listed below):
- Intro (0:00)
- Weka Introduction (1:10)
- Funding and Investors (3:48)
- The What and How of WekaFS (5:20)
- Linear Scalability of WekaFS (9:50)
- How Weka uses NVMe and SSD to scale metadata and files (12:05)
- Leveraging Ethernet and Infiniband (14:50)
- POSIX Filesystem (16:25)
- Software defined running on Commodity Hardware (17:55)
- Bursting out to AWS using Amazon S3 (19:50)
- Weka for Big Data, Genomics and fighting Covid (26:00)
- Weka Core Innovation and continuing to internally disrupt (30:35)
- Wrap-Up and Shoutouts (33:58)
In this episode I talk with Liran Zvibel, Co-Founder and CEO at #Weka. Weka has built the world’s fastest parallel file system, designed to solve the performance challenges in AI deep learning and technical computing. WekaFS delivers high performance, low latency storage. Weka provides a seamless hybrid cloud solution between on-premises data centers and the public cloud. With customers in several high performance markets including autonomous vehicle AI training, #genomics, financial modeling, EDA, software development, satellite imaging and media and entertainment.
Weka was founded in 2014 and is based San Francisco.