HopsFS
Open source, next generation scaleout metadata file system.

AI/ML needs a Key-Value store, and Redis is not up to it

RonDB shows higher availability and the ability to handle larger data sets in comparison with Redis, paving the way to be the fastest key-value store available.

read more
Beyond Brainless AI with a Feature Store

Evolve your models from stateless AI to Total Recall AI with the help of a Feature Store

read more
Elasticsearch is dead, long live Open Distro for Elasticsearch

Hopsworks now supports dynamic role-based access control to indexes in elasticsearch with no performance penalty by building on Open Distro for Elasticsearch.

read more

Unified extended,
Searchable Metadata

HopsFS provides a principled architecture for easily extending metadata for files and directories.

In particular, this is useful in the domain of machine learning where we have both artifacts (feature data, training data, programs, models, log files) that are typically stored as files and metadata (experiments, hyperparameters, tags, metrics, etc ) that are stored in a metastore (often a relational database).

HopsFS unifies the artifact store and metastore.

100x Times Faster than AWS S3

What If you could build on top of S3 a distributed file system with a HDFS API that gives you POSIX goodness and improved performance?

That’s what we have done with a cloud-native release of HopsFS that is highly available across availability zones, has the same cost as S3, but has 100X the performance of S3 for file move/rename operations, and 3.4X the read throughput of S3 (EMRFS) for the DFSIO Benchmark (peer reviewed at ACM Middleware 2020).