Hopsworks is an open-source platform for the development and operation of ML models, available as an on-premises platform (open-source or Enterprise version) and as a managed platform on AWS and Azure.
‣ End-to-End ML Pipelines - the Feature Store to training to serving;
‣ Manages ML Assets: Features, Experiments, Model Repository/Monitoring;
‣ Project-based multi-tenancy - collaborate with sensitive data in a shared cluster;
‣ Enterprise integrations with Active Directory, LDAP, OAuth2, Kubernetes;
‣ Full governance and provenance for ML assets, with GDPR compliance;
‣ Open-source ML with Spark, TensorFlow, PyTorch, Scikit-Learn;
‣ Solves the hardest scaling problems: training, hparam tuning, feature engineering.
Hopsworks 2.0 is an open-source platform for the development and operation of ML models, available as an on-premises platform (open-source or Enterprise version) and as a managed platform on AWS and Azure.
‣ End-to-End ML Pipelines - the Feature Store to training to serving;
‣ Manages ML Assets: Features, Experiments, Model Repository/Monitoring;
‣ Project-based multi-tenancy - collaborate with sensitive data in a shared cluster;
‣ Enterprise integrations with Active Directory, LDAP, OAuth2, Kubernetes;
‣ Full governance and provenance for ML assets, with GDPR compliance;
‣ Open-source ML with Spark, TensorFlow, PyTorch, Scikit-Learn;
‣ Solves the hardest scaling problems: training, hparam tuning, feature engineering.
Our Feature Store is the foundation for managing Data for AI across all industries. It is enabling collaboration between data science, data engineering, and ML operations teams. The full Hopsworks platform builds on the Feature Store to provide a rich Data Science platform built on open-source ML frameworks for feature engineering, model training, and model serving - all at scale.
Hopsworks manages and processes your data at scale for AI. The Hopsworks Feature Store manages features to be used in both training and serving models. It builds on HopsFS, the world's most scalable hierarchical HDFS- filesystem.
You can scale out training and hyperparameter optimization with as many GPUs as you can put in your cluster. And we provide framework support (Maggy and PySpark) to make distributed ML as Pythonic as possible.
ML pipelines have become the defacto way to productionize ML models. Hopsworks uses Airflow to orchestrate pipelines consisting of anything from (Py)Spark jobs, to Python programs on K8s, to Jupyter notebooks, to TensorFlow Extended (Beam/Flink).
JupyterLab is provided as a collaborative development environment, while jobs can also be deployed as programs: Python, PySpark, Scala/Java Spark, Beam/Flink.
Hopsworks can version all ML artifacts in ML pipelines: features in the feature store, train/test datasets, programs and pipelines in Github, and models in the model repository. Hopsworks also provides industry-leading support for provenance in ML pipelines: debug and explore lineage between processing steps and ML artifacts.
Unlike MLFlow and TFX, you do not need to re-write your piplines to add provenance: it is implicitly captured by our unique change-data-capture technology.
Our Feature Store is enabling Data Management across all industries for AI. Improving the workflow, simplifying communication between teams and allowing ground breaking work to be produced by in-house teams and partners with unparalleled ease using one of the industry's first Feature Store.
“Hopsworks keeps big organisations on their toes for what is possible and offers the implementation speed and options of on-premise and cloud based on data scientists’ needs”
Marcus Smed
Agile Product Owner
@Swedbank
‘’Hopsworks is a project-oriented data platform with an extensive range of functionality such as the Feature Store that integrates well with other platforms such as Databricks and Sagemaker"
.jpg)
Xavier Mehaut
Senior Architect Big Data, Cloud & AI
@AI & DATA
