Feed abonnieren

by Steve Watt, Chief Architect, Big Data, Red Hat

Red Hat and Continuum Analytics are pleased to announce a new solution that allows customers to deploy PySpark on top of Red Hat Storage GlusterFS. If you're attending Strata, you are encouraged to swing by the Red Hat Booth to grab a solution brief that describes how the solution is put together and how you can set it up. However, for those of you that are not at Strata, here's the overview -- and be sure to check out the technology brief, here.

Continuum Analytics are the makers of Anaconda, a leading Python distribution. At Strata, Continuum Analytics are announcing a new product, Anaconda Cluster, which is a highly-scalable cluster resource management tool. Red Hat Storage GlusterFS is a cost effective, easily scalable, POSIX compliant, distributed filesystem that runs on industry standard servers. Given that accessing data in HDFS from Python can be cumbersome, Red Hat and Continuum Analytics have built a solution that enables Anaconda Cluster to deploy PySpark on GlusterFS. This collocated solution keeps life simple for Python developers by providing a Python interface to Apache Spark that is able to read and write data on a distributed filesystem that looks and works like the local filesystems that they are used to. Furthermore, given that both Python and GlusterFS are written in C, this allows easy access to data from Python applications whether they are running on-premise or in the cloud.

If you'd like to try it out, please check out the demo posted in the video below and its accompanying tutorial: https://github.com/wattsteve/pyspark-tutorial


Über den Autor

Nach Thema durchsuchen

automation icon

Automatisierung

Das Neueste zum Thema IT-Automatisierung für Technologien, Teams und Umgebungen

AI icon

Künstliche Intelligenz

Erfahren Sie das Neueste von den Plattformen, die es Kunden ermöglichen, KI-Workloads beliebig auszuführen

open hybrid cloud icon

Open Hybrid Cloud

Erfahren Sie, wie wir eine flexiblere Zukunft mit Hybrid Clouds schaffen.

security icon

Sicherheit

Erfahren Sie, wie wir Risiken in verschiedenen Umgebungen und Technologien reduzieren

edge icon

Edge Computing

Erfahren Sie das Neueste von den Plattformen, die die Operations am Edge vereinfachen

Infrastructure icon

Infrastruktur

Erfahren Sie das Neueste von der weltweit führenden Linux-Plattform für Unternehmen

application development icon

Anwendungen

Entdecken Sie unsere Lösungen für komplexe Anwendungsherausforderungen

Original series icon

Original Shows

Interessantes von den Experten, die die Technologien in Unternehmen mitgestalten