The new OpenStack Kilo upstream release that became available on April 30, 2015 marks a significant milestone for the Manila project for shared file system service for OpenStack with an increase in development capacity and extensive vendors adoption. This project was kicked off 3 years ago and became incubated during 2014 and now moves to the front of the stage at the upcoming OpenStack Vancouver Conference taking place this month with customer stories of Manila deployments in Enterprise and Telco environments.
The project was originally sponsored and accelerated by NetApp and Red Hat and has established a very rich community that includes code contribution fromcompanies such as EMC, Deutsche Telekom, HP, Hitachi, Huawei, IBM, Intel, Mirantis and SUSE.
The momentum of cloud shared file services is not limited to the OpenStack open source world. In fact, last month at the AWS Summit in San Francisco, Amazon announced it new Shared File Storage for Amazon EC2, The Amazon Elastic File System also known for EFS. This new storage service is an addition to the existing AWS storage portfolio, Amazon Simple Storage Service (S3) for object storage, Amazon Elastic Block Store (EBS) for block storage, and Amazon Glacier for archival, cold storage.
The Amazon EFS provides a standard file system semantics and is based on NFS v4 that allows the EC2 instances to access file system at the same time, providing a common data source for a wide variety of workloads and applications that are shared across thousands of instances. It is designed for broad range of use cases, such as Home directories, Content repositories, Development environments and big data applications. Data uploaded to EFS is automatically replicated to different availability zones, and because EFS file systems are SSD-based, there should be few latency and throughput related problems with the service. The Amazon EFS file system as a service allows users to create and configure file systems quickly with no minimum fee or setup cost, and customers pay only for the storage used by the file system based on elastic storage capacity that automatically grows and shrinks when adding and removing files on demand.
The recent Amazon EFS Preview announcement is joining another commercial Cloud File Service Preview: The Microsoft Azure File Service that was announced last year at TechEd 2014: The Azure File service exposes file shares using the standard SMB 2.1 protocol that is currently limited to CIFS only. Applications and workloads can share files between VMs using standard file system APIs, along to a REST interface, which opens a variety of hybrid scenarios. The Azure Files is built on the same technology as the Blob, Table, and Queue Services, which means Azure Files is able to leverage the existing availability, durability, scalability, and geo redundancy that is built into its platform. From a use cases point of view, Azure Files makes it easier to “lift and shift” applications to the cloud that use on-premise file shares to share data between parts of the application.
Meet OpenStack Manila
Manila is a community-driven “Shared Filesystems as a service” project for OpenStack that aims to provide a set of services for management of shared filesystems across OpenStack Compute instances in a multi-tenant cloud environment, alongside the existing OpenStack Cinder block storage and Swift object storage services.
Manila has a pluggable infrastructure framework that provides a vendor neutral management API that allows for provisioning and attaching different shared file systems. In fact, it has support for multiple protocols such as GlusterFS, GPFS, HDFS and ZFS, and backend driver implementations. It provides provisioning and management in a “Distributed file system for cloud” fashion for VMs and physical hardware. Different from AWS EFS, Manila is not the actual shared file system rather than the control plane that can, for example, provide access to an existing CIFS share or create a new NFS export and map it between VM specific instances, where the service itself is not in the the data path.
Manila Use cases
Here are some key use cases that the Manila project can help to address:
- Replace home-grown NAS provisioning tools
- Support traditional enterprise applications
- On-Demand development and build environments
- Integration with existing automation frameworks through REST API or CLI
- Support “cloud-native” workloads, such DBaaS
- Big Data - via Manila’s HDFS native driver plugin
- Provide a secure cross-tenant file sharing
- Hybrid Cloud shares (external consumption of shares / migration of workloads to the cloud from on-premise file shares)
Behind The Scenes
The Manila service architecture consists of the following key components:
- manila-api - service that provides a stable RESTful API. The service authenticates and routes requests throughout the Shared Filesystem service.
- python-manilaclient - Command line interface to interact with Manila via manila-api and also a Python module to interact programmatically with Manila.
- manila-scheduler - Responsible for scheduling/routing requests to the appropriate manila-share service. It does that by picking one back-end while filtering all except one back-end.
- manila-share - Responsible for managing Shared File Service devices, specifically the back-end devices.
- Auth Manager - Component that is responsible for users/projects/and roles.
- SQL database - Manila uses a sql-based central database that is shared among Manila services in the system.
Manila has a “share network” notion - where share_network is a tenant-defined object that informs Manila about the security and network configuration for a group of shares. Manila also has a notion of a “Security Service” that is a set of options that defines a security domain for a particular shared file system protocol, such as an Active Directory domain, LDAP or a Kerberos domain. The security_service contains the information necessary for Manila to create a server that joins the given domain. Manila requires the user by default to create a share network, after the creation of the share network, the user can proceed to create the shares. Users in Manila can configure multiple back-ends just like Cinder. Manila has a share server assigned to every tenant. Similar to Cinder, it uses Intelligent scheduling of Shares using Filter Scheduler and Multi-backend support. Support for shares in filter scheduler allows the cloud administrator to manage large-scale shared storage by filtering backends based on predefined parameters.
Manila offers a full lifecycle shares management where tenants create, delete, list, get details, snapshot, and modify access for shares, coordinate mounting and unmounting of file system shares via the Horizon dashboard or the REST interface. It uses share access rules (ACL) to define which clients can access the shares within a single tenant space. It also supports full multi-tenancy so that drivers for storage controllers with support for secure multi-tenancy are able to automatically create virtual instances of storage servers in a multi-tenant configuration. (Manila supports the following Access Control types: IP address, User name or SSL certificate based)
Manila admins can create a unique “share-types” - where a share_type is an administrator-defined "type of service", comprised of a tenant visible description, and a list of non-tenant-visible key/value pairs (extra_specs) which the Manila scheduler uses to make provisioning decisions for each share request based on the capacity and capabilities of the storage made available to Manila (similar to Cinder scheduler), where it is up to the vendor driver to publish the backend capabilities back to the scheduler to match the names of the extra_specs. In fact, just like Cinder, this is where the different vendors can expose more unique capabilities such as deduplication or compression for example.
Manila’s Momentum
The new OpenStack Kilo upstream release, that became available on April 30th, 2015, marks significant momentum for the Manila project with its increased development capacity that was able to deliver more than 40 new blueprints that were implemented in this cycle. The Kilo release also brings a large increase of new and diverse ecosystem vendors drivers, such as EMC Isilon, Hitachi Scale-out-Platform, HDFS, HP 3PAR, Huawei V3 Storage, Oracle ZFS Storage Appliance & Quobyte.
This new Manila drivers velocity also raises the need for a Manila Third-Party Continuous integration (CI) testing that will require the vendors who wish to submit their drivers to setup a CI system in their labs to make sure their driver passes Tempest tests for every new Manila commit in order to maintain stability, but it also raises the need for more drivers interoperability and certifications testing.
Another notable addition to this release that was led by Red Hat is the new gateway mediated networking model with NFS-Ganesha that opens the door to allow vendor drivers support for multi-protocols such as NFSv2, v3, v4, v4.1, v4.2 & pNFS to enrich the Manila service catalog.
Using GlusterFS via NFS-Ganesha, the Manila file shares are abstracted from underlying hardware and can be grown, shrunk, or migrated across physical systems as necessary. Storage servers can be added or removed from the system dynamically with data rebalanced across the trusted pool of servers, where the data is always online - this addresses the File Share Elasticity required to provide a Scale-out / Scale-down NAS on demand. It also delivers File Share Availability as GlusterFS enables you to replicate the whole storage system between different data centers and across geographic locations.
A noteworthy improvement in Kilo was made around the networking model for Manila. The complexity involved in Manila networking has been vastly simplified enabling more options for deployment. (until Kilo release, it was pretty much Neutron and a share server for every tenant with L2 networking as the main choices). In addition to core framework and driver improvements, several features such as pool aware scheduler, access level for shares and private shares have been implemented.
The Road Ahead
The Manila Liberty roadmap is already in progress and will be the focus of the Mania tracks in the upcoming OpenStack Liberty Design Summit in Vancouver. Features such as support for Share Migration, Data Replication, Quality of Service, Consistency Groups, Thin Provisioning and IPv6 enablement should take the OpenStack File Share service to the next level.
The Manila File Share service is already available in OpenStack RDO community-supported distribution (via packstack) since the Juno release and is slated to be introduced in the upcoming Red Hat Enterprise Linux OpenStack Platform 7 as a Tech Preview this summer.
Join us in the upcoming Manila sessions at the OpenStack Vancouver Conference:
- An Update on OpenStack's Shared File Services Program
- Big Data Analytics and Docker: The Thrilla in Manila
- Manila 101 & Use Cases
- Taking OpenStack Shared File Storage to the Telco Cloud
- Thomson Reuters: Manila Deployment Customer Story
Image source http://upload.wikimedia.org/wikipedia/commons/4/42/A_corridor_of_files_at_The_National_Archives_UK.jpg
저자 소개
채널별 검색
오토메이션
기술, 팀, 인프라를 위한 IT 자동화 최신 동향
인공지능
고객이 어디서나 AI 워크로드를 실행할 수 있도록 지원하는 플랫폼 업데이트
오픈 하이브리드 클라우드
하이브리드 클라우드로 더욱 유연한 미래를 구축하는 방법을 알아보세요
보안
환경과 기술 전반에 걸쳐 리스크를 감소하는 방법에 대한 최신 정보
엣지 컴퓨팅
엣지에서의 운영을 단순화하는 플랫폼 업데이트
인프라
세계적으로 인정받은 기업용 Linux 플랫폼에 대한 최신 정보
애플리케이션
복잡한 애플리케이션에 대한 솔루션 더 보기
오리지널 쇼
엔터프라이즈 기술 분야의 제작자와 리더가 전하는 흥미로운 스토리
제품
- Red Hat Enterprise Linux
- Red Hat OpenShift Enterprise
- Red Hat Ansible Automation Platform
- 클라우드 서비스
- 모든 제품 보기
툴
체험, 구매 & 영업
커뮤니케이션
Red Hat 소개
Red Hat은 Linux, 클라우드, 컨테이너, 쿠버네티스 등을 포함한 글로벌 엔터프라이즈 오픈소스 솔루션 공급업체입니다. Red Hat은 코어 데이터센터에서 네트워크 엣지에 이르기까지 다양한 플랫폼과 환경에서 기업의 업무 편의성을 높여 주는 강화된 기능의 솔루션을 제공합니다.