피드 구독

In December 2012 Red Hat announced its participation in a three-year European Union initiative to create a Large-scale Elastic Architecture for Data-as-a-Service (LEADS). Those three years have come to an end, so what did we learn? Well, the major take home from this is that LEADS can work in principle and it has the potential to alter how companies work with distributed datacenters.

The intention with LEADS was to create a technology that could provide democratized access to public data sets such as from the open web, social media, and data exposed from various open data movements, index that data, and open it to query. This would in turn enable a third party to create a Data-as-a-Service offering.

It is important to note that while this is an EU initiative involving various academic institutions, the proposal was not necessarily to provide a ‘public good’ but to provide a technology that a commercial entity could use to provide a paid-for service targeted at either smaller companies where it would be less feasible economically or technically to accomplish the same thing, or for larger companies that do not wish to perform the operations in-house. The ability to reindex large amounts of data can be expensive for many companies, including those that are not driven by that core requirement. Large organizations such as Google and Apple may be able to afford to do it, but the costs are likely prohibitive for average sized corporations.

Important to the success of LEADS would be its ability to work across a widely distributed geographic area. Therefore one objective and vision was to spread the LEADS infrastructure across several ‘micro’ datacenters. Not only could this architecture make big data more affordable since it would be a shared resource, but it could also reduce latency as users would have a greater chance of being closer to a datacenter, as well as improve resiliency due to the shared nature of the data.

To index the web in this way, micro datacenters in France would, for example, index sites hosted in France to help lower the cost of peering and improve access to the bandwidth; datacenters in Germany would do German sites and so on. Ultimately, aggregated together, this could become a global index.

Another potential advantage of this architecture is the freedom of choice of supplier that it delivers. If you have technology that enables you to run the same software seamlessly across different datacenters in an efficient way, and if for some reason one of your Infrastructure-as-a-Service providers goes down or makes you unhappy, you may be able to switch suppliers more easily.

One of our goals now is to demonstrate that if one datacenter goes down, the system can still work. Already, we have demonstrated that even as a distributed system, efficiency can be maintained.

To be able to provide a more robust, reliable infrastructure the LEADS project members needed to enable a ‘multi-site, locality-aware distributed storage architecture’. Put simply, that means the data is not stored on a single machine nor a single datacenter, it is distributed across multiple machines and multiple datacenters. This is where Infinispan and the LEADS provided improvements came into the project. Locality-aware means the framework lets users decide where they want to push data. It is this ability to push data where users want that powers the potential advantages of LEADS – lower costs, reduced latency and greater resilience.

There are existing tools that let users push data into two or three datacenters, but with LEADS we have a framework that can give users a more fine-tuned and granular choice as to how and where they can push data. This capability can be used to make placement decisions based on where the index data comes from and the kind of queries being executed.

We see a lot of interesting potential for this capability to push data out across multiple sites; consequently we have put this technology into the Infinispan community (Infinispan Ensemble) to assess the pick up and see whether there are technical limitations that we need to overcome.

Of course, indexing the data is not enough – you need to be able to query the data, and the LEADS platform enables a user to have the ability to do that in a distributed fashion: the system can maximize local query execution as much as possible to help eliminate most of the cross-datacenter communication. This can keep costs down, reduce latency and improve resiliency.

Infinispan users are able to perform distributed queries across datacenters. In practice that means there is a single data grid in each of the datacenters, and that a a framework on top talks to all of the individual data grids, and the aggregation is then carried out to make it look like it’s a single data grid. It helps keep the programming model simple while offering the multi-datacenter approach.

What might this all look like in a commercial setting? Why build solutions on top of indexed public data? Say you sell goods in a competitive market. You could build software on top of the query system that provides sentiment analysis to determine whether a given item or item category is liked more or less over time, and why. This can help detect whether a blogger or group of influencers is unhappy about a certain product, and help map trends over time. The goal is to have the ability to spot a trend and then predict what could be popular. By having the potential ability to predict what could be popular four to six months in the future, you may be able to gain a competitive advantage.


저자 소개

UI_Icon-Red_Hat-Close-A-Black-RGB

채널별 검색

automation icon

오토메이션

기술, 팀, 인프라를 위한 IT 자동화 최신 동향

AI icon

인공지능

고객이 어디서나 AI 워크로드를 실행할 수 있도록 지원하는 플랫폼 업데이트

open hybrid cloud icon

오픈 하이브리드 클라우드

하이브리드 클라우드로 더욱 유연한 미래를 구축하는 방법을 알아보세요

security icon

보안

환경과 기술 전반에 걸쳐 리스크를 감소하는 방법에 대한 최신 정보

edge icon

엣지 컴퓨팅

엣지에서의 운영을 단순화하는 플랫폼 업데이트

Infrastructure icon

인프라

세계적으로 인정받은 기업용 Linux 플랫폼에 대한 최신 정보

application development icon

애플리케이션

복잡한 애플리케이션에 대한 솔루션 더 보기

Original series icon

오리지널 쇼

엔터프라이즈 기술 분야의 제작자와 리더가 전하는 흥미로운 스토리