As of yesterday, my most significant patch yet became a real part of GlusterFS. It’s not a big patch, but it’s significant because what it adds is enforcement of quorum for writes. In operational terms, what this means is that – if you turn quorum enforcement on – the probability of “split brain” problems is greatly reduced. It’s not eliminated entirely, because clients don’t see failures at the same time and might take actions that lead to split brain during that immediate post-failure interval. There are also some failure conditions that can cause clients to have persistently inconsistent models of who has quorum and who doesn’t. Still, for 99% of failures this will significantly reduce the number of files affects by split brain – often down to zero. What will happen instead is that clients attempting writes (actually any modifying operation) without quorum will get EROFS instead. That might cause the application to blow up; if that’s worse for you than split brain would be, then just don’t enable quorum enforcement. Otherwise, you have the option to avoid or reduce one of the more pernicious problems that affect GlusterFS deployments with replication.
There’s another significant implication that might be of interest to those who follow my other blog. As such readers would know, I’m an active participant in the endless debates about Brewer’s CAP Conjecture (I’ve decided that Gilbert and Lynch’s later Theorem is actively harmful to understanding of the issues involved). In the past, GlusterFS has been a bit of a mess in CAP terms. It’s basically AP, in that it preserves availability and partition tolerance as I apply those terms, but with very weak conflict resolution. If only one side wrote to a file, there’s not really a conflict. When there is a conflict within a file, GlusterFS doesn’t really have the information it needs to reconstruct a consistent sequence of events, so it has to fall back on things like sizes and modification times (it does a lot better for directory changes). In a word, ick. What quorum enforcement does is turn GlusterFS into a CP system. That’s not to say I like CP better than AP – on the contrary, my long-term plan is to implement the infrastructure needed for AP replication with proper conflict resolution – but I think many will prefer the predictable and well understood CP behavior with quorum enforcement to the AP behavior that’s there now. Since it was easy enough to implement, why not give people the choice?
About the author
Browse by channel
Automation
The latest on IT automation that spans tech, teams, and environments
Artificial intelligence
Explore the platforms and partners building a faster path for AI
Open hybrid cloud
Explore how we build a more flexible future with hybrid cloud
Security
Explore how we reduce risks across environments and technologies
Edge computing
Updates on the solutions that simplify infrastructure at the edge
Infrastructure
Stay up to date on the world’s leading enterprise Linux platform
Applications
The latest on our solutions to the toughest application challenges
Original shows
Entertaining stories from the makers and leaders in enterprise tech
Products
- Red Hat Enterprise Linux
- Red Hat OpenShift
- Red Hat Ansible Automation Platform
- Cloud services
- See all products
Tools
- Training and certification
- My account
- Developer resources
- Customer support
- Red Hat value calculator
- Red Hat Ecosystem Catalog
- Find a partner
Try, buy, & sell
Communicate
About Red Hat
We’re the world’s leading provider of enterprise open source solutions—including Linux, cloud, container, and Kubernetes. We deliver hardened solutions that make it easier for enterprises to work across platforms and environments, from the core datacenter to the network edge.
Select a language
Red Hat legal and privacy links
- About Red Hat
- Jobs
- Events
- Locations
- Contact Red Hat
- Red Hat Blog
- Diversity, equity, and inclusion
- Cool Stuff Store
- Red Hat Summit