Re-define How You Handle Data
Everpure is redefining the storage experience and empowering innovators by simplifying how people consume and interact with data.
Originally published in 2014, this piece remains relevant today—but the story goes deeper than you might think. What started as a bold design decision about power management reveals something fundamental about how Everpure approaches resilience across the entire stack.
In traditional storage, shutting down the array is kind of a big, scary, fingers-crossed kind of deal. Some legacy vendors make you engage their support or professional services teams just to power the thing down safely—the procedure is so risky they won’t let customers do it themselves. When Everpure started from scratch designing FlashArray, we challenged ourselves to completely change the paradigm. But the question wasn’t really about shutdown procedures; the question was whether we could build an array so confident in its ability to handle power loss that we could make pulling the plug our standard procedure.
And that question—that confidence requirement—turned out to be the seed of something much larger.
The Dreaded Double Failure
If you look at legacy storage architectures and analyze why they occasionally suffer data loss or corruption, most of those events trace back to double failures. Something relatively minor fails first—a drive, a controller, maybe an internal switch—which triggers one of the software resiliency features designed to save the day. Here’s where things get interesting (“interesting” = “terrifying”): that resiliency code is often a very under-exercised code function. It was more than likely written years ago to protect against some arcane failure case, tested well at the time, and then it started aging in the code base like a bottle of wine you forgot in the back of the closet, except instead of getting better with age, it gets more suspect.
This code is a fail-safe, so almost no one uses it in production, which makes it a magnet for software bugs—both when it’s initially written and as the code base evolves around it over the years. So on the one day you actually need that code path because of a real failure, you might discover this resiliency code is less reliable than the code it was supposedly protecting. This leads to the second failure achieved and data loss unlocked…be sure to tell your family you won’t be home for dinner for the rest of the week.
Everpure’s Philosophy: No Un-Exercised Code
When we started designing the high availability and resiliency features of FlashArray, our mantra of “no un-exercised code” became something close to religious doctrine with the Everpure engineers. And you can see that commitment manifest itself throughout the architecture—not just in the power-loss handling, but in how we think about writes, reads, and rebuilds across the entire system.
Take parity rebuilds as an example. We felt RAID rebuild code should be just as reliable and performant as the normal read/write path, so we designed an array that constantly reads from parity as part of normal operations. About 15 percent of the read I/O that comes from FlashArray comes from parity by design—it’s how we isolate drives for writes and make our I/O path non-blocking. The rebuild code isn’t some dusty corner of the codebase that only gets executed during emergencies; it’s running every single day, all day long.
We built the FlashArray so that controller failures and failovers were nothing to be afraid of. Its controllers are stateless—no persistent data in them, including in-flight writes—and HA events are designed to be non-events. Pull the power to any Everpure controller at any time, and you won’t see a performance hit. Better yet, you can upgrade Purity during middle-of-the-day with production workloads running without worrying about it (something else IT geeks love to do—prove the vendor wrong about their maintenance windows).
And then there’s the power-loss handling itself. FlashArray has to be able to handle full power loss with ease, which meant we needed to make the full-power-loss code some of the most exercised code in the industry rather than the least. Our design decision? Let’s make turning the array off and pulling the power one and the same function. We had to be so confident in our ability to manage power loss that we made it our standard shutdown procedure.
How Resilience Works When Everything’s Running
But let’s go a little deeper—because the power button story is really just the visible symptom of a much more comprehensive approach to resilience that continuously runs through every FlashArray I/O operation.
When a write arrives at FlashArray, the data gets written to four locations simultaneously before the acknowledgement happens—two copies in redundant NVRAM devices housed in the array’s storage shelves, and a working copy in the DRAM of both controllers. Notice what’s not happening here: the array is not frantically trying to de-stage data from volatile controller memory when power fails, and we’re not relying on some fragile UPS architecture to keep things running long enough to flush everything to disk. The write is committed to four protected locations before we tell the host it’s done, which means power loss at any point is just…not a problem.
This protection extends all the way down to the RAID layer. FlashArray uses a form of RAID to protect against drive failures, but there’s more under the hood: we designed the system so that RAID rebuild operations use the exact same code paths as normal read operations. That 15 percent of reads coming from parity I mentioned earlier? That’s not some background verification job running during idle time—that’s our normal I/O scheduler making intelligent decisions about how to satisfy read requests while simultaneously isolating drives for writes and keeping the I/O path non-blocking.
If a DFM fails and needs to be rebuilt (an extremely rare occurence), we’re not invoking some dusty emergency procedure that only runs during disasters…we’re using the same parity-read mechanisms that have been running constantly since day one that is processing billions of I/O operations. The rebuild code is battle-tested every single day by your production workload–not waiting in the wings hoping it still works when you need it.
The Stateless Controller Advantage
And this philosophy of “no un-exercised code” shows up in how we handle controller failures too. Remember that FlashArray uses a dual-controller architecture, and they are stateless—they don’t hold any persistent data, including in-flight writes. Everything that matters is already protected in those four locations I previously mentioned: two copies in NVRAM and working copies in both controllers’ DRAM.
What this means in practice: you can pull power from either controller at any time without impacting performance or risking data. HA failovers aren’t scary events that trigger emergency procedures— they’re non-events. And because controller failover is just another normal operation that happens regularly (software upgrades, maintenance, testing), that code path stays sharp. You’re not hoping the failover mechanism works when you need it; you’re exercising it routinely as part of normal operations.
Which brings us back to the power button (or lack thereof).
So How Do You Turn FlashArray Off?
By now you probably have the answer: you just pull the power cords. I know – it’s anticlimactic…that’s by design.
In full disclosure, since we use standard off-the-shelf hardware components for the controllers, there are technically legacy physical power buttons on the shelves and controllers—but their use is entirely optional and, in fact, not encouraged. There’s no shutdown button in the GUI or command in the CLI that initiates some elaborate shutdown procedure. If you want to turn it off, you just pull the power. Recovery time? That’s about three minutes—essentially the time it takes the controllers to boot.
This isn’t recklessness; this is confidence born from architectural decisions we made all the way down at the I/O level. Every write is protected in four locations before acknowledgment and every read is exercised in the same code paths that handle failures. These are supported by stateless controllers that can fail over without impacting operations. Every rebuild operation uses the exact same mechanisms that serve your production workload every single day.
What This Means For You
If you’re evaluating Everpure against other vendors making promises about resilience and availability—I’d suggest a few common-sense tests. Ask them about full power-loss scenarios: how does their code work, what levels of protection exist, are there any caveats, and how long does recovery take? And after you get your answer, test it during your proof-of-concept. Fire up a large workload, pull the power to the rack, restore power, and see how long it takes (and if) the array recovers. And if that procedure is not a part of the vendor’s proof of concept guide, insist on it being included.
Better yet, ask them about their rebuild operations: how often does that code get exercised during normal operations? What happens to performance when a drive fails and rebuild kicks in? I talked about this in another blog – the performance quality of a recovery is just as important as the built resilience for a system to function when there is a failure.
And this is what we learned building FlashArray: the way you handle edge cases reveals everything about your architecture. If power loss is a scary special case that requires elaborate procedures to manage safely, that tells you something about the fundamental design assumptions baked into the system. If rebuilds are emergency operations that degrade performance and put you at precarious risk during the rebuild window, that tells you the system wasn’t designed with failure as a normal operating condition.
Everpure challenged all of those accepted risks when FlashArray was designed. This is because we think failure handling should be so reliable that it becomes invisible—exercised constantly, trusted implicitly, and built into the normal flow of every I/O operation, rather than cordoned off into special-case code that only runs when everything’s already gone wrong.
Take the time to understand these differences. They matter—not just when you’re powering down for maintenance, but every single day your storage is running mission critical production workloads. That’s when you want resilience that’s been tested by billions of I/O operations, not resilience that only gets tested when you’re already having a very bad day.
Want to talk about how the Everpure platform supports long term enterprise storage strategies? Let’s have that conversation.
