Purity DeepReduce: The Next Frontier in Data Efficiency

This is the last installment in our series on rethinking storage efficiency for the AI era. As we’ve already established in previous blog posts in the series, storage costs don’t lie. Ask any data protection or infrastructure professional what keeps them up at night, and runaway storage capacity costs will be near the top of the list. Backup-heavy environments, archival workloads, and rapidly growing modern workload storage deployments can eat through raw capacity faster than most budgets can keep up.

What if you could dramatically shrink the amount of physical storage you actually need—without sacrificing performance or reliability?

That’s exactly what Everpure™ Purity DeepReduce™ is built to do. And it’s changing the game for customers. So, let’s dive right into the specifics on what Purity DeepReduce is and how it operates.

What is Purity DeepReduce?

Purity DeepReduce is next-generation data reduction technology engineered by Everpure. It goes well beyond traditional compression by layering in intelligent, system-wide data reduction—finding similar data across your entire storage system and eliminating unnecessary copies automatically.

Think of it this way: Rather than just compressing what’s already there, DeepReduce actively hunts for data that looks alike across your storage environment and stores only one efficient version. The result? You store far less while your applications see exactly what they expect.

DeepReduce operates quietly in the background while your system continuously optimizes behind the scenes—no disruption, no tradeoffs—only predictability.

How DeepReduce works: Smarter than conventional deduplication

Most data reduction technologies on the market today rely on two basic tools: compression (making individual files smaller) and deduplication (removing exact duplicate copies of data). Both are useful, but both have meaningful blind spots that we’ve discussed in a previous blog post in this series. DeepReduce is designed to address those blind spots head-on, combining state-of-the-art similarity detection with a system-wide engine that works completely behind the scenes.

Let’s break down exactly how it works.

The problem with conventional deduplication

Classic deduplication operates on a simple binary rule: If two blocks of data are bit-for-bit identical, one of them gets removed. This works reasonably well for static data sets where the same file is stored multiple times unchanged.

But the modern real world doesn’t work that cleanly. In backup environments, for example, each nightly backup is a slightly modified version of yesterday’s—not an exact copy. In archival workloads, documents evolve over time with minor edits. In AI data sets, media, and healthcare, files are often structurally similar but not identical.

Conventional deduplication completely misses all of these savings opportunities. DeepReduce doesn’t!

 Figure 1: Conventional deduplication only removes exact duplicates. DeepReduce detects redundancies within sub-blocks instead of entire blocks and stores only the difference—unlocking far greater savings.

Similarity-based reduction: Finding the ‘almost identical’ opportunities

At the heart of Purity DeepReduce is a technique called similarity-based reduction. Here’s how it works in plain terms:

1. Chunking: Each block of data is divided into several smaller sub-sections. DeepReduce keeps the chunking granular and variable sized to get optimal similarity matches globally across the system.
2. Fingerprinting: Each granular sub-section gets a unique digital fingerprint (a short mathematical hash signature that represents its content).
3. Similarity matching: The system queries a global index (maintained and stored by Purity) to find other sub-sections from data blocks that share similar fingerprints—even if they aren’t perfectly identical.
4. DeepReduce compression filter: When a similar pair is found, DeepReduce identifies one as the parent data block (the reference) and compresses the other block against it—storing only the differences, called the referencing data block.

The result? Instead of storing two large, similar blocks, the system stores one full block and one tiny “difference file.” For workloads with lots of incremental changes, like backup, archival, or object version-controlled data sets, this approach dramatically shrinks the physical storage footprint.

 Figure 2: Overview of Purity DeepReduce workflow.

Here’s how DeepReduce compares to older approaches:

Global scope of data reduction

Not all data reduction approaches are created equal. Where a system looks for data reduction opportunities makes a massive difference in how much space it can save.

Purity DeepReduce does the following to maximize data reduction returns:

• Global-level range: Search for similar blocks across the entire storage system, spanning all storage partitions simultaneously. This delivers the highest possible data reduction and is the approach DeepReduce uses.

The choice of global scope is what truly sets Purity DeepReduce apart, but making it work across a distributed system like FlashBlade^® required certain fundamental services to be added to Purity//FB that would help optimize operations and gain all the data reduction possible from the data set in the system.

The global index: The brain of DeepReduce

To make system-wide similarity matching possible, DeepReduce maintains a global index service—essentially a massive, distributed lookup table of data fingerprints.

Every data block chunk’s fingerprints are registered in this index. However, only the fingerprints of the parent data block chunks are kept as the reference data block chunks are referencing parent data block chunks and the reduced block is already written on DirectFlash^® Modules. 

When the background optimization engine encounters a new block to evaluate, it queries the index and gets back a list of similar chunks from various data blocks anywhere in the system—across all storage partitions. The index is intelligently distributed so that the fingerprint space is evenly spread across all partitions, avoiding bottlenecks.

The background optimization engine: Step by step

One of the most important design decisions for DeepReduce is that data reduction happens entirely in the background—completely separate from the inline data write path. Your applications write data at full speed, and DeepReduce then works asynchronously to identify and act on savings opportunities.

Here’s the step-by-step flow:

Figure 3: DeepReduce’s five-step background optimization process—from data ingestion to space reclamation.

1. Data lands at full speed. Write operations proceed normally. There is no inline overhead on the write path.
2. A background scanner comes to work. It scans data blocks and gets the other services involved to start DeepReduce processing.
3. Fingerprinting and index lookup. For each data block, the system breaks it into granular chunks, generates fingerprints, and queries the global index service to find similar blocks anywhere in the system. In certain cases, there might be multiple choices of similar parent data blocks presented, and DeepReduce chooses the best option from the choices available—optimized referencing.
4. Reference link creation. If a similar match is found, the system designates one block as the parent data block and compresses the other into a referencing data block. It creates a lightweight link between the two—recording the relationship so data can always be reconstructed accurately on a read request. 
5. Space is reclaimed. The delta version of the referencing data block is stored on DirectFlash Modules (DFMs), the redundant full copy of that block already residing on the DFMs is released, and the freed physical storage space becomes available. Applications continue reading data normally—they see exactly what they stored, with no awareness of the underlying optimization.

Global reduction across all storage partitions

Here’s a visual look at how DeepReduce operates at a system-wide level—something no siloed, partition-only solution can match:

Figure 4: The global index service enables DeepReduce to link a referencing data block in one storage partition to a parent data block in another—maximizing data reduction across the full system.

When a referencing data block in Partition 1 is linked to a parent data block in Partition 3, the system maintains this relationship carefully. Every operation—reads, snapshots, replication, high availability failover, and even data deletion—is aware of these cross-partition links and handles them correctly. No data integrity is ever compromised.

When a referencing data block needs to be read, FlashBlade transparently retrieves both the referencing data block and the parent data block, reconstructs the original data in memory, and delivers it to the application—all without the application ever knowing a reference link existed.

Staying smart over time: Continuous re-optimization and handling HA events

Data isn’t static. Files get overwritten, backup policies change, and data sets evolve. DeepReduce accounts for this with continuous re-optimization.

As data changes over time, the background engine can evaluate whether existing deduplication links still make sense:

• If a parent data block was deleted, and there are reference data blocks referencing that parent data block, then the parent data block is removed from the filesystem/bucket space and reports it under “shared space,” and makes sure that the parent data block is deleted only when all references to that block are dissolved.
• Indices for chunks (of parent data blocks) are stored in memory and written to DFMs at regular intervals; however, if there are any HA events that happen before the indices are written to DFMs, the background engine is equipped with shadowing links that keep the links and index information intact during such unforeseen scenarios.

This means DeepReduce doesn’t just optimize data reduction—it keeps track of all scenarios and runs data reduction efficiently as your environment evolves.

What makes DeepReduce different from the competition?

Plenty of storage vendors offer different flavors of data reduction techniques. So what sets DeepReduce apart?

Global scope with granular approach, not local scope

Many data reduction solutions only look within a single volume or partition. DeepReduce scans across the entire storage system—unlocking reduction opportunities that siloed approaches simply miss. And most importantly, DeepReduce works on granular chunks (sub-sections of a block) that help in maximizing similarity reduction opportunities.

Similarity based, not just exact match

Conventional deduplication only removes perfect duplicates. DeepReduce goes further by catching near-duplicates—the kind of data that modern workloads generate constantly—and reduces them efficiently.

Transparent and non-disruptive

There’s no separate feature to manage, no windows to schedule, and no manual tuning required. DeepReduce runs entirely in the background as part of the FlashBlade system’s normal operation and is always-on. Additionally, DeepReduce is client protocol- or tenant-agnostic, where it does not matter if the data is from a file system or bucket or Tenant 1 or Tenant 2. DeepReduce just transparently works on reducing the data.

Unified data reduction reporting

With DeepReduce enabled, your data reduction ratio now reflects the combined benefit of both inline compression and global similarity reduction—giving you a single, accurate picture of your true storage efficiency in the FlashBlade management interface.

No bytes left behind

DeepReduce works at the data layer—exercising a “no bytes left behind” strategy, i.e., all data in the system becomes a candidate for DeepReduce, whether it’s snapshot data or object version data. Every block is given an opportunity for further possible reduction.

The TCO impact: What this means for your budget

Better data reduction ratios have a direct effect on your total cost of ownership (TCO), enabling you to:

• Buy less raw capacity to hold the same amount of effective data
• Reduce your cost per usable terabyte significantly while providing predictability in unforeseen storage price increase scenarios
• Scale without proportional cost increases—DeepReduce efficiency holds as your environment grows
• Simplify your storage footprint—fewer drives, lower power consumption, smaller rack space

The bottom line

Data reduction has always been a core part of the Everpure value proposition. With Purity DeepReduce, we’re taking it to a new level—combining the best of compression and intelligent global similarity reduction in a solution that’s fully transparent, massively scalable, and built for the modern workloads that matter most.

If you’re running modern workloads at scale and you’re not getting the data reduction ratios your TCO model demands, it’s time to take a closer look at what Purity DeepReduce can do for you.

Talk to an Everpure specialist to learn how FlashBlade DeepReduce technology can cut your storage costs and maximize efficiency across your environment.