Oracle ASM Filter Driver Filtering and 5.15 UEK7 Kernels

This article on Oracle ASM Filter Driver filtering originally appeared on Ron Ekins’s blog. It has been republished with the author’s credit and consent. 

I’m a long-time fan of the Oracle ASM Filter Driver (AFD). One of its major benefits is AFD filtering.

AFD can ensure writes are only performed by trusted and registered Oracle processes, blocking potentially damaging writes that would have otherwise resulted in data corruption.

Unfortunately, a recent Linux kernel change has impacted the AFD and the latest recommendation from Oracle is to disable AFD filtering until a fix is available.

From Oracle Database 19.24, filtering is disabled by default, but for earlier versions, Oracle has provided Patch 35618858: Disable Filtering by Default for 5.15 and above Kernels.

Available Oracle Release Update (RU) Patches

• Oracle Clusterware 19 Release (19.23.0.0.240416ACFSRU)
• Oracle Clusterware 19 Release (19.24.0.0.0ACFSRU)
• Oracle Clusterware 19 Release (19.25.0.0.0ACFSRU)

In this blog post, I’ll explore the impact of disabling AFD filtering on space reclamation on all-flash storage arrays. 

Kernel Version

I’ve recently upgraded my Oracle RAC servers to Oracle Linux 8.10 using kernel 5.15. You can check your kernel version using uname -r:

[oracle@z-rac1 ~]$ cat /etc/oracle-release

Oracle Linux Server release 8.10

[oracle@z-rac1 ~]$ uname -r

5.15.0-208.159.3.el8uek.x86_64

Oracle ASMFD State

We can use the asmcmd afd_state command to get the ASM Filter Driver status, for example:

[oracle@z-rac1 ~]$ asmcmd afd_state

ASMCMD-9526: The AFD state is 'LOADED' and filtering is 'ENABLED' on host 'z-rac1.uklab.purestorage.com'

In a RAC cluster, use the asmcmd afd_state –all option to see the status of all nodes. All nodes should have the same Filtering Status.

[oracle@z-rac1 ~]$ asmcmd afd_state --all

node 'z-rac1': State of AFD: LOADED and ASMFD Filtering Status: ENABLED

node 'z-rac2': State of AFD: LOADED and ASMFD Filtering Status: DISABLED

Disable ASMFD Filtering

Disable filtering using the asmcmd afd_filter -d command:

[oracle@z-rac1 ~]$ asmcmd afd_filter -d

[oracle@z-rac1 ~]$ asmcmd afd_state

ASMCMD-9526: The AFD state is 'LOADED' and filtering is 'DISABLED' on host 'z-rac1.uklab.purestorage.com'

Enable ASMFD Filtering

If required, you can re-enable filtering using the asmcmd afd_filter -e command, for example:

[oracle@z-rac1 ~]$ asmcmd afd_filter -e

[oracle@z-rac1 ~]$ asmcmd afd_state

ASMCMD-9526: The AFD state is 'LOADED' and filtering is 'ENABLED' on host 'z-rac1.uklab.purestorage.com'

Storage Space Reclamation

Now, this raises a big question: If I disable the ASM Filter Driver, do I lose the ability to send SCSI UNMAP commands back to an all-flash storage array to release unused space?

OK, I have a two-node RAC cluster with ASMFD filtering DISABLED.

[oracle@z-rac1 ~]$ asmcmd afd_state --all

node 'z-rac1': State of AFD: LOADED and ASMFD Filtering Status: DISABLED

node 'z-rac2': State of AFD: LOADED and ASMFD Filtering Status: DISABLED

Create Oracle Tablespace

To demonstrate this, let’s create a 4TB tablespace and wait for Oracle to initialise the datafile.

Let’s confirm there is no space being used by using my v$asm_disk query:

ASMFD Disks

Disk Disk Disk Total   Used     Free

Nbr Name Label   Header  Mount   State   Path     GB     GB       GB

---- ---------- ---------- ------- ------- ------- ---------- -------- -------- --------

  0 DATA08 DATA08   MEMBER  CACHED  NORMAL  AFD:DATA08 12,288       0   12,288

  1 DATA07 DATA07   MEMBER  CACHED  NORMAL  AFD:DATA07 12,288       0  12,288

  2 DATA05 DATA05   MEMBER  CACHED  NORMAL  AFD:DATA05 12,288       0   12,288

  3 DATA06 DATA06   MEMBER  CACHED  NORMAL  AFD:DATA06 12,288       0   12,288

And now create the 4TB bigfile tablespace.

SQL> create BIGFILE tablespace test datafile '+AFDATA' size 4T;

Tablespace created.

Using the v$asm_disk query, I can see my four ASMFD volumes have used in total 4TB as expected.

ASMFD Disks

Disk Disk Disk Total   Used     Free

Nbr Name Label   Header  Mount   State   Path     GB     GB       GB

---- ---------- ---------- ------- ------- ------- ---------- -------- -------- --------

  0 DATA08 DATA08   MEMBER  CACHED  NORMAL  AFD:DATA08 12,288   1,024   11,264

  1 DATA07 DATA07   MEMBER  CACHED  NORMAL  AFD:DATA07 12,288   1,024   11,264

  2 DATA05 DATA05   MEMBER  CACHED  NORMAL  AFD:DATA05 12,288   1,024   11,264

  3 DATA06 DATA06   MEMBER  CACHED  NORMAL  AFD:DATA06 12,288   1,024   11,264

Using the asm command line utility asmcmd with the list attribute option, I see that thin provisioning is enabled on my AFDATA diskgroup:

[oracle@z-rac1 ~]$ asmcmd lsattr -G AFDATA -l | grep thin

thin_provisioned               TRUE

Using the FlashArray™ CLI and performing the purevol list <volume name> –-space –-total command, we can confirm the space used by the tablespace in the AFD diskgroup. For example:

purevol list z-racpod::afd_data* –space –total

pureuser@z-x90-a> purevol list z-racpod::afd_data* --space --total

Name                  Size  Virtual   Thin Provisioning  Data Reduction  Total Reduction  Unique   Snapshots  Total

z-racpod::afd_data05  12T   1.00T    92%                >100 to 1       >100 to 1        4.95G   0.00       4.95G

z-racpod::afd_data06  12T   1.00T    92%                >100 to 1       >100 to 1        4.95G   0.00       4.95G

z-racpod::afd_data07  12T   1.00T    92%                >100 to 1       >100 to 1        4.95G   0.00       4.95G

z-racpod::afd_data08  12T   1.00T    92%                >100 to 1       >100 to 1        4.95G   0.00       4.95G

(total)               48T  4.00T   92%                >100 to 1       >100 to 1        19.79G  0.00       19.79G

Drop Tablespace

Let’s now drop the Oracle tablespace using DROP TABLESPACE <tablespace_name> INCLUDING CONTENTS AND DATAFILES, and see the impact on the storage platform and Oracle database.

SQL> DROP TABLESPACE test INCLUDING CONTENTS AND DATAFILES;

Tablespace dropped.

If we re-run the v$asm_view query, Oracle is now reporting the 4TB test tablespace has been released.

ASMFD Disks

Disk Disk Disk Total   Used     Free

Nbr Name Label   Header  Mount   State   Path     GB     GB       GB

---- ---------- ---------- ------- ------- ------- ---------- -------- -------- --------

  0 DATA08 DATA08   MEMBER  CACHED  NORMAL  AFD:DATA08 12,288       0   12,288

  1 DATA07 DATA07   MEMBER  CACHED  NORMAL  AFD:DATA07 12,288       0   12,288

  2 DATA05 DATA05   MEMBER  CACHED  NORMAL  AFD:DATA05 12,288       0   12,288

  3 DATA06 DATA06   MEMBER  CACHED  NORMAL  AFD:DATA06 12,288       0   12,288

Returning to the Pure Storage® FlashArray, we can see that we are still consuming space even though the Oracle tablespace has been deleted.

pureuser@z-x90-a> purevol list z-racpod::afd_data* --space --total

Name                  Size  Virtual   Thin Provisioning  Data Reduction  Total Reduction  Unique   Snapshots  Total

z-racpod::afd_data05  12T   1.00T    92%                >100 to 1       >100 to 1        4.95G   0.00       4.95G

z-racpod::afd_data06  12T   1.00T    92%                >100 to 1       >100 to 1        4.95G   0.00       4.95G

z-racpod::afd_data07  12T   1.00T    92%                >100 to 1       >100 to 1        4.95G   0.00       4.95G

z-racpod::afd_data08  12T   1.00T    92%                >100 to 1       >100 to 1        4.95G   0.00       4.95G

(total)               48T   4.00T   92%                >100 to 1       >100 to 1        19.79G  0.00       19.79G

Oracle ASMFD Rebalance Compact Phase

When using Oracle ASM Filter Driver, we can manually trigger a SCSI UNMAP command to release space back to the storage array using the ALTER DISKGROUP <NAME> REBALANCE WITH BALANCE COMPACT.

I’ve included the optional WAIT option to only return once complete.

SQL> ALTER DISKGROUP AFDATA REBALANCE WITH BALANCE COMPACT WAIT;

Diskgroup altered.

If we now return to the FlashArray CLI and repeat the purevol list <volume name> –space –total command, we can see the impact of the ALTER DISKGROUP <name> REBALANCE WITH COMPACT.

We can see that even though AFD filtering has been disabled, ASMFD is still able to send the SCSI UNMAP request to the storage array to release the space.

pureuser@z-x90-a> purevol list z-racpod::afd_data* --space --total

z-racpod::afd_data05  12T   1.00T    92%                >100 to 1       >100 to 1        4.95G   0.00       4.95G

z-racpod::afd_data06  12T   1.00T    92%                >100 to 1       >100 to 1        4.95G   0.00       4.95G

z-racpod::afd_data07  12T   1.00T    92%                >100 to 1       >100 to 1        4.95G   0.00       4.95G

z-racpod::afd_data08  12T   1.00T    92%                >100 to 1       >100 to 1        4.95G   0.00       4.95G

(total)               48T   4.00T    92%                >100 to 1       >100 to 1        19.79G  0.00       19.79G

pureuser@z-x90-a> purevol list z-racpod::afd_data* --space --total

Name                  Size  Virtual  Thin Provisioning  Data Reduction  Total Reduction  Unique  Snapshots  Total

z-racpod::afd_data05  12T   92.00M   100%               6.9 to 1        >100 to 1        9.29M   0.00       9.29M

z-racpod::afd_data06  12T   92.00M   100%               6.6 to 1        >100 to 1        10.47M  0.00       10.47M

z-racpod::afd_data07  12T   92.00M   100%               6.7 to 1        >100 to 1        9.82M   0.00       9.82M

z-racpod::afd_data08  12T   92.00M   100%               7.0 to 1        >100 to 1        8.63M   0.00       8.63M

(total)               48T   368.00M  100%               6.8 to 1        >100 to 1        38.21M  0.00       38.21M

Summary

In this blog post, I’ve confirmed that disabling ASMFD filtering has no impact on space reclamation.

The Oracle AFD development team is actively working on a new filtering and protection mechanism for the above-mentioned and newer kernels. When the new filtering solution becomes available, I will give it a try and share my findings.

If you want to learn more about using the ASM Filter Driver to reclaim storage space, check out this blog post.