If you use Oracle database on NFS, make sure you use Oracle DirectNFS, which is a built in NFS client in Oracle database kernel. This makes it possible for the database software to access files on NFS share without calling the operating system kernel. The performance improvement is significant!

I use NFS a lot for Oracle databases, because it is so much easier to manage than any block device based system, specially when using Oracle RAC. Or when you need to clone the database.

And then one day I noticed that some Data Guard standby databases, rather large and busy ones, started to lag – more and more. Confusion started – why – same standby on another datacenter on same type of storage is completely fine. It was fine on this system also up to last week. What has changed? The long confusion part is not important… Until realised, that on the system that was working fine, Linux OS command nfsiostat (displaying Kernel level NFS statistics) – it showed hardly any traffic at all. But on the problematic system, nfsiostat displayed a very busy NFS traffic. The aha moment – DNFS! Linux kernel should not be able to see DNFS traffic.

When doing major OS upgrades, after you should also relink your Oracle database binaries:

Relinking Oracle Home FAQ ( Frequently Asked Questions) (Doc ID 1467060.1)

The note above even states that you should relink after every OS patch… So I just incuded relinking to the playbook that replaces database VM OS root disk. But since DNFS (probably other similar features too, like Unified Auditing??) needs to be linked into Oracle kernel separately – executing “relink all” silently disables DNFS.

Lets test. I’m using Oracle RDBMS 19.15.2, single instance. First I have DNFS enabled and working:

SQL> select id, svrname, dirname, nfsversion from v$dnfs_servers;

        ID SVRNAME    DIRNAME         NFSVERSION
---------- ---------- --------------- ----------------
         1 10.122.9.9 /pt101          NFSv3.0

Then I shut down the database and execute relink all.

SQL> shu immediate
oracle$ lsnrctl stop
oracle$ umask 022 && $ORACLE_HOME/bin/relink all
root$ $ORACLE_HOME/root.sh

Lets start up and see what happened.

SQL> startup mount
Database mounted.

SQL> alter database open;

Database altered.

SQL> select id, svrname, dirname, nfsversion from v$dnfs_servers;

no rows selected

And no DNFS! alert log is also missing the DNFS message:

Oracle instance running with ODM: Oracle Direct NFS ODM Library Version 6.0

Let shut it down and link DNFS back.

SQL> shu immediate

oracle$ cd $ORACLE_HOME/rdbms/lib && make -f ins_rdbms.mk dnfs_on
root$ $ORACLE_HOME/root.sh

SQL> startup mount
SQL> select id, svrname, dirname, nfsversion from v$dnfs_servers;

        ID SVRNAME    DIRNAME         NFSVERSION
---------- ---------- --------------- ----------------
         1 10.122.9.9 /pt101          NFSv3.0

And I have my precious DNFS back. So keep that in mind when you relink Oracle home binaries. Would be good to test if the same problem affects Unified Auditing.