Patching 19c RAC Environment with the updated version of January, 2023 Release Update(RU)

After pulling out the Oracle 19.18 Jan,2023 RU due the bug mentioned in below article, Oracle has re-released January 2023 19.18 RU which also includes the patch for unpublished bug# 35001966

http://myoracle-world.blogspot.com/2023/02/oracle-re-released-1918-ru-with-patch.html

19.18 RU details

Patch 34762026 - GI Release Update 19.18.0.0.230117 (REL-JAN230131)
Patch 34765931: DATABASE RELEASE UPDATE 19.18.0.0.0 (REL-JAN230131)
Patch 34773489: COMBO OF OJVM RU COMPONENT 19.18.0.0.230117 + DB RU 19.18.0.0.230117 (REL-JAN230131)

In this post, we will see how we can apply the 19.18 RU to both the Database and Grid homes in our 19c RAC Lab environment.

My Lab environment is a 2 node RAC cluster with patch set 19.17 applied. Below is the current RU version applied to the existing GI and DB homes

Source Environment setup
RAC Database : RENODBPR ( renodbpr1 & renodbpr2)
GRID Home : /u01/app/19.3.0.0/grid
RDBMS Home    : /u01/app/oracle/product/19.3.0.0/dbhome_1
Version         : Oracle Database 19c EE - Production Version 19.17.0.0.0
hosts : labhost01
                          labhost02

Grid Home Current patch level

[oracle@labhost01 software]$ echo $ORACLE_HOME
/u01/app/19.3.0.0/grid
[oracle@labhost01 software]$ $ORACLE_HOME/OPatch/opatch lspatches
34580338;TOMCAT RELEASE UPDATE 19.0.0.0.0 (34580338)
34444834;OCW RELEASE UPDATE 19.17.0.0.0 (34444834)
34428761;ACFS RELEASE UPDATE 19.17.0.0.0 (34428761)
34419443;Database Release Update : 19.17.0.0.221018 (34419443)
33575402;DBWLM RELEASE UPDATE 19.0.0.0.0 (33575402)
OPatch succeeded.
[oracle@labhost01 software]$

Database home current patch level

[oracle@labhost01 software]$ echo $ORACLE_HOME
/u01/app/oracle/product/19.3.0.0/db_1
[oracle@labhost01 software]$ $ORACLE_HOME/OPatch/opatch lspatches
34444834;OCW RELEASE UPDATE 19.17.0.0.0 (34444834)
34419443;Database Release Update : 19.17.0.0.221018 (34419443)
OPatch succeeded.
[oracle@labhost01 software]$

Download the Grid Patch , Database patch and the latest OPatch from Oracle and stage it on all the nodes of the cluster(if its RAC)

p34762026_190000_Linux-x86-64.zip - Grid Infrastructure Release Update 19.18.0.0.230117
p34765931_190000_Linux-x86-64.zip - Database Release Update 19.18.0.0.0
p6880880_190000_Linux-x86-64.zip   - Latest Opatch for 19c

The Oracle Grid Infrastructure and  Database patches are cumulative and include the database CPU program security content.

Patch Installation 

It is highly recommended to take a backup of the Oracle home binaries, the Grid home binaries, and Central Inventory prior to applying patches

Patch Installation prerequisites

1. Opatch utility version information

You must use the OPatch utility version 12.2.0.1.34 or later to apply this patch. Oracle recommends that you use the latest released OPatch version for 12.2 which is available for download from My Oracle Support note 6880880 by selecting the ARU link for the 12.2.0.1.0 OPatch release.

Check the current OPatch version on the server

[oracle@labhost01 software]$ $ORACLE_HOME/OPatch/opatch version
OPatch Version: 12.2.0.1.33
OPatch succeeded.
[oracle@labhost01 software]$

Download and unzip the latest OPatch utility to both the Grid and Database homes.

Grid home : 

Node 1

Perform the below steps as root user
[root@labhost01 ~]#  mv /u01/app/19.3.0.0/grid/OPatch /u01/app/19.3.0.0/grid/OPatch.orig1
[root@labhost01 ~]# unzip p6880880_190000_Linux-x86-64.zip -d /u01/app/19.3.0.0/grid
[root@labhost01 ~]# chown -Rf oracle:dba /u01/app/19.3.0.0/grid/OPatch

let's check the OPatch version to see if its updated.

[oracle@labhost01 software]$ $ORACLE_HOME/OPatch/opatch version
OPatch Version: 12.2.0.1.35
OPatch succeeded.
[oracle@labhost01 software]$

Node2

[root@labhost02 ~]#  mv /u01/app/19.3.0.0/grid/OPatch /u01/app/19.3.0.0/grid/OPatch.orig
[root@labhost02 ~]# unzip p6880880_190000_Linux-x86-64.zip -d /u01/app/19.3.0.0/grid
[root@labhost02 ~]# chown -Rf oracle:dba /u01/app/19.3.0.0/grid/OPatch

let's check the OPatch version to see if its updated.
[oracle@labhost02 software]$ $ORACLE_HOME/OPatch/opatch version
OPatch Version: 12.2.0.1.35
OPatch succeeded.
[oracle@labhost02 software]$

Database home 

Node 1:

[oracle@labhost01 software]$mv /u01/app/oracle/product/19.3.0.0/db_1/OPatch /u01/app/oracle/product/19.3.0.0/db_1/OPatch.orig
[oracle@labhost01 software]$ unzip p6880880_190000_Linux-x86-64.zip -d /u01/app/oracle/product/19.3.0.0/db_1/

[oracle@labhost01 software]$ . oraenv
ORACLE_SID = [+ASM1] ? renodbpr1
The Oracle base remains unchanged with value /u01/app/oracle
[oracle@labhost01 software]$ echo $ORACLE_HOME
/u01/app/oracle/product/19.3.0.0/db_1
[oracle@labhost01 software]$  $ORACLE_HOME/OPatch/opatch version
OPatch Version: 12.2.0.1.35
OPatch succeeded.
[oracle@labhost01 software]$

Node 2:
[oracle@labhost02 software]$mv /u01/app/oracle/product/19.3.0.0/db_1/OPatch /u01/app/oracle/product/19.3.0.0/db_1/OPatch.orig
[oracle@labhost02 software]$ unzip p6880880_190000_Linux-x86-64.zip -d /u01/app/oracle/product/19.3.0.0/db_1/

[oracle@labhost02 software]$ . oraenv
ORACLE_SID = [+ASM2] ? renodbpr2
The Oracle base remains unchanged with value /u01/app/oracle
[oracle@labhost02 software]$ echo $ORACLE_HOME
/u01/app/oracle/product/19.3.0.0/db_1
[oracle@labhost02 software]$  $ORACLE_HOME/OPatch/opatch version
OPatch Version: 12.2.0.1.35
OPatch succeeded.
[oracle@labhost02 software]$

2. Check the consistency for the Oracle inventory of both Grid and Database homes

Before we begin apply any patches, it is always strongly  recommended to check the consistency of the inventory information for both the Grid home and Oracle homes where we are applying the patch. 

$ <ORACLE_HOME>/OPatch/opatch lsinventory -detail -oh <ORACLE_HOME>

If this command succeeds, it lists the Oracle components that are installed in the home. Save the output so that you have the status prior to the patch application. However if this command fails, contact Oracle Support Services for assistance.

In our case, its checks out without any issues. Below is formatted output as the output from above command is very huge.

3. Run the Opatch conflict check 

Determine whether any currently installed one-off patches conflict with this patch 34416665 as follows:

Run OPatch Conflict Check

For Grid Infrastructure Home, as oracle user:

export ORACLE_HOME=/u01/app/19.3.0.0/grid
export PATH=$ORACLE_HOME/bin:$PATH

$ORACLE_HOME/OPatch/opatch prereq CheckConflictAgainstOHWithDetail -phBaseDir /u01/software/34762026/34765931
$ORACLE_HOME/OPatch/opatch prereq CheckConflictAgainstOHWithDetail -phBaseDir /u01/software/34762026/34768559
$ORACLE_HOME/OPatch/opatch prereq CheckConflictAgainstOHWithDetail -phBaseDir /u01/software/34762026/34768569
$ORACLE_HOME/OPatch/opatch prereq CheckConflictAgainstOHWithDetail -phBaseDir /u01/software/34762026/34863894
$ORACLE_HOME/OPatch/opatch prereq CheckConflictAgainstOHWithDetail -phBaseDir /u01/software/34762026/33575402

Below is the formatted output of above commands from the node1.

For Oracle database home, as home user:

$ORACLE_HOME/OPatch/opatch prereq CheckConflictAgainstOHWithDetail -phBaseDir /u01/software/34762026/34765931
$ORACLE_HOME/OPatch/opatch prereq CheckConflictAgainstOHWithDetail -phBaseDir /u01/software/34762026/34768559

Below is the formatted output of above commands from the node1.

Repeat the steps for all the nodes of the cluster.

4) Run the OPatch System Space Check

Check if enough free space is available on the ORACLE_HOME filesystem for the patches to be applied as given below:

For Grid Infrastructure home, as home user:
Create file /tmp/patch_list_gihome.txt with the following content:

/u01/software/34762026/34765931
/u01/software/34762026/34768559
/u01/software/34762026/34768569
/u01/software/34762026/34863894
/u01/software/34762026/33575402

Run the OPatch command to check if enough free space is available in the Grid Infrastructure home:

5) One-off Patch Conflict Detection and Resolution

The following commands check for conflicts in both the 19c Grid home and the 19c DB homes.

In case you are applying the patch, run this command:

As root user,

[root@labhost01 ~]# export ORACLE_HOME=/u01/app/19.3.0.0/grid

[root@labhost01 ~]# export PATH=$ORACLE_HOME/bin:$PATH

[root@labhost01 ~]# $ORACLE_HOME/OPatch/opatchauto apply /u01/software/334762026 -analyze

6) Patch Installation 

Lets apply the patch to both Grid and DB homes using 'Opatchauto' 

The OPatch utility has automated the patch application for the Oracle Grid Infrastructure (Grid) home and the Oracle RAC database homes. It operates by querying existing configurations and automating the steps required for patching each Oracle RAC database home of same version and the Grid home.

The utility must be executed by an operating system (OS) user with root privileges, and it must be executed on each node in the cluster if the Grid home or Oracle RAC database home is in non-shared storage. 

To patch the Grid home and all Oracle RAC database homes of the same version: we can use the below command.

export ORACLE_HOME=/u01/app/19.3.0.0/grid

export PATH=$ORACLE_HOME/bin:$PATH

$ORACLE_HOME/OPatch/opatchauto apply /u01/software/34762026 

Node 1: 

Check the applied patches for the database and grid homes.

Repeat the same steps in all the nodes of the cluster.

7) Check the DBA registry of the database

Thanks

Sambaiah Sammeta