oracle 11g RAC :ORA-17503: ksfdopn:2 Failed

SQL> startup
ORA-01078: failure in processing system parameters
ORA-01565: error in identifying file '+SYSTEM/itpux/spfileitpux.ora'
ORA-17503: ksfdopn:2 Failed to open file +SYSTEM/itpux/spfileitpux.ora
ORA-12547: TNS:lost contact
SQL> exit

 

Cause
The issue is caused by incorrect permission of $GRID_HOME/bin/oracle and $ORACLE_HOME/bin/oracle, which lead to connection failure to ASM instance:

ls -al $GRID_HOME/bin/oracle
-rwxr-x--x 1 grid oinstall 200678464 Feb 28 14:54 oracle

ls -al $ORACLE_HOME/bin/oracle
-rwxr-x--x 1 oracle asmadmin 228886191 Feb 28 15:41 oracle

 

Solution
Oracle binary should have permission of 6751. To correct the permission, as owner of oracle binary:

cd $GRID_HOME/bin
chmod 6751 oracle

cd $ORACLE_HOME/bin
chmod 6751 oracle

After the change, permission should show:

ls -l oracle
-rwsr-s--x 1 grid oinstall 152400480 Nov 24 15:49 oracle

Once the permission is fixed, if database still fails to come up, restart Grid Infrastructure.