ORACLE opatch 打补丁fuser command output for /u01/....../bin/crsctl.bin is Failure
最近，本地测试数据库oracle 11.2.0.4 for rhel7打补丁的时候失败了，检查opatch应用补丁的日志显示错误提示如下：
CRS-4133: Oracle High Availability Services has been stopped.
Successfully unlock /u01/app/11.2.0/grid
2020-06-26 17:05:46: Invoking removeproc to clean oracle client procs
2020-06-26 17:05:46: Executing cmd: /sbin/fuser -k /u01/app/11.2.0/grid/bin/crsctl.bin
2020-06-26 17:05:46: Command output:
> Failure in execution (rc=-1, 0, No such file or directory) for command /sbin/fuser -k /u01/app/11.2.0/grid/bin/crsctl.bin
>End Command output
2020-06-26 17:05:46: fuser command output for /u01/app/11.2.0/grid/bin/crsctl.bin is Failure in execution (rc=-1, 0, No such file or directory) for command /sbin/fuser -k
/u01/app/11.2.0/grid/bin/crsctl.bin
2020-06-26 17:05:46: Oracle user for /u01/app/11.2.0/grid is grid
2020-06-26 17:05:46: Opening file /etc/oracle/ocr.loc
2020-06-26 17:05:46: Value (FALSE) is set for key=local_only
2020-06-26 17:05:46: Running as user grid: /u01/app/11.2.0/grid/OPatch/opatch prereq CheckApplicable -ph /tmp/psu/27107360/26925576 -oh /u01/app/11.2.0/grid
2020-06-26 17:05:46: s_run_as_user2: Running /bin/su grid -c ' /u01/app/11.2.0/grid/OPatch/opatch prereq CheckApplicable -ph /tmp/psu/27107360/26925576 -oh /u01/app/11.2.0
/grid '
2020-06-26 17:05:47: Removing file /tmp/filewsfQVx
2020-06-26 17:05:47: Successfully removed file: /tmp/filewsfQVx
2020-06-26 17:05:47: /bin/su exited with rc=2
2020-06-26 17:05:47: Error : The opatch Applicable check failed. The patch /tmp/psu/27107360/26925576 is not applicable to /u01/app/11.2.0/grid
2020-06-26 17:05:47: Running as user grid: /u01/app/11.2.0/grid/OPatch/opatch prereq CheckApplicable -ph /tmp/psu/27107360/26609929 -oh /u01/app/11.2.0/grid
2020-06-26 17:05:47: s_run_as_user2: Running /bin/su grid -c ' /u01/app/11.2.0/grid/OPatch/opatch prereq CheckApplicable -ph /tmp/psu/27107360/26609929 -oh /u01/app/11.2.0
/grid '
2020-06-26 17:05:48: Removing file /tmp/filePKM5qO
2020-06-26 17:05:48: Successfully removed file: /tmp/filePKM5qO
2020-06-26 17:05:48: /bin/su exited with rc=2
2020-06-26 17:05:48: Error : The opatch Applicable check failed. The patch /tmp/psu/27107360/26609929 is not applicable to /u01/app/11.2.0/grid
2020-06-26 17:05:48: Running as user grid: /u01/app/11.2.0/grid/OPatch/opatch prereq CheckApplicable -ph /tmp/psu/27107360/22502505 -oh /u01/app/11.2.0/grid
2020-06-26 17:05:48: s_run_as_user2: Running /bin/su grid -c ' /u01/app/11.2.0/grid/OPatch/opatch prereq CheckApplicable -ph /tmp/psu/27107360/22502505 -oh /u01/app/11.2.0
/grid '
2020-06-26 17:05:49: Removing file /tmp/filecn51l7
2020-06-26 17:05:49: Successfully removed file: /tmp/filecn51l7
2020-06-26 17:05:49: /bin/su exited with rc=2
2020-06-26 17:05:49: Error : The opatch Applicable check failed. The patch /tmp/psu/27107360/22502505 is not applicable to /u01/app/11.2.0/grid
2020-06-26 17:05:49: Status of Applicable check for /u01/app/11.2.0/grid is 0
2020-06-26 17:05:49: Error:Patch Applicable check failed for /u01/app/11.2.0/grid
2020-06-26 17:05:49: Executing cmd: /bin/rpm -q sles-release
2020-06-26 17:05:49: Command output:
> package sles-release is not installed
观察日志，提示比较明显的错误信息是：The opatch Applicable check failed. The patch /tmp/psu/27107360/...... is not applicable to /u01/app/11.2.0/grid。
按照以往的补丁应用经验，按照如下过程进行异常排查：
1、重新上传补丁包，排除补丁包不完整的情况
2、赋予补丁集777权限，grid解压，排除补丁包宿主权限问题
3、将补丁集放在单独的空目录，grid执行解压
经过上诉三个步骤，在应用补丁集时还是提示相同的错误，仔细观察补丁集应用日志，发现如下观察错误提示：
2020-06-26 17:05:46: Command output:
> Failure in execution (rc=-1, 0, No such file or directory) for command /sbin/fuser -k /u01/app/11.2.0/grid/bin/crsctl.bin
>End Command output
2020-06-26 17:05:46: fuser command output for /u01/app/11.2.0/grid/bin/crsctl.bin is Failure in execution (rc=-1, 0, No such file or directory) for command /sbin/fuser -k
/u01/app/11.2.0/grid/bin/crsctl.bin
难道是/sbin/fuser命令不存在，经过排查，发现/sbin/fuser命令真的不在，死马当活马医，执行yum install psmisc，在集群
两个节点安装了psmisc.x86_64 0:22.20-15.el7，再次执行补丁集应用，补丁集应用成功。
# /u01/app/11.2.0/grid/OPatch/opatch auto /u01/soft/27107360 -oh /u01/app/11.2.0/grid
Executing /u01/app/11.2.0/grid/perl/bin/perl /u01/app/11.2.0/grid/OPatch/crs/patch11203.pl -patchdir /u01/soft -patchn 27107360 -oh /u01/app/11.2.0/grid -paramfile /u01/app/11.2.0/grid/crs/install/crsconfig_params
This is the main log file: /u01/app/11.2.0/grid/cfgtoollogs/opatchauto 2020-06-26_18-20-05.log
This file will show your detected configuration and all the steps that opatchauto attempted to do on your system:
/u01/app/11.2.0/grid/cfgtoollogs/opatchauto 2020-06-26_18-20-05.report.log
2020-06-26 18:20:05: Starting Clusterware Patch Setup
Using configuration parameter file: /u01/app/11.2.0/grid/crs/install/crsconfig_params
Stopping CRS...
Stopped CRS successfully
patch /u01/soft/27107360/26925576 apply successful for home /u01/app/11.2.0/grid
patch /u01/soft/27107360/26609929 apply successful for home /u01/app/11.2.0/grid
patch /u01/soft/27107360/22502505 apply successful for home /u01/app/11.2.0/grid
Starting CRS...
Installing Trace File Analyzer
ACFS driver install actions failed
CRS-4123: Oracle High Availability Services has been started.
opatch auto succeeded.