Troubleshooting 'cursor: pin S wait on X' waits. (文档 ID 1349387.1)
In this Document
| Purpose |
| Troubleshooting Steps |
| What is a 'Cursor: pin S wait on X' wait? |
| What causes 'Cursor: pin S wait on X' waits? |
| How to diagnose the cause. |
| How to Examine the Diagnostics. |
| SQL ordered by Parse Calls |
| SQL ordered by Version Count |
| Potential Solutions |
| Troubleshooting Other Issues |
| References |
APPLIES TO:
Oracle Database - Enterprise Edition - Version 10.2.0.1 and laterInformation in this document applies to any platform.
***Checked for relevance on 10-June-2013***
PURPOSE
The purpose of this article is to help troubleshoot 'Cursor: pin S wait on X' waits.
TROUBLESHOOTING STEPS
What is a 'Cursor: pin S wait on X' wait?
A cursor wait is associated with parsing in some form. A session may wait for this event when it is trying to get a mutex pin in Share mode but another session is holding the mutex pin on the same cursor object in exclusive. Frequently, waits for 'Cursor: pin S wait on X' is a symptom and not the cause. There may be underlying tuning requirements or known issues.
What causes 'Cursor: pin S wait on X' waits?
- Frequent Hard Parses
If the frequency of Hard Parsing is extremely high, then contention can occur on this pin. - High Version Counts
When Version counts become excessive, a long chain of versions needs to be examined and this can lead to contention on this event - Known bugs
How to diagnose the cause.
Obtain information and diagnostics to help locate the cause.
1. Run AWR and ADDM reports during the time that you see 'Cursor: pin S wait on X' waits and also collect baseline reports when you do not see the waits to make comparisons. The baseline shows us the typical 'background' concurrency and activity going on during the time of both the issue and the baseline and may help identify (for example) culprit SQLs with high version counts.
To gather this it is suggested to run AWR and ADDM for half an hour to an hour interval as follows:
SQL>@$ORACLE_HOME/rdbms/admin/addmrpt.sql
2. Sometimes system state dump is necessary to match known issues. For example, if there is no obvious candidate SQL in AWR, capturing holder or waiter processes in systemstate allows you to focus in on potential problems. Run system state when processes appear hung on 'Cursor: pin S wait on X' waits :
(a) Non-Rac
oradebug setmypid
oradebug unlimit
oradebug dump systemstate 266
wait 90 seconds
oradebug dump systemstate 266
wait 90 seconds
oradebug dump systemstate 266
quit
(b) RAC
oradebug setmypid
oradebug unlimit
oradebug setinst all
oradebug -g all hanganalyze 4
oradebug -g all dump systemstate 266
quit
3. Errorstacks: Another way to obtain process information is with errorstack. Assuming you can identify a blocker, taking errorstacks will provide much the same information as systemstates but with a much reduced disk footprint for trace. Once the ospid of the blocker has been found, an errorstack can be generated:
SQL> oradebug setospid <p.spid from above>
oradebug dump errorstack 3
<< wait 1min>>
oradebug dump errorstack 3
<< wait 1min>>
oradebug dump errorstack 3
exit
In particular, the stack from the resultant trace can be used to match known issues.
The system state and errorstacks are not easily readable; so a Service Request may need to be opened to read the files.
4. It is not always feasible to run system state dump. Thus, this note on finding the blocker can be used as well:
5. Furthermore, the following sqls can be ran to identify the waiters:
FROM v$session s, v$sql t
WHERE s.event LIKE '%cursor: pin S wait on X%'
AND t.sql_id = s.sql_id
6. In 11g RAC, there is another less resource intensive tool that can be used when compared with taking system state dumps:
How to Examine the Diagnostics.
1. Look for high parsing and high version counts from AWR.
Click on *SQL Statistics under Main Report of AWR:
Then, under SQL Statistics click on 'SQL ordered by Parse Calls' or 'SQL ordered by Version Count' to view that information:
SQL ordered by Parse Calls
* Captured SQL account for 95.6% of Total
Parse Calls Executions % Total Parses SQL Id SQL Module SQL Text
668,174 668,014 22.76 gpkdazvcfwkv select DATA_TABLE (...
667,440 668,234 22.74 5p9vjzht9jqb INSERT INTO DATA_TABLE (DATA_I...
From this list, investigate the top SQLs to determine whether this volume of parse calls is excessive or can be reduced.
SQL ordered by Version Count
Version Count Executions SQL Id SQL Module SQL Text
277 392,737 30d5a6v16mpb select FLOW_CONTEXT (...
78 131,104 7c0gj35488xs INSERT INTO PROJECT (DOC_ID, ...
From this list, investigate the SQLs with the high version count. What are the reasons that these statements are not shared? Can this be addressed?
2. From systemstate and errorstack, we are looking for stacks to match known issues.
For systemstates/errorstacks to be useful, it relies them being taken "at the right time". On a fast moving system, processes may have moved on so holders and waiters are not shown. In a perfect example the systemstates should show the blocker and processes being blocked or one of these. The stack from blocker may indicate known issues.
Potential Solutions
1. Tune sqls with high parse count by either investigating the application or the sql.
From AWR, the parse statistics is at the top of the report under load profile:
In this case there are mainly soft parses, however, if there were high hard parses, this may indicate high usage of literals or introduction of many new sqls. In this case consider using bind variables in place of literals.
If there is high percentage of soft parses, then check the application to see if it is using shareable sqls.
Ideally, Execute to Parse should be closer to 100%. The application should parse less and execute many times. See:
Also remember if the shared pool is flushed, then sqls will need to be hard parsed. This also may cause mutex waits. So make sure sqls are in memory once hard parsed and monitor to see if mutex waits are alleviated.
2. High version counts can also cause 'cursor: pin S wait on X' waits.
Check V$SQL_SHARED_CURSOR to see the potential reason for the high version count using:
Document 296377.1 Troubleshooting: High Version Count Issues
There are some notable bugs where high version counts have been a factor:
Document 9689310.8 Bug 9689310 - Excessive child cursors / high VERSION_COUNT / OERI:17059 due to bind mismatch
3. For more known defects, please go to following note and click on known bugs:
Click on the version that applies and review bug or bugs with similar scenario.
4. If the database has been migrated from 10g to 11g and mutex performance issue surfaces, please consider the 11.2.0.2.2 psu + fix for Unpublished Bug 12431716. Many mutex fixes are already included in this patch:
Troubleshooting Other Issues
For guidance troubleshooting other performance issues see:
REFERENCES
NOTE:9689310.8 - Bug 9689310 - Excessive child cursors / high VERSION_COUNT / OERI:17059 due to bind mismatchBUG:10157392 - HIGH VERSION COUNT OF RECURSIVE SQL THOUGH THE FIX FOR BUG 10086843 WAS APPLIED
BUG:9689310 - SPORADIC BUNCHES OF ORA-600 [17059]
NOTE:10157392.8 - Bug 10157392 - High version counts for SQL with binds (BIND_MISMATCH)
NOTE:1291879.1 - Oracle Database Patch Set Update 11.2.0.2.2 Known Issues
NOTE:1298015.1 - WAITEVENT: "cursor: pin S wait on X" Reference Note
NOTE:1377446.1 - * Troubleshooting Performance Issues
NOTE:296377.1 - Troubleshooting: High Version Count Issues
NOTE:438755.1 - High SQL Version Counts - Script to determine reason(s)
NOTE:459694.1 - Procwatcher: Script to Monitor and Examine Oracle DB and Clusterware Processes
NOTE:62143.1 - Troubleshooting: Tuning the Shared Pool and Tuning Library Cache Latch Contention
NOTE:786507.1 - How to Determine the Blocking Session for Event: 'cursor: pin S wait on X'
- Parse_Vers.png(13.75 KB)
- Parses.png(16.69 KB)
- SQL_Stats_link.png(9.94 KB)
- Soft_Parse.png(13.39 KB)
- Oracle Database Products > Oracle Database > Oracle Database > Oracle Database - Enterprise Edition > RDBMS > Database Level Performance Issues (not SQL Tuning)