近期进行压力测试时发现,平均响应时间较上一个版本延长10%以上、应用服务器CPU超过90%,这很不科学。。。
随机抓取两个dump文件,查看发现存在不少Oracle ODP.NET提交事务或关闭连接的堆栈,感觉上来说这不是很正常,此类操作很慢吗?很消耗CPU?
查阅当前的Oracle客户端为最新的12.2.0.1(Oracle.DataAccess.dll版本:4.122.1.0),而产品性能测试环境的上一步版本的Oracle客户端是11.2.0.4(Oracle.DataAccess.dll版本:4.112.4),难道新版本的Oracle客户端出现了性能下降?马上卸载并重新安装11.2.0.4后,响应时间果然下降了。。。哦哦
OS Thread Id: 0xd10 (62) Child SP IP Call Site 000000000f0ac5f8 00000000779abd7a [HelperMethodFrame: 000000000f0ac5f8] System.Threading.Thread.SleepInternal(Int32) 000000000f0ac720 000007ff010d9387 Oracle.DataAccess.Client.OracleResourceHolder.TransactionCompleted(System.Object, System.Transactions.TransactionEventArgs) 000000000f0ac870 000007fedd8e26d0 System.Transactions.TransactionStateCommitted.EnterState(System.Transactions.InternalTransaction)*** WARNING: Unable to verify checksum for System.Transactions.ni.dll 000000000f0ac8d0 000007fedd8dc6c4 System.Transactions.SinglePhaseEnlistment.Committed() 000000000f0ac9a0 000007ff010d8b8a Oracle.DataAccess.Client.PromotableTxnMgr.SinglePhaseCommit(System.Transactions.SinglePhaseEnlistment) 000000000f0aca10 000007fedd8e8d2f System.Transactions.DurableEnlistmentCommitting.EnterState(System.Transactions.InternalEnlistment) 000000000f0acae0 000007fedd8df2c9 System.Transactions.CommittableTransaction.Commit()
000000000f0acb90 000007fedd8ecbee System.Transactions.TransactionScope.InternalDispose() 000000000f0acc80 000007fedd8ec90c System.Transactions.TransactionScope.Dispose() OS Thread Id: 0x1130 (69) Child SP IP Call Site 00000000068abd28 00000000779abd7a [NDirectMethodFrameStandalone: 00000000068abd28] Oracle.DataAccess.Client.OpsCon.CheckConStatus(IntPtr, Oracle.DataAccess.Client.OpoConValCtx*, IntPtr, Int32, Int32 ByRef, Int32, Int32) 00000000068abcd0 000007ff005612c4 DomainBoundILStubClass.IL_STUB_PInvoke(IntPtr, Oracle.DataAccess.Client.OpoConValCtx*, IntPtr, Int32, Int32 ByRef, Int32, Int32) 00000000068abdd0 000007ff005609dc Oracle.DataAccess.Client.ConnectionPool.CheckLifeTimeAndStatus(Oracle.DataAccess.Client.OpoConCtx ByRef, Boolean, Boolean ByRef, Int32, Boolean, Boolean, Boolean) 00000000068abf40 000007ff0055c65b Oracle.DataAccess.Client.ConnectionPool.PutConnection(Oracle.DataAccess.Client.OpoConCtx ByRef, Boolean, Boolean, Boolean, Int32, Boolean, Boolean) 00000000068ac280 000007ff0055a270 Oracle.DataAccess.Client.ConnectionDispenser.Close(Oracle.DataAccess.Client.OpoConCtx ByRef, Boolean) 00000000068ac330 000007ff00559712 Oracle.DataAccess.Client.OracleConnectionOCP.Close(Oracle.DataAccess.Client.OracleConnection) 00000000068ac3c0 000007ff00558fe8 Oracle.DataAccess.Client.OracleConnection.Close()
00000000068ac400 000007ff005558c1 Oracle.DataAccess.Client.OracleDataReader.Dispose(Boolean) 00000000068ac510 000007ff0057620e Oracle.DataAccess.Client.OracleDataReader.Close()
0:000> lmvm Oracle_DataAccess
Browse full module list
start end module name
00000000`72e90000 00000000`7306c000 Oracle_DataAccess (no symbols)
Loaded symbol image file: Oracle.DataAccess.dll
Image path: C:\app\client\Administrator\product\12.2.0\client_1\odp.net\bin\4\Oracle.DataAccess.dll
Image name: Oracle.DataAccess.dll
Browse all global symbols functions data
Has CLR image header, track-debug-data flag not set
Timestamp: Sat Dec 17 05:28:01 2016 (58545C61)
CheckSum: 001DB314
ImageSize: 001DC000
File version: 4.122.1.0
Product version: 4.122.1.41664
File flags: 0 (Mask 3F)
File OS: 4 Unknown Win32
File type: 2.0 Dll
File date: 00000000.00000000
Translations: 0000.04b0
CompanyName: Oracle Corporation
ProductName: Oracle Data Provider for .NET, Unmanaged Driver
InternalName: Oracle.DataAccess.dll
OriginalFilename: Oracle.DataAccess.dll
ProductVersion: 4.122.1.20161216
FileVersion: 4.122.1.0
FileDescription: Oracle.DataAccess.dll
LegalCopyright: Copyright (C) Oracle Corporation 1998-2016. All Rights Reserved.
Comments: Oracle.DataAccess.dll
同时我们开启的性能技术器显示每秒的Exception数比较恐怖,900左右,这太不正常了!!
attach到w3wp进程后,输出内部异常后发现,一个基础的handler和aspx页面,有如下堆栈,查阅资料后发现:如果使用 Response.End、Response.Redirect 或 Server.Transfer 方法,则出现 ThreadAbortException 异常。
原因是Response.End 方法会停止页的执行,并将该执行变换到应用程序的事件管线中的 Application_EndRequest 事件, Response.End 后面的代码行将不执行。
对于Response.Redirect 和 Server.Transfer 方法来说,在内部也是调用了 Response.End()
对应的正确写法如下:
1、对于 Response.End,调用 ApplicationInstance.CompleteRequest 方法而不调用 Response.End,以便跳过 Application_EndRequest 事件的代码执行。
2、对于 Response.Redirect,使用重载方法 Response.Redirect(String url, bool endResponse),对 endResponse 参数它传递 false以取消对 Response.End 的内部调用。
例如: Response.Redirect ("nextpage.aspx", false); 如果使用这种解决方法,Response.Redirect 后面的代码将得到执行。
3、对于 Server.Transfer,请改用 Server.Execute 方法。
按照上述方法调整代码后,内部异常消失,平均响应时间进一步下降,结束。
0:111> sxe -c "!pe;!clrstack;g" clr
0:111> g
(ed4.3c4): CLR exception - code e0434352 (first chance)
Exception object: 000000048148b648
Exception type: System.Threading.ThreadAbortException
Message: Thread was being aborted. InnerException:StackTrace (generated): StackTraceString: HResult: 80131530 OS Thread Id: 0x3c4 (44) Child SP IP Call Site 00000000147eda28 000007fefd05a06d [HelperMethodFrame_1OBJ: 00000000147eda28] System.Threading.Thread.AbortInternal() 00000000147edb20 000007fef8764917 System.Threading.Thread.Abort(System.Object) 00000000147edb60 000007fef24ac5cc System.Web.HttpResponse.AbortCurrentThread() 00000000147edba0 000007fe9a8a8fe4 xxxxxxxxxx.xxxHandler.ProcessRequest(System.Web.HttpContext) 00000000147edc40 000007fef24c430c System.Web.HttpApplication+CallHandlerExecutionStep.System.Web.HttpApplication.IExecutionStep.Execute() 00000000147edcc0 000007fef248694c System.Web.HttpApplication.ExecuteStep(IExecutionStep, Boolean ByRef) 00000000147edd10 000007fef2e91d5a System.Web.HttpApplication+ApplicationStepManager.ResumeSteps(System.Exception) 00000000147edda0 000007fef2b8d011 System.Web.HttpApplication.System.Web.IHttpAsyncHandler.BeginProcessRequest(System.Web.HttpContext, System.AsyncCallback, System.Object) 00000000147eddf0 000007fef2bbd9bd System.Web.HttpRuntime.ProcessRequestInternal(System.Web.HttpWorkerRequest) 00000000147ede50 000007fef2bbf3ea System.Web.HttpRuntime.ProcessRequestNoDemand(System.Web.HttpWorkerRequest) 00000000147ede90 000007fef2e36f05 System.Web.Hosting.ISAPIRuntime.ProcessRequest(IntPtr, Int32) 00000000147edf70 000007fef2b7cbe3 DomainNeutralILStubClass.IL_STUB_COMtoCLR(Int64, Int32, IntPtr) 00000000147ee258 000007fef9762e39 [ContextTransitionFrame: 00000000147ee258] 00000000147ee5a0 000007fef9762e39 [ComMethodFrame: 00000000147ee5a0] (ed4.1244): CLR exception - code e0434352 (first chance) Exception object: 00000002015cc050 Exception type: System.Threading.ThreadAbortException
Message: Thread was being aborted. InnerException:StackTrace (generated): StackTraceString: HResult: 80131530 OS Thread Id: 0x1244 (47) Child SP IP Call Site 000000001938dda8 000007fefd05a06d [HelperMethodFrame_1OBJ: 000000001938dda8] System.Threading.Thread.AbortInternal() 000000001938dea0 000007fef8764917 System.Threading.Thread.Abort(System.Object) 000000001938dee0 000007fef24ac5cc System.Web.HttpResponse.AbortCurrentThread() 000000001938df20 000007fef24ac4a9 System.Web.HttpResponse.Redirect(System.String, Boolean, Boolean) 000000001938e020 000007fe9be585fa xxxxxxxxxx.xxxxPage.Page_Load(System.Object, System.EventArgs) 000000001938e090 000007fef24b143d System.Web.UI.Control.OnLoad(System.EventArgs) 000000001938e0d0 000007fef24b1484 System.Web.UI.Control.LoadRecursive() 000000001938e120 000007fef24bfeec System.Web.UI.Page.ProcessRequestMain(Boolean, Boolean) 000000001938e570 000007fef24bf72d System.Web.UI.Page.ProcessRequest(Boolean, Boolean) 000000001938e5c0 000007fef24bf64c System.Web.UI.Page.ProcessRequest() 000000001938e610 000007fef24bf5c6 System.Web.UI.Page.ProcessRequest(System.Web.HttpContext) 000000001938e650 000007fef24c430c System.Web.HttpApplication+CallHandlerExecutionStep.System.Web.HttpApplication.IExecutionStep.Execute() 000000001938e6d0 000007fef248694c System.Web.HttpApplication.ExecuteStep(IExecutionStep, Boolean ByRef) 000000001938e720 000007fef2e91d5a System.Web.HttpApplication+ApplicationStepManager.ResumeSteps(System.Exception) 000000001938e7b0 000007fef2b8d011 System.Web.HttpApplication.System.Web.IHttpAsyncHandler.BeginProcessRequest(System.Web.HttpContext, System.AsyncCallback, System.Object) 000000001938e800 000007fef2bbd9bd System.Web.HttpRuntime.ProcessRequestInternal(System.Web.HttpWorkerRequest) 000000001938e860 000007fef2bbf3ea System.Web.HttpRuntime.ProcessRequestNoDemand(System.Web.HttpWorkerRequest) 000000001938e8a0 000007fef2e36f05 System.Web.Hosting.ISAPIRuntime.ProcessRequest(IntPtr, Int32) 000000001938e980 000007fef2b7cbe3 DomainNeutralILStubClass.IL_STUB_COMtoCLR(Int64, Int32, IntPtr) 000000001938ec68 000007fef9762e39 [ContextTransitionFrame: 000000001938ec68] 000000001938efb0 000007fef9762e39 [ComMethodFrame: 000000001938efb0]