Memory Leaks - Part 2: Debugging Tools to Diagnose a Memory Leak

If you are reading this you may be in the unfortunate position of looking for an elusive and possibly intermittent problem in your production application. Debugging and diagnosing a memory leak or an OutOfMemoryException can be a daunting, intimidating and challenging task. Fortunately, there are a number of tools to help; some of these are "paid license" apps, but there are even more tools that are free. I'll discuss some .NET Debugging Tools available. I've used all of the following tools with the exception of SciTech .NET Memory Profiler. Each has its advantages and disadvantages. Personally, I prefer the free tools for several reasons:

1. Getting approval for licensing and purchase of a 3rd-party tool is an uphill task and you rarely have time to wait for the approval process when it is a production issue.
2. I find the feature set of a combination of the free tools gives you the largest "surface area" of features to help. Some are better at strict data collection, others have graphical views, some are static for post-mortem debugging, and others can do real-time analysis.
3. Even though these are free, they are very robust and provide just as much data as the paid license tools. A good example of this is WinDbg which was developed in 1993 by Microsoft for in-house debugging of the Windows Kernel.

Free Tools

Advantages

• Duh! It's free
• There are lots of tools to choose from
• All of the ones I've seen are from reputable and well-known companies.
• You can often find blog posts (ahem...), articles, reddit threads, etc. that can provide some direction for getting started.

Disadvantages

• Formal documentation can be lacking. Finding a blog post or article is great but comprehensive detail is often missing or at best glossed over. This can make getting started a bit more of a challenge if the tool is new to you.
• Your company may have restrictions on using free tools for fear of malware, liability, or other reasons.

• Windows Task Manager: taskmgr.exe. Make sure to make the following columns visible:
• Memory - Working Set
• Memory - Private Working Set
• Memory - Commit Size
• Threads
• Handles
• USER Objects (if WinForm or Native app)
• GDI Objects (if WinForm or Native app)
• Windows Performance Monitor: perfmon.exe
• Process Explorer (SysInternals)
• VM Map (SysInternals)
• Debugging Tools for Windows (WinDbg, SOS, Kernel Debugger, etc.)
• Microsoft Debug Diagnostic Tool (DebugDiag)
• Psscor2 - WinDbg debugger extension for ASP.NET 2.0, 3.5
• Psscor4 - WinDbg debugger extension for ASP.NET 4.x
• SOSEX from Steve Johnson (.NET 2.0, 3.5)
• SOSEX 4 from Steve Johnson (.NET 4.x)
• IISInfo - WinDbg debugger extension (included with DebugDiag)
• Microsoft Application Verifier
• CLR Profiler 2.0 (good for finding heap fragmentation)
• CLR Profiler 4.0 (good for finding heap fragmentation)
• Windows Performance Toolkit (xperf)

Licensed Tools

• RedGate Ants Memory Profiler
• Micro Focus DevPartner Studio (formerly Compuware)
• SciTech .NET Memory Profiler
• AQ Time Pro

Hope this helps!

Finally a good free multi-monitor tool

timstall: Tool - WinSplit Revolution - move windows between ...: "A coworker showed me a useful (and free!) tool that conveniently positions open windows on your screen. The tool is Winsplit Revolution , ..."

New WinDbg Extension available

I'm happy to announce that a new version of the WinDbg debugging extension PSSCOR2 (now called PSSCOR4) has been released.  It is available for public download here: 

http://www.microsoft.com/downloads/en/details.aspx?FamilyID=a06a0fea-a4d4-434e-a527-d6afa2e552dd

Whitepaper on the Debug Diagnostic Tool v1.1 - IIS debugging and debugdiag - Site Home - MSDN Blogs

Whitepaper on the Debug Diagnostic Tool v1.1 - IIS debugging and debugdiag - Site Home - MSDN Blogs

IIS Tools that you should have...

Here is a list of IIS tools I've used in the past and their respective links.  Please note that all of these tools are FREE and widely used in the industry.

 

IIS Web Deploy 2.0

http://www.iis.net/download/WebDeploy

 

 

IIS 6.0 Resource Kit (many tools included here – most helpful are probably IIS 6.0 Migration Tool and the IIS Metabase Explorer)

http://www.microsoft.com/downloads/en/details.aspx?FamilyID=56fc92ee-a71a-4c73-b628-ade629c89499&displaylang=en

 

 

IIS Diagnostics Kit – IIS Diag

http://www.microsoft.com/downloads/en/details.aspx?FamilyID=9bfa49bc-376b-4a54-95aa-73c9156706e7

 

This includes a number of tools which help diagnose various types of problems with IIS setup and configuration:

• AuthDiag - authentication and ACL problems
• SSL Diag - SSL-related problems
• SMTP Diag - smtp-related issues
• Log Parser - allows for easy reading of IIS log files
• Debug Diag - allows you to create memory dumps manually or automatically when IIS hangs.  It also can run a memory ananlysis from the memory dump captured!
• and others...

Event Log Explorer - allows you to create custom queries and filters against anything in the Event Viewer – very cool!

http://www.eventlogxp.com/

And a whole list of the other great tools from the IIS Team at Microsoft:

http://www.iis.net/download/All

 

 

Tools for Assisting with Collection and Analyzing of Data for Problem Diagnosis

I was going through my Google Bookmarks and found a bunch of tools that are extremely helpful in diagnosing  many problems that occur on a web/app server and and interpreting the data collected. I’ve used all of them; they are all free and well known and used within the Community.  By no means is this an exhaustive list of all of the Tools and Add-Ins that I use - I'll get around to posting those someday : )

I’d start with the Data Collector. You can collect everything with Data Collector and then use the Log Parser and Visual Log Parser to help aggregate the data and even run queries against it. You may already know about some of these – but for those you haven’t seen, you should take a look!

Data Collector (supports IIS6 & IIS7, and can pull most other logs from any server):
http://blogs.msdn.com/b/carloc/archive/2010/02/01/idevdatacollector.aspx

Log Parser 2.2:
http://www.microsoft.com/downloads/en/details.aspx?FamilyID=890cd06b-abf8-4c25-91b2-f8d975cf8c07&displaylang=en

Visual Log Parser:
http://visuallogparser.codeplex.com/

Log File Analysis:
http://blogs.msdn.com/b/delay/archive/2007/06/21/powerful-log-file-analysis-for-everyone-releasing-textanalysistool-net.aspx

Event Log Explorer:
http://www.eventlogxp.com/

IIS Log Analyzer (does graphs and reports):
http://www.iis.net/community/default.aspx?tabid=34&g=6&i=1864

IIS Diagnostics Toolkit (a suite of tools diagnosing Authentication, Authorization, SSL, Crash/Hang, etc.):
http://www.microsoft.com/downloads/en/details.aspx?FamilyID=9bfa49bc-376b-4a54-95aa-73c9156706e7

and then of course the "big daddy" bunch of them all - Debugging Tools for Windows:
http://www.microsoft.com/whdc/devtools/debugging/default.mspx

I hope you find these useful : )