I've been assisting someone with a memory leak issue over the past few days. They described their symptoms as:

The server will suddenly stop accepting any form of request from other servers. If you try to do a snap-in from another server using MMC, it will say "There is not enough storage to complete this operation". You may not even connect to the server using PCAnywhere. The only way to recover the server is to reboot it. Looking at the Event Viewer, it says:

Event Type: Error
Event Source: Srv
Event Category: None
Event ID: 2020
Date: 5/9/2006
Time: 9:09:34 AM
User: N/A
Computer: ProdSvr
Description:
The server was unable to allocate from the system paged pool because the pool was empty.

The user, based on their research, had narrowed down the possibilities to printer related or AV related or the server being under load or... (i.e. a number of possibilities). They had additionally looked at MS KB articles 312362 (tuning Memory Manager) and 177078 (IRPStackSize) to no avail. The user had been attempting to troubleshoot the problem for months.

In all debugging cases, you need to decide whether you want to find out the cause, or you want to fix the symptoms. They are not necessarily one and the same. Sometimes you can make the symptoms go away by rebuilding the server. Does that mean you found out what the problem is? Not really. Can the problem simply re-appear? Absolutely.

On the otherhand, many IT departments have SLAs and other time pressures. It may not be feasible to spend days or weeks debugging issues to determine the root cause of the problem, only to then discover that there may be nothing that can be done because it's a bug in a 3rd party application!

If you do want to find out the cause of memory issues like this, the tool to start off with is Poolmon.exe (Pool Monitor). Microsoft KB article 177415 shows you how to enable use Poolmon, and to use the Gflags.exe utility to enable Pool Tagging (not required on Windows Server 2003). Poolmon itself it located in the \support\tools folder on your Windows2000/XP/2003 CD, or in the NT Resource Kit.

For our user, Poolmon reporting the following: 

Memory: 3931568K Avail: 1193552K  PageFlts:  2497   InRam Krnl: 6036K P:303864K
 Commit:2602024K Limit:12150988K Peak:2626284K            Pool N:21844K P:304068
 Tag  Type     Allocs            Frees            Diff   Bytes      Per Alloc

 SevI Paged   1283625 ( 147)   1089572 (   0)   194053 223442368 (169344)   1151
 CM   Paged    154222 (   0)    149554 (   0)     4668 20050080 (     0)   4295
 NAI0 Paged        16 (   0)        12 (   0)        4 8011776 (     0) 2002944
 Gh 5 Paged     60378 (   0)     60060 (   0)      318 3228896 (     0)  10153
 Ntff Paged     17223 (   9)     16182 (   0)     1041  899424 (  7776)    864
 MmSt Paged     37044 (  34)     35944 (  29)     1100  835872 (  3200)    759
 Usqm Paged     93353 (   0)     83313 (   0)    10040  642560 (     0)     64
 Obtb Paged       547 (   0)       392 (   0)      155  634880 (     0)   4096
 Gla1 Paged      1186 (   0)       891 (   0)      295  462560 (     0)   1568
 NtfF Paged       908 (   0)       653 (   0)      255  252960 (     0)    992
 CMDa Paged    121768 (   9)    120191 (   6)     1577  231104 (   768)    146
 Ttfd Paged       753 (   0)       636 (   0)      117  215232 (     0)   1839

I think it's fairly clear what's consuming paged pool here. Our top consumer (SevI) has made 194053 more memory allocations than frees (deallocations), and is consuming over 200MB of paged pool.

Using the following command, we searched for drivers containing the string SevI: 

findstr /l /m SevI *.sys

which returned symevent.sys - a component of Symantec's AV product. Interestingly enough, it appears that we have some Network Associates product running on this server as well (NAI0).

And after all that, it turns out there's a KB article on Microsoft's website: 272568 which warns of exactly this problem using some older versions of Symantec AV on Windows 2000 Server! But at least the investigation was interesting, and we can be relatively satisfied that the KB article does apply to the situation.