none
Powershell ‘system.outofmemoryexception’ error RRS feed

  • Question

  • Hi, does anybody have any tips on how to diagnose the ‘system.outofmemoryexception’ error?
     I’ve written various memory tests in my script and output the results to a log file e.g. free physical memory/total available memory but no limits are exceeded. I'm running powershell V2 on a 64 bit environment, the line that is failing is:-

    [xml]$FullDetails = "<Root><device>" + $Details1 + $Details2 + "</device></Root>"

    So far I've tested free physical memory and I've also checked the length of the $Details variables. When the process fails the length of these is no greater than previous successful executions of this line of code.

    I was in the process of looking into the powershell internal variables, in particular I was hoping that maybe 'maxmemorypershell' was being exceeded but another thread suggests that this value only relates to processes running via remote connections, so what else can I test?

    Thanks in anticipation\hope\desperation....

    Thursday, November 10, 2016 11:46 AM

Answers

  • Of course, I'm paraphrasing your question.

    The point is that without a script and a particular reproducible behavior, it's simply not possible to answer the question based on only the information you posted.


    -- Bill Stewart [Bill_Stewart]



    Wednesday, November 16, 2016 3:58 PM
    Moderator

All replies

  • Hi Will,

    first of all, is your PowerShell also 64? You can test that using this:

    # True if 64 bit, false otherwise
    ([IntPtr]::Size -eq 8)

    If it's a 64bit PowerShell and you are not hitting your max memory on the system, I'm a bit stymied ... you may have a memory leak somewhere, if the system is running for quite a bit time that would take a significant portion of your memory.

    Either way, repeatedly building large strings will fill your memory. The Garbage Collector is not magic, just because you have no handle to a specific object, it isn't cleaned up immediately.

    You may also want to try running it on another machine - maybe the local machine has a defect. Verify your .NET health.

    Other than those, I don't know what it might be.

    Cheers,
    Fred


    There's no place like 127.0.0.1

    Thursday, November 10, 2016 12:38 PM
  • Hi Fred, thanks for your help. The expression returns true. I'll give some consideration to the other points you mention too.

    Thanks again,

    WILL

    Thursday, November 10, 2016 2:08 PM
  • The process runs for close on 8hrs before it fails so plenty of time there for a memory leak I would imagine? Not sure how I would further diagnose that leak though.

    Do you know if powershell will attempt to use as much of the 'free physical memory' as it needs, or is there some internal memory setting similar to 'maxmemorypershell' that may be influencing things? The reason I ask this is that when the process fails the 'free physical memory' measure (taken from the wmiobject) shows 4gb of 8gb is still free.

    I'll look into a .net health check (its a VM by the way), and maybe look for ways to initialize the garbage collector.

    Regards,

    WILL

    Thursday, November 10, 2016 2:22 PM
  • It is more likely that you have something else in the system that is using all of your memory.

    What size is your page file and how much is in use.  You wuill have to trace this using perfmon.


    \_(ツ)_/

    Thursday, November 10, 2016 3:16 PM
  • Thanks JRV, I've just looked at the log from the latest attempt and the process failed on a different line of code.

    previously it failed on this line:

    [xml]$FullDetails = "<Root><device>" + $Details1 + $Details2 + "</device></Root>"

    yesterday it failed on:

    [xml]$OtherInfo = "$Variable ...etc...

    So the common theme is the move to xml

    Friday, November 11, 2016 11:25 AM
  • XML has nothing to do with this.  YOU have other issues and they are not from a script.  You must use perfmon to track system memory over time to detect what is causing you memory issue.

    If you are running non PS code inside of your script this can cause issues.

    There is not enough information to even take a rough guess.  This is one task a trained tech must master.  You must learn how to troubleshoot system problems.

    If you are creating a very large object collection in an endless loop then you can run out of memory.  This would be a design issue but there is no way we can tell as we cannot see your system or your code.


    \_(ツ)_/

    Friday, November 11, 2016 11:37 AM
  • Keep in mind that nobody has access to your computer, and we can't see your screen.


    -- Bill Stewart [Bill_Stewart]

    Friday, November 11, 2016 3:14 PM
    Moderator
  • Yes thanks Bill, I would like to point out though that my questions were clear and basic and did not really need access to code\screen. My first question was:

    Other than 'FreePhysicalMemory' and '$variables', what else can I test? and is 'MaxMemoryperShell' relevant?

    I then asked:

    Do you know if powershell will attempt to use as much of the 'free physical memory' as it needs, or is there some internal memory setting similar to 'maxmemorypershell' that may be influencing things?

    it's tricky structuring a question for these forums because the question needs to be concise but I think it is also a good idea to provide enough info to avoid the "have you tried the obvious..." responses.

    Anyway, not to worry, I'll carry on, on my own.

    \_(ツ)_/

    :-)

    Monday, November 14, 2016 11:47 AM
  • On current versions of PowerShell this is the standard setting:

    Get-Item WSMan:\localhost\Shell\MaxMemoryPerShellMB

    It is set to a default of 2147483647Mb.

    Accumulating very large objects in memory can eventually eat this up but it is unlikely. If this is the issue then you have a design problem.

    Perhaps you are trying to execute an external program that is causing this error.

    Also accumulating a string can exceed the memory limit for a string object.  I believe that maximum is set by the architecture and is much lower.


    \_(ツ)_/

    Monday, November 14, 2016 12:30 PM
  • Thanks for the additional jrv
    Monday, November 14, 2016 1:16 PM
  • If your page file is too small you will also get OOM errors.

    \_(ツ)_/

    Monday, November 14, 2016 1:42 PM
  • I'm just running the process now with a perfmon watch, so hopefully will be able to see any page file issues.

    Thanks again..

     
    Monday, November 14, 2016 2:17 PM
  • Yes thanks Bill, I would like to point out though that my questions were clear and basic and did not really need access to code\screen. My first question was:

    Other than 'FreePhysicalMemory' and '$variables', what else can I test? and is 'MaxMemoryperShell' relevant?

    I then asked:

    Do you know if powershell will attempt to use as much of the 'free physical memory' as it needs, or is there some internal memory setting similar to 'maxmemorypershell' that may be influencing things?

    it's tricky structuring a question for these forums because the question needs to be concise but I think it is also a good idea to provide enough info to avoid the "have you tried the obvious..." responses.

    So essentially, you are asking: "I am running some script on my computer and something is using up all my free memory. Can you somehow magically tell me what's happening?"

    The answer to that is no, because we don't know what your script is doing, or even if it really is your script that is exhausting all of your physical memory.

    What you need to do is write a short script that contains only the absolute minimum amount of code needed to reproduce the problem. You also need to run said script on another machine (such as an isolated VM, for example) to reproduce the problem on another system.

    If you are going to ask a question in a forum, you need to make it easy as possible for others to reproduce the problem. As I stated earlier: We don't have access to the problem system, nor can we observe any actual script behavior.

    Your question is not answerable as stated; sorry.


    -- Bill Stewart [Bill_Stewart]

    Monday, November 14, 2016 3:26 PM
    Moderator
  • Bill, I did not ask the question - "I am running some script on my computer and something is using up all my free memory. Can you somehow magically tell me what's happening?"

    If you read the original question it was basically - "here are the points I've tested, can you think of any other points I can test?" Simple, if there are no other points to test then just say "no"!

    I then asked "if powershell will attempt to use as much of the 'free physical memory' as it needs, or is there some internal memory setting similar to 'maxmemorypershell' that may be influencing things?"

    As you can see, the second question was really about the inner workings of the powershell engine and how it uses memory. Sorry about that, I thought this was a powershell forum!

    So, two pretty clear straightforward questions, neither of which were answered. Some people did offer useful info and tips along the way so thanks to them.

    Wednesday, November 16, 2016 11:43 AM
  • Of course, I'm paraphrasing your question.

    The point is that without a script and a particular reproducible behavior, it's simply not possible to answer the question based on only the information you posted.


    -- Bill Stewart [Bill_Stewart]



    Wednesday, November 16, 2016 3:58 PM
    Moderator