The Netduino forums have been replaced by new forums at community.wildernesslabs.co.
This site has been preserved for archival purposes only
and the ability to make new accounts or posts has been turned off.
OutOfMemoryException, but I still have free memory!
I am getting the exception below. I am playing with that NetduinoPlusWebServer app from this thread. It seems to keep throwing an exception, usually on the substring command. I figured I was probably carelessly leaking memory, so I used the following command to watch the memory run out:
It's a little puzzling. I think the program does a couple of needless copies, but even so I don't see why it's running out of memory. Would you be willing to try something just to satisfy my curiosity?
in Request.cs, substitute this routine for ProcessRequest() and then tell me what the output is.
private void ProcessRequest(char[] data)
{
Debug.Print("data length is "+data.Length);
string content = new string(data);
var indexOf=content.IndexOf('\n');
Debug.Print("indexOf is "+indexOf);
string firstLine = content.Substring(0, indexOf);
// Parse the first line of the request: "GET /path/ HTTP/1.1"
string[] words = firstLine.Split(' ');
method = words[0];
url = words[1];
// Could look for any further headers in other lines of the request if required (e.g. User-Agent, Cookie)
}
Really interesting. Could you try something in your code right before the function which causes the "compacting memory" message?
Debug.GC(true);
We identified an issue in .NET MF a while ago where the garbage collector was not collecting/compacting properly. Basically, if there were huge chunks of memory available...but not contiguously, the automatic garbage collector was not compacting fully (and the memory allocator couldn't allocate a contiguous block of memory because the memory in use wasn't leaving a big enough contiguous block of RAM)... Interestingly, a manual garbage collection request seemed to fix the issue...
...but we need a good repro case to fix it (or as is more likely in the case of GC, assist Microsoft in fixing it in the core).
If Debug.GC(true) fixes it, that gives us both a good short-term fix and hopefully a repro case so the root cause can be remedied.
Well, that's exactly what I did about 3 hours ago and I have not been able to reproduce the Memory exception since. I was serving pages continuously for 2 hours straight to 3 different computers and did not get even one exception. I switched that setting back to false ( Debug.GC(false); ) and within 10 seconds had another Memory exception. It reproduces 50-60% of the time and the stack shows different sources from time to time. The second run set to false did not throw an exception. The third time did again and here is that exception:
GC: 3msec 37464 bytes used, 13512 bytes available
Type 0F (STRING ): 576 bytes
Type 11 (CLASS ): 3576 bytes
Type 12 (VALUETYPE ): 60 bytes
Type 13 (SZARRAY ): 9516 bytes
Type 15 (FREEBLOCK ): 13512 bytes
Type 17 (ASSEMBLY ): 13788 bytes
Type 18 (WEAKCLASS ): 48 bytes
Type 19 (REFLECTION ): 24 bytes
Type 1B (DELEGATE_HEAD ): 252 bytes
Type 1D (OBJECT_TO_EVENT ): 168 bytes
Type 1E (BINARY_BLOB_HEAD ): 1884 bytes
Type 1F (THREAD ): 2304 bytes
Type 20 (SUBTHREAD ): 288 bytes
Type 21 (STACK_FRAME ): 3156 bytes
Type 27 (FINALIZER_HEAD ): 360 bytes
Type 31 (IO_PORT ): 252 bytes
Type 34 (APPDOMAIN_HEAD ): 72 bytes
Type 36 (APPDOMAIN_ASSEMBLY ): 1140 bytes
Could not allocate 89 blocks, 1068 bytes. Compacting memory.
#### Exception System.OutOfMemoryException - CLR_E_OUT_OF_MEMORY (5) ####
#### Message:
#### Microsoft.SPOT.IO.NativeFileStream::.ctor [IP: 0000] ####
#### System.IO.FileStream::.ctor [IP: 00fd] ####
#### System.IO.FileStream::.ctor [IP: 0010] ####
#### NetduinoPlusWebServer.Request::SendFile [IP: 00aa] ####
#### NetduinoPlusWebServer.Program::TrySendFile [IP: 00ba] ####
#### NetduinoPlusWebServer.Program::RequestReceived [IP: 0020] ####
#### NetduinoPlusWebServer.Listener::StartListening [IP: 0114] ####
A first chance exception of type 'System.OutOfMemoryException' occurred in Microsoft.SPOT.IO.dll
A first chance exception of type 'System.OutOfMemoryException' occurred in System.IO.dll
An unhandled exception of type 'System.OutOfMemoryException' occurred in System.IO.dll
You can see that this time it happened in another part of the code. I have this version checked in to my SVN server so I can get it back for testing in the future.
P.S. Corey, I could see what you were getting at with the extra debug statement, but when the exception happened I could see that the IndexOf('\n') was NOT some huge number, creating some huge string. Also, the input was always less than 600 characters... I'll revisit this IF I get any more Memory Exceptions with the Debug.GC(true).
Thanks, Valkyrie-MT. I'll see if I can create a good repro for Microsoft and see if we can take care of this in a an out-of-band patch after the holidays...
Chris
I was reading this very nervously expecting that you'd find a horrendous memory leak in my code. I intentionally kept it simple rather than optimized, but glad I didn't do anything too bad!
Hey guys,
I had the same issue but the Debug.GC(true) did not work (it helped but didnt fix) for me.
My application is a webserver that sends output to a sound board via netduino plus (firmware v4.1.0 update 6). that will turn on/off channels.
Anyways, i run into problems when i compile my html string to send as a response. I then load that string into an array and send it. I put Debug.GC(true) all over the place, but after a few requests (about 19) it bombs. Granted for each request I log an event, so i understand that these events will take up more and more memory as they get added, but like others mentioned above, the system says i have plenty of memory left, before i get the error.
Attached is the code for my application. It is based off of the multi-threaded webserver found here:
http://forums.netdui...h__1#entry11770
but my HTML responses are pretty big (about 4-6KB)
Here is where I run into problems:
// Send HTML Response to web user.
Debug.Print("About to load string into bytes. No significant memory drop should have occured.");
Debug.GC(true);
string htmlResponse = getHomepageHTML(activeChannel, turnOn, e.Command.ClientIP);
Debug.Print("String loaded.");
Debug.GC(true);
e.ReturnString = Encoding.UTF8.GetBytes(htmlResponse);
Debug.Print("Loaded Byte array.");
Debug.GC(true);
Can anyone help? What am i doing wrong?
In .NET MF 4.1, the garbage collector sometimes doesn't sweep memory completely. This has been fixed in .NET MF 4.2 RC1. If you haven't tried it yet, I'd recommend updating your board to the newest .NET MF 4.2 beta and see if this problem is fixed for you
Also, some objects are pinned in memory. This is necessary for a few things like native code interop, hardware interrupt code, etc. A few items are pinned early in the memory space (which is why you might be able to allocate small--but not large objects when memory starts getting tight...not enough consecutive memory).
All that said, I'm guessing that .NET MF 4.2 will clear the issue up for you right away.
Hi Chris,
Just got 4.2 RC1 installed everywhere.
I'm not at present getting the OUT_OF_MEMORY exception (But not at a point to thrash out things just yet, given the new exception that's crept in with 4.2 RC1:
My Serial Library is attached.
History:
under 4.1.0.6 I never had this type of Exception inside serialPort_DataReceived() at all, and its nearly occurring every time something is received (even 3-5 chars)
Under 4.1.0.6 I used to get an exception when inside Read() as not enough memory was allocated for Chars[] to copy from mBuffer[].
Code:
attached
Implementation:
COM2 on NetduinoPlus
Note: My Original Exception entry point seems to persist, however the exception it raises is different:
4.1.0.6 Exception: OUT OF MEMORY
4.2. RC1: A first chance exception of type 'System.ArgumentOutOfRangeException' occurred in mscorlib.dll
Exception seems to occur on Line:
sChars = new string(chars, 0, mBufferPtr);
