Netduino home hardware projects downloads community

Jump to content


The Netduino forums have been replaced by new forums at community.wildernesslabs.co. This site has been preserved for archival purposes only and the ability to make new accounts or posts has been turned off.
Photo

Garbage Collector, SD Card, Stream Writer, Ram

Garbage Collector SD Card Stream Writer Ram

  • Please log in to reply
4 replies to this topic

#1 stotech

stotech

    Advanced Member

  • Members
  • PipPipPip
  • 143 posts
  • LocationAustralia

Posted 06 April 2015 - 12:03 AM

Please Help!
I'm running out of ram all the time when i try and write to the SD card, But it says I've got heaps spare. Here is my code.
 
public static void Writer(string msg, string path, string file)
        {
            Debug.GC(true);
            Debug.Print("mem:  " + Debug.GC(true));

            lock (CardLock)
            {
                //SDC_flushSlow();
                int bufferSize = msg.Length;

                Debug.Print("Attempting to write with buffer of " + bufferSize.ToString());

                try
                {
                    if (Detect.SD_Card() & File.Exists(path + file))
                    {
                        using (var writestream = new FileStream(path + file, FileMode.Create, FileAccess.ReadWrite, FileShare.ReadWrite, bufferSize))
                        {
                            StreamWriter SW = new StreamWriter(writestream);
                            SW.Write(msg);
                            SW.Flush();
                            SW.Close();
                        }
                    }
                    else if (Detect.SD_Card())
                    {
                        if (!Directory.Exists(path))
                        {
                            Directory.CreateDirectory(path);
                        }
                        using (var writestream = new FileStream(path + file, FileMode.Create, FileAccess.ReadWrite, FileShare.ReadWrite, bufferSize))
                        {
                            StreamWriter SW = new StreamWriter(writestream);
                            SW.Write(msg);
                            SW.Flush();
                            SW.Close();
                        }
                    }
                }
                catch (Exception e)
                {
                    Debug.Print("SD_Writer Failed " + e.Message);
                }

                DebugLED.Show(15, 500);
            }
        }
But I get this every time.
 

SDC_Flushed and remounted
mem: 24732
Attempting to write with buffer of 1074
Failed allocation for 343 blocks, 4116 bytes
Failed allocation for 343 blocks, 4116 bytes
A first chance exception of type 'System.OutOfMemoryException' occurred in System.IO.dll
SD_Writer FailedException was thrown: System.OutOfMemoryException
 
Ok, Ive got an idea that might be causing it. Do you think that it could be that I have about three threads running at this stage?
 
if I call this:
 
Debug.GC(true)

Debug.Print("mem: " + Debug.GC(true));

 

Then it says I have 24k spare.
But the it cant find a 4k in a row to open the stream writer.
Could it be that the 2 other threads are preventing the garbage collector from moving memory objects. Does the garbage collector run across all threads or only the one it is called in?
 
Thanks So much in advance.


#2 Chris Walker

Chris Walker

    Secret Labs Staff

  • Moderators
  • 7767 posts
  • LocationNew York, NY

Posted 06 April 2015 - 07:56 AM

Hi Grant,

The one thing the garbage collector can't do is move around pinned memory. This applies to both managed memory which is pinned and also unmanaged resources (which are, by nature, pinned).

When NETMF boots, it pre-allocated most of the native memory it will need. Most of this is in one large chunk at the beginning of RAM--but some bits could technically be allocated between managed code objects (which might be moved/erased later).

The other thing to remember is that classes can have quite a few temporary memory variables that get created and destroyed in rapid succession. This can particularly be an issue when working with strings, files, byte arrays, etc.--since extra buffers to pass responses and handle data are often created.

In your particular scenario: what happens if you try to create a bunch of small 200-byte strings in memory? Is it failing to allocate the memory completely--or just a contiguous chunk?

Finally: which board are you using?

Chris

#3 stotech

stotech

    Advanced Member

  • Members
  • PipPipPip
  • 143 posts
  • LocationAustralia

Posted 06 April 2015 - 09:44 AM

Thanks for that detailed response. The program continues to run fine even though the writer fails. So I'm assuming it's a problem finding sequential RAM as opposed to any RAM. Does everything i declare in managed code get "pinned"? Is there a way to pre-allocate memory for a managed function. Or do i just need to go over the code i've got.



#4 Chris Walker

Chris Walker

    Secret Labs Staff

  • Moderators
  • 7767 posts
  • LocationNew York, NY

Posted 06 April 2015 - 11:12 PM

Hi Grant,

All managed code, unless pinned, should be able to be moved around by the garbage collector. That said...there needs to be an EMPTY contiguous piece of RAM to move the entire managed memory chunk to...so with large objects you can still end up with the inability to "garbage collect" without a full "push everything to the front of RAM" sweep. And even then...any pinned objects are in the way.

If a managed code library has behind-the-scenes native code, and that native library does an explicit MALLOC (i.e. dynamically allocates and pins native memory resources in an arbitrary place on the heap), then you can end up with native memory (which is always pinned) in an undesirable spot.

This is actually one of the things we generally avoid on the Netduino platform by pre-allocating frequently-used buffers. I've seen board makers quote arbitrarily high free RAM figures, presumably by malloc'ing memory which creates memory pinning issues down the road.

To pre-allocate memory for a managed function, just allocate it at the class level. We do this a lot in our driver code. Then use that pre-allocated memory in your function. No dynamic memory allocation or GC required.

Chris

P.S. Then there is the topic of the managed stack--but that's a conversation for another day :)

#5 stotech

stotech

    Advanced Member

  • Members
  • PipPipPip
  • 143 posts
  • LocationAustralia

Posted 07 April 2015 - 12:30 AM

Thanks A heap. I'll mull over this with my mate google for a few days and see if i can make use of it. 







0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users

home    hardware    projects    downloads    community    where to buy    contact Copyright © 2016 Wilderness Labs Inc.  |  Legal   |   CC BY-SA
This webpage is licensed under a Creative Commons Attribution-ShareAlike License.