Netduino home hardware projects downloads community

Jump to content


The Netduino forums have been replaced by new forums at community.wildernesslabs.co. This site has been preserved for archival purposes only and the ability to make new accounts or posts has been turned off.
Photo

Running out of memory


  • Please log in to reply
14 replies to this topic

#1 Patrick

Patrick

    Advanced Member

  • Members
  • PipPipPip
  • 54 posts
  • LocationTampa

Posted 14 October 2012 - 08:49 PM

This code seems to cause my N+ to run out of memory. There are no errors but I figure that it's running out of memory because it will insert 536 records every time and if I add a couple references to the project and increase the size of the payload, it will insert a few less records. Scientific huh? :P

Any ideas? Is it possible to do a POST using sockets instead?

int numTests = 1000;

            while (numTests > 0)
            {
                //write test data and show LED
                led.Write(true);
                Thread.Sleep(250);

                try
                {
                    string strNewValue;
                    //string strResponse;
                    req = (HttpWebRequest)WebRequest.Create("http://SERVERNAME.com/PATH/TestInsert.php?");
                    req.Method = "POST";
                    req.ContentType = "application/x-www-form-urlencoded";
                    req.KeepAlive = false;

                    // Set values for the request back                
                    strNewValue = "TestID=''&TestDesc=AWholeBunchOfTestDataDoesItMatterHowMuchWePass";
                    req.ContentLength = strNewValue.Length;

                    // Write the request
                    stOut = new StreamWriter(req.GetRequestStream());
                    stOut.Write(strNewValue);
                    stOut.Dispose();
                    stOut.Close();

                    req.Dispose();
                }
                catch (Exception ex)
                {
                    Debug.Print(ex.Message.ToString());
                }

                //hide LED
                led.Write(false);
                Thread.Sleep(250);

                numTests--;
            }

            Debug.Print("Test complete");


#2 carb

carb

    Advanced Member

  • Members
  • PipPipPip
  • 352 posts
  • LocationCrystal River, Florida

Posted 14 October 2012 - 09:17 PM

Patrick,

I didn't see anything obviously wrong in the code, it could be a garbage collection issue.

I would add a debug.print to show available memory and cycle through a few times to see if you are indeed using up the memory.

If so move the debug.print to find out where. See this post Determining Available memory
Chuck

#3 Patrick

Patrick

    Advanced Member

  • Members
  • PipPipPip
  • 54 posts
  • LocationTampa

Posted 14 October 2012 - 09:59 PM

According to the Debug.GC method, it doesn't seem like the device is running out of memory. It shows the following then stays at 40812 right up until iteration 556, then it just stops.

Debug.Print("Memory Used: " + Debug.GC(true).ToString());
Memory Used: 35160
Memory Used: 36900
Memory Used: 36900
Memory Used: 38364
Memory Used: 40812
...

Same thing if I pass the Debug.GC method false as well but it goes to iteration 563.

Once the program halts, If I hit pause in VS, the compiler highlights this line:
stOut = new StreamWriter(req.GetRequestStream());


#4 carb

carb

    Advanced Member

  • Members
  • PipPipPip
  • 352 posts
  • LocationCrystal River, Florida

Posted 14 October 2012 - 10:09 PM

Patrick,

Your beyond my experience level. It is slow on the forum right now, but someone should be on later that can help.

One question that I had was wouldn't you normal close an item before disposing of it?
stOut.Dispose(); 
stOut.Close();
Chuck

#5 Patrick

Patrick

    Advanced Member

  • Members
  • PipPipPip
  • 54 posts
  • LocationTampa

Posted 14 October 2012 - 10:43 PM

Thanks for the help Chuck; I appreciate it! According to my research, Close() calls Dispose(true) internally and I did it in that order just in case what I found was inaccurate... Well that and I tried it both ways. :D You're in Crystal River? Cool! I'm in Tampa.

#6 Chris Walker

Chris Walker

    Secret Labs Staff

  • Moderators
  • 7767 posts
  • LocationNew York, NY

Posted 14 October 2012 - 11:36 PM

Hi Patrick, Are you running .NET MF 4.1 or 4.2? Also yes, if you want you can do POST with sockets; you'll need to build your own HTTP headers though. Check out the SocketClient sample from the .NET MF SDK. Chris

#7 Patrick

Patrick

    Advanced Member

  • Members
  • PipPipPip
  • 54 posts
  • LocationTampa

Posted 14 October 2012 - 11:41 PM

I'm running 4.2. Getting some help in chat atm and will check out the SocketClient example regardless; thanks!

#8 carb

carb

    Advanced Member

  • Members
  • PipPipPip
  • 352 posts
  • LocationCrystal River, Florida

Posted 14 October 2012 - 11:45 PM

Patrick, If Close calls Dispose then you should be able to comment the Dispose out and not see any change in the number of iterations. I saw that you were in Tampa, my house is in Crystal River about 80 miles to the North. I would like to see Tampa or Orlando get a Makers Faire to help increase the interest in Micro Processors like the Netduino. The new Go board has a lot of potential and is growing fast. You may want to restate your question and what you know at this point. Take care, Chuck

#9 Nicky

Nicky

    Advanced Member

  • Members
  • PipPipPip
  • 78 posts
  • LocationDenmark

Posted 15 October 2012 - 06:32 AM

To make a HTTP request with a Socket, see http://forums.netdui...es-a-long-time/ Replace GET with POST. Refer to HTTP 1.1 documentation for posting datas.

ntools
TCP Listener (Beta) · FTP Server (Alpha)
Netduino Plus Go Module · Xml Parser
http://ntools.codeplex.com/


#10 Patrick

Patrick

    Advanced Member

  • Members
  • PipPipPip
  • 54 posts
  • LocationTampa

Posted 15 October 2012 - 10:28 PM

Same exact failure with using sockets and a POST. 563 inserts, then locked.

private static void TestSockettPost()
        {
            int numTests = 1000;

            while (numTests > 0)
            {
                //write test data and show LED
                led.Write(true);
                Thread.Sleep(250);

                try
                {
                    IPHostEntry host = Dns.GetHostEntry("www.SERVER.com");
                    IPEndPoint endPoint = new IPEndPoint(host.AddressList[0], 80);

                    using (Socket socket = new Socket(AddressFamily.InterNetwork, SocketType.Stream, ProtocolType.Tcp))
                    {
                        socket.Connect(endPoint);

                        using (NetworkStream ns = new NetworkStream(socket))
                        {
                            //POST /somepage.php HTTP/1.1
                            //Host: example.com
                            //Content-Type: application/x-www-form-urlencoded
                            //Content-Length: 19

                            byte[] bytes = System.Text.Encoding.UTF8.GetBytes(
                                "POST /PATH/FILE.php HTTP/1.1\r\n" +                
                                "Host: www.SERVER.com\r\n" +
                                "Connection: keep-alive\r\n" +
                                "Content-Length: 21\r\n" +
                                "Cache-Control: max-age=0\r\n" +
                                "Origin: null\r\n" +
                                "User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.4 (KHTML, like Gecko) Chrome/22.0.1229.94 Safari/537.4\r\n" +
                                "Content-Type: application/x-www-form-urlencoded\r\n" +
                                "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8\r\n" +
                                "Accept-Encoding: gzip,deflate,sdch\r\n" +
                                "Accept-Language: en-US,en;q=0.8\r\n" +
                                "Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.3\r\n" +
                                "\r\n" +
                                "TestID=&TestDesc=TEST");

                            ns.Write(bytes, 0, bytes.Length);

                            Debug.Print("POST done");
                        }
                    }
                }
                catch (Exception ex)
                {
                    Debug.Print(ex.Message.ToString());
                }

                //hide LED
                led.Write(false);
                Thread.Sleep(250);

                numTests--;
            }

            Debug.Print("Test complete");
        }

This just seems crazy to me. Anybody have any idea what could be going on or any input on how to troubleshoot?

#11 Chris Walker

Chris Walker

    Secret Labs Staff

  • Moderators
  • 7767 posts
  • LocationNew York, NY

Posted 15 October 2012 - 11:08 PM

Do you have a copy of Wireshark? Is it possible that it's the server which is locking the socket connection after exactly that many requests, as a denial-of-service protection strategy? Chris

#12 Patrick

Patrick

    Advanced Member

  • Members
  • PipPipPip
  • 54 posts
  • LocationTampa

Posted 15 October 2012 - 11:42 PM

Thanks Chris! I have WireShark installed but I don't have access to the shared host's OS nor do I know how to monitor the traffic from the N+ using it. I moved my test loop into the SocketExample you recommended (much smaller deployment) and it locks up at 561 iterations with no output. I'm going to try to access some local web services to eliminate the possibility of a server side issue.

#13 Nobby

Nobby

    Advanced Member

  • Members
  • PipPipPip
  • 70 posts

Posted 16 October 2012 - 04:31 AM

Hey Patrick, this may or may not be of any use because it depends on the back-end code for sockets. With the regular .Net framework, socket objects can be disposed but underlying socket resources are maintained by the CLR in the event that in a short space of time, you decide to create a new socket to the same remote host on the same port. This is true even if you call dispose and try to trash the socket as hard as you can. When you use the 'using' clause, it just calls Dispose on the IDisposable object. Your network stream will dispose fine but your socket objects might be doing funny things if the implementation of System.Net.Socket is the same under MicroFramework as it is for regular framework. This is the only thing I can think of that hasn't been explored yet.

#14 Arbiter

Arbiter

    Advanced Member

  • Members
  • PipPipPip
  • 132 posts
  • LocationBrisbane, Australia

Posted 16 October 2012 - 10:20 AM

Declare the socket outside the loop and just open and close the connection in the loop. Don't create stuff dynamically. In embedded systems you need resource usage to be very predictable if you want long term stability. You could also leave out most of that post header, it's unnecessary. You only need host, content-type and content-length. If you put content-length at the end you can make up a single string constant and greatly reduce the amount of gratuitous object creation.
One day, all this too shall parse.

#15 ericcox

ericcox

    Member

  • Members
  • PipPip
  • 19 posts

Posted 20 October 2012 - 08:17 AM

Hi Patrick,

I don't know that this is the problem, but in your original post, I noticed this line:

stOut = new StreamWriter(req.GetRequestStream());

This line allocates a Stream object, and then wraps that into a StreamWriter. The StreamWriter is being disposed, but it's possible, since the Stream is not being explicitly closed and disposed, that something is holding a reference to the underlying Stream.

You could try managing object lifetime with using() statements:

using ( Stream responseStream = req.GetRequestStream() )
{
    using ( stOut = new StreamWriter(responseStream ) ) 
    {
        stOut.Write(strNewValue);
        stOut.Close();
    }
}

Using is the equivalent of try {...} finally { <dispose object> }. So even if an exception is thrown, the object will be disposed, and the exception re-thrown.
It also makes it easier to see when objects are created/destroyed.

Hope that helps...




0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users

home    hardware    projects    downloads    community    where to buy    contact Copyright © 2016 Wilderness Labs Inc.  |  Legal   |   CC BY-SA
This webpage is licensed under a Creative Commons Attribution-ShareAlike License.