29 replies to this topic

[skobyjay]

Just init your string array to be the count of lines in your file. Then populate, see what memory looks like

[skobyjay]

I know Get requests are typically lighter than Postbut have you tried to use Post to populate your database? I don't know how large your files are but it might be worth a shot. If the files are large then you could create smaller files. This would reduce the HTTP activity anyway with fewer and larger requests.

[skobyjay]

Does the http response class you are using implement Idisposable? If so, wrap it with using. Just another thought

[don664]

I know Get requests are typically lighter than Postbut have you tried to use Post to populate your database? I don't know how large your files are but it might be worth a shot. If the files are large then you could create smaller files.

 

The files are only a couple of kb each... very small but it doesn't seem that the wifi SimpleSocket allows HTTP Posts for transferring files.

 

Was really hoping that could be a solution to replace FTP or the need for an HTTP request for each count; but it looks like that is the only option at the moment 

 

As for the memory leak I'm considering just restarting the Netduino (using PowerState.RebootDevice(true);) as soon as the memory gets too low. This isn't a solution though as you'll invariably lose a few counts while the Netduino restarts.

 

Is there a way to simply restart the threads? I've tried Thread.Abort(); but this simply throws an exception when I try and abort a thread. I had hoped that by aborting a thread once I was done with it, it would release any memory resources it was using.

 

To do this I tried (just an outline of the program):

 

class program{     // Define the thread    public static Thread populateArrayThread = new Thread(populateArray);    Main    {        connectToWifi();    }    connectToWifi()    {        // connect to wifi stuff here        populateArrayThread.Start();    }    populateArray()    {        //populate array from SD card file        httpDataSend();    }    httpSendData()    {        populateArrayThread.Abort();        // send HTTP querystring requests from array variables        cleanUp();    } }

 

When I try and abort the thread I get:

 

A first chance exception of type 'System.ArgumentException' occurred in mscorlib.dll

An unhandled exception of type 'System.ArgumentException' occurred in mscorlib.dll

 

Also the wifi response does not include "Idisposable"... not that I'd know how to use it anyway 

 

To clean up the array, as it's being used, I'm removing each value after it's been set via HTTP... still not helping much though.

 

 

 

[emg]

Have looked at any other WiFi options? I have played about with an OpenPicus Flyport and it has lots of configuration options, send data via FTP, email, UDP, etc or has a built in webserver. 49 euros. I've tested mine with sending/receiving UDP packets to my PC via Wifi and it seems solid.

 

http://wiki.openpicu...le=Flyport_WiFi

 

http://www.openpicus.com/

 

 

[don664]

Thanks emg,This looks very promising and seems to be what I'm looking for......from what I've read briefly this is a stand alone product and won't simply replace the wifi module on my Netduino?

[emg]

It could be either, really.  I plan on using mine as a WiFi Ethernet to SPI bridge. I guess it depends how smart your counter devices need to be. If you want to 'process' or manipulate the data, cache it locally onto SD card, etc or just count and send totals. You might get way with just the Flyport if you don't need all the power of a N2+.

 

Here are some more links to libraries and sample applications:

 

http://wiki.openpicu...title=Tutorials

 

http://wiki.openpicu...munity_Projects

 

http://wiki.openpicu...title=Libraries

 

 

[skobyjay]

By implement Idisposable I mean can you wrap it with a using statement like you do with the filestream object? Did you try to use a simple string array with only the size that you need rather than using array list? I think this should reduce memory used. Because array list is built on a basic array and a basic array only has memory allocated for the size it needs to be rather than having to grow dynamically with each add.

[don664]

Hi skobyJay,

 

I wasn't sure how to modify the  array as you suggested, or implement the iDisposable using method so I thought the best thing was to simply try and put the whole project together including the sensor.and this seems to have helped with the memory leak.

 

Previously I was setting up a loop to simply read from a SD card file, populate the array and send via HTTP then read from a second file, send and start at the first file again. This loop seems to have been the real issue as it never "released" any of the threads?

 

Anyway now that the SD card read/array populate/http send are triggered by the sensor count the memory loss has stopped (apart from a few bytes for the counter variables obviously). So i can only imagine it's because that now there isn't a loop the garbage collector can release resources held by the various threads.

 

I'm still having timeout issues with the wifi module but at least it feels like another step forward.

 

The most recent version of the code below:

 

using System;using System.Collections;using System.IO;using System.Net;using System.Net.Sockets;using System.Text;using System.Threading;using Microsoft.SPOT;using Microsoft.SPOT.Hardware;using SecretLabs.NETMF.Hardware;using SecretLabs.NETMF.Hardware.Netduino;using Toolbox.NETMF.Hardware;using Toolbox.NETMF.NET;using VariableLabs.PowerManagment;namespace testToolbox21240{    public class Program    {        // Hardware: wifly, sensor and LCD        static WiFlyGSX WifiModule;        static InterruptPort mySensor;        // SD Card file variables        static string strWriteFile = "dataone";        static string strReadFile = "datatwo";        static int intDeviceID = 1;        // WiFi SSID and password        static string strSSID = "mySSID";        static string strPassword = "myPassword";        // Remote host URL        static string strRemoteHost = "myRemoteHost";        // Counter variables        static uint intCounter = 0;        static int intSendCounter = 0;        static int intArrayCounter = 0;        static uint intTimeoutCounter = 0;        static ArrayList counterDataArray = new ArrayList();        static OutputPort led = new OutputPort(Pins.ONBOARD_LED, false);        public static SimpleSocket Socket;        public static HTTP_Client WebSession;                public static void Main()        {            PowerManagment.SetPeripheralState(Peripheral.Ethernet, false);            Debug.EnableGCMessages(true);            connectToWifi();            // setup sensor interrupt            mySensor = new InterruptPort(Pins.GPIO_PIN_D7, false, Port.ResistorMode.Disabled, Port.InterruptMode.InterruptEdgeHigh);            mySensor.OnInterrupt += new NativeEventHandler(mySensor_OnInterrupt);            Thread.Sleep(Timeout.Infinite);        }        public static void mySensor_OnInterrupt(uint port, uint state, DateTime time)        {            string strDateTime = (DateTime.Now.AddHours(2).Month.ToString() + "/" + DateTime.Now.AddHours(2).Day.ToString() + "/" + DateTime.Now.AddHours(2).Year.ToString() + "%20" + DateTime.Now.AddHours(2).Hour.ToString() + ":" + DateTime.Now.AddHours(2).Minute.ToString() + ":" + DateTime.Now.AddHours(2).Second.ToString());            using (var filestream = new FileStream(@"SD" + strWriteFile + ".txt", FileMode.Append))            {                StreamWriter sw = new StreamWriter(filestream);                sw.WriteLine("deviceID=" + intDeviceID + "&count=" + intCounter + "&dateTimeStamp=" + strDateTime);                sw.Flush();                sw.Close();            }            intCounter++;            intSendCounter++;            Debug.Print("intCounter: " + intCounter + "intSendCounter: " + intSendCounter);            Thread.Sleep(250);            if (intSendCounter == 5)            {                intSendCounter++;                populateArray();            }        }        public static void populateArray()        {            if (strWriteFile == "dataone")            {                strWriteFile = "datatwo";                strReadFile = "dataone";            }            else            {                strWriteFile = "dataone";                strReadFile = "datatwo";            }            Debug.Print("Populate array from: " + strReadFile);            using (var filestream = new FileStream(@"SD" + strReadFile + ".txt", FileMode.Open))            {                string strSendLine;                StreamReader sr = new StreamReader(filestream);                while ((strSendLine = sr.ReadLine()) != null)                {                    counterDataArray.Add(strSendLine);                }                sr.Close();                sr.Dispose();            }            Thread httpSendThread = new Thread(httpSendData);            httpSendThread.Start();                   }        public static void httpSendData()        {            try            {                foreach (string value in counterDataArray)                {                    HTTP_Client.HTTP_Response Response = WebSession.Get("/default.aspx?" + value);                }            }            catch (System.ApplicationException e)            {                Debug.Print("Error Code: " + e);                intTimeoutCounter++;                Debug.Print("intTimeoutCounter: " + intTimeoutCounter);                using (var filestream = new FileStream(@"SDlog.txt", FileMode.Append))                {                    StreamWriter sw = new StreamWriter(filestream);                    sw.WriteLine("DateTime: " + DateTime.Now.AddHours(2) + "rnError Code: " + e + "rnintTimeoutCounter: " + intTimeoutCounter + "rn");                    sw.Flush();                    sw.Close();                    sw.Dispose();                }                Thread.Sleep(1500);                httpSendData();            }            catch (System.NullReferenceException f)            {                Debug.Print("Error Code: " + f);                using (var filestream = new FileStream(@"SDlog.txt", FileMode.Append))                {                    StreamWriter sw = new StreamWriter(filestream);                    sw.WriteLine("DateTime: " + DateTime.Now.AddHours(2) + "rnError Code: " + f + "rnintTimeoutCounter: " + intTimeoutCounter + "rn");                    sw.Flush();                    sw.Close();                    sw.Dispose();                }            }            finally            {                cleanUp();            }        }        public static void cleanUp()        {            File.Delete(@"SD" + strReadFile + ".txt");                        counterDataArray.Clear();                        Debug.Print("Free Memory: " + Debug.GC(true).ToString());            intSendCounter = 0;                        Debug.GC(true);                    }        public static void connectToWifi()        {            WifiModule = new WiFlyGSX();            WifiModule.EnableDHCP();            WifiModule.JoinNetwork(strSSID, 0, WiFlyGSX.AuthMode.MixedWPA1_WPA2, strPassword);            Thread.Sleep(3500);            // Showing some interesting output            Debug.Print("Local IP: " + WifiModule.LocalIP);            Debug.Print("MAC address: " + WifiModule.MacAddress);            // Set the RTC for GMT + 2            SNTP_Client TimeClient = new SNTP_Client(new WiFlySocket("ntp2.is.co.za", 123, WifiModule));            Thread.Sleep(1000);            TimeClient.Synchronize();            Debug.Print("DateTime.Now: " + DateTime.Now.AddHours(2));            Socket = new WiFlySocket(strRemoteHost, 80, WifiModule);            WebSession = new HTTP_Client(Socket);        }    }} 

 

Thanks for the help... I'll keep you posted on any further progress!

 

Have a great weekend!

[don664]

Hi emg,

 

Thanks for the lead. I have spoken with OpenPicus and ordered a board which is on it's way for me to test. Hopefully their wifi module can handle FTP (which they claim) and will help my project along.

 

Cheers,

Donovan