Netduino home hardware projects downloads community

Jump to content


The Netduino forums have been replaced by new forums at community.wildernesslabs.co. This site has been preserved for archival purposes only and the ability to make new accounts or posts has been turned off.

ukkiwisurfer's Content

There have been 33 items by ukkiwisurfer (Search limited from 29-March 23)


By content type

See this member's


Sort by                Order  

#64773 Netduino framework

Posted by ukkiwisurfer on 19 January 2016 - 03:01 PM in Netduino 3

Hi,

 

I've come to the painful conclusion that the .NET Micro framework just isn't worth the effort at the moment, purely from the toolchain perspective.

 

This isn't a decision I've come to lightly, especially after spending hundreds of hours on the platform (which seemed so very promising at the beginning). After upgrading to the Netduino 3 my issues of running out of memory were mostly resolved. But the issue of finding and fixing memory leaks (in my code I admit) became extraordinarily difficult and time consuming.  

 

Having to resort to print statements in the 21st century in order to identify memory leaks is not just painful - its also not very productive. Having to do so with a multi-threaded application makes it even more difficult. Using an interpreted language limits much of what the .NET tools and ecosystem can provide.

 

I've released the framework that I was working on as part of a larger project for .NET Micro under the MIT licence.

 

It can be found at https://github.com/u...mework.Embedded.

An example of how to use the framework can be found at: https://github.com/u.../UtilityMonitor 

 

Hopefully at some point in the near future the whole toolchain for .NET Micro can move into same same league as the traditional desktop .NET development. And hopefully some of what is in in the framework will be of benefit to the community.

 

Kind regards,

Jason.




#64522 Buffer OVFLW not being caught by try/catch

Posted by ukkiwisurfer on 16 November 2015 - 10:02 AM in General Discussion

Hi KiwiBryn,

 

I too have try/catch blocks (normally) around critical code sections. I realised over the weekend that even though I have an application level try/catch block, as the UDP capture code is running in its own thread, I was missing the associated try/catch block from the relevant thread looping code. Since making this change I'm not getting buffer overflow messages any more. 

 

The UDP thread captures whatever is in the buffer and then adds it to an internal (memory) queue. Another thread writes these blocks to the SD card (think micro-service architecture) using a StreamWriter in order to add additional metadata around the payload body. A third thread(service) reads these files off the SD and publishes to AMQP.

 

My next set of challenges is around an Unhandled Exception message being thrown from within the framework code (not a .NET exception) and a memory leak.

 

I have no idea where the un-handled exception is being thrown from but it seems to be happening around the point that my memory leak forces memory levels to a critical level. So my first plan of attack is to find the memory leak. To do this I'm going to disable all but one service and see if I can identify which service is leaking. Any other suggestions on approach? 

 

I've introduced a Debug.GC(true) into the code where I persist the file data via AMQP, which has helped considerably and I print out the free memory for reference. However over a couple of hours I can see the memory level gradually dropping. But obviously Debug.GC() just sticking plaster over a much more fundamental issue.

As I've introduced extensive logging statements into the code that also muddies the waters with regards to memory allocations for strings.

 

It would be nice if the tooling around .NETMF development moved into the late 20th century and not require print statements to identify leaks and the like. In cases like this I miss my crash dumps and WinDbg (however arcane the commands are) it definitely takes the guess work out of where objects are being pinned in memory.

 

Just imagine how more productive we would be.... :-)

 

Thanks,

Jason.




#64514 Buffer OVFLW not being caught by try/catch

Posted by ukkiwisurfer on 15 November 2015 - 10:31 AM in General Discussion

Hi KiwiBryn,

Sure, I can shed some details on the project. I'm building out a data capture platform, initially as a PoC for within the home.

The data is captured via a UDP broadcast that is happening elsewhere on the network. These packets are then saved as files on the SD card.

A separate thread runs, looking for files in a specific subdirectory, to then load each, one at a time into memory, before using amqp lite to transmit the data to a gateway (rabbitMQ).

Due to the lack of real analysis tools for the net mf platform that the desktop versions of .net support I've also had to write a generalised logging framework. To work out what is happening.

I wonder how you approach the same w.r.t logging and tracing on a constrained device?

Thanks,
Jason.



#64508 ND3 SD Filestream.Read() seems to become broken

Posted by ukkiwisurfer on 14 November 2015 - 11:29 PM in Netduino 3

Hi KiwiBryn,

 

I hadn't actually considered File.ReadAllBytes(). I'll give it a try. With regards to payload size its consistently about 556 bytes, so quite small but I get about 1 message every 5 seconds (thereabouts). 

 

It does sound very similar. The only difference is I'm publishing to a RabbitMQ instance running locally for further processing before being pushed into the cloud. I've followed your other posts with interest. :-)

 

Your insight with regards to VolumeInfo.FlushAll() will probably help mitigate the corruptions I sometimes see on the SD. ie. https://github.com/N...eter/issues/235. So thanks for this as well.

 

Thanks,

Jason.




#64500 Buffer OVFLW not being caught by try/catch

Posted by ukkiwisurfer on 12 November 2015 - 10:39 PM in General Discussion

I would add that this error message isn't being recorded through my logging system, and when it happens it takes the whole board down. Only a restart will recover it. No .NET exception is being raised either which makes me highly suspicious.

 

1. Has anyone encountered this type of error before?

2. Any suggestions as to how to capture more error information around what is actually happening? If its not raising a .NET exception, are their any type of crash dump files that get generated? 

 

Thanks,

Jason.




#64494 General support from Secret labs

Posted by ukkiwisurfer on 12 November 2015 - 01:07 PM in Netduino 3

Hi,

 

Thanks for the feedback. It confirms what I thought. There are some really helpful people out there in the community as I've already encountered.

 

I was just surprised there was only one primary contributor from Secret Labs.

 

Thanks,

Jason.




#64487 Buffer OVFLW not being caught by try/catch

Posted by ukkiwisurfer on 11 November 2015 - 09:22 PM in General Discussion

Hi Dankline,

 

Thanks for your reply. Unfortunately I don't know where its crashing, hence my attempt to use an application level try/catch block. I have the process running as I type with the breakpoint set to the debug.print statement within the catch block.

 

Its a multi-threaded application with separate data capture, data storage and transmission threads, accessing the ND3's inbuilt SD card.

 

My suspicion is that because I'm pushing the SD card in terms of volume of writes and reads, the SD card is getting corrupt. The error seems to appear around the point where it enumerates over folders to get the data files.

 

However its not consistent in terms of when it happens. Sometimes its within 10 minutes, other times its after a couple of hours consistent running.

 

I have noticed that if I try running it debug from within VS2013 it usually takes much longer to happen than if I just run the board and connect to it via USB using the .NET micro framework deployment tool (and view the output via the console listview). Which makes me wonder whether the debugging is actually masking the issue because of the additional latency it introduces.

 

In the normal .NET world you can use WinDbg.exe to analyse the crash dump files. Is there anything equivalent for .NET MF?




#64485 Buffer OVFLW not being caught by try/catch

Posted by ukkiwisurfer on 11 November 2015 - 07:41 PM in General Discussion

Hi,

 

I'm getting a 'Buffer OVFLW' message being logged to the console window. I have a general try/catch block at the application level, but it appears that this error is not being caught by the try/catch block. Can someone explain how this could happen? 

 

 try

 {

    var serviceManager = new ServiceManager();

    serviceManager.Run();

  }

  catch (Exception ex)

  {

     Debug.Print(ex.StackTrace);

  } 

 

And from the logging output (console window) I see it as:

 

2015-11-11 17:32:02.288 | 14 | FileSystemLogger | File matches pattern. File: '\SD\data\tosend\20151111173150715.data'.
Buffer OVFLW
Buffer OVFLW
 
It appears that it isn't being caught as I'd expect - I wanted to see what the stack trace was to work out what was causing the software to fail.



#64474 General support from Secret labs

Posted by ukkiwisurfer on 10 November 2015 - 09:10 AM in Netduino 3

A general question to the community...

 

Is there anyone else other than Chris from SecretLabs that monitors the forum and interacts with the community? Or is it literally just one person supporting the entire community (from the manufacturers perspective)?

 

Thanks,

Jason.

 




#64468 Corruption in forum post

Posted by ukkiwisurfer on 09 November 2015 - 10:22 AM in Netduino 3

Chris,the following post seems to have become corrupt.

 

http://forums.netdui...ing-filedelete/

 

Thanks,

Jason.




#64467 ND3 hangs calling File.Delete()

Posted by ukkiwisurfer on 09 November 2015 - 10:02 AM in Netduino 3

What is also interesting is whether there is any relationship between this issue and http://forums.netdui...become-broken/ 

 

I've seen this File.Delete() issue before and I've also seen the FileStream.Read() operation issue as well. The FileStream.Read() issue reared its head yesterday afternoon and late last night I saw the first incidence of the File.Delete() blocking issue. I powered off the board and continued running the soak test.

 

This morning found the File.Delete issue had re-occurred. I find it difficult to see it as a simple coincidence given they are both File.IO issues against the same SD card.

 

Thanks,

Jason. 




#64466 ND3 hangs calling File.Delete()

Posted by ukkiwisurfer on 09 November 2015 - 09:53 AM in Netduino 3

Hi Mark,

 

It's a 2GB SanDisk micro SD card. However after my last post it seems the problem is back on a different card. 

 

The thread ran from 2015-11-09 00:28:09.152 until 2015-11-09 00:52:44.284 this morning. Interestingly, after looking at the log files the last entry in the log file is:

 

2015-11-09 00:52:44.284 | 13 | ArchiveDataService | Deleting 6 of 7 files.

 

The code is as below:

 

    m_Logger.Debug("Deleting {0} of {1} files.", fileIndex, fileCount);

 

    long fileSize = m_FileHelper.GetFileSize(m_Configuration.TargetPath, fileName);

    if (fileSize >= 0)

    {

      m_Logger.Debug("Deleting file. Filesize: {0} bytes, ArchivePath: '{1}', Filename: '{2}'",  fileSize, m_Configuration.ArchivePath, fileName);

      m_FileHelper.DeleteFile(m_Configuration.ArchivePath, fileName);

     }

 

which implies that the code blocked on attempting to get the file size. And getting the file size operation is as follows:

 

 public long GetFileSize(string targetPath, string fileName)

 {

            long fileSize = 0;

 

            string filePath = Path.Combine(targetPath, fileName);

            if (File.Exists(filePath))

            {

                var info = new FileInfo(filePath);

                fileSize = info.Length;

            }

 

            return fileSize;

  }

 

Any suggestions?

 

Thanks,

Jason.




#64463 ND3 SD Filestream.Read() seems to become broken

Posted by ukkiwisurfer on 08 November 2015 - 11:34 PM in Netduino 3

My only solution at the moment is to manually seek in the FileStream such that the previous code now looks like:

 

 using (var stream = m_FileHelper.OpenStreamForRead(m_Configuration.TargetPath, fileName, m_BufferSize))

 {

   int index = 1;

   int offset = 0;

   int bytesRead = 0;

 

   while (offset < fileSize)

   {

     LogDebug("Seeking position in filestream. Position: {0}", offset);

 

     stream.Seek(offset, SeekOrigin.Begin);

     bytesRead = stream.Read(buffer, 0, buffer.Length);

     offset += bytesRead;

 

     LogDebug("Reading filestream. Loop: {0}, BytesRead: {1}, Position: {2}", index, bytesRead, stream.Position);

     memoryStream.Write(buffer, 0, bytesRead);

 

     index++;

    }

 

This implies that somehow that (in the original approach) the file stream's position after a Read() is either being corrupted or altered by something else. If it is the latter, how does this correlate with a FileShare.None flag for the file stream?




#64462 ND3 hangs calling File.Delete()

Posted by ukkiwisurfer on 08 November 2015 - 11:27 PM in Netduino 3

I've swapped out the SD card from this ND3 for another and the issue seems to have gone away. However it implies that the file.IO logic is choking on something and completely killing the ND3. That doesn't seem right...

 

Chris, is there any known issues around this that you are aware of?




#64461 ND3 SD Filestream.Read() seems to become broken

Posted by ukkiwisurfer on 08 November 2015 - 11:24 PM in Netduino 3

Hi,

 

I'm seeing some odd behaviour sometimes when reading off an SD card. I have a dedicated thread that reads data to be sent, and reads it off the SD and into a MemoryStream, before being transmitted via an AMQP connection.

 

The MSDN documentation implies that the Read operation will leave the underlying stream at the position that the previous FileStream.Read() left it at. However I am seeing situations where it appears to be going backwards, but apparently only on certain files.

 

The FileStream has been opened only for reading, with no share allowed for any other process. Attached below is a snapshot of the logging file. The structure is:   Timestamp | Thread Id | Logical component name | Log message

 

2015-11-08 23:00:31.128 | 16 | DataTransferService | Started processing file - FileSize: '20151108225110431.data'

2015-11-08 23:00:31.185 | 16 | DataTransferService | File is within allowable size. FileSize: '554'

2015-11-08 23:00:31.274 | 16 | DataTransferService | Reading filestream. Loop: 1, BytesRead: 400, Position: 400

2015-11-08 23:00:31.358 | 16 | DataTransferService | Reading filestream. Loop: 2, BytesRead: 400, Position: 246

2015-11-08 23:00:31.440 | 16 | DataTransferService | Reading filestream. Loop: 3, BytesRead: 400, Position: 92

2015-11-08 23:00:31.526 | 16 | DataTransferService | Reading filestream. Loop: 4, BytesRead: 400, Position: 400

2015-11-08 23:00:31.582 | 16 | DataTransferService | Reading filestream. Loop: 5, BytesRead: 400, Position: 246

2015-11-08 23:00:31.667 | 16 | DataTransferService | Reading filestream. Loop: 6, BytesRead: 400, Position: 92

 

The relevant code where this is being detected is as follows (where m_BufferSize defaults to 512):

 

 ...

 byte[] buffer = new byte[m_BufferSize];

 ...

 

 using (var stream = m_FileHelper.OpenStreamForRead(m_Configuration.TargetPath, fileName, m_BufferSize))

 {

   int index = 1;

   int offset = 0;

   int bytesRead = 0;

 

   while ((bytesRead = stream.Read(buffer, 0, buffer.Length)) > 0)

   {

     LogDebug("Reading filestream. Loop: {0}, BytesRead: {1}, Position: {2}", index, bytesRead, stream.Position);

     memoryStream.Write(buffer, 0, bytesRead);

 

     index++;

    }

  }

 

Anyone else come across this odd behaviour?




#64456 ND3 hangs calling File.Delete()

Posted by ukkiwisurfer on 06 November 2015 - 09:19 AM in Netduino 3

Hi,

 

I've had a search through the forums and found questions being asked (with no solutions from Secret Labs) on how to address hanging NDs with respect to corruption on SDs.

 

Scenario: ND3 (Wifi) on mains, power fails while ND is writing to SD. Subsequent to this the File.Delete() operation will hang the ND indefinitely. Theres no exception being raised and (obviously) there is no timeout you can provide when requesting a delete operation.

 

One solution is to provide a battery back up for the ND so you don't get this sort of scenario, but I'd like to understand what is actually happening so I actually address the issue and not the symptom.

 

Has anyone else had any luck remedying this sort of issue, especially for remotely deployed devices that are difficult to get access to?

 

Thanks,

Jason.




#64365 Netduino 3 Wifi connectivity failure mode handling

Posted by ukkiwisurfer on 24 October 2015 - 12:45 PM in Netduino 3

Hi Chris,

 

Is there any progress on this issue? 

 

Thanks,

Jason.




#64364 No WiFi signal

Posted by ukkiwisurfer on 23 October 2015 - 10:37 PM in Netduino 3

Hi,

 

Have you tried setting the Authentication in your configuration to 'Shared'?

 

Thanks,

Jason.




#64362 UDP Broadcast receive on ND 3 (Wifi)

Posted by ukkiwisurfer on 23 October 2015 - 09:14 PM in Netduino 3

Hi,

 

I'm looking into an issue with listening to UDP broadcasts on a Netduino 3 (Wifi). If I run the 4.3.2.0 firmware on the board I get the data packets I'm expecting. If I upgrade to 4.3.2.1 (or later) the UDP broadcasts aren't being received (with no code changes between deployments).

 

The associated code is summarised below. 

 

public UdpSocket(string serverAddress, int port)
{
           m_Client = new Socket(AddressFamily.InterNetwork, SocketType.Dgram, ProtocolType.Udp);
           m_Endpoint = new IPEndPoint(IPAddress.Parse(serverAddress), port);

 }

 

public void Open()
{      
           m_Client.Bind(new IPEndPoint(IPAddress.Any, m_Endpoint.Port));

}

 

public bool CheckForIncomingData()

 {        
           bool hasPayload = m_Client.Poll(PeekTimeoutInMilliseconds, SelectMode.SelectRead);
           return hasPayload;

 }

 

The CheckForIncomingData() detects incoming packets and indicates whether an associated message handling routine should commence - based on Poll() returning true (4.3.2.0) if data has been received. On 4.3.2.1 and higher Poll() always returns false.

 

Any suggestions?

 

Thanks,

Jason.




#64357 Waiting for a network connection

Posted by ukkiwisurfer on 22 October 2015 - 08:30 PM in Netduino 3

The following code should remedy the above issue for the Netduino Plus 2 and will still work for the Netduino 3 (Ethernet and Wifi) boards. Short summary: specify a timeout in milliseconds and loop until the DHCP address has been assigned.

 

/// <summary>
/// Returns the IP address of the device.
/// </summary>

public NetworkInformation GetNetworkDetails(int interfaceIndex)
{
           NetworkInformation information = null;

           // Set up event handlers for network state changes.
           NetworkChange.NetworkAvailabilityChanged += OnInternalNetworkAvailabilityChanged;
           NetworkChange.NetworkAddressChanged += OnNetworkAddressChanged;

           try
           {
               // Wait for the network to become available and for a DHCP address to be allocated.
               bool isWaitingToInitialise = true;
               while (isWaitingToInitialise)
               {
                   var isSignalled = WaitHandle.WaitAll(new[] { m_WaitForAddressChange, m_WaitForNetwork }, m_WaitForNetworkEventInMilliseconds, false);
                   if (!isSignalled)
                   {
                       // Retrieve network interface details to see if the initialisation had already occurred.
                       information = QueryInterface(interfaceIndex);
                       if ((information.IsDHCPEnabled) && (information.IpAddress != "0.0.0.0"))
                       {
                           isWaitingToInitialise = false;
                       }
                   }
                   else isWaitingToInitialise = false;
               }

               information = QueryInterface(interfaceIndex);
           }
           finally
           {
               m_WaitForAddressChange.Reset();
               m_WaitForNetwork.Reset();
           }

           return information;
       }

       /// <summary>
       /// Queries a network device for details about its connection state.
       /// </summary>
       /// <param name="interfaceIndex">
       /// The index of the device to query.
       /// </param>
       /// <returns>
       /// The status information on the network device.
       /// </returns>
       public NetworkInformation QueryInterface(int interfaceIndex)
       {
           NetworkInformation information = null;

          
               var networkInterface = GetInterface(interfaceIndex);
               if (networkInterface != null)
               {
                   // Then capture the network details. 
                   information = new NetworkInformation();
                   information.IpAddress = networkInterface.IPAddress;
                   information.IsDHCPEnabled = networkInterface.IsDhcpEnabled;
                   information.SubnetMask = networkInterface.SubnetMask;
                   information.MacAddress = networkInterface.PhysicalAddress;
                   information.NetworkInterfaceType = networkInterface.NetworkInterfaceType.ToString();
               }

           return information;
       }




#64356 Waiting for a network connection

Posted by ukkiwisurfer on 22 October 2015 - 08:21 PM in Netduino 3

In an interesting twist to this issue, running this code on a Netduino Plus 2 highlights a race condition. The issue is that the ethernet and Wifi networking capabilities on the Netduino 3 take a lot longer to initialise (3+ seconds) than the Netduino Plus 2's (4.3.2.1 Firmware) Ethernet does (milliseconds).

 

The consequence is, the above code will block indefinitely on Netduino Plus 2 at the WaitHandle.WaitAll() call because the NetworkChange event handlers never trigger. The network has already been initialised and configured (DHCP) even before the code has had a chance to set up the event handlers.

 

However the Netduino 3 (Ethernet and Wifi) take several seconds to initialise, allowing the NetworkChange event handlers to be set up and to fire.




#64317 DNS.GetNetworkTime

Posted by ukkiwisurfer on 16 October 2015 - 11:58 AM in Netduino 3

After investigating further it appears that this issue is related to the time it is taking for the board to have a DHCP IP address allocated to it. i.e. The board (unless you wait for the network address to change) will have a default IP address of 0.0.0.0.

 

See http://forums.netdui...ork-connection/




#64315 Waiting for a network connection

Posted by ukkiwisurfer on 16 October 2015 - 11:43 AM in Netduino 3

Following on from Chris' comments about waiting for a network connection on the Wifi version of the Netduino 3, I've written a Network helper class to use for both the Ethernet and Wifi boards:

 

 public NetworkInformation GetNetworkDetails(int interfaceIndex)
 {
     NetworkInformation information = null;
 
     NetworkChange.NetworkAvailabilityChanged += OnNetworkAvailabilityChanged;
     NetworkChange.NetworkAddressChanged += OnNetworkAddressChanged;
 
     // Wait for the network to become availableand for a DHCP address to be allocated.
     WaitHandle.WaitAll(new[] { m_WaitForAddressChange, m_WaitForNetwork });
....
}
 
The Ethernet version is configured to use DHCP. For reference the associated code to enable the WaitHandle.WaitAll() is below.
 
        /// <summary>
        /// Initialises an instance of the <see cref="NetworkHelper"/> class.
        /// </summary>
        public NetworkHelper()
        {
            m_WaitForNetwork = new AutoResetEvent(false);
            m_WaitForAddressChange = new AutoResetEvent(false);
        }
 
        /// <summary>
        /// Event handler to signal when the network address has changed.
        /// </summary>
        /// <param name="sender"></param>
        /// <param name="eventArgs"></param>
        private void OnNetworkAddressChanged(object sender, EventArgs eventArgs)
        {
            m_WaitForAddressChange.Set();
        }
 
        /// <summary>
        /// Event handler to signal that the network is now available.
        /// </summary>
        /// <param name="sender"></param>
        /// <param name="networkAvailabilityEventArgs"></param>
        private void OnNetworkAvailabilityChanged(object sender, NetworkAvailabilityEventArgs eventArgs)
        {
            if (eventArgs.IsAvailable)
            {
                m_WaitForNetwork.Set();
            }
        }



#64313 DNS.GetNetworkTime

Posted by ukkiwisurfer on 15 October 2015 - 10:06 PM in Netduino 3

Hi,

 

I've noticed an interesting issue with the Netduino 3 (Ethernet) today. Thanks to TechnoGuy for pointing out I had posted in an old forum...

 

I deployed the same (unmodified) application code that runs perfectly fine on a Netduino Plus 2 onto the Netduino 3, and on the Netduino 3 it throws an exception in the code below

 

public DateTime GetNetworkTime(string hostName = "time-a.nist.gov")

{

            var hostEntry = Dns.GetHostEntry(hostName);

.....

}

 

An unhandled exception of type 'System.Exception' occurred in Netduino.IP.LinkLayers.AX88796C.

 

My Netduino Plus 2 has been upgraded to 4.3.2.1 and is using the new IP stack. I'm using Windows 7, VS2013 and using .Net MF SDK 4.3. 

 

Any suggestions?

 

Thanks.




#64310 Netduino Plus 2 Firmware v4.2.2 (update 2)

Posted by ukkiwisurfer on 15 October 2015 - 07:14 PM in Netduino Plus 2 (and Netduino Plus 1)

Hi Chris,

 

Following on from my comment to Bradygaster I discovered that the issue I was having was related to calling File.Delete(filePath). At this point the board became unresponsive and I had to wipe the application from the board (Netduino Plus 2).

 

The code had worked for some time without an issues and then started having these freezes (reproducible every time). In the end I had to reformat the SD card and since then it seems to have been working ok.

 

My only guess is that buried down in the NETMF code it got stuck trying to access the SD filesystem and couldn't break out of either a resource contention or attempted to access the required resources without any sort of timeout.

 

Thanks,

Jason.





home    hardware    projects    downloads    community    where to buy    contact Copyright © 2016 Wilderness Labs Inc.  |  Legal   |   CC BY-SA
This webpage is licensed under a Creative Commons Attribution-ShareAlike License.