Hi,

I've come to the painful conclusion that the .NET Micro framework just isn't worth the effort at the moment, purely from the toolchain perspective.

This isn't a decision I've come to lightly, especially after spending hundreds of hours on the platform (which seemed so very promising at the beginning). After upgrading to the Netduino 3 my issues of running out of memory were mostly resolved. But the issue of finding and fixing memory leaks (in my code I admit) became extraordinarily difficult and time consuming.  

Having to resort to print statements in the 21st century in order to identify memory leaks is not just painful - its also not very productive. Having to do so with a multi-threaded application makes it even more difficult. Using an interpreted language limits much of what the .NET tools and ecosystem can provide.

I've released the framework that I was working on as part of a larger project for .NET Micro under the MIT licence.

It can be found at https://github.com/u...mework.Embedded.

An example of how to use the framework can be found at: https://github.com/u.../UtilityMonitor 

Hopefully at some point in the near future the whole toolchain for .NET Micro can move into same same league as the traditional desktop .NET development. And hopefully some of what is in in the framework will be of benefit to the community.

Kind regards,

Jason.

Chris,the following post seems to have become corrupt.

http://forums.netdui...ing-filedelete/

Thanks,

Jason.

Hi,

I'm looking into an issue with listening to UDP broadcasts on a Netduino 3 (Wifi). If I run the 4.3.2.0 firmware on the board I get the data packets I'm expecting. If I upgrade to 4.3.2.1 (or later) the UDP broadcasts aren't being received (with no code changes between deployments).

The associated code is summarised below. 

public UdpSocket(string serverAddress, int port)
{
           m_Client = new Socket(AddressFamily.InterNetwork, SocketType.Dgram, ProtocolType.Udp);
           m_Endpoint = new IPEndPoint(IPAddress.Parse(serverAddress), port);

 }

public void Open()
{      
           m_Client.Bind(new IPEndPoint(IPAddress.Any, m_Endpoint.Port));

}

public bool CheckForIncomingData()

 {        
           bool hasPayload = m_Client.Poll(PeekTimeoutInMilliseconds, SelectMode.SelectRead);
           return hasPayload;

 }

The CheckForIncomingData() detects incoming packets and indicates whether an associated message handling routine should commence - based on Poll() returning true (4.3.2.0) if data has been received. On 4.3.2.1 and higher Poll() always returns false.

Any suggestions?

Thanks,

Jason.

After investigating further it appears that this issue is related to the time it is taking for the board to have a DHCP IP address allocated to it. i.e. The board (unless you wait for the network address to change) will have a default IP address of 0.0.0.0.

See http://forums.netdui...ork-connection/

Hi,

I've noticed an interesting issue with the Netduino 3 (Ethernet) today. Thanks to TechnoGuy for pointing out I had posted in an old forum...

I deployed the same (unmodified) application code that runs perfectly fine on a Netduino Plus 2 onto the Netduino 3, and on the Netduino 3 it throws an exception in the code below. 

public DateTime GetNetworkTime(string hostName = "time-a.nist.gov")

{

            var hostEntry = Dns.GetHostEntry(hostName);

.....

}

An unhandled exception of type 'System.Exception' occurred in Netduino.IP.LinkLayers.AX88796C.

My Netduino Plus 2 has been upgraded to 4.3.2.1 and is using the new IP stack. I'm using Windows 7, VS2013 and using .Net MF SDK 4.3. 

Any suggestions?

Thanks.

Hi Chris,

My apologies for the lack of clarification -it is multicast that seems to be the problem. The nature of your reply seems to confirm that UDP multicast is not currently supported with the existing IP stack. Is that correct?

I'd like to help with testing on the new IP stack; however the existing issue does pose a challenge. If I read your new IP rollout correctly, 12 weeks from now we should have a new stack, feeding into a beta programme. I can't really afford to wait 12 weeks to resolve this existing issue, blocking development in the process. I've already lost a week trying to resolve issues around the existing IP stack (which thankfully your 4.3.1 firmware partially resolved).

I'm also concerned about the lack of clarity on what features in the MF are not supported by the firmware (or have issues). Do you have a defect list, by firmware version, that identifies issues, severity, status and timeframes to resolve?

I'm hoping to soon be in a state to release the framework I've built out for MF, which may be of use to others. Nothing fancy: multi-threaded services (not web services), logging support for AMQP and MQTT providers, basic assertions, and a standardised set of error classifications. The aim is to support integrating with a server side framework I've built up around WCF. 

Finally, thanks for the welcome!

Regards,

Jason.

Following on from Chris' comments about waiting for a network connection on the Wifi version of the Netduino 3, I've written a Network helper class to use for both the Ethernet and Wifi boards:

The following code should remedy the above issue for the Netduino Plus 2 and will still work for the Netduino 3 (Ethernet and Wifi) boards. Short summary: specify a timeout in milliseconds and loop until the DHCP address has been assigned.

/// <summary>
/// Returns the IP address of the device.
/// </summary>

public NetworkInformation GetNetworkDetails(int interfaceIndex)
{
           NetworkInformation information = null;

           // Set up event handlers for network state changes.
           NetworkChange.NetworkAvailabilityChanged += OnInternalNetworkAvailabilityChanged;
           NetworkChange.NetworkAddressChanged += OnNetworkAddressChanged;

           try
           {
               // Wait for the network to become available and for a DHCP address to be allocated.
               bool isWaitingToInitialise = true;
               while (isWaitingToInitialise)
               {
                   var isSignalled = WaitHandle.WaitAll(new[] { m_WaitForAddressChange, m_WaitForNetwork }, m_WaitForNetworkEventInMilliseconds, false);
                   if (!isSignalled)
                   {
                       // Retrieve network interface details to see if the initialisation had already occurred.
                       information = QueryInterface(interfaceIndex);
                       if ((information.IsDHCPEnabled) && (information.IpAddress != "0.0.0.0"))
                       {
                           isWaitingToInitialise = false;
                       }
                   }
                   else isWaitingToInitialise = false;
               }

               information = QueryInterface(interfaceIndex);
           }
           finally
           {
               m_WaitForAddressChange.Reset();
               m_WaitForNetwork.Reset();
           }

           return information;
       }

       /// <summary>
       /// Queries a network device for details about its connection state.
       /// </summary>
       /// <param name="interfaceIndex">
       /// The index of the device to query.
       /// </param>
       /// <returns>
       /// The status information on the network device.
       /// </returns>
       public NetworkInformation QueryInterface(int interfaceIndex)
       {
           NetworkInformation information = null;

          
               var networkInterface = GetInterface(interfaceIndex);
               if (networkInterface != null)
               {
                   // Then capture the network details. 
                   information = new NetworkInformation();
                   information.IpAddress = networkInterface.IPAddress;
                   information.IsDHCPEnabled = networkInterface.IsDhcpEnabled;
                   information.SubnetMask = networkInterface.SubnetMask;
                   information.MacAddress = networkInterface.PhysicalAddress;
                   information.NetworkInterfaceType = networkInterface.NetworkInterfaceType.ToString();
               }

           return information;
       }

Hi,

Has anyone had any luck with UDP on a Netduino Plus 2 with MF 4.3.1? I seem to get errors when using the SetSocketOption call. The exception returns an internal error id of 10042 (Bad protocol option).

I'm starting to wonder whether using the Netduino boards are worth it (given this sort of issue and other bugs - like the network stack not detecting ethernet plug disconnects). I've noticed that someone has tried this on another MF board and it worked fine. Any suggestions?

I'm trying to capture UDP messages broadcast by another device on the local network.

        /// <summary>

        /// Starts the client connection.

        /// </summary>

        public void Open()

        {

            m_IsOpen = false;

            lock (m_SyncLock)

            {

                var ipAddress = IPAddress.Parse("224.192.32.19");

                var ipBytes = ipAddress.GetAddressBytes();

                m_Client.SetSocketOption(SocketOptionLevel.IP, SocketOptionName.MulticastInterface, ipBytes);

                m_Client.Bind(m_Endpoint);

                m_IsOpen = true;

            }

        }

In an interesting twist to this issue, running this code on a Netduino Plus 2 highlights a race condition. The issue is that the ethernet and Wifi networking capabilities on the Netduino 3 take a lot longer to initialise (3+ seconds) than the Netduino Plus 2's (4.3.2.1 Firmware) Ethernet does (milliseconds).

The consequence is, the above code will block indefinitely on Netduino Plus 2 at the WaitHandle.WaitAll() call because the NetworkChange event handlers never trigger. The network has already been initialised and configured (DHCP) even before the code has had a chance to set up the event handlers.

However the Netduino 3 (Ethernet and Wifi) take several seconds to initialise, allowing the NetworkChange event handlers to be set up and to fire.

My only solution at the moment is to manually seek in the FileStream such that the previous code now looks like:

 using (var stream = m_FileHelper.OpenStreamForRead(m_Configuration.TargetPath, fileName, m_BufferSize))

 {

   int index = 1;

   int offset = 0;

   int bytesRead = 0;

   while (offset < fileSize)

   {

     LogDebug("Seeking position in filestream. Position: {0}", offset);

     stream.Seek(offset, SeekOrigin.Begin);

     bytesRead = stream.Read(buffer, 0, buffer.Length);

     offset += bytesRead;

     LogDebug("Reading filestream. Loop: {0}, BytesRead: {1}, Position: {2}", index, bytesRead, stream.Position);

     memoryStream.Write(buffer, 0, bytesRead);

     index++;

    }

This implies that somehow that (in the original approach) the file stream's position after a Read() is either being corrupted or altered by something else. If it is the latter, how does this correlate with a FileShare.None flag for the file stream?

Hi KiwiBryn,

I hadn't actually considered File.ReadAllBytes(). I'll give it a try. With regards to payload size its consistently about 556 bytes, so quite small but I get about 1 message every 5 seconds (thereabouts). 

It does sound very similar. The only difference is I'm publishing to a RabbitMQ instance running locally for further processing before being pushed into the cloud. I've followed your other posts with interest. :-)

Your insight with regards to VolumeInfo.FlushAll() will probably help mitigate the corruptions I sometimes see on the SD. ie. https://github.com/N...eter/issues/235. So thanks for this as well.

Thanks,

Jason.

Hi,

I'm seeing some odd behaviour sometimes when reading off an SD card. I have a dedicated thread that reads data to be sent, and reads it off the SD and into a MemoryStream, before being transmitted via an AMQP connection.

The MSDN documentation implies that the Read operation will leave the underlying stream at the position that the previous FileStream.Read() left it at. However I am seeing situations where it appears to be going backwards, but apparently only on certain files.

The FileStream has been opened only for reading, with no share allowed for any other process. Attached below is a snapshot of the logging file. The structure is:   Timestamp | Thread Id | Logical component name | Log message

2015-11-08 23:00:31.128 | 16 | DataTransferService | Started processing file - FileSize: '20151108225110431.data'

2015-11-08 23:00:31.185 | 16 | DataTransferService | File is within allowable size. FileSize: '554'

2015-11-08 23:00:31.274 | 16 | DataTransferService | Reading filestream. Loop: 1, BytesRead: 400, Position: 400

2015-11-08 23:00:31.358 | 16 | DataTransferService | Reading filestream. Loop: 2, BytesRead: 400, Position: 246

2015-11-08 23:00:31.440 | 16 | DataTransferService | Reading filestream. Loop: 3, BytesRead: 400, Position: 92

2015-11-08 23:00:31.526 | 16 | DataTransferService | Reading filestream. Loop: 4, BytesRead: 400, Position: 400

2015-11-08 23:00:31.582 | 16 | DataTransferService | Reading filestream. Loop: 5, BytesRead: 400, Position: 246

2015-11-08 23:00:31.667 | 16 | DataTransferService | Reading filestream. Loop: 6, BytesRead: 400, Position: 92

The relevant code where this is being detected is as follows (where m_BufferSize defaults to 512):

 ...

 byte[] buffer = new byte[m_BufferSize];

 ...

 using (var stream = m_FileHelper.OpenStreamForRead(m_Configuration.TargetPath, fileName, m_BufferSize))

 {

   int index = 1;

   int offset = 0;

   int bytesRead = 0;

   while ((bytesRead = stream.Read(buffer, 0, buffer.Length)) > 0)

   {

     LogDebug("Reading filestream. Loop: {0}, BytesRead: {1}, Position: {2}", index, bytesRead, stream.Position);

     memoryStream.Write(buffer, 0, bytesRead);

     index++;

    }

  }

Anyone else come across this odd behaviour?

I've swapped out the SD card from this ND3 for another and the issue seems to have gone away. However it implies that the file.IO logic is choking on something and completely killing the ND3. That doesn't seem right...

Chris, is there any known issues around this that you are aware of?

Hi, 

I've also decided that I'm not going to continue with the Netduino platform primarily because of 2 factors: 

1) The inability of the Netduino team to deliver basic working infrastructure code on time (to their delivery schedule).

2) with the open sourcing of .NET (and specifically WCF) I see no need to constrain myself to the ancient Micro .NET platform and it's restrictions.

I've 25 years of commercial software engineering experience, most of it on server side and in multi-threaded development, and I find the basic networking issues on the platform inexcusable in this day and age.

I'm moving off the Netduino platform to more open and capable hardware that can accommodate the demands of the 21st century. I'm also removing myself from being dependent on the Netduino team, who seem unable to project manage themselves to deliver on time.

It might seem a little harsh but there is no excuse for the manner in which the community here have been treated by the Netduino team over this new build. I can't justify relying on a supplier who doesn't communicate what they are doing and also doesn't honour their communicated goals. 

I will be shortly releasing my Micro.NET libraries for general use on GitHub (under an open licence) as I move my focus back to the Core .NET framework for embedded development.

Regards,

Jason.

Hi Chris,

Is there any progress on this issue? 

Thanks,

Jason.

What is also interesting is whether there is any relationship between this issue and http://forums.netdui...become-broken/ 

I've seen this File.Delete() issue before and I've also seen the FileStream.Read() operation issue as well. The FileStream.Read() issue reared its head yesterday afternoon and late last night I saw the first incidence of the File.Delete() blocking issue. I powered off the board and continued running the soak test.

This morning found the File.Delete issue had re-occurred. I find it difficult to see it as a simple coincidence given they are both File.IO issues against the same SD card.

Thanks,

Jason. 

Hi,

I've had a search through the forums and found questions being asked (with no solutions from Secret Labs) on how to address hanging NDs with respect to corruption on SDs.

Scenario: ND3 (Wifi) on mains, power fails while ND is writing to SD. Subsequent to this the File.Delete() operation will hang the ND indefinitely. Theres no exception being raised and (obviously) there is no timeout you can provide when requesting a delete operation.

One solution is to provide a battery back up for the ND so you don't get this sort of scenario, but I'd like to understand what is actually happening so I actually address the issue and not the symptom.

Has anyone else had any luck remedying this sort of issue, especially for remotely deployed devices that are difficult to get access to?

Thanks,

Jason.

Hi Mark,

It's a 2GB SanDisk micro SD card. However after my last post it seems the problem is back on a different card. 

The thread ran from 2015-11-09 00:28:09.152 until 2015-11-09 00:52:44.284 this morning. Interestingly, after looking at the log files the last entry in the log file is:

2015-11-09 00:52:44.284 | 13 | ArchiveDataService | Deleting 6 of 7 files.

The code is as below:

    m_Logger.Debug("Deleting {0} of {1} files.", fileIndex, fileCount);

    long fileSize = m_FileHelper.GetFileSize(m_Configuration.TargetPath, fileName);

    if (fileSize >= 0)

    {

      m_Logger.Debug("Deleting file. Filesize: {0} bytes, ArchivePath: '{1}', Filename: '{2}'",  fileSize, m_Configuration.ArchivePath, fileName);

      m_FileHelper.DeleteFile(m_Configuration.ArchivePath, fileName);

     }

which implies that the code blocked on attempting to get the file size. And getting the file size operation is as follows:

 public long GetFileSize(string targetPath, string fileName)

 {

            long fileSize = 0;

            string filePath = Path.Combine(targetPath, fileName);

            if (File.Exists(filePath))

            {

                var info = new FileInfo(filePath);

                fileSize = info.Length;

            }

            return fileSize;

  }

Any suggestions?

Thanks,

Jason.

Hi,

Have you tried setting the Authentication in your configuration to 'Shared'?

Thanks,

Jason.

Hi KiwiBryn,

I too have try/catch blocks (normally) around critical code sections. I realised over the weekend that even though I have an application level try/catch block, as the UDP capture code is running in its own thread, I was missing the associated try/catch block from the relevant thread looping code. Since making this change I'm not getting buffer overflow messages any more. 

The UDP thread captures whatever is in the buffer and then adds it to an internal (memory) queue. Another thread writes these blocks to the SD card (think micro-service architecture) using a StreamWriter in order to add additional metadata around the payload body. A third thread(service) reads these files off the SD and publishes to AMQP.

My next set of challenges is around an Unhandled Exception message being thrown from within the framework code (not a .NET exception) and a memory leak.

I have no idea where the un-handled exception is being thrown from but it seems to be happening around the point that my memory leak forces memory levels to a critical level. So my first plan of attack is to find the memory leak. To do this I'm going to disable all but one service and see if I can identify which service is leaking. Any other suggestions on approach? 

I've introduced a Debug.GC(true) into the code where I persist the file data via AMQP, which has helped considerably and I print out the free memory for reference. However over a couple of hours I can see the memory level gradually dropping. But obviously Debug.GC() just sticking plaster over a much more fundamental issue.

As I've introduced extensive logging statements into the code that also muddies the waters with regards to memory allocations for strings.

It would be nice if the tooling around .NETMF development moved into the late 20th century and not require print statements to identify leaks and the like. In cases like this I miss my crash dumps and WinDbg (however arcane the commands are) it definitely takes the guess work out of where objects are being pinned in memory.

Just imagine how more productive we would be.... :-)

Thanks,

Jason.

A general question to the community...

Is there anyone else other than Chris from SecretLabs that monitors the forum and interacts with the community? Or is it literally just one person supporting the entire community (from the manufacturers perspective)?

Thanks,

Jason.

Hi,

I'm getting a 'Buffer OVFLW' message being logged to the console window. I have a general try/catch block at the application level, but it appears that this error is not being caught by the try/catch block. Can someone explain how this could happen? 

 try

 {

    var serviceManager = new ServiceManager();

    serviceManager.Run();

  }

  catch (Exception ex)

  {

     Debug.Print(ex.StackTrace);

  } 

And from the logging output (console window) I see it as:

Hi Dankline,

Thanks for your reply. Unfortunately I don't know where its crashing, hence my attempt to use an application level try/catch block. I have the process running as I type with the breakpoint set to the debug.print statement within the catch block.

Its a multi-threaded application with separate data capture, data storage and transmission threads, accessing the ND3's inbuilt SD card.

My suspicion is that because I'm pushing the SD card in terms of volume of writes and reads, the SD card is getting corrupt. The error seems to appear around the point where it enumerates over folders to get the data files.

However its not consistent in terms of when it happens. Sometimes its within 10 minutes, other times its after a couple of hours consistent running.

I have noticed that if I try running it debug from within VS2013 it usually takes much longer to happen than if I just run the board and connect to it via USB using the .NET micro framework deployment tool (and view the output via the console listview). Which makes me wonder whether the debugging is actually masking the issue because of the additional latency it introduces.

In the normal .NET world you can use WinDbg.exe to analyse the crash dump files. Is there anything equivalent for .NET MF?

Hi,

Thanks for the feedback. It confirms what I thought. There are some really helpful people out there in the community as I've already encountered.

I was just surprised there was only one primary contributor from Secret Labs.

Thanks,

Jason.