Netduino home hardware projects downloads community

Jump to content


The Netduino forums have been replaced by new forums at community.wildernesslabs.co. This site has been preserved for archival purposes only and the ability to make new accounts or posts has been turned off.
Photo

Netduino Firmware v4.2.0 RC5 (Netduino + Netduino Plus)


  • Please log in to reply
138 replies to this topic

#121 MartinK

MartinK

    New Member

  • Members
  • Pip
  • 3 posts

Posted 07 August 2012 - 01:47 PM

Hi,
I am using Netduino Plus (v4.2.0.0 RC5) and I've got the following problem:

When reading some files from an Sd-Card an ArgumentOutOfRangeException is thrown when calling StreamReader.ReadLine().

using (FileStream fs = File.OpenRead(filePath))
{
  using (TextReader tr = new StreamReader(fs))
  {
    string line;
    while ((line = tr.ReadLine()) != null)
    ...
  }
}

The stacktrace shows that the exception is thrown inside StreamReader.Peek() which is called internally by StreamReader.ReadLine().

StackTrace:
IndexOutOfRangeException HRESULT: 2835349504
System.IO.StreamReader::Peek
System.IO.StreamReader::ReadLine

The exception
- is only thrown in some files
- is never thrown on the first line
- is always thrown on reading the same line in a file, so I can reproduce it
- is thrown in bigger files (24Kb+), in smaller ones it's working (could be coincidence)

I checked the lines in question and they are neither longer than lines where it works, nor are there any special characters. The only common factor I see atm is the size of the file.

I'd be grateful for suggestions.

EDIT:

I refactored the code to not use the StreamReader and ReadLine and got rid of the exception.

int read;
var buffer = new byte[100];

using (FileStream fs = File.OpenRead(filePath))
{
  while ((read = fs.Read(buffer, 0, buffer.Length)) != 0)
  ..
}

Take care,
Martin

#122 AxelG

AxelG

    Advanced Member

  • Members
  • PipPipPip
  • 52 posts

Posted 10 August 2012 - 02:28 AM

HI AxelG,


Can you post a small repro app for us to look at? Generally speaking, NETMF 4.1 code should run as-is under NETMF 4.2.

Chris


I wound up flashing my ND+ to 4.2 anyway and noticed a couple things:
1) My USB is a lot more stable - no more BSODs (I also removed a bluetooth driver that may have been causing problems)
2) The extra code space is nice! Thanks.
3) I am seeing strange behavior with the Serial Port - losing data. (Posting a separate thread)
4) The COM2 problem went away (ArgumentOutOfRange no longer happens when opening the COM port.

#123 AxelG

AxelG

    Advanced Member

  • Members
  • PipPipPip
  • 52 posts

Posted 10 August 2012 - 02:51 AM

Unusual behavior with Serial Ports in 4.2 RC5 on ND and ND+ (Losing data…)

I have a project that requires two XBee/ZigBee modems to communicate and transfer large (for Zigbee) data packets from 1k to about 100k in size.

My plan is to use SD cards to buffer data as it come in and goes out the modems, stitching the packets together as they come in. Simple enough….

I am using the Grommet (Copyright © 2009 http://grommet.codeplex.com) drivers for the low-level networking, which I really like because they started out very light-weight and I made them lighter by removing some methods I will not need.

The project is too large to post the entire code base in this thread, but I plan to post later on as an extension of these drivers.

Here is my trouble: I am losing data after the receiving XBee passes it to the ND/ND+ and when it is read out of the serial buffer. Attached is the Logic trace showing data leaving XBee #1, and being passed from XBee #2 into the ND. The data loss happens at different places at different times. If I slow the transmissions way down (one message every two seconds) I get nearly zero lost bytes. As soon as I remove this delay; lost data.

Here is the setup:
  • ND with XBee1 and SD shield on SPI
  • ND+ with XBee2 and SD card installed
  • Both ND/ND+ share common XBee drivers codebase (All XBee and serial port code is shared)
  • I have tried hardware flow control on both ND/ND+ and is enabled on XBee.
  • (As a footnote, CTS never de-asserts on either XBee, it always remains low, and never goes low on ND)
  • I have SerialErrorReceivedEventHandler enabled, and it never fires.
  • Everything is running at 9600 baud, 8N1. (I want to try faster speeds to see if that makes a difference, but CTS flow control should handle buffering data for me?)

Attached is a far-away view of the data in the LA. The left side is showing the modem setup exchange, and the right side is showing the data packets: you can see the packeting leaving one XBee and arriving at the other… These are two messages with 200 byte packages.

Also attaching a zoomed in view to compare with what I am reading out of the Serial Port buffer.

Here is the code that is reading the buffer. Fairly simple. Called by the SerialDataReceivedEventHandler event:
private void Serial_DataReceived(object sender, SerialDataReceivedEventArgs e)
        {

            //One at a time please....
            lock (readLock)
            {
                Debug.Print("Read unlocked with " + SerialPort.BytesToRead + " bytes to read");

                //FIXIT - May not need, but for some reason the event fires with nothing to read....
                int timeout = 100;
                while (SerialPort.BytesToRead <= 0 && timeout-- > 0) Thread.Sleep(10);

                byte[] buf = new byte[1];
                if (SerialPort.BytesToRead > 0)
                {
                    try
                    {
                        while (SerialPort.BytesToRead > 0)
                        {
                                
			   //If there is data, there is a incomplete frame, and we have a complete header
                            while ((SerialPort.BytesToRead > 0) && 
                                  (!frameBuilder.IsComplete) && 
                                  (frameBuilder.HeaderIsComplete)))
                            {
			       //Don’t forget about the checksum byte…  The + 1
                                byte[] buffer = new byte[frameBuilder.dataNeeded + 1];
                                //reading the data frame and the checksum byte
                                int bytesRead = SerialPort.Read(buffer, 0, frameBuilder.dataNeeded + 1);
                                Debug.Print("Read chunk of " + bytesRead.ToString() + " f: " + buffer[0].ToString() + " l: " + buffer[bytesRead-1].ToString() + " need " + (frameBuilder.dataNeeded + 1).ToString());
                                //Add the newly read data to the framBuilder buffer
                                frameBuilder.Append(buffer, bytesRead);
                            }
                            if (frameBuilder.IsComplete)
                            {
                                // got a frame, do something
                                ReceivedApiFrame(frameBuilder.GetApiFrame());
                                // Create a thread, launch fire event, etc
                                frameBuilder.Reset();
                            }
			    //Check to see is a header needs to be read (first three bytes)
                            if ((!frameBuilder.HeaderIsComplete) && (SerialPort.BytesToRead > 0))
                            {
                                SerialPort.Read(buf, 0, 1);
                                Debug.Print("Read byte for header " + buf[0].ToString());
                                frameBuilder.Append(buf, 1);
                            }
                            //Don't beat a hasty retreat after reading the frame…
                            timeout = 100;
                            while (SerialPort.BytesToRead <= 0 && timeout-- > 0) Thread.Sleep(10);
                        }
                    }
                    catch (Exception ex)
                    {
                        Debug.Print("Error reading data from serial " + ex.Message);
                    }
                }
                else Debug.Print("Nothing to read");//FIXIT – Get rid of this if() in production, ignore

                Debug.Print("Ending read " + local.ToString());
            }
        }

Here is the relevant Debug file to try and convince myself I am not going nuts... The data is really not there...

Read byte for header 126 <- good
Read byte for header 0 <- good
Read byte for header 212 <- good
Allocated 212 bytes to API Frame <- reading 200 bytes plus Zigbee frame
Read chunk of 16 f: 144 l: 52 need 213 <- extra room for checksum byte
Read chunk of 16 f: 34 l: 109 need 197
Read chunk of 15 f: 101 l: 109 need 181
Read chunk of 14 f: 32 l: 50 need 166
Read chunk of 15 f: 48 l: 87 need 152
Read chunk of 15 f: 97 l: 100 need 137
Read chunk of 14 f: 60 l: 48 need 122
Read chunk of 15 f: 54 l: 48 need 108
Read chunk of 15 f: 58 l: 99 need 93
Read chunk of 14 f: 114 l: 100 need 78
Read chunk of 15 f: 97 l: 49 need 64
Read chunk of 14 f: 32 l: 97 need 49
Read chunk of 15 f: 114 l: 105 need 35
Read chunk of 17 f: 99 l: 100 need 20
Read chunk of 3 f: 97 l: 110 need 3 <- finished, without checksum error!
Resetting frame
Staring processing thread… <- thread off to process the packet
(Here is where it goes horribly wrong….)
Read byte for header 126 <- good, found header start byte
Read byte for header 162 <- BAD! This is NOT a valid high-byte length
Error in length: 162
Resetting frame
Read byte for header 0
ERROR!!! Expected start byte: 0
Read byte for header 64
ERROR!!! Expected start byte: 64
Read byte for header 141
ERROR!!! Expected start byte: 141
Read byte for header 28
ERROR!!! Expected start byte: 28
Read byte for header 195
ERROR!!! Expected start byte: 195


The serial buffer contained the following bytes: (Refer to the attached LA image)
126(~), 162, 0, 64(@), 141, 28, 195…
But the XBee sent the following to the ND:
126(~), 0, 212, 144, 0, 19, 162, 0, 64(@), 141, 28, 195…
MISSING BYTES

Any ideas are welcome. I need to get some sleep!

Attached Files



#124 Chris Walker

Chris Walker

    Secret Labs Staff

  • Moderators
  • 7767 posts
  • LocationNew York, NY

Posted 10 August 2012 - 06:02 AM

Hi AxelG, From the logic analyzer...can you tell where the data is being lost? Is it being lost on the Netduino (i.e. buffer overflow)? Chris

#125 AxelG

AxelG

    Advanced Member

  • Members
  • PipPipPip
  • 52 posts

Posted 10 August 2012 - 01:02 PM

Hi AxelG,

From the logic analyzer...can you tell where the data is being lost? Is it being lost on the Netduino (i.e. buffer overflow)?

Chris


If you look at the data in Data.jpg, that is the data coming from the XBee into the ND, and all of the bytes are in that stream. I can only assume the data is getting lost after it enters the Netduino. I am working on a Netduino to Netduino set of applications to reproduce. So far I am able to force the same exception I was seeing in the 4.1 release:

"A first chance exception of type 'System.ArgumentException' occurred in Microsoft.SPOT.Hardware.SerialPort.dll
Exception: Exception was thrown: System.ArgumentException param name stack System.IO.Ports.SerialPort::Read
NDPlusSerialReader.Program+DoReader::Serial_DataReceived
System.IO.Ports.SerialPort::DataEventHandler"

I will post more later today.

#126 AxelG

AxelG

    Advanced Member

  • Members
  • PipPipPip
  • 52 posts

Posted 10 August 2012 - 03:54 PM

Testing scenario for Serial Ports in 4.2 RC5

Scenario:
  • Hooked up a ND to ND+ on COM2 (TX to RX)
  • ND was sender, ND+ was reader
  • ND had no flow control, ND+ was set to Handshake.RequestToSend
  • Logic Analyzer was hooked to RX, TX and RTS/CTS pins on ND+
  • Wrote two applications, Reader and Sender (Projects.zip)
  • Sender running on ND sends 200 byte packages to reader running on ND+
  • Reader reads the 200 byte blocks and reports on whether it got all of the data.

Outcome:
  • Handshake.RequestToSend seems to have no affect on the hardware.
  • Sending the data slowly enough for the reader to stay caught up with the writer works fine.
  • When the reader goes off to do something else (simulated by a Thread.Sleep()) the SerialPort.Read() method throws “System.ArgumentException” (Debug 2.txt)
  • If the writer sends data to the buffer (SerialPort.Write() method) too fast, the “SerialErrorReceivedEventHandler” handler fires with “SerialError.TXFull”
One Solution Idea:
  • Use XON/XOFF flow control.
  • Set no delay in reading the data.
  • Set one second delay between sending packages.
Outcome:
  • Lost data. Reading the packages as fast as possible shows lost data (Debug 3.txt) This happens even when sending data in 200 byte packages with a delay between packages.
  • Turning off flow control, fixes the problem at slow speeds, until you set a read delay which causes data loss.

Any ideas on how I can send large blocks of data without losing bytes?

Attached Files



#127 Chris Walker

Chris Walker

    Secret Labs Staff

  • Moderators
  • 7767 posts
  • LocationNew York, NY

Posted 11 August 2012 - 09:51 AM

Hi AxelG, Just to confirm... You're connecting two Netduinos together and are using flow control...and are losing data? If that's the scenario, does the same thing happen when you connect two serial ports together on one Netduino? We use 115200 bps data on Netduinos regularly and never see data loss...so I'd really like to figure out what's going on in this scenario. Also...there appears to be an errata on the SAM7X microcontroller regarding hardware flow control. Let's stick with software flow control at the moment. Thank you for helping narrow this down so we can repro and help push a fix to the NETMF codebase. Chris P.S. RC6 should be shipping in a few days. There is a serial port bugfix that recently made it into the NETMF core...so we may want to hold off digging in for a few days and see if the bugfix is related.

#128 AxelG

AxelG

    Advanced Member

  • Members
  • PipPipPip
  • 52 posts

Posted 11 August 2012 - 03:54 PM

Hi AxelG,

Just to confirm... You're connecting two Netduinos together and are using flow control...and are losing data?


Yes, that is the scenario. Using no flow control works well as long as I can read data fast enough. I am going to play with various baud rates and see what changes.

If that's the scenario, does the same thing happen when you connect two serial ports together on one Netduino?

I will try that.


P.S. RC6 should be shipping in a few days. There is a serial port bugfix that recently made it into the NETMF core...so we may want to hold off digging in for a few days and see if the bugfix is related.

Cool. I will try that as well.

#129 AxelG

AxelG

    Advanced Member

  • Members
  • PipPipPip
  • 52 posts

Posted 11 August 2012 - 08:43 PM

Chris:
Thanks for the feedback. Here is how I spent my morning.

I used one ND with COM1 hooked to COM2 and got exactly the same results. All works fine as long as I can read the data fast enough. If not, there are very unpredictable results including crashing the ND. If I use any type of handshake; all bets are off and data is lost. I will eagerly await the RC6 build and re-test.

(I have not ported this back to 4.1 to see what happens there, that would require reflashing my NDs. and I am basically lazy that way! :) )


Here is another very peculiar thing related to serial ports. I posted about this in an earlier thread concerning 'System.InvalidOperationException' in the 'System.IO.Ports.SerialPort.HandlePinReservations' call. I am wondering if my 4.2 implementation is FUBAR? Here is the scenario:

Running a simple test application:
using System.IO.Ports;
using Microsoft.SPOT;
using Microsoft.SPOT.Hardware;
using SecretLabs.NETMF.Hardware.Netduino;


namespace NetduinoApplication2
{
    public class Program
    {
        public class netduino
        {
            //Changing from const to static causes the sp.Open() method to throw a 'System.InvalidOperationException' Exception
            public const Cpu.Pin SomePin = Pins.GPIO_PIN_A4;
            //public static Cpu.Pin SomePin = Pins.GPIO_PIN_A4;
            public InterruptPort SomeInterrupt = new InterruptPort(SomePin, true, Port.ResistorMode.Disabled, Port.InterruptMode.InterruptEdgeBoth);
        }

        protected static netduino MCU = new netduino();

        public static void Main()
        {
            SerialPort sp = new SerialPort("COM2", 9600, Parity.None, 8, StopBits.One);
            sp.Open();
            Debug.Print("finished OK!");
        }
    }
}

Notice the definition for SomePin. If I define SomePin as static, the debug trace looks like this on the sp.Open() call:
Step into: Stepping over non-user code 'System.IO.Ports.SerialPort.Open'
Step into: Stepping over non-user code 'System.IO.Ports.SerialPort.Open'
Step into: Stepping over non-user code 'System.IO.Ports.SerialPort.HandlePinReservations'
Step into: Stepping over non-user code 'Microsoft.SPOT.Hardware.HardwareProvider.HwProvider.get'
Step into: Stepping over non-user code 'System.IO.Ports.SerialPort.HandlePinReservations'
Step into: Stepping over non-user code 'System.IO.Ports.SerialPort.HandlePinReservations'
A first chance exception of type 'System.InvalidOperationException' occurred in Microsoft.SPOT.Hardware.dll
Step into: Stepping over non-user code 'System.IO.Ports.SerialPort.HandlePinReservations'
A first chance exception of type 'System.InvalidOperationException' occurred in Microsoft.SPOT.Hardware.SerialPort.dll
An unhandled exception of type 'System.InvalidOperationException' occurred in Microsoft.SPOT.Hardware.SerialPort.dll

If I define as a const, all is good and no exception.

Not to make this even more confusing, but when I run the debugger and step through the code, when I get to the call for 'HandlePinReservations' I get a dialog box asking me to locate the following file:
C:\Documents\Secret Labs\Projects\Production\SecretLabs.NETMF.Hardware.Netduino\NetduinoHardwareProvider.cs (See attachment SourceScreen.jpg)

Any ideas on what is going on and where I might go next? The loss of data part is really the troubling part. I need to have an effective handshake so I can read and process larger chunks of data.

Attached Files



#130 Arbiter

Arbiter

    Advanced Member

  • Members
  • PipPipPip
  • 132 posts
  • LocationBrisbane, Australia

Posted 13 August 2012 - 04:39 AM

Axel, what happens if you don't use SomePin at all and just specify a literal like this?

SomeInterrupt = new InterruptPort(Pins.GPIO_PIN_A4, true, Port.ResistorMode.Disabled, Port.InterruptMode.InterruptEdgeBoth);

One day, all this too shall parse.

#131 AxelG

AxelG

    Advanced Member

  • Members
  • PipPipPip
  • 52 posts

Posted 18 August 2012 - 10:43 PM

Arbiter: When I do that, all works well. I am trying to abstract the hardware later from the rest of my code. I need to know what pin was used later because I share interrupts and what to know which pin caused the interrupt.

#132 AxelG

AxelG

    Advanced Member

  • Members
  • PipPipPip
  • 52 posts

Posted 18 August 2012 - 11:24 PM

Chris: I just confirmed that the same serial port issue exists in the latest 4.2 build (posted three days ago) If I use any flow control, I lose data. If I don't use flow control, I have to make sure to read data fast enough.... This happens with one ND, COM1 to COM2 or two netduinos connected with null-modem cable.

#133 AxelG

AxelG

    Advanced Member

  • Members
  • PipPipPip
  • 52 posts

Posted 02 September 2012 - 04:28 PM

Any idea on what I should try next? I hit a wall with the serial port problems in this 4.2 build and really can't move forward until they are working.

#134 mohammad

mohammad

    Advanced Member

  • Members
  • PipPipPip
  • 79 posts

Posted 30 October 2012 - 06:05 PM

Hi all,

No updates yet for Netduino Plus firmware to support Micro SD 4GB?

I developed a program for Netduino Plus to work (read/write) with Micro SD 4GB.
This program worked perfect with firmware v4.1.1 betta, but after upgrading to firmware v4.2.0 RC4, I got these errors:

A first chance exception of type 'System.NotSupportedException' occurred in Microsoft.SPOT.IO.dll
A first chance exception of type 'System.NotSupportedException' occurred in System.IO.dll
System.NotSupportedException
Would you please help me solve this problem?


Special Thanks,
Mohammad

#135 Chris Walker

Chris Walker

    Secret Labs Staff

  • Moderators
  • 7767 posts
  • LocationNew York, NY

Posted 02 November 2012 - 03:05 AM

Hi mohammad,

No updates yet for Netduino Plus firmware to support Micro SD 4GB?

KodeDaemon sent over some updates before the storm. We'll be checking them in next week, and integrating into the newest firmware...

Chris

#136 Martin Ayotte

Martin Ayotte

    Member

  • Members
  • PipPip
  • 15 posts

Posted 24 November 2012 - 11:44 PM

Hi Chris, Any news for the KodeDaemon code been integrated in latest firmware ? Thanks.

#137 Chris Walker

Chris Walker

    Secret Labs Staff

  • Moderators
  • 7767 posts
  • LocationNew York, NY

Posted 30 November 2012 - 09:42 AM

Hi Martin,

Any news for the KodeDaemon code been integrated in latest firmware ?

The Netduino Plus 2 bug fixes snuck up to the top of the list, but we have these on our short-list after those are complete.

[And thank you again to KodeDaemon for his submission]

BTW, did you try formatting your card as 2GB or 4GB? That may help as a temporary workaround...albeit not optimal ;)

Chris

#138 Martin Ayotte

Martin Ayotte

    Member

  • Members
  • PipPip
  • 15 posts

Posted 30 November 2012 - 09:42 PM

Hi Chris, Thanks for the reply. I will wait for KodeDaemon's submission ... In the mean time, yes, I've tried formatting my 8GB as a 1.5GB, either with gparted or DiskPart as suggested in forum, either in FAT16 or FAT32, without any success. Application thread not responding during 30secs, before automount fails, then my application see a unformatted volume. (it a Lexar uSDHC, but according to wikipedia, every uSDHC should be able to speak with SPI.) Also, I've ordered a Kingston 2GB uSD on eBay, but it will proably arrived only in 2 weeks. With that one, I will make sure my netduino can at least see this one. Martin.

#139 Martin Ayotte

Martin Ayotte

    Member

  • Members
  • PipPip
  • 15 posts

Posted 09 December 2012 - 02:21 AM

Hi again Chris, Just to let you know : since I didn't received yet my Kingston 2GB uSD, I can't give more news on that. But, in the mean time, I've received a Arduino Network Shield with SD, I've tried out with my old Arduino Mega 2560, and I figured out that the Lexar uSDHC 8GB was working fine with it, even after reformatted to full 8GB FAT32. So, my conclusion is the bug is really related to auto-mount in Netduino .NETMF micro-framework. I hope KodeDaemon's submission will give us great news. In the mean time, since I've also received a fullsize SD adapter, I will try it on my N+2 in the following days, hookup on SPI, to see if I can mount several fullsize SDs.




0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users

home    hardware    projects    downloads    community    where to buy    contact Copyright © 2016 Wilderness Labs Inc.  |  Legal   |   CC BY-SA
This webpage is licensed under a Creative Commons Attribution-ShareAlike License.