The Netduino forums have been replaced by new forums at community.wildernesslabs.co.
This site has been preserved for archival purposes only
and the ability to make new accounts or posts has been turned off.
I am having an ongoing problem reading from the Netduino plus 2 serial port.
I have a serial stream coming in from a GPS shield and I know its very constant, right baud rate, stop bits etc. I have it set to 4800 baud for test. It is byte data (not ASCII) and it is always a fixed number of bytes.
The very minimum code (below) shows the incoming number byte count as expected but when it reaches 256 it cycles a couple of times and then locks up.
I'm sure its reached a buffer limit but how to I take care of that? The Flush doesn't seem to work.
Thanks.
using System;using System.Net;using System.Net.Sockets;using System.Threading;using Microsoft.SPOT;using Microsoft.SPOT.Hardware;using SecretLabs.NETMF.Hardware;using SecretLabs.NETMF.Hardware.Netduino;using System.IO.Ports;namespace Test{ public class Program { public static void Main() { int[] Message = new int[36]; // get the waiting data SerialPort serialPort; serialPort = new SerialPort("COM2", 4800, Parity.None, 8, StopBits.One); serialPort.Open(); serialPort.Flush(); while (true) { int x; x = serialPort.BytesToRead; Debug.Print(x.ToString()); } } }}
The behavior is probably correct. By the way, you are querying for the quantity of bytes allocated in the temporary (internal) buffer, but you never pull them out. At a certain point, the buffer overflows, so...well, maybe it's 256 bytes.
Your approach is correct, in the sense that you should run a loop (not so tight hopefully), to test for incoming data. When some new bytes are pushed into the temporary buffer, you have to pull them out and store them in your own (external) buffer. Doing so the serial is working much as a "streaming" source, other than a protocol oriented device. A particular protocol could be implemented by your code, though.
Hope it helps.
Cheers
Biggest fault of Netduino? It runs by electricity.
yes as mario said you must pull the bytes out as they, or they are stuck in the com buffer and will eventually overflow.
I suspect you have a netduiono plus 2; the internal com buffer there (receive and transmit, per com port (4)) is 256 bytes. (somewhat interestingly, on the netduino 2 it is 512, but either way those are compiled-in choices of the firmware).
There's also a DataReceived event that's meant to fire when there is data is available; however, fair warning, I have not found it to be reliable in my experience. It works for a while after opening the port, then ceases to be fired anymore.
Yes Zigg - I did try the event and yes it did stop working after a while. The part that bothers me is that no exception is generated when the buffer overflows and the Netduino simply locks up tight. I have to press the reset button, several times usually, to get it to load again.
I would actually prefer to receive each byte and process it as they come, because the GPS format is embedded in the payload, but if the whole thing locks up as I am trying to process a message I need to rethink using the Netduino at all. The Arduino worked fine except that I needed more memory.
interesting. I'm doing some serial stuff as we speak (write); I've never explicitly let it overrun -- i agree it shouldn't lock up. When I get a moment I'll try to let it overflow on pupose is see how it behaves. Also, there is also an event for errors which I would /thing/ would get fired if you had an overruns, but again I've not tried stimulating it explicitly to say whether it works or not. As mentioned before, I've had a bad experience recently with the DataAvailable event, so Ive avoided events for now.
That being said, you certainly can process a character at a time as it comes in, but you have to retrieve the byte before you process it! Here is a trivial modification your your loop (not a model of good code, but brief and illustrative I hope, oh and I havent' actually run it, but I think it should work!):
while (true){ int x; //this is a count of what is available, not the data itself x = serialPort.BytesToRead; if ( 0 == x ) { Thread.Sleep ( 100 ); //wait a little bit } else { //get all that is available byte[] aby = new byte[x]; serialPort.Read(aby, 0, x); Debug.Print ( "I just read: " + x + " and got '" + HexStr(aby, 0, x) + "'" ); }}
as I write that, I wonder now if the misunderstanding is that BytesToRead returns the data. It doesn't, it returns the amount of data. You need Read() to actually get it (into a buffer of your creation). If you consume it as it's produced, you won't overflow the internal buffers.
I'm about to give up on the Netduino. The serial port is simply not reliable enough to use.
serialPort.BytesToRead reports 96 every time I send a block of bytes. I know for sure its sending 36 bytes. I'm also sure I'm sending N81 for the serial port setup. I have tried everything from 4800 to 115200 baud rate all with the same result - always 96 bytes.
The serial buffer does have the first few bytes correct but after the seventh byte it is in error. The error is always the same buffer after buffer.
The UART on my shield is sending:
B5 62 01 02 ......
This program reports.
Read: 96 and got 'B5 62 01 06 ......
The first error (read 06 should be 02) is exactly the same on each buffer
I have tried DiscardInBuffer and Flush with no change.
I am also have a problem deploying this program. Most of the time I get:
Attaching to device...
Waiting for device to initialize...
The debugging target and the debugger engine failed to initialize because of unspecified device errors.
The debugger engine thread has terminated unexpectedly with error 'Object reference not set to an instance of an object.'.
If I try it again I may get:
Attaching to device...
Deployment error: failed to (re)connect debugger engine to debugging target.
The debugging target and the debugger engine failed to initialize because of unspecified device errors.
The debugger engine thread has terminated unexpectedly with error 'Debugger engine could not attach to debugging target.'.
I can deploy 'Blinky' without problems so I think its something to do with System.IO.Ports.
Has anyone managed to send bytes to the Netduino UART with success?
sing System;using System.Net;using System.Net.Sockets;using System.Threading;using Microsoft.SPOT;using Microsoft.SPOT.Hardware;using SecretLabs.NETMF.Hardware;using SecretLabs.NETMF.Hardware.Netduino;using System.IO.Ports;namespace Test{ public class Program { public static string HexStr(byte[] p, int nIdxStart, int nCount) { char[] c = new char[nCount * 2]; byte b; for (int y = 0, x = 0; y < nCount; ++y, ++x) { int nIdx = nIdxStart + y; b = ((byte)(p[nIdx] >> 4)); c[x] = (char)(b > 9 ? b + 0x37 : b + 0x30); b = ((byte)(p[nIdx] & 0xF)); c[++x] = (char)(b > 9 ? b + 0x37 : b + 0x30); } return new string(c); } public static void Main() { SerialPort serialPort; serialPort = new SerialPort("COM1", 115200, Parity.None, 8, StopBits.One); serialPort.Open(); while (true) { int x; x = serialPort.BytesToRead; if (0 == x) { Thread.Sleep(100); } else { byte[] aby = new byte[x]; serialPort.Read(aby, 0, x); Thread.Sleep(100); Debug.Print("Read: " + x + " and got '" + HexStr(aby, 0, x) + "'"); } } } }}
Hi JamesBrown,
If the serial buffer overflows, it should just discard the extra bytes automatically. If NETMF is locking up because of too much incoming serial data, that's a bug. It may be a glitch in NETMF 4.3. If that happens to you in 4.2 too (release firmware), we can expedite a fix for that.
Regarding data corruption... What voltage is your GPS receiver sending data at? Anything from 3.3-5.0V should work fine. Do you have a logic analyzer by any chance, to see what the data looks like there?
BTW, we have applications which use the SerialPort extensively and never see any issues. As an example the Shield Base currently uses the UART transport and runs for days on end.
Please let me know if the buffer overflow lockup occurs on NETMF 4.2 (release firmware) as well or is just limited to NETMF 4.3 (beta firmware). In either case we'll want to fix it.
Thank you, and sorry for the troubles.
Chris
Noom - The GPS shield I'm working with has a ubox module on it and I have it configured to only send UBX protocol rather than NMEA. UBX is byte oriented where there are 2 sync bytes, then 1 class byte then 1 ID byte followed by 2 bytes of length of the payload. Then the payload followed by 2 checksum bytes. I was hoping to watch the data stream for the two sync bytes then collect the length and read the payload in with a serialPort.Read(aby,0,x) type command. It seems like I should be able to do that - right.
Chris - I switched to NETMF 4.2 by simply selecting .NET Micro Framework 4.2 in the application properties. For a test I tried 'Blinky' and now the user interface hangs with:
Adding pe file C:Program FilesSecret LabsNetduino SDKAssembliesv4.2lesecretlabs.netmf.hardware.netduino.pe to deployment bundle
Adding pe file C:Program FilesMicrosoft .NET Micro Frameworkv4.2AssembliesleMicrosoft.SPOT.Hardware.PWM.pe to deployment bundle
Adding pe file C:Program FilesSecret LabsNetduino SDKAssembliesv4.2lesecretlabs.netmf.hardware.pe to deployment bundle
Adding pe file C:Program FilesMicrosoft .NET Micro Frameworkv4.2AssembliesleMicrosoft.SPOT.Net.pe to deployment bundle
Do I have to burn the Netduino back to 4.2 as well?
Has anyone managed to send bytes to the Netduino UART with success?
...
I can attest that I have (and am currently) using the serial port successfully. I am interfacing with a GSM modem at 19.2kpbs N81, and have run it for days without incident, passing data back and forth as part of my testing. (mind you, /not/ using the DataReceived event, but I've mentioned that before)
Here's a thing I that has made my development life easier:
* get one of those FTDI usb serial dodads. you can get it on ebay for about $10. Then you can plug it directly into your shield, and do your debugging on the desktop (since it's dotNet you can), which I find faster going development-wise. You can prove your code here, separate from any netduinic aspects. Its also a great sanity check.
* second, when you do run it on the netduino, use that same FTDI adapter to monitor the communication. I plug the RX lead into whichever (RX or TX) side I'm interested in and monitor the communciaitons from Putty. Actually, this is how I diagnosed my 'event' problem -- I could see the modem was sending what it should, but but my debug log showed the reception abruptly terminating from the netduino's viewpoint. Once I changed from an event to a workerthread design (which happily wound up being trivial because the rest of the code worked the same way), all was right in the world.
Also, and it's probably not the case for you, but I'll mention it anyway:
* sanity check on your serial params. I have made this mistake before, and I get garbage data.
* also polarity of the serial data. I think you're using a 'shield' so thats probably not your problem, however I recently interfaced with a Motorola DS457 barcode scanner and guess what? the serial data is inverted! ha! so I needed to invert it to get sensible data.
Hope that helps, don't give up, it's something silly. Serial is very very fundamental (I know you know this already).
I took Dave's suggestion and hooked up a FTDI to the output of the shield and did find that I had two messages flowing, not one as I thought. Shutting off the second on and the number of bytes settled down at any baud rate I choose.
The second problem is still with me. I cannot deploy worth a hill of beans.
Once I get the Main() running I simply cannot get it to stop so that I can re-deploy a new compile. No amount of on Netduino button pushing or USB unplugging will get the job done. I various errors like:
Waiting for device to initialize...
The debugging target and the debugger engine failed to initialize because of unspecified device errors.
The debugger engine thread has terminated unexpectedly with error 'Object reference not set to an instance of an object.'.
or:
Attaching debugger engine...
... cannot attach debugger engine!
Its as if once in a loop waiting for bytes it cannot be interrupted even with the reset button.
It seems like there are at least two problems. The first is that there is no indication when a buffer overflow happens and the second is that even the reset button will not clear it. That has to be a firmware problem.
It seems like there are at least two problems. The first is that there is no indication when a buffer overflow happens and the second is that even the reset button will not clear it. That has to be a firmware problem.
are you still spinning in a tight loop, or did you put in some Sleep()s?
I have accidentally hung my board in a tight loop before and had to flash just to erase the program, since once it started, the board became unreachable to the MFDeploy protocol. So much so that I made a DFU that simply erases the deployment (your program) but not the firmware. Another trick that at least seemed to work for me was to put visual studio in the 'run program mode' with the board disconnected. Then it will wait a moment trying to connect via MFDeploy. Then you plug the board in, and with luck the debug connection will get made before your program locks up the board.
I just realized you were using some GPS module and remembered that Lo and Behold I have a GPS module I bought years ago from SparkFun laying around that I had never used (GPS-08936; pretty spendy too as I recall). So I was curious and made this quicky:
using System;using System.IO.Ports;using System.Net;using System.Net.Sockets;using System.Threading;using Microsoft.SPOT;using Microsoft.SPOT.Hardware;using SecretLabs.NETMF.Hardware;using SecretLabs.NETMF.Hardware.Netduino;namespace NP2GPSTest{ public class Program { SerialPort _comm; byte[] _abyBuff = new byte[128]; //our own COM buffer int _nIdxBuff = 0; //how much is valid int _nCommPollSleep = 100; //polling period public static void Main() { Program pgm = new Program(); } //'objectified' main Program() { //make our com port _comm = new SerialPort("COM1", 4800, Parity.None, 8, StopBits.One); _comm.Handshake = Handshake.None; //YYY maybe _comm.Open(); if (_comm.IsOpen) //well, is it? { } //monitor the com port for data, and dispatch it ReadCOMAndDispatch(); //you could wrap this in a thread if you wanted to //done with com port _comm.Close(); _comm.Dispose(); } //ReadCOMAndDispatch // this is a loop looking for COM data. It's similar to what I would //do in a DataReceived event handler, but I have found those to be //unreliable on the netduino (or probably netmf itself is more to blame). //If it /were/ in a DataReceived event, the while loop, and the sleep //would go away, leaving the stuff in the first 'else' clause. void ReadCOMAndDispatch() { bool bQuit = false; //this could be a member var, that some other thread could set to 'true' to make this loop end while (!bQuit) { int nReadable; if (null == _comm || !_comm.IsOpen || 0 == (nReadable = _comm.BytesToRead)) { Thread.Sleep(_nCommPollSleep); } else { //gots some datas! stick them it in our buffer, or throw it away -- either way we must dispense with it lock (_abyBuff) { int nRoom = _abyBuff.Length - _nIdxBuff; if (nReadable > nRoom) //insuf buff, lose some stuff { int nLoss = nReadable - nRoom;// Debug.Print("losing " + nLoss); //some is lost from the buffer... int nSatLoss = nLoss; //can't lose more than you have, unlike other things in life, alas if (nSatLoss > _nIdxBuff) nSatLoss = _nIdxBuff; if (_nIdxBuff != nSatLoss) Array.Copy(_abyBuff, nLoss, _abyBuff, 0, _nIdxBuff - nSatLoss); _nIdxBuff -= nSatLoss; //reflect the space we liberated in our buffer nRoom += nSatLoss; //...and some may still have to be lost from the line int nEat = nLoss - nSatLoss; int nEatRem = nEat; while (0 != nEatRem) { int nToEat = System.Math.Min(_abyBuff.Length, nEatRem); int nAte = _comm.Read(_abyBuff, 0, nToEat); nEatRem -= nToEat; } nReadable -= nEat; //reflect what we discarded from the com port } //OK, now we can stick some stuff in our buff int nRead = _comm.Read(_abyBuff, _nIdxBuff, nReadable);// Debug.Print("reading " + nReadable); _nIdxBuff += nRead; } //now, parse and dispatch the data ParseAndDispatch(); } } } //ParseAndDispatch // Hunt for text lines, pull them out, and pass them off. void ParseAndDispatch() { bool bParsing = true; while (bParsing) { //OK, this particular GPS returns NMEA text lines string strLine = null; //find text lines, which are terminated with 0x0d 0x0a //skip any leading CRLF that might be laying around int nIdxStart = 0; while (nIdxStart < _nIdxBuff) { if (0x0d == _abyBuff[nIdxStart]) { } //skip else if (0x0a == _abyBuff[nIdxStart]) { } //skip this one too else { //this is the start of a new line break; } ++nIdxStart; } //now, hunt for the CR int nIdxCR = Array.IndexOf(_abyBuff, (byte)0x0d, nIdxStart, _nIdxBuff - nIdxStart); //(yes you MUST have the (byte) before the 0x0d) if (nIdxCR < nIdxStart) { bParsing = false; } //not found, carry on building up lines else { //found a CR! consider this a line, unless it's empty if (nIdxCR > nIdxStart) { try { strLine = new String(System.Text.Encoding.UTF8.GetChars(_abyBuff, nIdxStart, nIdxCR - nIdxStart)); } catch (Exception) { strLine = "OMG trash?"; } } //now ditch this stuff in the buff int nKeep = _nIdxBuff - nIdxCR; //what we will keep (at the end) if (0 != nKeep) { Array.Copy(_abyBuff, _nIdxBuff - nKeep, _abyBuff, 0, nKeep); } _nIdxBuff = nKeep; if (null == strLine) //no lines? { bParsing = false; } //carry on building up lines else { Dispatch(strLine); //pass on to the dispatcher } } } } //Dispatch // Now we have a text line (ostensibly an NMEA line). Interpret it. void Dispatch(string strLine) { //OK we have an NMEA text line, let us parse it Debug.Print("Found an NMEA line! " + strLine); //XXX todo //inspect the line; I would probably use RegEx's to classify and parse the //fields into capture groups, but that's me -- because I hatehatehate //manual parsing. } }}
when run it will spew out the NMEA lines, like this:
'Microsoft.SPOT.Debugger.CorDebug.dll' (Managed): Loaded 'C:ExperimentsNP2GPSTestbinDebugleNP2GPSTest.exe', Symbols loaded.The thread '<No Name>' (0x2) has exited with code 0 (0x0).Found an NMEA line! $GPGGA,235948.058,,,,,0,00,,,M,0.0,M,,0000*5AFound an NMEA line! $GPGSA,A,1,,,,,,,,,,,,,,,*1EFound an NMEA line! $GPRMC,235948.058,V,,,,,,,310109,,,N*4BFound an NMEA line! $GPVTG,,T,,M,,N,,K,N*2CFound an NMEA line! $GPGGA,235949.058,,,,,0,00,,,M,0.0,M,,0000*5BFound an NMEA line! $GPGSA,A,1,,,,,,,,,,,,,,,*1EFound an NMEA line! $GPRMC,235949.058,V,,,,,,,310109,,,N*4AFound an NMEA line! $GPVTG,,T,,M,,N,,K,N*2CFound an NMEA line! $GPGGA,235950.058,,,,,0,00,,,M,0.0,M,,0000*53Found an NMEA line! $GPGSA,A,1,,,,,,,,,,,,,,,*1EFound an NMEA line! $GPGSV,1,1,00*79Found an NMEA line! $GPRMC,235950.058,V,,,,,,,310109,,,N*42Found an NMEA line! $GPVTG,,T,,M,,N,,K,N*2CFound an NMEA line! $GPGGA,235951.060,,,,,0,00,,,M,0.0,M,,0000*59Found an NMEA line! $GPGSA,A,1,,,,,,,,,,,,,,,*1E
I have no idea what that means; hopefully I haven't given away the location of my secret lair (I don't think I did, it doesn't get a lock in here). You'll have to figure out that. But anyway, it generated some 3000 lines in the time it took my to write this post.
