Reading data chunks from a COM port question
#1
Posted 08 September 2011 - 01:58 PM
#2
Posted 08 September 2011 - 02:57 PM
Hi everybody
I've got a question about Serial port communication using the netduino. This is the first time I've built something like this - my first week with a Netduino - but I hope this isn't a dumb question!!!.
Where I've got to so far:
I'm using a barebone FTDI USB to Serial mini adapter - I open COM1 at 115200, and repeatedly send a byte array (each packet being 16 bytes long) from my netduino.
I also have a C# client application which reads from the com port at 115200, and successfully sees the stream of bytes that I'm sending from the netduino. So far so good.
Where I'm hitting the problem:
The client application isn't able to determine where my packets of 16 bytes start and end. I mean if I tell the my client app to read 16 bytes from the port, I find that it's possible it'll pick up the last 7 bytes of one packet and then the first 9 bytes of the second.
I think this is probably expected behaviour.
My current workaround is a bit of code that looks for a certain byte pattern in what comes in from COM1 - when it finds that, then read the next 16 bytes of data and assume it's a valid packet of my data - and I just repeat that.
But do you guys know if there's a more elegant or robust solution to my problem, or if there's a protocol I should look into using to tokenise my data stream?
Thanks in advance for any help!
Jeremy
One general solution to this kind of problem is using a Self-Synchronizing code, another one is using markers to delimit token boundaries. How I would design the protocol specifically is dependant on the exact setting. If there's a byte (sequence) that will never appear in the datastream, you can use it between every N tokens (low N -> fast synchronization, high overhead, high N -> slower synchronization, lower overhead) to mark a boundary.
-- H.L. Mencken, "What I Believe"
#3
Posted 08 September 2011 - 03:06 PM
Hi everybody
I've got a question about Serial port communication using the netduino. This is the first time I've built something like this - my first week with a Netduino - but I hope this isn't a dumb question!!!.
Where I've got to so far:
I'm using a barebone FTDI USB to Serial mini adapter - I open COM1 at 115200, and repeatedly send a byte array (each packet being 16 bytes long) from my netduino.
I also have a C# client application which reads from the com port at 115200, and successfully sees the stream of bytes that I'm sending from the netduino. So far so good.
Where I'm hitting the problem:
The client application isn't able to determine where my packets of 16 bytes start and end. I mean if I tell the my client app to read 16 bytes from the port, I find that it's possible it'll pick up the last 7 bytes of one packet and then the first 9 bytes of the second.
I think this is probably expected behaviour.
My current workaround is a bit of code that looks for a certain byte pattern in what comes in from COM1 - when it finds that, then read the next 16 bytes of data and assume it's a valid packet of my data - and I just repeat that.
But do you guys know if there's a more elegant or robust solution to my problem, or if there's a protocol I should look into using to tokenise my data stream?
Thanks in advance for any help!
Jeremy
Probably one of the easiest things to do is put a single byte at the start of your byte array that would otherwise be invalid data, say byte 0 (null) then you can look for that byte and read the next N number of bytes for a completed message.
I have some code for the desktop client from a my personal stock that you can use, I need to clean it up; I will do that after work and post it for you.
#4
Posted 08 September 2011 - 03:26 PM
will repost after i fix some things in it.
#5
Posted 08 September 2011 - 10:14 PM
#6
Posted 09 September 2011 - 12:47 AM
using System; using System.Collections; using System.Collections.Generic; using System.Diagnostics; using System.Linq; public class DataHolder :IDisposable { #region Private Variables private Queue<byte[]> _completedMessages = new Queue<byte[]>(); private object _syncObject = new object(); private System.IO.MemoryStream _pendingMessages = new System.IO.MemoryStream(); private byte _headerByte = Convert.ToByte(0); #endregion #region Properties public object SyncRoot { get { return _syncObject; } } public Queue<byte[]> CompletedMessages { get { return _completedMessages; } } #endregion #region Constructors public DataHolder(byte headerByte) { _headerByte = headerByte; } #endregion #region Public Methods public bool AddData(byte[] data) { bool messageWasCompleted = false; // The synclock ensures only one thread at a time can modify the queue lock (_syncObject) { _pendingMessages.Write(data, 0, data.Length); // This takes all of the data from the memory stream and puts it in an array byte[] cachedData = _pendingMessages.ToArray(); //Uh we have no data?? then we cannot continue, so exit if (cachedData == null) return false; //Trim the start of the array to the start of the first message int indexOfStartofMessage = Array.IndexOf<byte>(cachedData, _headerByte); if (indexOfStartofMessage > 0) cachedData = cachedData.Skip(indexOfStartofMessage).ToArray(); //If there are at least 4 bytes then we know how long the first message is supposed to be if (cachedData.Length > 4) { // // A couple different ways to check message size // // Check Message size by reading the two bytes after the header byte byte[] lengthBytes = { cachedData[2], cachedData[3] }; int length = BitConverter.ToInt16(lengthBytes, 0); // Use a fixed message size //int length = 17; (16 byte message + 1byte header) //We have a complete message if (cachedData.Length >= length) { byte[] message = new byte[length]; Array.Copy(cachedData, message, length); // Copy the bytes into the message array _completedMessages.Enqueue(message); // Add the new byte array to our list of completed messages byte[] leftOverData = new byte[cachedData.Length - length]; //Copy the left over data into the new array Array.Copy(cachedData, length, leftOverData, 0, cachedData.Length - length); //Reset the memory stream _pendingMessages.SetLength(0); _pendingMessages.Position = 0; //Write the left over data back out _pendingMessages.Write(leftOverData, 0, leftOverData.Length); messageWasCompleted = true; } } } return messageWasCompleted; } public void Dispose() { if (_pendingMessages != null) { _pendingMessages.Close(); _pendingMessages.Dispose(); } } #endregion }
using System; using System.Collections.Generic; using System.Linq; using System.Text; public class Serial { #region Private Variables //The Convert.ToByte(0) assumes you are using the NULL byte, byte 0 as the indicator to the start of your packet DataHolder _messageHolder = new DataHolder(Convert.ToByte(0)); System.IO.Ports.SerialPort _serial; //Processing Threads System.Threading.Thread _processingThread; bool _stop = false; #endregion #region Constructor public Serial(string comPort, int baudRate) { //Setup the processing threads _processingThread = new System.Threading.Thread(ProcessSerialData); _processingThread.IsBackground = true; _processingThread.Start(); _serial = new System.IO.Ports.SerialPort(comPort, baudRate); _serial.Open(); } #endregion #region Public Methods public void Stop() { _stop = true; if (_processingThread != null && _processingThread.IsAlive) _processingThread.Join(); if (_serial != null) { if (_serial.IsOpen) _serial.Close(); _serial.Dispose(); } } #endregion #region Private Methods private void ProcessSerialData() { while (!_stop) { //TODO: You can process the message here or below System.Threading.Thread.Sleep(100); } } private void Serial_DataRecieved(object sender, System.IO.Ports.SerialDataReceivedEventArgs e) { using (System.IO.MemoryStream ms = new System.IO.MemoryStream) { while (_serial.BytesToRead > 0) { byte[] data = new byte[1024]; int bytesRead = _serial.Read(data,0,1024); ms.Write(data,0,bytesRead); } if (_messageHolder.AddData(ms.ToArray()) == true) { lock(_messageHolder.SyncRoot) { //TODO: You can process the message here // If you process it in the seperate processing thread you can commed out the lock part } } } } #endregion }
#7
Posted 09 September 2011 - 04:22 AM
Absolutely, and it's because the PC management. Your Netduino is not guilty!The client application isn't able to determine where my packets of 16 bytes start and end. I mean if I tell the my client app to read 16 bytes from the port, I find that it's possible it'll pick up the last 7 bytes of one packet and then the first 9 bytes of the second.
I think this is probably expected behaviour.
I can't rely on timings.
That's also normal. You are adding the missing "OSI layer" to the protocol.My current workaround is a bit of code that looks for a certain byte pattern in what comes in from COM1 - when it finds that, then read the next 16 bytes of data and assume it's a valid packet of my data - and I just repeat that.
There are plenty of UART-specific protocols, the most used are XON/XOFF, and they rely on a certain pattern for the start and for the termination.
Always we need to define what "elegant" means.But do you guys know if there's a more elegant or robust solution to my problem, or if there's a protocol I should look into using to tokenise my data stream?
Generally, such a protocol could be composed as follows:
- start marker: often it uses the SOH character (0x01), that has historically just this function;
- address: (optional) if your UART is connected to many slaves, and you need to address one of them;
- length: the remaining count of bytes;
- body: the useful data;
- crc/checksum: a redundancy check to verify the message integrity.
I would suggest to take a look at the Modbus-RTU specs.
Cheers
#8
Posted 11 September 2011 - 11:39 AM
0 user(s) are reading this topic
0 members, 0 guests, 0 anonymous users