Serial Data Transfer Protocol problem

I have been working on a universal serial protocol for quite some time now between the PC and Netduino that allows me to create plugin libraries that snap into a console app so I can dynamically plug in different components to work on the fly and not have to create from scratch every time.

So the console works wonderful and I can load dll's onto the SD card and send messages to the controller to load the dlls depending on what library I have made for each component I want to work with. Each controller library communicates in sync with a dynamic Windows user control to exchange events and values through the protocol and as a developer I no longer have to worry about the communication part and now can just create controls and snap them in on the fly depending on what I want the Netduino to control. So in theory I can control the Netduino from a windows app any way I want or control the windows app from the Netduino any way I want without knowing anything about the message layer.

Here is the problem. I got a bit cocky in that I got tired of compiling a Micro Framework dll and pulling the sd card loading the dll and resetting the netduino. So I decided to add streaming into the serial protocol that will send the dll file through serial and auto save to sd card and then load.

The main problem is since this is a protocol that has many message types once entering into a stream mode, since any byte in the file can represent 0 to 255 I no longer have a message terminator to let me know when I have sent the last byte of the file.

How to I tell my data received event I am done sending any byte 0 - 255? What kind of message End of Transmission can I use if binary data uses all of them?

I tried looking at the way the FTP protocol works to see if that would help but FTP sends a binary stream on a separate port from the one it actually exchanges messages. I do not want to use two different com ports with FTDI connections. There has got to be a better way.

I swear, every time I post something I think of the answer right after I post... almost every time. BASE64! Is there a helper routine around here? I would need to get a Base 64 string back into the original byte[]

What do you know... Base 64 is actually included in Micro Framework. Well I'll be. Works like a charm.

In my opinion a robust protocol should have some king of checksum/acknowledgment and send the length of the message. say something like LL LL CC CC CC CC TT PL PL PP PP PP PP PP...... LL LL=total packet length(2 bytes) CC CC CC CC=checksum over TT PL PL PP PP PP..(4 bytes) TT=packet type(1 byte) PL PL=payload length(2 bytes) PP PP PP ...=payload(length equal to the value of PL PL) This is the protocol implemented by a game I used to play and its pretty simple to implement. Pardon me if I misunderstood your first question:D

The problem is that I can't get a buffer to not run out of memory over about 150 bytes. My messages are longer than that... or can be so I have to use a terminator to let the protocol engine know to go ahead and process that message. Otherwise I would say yes... checksum is the way to go. But hey I got it working with Base 64. This console app is a time saver when I want to give netduino a full dynamic interface.

If you got it figured already, its cool then . Using a protocol like I mentioned you can also break data in pieces as small as you want, it's just a matter of creating appropriate headers that can specify such things.

Pardon me, guys, but...why reinventing the wheel?
The Modbus protocol has been designed exactly for purposes like those.
Set this, read that...it's ridicolously simple, yet well standardized.
Have a look here:
http://highfieldtale...y-for-netduino/

Moreover, using such as protocol, there's literally plenty of 3rd party devices of *ANY* kind which can be connected with *no* effort.

Here is a "dummy-like" explanation of the Modbus:
http://www.simplymodbus.ca/FAQ.htm

Limitation of the Modbus itself:

• any message has to be no longer than 256 bytes;
• there's no any (standard) packetization for long-streams;
• no encryption;
• many more...

Although the above, it's maybe the most used in the world, after HTTP.

Hope it helps
Cheers

Mario, Thanks for the links. I had a chance to glance at the Object diagrams for Modbus. Funny but I've never heard of it before. Though in many cases the command bus is very similar to mine, Just as the Netduino Go can hide much of the low level work done in making things happen with plugin components, my protocol is wrapped by a command/response interface that hides the low level byte communication. Write a windows control inherited by a base control that implements the console interface, write a plugin class that implements the Netduino controller interface and the engine does all the work for you. So, its not necessarily reinventing the wheel. You be the judge once I finish it. I will post it under the projects forum when I'm done.

Thank you very much for the invitation of judging. You know, I like much more bumping toward hardware problems, than software ones. However, your framework looks nice, so I'll wait for your presentation. Good luck. Cheers

"universal serial protocol" - that must be what they use on Star Trek to set up comms with alien races the Enterprise computers have never encountered before. I always wondered how they did that. Commander Bendage to the bridge!

LOL! Paul you crack me up

"universal serial protocol" - that must be what they use on Star Trek to set up comms with alien races the Enterprise computers have never encountered before.

I always wondered how they did that. Commander Bendage to the bridge!

Hey Paul. Just wanted to let you know it's done. Take a look and tell me what you think.

http://forums.netdui...6233#entry36233

Nah - Sorry Bendage, its not the same as Star Trek with screw heads in the corners of the GUI widgets. Close though! Paul