Netduino home hardware projects downloads community

Jump to content


Photo

Serial Data Transfer Protocol problem


  • Please log in to reply
12 replies to this topic

#1 Bendage

Bendage

    Advanced Member

  • Members
  • PipPipPip
  • 153 posts
  • LocationIrvine, CA

Posted 13 April 2012 - 08:00 PM

I have been working on a universal serial protocol for quite some time now between the PC and Netduino that allows me to create plugin libraries that snap into a console app so I can dynamically plug in different components to work on the fly and not have to create from scratch every time.

So the console works wonderful and I can load dll's onto the SD card and send messages to the controller to load the dlls depending on what library I have made for each component I want to work with. Each controller library communicates in sync with a dynamic Windows user control to exchange events and values through the protocol and as a developer I no longer have to worry about the communication part and now can just create controls and snap them in on the fly depending on what I want the Netduino to control. So in theory I can control the Netduino from a windows app any way I want or control the windows app from the Netduino any way I want without knowing anything about the message layer.

Here is the problem. I got a bit cocky in that I got tired of compiling a Micro Framework dll and pulling the sd card loading the dll and resetting the netduino. So I decided to add streaming into the serial protocol that will send the dll file through serial and auto save to sd card and then load.

The main problem is since this is a protocol that has many message types once entering into a stream mode, since any byte in the file can represent 0 to 255 I no longer have a message terminator to let me know when I have sent the last byte of the file.

How to I tell my data received event I am done sending any byte 0 - 255? What kind of message End of Transmission can I use if binary data uses all of them?

I tried looking at the way the FTP protocol works to see if that would help but FTP sends a binary stream on a separate port from the one it actually exchanges messages. I do not want to use two different com ports with FTDI connections. There has got to be a better way.

Posted Image

#2 Bendage

Bendage

    Advanced Member

  • Members
  • PipPipPip
  • 153 posts
  • LocationIrvine, CA

Posted 13 April 2012 - 08:58 PM

I swear, every time I post something I think of the answer right after I post... almost every time. BASE64! Is there a helper routine around here? I would need to get a Base 64 string back into the original byte[]

#3 Bendage

Bendage

    Advanced Member

  • Members
  • PipPipPip
  • 153 posts
  • LocationIrvine, CA

Posted 13 April 2012 - 09:25 PM

What do you know... Base 64 is actually included in Micro Framework. Well I'll be. Works like a charm.

#4 Geancarlo2

Geancarlo2

    Advanced Member

  • Members
  • PipPipPip
  • 70 posts

Posted 13 April 2012 - 09:30 PM

In my opinion a robust protocol should have some king of checksum/acknowledgment and send the length of the message. say something like LL LL CC CC CC CC TT PL PL PP PP PP PP PP...... LL LL=total packet length(2 bytes) CC CC CC CC=checksum over TT PL PL PP PP PP..(4 bytes) TT=packet type(1 byte) PL PL=payload length(2 bytes) PP PP PP ...=payload(length equal to the value of PL PL) This is the protocol implemented by a game I used to play and its pretty simple to implement. Pardon me if I misunderstood your first question:D

#5 Bendage

Bendage

    Advanced Member

  • Members
  • PipPipPip
  • 153 posts
  • LocationIrvine, CA

Posted 13 April 2012 - 09:39 PM

The problem is that I can't get a buffer to not run out of memory over about 150 bytes. My messages are longer than that... or can be so I have to use a terminator to let the protocol engine know to go ahead and process that message. Otherwise I would say yes... checksum is the way to go. But hey I got it working with Base 64. This console app is a time saver when I want to give netduino a full dynamic interface.

#6 Geancarlo2

Geancarlo2

    Advanced Member

  • Members
  • PipPipPip
  • 70 posts

Posted 13 April 2012 - 10:52 PM

If you got it figured already, its cool then :D. Using a protocol like I mentioned you can also break data in pieces as small as you want, it's just a matter of creating appropriate headers that can specify such things.

#7 Mario Vernari

Mario Vernari

    Advanced Member

  • Members
  • PipPipPip
  • 1720 posts
  • LocationVenezia, Italia

Posted 14 April 2012 - 04:58 AM

Pardon me, guys, but...why reinventing the wheel?
The Modbus protocol has been designed exactly for purposes like those.
Set this, read that...it's ridicolously simple, yet well standardized.
Have a look here:
http://highfieldtale...y-for-netduino/

Moreover, using such as protocol, there's literally plenty of 3rd party devices of *ANY* kind which can be connected with *no* effort.

Here is a "dummy-like" explanation of the Modbus:
http://www.simplymodbus.ca/FAQ.htm

Limitation of the Modbus itself:
  • any message has to be no longer than 256 bytes;
  • there's no any (standard) packetization for long-streams;
  • no encryption;
  • many more...
Although the above, it's maybe the most used in the world, after HTTP.

Hope it helps
Cheers
Biggest fault of Netduino? It runs by electricity.

#8 Bendage

Bendage

    Advanced Member

  • Members
  • PipPipPip
  • 153 posts
  • LocationIrvine, CA

Posted 14 April 2012 - 05:28 PM

Mario, Thanks for the links. I had a chance to glance at the Object diagrams for Modbus. Funny but I've never heard of it before. Though in many cases the command bus is very similar to mine, Just as the Netduino Go can hide much of the low level work done in making things happen with plugin components, my protocol is wrapped by a command/response interface that hides the low level byte communication. Write a windows control inherited by a base control that implements the console interface, write a plugin class that implements the Netduino controller interface and the engine does all the work for you. So, its not necessarily reinventing the wheel. You be the judge once I finish it. I will post it under the projects forum when I'm done.

#9 Mario Vernari

Mario Vernari

    Advanced Member

  • Members
  • PipPipPip
  • 1720 posts
  • LocationVenezia, Italia

Posted 14 April 2012 - 05:35 PM

Thank you very much for the invitation of judging. You know, I like much more bumping toward hardware problems, than software ones. However, your framework looks nice, so I'll wait for your presentation. Good luck. Cheers
Biggest fault of Netduino? It runs by electricity.

#10 Paul Newton

Paul Newton

    Advanced Member

  • Members
  • PipPipPip
  • 710 posts
  • LocationBerkshire, UK

Posted 14 April 2012 - 08:29 PM

"universal serial protocol" - that must be what they use on Star Trek to set up comms with alien races the Enterprise computers have never encountered before. I always wondered how they did that. Commander Bendage to the bridge!

#11 Bendage

Bendage

    Advanced Member

  • Members
  • PipPipPip
  • 153 posts
  • LocationIrvine, CA

Posted 19 April 2012 - 12:17 AM

LOL! Paul you crack me up

#12 Bendage

Bendage

    Advanced Member

  • Members
  • PipPipPip
  • 153 posts
  • LocationIrvine, CA

Posted 29 September 2012 - 05:00 AM

"universal serial protocol" - that must be what they use on Star Trek to set up comms with alien races the Enterprise computers have never encountered before.

I always wondered how they did that. Commander Bendage to the bridge!



Hey Paul. Just wanted to let you know it's done. Take a look and tell me what you think.

http://forums.netdui...6233#entry36233

#13 Paul Newton

Paul Newton

    Advanced Member

  • Members
  • PipPipPip
  • 710 posts
  • LocationBerkshire, UK

Posted 30 September 2012 - 03:46 PM

Nah - Sorry Bendage, its not the same as Star Trek with screw heads in the corners of the GUI widgets. :P Close though! Paul




0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users

home    hardware    projects    downloads    community    where to buy    contact Copyright © 2010-2014 Secret Labs LLC  |  Legal   |   CC BY-SA
This webpage is licensed under a Creative Commons Attribution-ShareAlike License.