Introducing Netduino.IP, the shiny new TCP/IP stack for NETMF - Netduino.IP Technical Preview - Netduino Forums
   
Netduino home hardware projects downloads community

Jump to content


The Netduino forums have been replaced by new forums at community.wildernesslabs.co. This site has been preserved for archival purposes only and the ability to make new accounts or posts has been turned off.
Photo

Introducing Netduino.IP, the shiny new TCP/IP stack for NETMF


  • Please log in to reply
35 replies to this topic

#1 Chris Walker

Chris Walker

    Secret Labs Staff

  • Moderators
  • 7767 posts
  • LocationNew York, NY

Posted 14 January 2015 - 09:15 AM

UPDATE: Netduino.IP is now in production.
* Netduino Plus 2 mainboards -- for firmware, see sticky posts at the top of this forum.
* Netduino 3 mainboards -- Netduino.IP is included in all standard firmware releases.


In the next decade, the billions of people on the Internet will be vastly outnumbered by even more billions of intelligent Internet-connected devices ("things"). A seismic shift in the Internet's digital demographics is underway.

Of course, .NET Micro Framework is poised to play a very important role in this exploding Internet of Things. The processor price difference using NETMF vs. native code is rapidly closing or disappearing entirely. And a whole new class of low-cost, NETMF-capable microcontrollers with integrated wireless connectivity is now arriving on the market.

TCP/IP support today
Over our community's four and a half years together, Netduino has powered a lot of Internet-connected gadgets. Makers have built a ton of cool Internet-connected projects. And commercial entities have derived some pretty great Internet-connected gadgets from Netduino.

But our networking hardware is not running at 100%. While the open source network stack we use (lwIP) is pretty awesome, it can be fickle and it consumes a lot of expensive RAM. Implementing support for new networking chips today can be time consuming and requires a lot of low-level code. And debugging the native lwIP stack can be, in a word, intense; while we do love lwIP, it also has quite a few...let's call them bugs.

Beyond that, the future of the Internet is IPv6-based: Zigbee, Bluetooth 4.2, Wifi and Ethernet all support IPv6. We need a future-ready networking stack, optimized for NETMF.

Introducing Netduino.IP
So we've created a brand new open-source TCP/IP stack for NETMF; it is called Netduino.IP.

The entire Netduino.IP stack, from link layer interface to the networking protocols, is written in C#. It works with the standard System.Net classes (Socket, HttpWebRequest, etc.) for backwards compatibility, so existing NETMF projects should "just work".

Many commercial IoT projects are cost sensitive. And we want to move NETMF into even lower-cost microcontrollers while retaining its awesome networking capabilities. So we designed Netduino.IP to use as little as 4KB of RAM (vs 10s of KBs for lwIP). Plus the new stack only consumes RAM when networking is used; garbage collection is minimized by design.

The new Netduino.IP stack is designed for modularity and for use across many communication mediums and many different networking chips. One of our main design goals was to make sure that Netduino.IP enabled a full network stack with lower-level chips like the ENC28J60 used on Netduino Plus 2...while also supporting a more minimal stack with higher-level chips like the Wiznet chips used on Ethernet shields. And of course we want to support IP over not only Ethernet but also over Wi-Fi and Bluetooth and any other arbitrary medium.

Preview program
Netduino.IP is shiny and new, and it contains many thousands of lines of new code. There are a lot of layers to this protocol stack. And regardless of how much testing we do internally, we simply cannot anticipate all of the issues that might crop up in the wild.

Next week we will be releasing the first Technical Preview build of Netduino.IP for Netduino Plus 2. While we have tested HTTP requests, DHCP, DNS resolution and more with Netduino.IP, the preview program will be progressive. This means that we will be testing one piece of the stack at a time.

We will start testing with network adapter configuration and static IP. Then we will add ARP. And then DHCP. And UDP. And DNS. And ICMP. And TCP. There is a lot of ground to cover, and we want to make sure we can focus in and test every part of the stack individually.

The preview program is estimated to last twelve weeks, with a new build released on average every two weeks--incorporating feedback and bugfixes from the previous build. We may also post interim builds with smaller bugfixes.

Today Netduino.IP is an IPv4 stack. During the technical preview program, we will start to add IPv6 support. I would really appreciate any/all testing of the IPv6 features as they are implemented. [If your ISP doesn't support native IPv6, you can get a free IPv6 tunnel for many routers, etc. through Hurricane Electric.]

Once the technical preview program is complete, we will publish the full-featured Netduino.IP stack as part of Netduino 4.3.2 beta firmware. If IPv6 implementation/testing is slower, we may continue its test program beyond the IPv4 technical preview program and release completed IPv6 support as part of official firmware later in the year.

Call for testers
If you're interested in taking the new networking stack for a spin, please follow this forum ("Follow this forum" button in upper-right corner). If you have a few spare cycles, please help us prove out the new stack by testing it on your network. I really, really appreciate your time and expertise perfecting and polishing this new networking stack.

During the technical preview period, please post your feedback and experiences in this forum. I'll personally be actively reading and responding to Netduino.IP posts to help ensure that we deliver a first-class networking stack, designed from the ground up for NETMF.

Chris

P.S. Netduino.IP can optionally be deployed at runtime rather than being embedded in the base firmware. There are a lot more cool features which we'll highlight throughout the preview program.

P.P.S. I also want to give a shout-out to community member Valkyrie-MT. He has done a bunch of work with networking on NETMF with his very cool mIP stack. In many ways, you got us on this path...so please accept my gratitude for all your work!
  • KodeDaemon , EricMeyer , beastyboy and 1 other like this

#2 Cuno

Cuno

    Advanced Member

  • Members
  • PipPipPip
  • 144 posts
  • LocationZürich / Switzerland

Posted 14 January 2015 - 01:10 PM

while we do love lwIP, it also has quite a few...let's call them bugs.

The problem has never been with lwIP. The problem was a bad architecture in NETMF for integrating lwIP. The new Microsoft NETMF team is aware of that and working towards a fundamentally sound new integration architecture. This will allow lwIP to work just fine, at native code speed.



#3 Chris Walker

Chris Walker

    Secret Labs Staff

  • Moderators
  • 7767 posts
  • LocationNew York, NY

Posted 14 January 2015 - 06:48 PM

Hi Cuno,

We still love lwIP. But we're looking forward to something even better, more flexible, and designed from the ground-up for NETMF.

The largest category of user issues and bugs we run into are networking-related, and it's going to be awesome to be able to diagnose those issues easily in managed code.

And, like lwIP, Netduino.IP is open source. It's designed to be used with all .NET MF boards and all .NET MF-based end products. Using a lot less memory, with greater stability, and more ready for the Internet's future.

Chris

P.S. The lwIP community has been fixing quite a few of the bugs, but merging those back into NETMF can be a big project (as you alluded).

#4 Cuno

Cuno

    Advanced Member

  • Members
  • PipPipPip
  • 144 posts
  • LocationZürich / Switzerland

Posted 14 January 2015 - 09:26 PM

The lwIP community has been fixing quite a few of the bugs, but merging those back into NETMF can be a big project (as you alluded).

Microsoft confirmed that the various networking issues in NETMF come from the NETMF design, not from lwIP. They intend to tackle the problem at the core rather than symptomatically - by changing the way in which NETMF integrates lwIP. They verified that a correctly used and configured lwIP stack has none of the known NETMF networking issues.



#5 iced98lx

iced98lx

    Advanced Member

  • Members
  • PipPipPip
  • 134 posts
  • LocationSouth Dakota

Posted 14 January 2015 - 04:23 PM

Chris- This is excellent to hear. Some of the flaky nature of the current solution has held me back from making my project more network intensive, as a failure that locks my board up could mean thousands of dollars of lost livestock. Easing memory usage while providing a stable experience to build on is crucial. It's fun to speculate about the neat things that can be built, but the real answer is that until stability is proven it is still only good for tinkering. Many have worked around the current issues and ended up with stable solutions, I look forward to a time when bugs can be more proactively fixed and I don't have to be a IwIP expert to know how to work around issues I experience. 

 

Again, back to the main upside of .NetMF and boards like Netduino + 2- Abstraction and easy interactions in familiar language with predicable stable results makes for fast prototyping and less complex, easier to debug and maintain code solutions to existing problems.

 

I applaud the effort. 

 

(I have other, distinct feelings on just exactly when my watch will arrive in the mail, if you're keeping track)



#6 Mario Vernari

Mario Vernari

    Advanced Member

  • Members
  • PipPipPip
  • 1768 posts
  • LocationVenezia, Italia

Posted 15 January 2015 - 06:41 AM

I may put my modest contribution to the project, although I'd have a couple of questions...

  1. coding the stack as managed code, would offer a decent performance yet reliability? From the experience I've collected with N+/N+2, the low-level layer are the most critical section, and coding them as managed unfortunately yields to a mediocre result.
  2. is there anything "interesting" at the horizon, like...for instance...ability to compile a C# app natively?

Chris, I know that you won't disclose anything, even under heavy torture...but could you justify a bit more this decision of porting the stack on C# (apart the bugs)?

Cheers


Biggest fault of Netduino? It runs by electricity.

#7 Paolo Patierno

Paolo Patierno

    Advanced Member

  • Members
  • PipPipPip
  • 169 posts
  • LocationItaly

Posted 15 January 2015 - 09:12 AM

Hi Mario,

 

regarding point 2, you can read an interesting discussion here : https://netmf.codepl...m/workitem/2368

 

As we know, the .Net MF team is working on improving TCP/IP stack integration and fixing bug on lwIP.

 

Paolo.


Paolo Patierno

Microsoft MVP on Windows Embedded & IoT

Azure Advisor

Twitter : @ppatierno
Linkedin : paolopatierno
Blog : DevExperience

Blog : Embedded101
?


#8 Chris Walker

Chris Walker

    Secret Labs Staff

  • Moderators
  • 7767 posts
  • LocationNew York, NY

Posted 15 January 2015 - 10:57 PM

Hi Mario,
 

I may put my modest contribution to the project, although I'd have a couple of questions...
...
Chris, I know that you won't disclose anything, even under heavy torture...but could you justify a bit more this decision of porting the stack on C# (apart the bugs)?
Cheers

Netduino.IP is designed for the future of NETMF.

It's also a giant bugfix for the most pressing issues. And it introduces some really exciting new capabilities like IPv6 to NETMF (thanks to the IPv6 code we're contributing from a commercial project). And it enables on-the-fly configuration of networking chips. And...

There are some really interesting things coming in the NETMF world. In meetings with Microsoft last year, Secret Labs agreed to take on the future network stack project to let the NETMF team focus in on the things they do best (i.e. creating the future of the NETMF core). In the short term Microsoft's engineers may also fix some of the lwIP issues for applications that want to use that stack.

As far as performance goes: I think that you will be pleasantly surprised by the performance of Netduino.IP. Some operations are even quicker, today, with Netduino.IP rather than with lwIP. For the vast majority of applications, Netduino.IP will be the best solution.

Chris

#9 JoopC

JoopC

    Advanced Member

  • Members
  • PipPipPip
  • 148 posts

Posted 16 January 2015 - 08:15 AM

I'm more curious by the day, we hope that the most irritating network bug (network up but the internet down) is solved so that we can take advantage of all the settings with the highest speed. Count us in as testers.

#10 Chris Walker

Chris Walker

    Secret Labs Staff

  • Moderators
  • 7767 posts
  • LocationNew York, NY

Posted 16 January 2015 - 10:33 AM

iced -- thanks for the kind words and enthusiasm! We're super-excited about Netduino.IP.

JoopC -- link status updates and network reconnection should be fixed :) Please let me know how it works for you as the technical preview progresses. First progressive build (network adapter configuration and static IP) will be posted next week.

Chris

#11 Chris Walker

Chris Walker

    Secret Labs Staff

  • Moderators
  • 7767 posts
  • LocationNew York, NY

Posted 18 January 2015 - 01:44 AM

For those eager to dig into Netduino.IP source code (or wanting to know more about how it works)...

Article:
Managed code network drivers and memory management

Source code:
ENC28J60 driver source code

Chris

#12 xmen

xmen

    Advanced Member

  • Members
  • PipPipPip
  • 72 posts

Posted 20 January 2015 - 06:17 AM

Sounds pretty interesting. I see many places people/developers dislikes things written in C#. Yes there could be performance difference but doesn't mean C# is not strong and can not get better in future. I would love to have an OS written in C#.

 

I've one question Chris, how will I able to use Wi-Fi using Netduino.IP ?



#13 Chris Walker

Chris Walker

    Secret Labs Staff

  • Moderators
  • 7767 posts
  • LocationNew York, NY

Posted 20 January 2015 - 06:28 AM

Hi xmen,

I've one question Chris, how will I able to use Wi-Fi using Netduino.IP ?


Of course. Netduino.IP is a full and modular TCP/IP stack (and we're adding IPv6 to it), so you could use it over Wifi or Bluetooth 4.2 or 6lowpan, etc.

Chris

#14 xmen

xmen

    Advanced Member

  • Members
  • PipPipPip
  • 72 posts

Posted 20 January 2015 - 08:05 AM

Hi xmen,


Of course. Netduino.IP is a full and modular TCP/IP stack (and we're adding IPv6 to it), so you could use it over Wifi or Bluetooth 4.2 or 6lowpan, etc.

Chris

 

What hardware is require for WiFi connectivity ?



#15 Chris Walker

Chris Walker

    Secret Labs Staff

  • Moderators
  • 7767 posts
  • LocationNew York, NY

Posted 20 January 2015 - 08:22 AM

Hi xmen,

What hardware is require for WiFi connectivity ?

During the technical preview we will be sticking with NP2 hardware; there is a lot of new code in this stack and we want to focus the testing on one known-good set of hardware.

Once the technical preview is complete, we can build a C# driver for a Wi-Fi chip as well. There are quite a few Wi-Fi expansion options out there--and Netduino.IP can configure networking expansion boards on the fly. :)

Chris

#16 xmen

xmen

    Advanced Member

  • Members
  • PipPipPip
  • 72 posts

Posted 22 January 2015 - 08:37 AM

Hi xmen,

During the technical preview we will be sticking with NP2 hardware; there is a lot of new code in this stack and we want to focus the testing on one known-good set of hardware.

Once the technical preview is complete, we can build a C# driver for a Wi-Fi chip as well. There are quite a few Wi-Fi expansion options out there--and Netduino.IP can configure networking expansion boards on the fly. :)

Chris

 

That is just awesome.



#17 ukkiwisurfer

ukkiwisurfer

    Advanced Member

  • Members
  • PipPipPip
  • 32 posts
  • LocationLondon

Posted 24 January 2015 - 09:24 AM

Hi Chris,

 

Are you open to code reviews from the community? Or alternatively do you have a set of coding standards and principles that guides the actual development?

 

Thanks,

Jason.



#18 Chris Walker

Chris Walker

    Secret Labs Staff

  • Moderators
  • 7767 posts
  • LocationNew York, NY

Posted 24 January 2015 - 12:03 PM

Hi Jason,

Are you open to code reviews from the community? Or alternatively do you have a set of coding standards and principles that guides the actual development?

Our goal with the technical preview program is to validate each layer and part of the network stack.

All feedback is welcome, of course. Please submit bug reports and issues directly through GitHub so we make sure we address each one individually.

Chris

#19 Chris Walker

Chris Walker

    Secret Labs Staff

  • Moderators
  • 7767 posts
  • LocationNew York, NY

Posted 25 January 2015 - 09:32 AM

In conjunction with the Netduino.IP Technical Preview, we've put together an update to the MFDeploy tool. The update polishes the Networking Configuration options a bit--and critically it adds in IPv6 configuration.

You can grab a copy of the updated MFDeploy tool (with IPv6 support) here:
Download: New MFDeploy tool with IPv6 support

We've also posted its source code in the Netduino repository.

The new MFDeploy tool also works with existing hardware--and adds the option to automatically obtain DNS addresses via DHCP.

Chris

#20 Chris Walker

Chris Walker

    Secret Labs Staff

  • Moderators
  • 7767 posts
  • LocationNew York, NY

Posted 25 January 2015 - 09:57 AM

The first build of the Netduino.IP technical preview (with source) is now available for download:
Download: Netduino.IP Technical Preview (Build 1)

For this preview build, we're focused on testing out network configuration (using the new MFDeploy tool). If you have some spare cycles, please test out the IPv4 and IPv6 configuration options.

Next up...ARP (address resolution protocol) and link status events. Then DHCP. Then ICMP, UDP, TCP, etc. Lots to test :)

Thanks for your participation and feedback!

Chris
  • beastyboy and xmen like this




0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users

home    hardware    projects    downloads    community    where to buy    contact Copyright © 2016 Wilderness Labs Inc.  |  Legal   |   CC BY-SA
This webpage is licensed under a Creative Commons Attribution-ShareAlike License.