25 replies to this topic

[Mario Vernari]

Hello folks.

*** OFF-TOPIC DESPERATION ***
It was around the mid '80, in the heat of the "turbo-age". I was programming Motorola MCUs in assembler, and the cycle was *veeery* tedious!
An editor to create the source, then a compiler, then a linker, finally an utility to deploy the code to the emulator. That emu was connected via RS232 at 1200bps (at 2400 there were ton of errors), and making even a simple step-by-step was a nightmare.
So I decided to create my own "ide". That was blazing fast (on a PC-XT 4.77MHz): an editor, a compiler, an embedded simulator. Consider having a "turbo-whatever". My job speed up 100x more.
Now it's 2011 (about 30yrs later). We have jit-compilers able to work in background: amazing.
I ask you: having a gigahertz+gigabyte+Ncores PC, why to compile the source for a small MCU should we wait MINUTES???...
I guess that's Alice in Wonderland.
Flags, addresses, preprocess-directives everywhere...why all that?
Masochism?
*** END OFF-TOPIC ***

OK, never mind...life is beautiful anyway.
I am trying to compiling the Netduino plus firmware (ed. 4.1.0.6) using gcc 4.5.
I have not still added anything but the SecretLabs edition.
I have followed the walkthrough described by CW2 here:
http://forums.netdui...h-vs2010-stuck/
It works at all, except for the flash occupation.

Well, I have read that the latest gcc (e.g. 4.4) is able to generate native code not so larger than the mythical RVDS. So, I'd expect having a little overhead (reasonable).
At the moment, all my attempts are just on compilation, so the board has still the original firmware provided. So, this is the flash map:

Flash Sector Map Command
 Sector    Start       Size        Usage
-----------------------------------------------
    0    0x00100000  0x0000c000   Bootstrap
    1    0x0010c000  0x00012000   Code
    2    0x0011e000  0x0004c000   Code
    3    0x0016a000  0x00004000   Deployment
    4    0x0016e000  0x0000c000   Deployment
    5    0x0017a000  0x00002000   Storage (A)
    6    0x0017c000  0x00002000   Storage (B)
    7    0x0017e000  0x00002000   Configuration
    8    0x00000000  0x75e00000   File System
Flash Sector Map Complete

I have then tried to compile the firmware using this command:

C:\Netduino4106>msbuild solutions/netduinoplus/dotnetmf.proj /p:flavor=release /v:m /m

but the compilation halts because the code overflows the available flash: it exceeds by over 130K!!!

Then I have followed the indication of CW2, to cut off some libraries from the solution wizard (file-system, stream, sd, etc). I don't like it, because I'd need those features, but...never mind.
By running the compilation again, it halts again showing the same problem:

c:\codesourcery\bin\arm-none-eabi-ld.exe: C:\Netduino4106\BuildOutput\THUMB\GCC4.2\le\FLASH\release\NetduinoPlus\bin\tinyclr.axf section `ER_FLASH' will not fit in region `LR_FLASH'
  c:\codesourcery\bin\arm-none-eabi-ld.exe: section ER_CONFIG loaded at [0017e000,0017e90b] overlaps section ER_FLASH loaded at [0011e000,001842b3]
  c:\codesourcery\bin\arm-none-eabi-ld.exe: region `LR_FLASH' overflowed by 80568 bytes

Here is the block range:

const BlockRange g_SAM7X_BS_BlockRange[] =
{
    //
    { BlockRange::BLOCKTYPE_BOOTSTRAP       ,  0,  5 },
    { MEMORY_BLOCKTYPE_GCC_SPECIAL_BOOTSTRAP,  6, 14 },
    { BlockRange::BLOCKTYPE_CODE            , 15, 52 },
    { MEMORY_BLOCKTYPE_GCC_SPECIAL_CODE     , 53, 54 },
    { BlockRange::BLOCKTYPE_DEPLOYMENT      , 55, 60 },
    { BlockRange::BLOCKTYPE_STORAGE_A       , 61, 61 },
    { BlockRange::BLOCKTYPE_STORAGE_B       , 62, 62 },
    { BlockRange::BLOCKTYPE_CONFIG          , 63, 63 }
};

and the addresses:

        <Set Name="Code_BaseAddress"    Value="0x0011e000"/>
        <Set Name="Deploy_BaseAddress"  Value="0x00172000"/>
        <Set Name="Code_Size"           Value="%Deploy_BaseAddress - Code_BaseAddress%"/>
        <Set Name="Config_BaseAddress"  Value="0x0017E000"/>    
        <Set Name="Config_Size"         Value="0x00002000"/>

The blocks show about 48K of free flash (deployment), so -by using gcc- I'd expect a loss of 50% (pessimistic). That is about 24K for deployment and 24K for the compiler overhead.

Where is the trick?

Thank you so much in advance.

[Chris Walker]

Hi Mario, Have you compiled the firmware with the older, .NET MF-supported version of GCC? As a baseline? Then after that works, recompile with GCC 4.5? Chris

[Mario Vernari]

Hi Chris and thank for the help.
I have made several attempts, but those I have described is the only that ran pretty well.
Every time I started with a brand new folder, so I guess there is no pollution on files.

EDIT: I didn't try with the old 4.2, because CW2 instructed how to do directly with the 4.5.

I have followed the instructions of CW2 (frame #12) on the thread I have indicated.
From the command prompt the result is as follows:

C:\CodeSourcery\arm-none-eabi\bin>gcc --version
gcc (Sourcery G++ Lite 2010.09-51) 4.5.1
Copyright (C) 2010 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

To tell the truth, on this last attempt, I didn't perform the preliminary building:

MSBuild build.dirproj

Because CW2 has not listed, so...maybe...was unnecessary?

What should do instead?

[CW2]

I ask you: having a gigahertz+gigabyte+Ncores PC, why to compile the source for a small MCU should we wait MINUTES???...

I guess that's because the firmware has ~600 000 lines of code (excluding comments and blanks) and complex structure (i.e. many dependencies)

Just out of curiosity, have you used minimal verbosity and parallel build options (msbuild ... /v:m /m)?

[Mario Vernari]

Yes, I have added the flags, *AND* (I apologize the missing) the lwip stack option.
Here is the exact command I have used:

C:\Netduino4106>msbuild solutions/netduinoplus/dotnetmf.proj /p:flavor=release;tcp_ip_stack=lwip /v:m /m

as Chris specified here.
Otherwise won't compile because an unresolved symbol.

I apologize again for the mistake above.

I guess that's because the firmware has ~600 000 lines of code

If that's true, it is a case of inefficient programming: 5M of chars to produce 250K of opcodes. (Assuming around 10chars/line)
CW2, I had tons of discussions about C/C++ with my collegues, because I find non-sense many things of those languages.
I don't blame several features of C at the time was created, because PCs were poor of resources. But there is no valid reason having a so complicated environment under any modern PC.
IMHO C/C++ are so chaotic because there was too "anarchy" for the developers, and now they have tons of old sources that cannot be "modernized" anymore.
Thank you anyway for your point.

[Mario Vernari]

I consider this attempt almost a garbage, so I made some experiment on.
Firstly, I made the preliminary build, as the PK says:

msbuild build.dirproj

Secondly, hereinafter I always used the /t:rebuild flag for the compilation.

Finally I was able to compile a kind of "firmware" (even I have noticed some reddish text passing through during the compilation).
The problem now is that I hardly may admit this is still a Netduino+, since I have cut off almost any feature.
This is still selected in the solution wizard:

Projects selected:

• tinyclr
• tinyBooter
• tinyBooterDecompressor

Features:

• debugger
• diagnostic
• interrupthandler
• secretlabs.netmf.diagnostic
• secretlabs.netmf.hardware
• usbclient

EDIT: the ER_FLASH file is 307K.

Moral.
If the brand-new Ferrari is too expensive for you, consider cutting-off some optionals such as the engine, chassis, brakes, instrumentation, etc. That probably fit your budget.

Thanks for the patience

[CW2]

If that's true, it is a case of inefficient programming: 5M of chars to produce 250K of opcodes. (Assuming around 10chars/line)

Just for clarification, the original number was for Netduino firmware source only, with some unnecessary/unused stuff removed (such as security libraries, samples, solutions for other boards etc.). The following table contains the most significant results from cloc run over the complete 4.1 QFE1 Porting Kit source:

Language                     files          blank        comment           code
-------------------------------------------------------------------------------
C++                           1831          83131          89910         390481
C#                            1226          44808          66057         325136
C/C++ Header                   754          30944          39972         139219
C                              238          12448          22811          78130
...
Assembly                        69           2620           2084          14068
...
-------------------------------------------------------------------------------
SUM:                          4843         189353         228512        1035400

I don't really want to argue about programming efficiency here, but IMVHO one cannot conclude much just from the number of SLOC.

But there is no valid reason having a so complicated environment under any modern PC.

Well, pick any similar project from [insert your favorite open source repository here] and try it for yourself - you'll see that .NET MF is not really that bad

[Mario Vernari]

*** BEGIN OFF-TOPIC *** I wouldn't going too much off-topic with *my own* ideas about what is chaotic and what is not. That could be a good discussion, if you want, but here is not the right thread, I think. Just to close and explain *my* viewpoint: I was student and after I became a professional. There are thousands of reasons that move a professional development toward a certain direction, that often they differ with an academic/researcher subject. The professional *must* answer their bosses (then customers) within hours/days/weeks: "It can be done that? How many days do you need? Will be reliable? May I begin to advertise?". That leads to use products that *must* be real RAD-like. That is because I think this kind of modus-operandi is too chaotic. My collegues are working with C/C++ with a similar scenario are losing at least 10x the time I need using C# and Visual Studio. That is a real RAD. However there are many other products non-MS, that is RAD too. *** END OFF-TOPIC *** At the moment I am stuck with the compilation of the basic firmware. I have followed your instructions (I think at all), with the only difference that I own a Netduino+. Do you think I'll be able to add the standard libs of N+, so they fit in the available flash (512K)? Of course that would only the first step, 'cos after I should be able to add my own native libs and some managed code to test them. I don't think the libs will be heavy, though. Thank you very much for your support. Cheers

[CW2]

Do you think I'll be able to add the standard libs of N+, so they fit in the available flash (512K)?

I have never compiled NetduinoPlus firmware with GCC - I will have a look at it... Please note the problem is not with GCC itself, but with the size of C/C++ runtime libraries included. Even newlib optimized for embedded systems has significantly larger implementation of s/v/nprintf family functions (as I mentioned in Toolchain roundup post).

[Mario Vernari]

I guess the problem *is* GCC, or -better- anything but RVDS. According on what you are written on the roundup, it seems that GCC (even the latest versions) produces a binary much larger than the same source compiled with RVDS. That is what I have understand. I have read that GCC 4.4 produces much more compact object, though. If so, that means I'll have no chances to build something decent, because: - RVDS has no free/limited editions, so it's out for most of us; - GCC has an excessive impact on the code produced, to oblige us to cut almost any library. I really cannot see any reasonable way to write some libraries (at least trying to do them). It sounds much like trying to fit the water of a pool into a bottle.

[CW2]

- RVDS has no free/limited editions, so it's out for most of us;

ARM provides time-limited evaluation version of RVDS 4.1 Pro (e-mail registration required).

[Mario Vernari]

I know and I have registered yet, but -for me- 45 days is just moving few steps. EDIT: Anyway, it seems to me shocking that a 512K flash is not enough to fit the tinyclr, without any other extension.

[FelixEmman]

Mario, I had the same problem with GCC. I wanted to add a firmware feature to the N+ code, and after lots of frustration with making it even compile, I realized that I had to build with tcpip_stack=lwip to be able to use the ethernet. That ended in disappointment because the binaries produced by GCC were now way over the capacity, so I went ahead and tried what you did, strip the Ferrari's engine, brakes, suspension and gearbox, hoping it will work. I was too optimistic. Conclusion: GCC might work, but I lost my patience. The real issue is that currently the 512Kb of flash is by all means a BIG flash, but when you're trying to run .NET in a nutshell on that 512kb, you can't expect much. So a bigger flash would be the first improvement that would make a difference for those who want to hack. Right now the stock N+ firmware enables 64kb of free space, which is GOOD. But it's really pushing the limits when it comes to adding new features to N+, so unless Secret Labs will revision with a new CPU with more flash/RAM, I think they'll run out of flash pretty soon, even using RVDS. Let's keep the fingers X-ed for that to happen sooner ;-)

[Mario Vernari]

unless Secret Labs will revision with a new CPU with more flash/RAM, I think they'll run out of flash pretty soon

I agree at all.
I am wondering whether Chris has any plan to develop a "Netduino super" with an ARM 9 MCU. That will have 180MHz and memory expandibility: that is not for production, but of sure could make our life a lot better. (At least until that board keep below a reasonable price!)
Cheers

[Chris Walker]

Hi Mario,

EDIT: Anyway, it seems to me shocking that a 512K flash is not enough to fit the tinyclr, without any other extension.

I totally understand your pain.

Luckily, there is actually quite a bit of elbow room with 512KB of Flash. The TinyCLR itself only takes about 300-350KB of flash (using RVDS). lwIP takes up about 64KB-80KB. It all depends on which features you do or do not want, I suppose.

We are _very_ interested in fitting a GCC-compiled version into Flash without removing any of the extra features. We'll be starting a thread soon working with the community to compile the Netduino firmware using the latest GCC...and then moving onto the Netduino Plus firmware. No guarantees on GCC's performance, but we'd really like to empower custom firmware solutions.

Also, the upcoming runtime interop feature will let you add features without recompiling the core firmware...so that will also enable some good scenarios.

Fun fact: we've actually reserved 8KB-24KB (depending on Netduino model) to be able to add extra features to the Netduino firmware without affecting code storage space. We've only used a few KB of that so far, and the extra elbow room helps community members have room for their own enhancements (or less efficient compiled code) as well.

I am wondering whether Chris has any plan to develop a "Netduino super" with an ARM 9 MCU. That will have 180MHz and memory expandibility: that is not for production, but of sure could make our life a lot better. (At least until that board keep below a reasonable price!)

We work with ARM9 MCUs and .NET MF for our commercial clients, so this is not out of the question. Although price would be a big concern for us...Netduino isn't really meant to be a $200 devboard. All that said, we have some ideas for new hardware; more on that later.

Chris

[Species8372]

Hi Chris,

because of being in desperate need of some bit-banging possibility the SPI doesnt have (need to wait between write and read for some microseconds) i wondered if there has been some progress going on with the gcc-compiling for the community?

We are _very_ interested in fitting a GCC-compiled version into Flash without removing any of the extra features. We'll be starting a thread soon working with the community to compile the Netduino firmware using the latest GCC...and then moving onto the Netduino Plus firmware. No guarantees on GCC's performance, but we'd really like to empower custom firmware solutions.

In the meantime, i guess not only myself would be very very glad to get hands on a netduino Plus Firmware with your mentioned Interop-Feature (or at least the SimpleNGen-Entry-Point):

Also, the upcoming runtime interop feature will let you add features without recompiling the core firmware...so that will also enable some good scenarios.

I hope you can help some of us?

Species8372


PS: If someone has the RVDS i'd like to help setting up some service like mbed-Compiler to allow the community to self-compile the Netduino-Plus-Firmware....

[CW2]

PS: If someone has the RVDS i'd like to help setting up some service like mbed-Compiler to allow the community to self-compile the Netduino-Plus-Firmware....

This has been proposed already, but IMHO it is not possible due to the RVDS license - it is very detailed about the various usage scenarios (I am not a lawyer, though). mbed can offer online "compiler-as-service", because it is backed by ARM itself ("project was initiated by two ARM employees, and later became an official research project within ARM"), so they probably have a special license (or just do not have to care about it).

I have not performed any measurements with the recent versions of GCC, but from what I recall, the problem is the size of C/C++ Runtime Library functions - get the printf() implementation that produces code of similar size compared to RVDS (resp. RVCT), and the output will fit into flash...

[Species8372]

So i will never ever get a patched firmware. Because for several thousand bucks for the compiler i'd rather go to some specialists and let them program and compile to my wishes (if i had the money). What about: * a really tiny firmware * all neccessary addons are loaded from the sd-card (even older digi-cams can do that, use a firmware on sd-card instead of the built-in one) I dont see a way for anyone in the community to implement new device drivers or extend the firmware for the netduino plus. Except the ones who have access to this magic ARM-Compiler from their workplace.

[Chris Walker]

Hi CW2, Species8372: We'll be posting the source for .NET MF 4.2 RC3 firmware in the next few days. Would one of you like to test it with the latest supported version of GCC, and see how big the compiled firmware comes out? Chris

[KodeDaemon]

Hi CW2, Species8372:

We'll be posting the source for .NET MF 4.2 RC3 firmware in the next few days. Would one of you like to test it with the latest supported version of GCC, and see how big the compiled firmware comes out?

Chris

I was waiting for the firmware source code release to do this with my GCC support patch.