9 replies to this topic

[Chris Walker]

Steven Behnke and and Blake Ramsdell have kindly offered to help port the MetaDataProcessor to Linux/Mac. Steven has been working on tweaking the compiler files and I'll post excerpts from his e-mails below. He started tweaking the header files to work with Mac/Linux and is looking for some files/functions. I'll post his first message below (Steven, feel free to follow up with your thoughts; Blake, please feel free to ask questions about where to get started). I'm going to put the MetaDataProcessor Mac/Linux project up on GitHub in the next few days. Should we just start with the MetaDataProcessor directory itself--and then we can all add files to the project as we find all the dependencies? The .NET Micro Framework code is OSS (Apache 2.0) and we will be making the modified port OSS (Apache 2.0) as well. Steven's first post follows...

[Chris Walker]

So far I'm at a loss on these header files:

Description Resource Path Location Type
ole2.h: No such file or directory MetaDataProcessor line 10 C/C++ Problem
poppack.h: No such file or directory MetaDataProcessor line 2064 C/C++ Problem
pshpack1.h: No such file or directory MetaDataProcessor line 2057 C/C++ Problem

winerror.h should be relatively easy. ole2.h, poppack.h and pshpack1.h are very platform specific from what I understand. I believe poppack.h and pshpack1.h have to do with structure alignment.

Of course I'm also missing these files:

/home/steven/Downloads/MicroFrameworkPK_v4_1/CLR/Tools/Include/corsym.h:25:17: error: rpc.h: No such file or directory
/home/steven/Downloads/MicroFrameworkPK_v4_1/CLR/Tools/Include/corsym.h:26:20: error: rpcndr.h: No such file or directory
/home/steven/Downloads/MicroFrameworkPK_v4_1/CLR/Tools/Include/corsym.h:29:2: error: #error this stub requires an updated version of <rpcndr.h>
/home/steven/Downloads/MicroFrameworkPK_v4_1/CLR/Tools/Include/corsym.h:33:21: error: windows.h: No such file or directory

Any ideas where I could find: PCCOR_SIGNATURE would be useful too.

I've attached the header file I've built so far to provide Windows Compatibility on Linux.

Attached Files

•  windows_compatability.h.txt   1.64KB   2 downloads

[Steven Behnke]

Steven Behnke and and Blake Ramsdell have kindly offered to help port the MetaDataProcessor to Linux/Mac.

Steven has been working on tweaking the compiler files and I'll post excerpts from his e-mails below. He started tweaking the header files to work with Mac/Linux and is looking for some files/functions. I'll post his first message below (Steven, feel free to follow up with your thoughts; Blake, please feel free to ask questions about where to get started).

I'm going to put the MetaDataProcessor Mac/Linux project up on GitHub in the next few days. Should we just start with the MetaDataProcessor directory itself--and then we can all add files to the project as we find all the dependencies?

The .NET Micro Framework code is OSS (Apache 2.0) and we will be making the modified port OSS (Apache 2.0) as well.

Steven's first post follows...

You can't even begin to compile it would the CLR and DeviceCode include directories at the least. I think we might have to have the full Micro Framework available for modifying this tool.

[Chris Walker]

Steven, Blake...what's the best way to coordinate on this? The main MetaDataProcessor files are in: C:\MicroFrameworkPK_v4_1\CLR\Tools\MetaDataProcessor MetaDataProcessor appears to link to the TinyCLR itself, so pulling the necessary functionality out of there will probably be the bigger job here...

[Chris Walker]

You can't even begin to compile it would the CLR and DeviceCode include directories at the least. I think we might have to have the full Micro Framework available for modifying this tool.

So perhaps we put the entire "C:\MicroFrameworkPK_v4_1\" folder into a GitHub project...and then we simply remove the files that aren't needed once we're done?

Chris

[Steven Behnke]

So perhaps we put the entire "C:\MicroFrameworkPK_v4_1\" folder into a GitHub project...and then we simply remove the files that aren't needed once we're done?

Chris

That sounds the best to me. Then I can commit the modified header files that we can then build a diff from to submit to microsoft. Sorry for the typos. The ipad has a mind of it's own sometime while typing.

[Chris Walker]

That sounds the best to me. Then I can commit the modified header files that we can then build a diff from to submit to microsoft. Sorry for the typos. The ipad has a mind of it's own sometime while typing.

Cool. Then we can make one project and include all the MFDeployEngine code in it as well. I'm on it. It may be a few days before it all gets posted...

Chris

[Blake Ramsdell]

So I'm still coming up to speed, and I'm curious what the goal is. Like I'm trying to figure out the purpose of MetaDataProcessor. I would presume that the world on non-Windows looks something like: 1. Generate assembly with Mono 2. Prepare assembly for transmission to target environment 3. Transmit assembly to target environment I presume step 1 is fairly well understood. There may be some amount of getting the Netduino-specific classes for the pin definitions and stuff massaged into more of a Mono/MonoDevelop mindset. So my guess is we're on step 2, and it is believed that MetaDataProcessor is involved here. Looking at the code for MDP it looks like he does a bunch of crap for both converting the endianness of fields in an assembly, as well as crypto-related functions. I have a concern with the compatibility include file approach, because at the end of the day, even if we have HRESULT defined and that kind of stuff, the code is going to call some nontrivial Windows platform functions that we don't have. Like I don't think the "patch and pray" approach will work here -- this code may be hopelessly Windows-specific. So another approach might be to take a step back and determine what scope of functionality we need for my "step 2" above, and see how we can implement that. And the good news is that we can most likely do it in C# and avoid the knife juggling of C++ altogether. But I'm at a loss as to what step 2 entails, so this might be a bad idea on my part. Can anyone enlighten me about the big picture of "starting with C# source code, how does it get on the device, and what format it needs to be in"? And certainly an RTFM pointer would be welcome. I'll research this independently and see what I can come up with.

[Chris Walker]

Blake, We've thought about rewriting MetaDataProcessor in C#, but the side-effect of that is that _might_ make it harder for us to update MetaDataProcessor in the future to match new versions of .NET Micro Framework. And we could introduce lots of bugs into something that was already working, tried, and tested. That said, I wouldn't rule it out completely as an option either. We're more or less working backwards from #3 to #1. We've done some work internally to get MFDeployEngine working under Mono (mostly deconstructing and reconstructing the serial port code to get the computer and device to talk) and that's _mostly_ working. We did that with .NET MF 4.0 and Microsoft did a bunch of updates to the 4.1 edition (including updating the framework target to .NET 4.0--which Mono hasn't quite caught up with yet)...so we're now working on a hybrid of the two internally to make a ".NET MF 4.1 MFDeployEngine that runs under a .NET 3.5/Mono CLR". There are a few blog posts on the web talking about MetaDataProcessor. Here's one where someone explains how VB could be supported (in theory) using a more manual approach. When I spoke with Microsoft, they said that one piece is missing... Visual Studio uses the MFDeployEngine to check for what assemblies are integrated into the device firmware before creating a HEX file so that it doesn't deploy them twice. http://www.christec....og/archives/317 Chris

[Steven Behnke]

So I'm still coming up to speed, and I'm curious what the goal is. Like I'm trying to figure out the purpose of MetaDataProcessor. I would presume that the world on non-Windows looks something like:

1. Generate assembly with Mono
2. Prepare assembly for transmission to target environment
3. Transmit assembly to target environment

I presume step 1 is fairly well understood. There may be some amount of getting the Netduino-specific classes for the pin definitions and stuff massaged into more of a Mono/MonoDevelop mindset.

So my guess is we're on step 2, and it is believed that MetaDataProcessor is involved here. Looking at the code for MDP it looks like he does a bunch of crap for both converting the endianness of fields in an assembly, as well as crypto-related functions.

I have a concern with the compatibility include file approach, because at the end of the day, even if we have HRESULT defined and that kind of stuff, the code is going to call some nontrivial Windows platform functions that we don't have. Like I don't think the "patch and pray" approach will work here -- this code may be hopelessly Windows-specific.

So another approach might be to take a step back and determine what scope of functionality we need for my "step 2" above, and see how we can implement that. And the good news is that we can most likely do it in C# and avoid the knife juggling of C++ altogether.

But I'm at a loss as to what step 2 entails, so this might be a bad idea on my part. Can anyone enlighten me about the big picture of "starting with C# source code, how does it get on the device, and what format it needs to be in"? And certainly an RTFM pointer would be welcome. I'll research this independently and see what I can come up with.

The problem is that you can't simply send over the assembly as is to the Netduino. That's where the MetaDataProcessor comes in. It goes through the process of minimizing the assembly and outputting the smallest possible resulting binary file to be handed over to MFDeploy to load on the Netduino. I realize trying to provide Windows compatibility through a header file isn't the ideal solution, but I was at least trying to identify all of the functionality that doesn't exist and get the error list down to a minimal set so we see what we have to actually deal with. Some of the bit packing functionality appears to be completely missing and will have to be rewritten.

An example of how it is used is this:

"%MF_BIN%\MetaDataProcessor.exe"
-loadHints mscorlib "%MF_LIB%\mscorlib.dll"
-parse HelloWorldVB.exe
-minimize
-compile HelloWorldVB.pe

During the minimize step certain attributes are omitted and other metadata is changed to be smaller.

I too have thought about writing a managed version of this. I'm not sure which will be less work.