18 replies to this topic

[Mario Vernari]

Hi all.
Last weekend I have struggled around my attempt to make a *very basic* signal processor with Netduino+MF. The idea is stimulating the analog input with a wave, then Netduino should guess that frequency. Bear in mind we are talking of frequency from 0.5 to 2 Hz.
I really have lost any optimism, because there are tons of problems around that.

The target is very simple...
Every 200ms:

• collect an ADC reading;
• place it on a circular buffer (len=64);

Every 800ms:

• make the FFT on the buffer;
• search the peak frequency;
• display on the LCD (16x2) the result;

Ansynchronously:

• make available the FFT result for the embedded web server.

There were problems of RAM, problems of ROM, problems of optimization...all were solved with no particular troubles.
Well, believe me or not, the *only* problem is the jittering on the sampling period. It seems to me as impossible to obtain a decent precision on that timing.
The first attempt was using a "Timer", but it is far from being precise.
The last attempt is a tight loop on a dedicated thread, whose priority is the highest. That is much better, but there is still a jitter of -/+ 5% on the time. That's the "best" result I had!
That jitter is kind a frequency-modulation that destroys any possibility of detection.
I may "accept" for sampling time under the millisecond, but here are hundreds of ms!...I am sampling a frequency of just one Hertz!
That's going me crazy thinking a 48MHz MCU has difficulties to measure a so low frequency.

The discussion here is not about *my* problem, but the ability to collect data with a relative precision. I personally think that is a very common task on a device, but it seems very hard to have even with an external hardware (at least without another processor).
So I am wondering if anyone of you is interested in a similar library.
That should be done in C/C++ and should interop with the managed code in a very easy way.
At the moment I'm in the office and I can't have the code I have used for my app, but the sense is having a kind of "reader" for the analog port.
That (disposable) class should act as a "producer" of data and it needs at least two parameters: the port and the sampling period. At this point there are two possibilities where to lay the data read:

• the reader owns a private cache, where the samples are collected and the consumer (i.e. the host app) pulls out with "relative" imprecision. If the consumer is not able to pull out the data fast enough, the oldest will be lost;
• the reader has another param, which is a reference to the array to fill.

There are also multiple way to embellish such a reader:

• collect N samples only then stops;
• ability to collect even boolean data, from an InputPort;
• ability to collect mode inputs at the same time, yielding structs of samples;
• ability to count high-speed events withing a timeframe (e.g. number of InputPort edges within exactly 100ms).

This discussion is especially dedicated to Chris Walker, as referent for the N firmware. Do you think that could be an useful extension to embed?
Feel free anyone to say what it thinks about.
Thank you all.
Cheers

[Chris Walker]

Hi Mario, That could be a very useful feature to embed. It's certainly possible to do this today using a mixture of native and managed code, but having it built-in (or deployed via the upcoming runtime interop feature) would certainly be useful. Either way, you could tie to a high-priority hardware timer to get readings every #ns/us/ms. Chris

[Mario Vernari]

Some day ago I was talking with Gus-Fezzer about the possibility to write a signal-processing library to embed into the firmware.
Please, tell me if you are interested in also.

Either way, you could tie to a high-priority hardware timer to get readings every #ns/us/ms.

Could you explain better what you mean?


Anyway, I am just arrived at home and here is the code about the "AnalogReader" as promised.

using System;
using Microsoft.SPOT;
using System.IO;
using Microsoft.SPOT.Hardware;
using SecretLabs.NETMF.Hardware;
using System.Threading;
using SecretLabs.NETMF.Hardware.NetduinoPlus;


namespace Highfield.NetduinoHeartbeatMeter
{
    public struct AnalogReaderItem
    {
        public int Value;
        public TimeSpan Time;
    }



    public class AnalogReader
        : IDisposable
    {
        private const int CacheBufferLength = 4;
        private const int CacheBufferMask = CacheBufferLength - 1;



        public AnalogReader(AnalogInput input, TimeSpan period)
        {
            this._input = input;
            this._period = period;

            this.StartWorker();
        }



        private AnalogInput _input;
        private TimeSpan _period;

        private AnalogReaderItem[] _cache = new AnalogReaderItem[CacheBufferLength];
        private uint _cacheSamplerCount;
        private uint _cacheReaderCount;

        private bool _abort;
        private AutoResetEvent _semaphore;



        public AnalogReaderItem Read(bool abort)
        {
            AnalogReaderItem result;
            this._abort = abort;

            if (abort)
            {
                result = new AnalogReaderItem();
            }
            else
            {
                try
                {
                    this._semaphore.WaitOne();

                    uint lowerLimit = this._cacheSamplerCount - CacheBufferLength;
                    if (lowerLimit > this._cacheReaderCount)
                        this._cacheReaderCount = lowerLimit;

                    result = this._cache[this._cacheReaderCount & CacheBufferMask];
                    this._cacheReaderCount++;
                }
                finally
                {
                    this._semaphore.Set();
                }
            }

            return result;
        }



        private void StartWorker()
        {
            this._semaphore = new AutoResetEvent(true);

            var t = new Thread(Worker);
            t.Priority = ThreadPriority.Highest;
            t.Start();
        }



        private void Worker()
        {
            var prevTime = Utility.GetMachineTime();
            var nominalPeriodDoubled = this._period.Add(this._period) - TimeSpan.FromTicks(3 * TimeSpan.TicksPerMillisecond);
            var elapsed = this._period;

            while (this._abort == false)
            {
                var actualPeriod = nominalPeriodDoubled.Subtract(elapsed);
                TimeSpan currTime;
                do
                {
                    Thread.Sleep(1);
                    currTime = Utility.GetMachineTime();
                    elapsed = currTime.Subtract(prevTime);
                }
                while (elapsed < actualPeriod);

                var sample = new AnalogReaderItem
                {
                    Value = this._input.Read(),
                    Time = currTime
                };
                prevTime = prevTime.Add(elapsed);

                try
                {
                    this._semaphore.WaitOne();
                    this._cache[this._cacheSamplerCount & CacheBufferMask] = sample;
                    this._cacheSamplerCount++;

                    //Debug.Print(adc.ToString() + "   " + prevTime.Ticks);
                }
                finally
                {
                    this._semaphore.Set();
                }
            }
        }



        #region IDisposable implementation

        private bool _disposed;



        public void Dispose()
        {
            this.Dispose(true);
            GC.SuppressFinalize(this);
        }



        protected virtual void Dispose(bool disposing)
        {
            if (!this._disposed)
            {
                if (disposing)
                {
                    // dispose-only, i.e. non-finalizable logic
                }

                // shared cleanup logic
                this._disposed = true;
            }
        }



        ~AnalogReader()
        {
            this.Dispose(false);
        }

        #endregion
    }
}

Cheers

[Nevyn]

In the very beginning there was C-flat only. The C# came much later, after adding a whole tone upper.

Like the sig but I get the feeling it's a lot easier to cut yourself with C flat than C#.

Cheers
Mark

[CW2]

Slightly off-topic: Another way to make Netduino (or generally any .NET MF board) a little bit closer to a RTOS device is to port the .NET Micro Framework onto an existing real-time operating system. The .NET MF has only one thread of execution, so it basically means creating one 'task' in RTOS that calls .NET MF entry point function (and of course remapping and implementing the necessary APIs). The Emulator is exactly that, .NET MF ported onto Windows. For more information, see "Porting onto an Operating System" in the Porting Kit documentation.

I have also found ProConOS embedded CLR, which claims to be 'real-time capable' and uses Ahead-Of-Time compiler - I am not sure how this relates to .NET MF, but it is an interesting implementation. Though I have not had a chance to evaluate it yet.

[Mario Vernari]

Like the sig but I get the feeling it's a lot easier to cut yourself with C flat than C#.

Unfortunately that sounds true...but I feel like a warrior, fighting against the chaotic, messy, anarchical and old plain-C!...(Two seconds later I feel just Don Quixote)...
Jokes apart, I guess that if this platform promises usability with C# and MF, that should be relatively true. It seems that even simple tasks are such a mountains.
From my viewpoint, it makes no sense having a WPF-like board, with touch, web-services and other amazing stuffs, while the simple periodic reading of an adc is an hell. That could be for a PC, not for a MCU board.

I have also found ProConOS embedded CLR, which claims to be 'real-time capable' and uses Ahead-Of-Time compiler - I am not sure how this relates to .NET MF, but it is an interesting implementation. Though I have not had a chance to evaluate it yet.

I know also, but I never tried too. Anyway I guess that it outside our context, being amateur-like (the mine at least). I am scared about the price.
The only solution I see is to enrich the MF with a sort of basic helpers for a more realistic utilization.
Cheers

[Chris Walker]

Hi Mario, If you want to help create the helper classes, we'll be happy to provide a way to get them either (a) into the core firmware or ( hooked into the firmware via the upcoming runtime interop feature. Chris

[Mario Vernari]

If you want to help create the helper classes, we'll be happy to provide a way to get them either (a) into the core firmware or ( hooked into the firmware via the upcoming runtime interop feature.

That would be wonderful, but the only thing I can do is to make a C# library, maybe with a good doc. I don't think that could fit so much for a native implementation.
I have no experience with C/C++, but a lot with MCUs and C#. If anyone of you knows a well described walkthrough on how to make you own driver, that would at least an attempt to give you something useful.
Basically I would need:
- what I have to install on my Vista 32 PC and how to set up the environment una-tantum;
- how to keep in sync with the MS .Net MF libs and the Netduino SDK, in order to compile them;
- one or more sample drivers so that I may understand how exchange data to/from the managed layer;
- what else?...maybe a good friend for the moment of depression?

Thanks in advance.
Cheers

[Troll feeder]

I habent fallowed the whole saga, so pardon me if some of my points have been discussed. Have you thought about an external rtc ($5-10) and an event /interrupt handler assigned to the pin the rtc is on? Or Is there a problem with this technique? I've been hopping between c# on mcu's and vb .net on desktops and arduino so often I'm having ot double check everything i program. I tend to agree .net micro frame work needs to have a high percussion timing mechanism. It's really imperative for many mcu type projects. Mike.

[Mario Vernari]

Have you thought about an external rtc ($5-10) and an event /interrupt handler assigned to the pin the rtc is on? Or Is there a problem with this technique?

Making an external oscillator is very easy, even an high-precision crystal based. I guess the cost will be no more than a buck. That absolutely not a problem.
I think also that the internal timers offered by the MCU offer an high precision too: just take a look at the Atmel data-sheet. They are counters clocked by the MCU timing.

The problem is that there is a variable and indeterminate latency to serve an interrupt.
The "Timer" object present in the Micro Framework is hardware based (I guess), and it should fire the "Tick" event as soon the hardware timer interrupt calls its (native) service routine. I am sure that the native code is called immediately, but it takes from zero to even 20 ms to serve the managed event. That delay is pseudo-random, and is easy to see that is longer the higher is the load of the MCU.
That is a normal behavior because the garbage collecting, threading and other background activities. Unfortunately won't fit any signal processing base application.

The only way to solve is to collect the samples via native code, then pull the stream from within the managed environment.
I have just began to install the necessary stuffs to work in C++, but I'm still not sure to able to do it.
Let's see.
Cheers

[CW2]

The "Timer" object present in the Micro Framework is hardware based (I guess), and it should fire the "Tick" event as soon the hardware timer interrupt calls its (native) service routine. I am sure that the native code is called immediately, but it takes from zero to even 20 ms to serve the managed event. That delay is pseudo-random, and is easy to see that is longer the higher is the load of the MCU.

Well, the timers are implemented in software, they do not use hardware timer modules. There is currently only one hardware timer that the CLR engine uses for scheduling - it is also known as 'slow timer', its frequency on Netduino is 46875 Hz (~21 µs resolution) and this determines the response time for the fastest interrupt handling, so called ISR-mode completions (*). The delay (up to 20 ms) directly relates to scheduling mechanism, it is the time quantum after which the threads are switched. The execution engine checks for any hardware events (non-ISR mode, such as continuations) only in between scheduling the threads. Additionally, in the current implementation the multitasking is cooperative, and calls to native methods are blocking, so the execution engine has to wait until the native method finishes.

(*) For more details, please refer to "Understanding the Common Language Runtime" chapter in the Porting Kit documentation.

[Mario Vernari]

Well, the timers are implemented in software, they do not use hardware timer modules. There is currently only one hardware timer that the CLR engine uses for scheduling - it is also known as 'slow timer', its frequency on Netduino is 46875 Hz (~21 µs resolution) and this determines the response time for the fastest interrupt handling, so called ISR-mode completions (*). The delay (up to 20 ms) directly relates to scheduling mechanism, it is the time quantum after which the threads are switched. The execution engine checks for any hardware events (non-ISR mode, such as continuations) only in between scheduling the threads. Additionally, in the current implementation the multitasking is cooperative, and calls to native methods are blocking, so the execution engine has to wait until the native method finishes.

All these very useful infos make me even more convinced toward a native implementation of the sampler.
I wonder why anyone didn't ask before.

[Troll feeder]

Thanks for the Info, I now understand the problem a lot better and know what to look out for. It also explains some oddities that drove me to make some serial code event and queue driven.

[Jim Davies]

I would love to see this sort of project becoming easier in NetMF, but for anyone who needs to do it 'right now' I have only one word of advice (dare I say it...) - Arduino! Jim

[Chris Walker]

I would love to see this sort of project becoming easier in NetMF, but for anyone who needs to do it 'right now' I have only one word of advice (dare I say it...) - Arduino!

Also a good option. Or you could load FreeRTOS on your Netduino...or write C/C++ code on your Netduino (like people often do on Arduino).

It sounds like there are a few specific scenarios that people need the speed of native code. Such as counting the number of pulses over ## ns/us/ms. If we can isolate those specific cases, we can create a set of classes for them.

Chris

[Michel Trahan]

calls to native methods are blocking, so the execution engine has to wait until the native method finishes.

That made me think about how we used to do sampling of brainwaves for consciousness detection under anaesthesia. First capture data and nothing else, then massage it (cumulate to get base wave, filter the 60hz that is everywhere, etc...). Since it is blocking, does it gives you full access to the mcu ? for only one process : capture data. That would be perfect.

So, correct me if I'm wrong, native code would allow you to stop everything else (gc included?) to capture clean data (maximum time allowed ?) and then back to managed code for ease of programming and analysis. I want one

you could load FreeRTOS on your Netduino...

That would be an interesting tutorial for advanced users and what is the language used in FreeRTOS ?
I'll check, thanks for posting

[Mario Vernari]

Today morning (for me), JonnyBoats has posted a video of MS about the future of Win (and derivatives), that is the cloud.
This is not a new, but only a concrete sign that MS is going toward interconnection, gadgets and whatever else could fit the cloud.
That would be a very nice thing indeed, but it also true that it would be always less close to the hardware layer.
As a ex-hardware engineer, I really can't miss to observe serious gaps in the MF. They are not mistaken by the MF team, but -worser- sections totally cut off (probably because they move few money).

We build several devices from almost 30 yrs, especially smart-sensors, that is specialized devices (MCU based) to interface some kind of sensor. One of the most desired feature was/is the flexibility, that is some kind of "programmability" on that devices.
The Netduino board made me happy, because was a nice example to full-featured board, where even a child can create his own program.

Digging a bit deeper, I have realized that a board like Netduino (i.e. MCU+MF) has something wrong.
Can't be a 512K MCU must be treated by super-expertise to add even a common Modbus interface, just as an example.
Can't be that you will able to compile only having a 6K$ compiler.
Can't be that several little projects aren't feasible, because there's no way to grant time precision.
Can't be that I'll have to purchase a GHI AT9 running 180MHz to make what a 20$ Arduino can.

Speaking for myself, I would much more comfortable adding a couple of chips and some resistor to the Netduino, instead of learning C/C++. However, what I see on this forum, is that most of you don't know so well circuitry and prefer either write code or purchase some shield.

Well, I may be the only one pointing this considerations, but I hope someone will understand where I'd like to see Netduino and its world.

Ciao

[CW2]

That would be an interesting tutorial for advanced users and what is the language used in FreeRTOS ?

Even more advanced users can port .NET Micro Framework onto FreeRTOS (written in C) and gain benefits from the both worlds. I am not sure if I remember it correctly, but I think I saw a post somewhere on Microsoft forum that somebody is already working on that...

[Chris Walker]

So, correct me if I'm wrong, native code would allow you to stop everything else (gc included?) to capture clean data (maximum time allowed ?) and then back to managed code for ease of programming and analysis. I want one

Exactly.

Native code has two options (in the "stop everything in the system" realm):

• Keep interrupts enabled to service UARTs, Ethernet packets, etc.
• Pause all interrupts system-wide

In the case of #2, you have to completely in less than a few hundred instructions...or else you could lose a byte on the serial port etc.

Chris