The Netduino forums have been replaced by new forums at community.wildernesslabs.co.
This site has been preserved for archival purposes only
and the ability to make new accounts or posts has been turned off.
I have a project which is currently forced to use two separate microcontrollers, one to log GPS data and the other to govern an array of sensors. They're separate since the GPS logging comes in so fast that there's just too much data, while the sensors only need log every two seconds. Since I can't time things precisely, I've decided to go for the multithreading Netduino solution.
So, is it possible for two distinct threads to log their data to the same SD card simultaneously? Not to the same file, obviously.
The hardware that each thread is using will be distinct, one uses a COM port to chat with the GPS receiver, and the other used SPI and I2C to run the sensors. No overlap there, but I'm concerned about the datalogging.
I have written a couple applications that have multiple threads accessing the SD card at the same time; it works but you need to be careful with a couple things. You hit on the first one; one thread per file (reading and writing). If you need two threads, simple locking mechanism can ensure read/seek/write transactions remain pure. Since each thread keeps their own file structures (Assuming you are not sharing them!); mostly you are OK.
I have also found that dealing with the SD card is quite resource intensive; especially when mixed with time-sensitive sensor reading. It is easy to create bottlenecks in the code that create data flow problems. Multi-threading on the netduino is not parallel processing; it just allows you to branch execution into separate places in your code - processor time is shared between the threads so only one instruction is running at a time. The right locking scheme can make all of the difference.
SD cards also use SPI. This is another area to make sure you code is aware there are multiple SPI devices.
I have written a couple applications that have multiple threads accessing the SD card at the same time; it works but you need to be careful with a couple things. You hit on the first one; one thread per file (reading and writing). If you need two threads, simple locking mechanism can ensure read/seek/write transactions remain pure. Since each thread keeps their own file structures (Assuming you are not sharing them!); mostly you are OK.
I have also found that dealing with the SD card is quite resource intensive; especially when mixed with time-sensitive sensor reading. It is easy to create bottlenecks in the code that create data flow problems. Multi-threading on the netduino is not parallel processing; it just allows you to branch execution into separate places in your code - processor time is shared between the threads so only one instruction is running at a time. The right locking scheme can make all of the difference.
SD cards also use SPI. This is another area to make sure you code is aware there are multiple SPI devices.
Thanks for this reply. Could you explain what is meant by a "locking scheme"?
Here's how I usually implement this problem on the full version of .NET on the PC: use one thread for the GPS data, one for the sensor data, and a third thread to send the data to the SD card. You could use a Queue object to manage the data flow. Your worker threads would stuff the data into the queue, using a locking object to eliminate collisions. I haven't used this kind of thing on the NetMF however - there may be threading idiosyncrasies on NetMF for which I haven't accounted.
object lockObject = new Object();Queue queue = new queue();AutoResetEvent dataIsAvailable = new AutoResetEvent(false);public static void Main(){ ... Create Logging Thread (Make sure to start this one first) ... Create GPS Thread ... Create Sensor Thread Thread.Sleep(Timeout.Infinite);}public void GPSThreadStart(){ while (true) { .... raw GPS data received ... create data object from raw GPS data lock (lockObject) { queue.Enqueue(data); } dataIsAvailable.Set(); }}(Sensor thread is the same basic idea)public void LoggingThreadStart(){ while ( true ) { dataIsAvailable.WaitOne(); while (queue.Count > 0 ) { object data; lock (lockObject) { data = queue.DeQueue(); } ... Write data to SD card } }}
The AutoResetEvent allows the threads to wait until data is available without using any CPU cycles. The lock() statements make sure the Dequeue/Enqueue calls are completed before a thread context switch is allowed, avoiding collisions. Using this scheme, you can enqueue sensor data inside interrupt handlers, and even write the logging data to the same file if you like. When I do that, I usually use a base class to enforce a common record format, then derived classes for each different type of record.
If you need help with NeonMika.Webserver, please just leave a note in the thread and/or contact me via Skype
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------
--- Mistakes teach you important lessons. Every time you make one, you are one step closer to your goal. ----
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------
Pretty nice piece of code Eric, I think I will give this a try sometime.
Do you know if the AutoResetEvent is in the .Net MF too? This multi-threading-mechanism is pretty cool.
Greets, Markus
Thanks Markus.
Yep, AutoResetEvent and ManuResetEvent are in the NetMF. IIRC, in the full framework it's a single class called EventWaitHandle with the ability to set auto or manual at will.
Couple more things I thought of:
1. try to keep the code within your lock statements as small as possible. Rule of thumb in the full framework is 5 IL instructions, that's usually 1 or 2 lines of code.
2. If your system is designed to run forever, then this code should work, but if you build in the concept of a shutdown/restart or start/stop kind of thing, then you will have to manage the shutdown properly to avoid losing any data. Basically you send a signal to all threads to shutdown, then kick the dataIsAvailable handle (tricking the logger thread into emptying the queue), Then you have to wait for the logger thread to send a signal that it is done before shutting down. I usually just use "volatile bool" variables for all these signals. Of course, if a few lost sensor readings or GPS don't matter, then the point is moot )
Thanks for this reply. Could you explain what is meant by a "locking scheme"?
Sure:
In my code I open the Filestream once and keep it open. It would be a bad thing if multiple threads interacted with the the stream at "the same time". I therefore create an object to lock it (private object LockMe = new object() and any time I am reading or writing to the filestream I make sure the code holds a lock on this object (lock(LockMe) { //Code here } )
This way, you can ensure only one thread is interacting with the filestream at any given time. Just make sure the scope of LockMe is shared between all threads.
I hope this helps. You can google "C# lock()" for more reference.