Netduino home hardware projects downloads community

Jump to content


The Netduino forums have been replaced by new forums at community.wildernesslabs.co. This site has been preserved for archival purposes only and the ability to make new accounts or posts has been turned off.
Photo

writting to sd card - losing serialdata!


  • Please log in to reply
12 replies to this topic

#1 emence

emence

    Member

  • Members
  • PipPip
  • 17 posts

Posted 09 August 2012 - 02:05 PM

Hello!

I'm receiving serialdata via the eventHandler
static void camera_DataReceived(object sender, SerialDataReceivedEventArgs e)
        {
            lock(serialBuffer){
                int bytesReceived = serial.BytesToRead;
                //Debug.Print("" + bytesReceived);
                if (bytesReceived > 0)
                {
                    serial.Read(serialBuffer, 0, bytesReceived);
                    bufferLength += bytesReceived;
                }
            }
        }

works fine until i try to write the incomming data to the sd card.

            while (watchdog.ElapsedMilliseconds <= timeout && bytesWritten < size60)
            {
              
                fs.Write(serialBuffer, 0, bufferLength); //write received bytes to sd
                bytesWritten += bufferLength;
                Debug.Print("written: " + bytesWritten);
                bufferLength = 0;
               
            }
            fs.Close();

I'm receiving a picture from a camera.
if i lose the line
fs.Write(serialBuffer, 0, bufferLength);
I receive all the bytes of the picture.
When i write the bytes from buffer to SD, i lose some bytes.
I guess the write routine takes to long, and therefore the incoming serial-data can't be written to the buffer.

How can i buffer the incoming data without any loss?

Thanks in advance!

#2 Stefan

Stefan

    Moderator

  • Members
  • PipPipPip
  • 1965 posts
  • LocationBreda, the Netherlands

Posted 09 August 2012 - 02:13 PM

Hi, I see only snapshots of your code, I don't see where "fs" is set. Could it be that it gets destroyed by the garbage collector?
"Fact that I'm a moderator doesn't make me an expert in things." Stefan, the eternal newb!
My .NETMF projects: .NETMF Toolbox / Gadgeteer Light / Some PCB designs

#3 emence

emence

    Member

  • Members
  • PipPip
  • 17 posts

Posted 09 August 2012 - 02:32 PM

Hi,

I see only snapshots of your code, I don't see where "fs" is set. Could it be that it gets destroyed by the garbage collector?


I can write without any problems to the sd card and there are no errors.
I just can't receive all the bytes from the serial, while trying to write them to the sd card.
In the time i write to the sd card, a new set of data would be received. but since i'm still writing to the sd card, the incoming serial data is dismissed.

#4 Chris Walker

Chris Walker

    Secret Labs Staff

  • Moderators
  • 7767 posts
  • LocationNew York, NY

Posted 09 August 2012 - 03:07 PM

Hi emence, What speed are you using for your serial connection? If you use software flow control, are you able to capture all the data? Chris

#5 emence

emence

    Member

  • Members
  • PipPip
  • 17 posts

Posted 15 August 2012 - 03:12 PM

Hi emence,

What speed are you using for your serial connection?

If you use software flow control, are you able to capture all the data?

Chris


I'm on 115200 kbps/baud.
unfortunately the camera doesn't support any flow control at all. As soon as i request an image the camera starts sending one whole frame.

Whats weird is, that the serial-buffer(512bytes on the netduino) doesn't even fill up.
I wait till the bytes in the serial-buffer are > 0 and then immediately write to the SD card. So i write about 40 bytes each time. when i check the serial-Buffer again there are another 20ish bytes to write.
Still, at the end i'm losing data. As soon as I don't write to the SD-Card, every byte will be received. :angry:


static void camera_DataReceived(object sender, SerialDataReceivedEventArgs e)
        {
bytesReceived = serial.BytesToRead;
                if(bytesReceived > 0){
                    serial.Read(serialBuffer, 0, bytesReceived);
                    image.Write(serialBuffer, 0, bytesReceived); //data loss Filestream
                    bufferLength += bytesReceived;
                    Debug.Print("written: " + bufferLength);    
                }
}

I tried lowering the baudrate. still the same problem.
Does the interrupt-Routine (Event Handler) of the serial, interrupt the filestream to the sd card? :blink:

I'm really stuck on this... :(

#6 Stefan

Stefan

    Moderator

  • Members
  • PipPipPip
  • 1965 posts
  • LocationBreda, the Netherlands

Posted 15 August 2012 - 03:22 PM

I'm really stuck on this... :(

Could you maybe post the full project? It's hard to help you out with only one method available to analyse, especially when using global variables.
"Fact that I'm a moderator doesn't make me an expert in things." Stefan, the eternal newb!
My .NETMF projects: .NETMF Toolbox / Gadgeteer Light / Some PCB designs

#7 emence

emence

    Member

  • Members
  • PipPip
  • 17 posts

Posted 15 August 2012 - 03:46 PM

Could you maybe post the full project? It's hard to help you out with only one method available to analyse, especially when using global variables.


Sure, I can. If you really have the time :D I'm glad to.

Attached Files



#8 Stefan

Stefan

    Moderator

  • Members
  • PipPipPip
  • 1965 posts
  • LocationBreda, the Netherlands

Posted 15 August 2012 - 04:21 PM

Thermocam.cs:

public  class Program
    {
        //static SerialPortHelper serial1;
        static SerialPort serial1;
        static SerialPort serial2;
        static ThermalPrinter printer;
        static Ucamttl camera;

        static FileStream fs = File.Create("\\SD\\picture.raw");
        
        public static void Main()
        {
           
Is there a reason Program isn't declared Static, while all methods and properties inside are static?

Also, you're overwriting the first values all the time, in Ucamttl.cs:
public class Ucamttl
    {
        static SerialPort serial;
        static UInt16 _packageSize;
        static int bufferLength = 0;
        const int bufferMax = 513;
        static byte[] serialBuffer = new byte[bufferMax];
        static byte[] _command = new byte[Constants.CMD_SIZE];
        static Stopwatch watchdog = new Stopwatch();

        public Ucamttl(SerialPort sp)
        {
            serial = sp;
            serial.DataReceived += new SerialDataReceivedEventHandler(camera_DataReceived);
            watchdog.Start();

        }

        static void camera_DataReceived(object sender, SerialDataReceivedEventArgs e)
        {

            int bytesReceived = serial.BytesToRead;
            if (bytesReceived > 0)
            {
                serial.Read(serialBuffer, 0, bytesReceived);
                bufferLength += bytesReceived;
            }
        }

I think you want to append to the buffer, not overwrite it all the time.
I'm not sure which camera you're using? I know here is some code for Adafruits TTL camera, maybe that's helping?
"Fact that I'm a moderator doesn't make me an expert in things." Stefan, the eternal newb!
My .NETMF projects: .NETMF Toolbox / Gadgeteer Light / Some PCB designs

#9 emence

emence

    Member

  • Members
  • PipPip
  • 17 posts

Posted 17 August 2012 - 09:38 AM

Thanks for your time!

Is there a reason Program isn't declared Static, while all methods and properties inside are static?


No reason...

I think you want to append to the buffer, not overwrite it all the time.


Did that.
this allowed me to save a smaller picture of 19200 bytes directly to the netduino.
and write it to the SD, after the picture was fully transferred.

Since I can't allocate 76800 bytes for a 320x240 picture I would need to Thread the SD-write process somehow, without interrupting the eventhandler from the serial...

There are only 60KB RAM right? I'd need 76.8KB :P

#10 Stefan

Stefan

    Moderator

  • Members
  • PipPipPip
  • 1965 posts
  • LocationBreda, the Netherlands

Posted 17 August 2012 - 11:16 AM

No reason...

Maybe you should change that, this could cause an issue with the garbage collector, although I'm not sure. Best to keep your code consistent though.

Did that.
this allowed me to save a smaller picture of 19200 bytes directly to the netduino.
and write it to the SD, after the picture was fully transferred.

No you didn't :)
Your buffer doesn't sync right, that's the main issue I think.

public bool GetPicture(UInt32 pictureSize, FileStream fs)
That one writes the file;
static void camera_DataReceived(object sender, SerialDataReceivedEventArgs e)
That one reads the image;

There's a big chance camera_DataReceived is triggered twice before GetPicture gets some processor time. That way, you'll have data loss.
"Fact that I'm a moderator doesn't make me an expert in things." Stefan, the eternal newb!
My .NETMF projects: .NETMF Toolbox / Gadgeteer Light / Some PCB designs

#11 emence

emence

    Member

  • Members
  • PipPip
  • 17 posts

Posted 17 August 2012 - 01:15 PM

No you didn't

sorry. i meant, i did that now... it looks like this.

static void camera_DataReceived(object sender, SerialDataReceivedEventArgs e)
        {
            int bytesReceived = serial.BytesToRead;
            if (isPicture && bytesReceived > 0)
            {
              //  incomingSerial = true;
                
                serial.Read(Buffer, 0, bytesReceived);
                System.Array.Copy(Buffer, 0, serialBuffer, bufferLength, bytesReceived);

               // serial.Read(serialBuffer, bufferLength, bytesReceived);
                bufferLength += bytesReceived;
                Debug.Print("" + bufferLength);
               // incomingSerial = false;
            }
            else
            {
                if (bytesReceived > 0)
                {
                    serial.Read(serialBuffer, 0, bytesReceived);
                    bufferLength += bytesReceived;
                }
            }
}

Your buffer doesn't sync right, that's the main issue I think.


how can i check?!!!
i tried waiting until the serialbuffer on the netduino 512byte is filled up and write to the SD card while it's filling up. but most of the time there were only ~32bytes in the buffer then it stopped.
only after reading the serial, the serialbuffer from the serial filled again up to about 32bytes.

like i said

I would need to Thread the SD-write process somehow, without interrupting the eventhandler from the serial...


You've been a great help so far helping me comprehend what's going on. Thanks a lot!

#12 Stefan

Stefan

    Moderator

  • Members
  • PipPipPip
  • 1965 posts
  • LocationBreda, the Netherlands

Posted 17 August 2012 - 01:33 PM

I would move the code that writes the file to the camera_DataReceived method too. Or at least call the writing of the file in that method. That way you'll keep code execution a bit more under control.
"Fact that I'm a moderator doesn't make me an expert in things." Stefan, the eternal newb!
My .NETMF projects: .NETMF Toolbox / Gadgeteer Light / Some PCB designs

#13 emence

emence

    Member

  • Members
  • PipPip
  • 17 posts

Posted 17 August 2012 - 03:18 PM

I would move the code that writes the file to the camera_DataReceived method too. Or at least call the writing of the file in that method. That way you'll keep code execution a bit more under control.


I will do that, and post any helpful results here. Maybe its of any use someday...




0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users

home    hardware    projects    downloads    community    where to buy    contact Copyright © 2016 Wilderness Labs Inc.  |  Legal   |   CC BY-SA
This webpage is licensed under a Creative Commons Attribution-ShareAlike License.