8 replies to this topic

[per123]

Using NMF 4.2.2 on Plus2.

 

I cannot get SreamReader.Readline to read more than 185 lines from a text file before it throws an

'System.IndexOutOfRangeException' occurred in System.IO.dll

 

/*

// 4096 lines of values to be read from "myfile.txt"

14837,80

14845,80

14854,80

14862,80

14871,80

14879,80

14888,81

14896,81

14905,81

14913,81

14921,81

.......

.......

// total of 4096 lines

*/

/*

 

A first chance exception of type 'System.IndexOutOfRangeException' occurred in System.IO.dll An unhandled exception of type 'System.IndexOutOfRangeException' occurred in System.IO.dll

*/

static void xx()

{

string name = "SDmyfile.txt";

int line = 0;

using (FileStream fs = File.OpenRead(name)) {

StreamReader strm = new StreamReader(fs);

do {

string s = strm.ReadLine();

line++;

} while (strm.EndOfStream == false);

fs.Close();

}

}

 

When debugger breaks because of exception these are the values of StreamReader

BaseStream.Position = 2048

BaseStream.Length = 46755

BaseStream.EndOfStream = 1

m_curBufLen = 512

m_curBufPos = 512

GC reports 73500 kb free memory

 

Looks to me that there is a bug in streamReader.

The Exception is thrown at strm.ReadLine()

 

Can you try the above on a Plus2 to verify that my code is okay ?

Or am I doing something wrong ??

 

regards

Per

 

[per123]

Is there nobody that can help me with the StreamReader problem ?

[Shadi]

I have confirmed this defect in StreamReader.

 

It looks like you will have to read the filestream directly to get around it.

 

[color=#0000FF;]int[/color] lineCount = 0;
[color=#0000FF;]var[/color] files = System.IO.[color=#2b91af;]Directory[/color].GetFiles([color=#a31515;]"SD"[/color]);
[color=#0000FF;]using[/color] ([color=#0000FF;]var[/color] fs = System.IO.[color=#2b91af;]File[/color].OpenRead(files[0]))
{
[color=#0000FF;]var[/color] buffer = [color=#0000FF;]new[/color] [color=#0000FF;]byte[/color][1024];
[color=#0000FF;]int[/color] readLen = 0;
[color=#0000FF;]while[/color] ((readLen = fs.Read(buffer, 0, buffer.Length)) > 0)
{
[color=#0000FF;]for[/color] ([color=#0000FF;]int[/color] x = 0; x < readLen; ++x)
{
[color=#0000FF;]if[/color] (buffer[x] == [color=#a31515;]'r'[/color])
{
lineCount++;
}
}
}
}
[color=#2b91af;]Debug[/color].Print(files[0] + [color=#a31515;]" - Lines: "[/color] + lineCount.ToString());

[per123]

Thanks for an alternate way of reading the data. I have not thought of that

 

Per

[Chris Walker]

Hi Per, Very interesting. I've logged an internal bug report on this, and we'll raise it to the NETMF team in time as well. If the workaround is not working for you, please feel free to open a work item on netmf.codeplex.com. Chris

[Shadi]

I tracked it down in the Netduino source code:

 

StreamReader.Peek()

m_curBufPos is 511

m_curBufLen is 512

m_buffer is byte[512]

 

[color=#0000FF;]   if[/color] (m_curBufPos == m_curBufLen || (([color=#ff0000;]m_buffer[m_curBufPos + 1][/color] & 0x80) != 0 && m_curBufPos + 3 > m_curBufLen))

 

As you know, in a 512 length array, there is no position 512.

So if we change the code to:

  [color=#0000FF;]if[/color] (m_curBufPos <= (m_curBufLen - 1) || ((m_buffer[m_curBufPos + 1] & 0x80) != 0 && m_curBufPos + 3 > m_curBufLen))

 

There is also an endless loop inside Peek():

[color=#008000;]// Retry read until response timeout expires[/color]

[color=#0000FF;]if [/color](m_stream.Length > 0) [color=#0000FF;]while[/color] (i < m_buffer.Length && (noRead = m_stream.Read(m_buffer, i, ([color=#0000FF;]int[/color])(m_buffer.Length - i < m_stream.Length ? m_buffer.Length - i : m_stream.Length))) == 0)
{
  i += noRead; [color=#ff0000;]// gets stuck here[/color]
}

}

 

Fixing that with:

[color=#0000FF;]if[/color] (m_stream.Length > 0 && (m_stream.Position != m_stream.Length))

 

However, this probably breaks if you are reading from a network stream or something.

 

Anyways, here's a quick & dirty line reader:

http://pastebin.com/GpNy0Yr1

[per123]

Thanks...

Your fix works fine.

One thing to remember is to skip the first 3 bytes at the beginning of the file when reading an ASCII file. Somehow this is garbage.

 

Per

[Shadi]

those first 3 bytes are the "Byte Order Mark" for UTF-8 text files. it's going to be something like "" (0xEF 0xBB 0xBF)

[untitled]

I modified my StreamReader as directed, but that did NOT solve the problem for me.  Looking at the code Peek was only called when rn was encountered.  Since I'm writing the files I needed to read, I changed my write code to use n instead of rn.  Seems like that solved the problem for me.  ...But I'd love to know why my StreamReader is broken.

 

EDIT:

Also, I noticed my files are being modified.  I use the LastModifiedDate to sort my data on-screen.  Apparently there's some code in the StreamReader class that does this.  Maybe this bcl bug is related...

 

http://connect.micro...k-modifies-file

 

Any idea how I can avoid this?