x


Optimize the reading of a text file?

I've followed a lot of suggestions for optimizing graphics performance, and my projects currently sails along quickly and lightly in terms of rendering.

I do, however, have some overhead on loading that I'd like to minimize if possible. It comes down to reading a text file of objects and coordinates (this approach is questionable, but let's stick with it as a given).

I'm reading in a file that contains a subset of lines I'm interested in ("MARK" being the intial characters). Then, only a subset of these lines, I want to store coordinate information for - I check for the existence of the line type using a single character from the line string in a hash I populate at the start of the code.

I'm using builtin arrays for everything below.

I have the following:

if (textAsset == null) return;
reader = new StringReader(textAsset.text);
line = reader.ReadLine();

while (line != null)
{

    if(line.Substring(0,4) == "MARK")
    {

        var identifier = Regex.Replace(line.Substring(13,4),"( )+","");

        if (lineType[identifier])
        {
            i++;

            totalCounts++;


    vec[i] = Vector3(float.Parse(line.Substring(30,8)),float.Parse(line.Substring(38,8)),float.Parse(line.Substring(46,8)));
    vecSum += Vector3(float.Parse(line.Substring(30,8)),float.Parse(line.Substring(38,8)),float.Parse(line.Substring(46,8)));

            print(totalCounts);
            print(DateTime.Now);

        }
    }
}

I'm clocking about 200 lines read per second. I'm wondering if anyone sees anything in the above code that could be streamlined for a time gain?

Cheers, and thanks for reading this far!

more ▼

asked Jul 05 '10 at 03:54 PM

pickledzebra gravatar image

pickledzebra
286 31 35 44

(comments are locked)
10|3000 characters needed characters left

1 answer: sort voted first

First thing i see - you're parsing the vectors twice each - store the intermediary vector, then use it for vec[i] and adding to vecSum

Second, you really should be using the overload of float.Parse that takes a CultureInfo object - otherwise on european computers, it'll have problems reading US/UK floats (and vice versa)

Third, you're printing two items each time - that'll slow you down ridiculously (it has to print the current stacktrace to a file, as well as displaying it in the console), so remove those if you're testing performance

more ▼

answered Jul 05 '10 at 04:16 PM

Mike 3 gravatar image

Mike 3
30.5k 10 65 253

Thanks for the pointer on #2. #3 is just temporary while I figure this out.

Jul 05 '10 at 04:27 PM pickledzebra

Hmmm... fixed the #1 and deleted #3. Neither reduces the time outside the variance of total time from run to run.

Jul 05 '10 at 04:35 PM pickledzebra

I think parsing the input text file down will get me to where I need to go. I was hoping the readin would be sufficiently fast that I could ignore large chunks of non-interesting data in my input files. Thanks for looking, mike.

Jul 05 '10 at 04:43 PM pickledzebra

I'm assuming that you've skipped the bit of code which stops the above being an infinite loop though - line never gets modified, so it'll just hang. Also - I'm not sure if reading line by line will be faster than reading once then splitting, if you don't need to break out early

Jul 05 '10 at 05:05 PM Mike 3
(comments are locked)
10|3000 characters needed characters left
Your answer
toggle preview:

Up to 2 attachments (including images) can be used with a maximum of 524.3 kB each and 1.0 MB total.

Follow this question

By Email:

Once you sign in you will be able to subscribe for any updates here

By RSS:

Answers

Answers and Comments

Topics:

x555
x243
x78
x45

asked: Jul 05 '10 at 03:54 PM

Seen: 1656 times

Last Updated: Jul 05 '10 at 03:54 PM