Working with large files and Memory

FROM : Carl E. McIntosh
DATE : Tue Mar 11 17:54:51 2008

Can you please give advice about handling large data files with memory 
management techniques? I am attempting to read three large files (1 
GB, 208 MB, 725 MB) sequentially and place the data into arrays for 
processing. Here is my psuedocode:

1) Import a file into NSString.
      NSString *aFileString = [NSString stringWithContentsOfFile: 
fileLocation];    // Convert file at path to myFileString text holder;

2) Use NSScanner pull out integers and floats
      NSScanner *aFileScanner = [[NSScanner alloc] initWithString: 
aFileString];

3) Store values into arrays.
      float myFloats [100000][2000]; or
      float myInts [100000][2000];

4) repeat three times with 3 different files.

This algorithm works for smaller files but chokes on the larger files 
and I get malloc errors. I've attempted to use NSZone's to the same 
failure.

Can you please give advice about handling large data files with memory 
management techniques?

I have 4 GB ram and can hog off 2 - 3 GBs for the process. I don't 
know how to explicitly allocate real memory. I'd rather not use 
virtual memory. Any references or examples would be appreciated.


Thank you.
Carl