Mailing List Archive: 49091 messages
  • Home
  • Script library
  • AltME Archive
  • Mailing list
  • Articles Index
  • Site search
 

file splitting howto?

 [1/2] from: notofo::earthlink::net at: 31-May-2004 21:26


Hi guys, I would like to split a large (40m) file into 1m chunks, being sure to split on line breaks. Each chunk will be written to a seperate file. The linux utility "split" does almost what I want, execept for where it breaks the file. does my solution involve open/direct/lines, and copy/part, by any chance? :-) Thanks. -- signature drinking: "glugluglug" -tom

 [2/2] from: antonr:lexicon at: 16-Jun-2004 15:29


Hi Tom, This is not as easy as it sounds. I am close to a solution using open/direct/binary/read And yes, copy/part is used too, but in my algorithm, it is not used to specify the 1MB chunks directly, but smaller chunks (which are later joined). I found a problem was the length of the line terminator can be different by platform. Unix uses LF, Mac uses CR, and Windows uses CRLF (two characters). Being two characters means that a split at the wrong place could split a line terminator in half, so the end of one partial file has the CR, and the beginning of the next partial file has the LF. I am interested what you are splitting. I assume it is a log file of some sort ? Anton.