file splitting howto?
[1/2] from: notofo::earthlink::net at: 31-May-2004 21:26
Hi guys,
I would like to split a large (40m) file into 1m chunks,
being sure to split on line breaks. Each chunk will be
written to a seperate file. The linux utility "split" does
almost what I want, execept for where it breaks the file.
does my solution involve open/direct/lines, and copy/part,
by any chance? :-)
Thanks.
--
signature drinking: "glugluglug"
-tom
[2/2] from: antonr:lexicon at: 16-Jun-2004 15:29
Hi Tom,
This is not as easy as it sounds.
I am close to a solution using
open/direct/binary/read
And yes, copy/part is used too, but in my algorithm,
it is not used to specify the 1MB chunks directly, but
smaller chunks (which are later joined).
I found a problem was the length of the line terminator
can be different by platform.
Unix uses LF, Mac uses CR, and Windows uses CRLF (two characters).
Being two characters means that a split at the wrong place could
split a line terminator in half, so the end of one partial file
has the CR, and the beginning of the next partial file has the LF.
I am interested what you are splitting. I assume it is a log
file of some sort ?
Anton.