Buffered file reading
[1/2] from: mat:eurogamer at: 6-Apr-2001 19:02
Hi Guys,
I'm making a sort of batch URL mover in Rebol. It will take it's
commands via E-mail and download files from the net and upload them
where specified.
Naturally we need to use some sort of buffered scheme. I asked about
this before and drew a blank but then I found this document on
rebol.com: http://www.rebol.com/docs/network.html
Take a look at the following extract;
<-- Cut -->
Transferring large files requires special considerations. You may want
to transfer the file in chunks to reduce the memory required by your
computer and to provide user feedback while the transfer is happening.
Here is an example that downloads a very large binary file in chunks.
inp: open/binary/direct ftp://ftp.site.com/big-file.bmp
out: open/binary/new/direct %big-file.bmp
buf-size: 200000
buffer: make binary! buf-size + 2
while [not zero? size: read-io inp buffer buf-size][
write-io out buffer size
total: total + size
print ["transferred:" total]
]
<-- Cut -->
Well no, that code obviously wont ever work. (total not initialised)
But it's probably close.
The odd thing is that it only transfers about 200000 and I'm not sure
why that would be. It only seems to read a few K at a time and I've no
idea why size would be zero when the magical number of 200000 is hit.
Any ideas?
--
Mat Bettinson - EuroGamer's Gaming Evangelist with a Goatee
http://www.eurogamer.net | http://www.eurogamer-network.com
[2/2] from: mat::eurogamer::net at: 6-Apr-2001 19:13
Heya Mat,
MB> inp: open/binary/direct ftp://ftp.site.com/big-file.bmp
MB> out: open/binary/new/direct %big-file.bmp
MB> buf-size: 200000
MB> buffer: make binary! buf-size + 2
MB> while [not zero? size: read-io inp buffer buf-size][
MB> write-io out buffer size
MB> total: total + size
MB> print ["transferred:" total]
MB> ]
Actually, I got this working;
inp: open/binary/direct ftp://ftp.site.com/big-file.bmp
out: open/binary/new/direct %big-file.bmp
buf-size: 200000
buffer: make binary! buf-size + 2
total: 0
while [not zero? size: read-io inp buffer buf-size][
write-io out buffer size
clear buffer
total: total + size
print ["transferred:" total]
]
close inp
close out
This works. The notable difference being the clear buffer, which was
actually just a guess from me since I don't really know what it does
other than resetting a pointer.
--
Mat Bettinson - EuroGamer's Gaming Evangelist with a Goatee
http://www.eurogamer.net | http://www.eurogamer-network.com