[REBOL] Re: hook while downloading from html port [http][port]
From: moliad::aei::ca at: 15-Apr-2004 23:39
Scott,
this is gold!
I added a tag on the subject to be sure rebol.org indexes this properly, this
really is a worth-while exercise, thanks for all the time you put on this, it
will surely open up the mysterious port handling for many rebolers out there!
-MAx
----- Original Message -----
From: "Jones, Scott"
Sent: Thursday, April 15, 2004 8:35 PM
Subject: [REBOL] Re: hook while downloading from html port
From: Anton Rolls
AR> Scott, I think you are a bit negative about transfer rates. :)
AR> Surely there is a way.
....
It was one of those things were as soon as I sent my email, I, too, thought of a
way to show progress, but our satellite signals times are limited. My guess is
that there will already be 3 more ways to do it by the time this arrives!
Not unlike Anton's way in using read, I capture the actual bytes read from the
return of read-io and accumulate these in a variable. If the site provides
content length, then the patch can compare sent to expected. Unfortunately, not
all servers/websites send the content length.
At the end of the http scheme, and the variable "bytes-transferred", as in:
...
cache-size: 5
user-agent: "REBOL View 1.2.10.3.1"
bytes-transferred: 0
]
Then change the read function to (watch for line wrap):
read: func [
port "An open port spec"
data "A buffer to use for the read"
/local bytes-read
][
net-utils/net-log ["low level read of " port/state/num "bytes"]
bytes-read: read-io port/sub-port data port/state/num
bytes-transferred: bytes-transferred + bytes-read
if port/locals/headers/content-length <> none [print [to-integer
(bytes-transferred / (to-integer port/locals/headers/content-length) * 100)
percent
]]
bytes-read
]
Reading a page in which the server gives content-length will allow the
calculation of percent received. I realized that the read function needs to
return the bytes, read, hence the last line.
Expanding on the idea, I decided for fun to try to create a visual feedback
method that gives progress bar when the content-length is known, or simple
accumulated bytes retrieved, when content-length was unknown. It requires a
fair amount of patching.
Starting with a fresh copy of the http scheme, one first alters the
continue-post function:
continue-post: func [/tunnel] [
response-line: system/words/pick port/sub-port 1
net-utils/net-log response-line
either none? response-line [do error] [
either none? result: select either tunnel [tunnel-actions]
[response-actions]
response-code: to-integer second parse response-line none [do error] [do
get result]
]
if show-progress [
either port/locals/headers/content-length <> none [
progress-message/pane: layout/origin/offset [p: progress] 0x0 0x0
show progress-message
][
progress-message/pane: layout/origin/offset [t: text 200x16] 0x0 0x0
show progress-message
]
]
]
Then following the close block and build-port further down, one inserts these
lines
....
]
build-port ;insert following lines
if all [block? port/state/custom post-data: find port/state/custom to-lit-word
'progress block? post-data] [
show-progress: true
progress-message: get to-word select port/state/custom to-lit-word 'progress
]
Then one changes the close function:
close: func [port][
if show-progress [
progress-message/pane: layout/origin/offset [t: text 200x16] 0x0 0x0
show progress-message
t/text: "Done!"
show t
show-progress: none
]
system/words/close port/sub-port
bytes-transferred: 0
]
One changes the read function:
read: func [
port "An open port spec"
data "A buffer to use for the read"
/local bytes-read
][
net-utils/net-log ["low level read of " port/state/num "bytes"]
bytes-read: read-io port/sub-port data port/state/num
bytes-transferred: bytes-transferred + bytes-read
if show-progress [
either port/locals/headers/content-length <> none [
p/data: bytes-transferred / (to-integer
port/locals/headers/content-length)
show p
;loop 10000000 [] ;uncomment to see fast pages load
][
t/text: bytes-transferred
show t
;loop 10000000 []
]
]
bytes-read
]
Finally, one adds variables to the end of the scheme:
.....
user-agent: "REBOL View 1.2.10.3.1"
bytes-transferred: 0
show-progress: progress-message: p: t: none
]
One uses the custom refinement to pass the reference to a VID box:
view layout [
f: field 300x24
button "Get Page" [
read/custom trim to-url f/text ['progress :my-pane]
]
my-pane: box 200x16
button "Exit" [quit]
]
When one types in a url from a website where the server gives the
content-length, the box gets a progress bar embedded, which is then updated
based on the percentage of bytes read. When the url is served from a server
that does not give the content-length, then the bytes retrieved are displayed.
After completed, the progress box is converted to text that says "Done!" I have
commented out some do nothing loops in case your downloads are too fast, and you
are trying to see the changes at work.
I was just about to send this when I received the most recent set of emails,
including Max's own patch on read-net. Oh well, it still may prove useful to
him or someone else in another context and/or to take advantage of all the extra
features of the http protocol.
--Scott Jones
