tired of broken downloads?

Hello, in the age of advanced Internet browsers, we still can meet with very lame downloaders they provide. They offer no ability to start at the place where download was broken. So I created following 1KB in size small util. As you can see - no GUI. I tried, but I spent most of my time playing with look, so I gave up. It is simple, small, does what it is supposed to do, well, mostly. Every time I think I finally understand how something works, I stay starving at console, getting unexpected results. So, the script as it is seems to work. Just try to answer my following questions: 1) I tried to use simply "data: copy source" aproach. The strange thing is, that while I use /no-wait, it should perform body block of 'while. But I can see console hanging, and the body block is entered only once - once file is completly downloaded. But - 'copy should not block here, no? 2) With larger file, I obtain network time-out. So I tried to use some kind of "not error? try [data: copy source]", but well, once port is closed by the other side, 'none is generated, but 'none itself doesn't return false(error) state to 'try. So - how to adapt the script, so I could safely go home, and let my script to download-damned-thing-even-in-million-pieces, safely? :-) ... I turned it back to copy/part aproach, but - is open/no-wait on http port broken? Why does 'copy block, or - what happens at all? Any explanation appreciated. PS: used write/binary/append, untill open/skip works as expected .... -pekr- ------------------------------- REBOL [] source-url: http://www.rebol.cz/~cyphre/ source-file: %styles2.jpg target-path: %./ target-file: source-file source-size: size? join source-url source-file either exists? join target-path target-file [ target-size: size? join target-path target-file ][target-size: 0] if target-size == source-size [print "Nothing to download, file already complete ..." halt] if target-size > 0 [print ["Appending at " target-size "bytes"]] start: now/time forever [ while [error? try [source: open/binary/direct/no-wait/skip join source-url source-file target-size]][ print "Can't open source file .... waiting 10 min" wait 00:10 ] while [ wait source data: copy/part source 2048 ][ write/binary/append join target-path target-file data target-size: target-size + length? data print ["Source-size: " source-size tab "Downloaded: " target-size tab "Time: " now/time - start] ] if target-size >= source-size [break] ] close source

Hi Pekr source read-net open url set-modes port/sub-port [lines: false binary: true no-wait: true] seems open http:// goes to wait-mode to get the header. after that is done one can set-mode[no-wait: true]. -Volker Am Freitag, 16. August 2002 07:57 schrieb Petr Krenzelok:

; replying myself with updated versions .... downloaded safely more than 100 MB today, correctly appends to interrupted downloads, etc. REBOL [ Title: "download-it!" Author: "Petr Krenzelok" Email: [petr--krenzelok--trz--cz] File: %download-it!.r Version: 1.0 Comment: {Tries to download file, till complete. Still place for improvements: - GUI - not necessary imo - log events to file? - download list of files? - callbacks? E.g. notification by sending email, sms, etc. - whatever ... as for me though - it works, so - finished ... :-) } ] source-file: to-url ask "Paste URL: " target-file: first request-file/title "Save (append) to ..." "Select" source-size: size? source-file either exists? target-file [target-size: size? target-file][target-size: 0] if target-size == source-size [print "Nothing to download, file already complete ..." halt] if target-size > 0 [print ["Appending at " target-size "bytes"]] start: now/time forever [ while [error? try [source: open/binary/direct/no-wait/skip source-file target-size]][ print "Can't open source file .... waiting 10 min" wait 00:10 ] while [ wait source all [ not error? try [data: copy/part source 8192] data ] ][ write/binary/append target-file data target-size: target-size + length? data print ["Source-size: " source-size tab "Downloaded: " target-size tab "Time: " now/time - start] ] if target-size >= source-size [print "Download complete ..." break] print "Download interrupted ..." print ["Continuing at " target-size "bytes"] close source ] close source

Hello Petr, Friday, August 16, 2002, 7:57:35 AM, you wrote: PK> Hello, PK> in the age of advanced Internet browsers, we still can meet with very PK> lame downloaders they provide. They offer no ability to start at the PK> place where download was broken. So I created following 1KB in size PK> small util. As you can see - no GUI. I tried, but I spent most of my PK> time playing with look, so I gave up. It is simple, small, does what it PK> is supposed to do, well, mostly. PK> Every time I think I finally understand how something works, I stay PK> starving at console, getting unexpected results. So, the script as it is PK> seems to work. Just try to answer my following questions: PK> 1) I tried to use simply "data: copy source" aproach. The strange thing PK> is, that while I use /no-wait, it should perform body block of 'while. PK> But I can see console hanging, and the body block is entered only once - PK> once file is completly downloaded. But - 'copy should not block here, no? PK> 2) With larger file, I obtain network time-out. So I tried to use some PK> kind of "not error? try [data: copy source]", but well, once port is PK> closed by the other side, 'none is generated, but 'none itself doesn't PK> return false(error) state to 'try. So - how to adapt the script, so I PK> could safely go home, and let my script to PK> download-damned-thing-even-in-million-pieces, safely? :-) PK> ... I turned it back to copy/part aproach, but - is open/no-wait on http PK> port broken? Why does 'copy block, or - what happens at all? Any PK> explanation appreciated. PK> PS: used write/binary/append, untill open/skip works as expected .... PK> -pekr- PK> ------------------------------- PK> REBOL [] PK> source-url: http://www.rebol.cz/~cyphre/ PK> source-file: %styles2.jpg PK> target-path: %./ PK> target-file: source-file PK> source-size: size? join source-url source-file PK> either exists? join target-path target-file [ PK> target-size: size? join target-path target-file PK> ][target-size: 0] PK> if target-size == source-size [print "Nothing to download, file already PK> complete ..." halt] if target-size >> 0 [print ["Appending at " target-size "bytes"]] PK> start: now/time PK> forever [ PK> while [error? try [source: open/binary/direct/no-wait/skip join PK> source-url source-file target-size]][ PK> print "Can't open source file .... waiting 10 min" PK> wait 00:10 PK> ] PK> while [ PK> wait source PK> data: copy/part source 2048 PK> ][ PK> write/binary/append join target-path target-file data PK> target-size: target-size + length? data PK> print ["Source-size: " source-size tab "Downloaded: " target-size PK> tab "Time: " now/time - start] PK> ] PK> if target-size >= source-size [break] PK> ] PK> close source I thought that the 'skip on binary/direct modes is broken, but not sure... I was thinking to use somethink like that in my downloader script as well (skiping already downloaded parts) but not started yet.... BTW: you should also check if the file was not modified since your last downloading. And your script will be also unusable on some files where is no info available (usualy generated files) And one more comment: I'm not sure but maybe it will be better to make the checking (file info) after opening the port, because now you probably open the port two times.... but I'm not sure because the 'query function that's called on the port is native and the port is closed after that... see

Hi RebOldes, On Friday, August 16, 2002, 11:41:52 AM, you wrote: R> info) after opening the port, because now you probably open the port R> two times.... but I'm not sure because the 'query function that's R> called on the port is native and the port is closed after that... see QUERY is native!, but for protocols written in REBOL (such as HTTP) it just calls the QUERY function in the handler. Last time I looked into it, QUERY for HTTP opened the port and even downloaded the whole file; I assume it has been enhanced now to use the HEAD method instead of GET and so on, but I'd suggest you to check. :-) Regards, Gabriele. -- Gabriele Santilli <[g--santilli--tiscalinet--it]> -- REBOL Programmer Amigan -- AGI L'Aquila -- REB: http://web.tiscali.it/rebol/index.r