Mailing List Archive: 49091 messages
  • Home
  • Script library
  • AltME Archive
  • Mailing list
  • Articles Index
  • Site search
 

wget - was: Newbie needs some guidance...

 [1/2] from: antonr::iinet::net::au at: 24-Dec-2003 20:44


As seen here http://www.gnu.org/software/wget/wget.html it seems the desired behaviour is to mirror a site beginning at a url. Rebol does not have something like wget built in. However, you can make a recursive function of your own to do the job. I wanted to do something like this for a while, but only got to the stage of extracting the links from a single url. See: http://www.lexicon.net/anton/rebol/web/extract-html-links.r After extracting the links, you need to filter out the external links and keep the internal links. I've got some functions lying around somewhere for that.. Then save and recurse on the internal links. Anton.

 [2/2] from: SunandaDH:aol at: 24-Dec-2003 5:40


Anton:
> I wanted to do something like > this for a while, but only got to the stage of > extracting the links from a single url.
There's a complete spider here: http://www.searchlores.org/sono_bot.htm (I've not tried it, just bookmarked it a while back), Sunanda