Mailing List Archive: 49091 messages
  • Home
  • Script library
  • AltME Archive
  • Mailing list
  • Articles Index
  • Site search
 

Cookies

 [1/5] from: dlmayer:flashcom at: 27-Oct-2000 8:19


I'm new to Rebol and would appreciate some help. I'm writing a webcrawler and am running into a problem with sites that have cookies. Once a cookie is sent my script stops as it can't deal with it. Would appreciate any help on writing the code that will determine if a cookie has been sent and then respond (yes/no) to it. Thanks in advance... Doug Mayer

 [2/5] from: carlos:lorenz at: 27-Oct-2000 14:07


Do you intend to share your code or not? I'm new to REBOL too and would like to learn how to code a useful tool like a webcrawler, for instance :-) from Brazil Carlos ----- Original Message ----- From: Douglas Mayer <[dlmayer--flashcom--net]> To: <[rebol-list--rebol--com]> Sent: Friday, October 27, 2000 1:19 PM Subject: [REBOL] Cookies
> I'm new to Rebol and would appreciate some help. > > I'm writing a webcrawler and am running into a problem with sites that
have cookies. Once a cookie is sent my script stops as it can't deal with it.

 [3/5] from: dlmayer:flashcom at: 27-Oct-2000 14:16


Carlos... I have found the following two scripts on www.rebol.org helpful in starting my development of a webcraller. Web Crawler Tutorial v1.0 Web Miner My code is still very basic. It follows: startsite: ask "URL: " ; the response is the starting url for the search startsite: to-url startsite item: make block! ; hold lines read from site page: read/lines startsite ; read page line-by-line foreach item page [ parse item [thru {href="} copy text to {"} Now when I type in a site that has a cookie the program just hangs. I would very much appreciate someone who could give me advice on how to automatically answer No to cookies and progress through parsing the page. ... Doug ----- Original Message ----- From: Lorenz <[carlos--lorenz--net]> To: <[rebol-list--rebol--com]> Sent: Friday, October 27, 2000 9:07 AM Subject: [REBOL] Re: Cookies Do you intend to share your code or not? I'm new to REBOL too and would like to learn how to code a useful tool like a webcrawler, for instance :-) from Brazil Carlos ----- Original Message ----- From: Douglas Mayer <[dlmayer--flashcom--net]> To: <[rebol-list--rebol--com]> Sent: Friday, October 27, 2000 1:19 PM Subject: [REBOL] Cookies
> I'm new to Rebol and would appreciate some help. > > I'm writing a webcrawler and am running into a problem with sites that
have cookies. Once a cookie is sent my script stops as it can't deal with it.

 [4/5] from: bo:rebol at: 27-Oct-2000 14:42


And don't forget the webcrawler on www.rebol.com in the script library! -Bo On 27-Oct-2000/14:16:27-7:00, [dlmayer--flashcom--net] wrote:
>Carlos... >I have found the following two scripts on www.rebol.org helpful in starting
<<quoted lines omitted: 54>>
>[rebol-request--rebol--com] with "unsubscribe" in the >subject, without the quotes.
-- Bohdan "Bo" Lechnowsky REBOL Adventure Guide REBOL Technologies 707-467-8000 (http://www.rebol.com) The Official Source for REBOL Books (http://www.REBOLpress.com)

 [5/5] from: brett:codeconscious at: 29-Oct-2000 13:52


Hi Doug,
> Now when I type in a site that has a cookie the program just hangs. > > I would very much appreciate someone who could give me advice on how to > automatically answer No to cookies and progress through parsing the page.
I haven't used Rebol for this but I have seen other posts on this topic. I understand there is a script called cookies-client.r that might be able to help you. You could probably find it at www.rebol.org or www.rebol.com. It gives you the abilitity to accept an return cookies to the server. Without this or something similar Rebol does not handle cookies. By not supporting cookies at the moment, in effect you are already saying no - I think. Just saying "No" to the server may not be good enough - depending on the server. Some web sites require cookies, others are more forgiving. So, your program hanging maybe for a different reason. I hope this has helped.. Brett.

Notes
  • Quoted lines have been omitted from some messages.
    View the message alone to see the lines that have been omitted