Mailing List Archive: 49091 messages
  • Home
  • Script library
  • AltME Archive
  • Mailing list
  • Articles Index
  • Site search

[REBOL] Re: using 'open to read url redirection

From: ryan:christiansen:intellisol at: 28-Jun-2001 9:08

----- Forwarded by Ryan Christiansen/Intellisol on 06/28/2001 09:03 AM ----- Ryan Christiansen To: e [rebol-list--rebol--com] cc: 06/28/2001 Subject: Re: [REBOL] Re: using 'open to read url redirection(Document 09:08 AM link: Ryan Christiansen) Nope. That didn't solve the problem. I added the line http-get-header/host: port/host to system/ schemes/http/handler according to the instructions given in the discussion list archive URL you pointed out. The 'open operation still seems to get "stuck" on the URL. If it would just error out for me, that would be great, because my script could continue. But it doesn't error out, which makes things "hang." Any other suggestions? -Ryan From: Ryan Christiansen
> I have a script which uses the following to grab a redirect url from a > server... > > port: open > redirect-url: make url! probe port/url > > But for some reason the 'open operation gets "stuck" when using this > particular URL. REBOL just keeps reconnecting to the URL endlessly, it > never errors out.
Hi, Ryan, Perhaps the following link will help. --Scott Jones REBOL [] system/schemes/http/handler: make object! [ port-flags: 0 open-check: none close-check: none write-check: none init: func [ "Parse URL and/or check the port spec object" port "Unopened port spec" spec {Argument passed to open or make (a URL or port-spec)} /local scheme ][ if url? spec [net-utils/url-parser/parse-url port spec] scheme: port/scheme port/url: spec if none? port/host [ net-error reform ["No network server for" scheme "is specified"] ] if none? port/port-id [ net-error reform ["No port address for" scheme "is specified"] ] ] open: func [ port "the port to open" /local http-packet http-command response-actions success error response-line target headers http-version post-data result][ port/locals: make object! [list: copy [] headers: none] build-port: func [] [ open-proto port port/url: rejoin ["http://" port/host either port/port-id <> 80 [join #":" port/port-id] [copy ""] slash] if found? port/path [append port/url port/path] if found? port/target [append port/url port/target] ] build-port http-command: "GET" HTTP-Get-Header: make object! [ Accept: "*/*" Connection: "close" User-Agent: system/schemes/http/user-agent Host: join port/host any [all [port/port-id (port/port-id <> 80) join #":" port/port-id] #] ] HTTP-Header: make object! [ Date: Server: Last-Modified: Accept-Ranges: Content-Encoding: Content-Type: Content-Length: Location: Expires: Referer: Connection: Authorization: none ] do create-request: func [/local target user pass u] [ http-version: "HTTP/1.0^/" all [port/user port/pass HTTP-Get-Header: make HTTP-Get-Header [Authorization: join "Basic " enbase join port/user [#":" port/pass]]] user: get in port/proxy 'user pass: get in port/proxy 'pass if (port/proxy/type = 'generic) and (string? :user) and (not none? port/proxy/host) [ HTTP-Get-Header: make HTTP-Get-Header [ Proxy-Authorization: join "Basic " enbase join user [#":" pass] ] ] if port/state/index > 0 [ http-version: "HTTP/1.1^/" HTTP-Get-Header: make HTTP-Get-Header [ Range: rejoin ["bytes=" port/state/index "-"] ] ] target: next mold to-file join (join "/" either found? port/path [port/path] [""]) either found? port/target [port/target] [""] post-data: none if all [block? port/state/custom post-data: find port/state/custom 'post post-data/2] [ http-command: "POST" HTTP-Get-Header: make HTTP-Get-Header append [ Referer: either find port/url #"?" [head clear find copy port/url #"?"] [port/url] Content-Type: "application/x-www-form-urlencoded" Content-Length: length? post-data/2 ] either block? post-data/3 [post-data/3] [[]] post-data: post-data/2 ] http-packet: reform [http-command either port/proxy/type = 'generic [port/url] [target] http-version] append http-packet net-utils/export HTTP-Get-Header append http-packet "^/" if post-data [append http-packet post-data] ] send-and-check: func [] [ net-utils/net-log http-packet insert port/sub-port http-packet do continue-post: does [ response-line: system/words/pick port/sub-port 1 net-utils/net-log response-line either none? response-line [do error] [ either none? result: select response-actions response-code: to-integer second parse response-line none [ do error] [ do get result] ] ] ] response-actions: [ 100 continue-post 200 success 201 success 204 success 206 success 300 forward 301 forward 302 forward 304 success 407 proxyauth ] success: [ headers: make string! 500 while [(line: pick port/sub-port 1) <> ""] [append headers join line "^/"] port/locals/headers: headers: Parse-Header HTTP-Header headers port/size: 0 if querying [if headers/Content-Length [port/size: load headers/Content-Length]] if error? try [port/date: parse-header-date headers/Last-Modified] [port/date: none] port/status: 'file ] error: [ system/words/close port/sub-port net-error reform ["Error. Target url:" port/url "could not be retrieved. Server response:" response-line] ] forward: [ page: copy "" while [(str: pick port/sub-port 1) <> ""] [append page reduce [str newline]] headers: Parse-Header HTTP-Header page insert port/locals/list port/url either found? headers/Location [ either find/match headers/Location "http://" [ port/path: port/target: none net-utils/URL-Parser/parse-url port to-url port/url: headers/Location ] [ either (first headers/Location) = slash [port/path: none remove headers/Location] [either port/path [insert port/path "/"] [port/path: copy "/"]] port/target: headers/Location port/url: join http:// reduce [port/host either port/path [port/path] [""] either port/target [port/target] [""]] ] if find/case port/locals/list port/url [net-error reform ["Error. Target url:" port/url {could not be retrieved. Circular forwarding detected}]] system/words/close port/sub-port build-port http-get-header/host: port/host create-request send-and-check ] [ do error] ] proxyauth: [ system/words/close port/sub-port either all [(port/proxy/type = 'generic) port/proxy/host (not string? get in port/proxy 'user)] [ port/proxy/user: system/schemes/http/proxy/user: port/proxy/user port/proxy/pass: system/schemes/http/proxy/pass: port/proxy/pass ] [ net-error reform ["Error. Target url:" port/url {could not be retrieved: Proxy authentication denied}] ] build-port create-request send-and-check ] send-and-check ] open-proto: func [ {Open the socket connection and confirm server response.} port "Initalized port spec" /locals sub-port data in-bypass find-bypass bp ][ net-utils/net-log ["Opening tcp for" port/scheme] if not system/options/quiet [print ["connecting to:" port/host]] find-bypass: func [host bypass /local x] [ if found? host [ foreach item bypass [ if all [x: find/match/any host item tail? x] [return true] ] ] false ] in-bypass: func [host bypass /local item x] [ if any [none? bypass empty? bypass] [return false] if not tuple? load host [host: form system/words/read join dns:// host] either find-bypass host bypass [ true ] [ host: system/words/read join dns:// host find-bypass host bypass ] ] either all [port/proxy/host bp: not in-bypass port/host port/proxy/bypass find [socks4 socks5 socks] port/proxy/type][ port/sub-port: net-utils/connect-proxy port 'connect ] [ sub-port: system/words/open/lines [ scheme: 'tcp host: either all [port/proxy/type = 'generic bp] [port/proxy/host] [port/proxy/host: none port/host] user: port/user pass: port/pass port-id: either all [port/proxy/type = 'generic bp] [port/proxy/port-id] [port/port-id] ] port/sub-port: sub-port ] port/sub-port/timeout: port/timeout port/sub-port/user: port/user port/sub-port/pass: port/pass port/sub-port/path: port/path port/sub-port/target: port/target net-utils/confirm/multiline port/sub-port open-check port/state/flags: port/state/flags or port-flags ] close: func [port][system/words/close port/sub-port] write: func [ "Default write operation called from buffer layer." port "An open port spec" data "Data to write" ][ net-utils/net-log ["low level write of " port/state/num "bytes"] write-io port/sub-port data port/state/num ] read: func [ port "An open port spec" data "A buffer to use for the read" ][ net-utils/net-log ["low level read of " port/state/num "bytes"] read-io port/sub-port data port/state/num ] get-sub-port: func [ port "An open port spec" ][ port/sub-port ] awake: func [ prot "An open port spec" ][ none ] get-modes: func [ port "An open port spec" modes "A mode block" ][ system/words/get-modes port/sub-port modes ] set-modes: func [ port "An open port spec" modes "A mode block" ][ system/words/set-modes port/sub-port modes ] querying: false query: func [port][ if not port/locals [ querying: true open port ] none ] ] "GS Jones" <[gjones05--mail]. To: <[rebol-list--rebol--com]>> cc: Sent by: Subject: [REBOL] Re: using 'open to read url redirection [rebol-bounce--re] 06/28/2001 05:40 AM Please respond to rebol-list