World: r3wp

look in the URL-parser context within the prot-utils.r file.

that is where the url decoding occurs.

There's no 'dehex there

but I remember having the same issue a while back and traced it to 
the actual datatype always handling the hex values.

I don't think it matters

yes, Max ..

just using the to-url created the same headaches... IIRC

Brian's issue is not a problem

That is exactly where it matters. That is the whole problem.

We can fix it without worrying about that part

That part is the only part that needs fixing.

brian .. I agree.. the hexing should stay in the url datatype until 
the actual network scheme requires to handle it.  % characters are 
valid url so they should not get "fixed"

%XX  that is.

But you can't fix it because it's the way Rebol evaluates datatypes 
.. only Carl can fix that.

exactly.

Since when is that a constraint?

12 years I think now

Problems that only Carl can fix still need fixing.

so its a limitation in the URL datatype... akin to agressive error 
evaluation.

so in REBOL speak, url dehexing should, be "relaxed" :-)

in my app, I ended up doing all URL manipulation in strings, and 
then just converting to url at the time of network call

Or at least put off until it is appropriate to do.

IMHO the datatype can't know when. only the schemes and url processors 
know "when" is appropriate.

the problem is when we are programatically managing uri.  the datatype 
dehexing really gets in the way.

>> http://user%40rebol.com:[blah-:-www-:-rebol-:-com]/
== http://[user-:-rebol-:-com]:[blah-:-www-:-rebol-:-com]/
>> a: to-url "http://user%40rebol.com:[blah-:-www-:-rebol-:-com]"
== http://user%40rebol.com:[blah-:-www-:-rebol-:-com]
>> a
== http://user%40rebol.com:[blah-:-www-:-rebol-:-com]

It's really simple: The url! datatype should do no dehexing itself. 
The file! datatype can dehex, but not url!. Dehexing is only safe 
after decoding.

Graham, thanks for narrowing it down.

just tried a read, and when the second form of graham's test (using 
to-url on a string) the url parser doesn't dehex... so the username 
will be invalid.

but I guess the server is responsible for dehexing in that case.

Um, no. The HTTP standard for basic authentication doesn't hex-encode 
the user or password fields. The browser (or in our case, http scheme) 
does.

Only the path is hex-encoded when passed to the server.

ok so then the dehexing should be added in the url-parser and string 
notation used for @ containing passwords.  

just like we use string notation for files containing spaces.

this could a workaround until Carl stops dehexing in the loading 
phase.

Still haven't traced that.

>> c: load "http://user%40rebol.com:[blah-:-www-:-rebol-:-com]"
== http://[user-:-rebol-:-com]:[blah-:-www-:-rebol-:-com]

No, I mean I haven't traced it. LOAD calls other functions, maybe 
even in R2.

The 'solution' is: read [scheme: 'http user: "[user-:-rebol-:-com]" pass: 
.........]

: )

but that's not a uri  ;-)   the point of the url datatype is that 
we shoudn't need to use "specifications" but uri paths.

I know, I know.  I've just used it many times : )

The standard is very explicit on how encoding should work. Reserver 
characters must never be decoded BEFORE you split the URL into its 
parts (which can only be done if you understand the scheme). Any 
other character may be decoded just like REBOL does.


So, it's not that URL! should never decode, it's that it should never 
decode the reserved characters. Then, the handler for each protocol 
needs to decode the parts separately after splitting. Since the protocols 
we use in REBOL all have the same format basically, you can use a 
single function to handle all of them, eg. like my parse-url.

Excellent summary Gabriele. Thanks for posting that. It makes the 
problem and solution very clear.

Carl wrote:   I'm wrapping up R3 A107... so if we want to do some 
test builds for 2.7.8 on this, let me know.

The question is where. There was a similar call in R3 Chat at the 
beginning of the year, but no response there

Personaly I think putting out some test builds is always a good thing, 
 and the prudent thing to to is to make them available for the most 
popular platforms, best chance of someone doing something that results 
in useful feed back. unfortunatly as a unpopular platform user that 
means I am out.

I find the loss of data on reading a http/s page annoying.   ie, 
losing the http headers

I've inserted a save-header function here


   port/locals/headers: headers: save-header Parse-Header HTTP-Header 
   headers

but the question now is, where is the best place to store this?

actually thinking about it, it should really be 


   port/locals/headers: headers: Parse-Header HTTP-Header save-header 
   headers


as parse-header will remove duplicate header lines .. ie. if there 
is more than one cookie header, the parse-header function loses all 
but one.

This is one reason why I wrote a rest protocol.  The http protocol 
seems designed to get content the same way a browser would.  But 
as more services use http more completely, things like automatic 
redirects and thrown errors for 4xx/5xx status codes are not helpful 
(and good luck getting headers and content then).

rest.r is designed to make HTTP interaction behave more like a web 
service.  Which is the principle behind Rest, I supose.

Is anyone tracking what we need for 2.7.8 ??  http://www.rebol.com/docs/changes-2-7.html
This was supposed to be released months ago ...

Actually 2.7.8 was promised for jan 2010