World: r3wp

Well, DECODE-URL should probably not dehex the username and password 
unless the (unknown to me) code that reassembles the url! can be 
changed to rehex them. As it should, but I don't know which code 
to fix. The scheme, host and path should not be dehexed in any case.

Probably don't dehex anything in decode-url ...leave it to the schemes

true... since a scheme can potentially not even represent anything 
network related.

The problem is that while the scheme might not represent anything 
network-related, the standard for URI syntax is independent of network 
issues. And that standard is pretty strict about hex encoding, regardless 
of the scheme's internal rules. So schemes need to be hex-encoding-aware 
for their specs, whether they are network-related or not.

parse-url should not dehex .. we can fix the rest in the schemes

brian, true.  my error... I'm deep in calculus... my brain is a bit 
mushy ;-)


IIRC the RFC has an BNF-style breakdown, so there should be no surprise 
as to where hexing can and should be interpreted.

I did a fully compliant RFC URL parser which works better than the 
internal one... maybe I should look at it for more details...

My only concern is that I don't know where is the code that reassembles 
the url! from the results of DECODE-URL. So I don't know how to fix 
any issues in it.

the code is in the sdk

Which file? What is the function named?

Why do you think it is reassembled?

IIRC its all in a context.

All urls are deconstructed into an object

Right. But the problem I am trying to fix is this:
>> http://user%40rebol.com:[blah-:-www-:-rebol-:-com]/
== http://[user-:-rebol-:-com]:[blah-:-www-:-rebol-:-com]/

Somewhere in the middle of that process, DECODE-URL is called. What 
is called after that to reassemble the result into a url! value?

The dehex in that process is the one that we need to get rid of.

I suspect we don't have the source to that...

look in the URL-parser context within the prot-utils.r file.

that is where the url decoding occurs.

There's no 'dehex there

but I remember having the same issue a while back and traced it to 
the actual datatype always handling the hex values.

I don't think it matters

yes, Max ..

just using the to-url created the same headaches... IIRC

Brian's issue is not a problem

That is exactly where it matters. That is the whole problem.

We can fix it without worrying about that part

That part is the only part that needs fixing.

brian .. I agree.. the hexing should stay in the url datatype until 
the actual network scheme requires to handle it.  % characters are 
valid url so they should not get "fixed"

%XX  that is.

But you can't fix it because it's the way Rebol evaluates datatypes 
.. only Carl can fix that.

exactly.

Since when is that a constraint?

12 years I think now

Problems that only Carl can fix still need fixing.

so its a limitation in the URL datatype... akin to agressive error 
evaluation.

so in REBOL speak, url dehexing should, be "relaxed" :-)

in my app, I ended up doing all URL manipulation in strings, and 
then just converting to url at the time of network call

Or at least put off until it is appropriate to do.

IMHO the datatype can't know when. only the schemes and url processors 
know "when" is appropriate.

the problem is when we are programatically managing uri.  the datatype 
dehexing really gets in the way.

>> http://user%40rebol.com:[blah-:-www-:-rebol-:-com]/
== http://[user-:-rebol-:-com]:[blah-:-www-:-rebol-:-com]/
>> a: to-url "http://user%40rebol.com:[blah-:-www-:-rebol-:-com]"
== http://user%40rebol.com:[blah-:-www-:-rebol-:-com]
>> a
== http://user%40rebol.com:[blah-:-www-:-rebol-:-com]

It's really simple: The url! datatype should do no dehexing itself. 
The file! datatype can dehex, but not url!. Dehexing is only safe 
after decoding.

Graham, thanks for narrowing it down.

just tried a read, and when the second form of graham's test (using 
to-url on a string) the url parser doesn't dehex... so the username 
will be invalid.

but I guess the server is responsible for dehexing in that case.

Um, no. The HTTP standard for basic authentication doesn't hex-encode 
the user or password fields. The browser (or in our case, http scheme) 
does.

Only the path is hex-encoded when passed to the server.

ok so then the dehexing should be added in the url-parser and string 
notation used for @ containing passwords.  

just like we use string notation for files containing spaces.

this could a workaround until Carl stops dehexing in the loading 
phase.

Still haven't traced that.