decode-cgi bug (?)
[1/19] from: hallvard:ystad:oops-as:no at: 9-Feb-2004 23:07
Hi
I know some people say "garbage in - garbage out", but one can't always control what
comes into one's programs as input. Here's what came to one of mine:
http://www.bbc.co.uk/cgi-perl/h2/h2.cgi?state=threads&board=cult.tamaraswift&&sort=T
This crashes 'decode-cgi in /core 2.5.6:
>> decode-cgi "state=threads&board=cult.tamaraswift&&sort=T"
** Script Error: Out of range or past end
** Where: to-set-word
** Near: to set-word! :value
>>
I believe Andreas Bolka and/or Andrew Martin made a patch to 'decode-cgi some time back,
but can't seem to find it. Does anyone know whether the patch fixes this problem? Or
perhaps where I can find it?
Thanks,
HY
[2/19] from: SunandaDH::aol::com at: 9-Feb-2004 17:32
Hallvard:
> I believe Andreas Bolka and/or Andrew Martin made a patch to 'decode-cgi
some
> time back, but can't seem to find it. Does anyone know whether the patch
> fixes this problem? Or perhaps where I can find it?
decode-cgi works under View (at least where I am). (Core fails here, just
like your version). You could cut'n'paste the source from View to Core.
When I say "works", I mean doesn't crash. That extra ampersand is treated as
part of the field name -- may not be what you want:
probe cgi: context decode-cgi "state=threads&board=cult.tamaraswift&&sort=T"
make object! [
state: "threads"
board: "cult.tamaraswift"
&sort: "T"
]
Sunanda
[3/19] from: warp:reboot:ch at: 9-Feb-2004 23:36
Hi Hallvard;
this one work on your url, it also has some enhancement
most preferred is ability to set a: "" when http://dom.com?a
I'm not sure who is the author but thanks, and shouldn't this version
replace
rebol's one?
Hope that helps and have a great day!
Will Arp
[warp--reboot--ch]
;core 2.6 decode-cgi do not accept "a" only "a=" or "a&"
decode-cgi: func [
{Converts CGI argument string to a list of words and value strings.}
args [any-string!] "Starts at first argument word"
/local list equate value name name-chars val add-nv
] [
add-nv: func [ list name value /local val-ptr ] [
value: either none? value
[ copy "" ]
[ form dehex (replace/all value "+" " ") ]
either none? val-ptr: find list to-set-word name [
append list compose [ (to-set-word name) (value) ]
] [
idx: index? next val-ptr
poke list idx compose [ (pick list idx) (value) ]
]
]
list: make block! 8
name-chars: complement charset "&="
equate: [ copy name some name-chars value ]
value: [ "=" value
| "&" (add-nv list name "")
| [copy val to "&" "&" | copy val to end]
(add-nv list name val)
]
parse/all args [ some equate | none ]
list
]
On 9-feb-04, at 23:07, Hallvard Ystad wrote:
[4/19] from: hallvard:ystad:oops-as:no at: 9-Feb-2004 23:47
Thanks, Will,
but no, this doesn't fix the problem. The URL I found had two adjacent &s:
state=threads&board=cult.tamaraswift&&sort=T
I think whoever made this URL ment for sort=T to join in, but your code doesn't let it.
I only get state and board.
Any ideas? (I'll look into it more closely myself later on, but I'm off to bed right
now).
HY
Dixit Will Arp (23.36 09.02.2004):
[5/19] from: hallvard:ystad:oops-as:no at: 9-Feb-2004 23:52
Oh, I tried with the source from /view 1.2.10, but got the same result. Could you perhaps
send me the code? Getting the parameter with the ampersand is better than nothing, after
all...
Thanks,
HY
Dixit [SunandaDH--aol--com] (23.32 09.02.2004):
[6/19] from: tomc:darkwing:uoregon at: 9-Feb-2004 15:02
howdy,
I was fiddling with this recently hoping to get an improved version into
view 1.3. there are different thoughts on 'decode-cgi should be a
higher/lower level function my belief is it should be pretty dumb and
just break strings up on a structural basis. but that leads to newby
biting which is not good. so here are two functions:
the first 'vet-word attempts to return as much of an input string! that
can be formed into a valid rebol word! making char replacements as needed.
the second is dumb but a robust decode-cgi that should handle all sorts
of malformed query strings, which by default calls vet-word to filter
out funny stuff but can be called with a /raw refinement to skip it
if you feel you do not need the extra checks.
mind the wrap
vet-word: func[{return as much of a string is suitable to form a word
assumes no chars below 31 20x}
str[string!]
/local valid invalid result token
][
invalid: make bitset! [#"@" #"#" #"$" #"%" #"^^" #"," #"[" #"]" #"("
#")" #"{" #"}" #"^"" #":" #";" #"/" #" " #"^/"]
valid: complement invalid
result: make string! length? str
number: [copy token [opt["+" | "-"][integer! opt ["." integer!]]|["."
integer!]]
(insert tail result join "number_" token)
]
parse/all str[
opt number
any[ opt [[some invalid] (insert tail result "!")]
copy token [some valid] (insert tail result token)
]
]
if empty? result[result: "!"] ; if the string had no valid chars return
surprise
if not equal? str result[print["IN-PUT " str newline "OUTPUT " result]];
debugging
result
]
;;;----------------------------------------------------------------------------------------
decode-cgi: func [args [any-string!] /raw
/local cgi-block name value mark here
][
all[empty? args return ""]
structural: make bitset! "+&=%"
non-struct: complement structural
xchar: make bitset! {0123456789ABCDEFabcdef}
safe-xchar: make bitset! {23456789ABCDEFabcdef}
cgi-block: make block! 16
spc: [mark: #"+" (change :mark " ")]
hex: [mark: #"%" 2 xchar (change/part :mark dehex copy/part :mark 3 3)
:mark]
safe-hex: [mark: #"%"
[safe-xchar xchar (change/part :mark dehex copy/part
:mark 3 3) :mark] |
[2 xchar (change :mark "!" )]
]
parse/all args [
opt "&"
any[(name: copy "" value: copy "" here: copy "")
copy name some [non-struct | spc | safe-hex]
(all[not raw name: vet-word trim name]
either none? here: find cgi-block to set-word! name[
insert/only tail cgi-block to set-word! name
insert/only here: tail cgi-block none
][here: next here])
opt[#"="
opt[copy value some [non-struct | spc | hex]
(either none? first here
[change here value]
[either not block? first here
[poke here 1 to block! reduce[first here value]]
[insert/only tail first here value]
]
)
]
]
[ #"&" | end]
]
]
cgi-block
]
On Mon, 9 Feb 2004, Hallvard Ystad wrote:
[7/19] from: hallvard:ystad:oops-as:no at: 10-Feb-2004 0:24
Sorry, Tom,
your script fails to get the 'sort parameter too.
This time I'm _really_ off to bed.
HY
Dixit Tom Conlin (00.02 10.02.2004):
[8/19] from: tomc:darkwing:uoregon at: 9-Feb-2004 15:24
ok here is a version that allows multiple &'s
decode-cgi: func [args [any-string!] /raw
/local cgi-block name value mark here
][
all[empty? args return ""]
structural: make bitset! "+&=%"
non-struct: complement structural
xchar: make bitset! {0123456789ABCDEFabcdef}
safe-xchar: make bitset! {23456789ABCDEFabcdef}
cgi-block: make block! 16
spc: [mark: #"+" (change :mark " ")]
hex: [mark: #"%" 2 xchar (change/part :mark dehex copy/part :mark 3 3)
:mark]
safe-hex: [mark: #"%"
[safe-xchar xchar (change/part :mark dehex copy/part
:mark 3 3) :mark] |
[2 xchar (change :mark "!" )]
]
parse/all args [
any #"&"
any[(name: copy "" value: copy "" here: copy "")
copy name some [non-struct | spc | safe-hex]
(all[not raw name: vet-word trim name]
either none? here: find cgi-block to set-word! name[
insert/only tail cgi-block to set-word! name
insert/only here: tail cgi-block none
][here: next here])
opt[#"="
opt[copy value some [non-struct | spc | hex]
(either none? first here
[change here value]
[either not block? first here
[poke here 1 to block! reduce[first here
value]]
[insert/only tail first here value]
]
)
]
]
[some[#"&"] | end]
]
]
cgi-block
]
On Mon, 9 Feb 2004, Tom Conlin wrote:
[9/19] from: tim::johnsons-web::com at: 9-Feb-2004 14:47
Re: decode-cgi bug (imbedded spaces in names)
* Hallvard Ystad <[hallvard--ystad--oops-as--no]> [040209 13:31]:
It looks like Hallvard's problem is being addressed, but I want to
note that I modified decode-cgi so that a block of string pairs
is returned rather than a block of alternating to-set-word types
and strings.
Reason for it:
A rebol cgi program might be processing a form in which field
names have imbedded spaces. This isn't a practice of mine,
and I avoid it on any of our projects, but the eventuality
would yield unpredicatable results:
example:
>> res: context decode-cgi "field one=1&fieldtwo=2"
>> probe res
make object! [
field one: "1"
fieldtwo: "2"
]
>> res/fieldtwo
== "2"
>>res/field one
** Script Error: Invalid path value: field
** Near: res/field one
tim
> Hi
> I know some people say "garbage in - garbage out", but one can't always control what
comes into one's programs as input. Here's what came to one of mine:
<<quoted lines omitted: 12>>
> To unsubscribe from this list, just send an email to
> [rebol-request--rebol--com] with unsubscribe as the subject.
--
Tim Johnson <[tim--johnsons-web--com]>
http://www.alaska-internet-solutions.com
[10/19] from: hallvard:ystad:oops-as:no at: 10-Feb-2004 0:46
Re: decode-cgi bug (?)
Brilliant. Thanks!
But:
>> decode-cgi {url=http%3A//www.bbc.co.uk/cgi-perl/h2/h2.cgi%3Fstate%3Dthreads%26board%3Dcult.tamaraswift%26%26sort%3DT}
== [url: "http://www.bbc.co.uk/cgi-perl/h2/h2.cgi?state"]
>>
Phew. This time I _really_really_ am off to bed.
HY
Dixit Tom Conlin (00.24 10.02.2004):
[11/19] from: andreas:bolka:gmx at: 10-Feb-2004 0:50
Monday, February 9, 2004, 11:07:27 PM, Hallvard wrote:
> I know some people say "garbage in - garbage out", but one can't
> always control what comes into one's programs as input. Here's what
> came to one of mine:
> http://www.bbc.co.uk/cgi-perl/h2/h2.cgi?state=threads&board=cult.tamaraswift&&sort=T
thanks for reporting that, hallvard! i've extended my patched version
of decode-cgi to cope with that as well:
>> decode-cgi "state=threads&board=cult.tamaraswift&&sort=T"
== [state: "threads" board: "cult.tamaraswift" sort: "T"]
-- snip --
decode-cgi: func [
{Converts CGI argument string to a list of words and value strings.}
args [any-string!] "Starts at first argument word"
/local list nv-pair value name name-chars val
][
add-nv: func [ list name value /local val-ptr ] [
if none? name [ return ]
value: dehex replace/all any [ value (copy "") ] "+" " "
either val-ptr: find list to-set-word name
[ change/only next val-ptr compose [ (first next val-ptr) (value) ] ]
[ append list compose [ (to-set-word name) (value) ] ]
]
list: make block! 8
name-chars: complement charset "&="
nv-pair: [ copy name some name-chars value (add-nv list name val) | "&" ]
value: [ "=" value | "&" | [ copy val to "&" skip | copy val to end ] ]
parse/all args [ some nv-pair | none ]
list
]
-- snap --
if someone wants to submit it to the view 1.3 effort, do not hesitate
to do so.
--
Best regards,
Andreas
[12/19] from: tomc:darkwing:uoregon at: 9-Feb-2004 17:39
Re: decode-cgi bug (imbedded spaces in names)
that is exactly the sort of thing 'vet-word avoids.
it is obviously a work in progress and of all the unsatisfaftory
side effects I think the least unsatisfaftory is yet to be found.
On Mon, 9 Feb 2004, Tim Johnson wrote:
[13/19] from: andreas:bolka:gmx at: 10-Feb-2004 15:44
Tuesday, February 10, 2004, 12:47:11 AM, Tim wrote:
> I want to note that I modified decode-cgi so that a block of string
> pairs is returned rather than a block of alternating to-set-word
> types and strings.
> Reason for it: A rebol cgi program might be processing a form in
> which field names have imbedded spaces.
I think that it will be rather hard to get that kind of data to a CGI
in real world scenarios. You won't be able to pass it via method GET's
query string, as a blank is not a valid character there. So your url
will either look like http://foo/bar?first one=1 or like
http://foo/bar?first+one=1.
Quite the same holds for method POST. If you submit a 'input
type="text" name="first one"' with a value of "1" a browser will send
'first+one=1' as entity body.
However, if one _manually_ does a POST this kind of bad data is
certainly possible. For example
read/custom http://foo/bar [ post "first one=1" ]
will send 'first one=1' as entity body.
So, what's the point of those lengthy explanations? While this is a
scenario that is only created by 'broken' tools, 'decode-cgi should
certainly cope with this broken data.
Incidentially, my patched version of decode-cgi does _somehow_ cope
with it:
>> o: make object! b: decode-cgi "field one=1&fieldtwo=2"
>> probe o
make object! [
field one: "1"
fieldtwo: "2"
]
>> print mold b
[field one: "1" fieldtwo: "2"]
>> get in o to-word "field one"
== "1"
--
Best regards,
Andreas
[14/19] from: tim:johnsons-web at: 10-Feb-2004 7:38
* Tom Conlin <[tomc--darkwing--uoregon--edu]> [040209 17:04]:
>
> that is exactly the sort of thing 'vet-word avoids.
Tom:
Tell me more about 'vet-word,
if you can.
thanks
tim
> it is obviously a work in progress and of all the unsatisfaftory
> side effects I think the least unsatisfaftory is yet to be found.
<<quoted lines omitted: 64>>
> To unsubscribe from this list, just send an email to
> [rebol-request--rebol--com] with unsubscribe as the subject.
--
Tim Johnson <[tim--johnsons-web--com]>
http://www.alaska-internet-solutions.com
[15/19] from: tim:johnsons-web at: 10-Feb-2004 7:43
* Andreas Bolka <[andreas--bolka--gmx--net]> [040210 06:04]:
>
> Tuesday, February 10, 2004, 12:47:11 AM, Tim wrote:
>
> > I want to note that I modified decode-cgi so that a block of string
> > pairs is returned rather than a block of alternating to-set-word
> > types and strings.
>
> > Reason for it: A rebol cgi program might be processing a form in
> > which field names have imbedded spaces.
<..>
> >> probe o
> make object! [
<<quoted lines omitted: 4>>
> [field one: "1" fieldtwo: "2"]
> >> get in o to-word "field one"
What would the advantage of this example,
as opposed to
select o "field one" ;where o is stored as pairs of strings
thanks
tim
--
Tim Johnson <[tim--johnsons-web--com]>
http://www.alaska-internet-solutions.com
[16/19] from: tomc:darkwing:uoregon at: 10-Feb-2004 9:38
Hi Tim,
by my count it is in the sixth post of this thread.
the basic idea is to brighten up decode-cgi so it can
return a block of set words instead of a block of string pairs.
which can be coerced into an honest to goodness OBJECT
So CGI processing would seem more similar to the way some other
languages do it, for newbies to feel comfortable.
(I am not saying it is my first choice but I'm willing to work with it)
On Tue, 10 Feb 2004, Tim Johnson wrote:
[17/19] from: inetw3:mindspring at: 10-Feb-2004 17:29
Re: decode-cgi bug (?)
Hi Hallvard,
Would this help any?
you can replace the " replace/all "this" "this" "
with a "foreach char_block" and "parse cgi (replace cgi "this" "this")
----------------------------------------------------------------------------
------------------
cgi: "state=thread s&board=cult.tamaraswift&&sort=T"
char_block: ["=" {: "} "&" {" } { " } " "]
decode_cgi: func [cgi][
replace/all cgi " " " "
replace/all cgi "=" {: "}
replace/all cgi "&" {" }
replace/all cgi { " } " "
append cgi {"}
cgi: to-block cgi
cgi_object: make object! cgi
]
decode_cgi cgi
probe cgi_object
----- Original Message -----
From: Hallvard Ystad <[hallvard--ystad--oops-as--no]>
To: <[rebol-list--rebol--com]>
Sent: Monday, February 09, 2004 4:07 PM
Subject: [REBOL] decode-cgi bug (?)
> Hi
>
> I know some people say "garbage in - garbage out", but one can't always
control what comes into one's programs as input. Here's what came to one of
mine:
>
http://www.bbc.co.uk/cgi-perl/h2/h2.cgi?state=threads&board=cult.tamaraswift
&&sort=T
> This crashes 'decode-cgi in /core 2.5.6:
> >> decode-cgi "state=threads&board=cult.tamaraswift&&sort=T"
<<quoted lines omitted: 3>>
> >>
> I believe Andreas Bolka and/or Andrew Martin made a patch to 'decode-cgi
some time back, but can't seem to find it. Does anyone know whether the
patch fixes this problem? Or perhaps where I can find it?
[18/19] from: warp:reboot:ch at: 19-Feb-2004 17:18
Hi Andres,
there seems to be a bug in your function:
>> decode-cgi "a=1&b=&c=3"
== [a: "1" b: "1" c: "3"]
Will
On 10-feb-04, at 00:50, Andreas Bolka wrote:
[19/19] from: andreas:bolka:gmx at: 19-Feb-2004 18:02
Thursday, February 19, 2004, 5:18:19 PM, Will wrote:
> there seems to be a bug in your function:
>>> decode-cgi "a=1&b=&c=3"
> == [a: "1" b: "1" c: "3"]
thanks for reporting, will. fixed:
decode-cgi: func [
{Converts CGI argument string to a list of words and value strings.}
args [any-string!] "Starts at first argument word"
/local list nv-pair value name name-chars val
][
add-nv: func [ list name value /local val-ptr ] [
if none? name [ return ]
value: dehex replace/all any [ value (copy "") ] "+" " "
either val-ptr: find list to-set-word name
[ change/only next val-ptr compose [ (first next val-ptr) (value) ] ]
[ append list compose [ (to-set-word name) (value) ] ]
]
list: make block! 8
name-chars: complement charset "&="
nv-pair: [ copy name some name-chars
(val: none) value (add-nv list name val) | "&" ]
value: [ "=" value | "&" | [ copy val to "&" skip | copy val to end ] ]
parse/all args [ some nv-pair | none ]
list
]
--
Best regards,
Andreas
Notes
- Quoted lines have been omitted from some messages.
View the message alone to see the lines that have been omitted