CGI POST input
[1/10] from: joel:neely:fedex at: 18-Nov-2000 23:59
I recall some discussion relevant to this question, but can't find
a definitive answer.
I'm working on CGI code that takes input via POST from a form with
a textarea. The input can vary widely in length. Using the simple
scheme of
request-method: system/options/cgi/request-method
cgi-input: any [
if request-method = "get" [
system/options/cgi/query-string
]
if request-method = "post" [
input
]
]
I get failure if the form uses POST and the combined length of all
name/value pairs exceeds 4104 bytes, which is apparently the max
length returned by a single call to input (this is consistent).
I.e., the rest of the data are lost.
OTOH, if I try repeatedly calling for input, concatenating the
results into a single string until the total length equals the
content-length, I get a script that never terminates.
Does anyone know of an officially-sanctioned, reliable approach to
reading data from large forms (via POST, of course)?
-jn-
[2/10] from: sharriff:aina:med-iq at: 19-Nov-2000 6:22
Hi Joel!
I think you meant the problems that posted to the list concerning the out
of the text and content using TEXTAREAS. This is what I discovered:
1. I think it depends on the Browser, IE5 REFUSES to send more than 1600
bytes with GET, Netscape on the other hand, sends everything over with get,
I have sent over approximately 2 pages of text sucessfully.
2. According to the documentation on www.blooberry.com, its advisable to
use the POST method, since the data that post can handle is not limited,
IŽll have to try that out.
If youŽre interested, I solved my problem that I posted to the list by
using "wrap=hard" in the textarea attribute (using wrap=virtual or
wrap=physical is quirky or depending on the browser version not supported
at all!!) and adding a TRIM command to my CGI script that captured the data
from the textarea, If you want to keep your format intact without REBOL
adding excess TABS and linebreaks that is.
Sharriff Aina
med.iq information & quality in healthcare AG
Joel Neely
<[joel--neely--f] An: [rebol-list--rebol--com]
edex.com> Kopie:
Gesendet von: Thema: [REBOL] CGI POST input
rebol-bounce@
rebol.com
19.11.00
05:59
Bitte
antworten an
rebol-list
I recall some discussion relevant to this question, but can't find
a definitive answer.
I'm working on CGI code that takes input via POST from a form with
a textarea. The input can vary widely in length. Using the simple
scheme of
request-method: system/options/cgi/request-method
cgi-input: any [
if request-method = "get" [
system/options/cgi/query-string
]
if request-method = "post" [
input
]
]
I get failure if the form uses POST and the combined length of all
name/value pairs exceeds 4104 bytes, which is apparently the max
length returned by a single call to input (this is consistent).
I.e., the rest of the data are lost.
OTOH, if I try repeatedly calling for input, concatenating the
results into a single string until the total length equals the
content-length, I get a script that never terminates.
Does anyone know of an officially-sanctioned, reliable approach to
reading data from large forms (via POST, of course)?
-jn-
[3/10] from: allenk:powerup:au at: 19-Nov-2000 17:50
----- Original Message -----
From: "Joel Neely" <[joel--neely--fedex--com]>
To: <[rebol-list--rebol--com]>
Sent: Sunday, November 19, 2000 3:59 PM
Subject: [REBOL] CGI POST input
> I recall some discussion relevant to this question, but can't find
> a definitive answer.
<<quoted lines omitted: 24>>
> [rebol-request--rebol--com] with "unsubscribe" in the
> subject, without the quotes.
Hi Joel,
This a new entry in the Official FAQ which may be of use here.
http://www.rebol.com/faq.html#14982631
I haven't tested it on anything large yet.
Cheers,
Allen K
[4/10] from: g:santilli:tiscalinet:it at: 19-Nov-2000 12:02
Hello Joel!
On 19-Nov-00, you wrote:
JN> OTOH, if I try repeatedly calling for input, concatenating
JN> the results into a single string until the total length
JN> equals the content-length, I get a script that never
JN> terminates.
JN> Does anyone know of an officially-sanctioned, reliable
JN> approach to reading data from large forms (via POST, of
JN> course)?
Did you try opening console:// and reading from there? Like:
port: open/binary/no-wait console://
while [need-more-chars] [
wait port
insert tail form-input copy port
]
close port
(I didn't try if /NO-WAIT works with CONSOLE://, tough)
Regards,
Gabriele.
--
Gabriele Santilli <[giesse--writeme--com]> - Amigan - REBOL programmer
Amiga Group Italia sez. L'Aquila -- http://www.amyresource.it/AGI/
[5/10] from: joel:neely:fedex at: 19-Nov-2000 7:46
Hello, Sharriff,
Thanks for the reply...
[rebol-bounce--rebol--com] wrote:
> Hi Joel!
> I think you meant the problems that posted to the list concerning
<<quoted lines omitted: 7>>
> advisable to use the POST method, since the data that post can
> handle is not limited, IŽll have to try that out.
My problem was a REBOL problem, not a browser nor a GET-vs-POST
problem. I was already using POST. The problem was that I still
wasn't getting all of the data into REBOL. When I rewrote the
script using Perl, all of the data showed up.
I'm going to test the idea from the REBOL FAQ (URL in Allen's reply)
and report back on whether that solved the problem.
-jn-
[6/10] from: t_degrav:rhrk:uni-kl at: 20-Nov-2000 1:57
Hi Joel,
isn't this problem something caused by a little inconsistency with 'read
and the method of using 'read-io mentioned in the "How To" the (so far)
appropriate way to handle it?! I have to admit, I haven't much experimented
with the POST method myself so far, but 'read-io seems to work for me
even for somwhat larger files (at least larger than 4k) as you also can
see in the CGI test script I posted on thursday.
Tom
Joel Neely wrote:
[7/10] from: rex:smallandmighty at: 19-Nov-2000 20:59
On Sun, Nov 19, 2000, Joel Neely <[joel--neely--fedex--com]> wrote:
>My problem was a REBOL problem, not a browser nor a GET-vs-POST
>problem. I was already using POST. The problem was that I still
>wasn't getting all of the data into REBOL. When I rewrote the
>script using Perl, all of the data showed up.
>
>I'm going to test the idea from the REBOL FAQ (URL in Allen's reply)
>and report back on whether that solved the problem.
I was having the *exact* same problem in my Rebol webserver. The only
thing that worked reliably for me was looping on read-io, and my server
does regularly handles submissions from large ( >20k ) forms. One of my
tests was a text version of my Bachelor's thesis, and that was 216k.
Since my webserver has to handle multiple connections I use a couple
of queues, one for writing, and one for reading. Both send little chunks
of data with write-io or read-io, and keep track of how much was sent or
received so they know when to stop. I don't do keep-alives.
Incidentally, during my testing, I also noticed that Netscape and IE
have very different patterns of I/O. It is quite possible (I know because
I inadvertently did it...) to write some code that reads a POST reliably
from IE, but not Netscape. There also seems to be a maximum amount of
data that the browsers can send from a text area. Each browser has a
different max. Figure somewhere around 30k. I don't remember exactly. But
any more gets truncated by the browser, not Rebol. Also, IE will let you
use the PUT method with a form, whereas Netscape won't. My server handles
PUTs exactly like POSTs.
I know I should probably recode my read and write queue code to use
copy or insert, but it's been working really well since August, and I
have too much other code external to my core webserver code to fix. Plus,
I'd like to get it running on Mac OS X, but that's stuck with Rebol 2.2,
and I think I need 2.3+ for the new networking stuff to work.
My server's gotten too chubby (800+ lines, excluding handlers) to
post to the list, but it's pretty well commented, so I could send email
it to you if you like. When I've cleaned out some of the esoteric cruft I
need for the applications I'm using it for, I'll post it up on one of my
websites, so others can play with it.
.:Eric
[8/10] from: joel:neely:fedex at: 19-Nov-2000 23:27
Hello, Allen,
I tested it and it didn't work. |-(
Details below.
[rebol-bounce--rebol--com] wrote:
> From: "Joel Neely" <[joel--neely--fedex--com]>
> >
<<quoted lines omitted: 6>>
> http://www.rebol.com/faq.html#14982631
> I haven't tested it on anything large yet.
Here's the form I created for the test.
<html>
<head><title>Test CGI with form</title></head>
<body>
<h3>Test CGI with form</h3>
<form method="post" action="testcgi.r.cgi">
Text area:
<textarea name="area" rows="15" cols="50">Hello</textarea><br />
Text field:
<input type="text" name="field" size="10" value="Goodbye"><br />
<input name="submit" type="submit" value="Submit">
<input name="reset" type="reset" value="Reset">
<input name="secret" type="hidden" value="whisper">
</form>
</body>
</html>
... and here's the first script ...
#!/usr/local/bin/rebol -cs
REBOL []
read-post-input: func [/local cs] [
; cs: copy system/ports/input
cs: input
return cs
]
clen: to-integer system/options/cgi/content-length
reqm: system/options/cgi/request-method
cont: any [
if reqm = "get" [ system/options/cgi/query-string ]
if reqm = "post" [ input ]
]
print [
{Content-type: text/plain^/^/}
{^/request-method: } system/options/cgi/request-method
{^/content-length: } clen
{^/content: } cont
{^/length?: } length? cont
newline
]
When I put large quantities of text into the textarea, the script
produces the following output ...
request-method: POST
content-length: 6771
content:
area=12345678901234567890123456789012345678901234567890%0D%0A1...
length?: 4094
Note the "magic number" of 4094 bytes in the actual length retrieved,
even though the form sent over 6.5K.
The second attempt just swapped the source of data in the
Read-Post-Input function...
#!/usr/local/bin/rebol -cs
REBOL []
read-post-input: func [/local cs] [
cs: copy system/ports/input
; cs: input
return cs
]
clen: to-integer system/options/cgi/content-length
reqm: system/options/cgi/request-method
cont: any [
if reqm = "get" [ system/options/cgi/query-string ]
if reqm = "post" [ read-post-input ]
]
print [
{Content-type: text/plain^/^/}
{^/request-method: } system/options/cgi/request-method
{^/content-length: } clen
{^/content: } cont
{^/length?: } length? cont
newline
]
...which produced...
request-method: POST
content-length: 53
content: area=Hello&field=Goodbye&submit=Submit&secret=whisper
length?: 1
...using the default form content. Notice the peculiar fact that the
length is being reported as 1 (???) However, this one also hits the
proverbial wall when the input exceeds 4094 bytes in length.
Sooooo...
I'm still open for suggestions, but it's not looking too good for
the home team at this point.
-jn-
[9/10] from: joel::neely::fedex::com at: 19-Nov-2000 23:36
Hello, Gabriele,
I wasn't able to get this to work.
[rebol-bounce--rebol--com] wrote:
> Hello Joel!
> On 19-Nov-00, you wrote:
<<quoted lines omitted: 12>>
> ]
> close port
I used wait [port 5] hoping to get a timeout, but what I get
instead is the script (apparently) terminating with no code
following that point getting executed. It's late now, but I'll
post a copy of the code tomorrow.
> (I didn't try if /NO-WAIT works with CONSOLE://, tough)
>
It doesn't.
-jn-
[10/10] from: g:santilli:tiscalinet:it at: 20-Nov-2000 15:50
Joel Neely wrote:
> > (I didn't try if /NO-WAIT works with CONSOLE://, tough)
>
> It doesn't.
Hmmm... the following seems to work here:
--
REBOL []
port: open/binary/no-wait console://
bytes: 0
while [bytes < 500] [
wait port
probe bytes: bytes + length? copy port
]
close port
--
but I don't have a cgi environment set up currently to test it...
I think you should be able to make it work with read-io, if it
doesn't with copy, tough...
Regards,
Gabriele.
--
Gabriele Santilli <[giesse--writeme--com]> - Amigan - REBOL programmer
Amiga Group Italia sez. L'Aquila -- http://www.amyresource.it/AGI/
Notes
- Quoted lines have been omitted from some messages.
View the message alone to see the lines that have been omitted