Mailing List Archive: 49091 messages
  • Home
  • Script library
  • AltME Archive
  • Mailing list
  • Articles Index
  • Site search
 

CGI POST input

 [1/10] from: joel:neely:fedex at: 18-Nov-2000 23:59


I recall some discussion relevant to this question, but can't find a definitive answer. I'm working on CGI code that takes input via POST from a form with a textarea. The input can vary widely in length. Using the simple scheme of request-method: system/options/cgi/request-method cgi-input: any [ if request-method = "get" [ system/options/cgi/query-string ] if request-method = "post" [ input ] ] I get failure if the form uses POST and the combined length of all name/value pairs exceeds 4104 bytes, which is apparently the max length returned by a single call to input (this is consistent). I.e., the rest of the data are lost. OTOH, if I try repeatedly calling for input, concatenating the results into a single string until the total length equals the content-length, I get a script that never terminates. Does anyone know of an officially-sanctioned, reliable approach to reading data from large forms (via POST, of course)? -jn-

 [2/10] from: sharriff:aina:med-iq at: 19-Nov-2000 6:22


Hi Joel! I think you meant the problems that posted to the list concerning the out of the text and content using TEXTAREAS. This is what I discovered: 1. I think it depends on the Browser, IE5 REFUSES to send more than 1600 bytes with GET, Netscape on the other hand, sends everything over with get, I have sent over approximately 2 pages of text sucessfully. 2. According to the documentation on www.blooberry.com, its advisable to use the POST method, since the data that post can handle is not limited, IŽll have to try that out. If youŽre interested, I solved my problem that I posted to the list by using "wrap=hard" in the textarea attribute (using wrap=virtual or wrap=physical is quirky or depending on the browser version not supported at all!!) and adding a TRIM command to my CGI script that captured the data from the textarea, If you want to keep your format intact without REBOL adding excess TABS and linebreaks that is. Sharriff Aina med.iq information & quality in healthcare AG Joel Neely <[joel--neely--f] An: [rebol-list--rebol--com] edex.com> Kopie: Gesendet von: Thema: [REBOL] CGI POST input rebol-bounce@ rebol.com 19.11.00 05:59 Bitte antworten an rebol-list I recall some discussion relevant to this question, but can't find a definitive answer. I'm working on CGI code that takes input via POST from a form with a textarea. The input can vary widely in length. Using the simple scheme of request-method: system/options/cgi/request-method cgi-input: any [ if request-method = "get" [ system/options/cgi/query-string ] if request-method = "post" [ input ] ] I get failure if the form uses POST and the combined length of all name/value pairs exceeds 4104 bytes, which is apparently the max length returned by a single call to input (this is consistent). I.e., the rest of the data are lost. OTOH, if I try repeatedly calling for input, concatenating the results into a single string until the total length equals the content-length, I get a script that never terminates. Does anyone know of an officially-sanctioned, reliable approach to reading data from large forms (via POST, of course)? -jn-

 [3/10] from: allenk:powerup:au at: 19-Nov-2000 17:50


----- Original Message ----- From: "Joel Neely" <[joel--neely--fedex--com]> To: <[rebol-list--rebol--com]> Sent: Sunday, November 19, 2000 3:59 PM Subject: [REBOL] CGI POST input
> I recall some discussion relevant to this question, but can't find > a definitive answer.
<<quoted lines omitted: 24>>
> [rebol-request--rebol--com] with "unsubscribe" in the > subject, without the quotes.
Hi Joel, This a new entry in the Official FAQ which may be of use here. http://www.rebol.com/faq.html#14982631 I haven't tested it on anything large yet. Cheers, Allen K

 [4/10] from: g:santilli:tiscalinet:it at: 19-Nov-2000 12:02


Hello Joel! On 19-Nov-00, you wrote: JN> OTOH, if I try repeatedly calling for input, concatenating JN> the results into a single string until the total length JN> equals the content-length, I get a script that never JN> terminates. JN> Does anyone know of an officially-sanctioned, reliable JN> approach to reading data from large forms (via POST, of JN> course)? Did you try opening console:// and reading from there? Like: port: open/binary/no-wait console:// while [need-more-chars] [ wait port insert tail form-input copy port ] close port (I didn't try if /NO-WAIT works with CONSOLE://, tough) Regards, Gabriele. -- Gabriele Santilli <[giesse--writeme--com]> - Amigan - REBOL programmer Amiga Group Italia sez. L'Aquila -- http://www.amyresource.it/AGI/

 [5/10] from: joel:neely:fedex at: 19-Nov-2000 7:46


Hello, Sharriff, Thanks for the reply... [rebol-bounce--rebol--com] wrote:
> Hi Joel! > I think you meant the problems that posted to the list concerning
<<quoted lines omitted: 7>>
> advisable to use the POST method, since the data that post can > handle is not limited, IŽll have to try that out.
My problem was a REBOL problem, not a browser nor a GET-vs-POST problem. I was already using POST. The problem was that I still wasn't getting all of the data into REBOL. When I rewrote the script using Perl, all of the data showed up. I'm going to test the idea from the REBOL FAQ (URL in Allen's reply) and report back on whether that solved the problem. -jn-

 [6/10] from: t_degrav:rhrk:uni-kl at: 20-Nov-2000 1:57


Hi Joel, isn't this problem something caused by a little inconsistency with 'read and the method of using 'read-io mentioned in the "How To" the (so far) appropriate way to handle it?! I have to admit, I haven't much experimented with the POST method myself so far, but 'read-io seems to work for me even for somwhat larger files (at least larger than 4k) as you also can see in the CGI test script I posted on thursday. Tom Joel Neely wrote:

 [7/10] from: rex:smallandmighty at: 19-Nov-2000 20:59


On Sun, Nov 19, 2000, Joel Neely <[joel--neely--fedex--com]> wrote:
>My problem was a REBOL problem, not a browser nor a GET-vs-POST >problem. I was already using POST. The problem was that I still >wasn't getting all of the data into REBOL. When I rewrote the >script using Perl, all of the data showed up. > >I'm going to test the idea from the REBOL FAQ (URL in Allen's reply) >and report back on whether that solved the problem.
I was having the *exact* same problem in my Rebol webserver. The only thing that worked reliably for me was looping on read-io, and my server does regularly handles submissions from large ( >20k ) forms. One of my tests was a text version of my Bachelor's thesis, and that was 216k. Since my webserver has to handle multiple connections I use a couple of queues, one for writing, and one for reading. Both send little chunks of data with write-io or read-io, and keep track of how much was sent or received so they know when to stop. I don't do keep-alives. Incidentally, during my testing, I also noticed that Netscape and IE have very different patterns of I/O. It is quite possible (I know because I inadvertently did it...) to write some code that reads a POST reliably from IE, but not Netscape. There also seems to be a maximum amount of data that the browsers can send from a text area. Each browser has a different max. Figure somewhere around 30k. I don't remember exactly. But any more gets truncated by the browser, not Rebol. Also, IE will let you use the PUT method with a form, whereas Netscape won't. My server handles PUTs exactly like POSTs. I know I should probably recode my read and write queue code to use copy or insert, but it's been working really well since August, and I have too much other code external to my core webserver code to fix. Plus, I'd like to get it running on Mac OS X, but that's stuck with Rebol 2.2, and I think I need 2.3+ for the new networking stuff to work. My server's gotten too chubby (800+ lines, excluding handlers) to post to the list, but it's pretty well commented, so I could send email it to you if you like. When I've cleaned out some of the esoteric cruft I need for the applications I'm using it for, I'll post it up on one of my websites, so others can play with it. .:Eric

 [8/10] from: joel:neely:fedex at: 19-Nov-2000 23:27


Hello, Allen, I tested it and it didn't work. |-( Details below. [rebol-bounce--rebol--com] wrote:
> From: "Joel Neely" <[joel--neely--fedex--com]> > >
<<quoted lines omitted: 6>>
> http://www.rebol.com/faq.html#14982631 > I haven't tested it on anything large yet.
Here's the form I created for the test. <html> <head><title>Test CGI with form</title></head> <body> <h3>Test CGI with form</h3> <form method="post" action="testcgi.r.cgi"> Text area: <textarea name="area" rows="15" cols="50">Hello</textarea><br /> Text field: <input type="text" name="field" size="10" value="Goodbye"><br /> <input name="submit" type="submit" value="Submit"> <input name="reset" type="reset" value="Reset"> <input name="secret" type="hidden" value="whisper"> </form> </body> </html> ... and here's the first script ... #!/usr/local/bin/rebol -cs REBOL [] read-post-input: func [/local cs] [ ; cs: copy system/ports/input cs: input return cs ] clen: to-integer system/options/cgi/content-length reqm: system/options/cgi/request-method cont: any [ if reqm = "get" [ system/options/cgi/query-string ] if reqm = "post" [ input ] ] print [ {Content-type: text/plain^/^/} {^/request-method: } system/options/cgi/request-method {^/content-length: } clen {^/content: } cont {^/length?: } length? cont newline ] When I put large quantities of text into the textarea, the script produces the following output ... request-method: POST content-length: 6771 content: area=12345678901234567890123456789012345678901234567890%0D%0A1... length?: 4094 Note the "magic number" of 4094 bytes in the actual length retrieved, even though the form sent over 6.5K. The second attempt just swapped the source of data in the Read-Post-Input function... #!/usr/local/bin/rebol -cs REBOL [] read-post-input: func [/local cs] [ cs: copy system/ports/input ; cs: input return cs ] clen: to-integer system/options/cgi/content-length reqm: system/options/cgi/request-method cont: any [ if reqm = "get" [ system/options/cgi/query-string ] if reqm = "post" [ read-post-input ] ] print [ {Content-type: text/plain^/^/} {^/request-method: } system/options/cgi/request-method {^/content-length: } clen {^/content: } cont {^/length?: } length? cont newline ] ...which produced... request-method: POST content-length: 53 content: area=Hello&field=Goodbye&submit=Submit&secret=whisper length?: 1 ...using the default form content. Notice the peculiar fact that the length is being reported as 1 (???) However, this one also hits the proverbial wall when the input exceeds 4094 bytes in length. Sooooo... I'm still open for suggestions, but it's not looking too good for the home team at this point. -jn-

 [9/10] from: joel::neely::fedex::com at: 19-Nov-2000 23:36


Hello, Gabriele, I wasn't able to get this to work. [rebol-bounce--rebol--com] wrote:
> Hello Joel! > On 19-Nov-00, you wrote:
<<quoted lines omitted: 12>>
> ] > close port
I used wait [port 5] hoping to get a timeout, but what I get instead is the script (apparently) terminating with no code following that point getting executed. It's late now, but I'll post a copy of the code tomorrow.
> (I didn't try if /NO-WAIT works with CONSOLE://, tough) >
It doesn't. -jn-

 [10/10] from: g:santilli:tiscalinet:it at: 20-Nov-2000 15:50


Joel Neely wrote:
> > (I didn't try if /NO-WAIT works with CONSOLE://, tough) > > It doesn't.
Hmmm... the following seems to work here: -- REBOL [] port: open/binary/no-wait console:// bytes: 0 while [bytes < 500] [ wait port probe bytes: bytes + length? copy port ] close port -- but I don't have a cgi environment set up currently to test it... I think you should be able to make it work with read-io, if it doesn't with copy, tough... Regards, Gabriele. -- Gabriele Santilli <[giesse--writeme--com]> - Amigan - REBOL programmer Amiga Group Italia sez. L'Aquila -- http://www.amyresource.it/AGI/

Notes
  • Quoted lines have been omitted from some messages.
    View the message alone to see the lines that have been omitted