AltME groups: search

The cURL binding documentation looks very good Kaj, congratulations.

(-> cURL Binding)

sure, gracias for curl kaj. very usefull component and already working 
;-)

My cURL binding is now released under source control in the Syllable 
build system:

http://syllable.cvs.sourceforge.net/viewvc/syllable/syllable/system/apps/utils/Builder/packages/cURL-Binding--current/

It took a lot more time than I expected to create the documentation 
for the cURL binding; about as long as the binding itself. But here 
it is, together with the documentation for the example extension:

Curl: Do you mean Curl as basic system or transforming to Curl? IIRC 
there system is quite fat and requires special servers etc. They 
have a complicated licensing scheme etc.

So, I agree with Petr, why use Curl if we have Rebol?

It's just an HTML alternative ( HTML+CSS+JavaScript )

I was once an OS.2 Zealot and a Prolog Zealot.  I love Rebol, but 
it will not make me either a Rebol bigot or a Rebol zealot.  Even 
a good advocate should not come across as either ...  Curl is not 
server-side; it is just client-side.  If what you have is a 'layout', 
why transform to HTML just to get a page into a browser while waiting 
(6+ mo)  for a Rebol plug-in ?  Curl is a web content language.  
It's good for that.  And it has great sliding panes and all just 
like scriptaculous but a s simple and consistent as good Rebol code. 
 Think of it as Scheme/Dylan for web pages instesad of HTML.

RobertS - just don't try to suggest we are REBOL zealots. Maybe I 
was Amiga zealot, but that time is long gone. OTOH I strictly refuse 
to be shy of REBOL anymore? What does Curl gives me? It is just another 
technology. I was open about Curl, or was it other technology? And 
it was Robert who suggested that the thing is slow, and that his 
fried had to restart the project because of that.

Curl and client side? Wait, is that a plug-in, or what? I don't want 
another plugin. I accept already existant plugins, or the technologies 
I like, and it is REBOL. So, ROBERT is looking for the REBOL to browser 
converter. And it is news to me the world of web browsers supports 
Curl out of the box. It does not. And that is exactly the reason 
why Curl is absolutly zero solution here ...

For literal string delimiters, {~ followed by a space seems safe 
as I am unaware of where ~ is used followed by a space and at the 
other end I cannot think of where tilde is used followed by a closing 
curly brace.  Having Rebol always test for nested curl-braces in 
curl-brace values makes Rebol as difficult for my project as is Tcl 
- for the same reason.  Worka-arounds abound but the result has such 
poor usability and readability as to be unusable.

What is obscure about a syntax which permits literal strings to be 
literal strings?  Try assigning  set lit "{test} {" in your favorite 
Tcl interpreter.  I am not a JSON expert but [{ looks like JSON to 
me so #{[ "looks worrisome to me"   JSON or YAML or something other 
than XML is going to be important whether REBOL likes it or not. 
 Take RDF as one exmaple ( I prefer Topic Maps - please do not attack 
the example, but the isea ).  The fact that most people seem to think 
that RDF is XML does not make it so.  Tim Berners-Lee prefers some 
form of Triple notation for RDF.  Not XML.  As soon as a notation 
uses curly braces we have a problem using Curl on the server-side. 
 Please don't point to QM.  IT is not just tightly coupled to HTML 
it is married to it.  The web is not HTML it is HTTP with Content-Type: 
 set in the response header.  If that content type uses curly braces 
we have to start escaping characters in Rebol.  Awkward templating 
is dooked templating.  Let me repeat: doomed.  Folly.

1) it was one of the first RIAs back in the time, where world did 
not even know, what the RIA is - typical Amiga syndrome - concept, 
which was not embraced back at that time 2) then it slept for way 
too long, allowing Runtime Revolution, Curl, Adobe Air, Silverlight 
to take the place.

So now is the time to add ourselves to the above family ...

and this is the http request:  curl  -d '{"a": 1, "b": 2}' http://localhost:8080/?test

onetom ~/p/ob/jsondb $ curl http://localhost:8080/ver.r
2.7.8.2.5

i have an rsp like this:

<% probe delta-time [ ... switch request/method [ get  [...]   post 
[...] ]  ... ] %>

if i make a GET, it's fast:

$ time curl -s -D- http://localhost:8080/docs/rfq3 >/dev/null
0:00:00.003815
real    0m0.026s


if i make a post, the rsp part is still fast, but the overall request 
is damn slow (even consequent requests too):

$ time curl -d @xxx -D- http://localhost:8080/docs/rfq3
0:00:00.004595
real    0m2.034s

curl --http1.0 ... is fast

yes, the http connection will not close until it times out.


if the curl command line waits for ip close to return, its possible 
that is what is happening.

there are probably ways control this via some curl command line options?

Here is a workaround (by raping curl):


$ time curl -H 'Expect:' -d @xxx -D- http://localhost:8080/docs/rfq3
real    0m0.082s

regarding the Expect: 100-continue, i don't have any other realistic 
usecase other than this is the default behaviour of curl...

onetom ~/rebol/delete $ cat httpd.cfg 
modules [ internal extapp static rsp alias ]
globals [ listen [8888]    bind RSP to [ .r ] ]
default [ debug root-dir %./ alias "/test" %test.r ]

onetom ~/rebol/delete $ cat test.r 
<% print request %>


onetom ~/rebol/delete $ rm chey-*.log; rebol -qws ~/rebol/cheyenne-server-read-only/Cheyenne/cheyenne.r 
-w 0 -vvv


onetom ~/rebol/delete $ curl -X DELETE -D- http://localhost:8888/test
HTTP/1.1 404 Not Found
Server: Cheyenne/0.9.20
Date: Wed, 20 Apr 2011 11:14:04 GMT
Content-Length: 53
Content-Type: text/html
Connection: close

<html><body><h1>404 Page not found</h1></body></html>


>> p: open/no-wait/direct tcp://localhost:8888
>>  insert p "DELETE /show.rsp HTTP/1.0^/^/"
>> probe copy p
{HTTP/1.1 404 Not Found
Server: Cheyenne/0.9.20
Date: Wed, 20 Apr 2011 11:17:34 GMT
Content-Length: 53
Content-Type: text/html
Connection: close

<html><body><h1>404 Page not found</h1></body></html>}



u were doing 1.0 DELETE request though, but it didn't make any difference.

onetom ~/rebol/cheyenne-server-read-only/Cheyenne $ svn up
U    mods/mod-action.r
Updated to revision 131.

hmm... cheyenne binary and the source works differently w the same 
configfile.
$ cheyenne --version
0.9.20.129
WORKS
latest source doesn't work with the following config:

modules [ internal extapp static action rsp alias ]
globals [ listen [8080]    bind RSP to [ .r ] ]
guan-huat [ debug root-dir %./ alias "/docs" %jsondb/to-json.r ]

im testing w $ curl http://guan-huat:8080/docs/test/1

hmm... how can i map a directory under a certain path in vhost? im 
trying this:
yp [  root-dir %~/p/ob/yp  alias "/public" %../public/ ]

then for curl http://yp:8080/public/angular-0.9.15.min.js  i get

HTTP/1.1 301 Moved Permanently
Location: /public/angular-0.9.15.min.js/

wtf?

i remember seeing something like "/some/path" [ options ] in a vhost 
config block, but i can't find anything about it now

any example how do u test pages behind a session?

im trying curl -D- -d 'login=test&pass=letmein' -c jar http://localhost:8080/app/login.rsp

but subsequent curl -D- -c jar http://localhost:8080/app/some.html 
still gives me 302 to login.rsp

Dockimbel: i want the authentication. im just asking how can i test 
it programmatically easily. any idea whats wrong w the curl command 
lines?

onetom: sorry, I am not fluent in curl, I need to look for man pages 
for every options...Can't you use REBOL http client for testing?

Does curl handle cookies automatically?

i will read up on curl, im just asking how would YOU do it from rebol

i think i was using that once. it's just not default in rebol... 
:/ curl however is present on every machine i have access to...

so it's worth learning.

i was a wget user earlier, but since macs come w curl and i can easily 
apt-get install curl and even syllable comes w curl, i don't give 
a fuck anymore how an inefficient bloatware is curl, im still trying 
to use it and only it :)

curl -D- -d 'login=test&pass=letmein' -c jar http://localhost:8080/login.rsp
curl -D- -b jar -c jar http://localhost:8080/some.html
worked, btw

Curl is not on Windows, REBOL is.

Curl: I didn't know it was so widespread.

u know if rebol would come w a nicer header dump, cookie, https and 
url encode/decode support by default, i would advocate it as a curl 
replacement probably... :/

maybe an extension script would be enough, which anyone can remember, 
like http://json.org/json.r

Happens for my own extensions on both Win32 A110 and Linux A110. 
Also just tried it with Kaj's cURL extension, same problem.

When I split the cURL binding in two contexts, it works, so it seems 
to be some object limit

OK, let me upload the cURL binding source for you

How come? Computer is started. User may start a program on Monday 
that needs to load cURL extension. May decide to start a program 
on Tuesday that needs to load 0MQ extension

Do you have a problem with making an embedded delayed extension that 
loads cURL dynamically when the extension is imported? This is supposed 
to be possible, and is the main reason for delayed modules and extensions. 
Then only programs with Needs: [curl] would initialize curl.

Nonsense. R3 is cross-platform, cURL is cross-platform, and the extension 
compiled on Linux even runs on Syllable

R3 is cross-platform, and cURL is cross-platform, but both require 
a little work to make the platform distinctions go away. POSIX helps 
with this on POSIX platforms, but that's not everything. You can 
do a little extra work in your extension to make it work the same 
externally, so R3 code can't tell the difference, but if you want 
that to extend to filenames then you have to pick a cross-platform 
filename standard. Once the extension is loaded, embedded or delayed 
it can be referred to by module name. We don't want IMPORT to grab 
extensions in its module-paths lookup because that would make it 
possible to load an extension when you were looking for a safe module, 
so .rx and .so files aren't going to be in the search list unless 
your program sets system/options/default-suffix to .rx or .so, with 
the same security implications.

Do you have a problem with making an embedded delayed extension that 
loads cURL dynamically when the extension is imported? This is supposed 
to be possible, and is the main reason for delayed modules and extensions. 
Then only programs with Needs: [curl] would initialize curl.

You may want to review my documentation for the cURL binding

All you have to do to use it, is to import the 'extload module first, 
before you import any native extension. After that, use the generic 
suffix to load extensions.


For example, let's assume you want to load your platform's cURL-binding.* 
extension:

import %extload.r3
import %cURL-binding.rx


On Linux (in the system/platform/1 sense) and FreeBSD, this will 
first attempt to load %cURL-binding.rx and if that fails, %cURL-binding.so.


On Windows, %cURL-binding.rx and %cURL-binding.dll are tried (in 
this order).


On OSX, %cURL-binding.rx, %cURL-binding.dylib and %cURL-binding.so 
are tried (in this order).

Filenames not ending in the generic suffix are left as-is; so event 
with extload present, if can use e.g. import %cURL-binding.so directly 
and bypass the suffix translation mechanism.

There's a callback example in the cURL binding, for progress info

Just an object. Please refer to the cURL source and documentation

You could try to have the progress callback in the cURL binding pass 
a series. cURL is singlethreaded, so this would establish whether 
it's a threading problem or an R3 memory management/callback problem

For what it's worth, the cURL binding supports proxies

really, if we have to choose between encryption and threads... there 
is no contest.... all the "usability" stuff we can code as extensions 
and indeed, the cURL binding is a good example of this.


we need threads to be done... they have an over-arching effect on 
every aspect of REBOL... we can't put this off until later... its 
going to change the design of things for sure.  I can't understand 
why Carl is side-stepping this again.

is carl even aware that there is a cURL binding right now and that 
maybe he can skip this for now?

Carl has handled cURL related bug reports, but chances are he doesn't 
know what it is :-)

in any case a tool like cURL is always going to be better since its 
being developped using standard tools which evolve and keep pace 
with changing specs and are bug fixed on their own.


I'd rather that we be able to wrap cURL within schemes properly and 
that they be fixed or enhanced to make it happen.

the core already has enough features... we need more architecture 
work and polishing.

Brian tells me it should be possible to wrap cURL's features in R3 
schemes, but I can't make it a priority yet

I have doubts that scheme wrappers can support all efficiency tactics 
of cURL, but all REBOL scheme features should at least be possible

I have a question for Kaj .. does the curl binding mean we can use 
https as a scheme now?

You can use SSL in the cURL binding, but I haven't written scheme 
wrappers yet. Due to incomplete documentation, that's going to be 
a lot of work

Would Curl-to-R3 be able to solve the NTLM challenge that I have 
... similar to Graham's HTTPS for everything these days?

Thanks. Doc and I have had a series of discussions on it where he 
has helped me out. In particular was Cheyenne being able to pull 
IIS information authenticated to a Windows domain for presentation 
from Cheyenne. We got to ... some more detailed / dedicated Doc work 
needed because NTLM is tricky.   That's not an option now because 
Red is a priority for everyone including me.  But if Curl enables 
it ...

cURL supports NTLM

I guess cURL solves the https

I'm all for anything that moves us forward, but solutions like cURL 
and CALL via an extension means I know I'm going to have to change 
things later, or live with more pieces.

I would add following "negatives" (depends upon how you look into 
it):


- no /libary extension and easy wrapping of DLLs. There was a bounty 
started to bring in kind of R2 DLL capabilities using extensions, 
Max was working on something, but did not deliver. Some ppl claim, 
that working with extensions is easy enough, much more powerfull, 
and that in fact R2 /library interface was weak in comparison in 
capabilities.


- weak and underpowered CALL.No /output or /wait parameter IIRC. 
Carl said, that R2 C code to it was complex, and that the code is 
eventually awailable for volunteer to bring in to R3. The outcome 
is - CALL is limited in usage in comparison to what can be easily 
achieved in R2.


- protocols. The only protocol IIRC was available was HTTP, done 
by Gabriele. It was HTTP 1.1 compatible, but due to some bug (?) 
it was downgraded to 1.0 version. No proxy support. Other protocols 
were done by some other ppl, I do remember Graham doing some work 
here. In regards to protocols, IIRC there was some work done by Kaj, 
who brought Curl networking extension to R3.


- under Windows console is a bit more inconvenient in usage than 
in R2, we use native Windows console, yet we don't have full console 
support, so we can't replace the native R3 one by e.g. Console2 or 
some other version ...


- DBAccess - forget R2 protocols available. The rescue is ODBC extension 
for R3


- CGI - no native CGI support in R3, though it should not be difficult 
to emulate


- Sorting & Unicode - althought we have Unicode strings available, 
sort is not adapted to that, and the question is, if it can be easily 
done ...

There has only been one issue in the modukle system itself (aside 
from the possible cURL thing) that has been discovered since a110, 
and that is not as much a bug as it is a gotcha, and hasn't been 
reported yet.

http://www.osnews.com/story/24437/Syllable_Gets_First_REBOL_Binding_With_cURL

I thought you did a talk on cURL binding so R3 was represented

`curl ... | hd` gives me `61 0d 62 0a 63` as desired

Why should git be dependent on curl-config ?

Let me see if I can find a curl-config that runs... then I can try 
the build again.

(that contains curl-config)

try `make NO_CURL=1`

rebuilding with NO_CURL=1

Not sure what you've mentioned, but here is my config output:

core.symlinks=false
core.autocrlf=true
color.diff=auto
pack.packsizelimit=2g
help.format=html
http.sslcainfo=/bin/curl-ca-bundle.crt
sendemail.smtpserver=/bin/msmtp.exe
user.name=
user.email=
core.autocrlf=true
core.safecrlf=false

For porting the cURL binding, I will need callbacks. I'm sure cURL 
on most systems expects them to be cdecl

If you want to run Desktop, 0.6.6 indeed has a ready VMware image. 
On the other hand, the live CD you have has some enhancements. Such 
as R3 and the cURL binding :-)

Wget is not onboard. Curl is

By the way, do we want to keep PRIN? My toes curl a little every 
time I see it. I know it's longer, but I would be OK with replacing 
PRIN & PRINT with PRINT & PRINT-LINE - or PRINT/LINE in Red proper

I've started working on a cURL binding. It can print the cURL version 
now :-)

looking forward to more weird bugs discovered by the curl binding 
:)

Hm, a cURL progress callback needs floats. It's pretty limiting if 
you can't get progress feedback

Got cURL to read a website and print it:

curl-begin curl-global-all
session: curl-new-session
curl-set session curl-set-upload?  as variant! no
curl-set session curl-set-url  as variant! "http://rebol.com"
curl-do session
curl-end-session session
curl-end

curl-begin
session: curl-new-session
curl-read session
curl-do session "http://rebol.com"
curl-end-session session
curl-end

Of course, if you use the 0MQ binding in combination with the cURL 
binding, you can do multiple concurrent uploads from disk only

A callback used internally by cURL is fine, yes, on Linux. We'll 
see what a progress callback does with trying to print floats

But that was earlier, wasn't it? Then I have only tested the latest 
callbacks with cURL

I've extended the read function in the cURL binding to store a file 
being received directly to disk, instead of just printing it

I've confirmed that the Empty and Hello examples still run on Syllable, 
but the cURL binding currently segfaults

The cURL binding seems to work there, too

Kaj, do you have gdb on Syllable? If so, please run the curl example 
via gdb and add the output of "backtrace" to the bug report. Thanks!

Sounds like an invitation to rapidly fix the cURL binding issue. 
;-)

The problem with the current GDb on Syllable is that it doesn't know 
how to load shared libraries, so it can't load cURL

The issue with shared libraries seems to be limited to the fact that 
cURL can use libdl. I don't know what was different now, but I got 
a GDb trace. Will put it on the tracker

The crashes seem to happen when cURL calls the C library, so outside 
the Red program. I'm not sure it's useful to disassemble a minimal 
Red example

Got the cURL binding to work on Syllable, starting from Andreas' 
example. :-) Will research some more and report on the tracker

I'm about to for cURL, to receive an unknown quantity of data into 
memory

I've cleaned up my cURL binding enough to put it in a repository:

http://red.esperconsultancy.nl/Red-cURL

cURL link added to Contributions section: http://www.red-lang.org/p/contributions.html