World: r3wp

More about routines.

The next release of World will introduce the handle! datatype. It's 
used to hold pointers, which are normally handled by structs and 
integers in R2.

If we take the sqlite3_open routine as an example:
http://www.sqlite.org/c3ref/open.html


The routine takes a pointer to a pointer as its 2nd argument, and 
this is the db handle updated by the routine. The handle is again 
used in sqlite3_close:
http://www.sqlite.org/c3ref/close.html


, but this time, it's the handle itself as the argument, not its 
address.


In World, using routines should be as easy as possible, and the consequence 
is, the definition of the routine gets some complexity. World uses 
libffi as the underlying motor to carry out the calls. libffi defines 
15 datatypes, where World uses 14 of them (not longdouble). Beside 
"pointer", I will introduce "pointer-adr", and routines like sqlite3_open 
can use that. Then the address of the handle will be given as an 
argument. Routines like sqlite3_close should just use "pointer", 
and World will internally typecast the handle to an integer (32- 
or 64-bit depending on version), and give that to the routine.

The routines could then be defined as:

sqlite3-open: make routine! [
	[typecheck]
	sqlite "sqlite3_open" [
		filename [string!] pointer
		ppDb [handle!] pointer-adr
	]
	sint
]

sqlite3-close: make routine! [
	[typecheck]
	sqlite "sqlite3_close" [
		ppDb [handle!] pointer
	]
	sint
]

looks good

clear enough.

Notice that the world-datatype after sint (the return type) will 
be optional too, even when typecheck is on. The argument names are 
optional too. The World types of the arguments are not optional, 
if typecheck is on, else they're just treated as comments and can 
be left out, but not their blocks, as those are used to count no. 
of arguments.


Btw. argument types in functions and operators are treated as comments 
too in World. I plan to introduce [typecheck] special attribute for 
those too.

Naming pointer-adr, I considered also *pointer, but found it too 
C-like and 'pointer, so didn't quite like the syntax, even if it 
lead the thought to call-by-word.

I also considered leaving pointer-adr out, and let the call define, 
how the routine should be called. Something like:

h: make handle! none
sqlite3-open "test.db" 'h		; notice the lit-word!


But this will slow calling routines down, because the call will have 
to look h up.

So I ended with: pointer-adr

The reason, I put the world-types of the arguments into blocks is, 
that in the future, polymorphism could be introduced for routines. 
So routines could be called with different types of arguments, and 
internal conversion will then happen. This is for the future. Second 
reason is, that argument names are optional, so parsing the routine 
spec needs some way to figure out, what is name and what is type.

Today World types in routines are considered, when typecheck is on, 
and conversion is then carried out between the World type and the 
C type for the arguments, and the C type and the World type for the 
result. Examples with "clock" from libc under OS X:


w> clock: make routine! [libc "clock" sint64]	; The simple version
w> clock
== 79551135

w> clock: make routine! [[typecheck ]libc "clock" sint64 real!]		; 
Result as real!
w> clock
== 79576741.0

w> clock: make routine! [[typecheck ]libc "clock" sint64 complex!]	; 
Result as complex!
w> clock
== 79621776+0i

just brainstorming... don't you want to use something like datatype 
notation for world's types as well?

World types as exclamation mark (!) in the end. C types don't. Do 
I misunderstand the question?

as -> has

In the above, sint64 is a C type, real! is a World type.

(Shouldn't I had said: "World types *have*", because it's plural?)

New release at https://github.com/Geomol/World

- Added handle! datatype
- Gave routines an overhaul
- Added AS native to change type of series without copying.
- Fixed crash with function redefinition to reset compile state

- Updated cortex_alpha.pdf, the section about routine!, and added 
handle! and AS.

Regarding AS, the REBOL AS-BINARY and AS-STRING can be achieved with:

as binary! ...
as string! ...

But it's also possible to do:

w> as issue! "abc"
== #abc
w> as paren! [a b c]
== (a b c)
w> as tag! "title"
== <title>

and many other combinations.

Performance wise, it's of course faster to use AS than e.g. TO:

w> dt [loop 1000000 [to binary! "abc"]]
== 0:00:00.420204
w> dt [loop 1000000 [as binary! "abc"]]
== 0:00:00.133765

People trying out the routine! implementation, please speak up, if 
you find anything strange, so I can fix it, while it's fresh in memory.

Handles are made with:

	handle: make handle! none


Some routines might return handles, and those are not necessary to 
made yourself.

wow a lot of activity here since I last checked...

one little note about handle! is that it should be immutable and 
cannot be created from within the interpreter beyond defining a null 
pointer, as above.   value holding handles should only be returned 
from a routine.


the idea here is that you should not be able to mangle handles, for 
a variety of reasons.

its tempting to be able to do:  

i: to integer! my-handle
i : i + 1
some-routine i


but its very bad practice (from within a scripting language) IMHO 
since it allows all manner of crashes to occur, not to speak of security 
issues this can open up as well.

Yes. I made it possible to view the content of a handle with MOLD. 
That should be ok.

peeking is ok (often neccessary for debugging), as long as you can't 
change it  :-)

Right.

btw VERY happy you kept the lib-ffi type names within your routine! 
implementation  :-)

Having all the libffi C types should cover all possible libraries. 
(I guess.)

struct! isn't there yet though. I think, structures are needed for 
some routines? Else it's a hard job juggling with binaries.

Structures are needed for almost all routines

yep.

my only suggestion for structures...  make it so the semantics are 
exactly the same as C.    
i.e. you take a C header file, and do:

replace/all header-string "{" "["  
replace/all header-string "}" "]"  
replace/all header-string "char *" "pointer"
...


and basically you've got your simplest World version of the spec 
 ;-)

Cool, I'll check that out.

also, I would really like if the return value setup for a routine! 
could include an optional  function block which is run on the return 
value.  

it would prevent the very common instance where you have to write 
stubs for the routines (such as normalizing return values on errors 
or converting values to some better internal mechanism).

Better error! implementation is high on my list. Also needed to implement 
TRY.

btw, regarding the struct interface, making as close to C as possible 
allows you to support any binary api and will allow you support the 
more complex twists.


things like arrays of pointers and pointers to arrays of pointers, 
etc... these things are often overlooked (like they where in R2) 
and create major headaches or downright project blocking in some 
cases.

Geomol, I am reconnectin to the Rebol3 world after a long time of 
absence.

It comes as surprise you new REBOL inspired new language. It is inevitable 
that things like TOPAZ and RED are created after the long absence 
of CARL. There is a stong need from the market and the market is 
US programmers which cannot wait any longer.

First of all a question: which will be the difference and improvements 
comparing to REBOL ?

Now will come my considerations about Licensing and development.

Difference: Based on a virtual machine, that source can compile to. 
Many different design decisions, but many of those can be redone 
to be REBOL compatible by defining words. This is what %rebol.w is 
for.


Improvements: Faster in many cases because of the VM. I try to make 
it better in all those areas, where I find REBOL not good enough. 
Datatypes like complex! and more in the future. I also try to cut 
into the bone and only create that at the core, that is really needed. 
I see that as an improvement.

You are the project LEADER and you are the creator of this wonderful 
project. You have a VISION, a destination.

Back in time, when CARL was developing REBOL3 he created a roadmap 
explaining us what would be the structure of the language, its modules 
and how they cooperate.

This inspired suggestions from the community, changes in the roadmap 
but more preciosuly it created expectation, commitment and willingnes 
to cooperate.

I feel more like only reveal new stuff, when it's just about to emerge, 
or is already done. I've seen too many promises, that didn't materialize.

I'm very focused to get to version 1. When that's done (or almost 
done), it makes more sense to launch parallel projects. If someone 
offer me a business opportunity, I'm willing to change my focus or 
work on specific projects.

(Please, give me the opportunity to finish myt talk. I am quite slow 
at english writing !)

np :)

You are a programmer but you should be an inspirator and entrepreneur. 
Like a programmer you first released the alpha version before selling 
the idea ! :-) You came out from your BatCave after 2 years of silence 
telling us "look here what I have created". This is good as it proves 
you deliver other than promising. 


Somewhere, in the road to version 1.0 you will need help. It belongs 
to you teling us when you will have clear in your mind the whole 
project. Then, please SHARE THE VISION and ask for cooperation and 
involvement of other programmers.


You are not alone. There is lot of skilled people here. They will 
be happy to give their input to you.

This will avoid being World as a one man project and will put the 
basis for collaboration.

When you will be ready for this I ask you to make few thing:

1) Create a white paper with your vision to let us know what is in 
your mind
2) Set the goals of the project.
3) Open the source but maintain the leadership of the project.
4) Ask for cooperation
5) Build the comminication/documentation infrastructure .

I have read you are somehow confused about how to make revenues from 
your project. First of all lets alltogether finish it.