AltME groups: search

We have been aware that pointer [integer!] does not accurately describe 
a pointer [byte!] but you don't want to implement that in the current 
development cycle and that's fair enough. It's only a cosmetic problem 
right now

Will the NULL enhancement change the types handled by ALLOCATE and 
FREE to one generic pointer definition? Let's stop calling it void, 
but instead variant, which it really is

There is no generic pointer in Red/System.

Even if binary! was there, it would work as a byte! pointer.

I thought so, so ALLOCATE and FREE will still deliver types that 
I wouldn't expect from them, and that effectively introduce an extra 
definition of a variant pointer

But the more tangible problem is that currently, you don't have to 
cast when you allocate a string, and that will change when a proper 
byte pointer is introduced

Kaj: you have now a polymorphic NULL for all pointers (struct, pointer, 
c-string) and a new pointer! [byte!] type. byte-ptr! has been redefined 
to pointer! [byte!] instead of c-string!.

A long-term solution could be to disable buffering of C's stdout 
stream, using a call to setvbuf(). It requires passing a stream pointer, 
so I am not sure this is doable now.

NULL can't be used any more as the return value for a function that 
normally returns a pointer:

It is caused by NULL clashing with EITHER return type static analysis. 
I am giving NULL an any-pointer! pseudo-type to solve that.

*** Compiler Internal Error: Script Error : compare-func-specs expected 
cb argument of type: get-word 
*** Where: check-expected-type 
*** Near:  [unless any [
all [
block? expr 
object? expr/1 
expr/1/action = 'null 
type: expected 
any-pointer? expected/1
] 
all [
type 
any [

If you want the function to return a logic! status and a result to 
be able to handle it locally, the result needs to fetched through 
a pointer, which also raises complexity. We can't handle addresses 
very well yet, either, because get-words only work for native functions

Red/System will go beta as soon as I:
- fix the remaining bugs from the tracker,

- decide on 2 last syntax issues discussed on the ML (concerning 
struct/pointer literal syntax and aliases)
- make a pass on the specification to fix/update/complete it.

Kaj: we have decided to replace the "struct" and "pointer" by "declare". 
Please read this post: http://groups.google.com/group/red-lang/browse_thread/thread/9c407676e2335919?hl=en

16 bits arrays also need to be manipulated, with 16 bits access and 
pointer arithmetic

Don't the count and the pointer count?

Also for fixed arity functions, the function spec block might become 
quite verbose:


foo: func [[typed] arg1 [...] type1 [integer!] arg2 [...] type2 [integer!]...] 
[ ... ]


Maybe the type numbers could be packed in a byte array and passed 
as an array pointer to make it easier to declare and use...?

There are no "standard" arrays in Red/System though I'm sure that 
there will be, or something which provides the same functionality 
in Red (when it becomes available).


args is not a standard array and is navigated through pointer arithmetic 
just like  a c-string. I suspect that args points to a list of pointers 
each of which contains the address of a command-line argument.


So the first entry in Args holds the memory address of the first 
command-line entry, the second the second command line entry, etc.


The syntax args/item gets you to the data pointed at by the relevant 
entry in args.

So the compiler doesn't treat item as an index but as a command to 
derefence the pointer heid in the current position of args.

You can also do it with an index, but indexes are one-based. With 
pointer advancement you usually get the same effect as zero-based 
indexes

I think it should be reasonable enough to choose cdecl when a function 
pointer is passed to a variadic function.

The current resolver only works for simple cases, basically, only 
when you're directly passing a function pointer as argument to a 
function.

I don't really see a connection between the function a pointer is 
passed to and the function that will eventually call it

Well, it depends if the function taking a function pointer is invoking 
it directly or if it's passing it to another function (local or imported). 
In the former case, the compiler can set the right cconv itself, 
in the latter case, it can't and need some help from the user.

For float!, you need to do an extra type-casting on the arguments 
pointer to convert it to typed-float! instead of typed-value!. You 
have an example of that in %red-system/runtime/utils.reds in _print 
function.

Why is not c-string! available in pointer-syntax rule?

Because, pointer [byte!] == c-string!

is it possible to get pointer value from variable declared as c-string?

If you mean a pointer to the string payload, that's what a c-string 
already is

Kaj: my problem is, that I have a routine: MagickRelinquishMemory 
which should release memory from routines where some returns c-string, 
and some for example pointer to array values... So the best would 
be to accept pointer! with MagickRelinquishMemory, but still to be 
able return just c-string from some routines, if you understand me. 
I know there are workarounds, but it could be quite simple

What about implementing any-pointer! type?

and is it possible to cast pointer from c-string?

You can freely cast any pointer type to any other pointer type (struct 
and c-string are part of any-pointer! internal typeset).

I would prefere to be able simply define:   MagickRelinquishMemory: 
"MagickRelinquishMemory" [return: [any-pointer!]] which would accept 
c-string!, struct!, pointer! [byte!], pointer! [integer!] etc..

sorry..   MagickRelinquishMemory: "MagickRelinquishMemory" [resource 
[any-pointer!]]

You can already have it: #define any-pointer! byte-ptr!

After adding the float! support, the main thing is a series! datatyte. 
Should we wait for Red to implement it properly or do we want ti 
in red/system as well? (I know I can use pointer math to access values, 
but it's not so easy for ordinary scripting)

I think so, but the C code is also doing an indirection through a 
function pointer, hence the fp

typedef int (WINAPI *LSN_GETBRIGHT)()


^^ That defines LSN_GETBRIGHT to be another name for a pointer to 
a stdcall function that takes no arguments and returns an int. (WINAPI 
== __stdcall)

extern LSN_GETBRIGHT		fpGetBright	;


^^ That now is the actual "function", although held indirectly as 
a function pointer. So somewhere in a library is a fpGetBright value 
which holds a LSN_GETBRIGHT function pointer.


In C you can directly call that as e.g. `int brightness = fpGetBright();`. 
However, I fear in Red/System it's at the moment not possible to 
perform that call.

Then for each function you have to write a Red/System wrapper function 
and pass it the function pointer as if it were a callback function, 
and call it. There are examples of such constructs in my bindings

Hm, I'm not sure either if the wrapper function can currently be 
written like I said. You'd have to use the specific declaration as 
function! type for the function pointer, but the Red/System type 
checker may not currently support getting the function pointer in 
the right type

I've found the easiest way to raise that runtime error is to accidently 
treat an integer as a pointer.

This code will cause an access violation as clock in libc returns 
an integer not a pointer to an integer.

pi: declare pointer! [integer!] 
#import [
	LIBC-FILE cdecl [
	  read-cpu-clock: "clock" [
	    return:    [pointer! [integer!]]
	  ]
    ]
]
pi: read-cpu-clock
pi/value

I find this better:

x  =>  string!



->    looks more like a "move to here" (which is why they used it 
for C pointer struct dereferencing IMHO)

=>   having an = symbol, looks more like a "make into"  (for which 
"to" is a synonym).

Oldes, complex numbers hold 2 double = 128 bit, then OP-code, register 
pointer and maybe some more.

Uh, I'm not 100% on the technical terms. Registers are on a stack, 
and a register pointer is an offset. So you can have lots of registers, 
and the VM can access each of them very fast. Values can also float 
in memory (not on stack), and then we have a real pointer to it. 
See blog tomorrow.

Defining routines is very flexible, a bit against my simplistic philosophy 
for World, but now it's done. Having e.g. libc (OS X example):

libc: load/library %/usr/lib/libc.dylib

Defining PUTS can be as simple as:

puts: make routine! [
	libc "puts" [
		[string!] pointer
	]
	sint
]

or as full of info and features as:

puts: make routine! [
	"Writes a string to stdout followed by a newline."
	[typecheck]
	libc "puts" [
		string [string!] pointer "String to write to stdout"
	]
	sint integer!
]

At present without typecheck, World think string! when seeing a pointer 
in a routine spec. And the World types are just seen as comments. 
With typecheck on, the Would types are considered, and only certain 
combinations of World types and libffi (C) types makes sense, like 
string! and binary! for pointer.


I'm about to implement handle!, and maybe handle! should be the default 
for pointer. And then it's required to specify typecheck, if string! 
or binary! want to be used with pointer.

More about routines.

The next release of World will introduce the handle! datatype. It's 
used to hold pointers, which are normally handled by structs and 
integers in R2.

If we take the sqlite3_open routine as an example:
http://www.sqlite.org/c3ref/open.html


The routine takes a pointer to a pointer as its 2nd argument, and 
this is the db handle updated by the routine. The handle is again 
used in sqlite3_close:
http://www.sqlite.org/c3ref/close.html


, but this time, it's the handle itself as the argument, not its 
address.


In World, using routines should be as easy as possible, and the consequence 
is, the definition of the routine gets some complexity. World uses 
libffi as the underlying motor to carry out the calls. libffi defines 
15 datatypes, where World uses 14 of them (not longdouble). Beside 
"pointer", I will introduce "pointer-adr", and routines like sqlite3_open 
can use that. Then the address of the handle will be given as an 
argument. Routines like sqlite3_close should just use "pointer", 
and World will internally typecast the handle to an integer (32- 
or 64-bit depending on version), and give that to the routine.

The routines could then be defined as:

sqlite3-open: make routine! [
	[typecheck]
	sqlite "sqlite3_open" [
		filename [string!] pointer
		ppDb [handle!] pointer-adr
	]
	sint
]

sqlite3-close: make routine! [
	[typecheck]
	sqlite "sqlite3_close" [
		ppDb [handle!] pointer
	]
	sint
]

Naming pointer-adr, I considered also *pointer, but found it too 
C-like and 'pointer, so didn't quite like the syntax, even if it 
lead the thought to call-by-word.

I also considered leaving pointer-adr out, and let the call define, 
how the routine should be called. Something like:

h: make handle! none
sqlite3-open "test.db" 'h		; notice the lit-word!


But this will slow calling routines down, because the call will have 
to look h up.

So I ended with: pointer-adr

one little note about handle! is that it should be immutable and 
cannot be created from within the interpreter beyond defining a null 
pointer, as above.   value holding handles should only be returned 
from a routine.


the idea here is that you should not be able to mangle handles, for 
a variety of reasons.

my only suggestion for structures...  make it so the semantics are 
exactly the same as C.    
i.e. you take a C header file, and do:

replace/all header-string "{" "["  
replace/all header-string "}" "]"  
replace/all header-string "char *" "pointer"
...


and basically you've got your simplest World version of the spec 
 ;-)

Example of using struct! in routine definition:

	my-routine: make routine! [
		library "routine" [
			[struct!] pointer
		]
	]

A real example under OS X:

timeval: make struct! [[
	slong sec
	sint32 usec
] none]

timezone: make struct! [[
	sint minuteswest
	sint dsttime
] none]

gettimeofday: make routine! [
	[typecheck]
	libc "gettimeofday" [
		tp [struct!] pointer
		tzp [struct!] pointer
	]
	sint
]

w> gettimeofday timeval timezone
== 0
w> timeval/sec
== 1323951188
w> timeval/usec
== 314011
w> timezone/minuteswest
== -60
w> timezone/dsttime    
== 0

I was able to load MagickWand under Linux, and it seems to work with 
uint32 datatypes to hold the handle (a C pointer). But it doens't 
work so well when using the handle! datatype for that. It would be 
nice, if it worked, I guess. It's probably some type casting problem.

Suggestion:


Some routines return a pointer to a structure, like LOCALTIME (from 
LIBC). The structure is struct tm and consists of 11 fields.


In World, we can define LOCALTIME to return a pointer to a handle!, 
but how should we get to all the data fields?


I suggest, TO should be used to copy data from a handle to a structure, 
like:

tm: struct [
	sint sec
	sint min
	sint hour
	sint mday
	...
] none

h: localtime time	; time is some variable holding seconds
to tm h	; This will copy the data (pointed to by h) to tm

Comments? Concerns?

As I understand localtime is not thread safe, the thread safe version 
locatime_r requires the address of a tm structure as an argument. 
(Though it returns the pointer to that same structure if the call 
is succesful.


In general, isn't it a better option for strucutures to be allocated 
in World rather than the called function?

Yes, it's probably a better idea to use routines, where you can allocate 
the structure in World, and handle the routine a pointer to it. But 
some routines does the other thing. In the case of localtime, it's 
a static buffer. Some routines in some libraries dynamic allocate 
memory, that the user can deallocate again with other routines. (Oldes 
pointed me to such a case in ImageMagick.)


If World should support calling such routines and be able to operate 
on the result, we need something like my suggestion, I think.

I don't think so, as it's a simple memcpy. The C code looks like 
this:

	if (rb->type == STRUCT_T) {
		if (rc->type == HANDLE_T) {
			Struct *U = (Struct *) rb->value.rc;

   memcpy (U->u, (char *) ((Handle *) rc->value.rc)->pointer, U->size);
		} else
			invalid_argument (W, rc);
	} else
		invalid_argument (W, rb);

Routines able to operate on structures, you define in World and give 
a pointer to to the routine, doesn't need this memcpy, and it'll 
work today.

I'm not sure about allowing typecasting from handle!.  it sort of 
defeats the purpose of an opaque pointer handler IMHO.   the better 
alternative would be to allow the routine to define what struct pointer 
type it returns directly.