AltME groups: search

Ok, right. Using int32 and ser as struct names.

Same as APPLY, actually. Fortunately the C implementation and the 
REBOL declaration are bundled together, so you tend to be the one 
setting the positions in the first place. This makes the whole process 
easier.

Not at the moment. That is as good a method as any for now. Maxim 
has beeen doing some research on this, and the device model is supposed 
to solve this problem in the long run.

That's bad because it's IMO an enabler and promoter for R3. As long 
as the GUI is missing, at least R3 can be used on the server with 
extensions.

The API is versioned for exactly this reason. Carl came up with enough 
of an extensions API to actually function and to let people experiment 
with various techniques to make it better. Carl is not the only designer 
of R3 - we all help, and need to. We can't know how to design the 
extensions API until we get an idea of how it will need to be used.

I have been waiting for extensions for a decade, and its almost there.

a lot of stuff depends on the improvement of extensions and addition 
of device extensions.  not just for me but for Carl also.

Unfortunately I am not at liberty right now to tell what that is, 
but I can assure you extensions will have to improve in the short 
term because a new player (company) in the REBOL community needs 
this, already.


this company might become one of the levers to propel REBOL into 
adoption in (several) very large corporations (fortune 500) & scientific 
organisations around the world, so RT has vested interest into doing 
as much as it can to make this happen... and right now... the host 
code and extensions is the key to most of it.

Rebolek, that is not really true - the function still takes a fixed 
number of argument, and you're just passing a unset! value to some 
of them (which is a side effect of R2 passing unset! at the end of 
the block, i think R3 does not even do that)

Gabriele, you're right that it's just a R2 side-effect and it's true 
that it does not work in R3. Not that I miss it.

Could someone write gzip compress and decompress functions for rebol3 
extensions? what the cost would be?

Robert, if you are good at C macros and have a good idea about how 
to improve things, make suggestions. Good safe methods for bulk copying 
of string or binary data into the REBOL values to be returned, or 
from values passed in would be great. Look at the existing extension 
source for an idea about how the current macros work. Safety is a 
priority here, so don't forget the bounds checking.

Ok, so I've not yet provided everything that you'll need to do it. 
I divided the
 extensions release into a few stages:

1. simple - 
just simple access to commands and args
2. series - access to series 
values of various types
3. objects - access to objects (of all types)
4. 
codecs - support for codecs
5. host-lib - ability to bundle extensions 
with the host-lib itself.

So, I need to get you a bit more... in 
fact along the lines that you mention.

Re #6156: Pekr, we ARE NOT giving up on dialects!! There are many 
dialects in RE
BOL, and it is one of the main concepts.

What we 
are doing is removing the strong overlap in DELECT and COMMAND. If 
you l
ook at the DELECT method, it is a small subset of full dialects. 
It implements a
 form of function with optional arguments.

So, it's 
better to move that code into COMMANDS, and allow them to work that 
way
. This makes it much easier for people to learn and use. Even 
me!

Also, REBOL/Services will use this same method, because COMMANDS 
are not limited
 to just extensions... ah the secret is out: COMMANDS 
can also be attached to a
context, making them generally useful in 
REBOL code.

I will check the blog comments to make sure it's not 
misunderstood.

Please could someone translate to me, what does it mean that COMMANDS 
can be attached to a context, and that it will make them useful in 
REBOL code? :-)

The way I see it is  that the code inside a command probably can 
be late bound to a context, rather than the global context, as it 
is now.


when extensions will support objects, this can be pretty powerfull, 
since commands can become virtual and private methods for an object 
where the data is stored in a stuct in the binary (C) side... 


which is EXACTLY what I need for liquid, where I need rebol dispatchers 
but native data storage, so it can scale to hundreds of billions 
of nodes, and yes I already have the solution for the storage/memory 
engine if Carl can give me the means.   :-D

I already found a way to make callbacks extension callbacks in the 
current host distribution, even if nothing in the current rebol native 
code supports it  :-D


will be testing this out tonight and will report on this... I hope 
my idea works.  this would reactivate the OpenGL project along with 
other stuff on the backburner.

It's the dispatch. Right now with extensions, when you make a command! 
it makes a function that is dispatched by a function in the extension 
based on a number (which you can think of ay a key), to code that 
handles the command (the value associated with the key). In theory 
this is not that different from an object! grabbing data from one 
of its slots based on the keyword you pass it. Apparently commands 
will be able to dispatch to objects soon, and the functions assigned 
to slots of that object will handle the command code.


The DELECT dialect model was based on rebcode, mostly on its JIT 
binding. DELECT added the out-of-order, possibly optional argument 
handling to the dialect decoding phase, but the dispatch phase was 
mostly left out (I commented on this at the time). The command! type 
has the dispatch model, but uses the function call model for calling 
the commands. The overlap that Carl mentions is in the mapping of 
keys to command handlers.


If you unify the command mapping models between DELECT and command!, 
then that code can be shared. This means that the DELECT function 
could do the out-of-order dialect decoding, then dispatch the operations 
as commands. Values of the command! type would continue to be called 
like regular functions in DO code or by APPLY, and then dispatch 
using the same dispatch code as above. On the other end, commands 
would either dispatch to objects (including modules perhaps) or extensions.


By the sound of it, this might also allow the command! type to serve 
as a method pointer, but we'll have to wait to see about that :)

btw... I wish there more host <-> r3lib hooking.   I really wish 
he'd push some of the extension handling code into the host.  right 
now there is no real extension code within the host, and there is 
no integration possible from new runtime features into the extension... 
basically, the extensions are running blind.

just a single place where we can put data which is accessible by 
extensions.  that would already make the host that much more usefull, 
especially for testing new host models or devices.  which add new 
possibilities for extensions.


the event device is also not useable for my specific task and I'm 
not sure I can really play around with it without breaking the r3lib 
<-> host integrity... testing will provide clues, I guess.

and we are talking (non web) commercial rebol work here.

I know you are really chomping at the bit for callbacks, and were 
really hoping that the host code would allow you to hack them in 
yourself, but you're out of luck there. Your callback proposals didn't 
take tasks into account anyways. You're still going to need device 
extensions. Fortunately, you can probably help design the model for 
those :)

I think I could have something working for non multi-threaded stuff 
in a little while... I'm working on this now... its the time I have 
to do it ... after that...
I return from a sanity preserving week of vacation.


and yes I hope I can help with the development, especially since 
I have two different devices which need to be added.  

with the current host I might make my hackback do some usefull tests 
to help shape a working model aka prototype.

Well, with the current host code you can add devices in the host, 
and don't need to wait for the device extension model :)

I've been 3 times to the dentist in 2 weeks... (and going back next 
week!)...  needles and my gums are practically on first name basis 
as of now  :-(.

but the coupling with the core run-time is practically abscent.  
there is only one function I can use to have the run-time do anything 
and thats a pretty simple... do_rebol_string() which basically runs 
a block of code in the global space... beyond that I've only got 
network/file like ports, which basically are streamed I/O.  


I can't create data directly and leave it at the port, in a block, 
like I'd do for a proper event queue.   This is currently my pet 
peeve about the host... 


but let's not be judgmental... I'm VERY happy I have the host, so 
I can at least try to rig something up with bailing wire, duct tape, 
pliers, a bit of string & epoxy glue.  


 Extensions & the core allows me to hide this under a nice fiberglass 
 body  ;-)

But I will try to do this and specify the datatype. We will see.

:-)  and its probably really fast :-)

And, I directly build internal R3 block, this is directly used I 
think. So no copying, internal handling etc.

@ brian: I understand what you say about working around... but, with 
the host right now, I can't do anything else than hack up a solution 
in order for extensions to have callbacks.  99% of the real work 
is inside the extension.  when the host will be improved, the extension 
work won't change.  I can't wait for things to happen... this solution 
is just for my own use, and it gives me a good perspective on how 
to help with a real solution.  


I'm also brushing up on my C skills (I haven't done any in over a 
decade!) so doing this stuff is a good exercise anyways.


right now, if I can show to Carl how complicated it is to do some 
stuff, he will have an explicit example and have a better reference 
for practical solutions.  Doing anything... its all just theories. 
 look at the notes on delect and command... this is similar... lets 
start with something.  see where that leads us and then, we have 
a reference to pick on and critique.  


We can have better ideas for the api, just by having something bad 
to start with.... and believe me.. I know my hack sucks ... but I've 
got part of the solution complete, and compiling without warnings, 
and some code sharing between extensions api and the host.  :-)

Maxim, you do realize that the purpose of the current host release 
is to test and improve the host model, right? Not to build final 
projects? If you run into problems in the host model, try to fix 
them, not work around them. Otherwise your work is a waste since 
the host interfacing model is going to change in the next version, 
hopefully based on your and my feedback. And a callback solution 
that doesn't integrate with R3's multitasking model is worse than 
having none at all - since any code that might be written to use 
it would need rewriting, and probably rearchitecting, very soon.

On the other hand, if you are really trying to test the model to 
destruction as an example to base the next version's revisions on, 
then cool. I would like to see how your code integrates with devices, 
even if it has to be moved out of an extension and into the host 
for now, at least until we get device extensions. Code that works 
with the model won't need as much rearchitecting.

I've checked the host code and afaict, you can add your own device 
types. You don't have to stick with just file, network and clipboard.

jocko,  yes and no.   ;-)  


Glass is going to be rebol code only, but its going to be based on 
rebogl, the OpenGL extension I am currently working on (as I write 
this).  Rebogl its going to be an evolutionary process, starting 
with simple high-level pre-defined primitives and colors and then 
will get more and more customisable (deformers, animation, textures, 
programmable shaders, etc).


I am still not sure how the Glass engine will evolve, but there is 
a good chance that it will be based on the scene graph technology 
I am working on for the Scream game engine.  This has the benefit 
that Glass can be used to build the interfaces within the games themselves. 
 But it most definitely won't require you to load a complete  (and 
inherently complex) 3d world manager, just to build a window with 
a form.  if possible, I'd like to have window masks, so that the 
3D forms can actually live like 3d models direclty on the desktop... 
so some of the nice 3d feature isn't wrapped within an OS window 
border.

brian, yes we can add our own devices... in fact, it seems quite 
easy, and I will probably be adding a DB trigger device within a 
week or two.  :-)


the thing is that there aren't any exposed or documented *native* 
hooks from the host into the core... 


so far, I've got a callback library (called wire) working which executes 
rebol code in global context using the Reb_Do_String() r3lib.dll 
exported function  :-)


now I just need to use that library within the extension and see 
how it goes... the moment I have *something* which works... I'll 
stop improving the hack... from there on, I'll just work on the architecture 
of the caller and callee, to see how we could make it simple and 
easy to setup, from the extension and within the application using 
that extension... generically.


the code in between can change completely, it wouldn't actually change 
the extension or application code (that's the idea anyways)... just 
a few includes and headers which map how to link to the callback 
system.  I'll also try to build a device, just to see how that can 
be used instead of callbacks... but I still need to use a callback 
from the extension in order to access the host... so for now my hack 
is essential, whatever I do.


in this case, I'll be dispatching the GLUT events within the rebol 
using this architecture... I should have an interactive OpenGL window 
by tomorrow... crossing my fingers.


for now I am busy rebuilding my old OpenGL project within the new 
cleaned-up MSVS solution I've been working on for 2 days now... there 
are soooo many properties, its scary and long to setup... especially 
in this setup where there are several interdependent projects within 
the solution... but now, at least, when I change stuff at any layer 
and build, it builds all the stuff correctly in one step...

Guys - do you have any special comm channel with Carl? If not, then 
I feel some info might get lost here. E.g. Max expressing the need 
for Extensions being at least partially moved into Host. So my question 
is - does Carl know about your needs and opinions?

No I don't. there are zillions to choose from but I don't know which 
one is really good and leads to a result.

liquid is a dependency engine, its like a kernel but managing individual 
operations (functions/procedures) instead of whole applications (processes/tasks). 
 Scream uses liquid to build data and make sure it stays up to date 
with whatever data it is based on.. if you change sphere radius... 
the 3d model representing that sphere will rebuild itself... no need 
to know how the sphere model itself works.  


If Glass is based on some of the technology within scream, which 
uses liquid, then things like dependencies between input data, their 
 forms, and the result of that input become impossible to break. 
 there is, as such, no action function as we had in VID.  the interconnections 
from data and process is what defines an application.

Pekr, Carl already said in a recent blog that the ability to have 
extensions built into the host is already planned, and he is working 
on it.

I know have added callbacks to extensions using a little hack with 
an intermediary dll I built, loaded by the host and any extension 
that needs to run rebol code.   so its fun to know that in the end... 
we already have ways to tailor the executable to what we need even 
when it officially doesn't support what you need.  :-)


obviously we can't do everything, but this little test is already 
nice.  right now I execute code when the OpenGL window is resized....

I will be adding events for mouse clicks and keyboard presses, so 
I can start interacting with the 3D rendered stuff.

yes... R3 is a completely different ball game than R2    :-D

yay!  real event model in place and functional for the OpenGL extension... 
its not a permanent solution but it will do for now.f


now the tedious job of creating stubs for a few hundred functions 
begins!


and hopefully by next week the first applications to show this off 
will be demoable  :-)


currently, including callbacks which create an object at each refresh, 
I can't  resize the window faster than the engine can redraw it (up 
to 1440x900, in 32 bit color, with a few shaded polygons ) , and 
this includes hundreds of lines of rebol being printed in the DOS 
shell.

R3 SQlite update: I added SQL statement caching so that these are 
re-used if once seen, which speeds up things a lot. And driver supports 
multiple database connections as well now.

The trick is that the struct! and routine! types would only exist 
within the dialect - they would be handle! and command! outside of 
it.

And handles, for that matter.

so I can assign a handle to an integer?  and the extension with get 
a  64int *myhandle ?

A handle is a 64bit value that can be treated in any way that the 
relevant native code wants it to be treated. It is returned by native 
functions or commands and is taken by native functions or commands.

anyhow.  I wonder, can anyone tell me if its easy to load dll and 
map functions dynamically on LINUX?

Maxim: loading and using shared libs on linux is rather trivial. 
dlopen(3) and dlsym(3) are what you need

they are part of POSIX, and you'll have to link against the dl library

I think that some users and especially novices might still welcome 
a bit improved /library interface to the full fledged Extension possibilities 
...

doc, i guess maxim is referring to LoadLibrary and GetProcAddress 
in win32

/Library is tricky to implement, it requires constructing a C function 
call dynamically (using so-called trampoline functions) unrolling 
arguments in right format and order on the stack, then calling the 
C function, then retrieving result from stack.

how I see it, that is what /library does anyways.   and when you 
look at the host code includes, a lot of it is dynamically created 
vector lists.    so I bet the host  is loosely based on /library 
in fact.  :-)

still a complex system of registering functions, converting arguments 
and stuff, but for simple first tests, it can be done.

and we can expand as we go.

pekr, read Ladislav's DLL document a bit, and yess, it provides usefull 
information ...  gotcha's mainly which point out a few things, not 
to forget when defining the struct! dialect and other details.

I would also be very happy if there was /library in R3. I think many 
of use really don't want to write and compile c code (especially 
for multiple platforms) unless it's really really necesarry (thats 
why we use rebol).  Max I don't work in R3 for now, but if you make 
/library I will port the libharu pdf binding to R3.

and although you say library is limited is is *good enough* for many 
things.

once again - why not have best of both worlds? Carl stated, that 
Extensions might be internalised (packed into main distro). So - 
let's use default interfacing method - Extensions. And let's make 
/library extension. Should work no? If it would allow many ppl to 
wrap thing here or there, and even if it will not be so powerfull 
as full fledged extensions, why not to have it, right?

( as it was demonstrated by many comunity members with many bindings, 
and I could also make a pdf binding that is already used for serrious 
stuff, and works on linux and windows without me ever touching the 
hairy low level stuff)

... and if it will a bit more powerfull, than R2 version, even better 
(referring to Ladislav's proposed enhancements)

The thing is, if we can come up with corrent architecture. R3 plans 
on handle! type, as well as custom datatypes (not sure it is needed), 
but surely is R3 not planning to use routine! and struct! ones (IMO)

and another  100 usd if nested structs and clean pointer handling 
are included

It's not difficult to start; that's just a matter of doing the research 
on dynamic library loading and mocking up some code. And once extensions 
(not the host kit) are available on OS X, then the code can be tweaked 
and compiled.

You can start by going through R3 chat and helping Carl with his 
difficulties with porting the host kit and extensions to OS X.

And in rebdev message #6257 I explain how to get extensions working 
on Linux.

Good. So once we resume work on such things (we're working on bug 
fixes and tickets now) the information will be there. Thanks.

Because we have been working on other things, mostly R2 and the web 
site (and R3 mezzanines in my case).

I saw that one, but it doesn't seem to be the right approach. Two 
reasons:

- .so and .dynlib aren't dynamicly linked library formats on Windows, 
.dll isn't on Linux, etc.

- It kind of defeats the purpose of extensions: making all of that 
platform-specific stuff go away.


Don't get me wrong, the current behavior also has the second problem. 
Extensions are imported like modules, and so the code that is asking 
for them has to specify the filename to load them. However, *that* 
code is supposed to be cross-platform, so putting a platform-specific 
filename in their Needs list is inappropriate.


What do you think of using a generic filename extension like .rx 
for extensions, then having LOAD translate to the native filename? 
Or on platforms that don't require a specific filename extension 
for their dynamic library loader (like Windows), you could use the 
.rx extension directly.

And I'd also keep the platform specific extensions in system/options/file-types

Windows is the one that needs to use the .rx extension the most, 
and the one that we definitely can use it. And system/options/file-types 
has to have the extension we are trying to LOAD, not what it might 
be translated to.

Is there any harm in allowing .rx alongside with .dll .so and .dylib?

With both .rx and .dll having the same name? OK.

The harm comes from having .dll, .so or .dylib in the Needs lists 
of what would be otherwise portable REBOL modules and scripts.

It's better to not have .so in there at all. Part of the design strategy 
of the R3 language is to at least pretend that the language is portable, 
and keep all of the non-portable stuff wrapped in extensions or hidden 
in the host code.

And as allowing them to be used with their native extension does 
no harm as far as I can tell, I'd enable that usage alongside the 
.rx

We do. Simplifying the code would be to just support .rx and have 
the LOAD-EXTENSION native in the host code do the translation.

And keep .dll .so and .dylib as they are now

I want to keep .dll, .so and .dylib files as they are now. That is 
what we write .rx extensions to wrap.

Sounds reasonable and simple to me

So what? There's no reason why dynamic libraries should be called 
.dll on Windows and .so on Linux, none at all.

I.e. that i can ship linux, windows and osx extensions along with 
a simple script, and use the proper extension on the proper platform

and that's a hostkit compiled with my posix loading patch

And have you tested by adding .so to system/options/file-types at 
runtime and then using IMPORT on an extension?

So you'd get rid of the append in mezz-init.r completely, and add 
the extension filetype some place else?

just a quick reminder: on osx .dylib and .so are both ok

OK. They're just for convenience though, since every almost dylib 
that is loadable as an extension will have been written specificly 
for that purpose and probably can't be used otherwise, so sticking 
to the .rx filename makes sense.

then i could have a single script, say foo.r shipped with bar.dll 
bar.so and bar.dynlib, use import %bar.rx in foo.r and it will select 
the proper platform specific library

this will break down as soon as we have e.g. 32b and 64b builds for 
the same platform

Fortunately the module system is based around importing into the 
system, rather than importing into modules directly. This means that 
you can have all of your platform-specific requirements handled by 
one module or script and then have the rest just reference by module 
name, not file name.

There won't be 32 and 64 bit builds on the same platform, in theory; 
32bit builds get one platform number, 64bit builds another.

and why not submitting it to our eyes too ?

Further it's lost within the discussions at some point. I just wanted 
to provide the file so that it can be reviewed and included in the 
next release.

Some people put files on a website and post a link.

So I can choose between 5 different ways. And I don't know which 
way will be the best that the contribution is taken into account... 


Guys, you know that I'm a real Rebol supporter but as long as the 
most basic things (like submiting a patch, and being sure it's taken 
into account)  are not made clear and simple and people can see it's 
looked at we don't have to wonder about "low participance".

Robert - R3 Chat is official SVN for R3 and soon even for R2. I would 
learn to use the Chat. Hopefully once R3 Veiw is available, GUI client 
will emerge ..

Years, ago I said that information channel fragmentation in a small 
community is evil. We still have the same situation or add new channels.

But OK, I use R3-Chat and will post to Extension group.

R3 chat is just not being advertised enough, I think. And even if 
Carl forgets to check there, be sure to let him know, so he doesn't 
forget his own tools.

BrianH: looks good. Maybe we should add platform-linux? platform-win32? 
and platform-osx? predicates to get rid of those evil magic numbers 
in cases like the above.

Reported a boiled-down version of this wish as bug#1454 and submitted 
an initial implementation in chat#6785.

so did anyone start on the R3 /library  extension?  if not I can 
work on that for windows, and make it so its very easy for someone 
else to map it to linux ( as a few stubs to re-implement  ).