World: r3wp

RXA_Handle is used to allocate pointers and provide them to the core, 
without source code being able to tamper with them.  in fact, I don't 
even thing a handle!  can be created from R3 source...  only from 
the core or extensions.   so in general, if you are given a handle! 
type as an argument, you can rely its really some pointer generated 
by some native/extension code.

when you say REBOL source you mean script code?

yes.

so, basically you can do what I was asking for (calling scripts functions), 
but you have the limitation of having to know in advance the signature 
of the script function you're going to call, no?

not really,   Reb_Do_String  just executes a string of REBOL text 
on the global context.

it loads it, binds and runs the do dialect on it.

I was able to trigger button & keyboard events from  a GLut  (OpenGL) 
window,  running arbitrary code like normal view actions.


Nice thing is that the errors do not stop the interpreter, just that 
call to Reb_Do_String() so its like if your code was wrapped within 
an attempt by default  :-)

guess it's fairly flexible, but you're incurring quite a cost for 
this - need to stringify the script call and then this needs to be 
parsed again on the host side - seems wasteful

yep.   but it works.

I was still able to get more than 100fps running with one callback 
creating a small object per refresh IIRC.

sure - it'll have to do for now, in any case. You didn't answer about 
any enhancement to this that Carl might be working on, though - do 
you know if he is?

with view extraction from the coe, he has NO CHOICE in giving us 
a means to execute loaded, bound, code, since all a GUI really does 
is trigger code.
code which is managed by the core.


I also know that vector and image datatypes will definitely be part 
of next extension, its really important.


I am in the process of requesting that the argument frame be increased 
to 15 arguments by default... 7 is ridiculously low, especially if 
you want to use refinements.

so you think he's pretty much there?

wrt the first part

note that with the above callback system,  receiving a result from 
the callback is tricky.  


it has to use one of the extension's commands to store a return value 
there,  and the extension should then look for a return value after 
the callback returns.

there ... as in Done?  not sure...  its a lot of architecture and 
design decisions, probably more thinking than coding is happening 
right now.

its a big deal, it has to be secure i.e.  an extension must not be 
able to hijack the core and get an way in so it would start  probing 
core data, and ultimately go around all the work done to improve 
R3's safe use.

As I mentioned in chat, I'm thinking of implementing an external 
REBOL lexer for the Scintilla based editors. I was thinking that 
one way it could work is if the external lexer was a REBOL extension, 
but thinking about it again, maybe the whole callback thing could 
be avoided by having a custom host that is built as a dll, with the 
additional functions added to satisfy the Scintilla external lexer 
interface. Max, BrianH, do you know if it would be possible to compile 
the host as a dynamic library?

sure, but you'll probably have to tweak the distro, replace the main 
by the appropriate DLL init funcs.

not sure how easy it will be in linux, but on windows, exe and dlls 
actually are the same thing.  the os, just doesn't run main() on 
a dll.

yeah, you're right

do you know if Carl is still giving access to the hostkit if he's 
asked nicely in chat?

if you look at the startup code, its dead simple, so you probably 
just need to remove that and you're done... then use do_string, just 
like the read line loop does.

sure, especially to people with definite goals and cool ideas.  I 
thought it was made public, maybe not.

I wonder what performance you could expect if the host was called 
for evey keystroke in the editor and was passed the complete script 
(with possible included modules) to be parsed  - do you have any 
guess? Of course the host instance would be cahced between invocations.

if all you do is:


rebol_source = "PARSE {.... UTF-8 data from scintilla ...} parse-rules 
";
do_string (rebol_source);


probably very fast, enough for real time,  if script isn't huge :-)

if your app also has an extension which can give data to scintilla, 
then you'd just  add a command to the above string which calls the 
extension's command! with parse results.


the xtensions are very adept at parsing blocks, so converting that 
back to scintilla as native C data is possible there.

the only issue I see is intense RAM juggling, so if you can find 
ways to parse only subsets of current edit and cache the rest, then 
it might even be effective and real-time.

Why the need for the extension? If you already have a customized 
host, why would this be needed?

ah, yes, its possible the host also supports commands... but I'm 
not sure in the official release.  in the next one I think these 
have been built-in.

if you're having to do partial parsing, I'm guessing it can get pretty 
complicated quickly as you'd have to keep track of contexts that 
might be split, no?

thing is, things like intellisense in rebol aren't possible, because 
only the interpreter understands the complex binding that happens.

you can guess them, but not confidently, because you'd need access 
to actual context-like object from the C side to resolve words which 
are bound externally (not part of current context, but inherited)

but if you have control over the source building itself within scintilla, 
then it can know what things are and where they come from... that's 
the main point of using an IDE.

its not an easy thing to do in any case  ;-)

the module system provides enough information so that your editor 
could resolve the dependency on its own, and parse stuff accordingly.

now USING rebol as the scripting/macro language within an Editor 
 

THAT would be really cool.   :-D

not sure I follow...

yeah, that was my intent, if at all possible

but I think that the integration between the editor base and the 
lexer is too weak

if all you want to do is give rebol a string, and use the return 
string, and plug it back into scintilla, then its going to be a piece 
of cake.

if, on the other hand, you want to use REBOL to parse the source 
for things like syntax highlighting, then its complicated, which 
is what my notes above relate to.

I need to identify what benefit I'm hoping to get over the current 
lexer which uses a static word list and regexes for capturing various 
bits

using R3 to actually LOAD and identify tokens by their datatype seems 
to be the biggest deal to me.

that was my intent

and then to query it for the up-to-date words

you would let R3 parse the source, then call various internal functions 
to return the various types of words

trick is to find a way to parse just a subset... you might want to 
chat with steeve and shadwolf, their rebol editor was pretty responsive 
even with big files, so I guess they have an optimisation that lets 
them just token what is visible, with some form of caching for before 
and after pages of data.

you could then just apply the algorithm to the C side and let R3 
parse only a small portion of the script at a time.   That's how 
I'd try it.

using  a command! to tell scintilla what to refresh directly, based 
on parse results.