AltME groups: search

Each app choses its own IPC when the OS doesn't have one standard. 
 No single IPC method will be usefull.   


for REBOL itself, I do think that we should provide a construct which 
will be similar or exactly the same as what will be build for thread 
messaging.

+ lots of points for having a code base that's already done and tested!

Carl, by STDIO, I just mean the basics for writing pipe-and-filter 
apps. e.g. 

  while [data: input] [print data]


I just found http://www.rebol.net/cgi-bin/r3blog.r?view=0282so maybe 
we're close and just need to write up some examples and a doc page 
explaining limitations on different OSs. Gabriele has some things 
in his power-mezz package as well (chain, filter, pipe), which are 
worth keeping in mind. The blurring of lines between in-process and 
inter-process, and piping is where we need design direction from 
above. 


A REBOL way (passing around blocks of dialected data, i.e., messages) 
vital, but we also need gateways to other mechanisms. The REBOL way 
is critical because it reduces or eimilates external dependencies 
and provides a model for gatewys to emulate.

License note:


This software is licensed under the Spread Open Source License. This 
license is SIMILAR BUT NOT IDENTICAL to the BSD license. Specifically, 
the license includes the requirement that all advertising materials 
(including web pages) mentioning software that uses Spread display 
a specific acknowledgement.

That's not insignificant, as the advertising clause of the old BSD 
made it legally incompatible with a lot of other licenses. Particularly 
(L)GPL, but we as a community need to be avoiding (L)GPL code anyways 
for a lot of reasons. We'll have to be careful.

If it's rolled into a host, then you need to care. Having it built 
in would be very cool so, yeah, you're right that we need to care.

One: yes, it would still be a problem even as an external extension. 
Two: we would have to do it as a host-embedded extension if we want 
to use this for the core intertask messaging solution, and that is 
the same as linking the code in.

Yes, if it's built in, we're in sync. But if it's an external extension, 
doesn't it become a problem for us as individuals rather than RT?

Nope, because it becomes a problem for any person who uses that extension 
in their product.

But asking each application built with R3 to also include a notice 
like this seems a big pain

I wonder if including this "This product uses software developed 
by Spread Concepts LLC for use in the Spread toolkit. For more information 
about Spread see http://www.spread.org"as a text string in the source 
counts?

I don't think it's a biggie .. more of an issue is how good is it, 
how complex, has anyone used it etc

Brian, I do commercial app development in REBOL as well as in-house.


Petr, I'm not here to defend Spread. I mentioned it because when 
I looked at it before, it was something I marked to remember because 
it wasn't too large or complex and didn't try to do too much (compared 
to, say, AMQP). 

I only played with it breifly, I didn't put it into production. 

I want a REBOL solution too. :-)

Mathematics aside, it should be obvious by now that REBOL is not 
in any way attractive in mainstream IT. I believe it's insularity 
doesn't help though that is only one reason. I made the comment tongue-in-cheek 
as I see no relevance to the worthiness or otherwise of a library 
just because somebody who has worked for 15 years in "enterprise 
sphere" never having heard of it.

Whether Spread would be right for REBOL is another matter though 
it would offer a limited  alternative for secure inter-process communications 
with non-REBOL processes without having to resort to either cgi or 
"roll-your-own".

Maybe we should make a list of candidate libraries and decide what 
is the best fit to our as yet unstated needs?

Well - I don't mind. It is not about if I ever heard about it. It 
is all about - how much interoperability with outer world do we get? 
E.g. - can you talk to most Windows or Linux (Unix) apps with it, 
like Amiga Arexx? Can you talk to SAP, Orcle APPs, WebSphere, SharePoint, 
etc. enterprise apps? In fact and once again - -I expect having extensions 
to as many systems as possible, but we surely also know, how picky 
is Carl about what is going to be included into standard distro, 
and 50KB is considered being a big addition :-)

Spread requires a daemon to be running ... so this doesn't fit

Petr, Peter doesn't talk nonsense IMO. There are a lot of things 
REBOL could use for interop. SOAP would be a protocol, not an extension. 
REBOL IPC may be a protocol too. MQ et al would be for specific systems, 
not general IPC. I'm all for a Uniserve model.


Historically, memory mapped files were one of the most efficient 
IPC mechanisms on Windows, but if a more portable approach can be 
written based on named pipes, I can live with that. Sockets are great 
as long as you don't run into firewall issues (even locally), and 
might encourage us to think in terms of protocols and messages.


As Pavel said, and one of the main design points, is the need for 
a daemon. Obviously memory mapped files aren't going to work across 
the net.

Pavel, I pulled out my old QNX manual and it lists Messages, Ports, 
and Exeptions as their three IPC mechanisms. Messages can be sent 
on "virtual circuits" which have a node ID in addition to the task 
ID to send to; no implementation details given. Ports were like named 
pipes it seems, with 16 being numbered and reserved by the OS.

Do you really need a daemon if you're just doing IPC on the one PC?

Graham, you don't for a single machine, but you would need a memory 
mapped file, named pipe, or something else. The question is whether 
you want an IPC system that is limited to the local machine.

Graham, one advantage of using a daemon approach to inter-process 
communication is that it is very easy to upgrade an application from 
single-user, single-machine to multi-user, multi-machine.

If we want to pursue IPC chat, make suggestions, and get Carl involved 
(at least get some his thoughts), we should do it in a different 
group or somewhere else. Should we do it on AltME or somewhere else?

a good thing is to make a wiki open discussion about it.  collect 
ideas, wants, needs, gotchas, known tools... whatever.  when Carl 
addresses this, he will have a document to review and analyse.

Agreed, with the goal of gathering information, though I'm also a 
bit hesitant as he may have strong feelings and ideas already. 

I'm good with a wiki though. Where?

wow, looks like the framework for a thesis  ;-)

I have a lot of notes. :-) I tried to keep it minimal, focusing on 
a few key questions to start, with an eye on the bigger things that 
would be built on the foundation.

I added a threading section to the IPC document.

The most important questions are probably missing: _what_ kind of 
"processes" are we talking about. Is it about communication between 
R3 tasks (within a single process), about multiple R3 processes on 
the same machine, R3 processes across a LAN? WAN? Heterogenous processes?

I also added a little section on implementation specifics (directly 
in the host, as extension, embedded?, etc)

one advantage to using TCP/ip is that users/admins have some control 
over them using standard firewall HW & software.  also, if one needs 
to he can use port 80 so messages can to through normal HTTP messaging 
routes.  This improves flexibility of the IPC, without it being a 
requirement.

Has anyone tried to build a .NET DLL that can be used from Rebol? 
Is this possible? IMO it should be possible.

until host-kit-a100, the extensions that I produced worked properly. 
With a102, the tests fail at a certain point (r3 found a problem 
...), even with the sample extension example produced by Carl. Any 
other experience on compatibility problems ?

Having had a look to the extension pages of the R3 documentation, 
I see that the use of make-ext.r to generate the init_block is no 
more relevant.

Andreas: thanks? With A100+, it seems that we need some other headers: 
ext-types.h, reb-defs.h (also present in src/include ). However, 
although they compile, it seems that they cannot be opened under 
r3 ( access error: cannot open %xxx.dll ... reason: none). I suspect 
the absence of a given #define flag, but not easy to debug. Anyway, 
I think that it would be useful  to clarify. If anybody already succeded, 
please give us the set of files and the config used. If not, could 
somebody do the test ?

thhanks, I will try. But my problem is not here. I have two concerns 
:  for the extensions dealing with strings (not the simple example 
where one reverse the order of chars), the compatibility is no more 
achieved (because of changes in the representation or processing 
of strings ?) Then I have to compile using the new headers. And when 
I do so, the extension is no more loadable (even a simple one like 
your example)

I added a second file, sample2.c, to demonstrate working with strings 
in A102: https://gist.github.com/bc820cc3eb301c79c1ef#file_sample2.c

Also compiles and loads fine for me.

thanks, ill have a look

Well, I had the same problem that some months ago :  

 in my application, I must compile in c++, and there is a difference 
 in labelling the function calls in c and c++.  
The simplest solution is to modify a line in reb-ext-lib.h:  
#define RXIEXT _declspec(dllexport) to
#define extern "C" RXIEXT _declspec(dllexport) 

One shoud definitly insert in this header a condition like  
#ifdef _cplusplus   
 #define extern "C" RXIEXT _declspec(dllexport)

Yes, that's a good note. Should be used. I forward it to Carl.

http://www.rebol.com/r3/docs/concepts/extensions-making.html#section-17
says it's out of date and I'm really having trouble including words 
as symbols in a result block of an extension command. 

It works fine with an *INIT_BLOCK defined as


 const char *init_block = "REBOL [\nTitle: {Power Management}\nName: 
 power\nType: extension\nExports: [power-status]\n]\n"

     "lib-boot:           does [map-words words] ;-- Is that a good idea?\n"

     "words:             [ac-line battery remains of high low critical 
     charging]\n"
	    "map-words:    command [words [block!]]\n"
	    "power-status: command []\n"
;

if after IMPORT %power.dll I call SYSTEM/MODULES/POWER/LIB-BOOT.


Is there a way to have IMPORT automatically execute the LIB-BOOT 
code with a simple extension not included into the host code with 
host-kit?

I'm trying to return a block containg a word not taken from a commands 
arguments. Say, I want to return a block like [BATTERY CHARGING]: 

That works with

	u32 *word_ids = 0; // used to hold word identifiers
	enum power_words {W_POWER_BATTERY = 1, W_POWER_CHARGING};

	RXIEXT int RX_Call(int cmd, RXIFRM *frm, void *data) {
		/* .... */

		v.int32a = word_ids[W_POWER_BATTERY];
		RXI_SET_VALUE(s, pos++, v, RXT_WORD); 
		v.int32a = word_ids[W_POWER_CHARGING];
		RXI_SET_VALUE(s, pos++, v, RXT_WORD);
	
		/* .... */
   	}


only after importing the DLL I "manually" init the words with a MAP-WORDS 
command. I was thinking that IMPORT eventually triggers LIB-BOOT 
init-code from the INIT_BLOCK. It seems like the init code is not 
executed.

Ok, you need to make a call to your lob-boot function out of this 
init function.

Ouch! Now that problem above has a really simple solution: Just putting 
the module init code in the CONST CHAR *INIT_BLOCK works great:

	const char *init_block =
	    "REBOL [\n"
	        "Title:   {Power Management}\n"
	        "Name:    power\n"
	        "Type:    extension\n"
	        "Exports: [power-status capture-screen]\n"
	    "]\n"

     "words:          [ac-line battery remains of high low critical charging]\n"
	    "map-words:      command [words [block!]]\n"

     "power-status:   command [{Returns ac-line and battery status information.}]\n"

     "capture-screen: command [{Returns screenshot as an image.}]\n"
	    "map-words words\n";


The "magic" is the last line, where MAP_WORDS WORDS is evaluated 
ofter loading the module, that command inits the dll with the word 
ids used by the extension.

I have built a simple R3 extension for ODBC database access. Although 
more work needs to be done in the unicode area and configurable rowset 
max-rows retrieval as well as catching some GC-related bugs, basic 
functionality like selects, inserts, updates and statement parameters 
is there and working for most major types like LOGIC!, INTEGER!, 
DECIMAL!, TIME! and STRING! I have to test with more databases / 
odbc database drivers. 


I have, however, major problems in working with date values. I just 
don't manage to retrieve date values passed to a command or to return 
a proper date value. So, has anybody succeeded in working with date 
values and probably knows how to create, access and calculate them? 
Sadly, I've found no example code related to date values and there 
isn't much documentation too. Any info is greatly appreciated!

Ah, good question, now I think Christian meant the binary format 
of rebol's date! values, not any database's date format, which should 
be available as a string at least.

I've looked in DevBase, but haven't asked questions there yet. I've 
scanned all the extensions documentation on DocBase and the header 
files in the host-lib/extension packages. 


I didn't even manage to read a date value in a simple command like 
   TEST-DATE: COMMAND [DATE [DATE!]]      About every member n the 
REBOL value c-union type seemed to only contain zero values.

But I suppose if you need it badly enough and can't find how to move 
dates then you parse and transform between the string formats.

On the rebol side, should just need a MOLD and a LOAD / TO-DATE...

Hmm, I think I've to check this again. At least, a simple 

    RXIEXT int RX_Call(int cmd, RXIFRM *frm, void *data) {
        return RXR_VALUE
    }
    
returns a proper date value for

    test-date: command [date [date!]]
    
I'll see if I can find some time to work on this evening.

ChristianE, can we see your extension .. even if it is a WIP?

There are things that have to be cleaned up a bit first, Graham, 
but I hope I can publish it very soon (counting days, not weeks).

That's why I'm trying to DELAY. Delaying doesn't seem to work for 
embedded extensions according to curecode, but here it's a simple 
dll import ...

Yes, I saw the ticket ... maybe it gets fixed in a110, but not sure 
it is a cause. BrianH discussed it a bit today ...

We have been discussing adding a header option to tell the system 
to put off loading an embedded extension until after the mezzanines 
have loaded (or have that be the default, and have an option to load 
before the mezzanines). I haven't looked over the startup code yet 
to see how to do this, but it shouldn't be too hard. I'm waiting 
for Carl to decide on the option (he seems to be the namer of that 
kind of thing right now, I'm just an implementor).

Here's a minimal example of an extension:
https://gist.github.com/0677266b8511d83ce76a

i think you could also replace RXI_GET_CHAR by first obtaining a 
pointer using RL_SERIES(ser, RXI_SER_DATA) and then just indexing 
into this directly

I would appreciate a very simple example how to make extension using 
external DLL. So I could write for example ImageMagick extension.

funny, I intended also to try interfacing with ImageMagick (maybe 
the ImageMagick Com + component.), as I have a need for another project.

Concerning the extensions dll's and source files, I think that they 
are no more compatible. I should check and put the latest versions.

if you download the OpenGL demo, there is a complete working setup. 
 it has a makefile and a visual studio 2008 setup .  it should help 
you out.

I'm completely C/C++ newbie, so I would like to know, how to make 
the extension, if I have precompiled DLL.
For example, In R2 I can simply download the DLL like this one:
http://zlib.net/zlib125-dll.zip

And simply do:

zlib.dll: load/library %zlib1.dll

zlib-version: make routine! [ return: [string!] ] zlib.dll "zlibVersion"
zlib-version
;== "1.2.5"


Is there someone who can write a simple tutorial - C extension with 
the zlib-version command?

Oldes, I've built you a simple "zlib-version" extension.

Afterwards you should have a fresh "zlib.dll" which is the extension.

is there a reason why there is:
#include <stdio.h>
#include <string.h> 
?

stdio.h is a leftover and not needed.

And isn't the conversion to a REBOL string quite complicated?

Yes, a bit. There may be a simpler way. Or we could bundle a few 
such helper functions into a support library (as Max did in his OpenGL 
work).

I know. This is a good start for me:) btw. is it possible to create 
extzlib.dll which would contain the zlib1.dll? I don't need it, just 
asking.

developing an extension to wrap MagickWand is a huge work, because 
of the large number of functions, and data formats. It's better, 
in this case, to develop a full dedicated application in c, or, at 
least a dll with higher level functions ... wihch is approximately 
the case of ImageMagickObject. In other words, I am not convinced 
of the interest to program in Rebol if you want to use the full set 
of low level image processing functions.

yes, its VERY usefull... one doesn't want to have to compile applications 
and stuff... AFAIK there are binding to other languages which use 
the full IM suite.  

but yes its a lot of work.

That's also a question for me ...

Part of the answer is that using it in a cpp exe will allow to execute 
these functions without te burden of executing a c process while 
masking the dos window (which is a huge challenge here, and I do'nt 
know why). In the case of Rebol, yes, the interest is not so evident.

Maxim, yes, I have seen bindings to other languages, but I am still 
not convinced that its much simpler than coding directly in C or 
C++

Oldes, I did not know im-min.r. It's a nice work. I have not enough 
experience in Extensions to do such work, as the input-output methods 
of R3 extensions are a little tricky.

REBSER is a type of RXIARG, you build the RXIARG like so:

this is what carl will be adding to the next host-kit.... 


//------------------
//-    #RXV_xxx
//
// REBOL EXTENSION GET Macros
//
// provide direct RXIARG access macros
// with these macros, the single argument should be an RXIARG *
//

// this is usefull when the RXIARG is NOT being used from an argument 
frame

// but as a single value, like when we use RL_Get_Field() or RL_Get_Value()
//
// if the argument is dynamically allocated, ex:
//    RXIARG arg = OS_MAKE(sizeof(RXIARG)); 
// then use the macros like so:
//     RXV_WORD(*(arg));
//------------------
#define RXV_INT64(a)		(a.int64)
#define RXV_INT32(a)		(i32)(a.int64)
#define RXV_INTEGER(a)	(a.int64) // maps to RXT_INTEGER
#define RXV_DEC64(a)		(a.dec64)
#define RXV_DECIMAL(a)	(a.dec64) // maps to RXT_DECIMAL
#define RXV_LOGIC(a)		(a.int32a)
#define RXV_CHAR(a)		(a.int32a)
#define RXV_TIME(a)		(a.int64)
#define RXV_DATE(a)		(a.int32a)
#define RXV_WORD(a)		(a.int32a)
#define RXV_PAIR(a)		(a.pair)
#define RXV_TUPLE(a)		(a.bytes)
#define RXV_SERIES(a)		(a.series)
#define RXV_BLOCK(a)		(a.series)
#define RXV_INDEX(a)		(a.index)
#define RXV_OBJECT(a)	(a.addr)
#define RXV_MODULE(a)	(a.addr)
#define RXV_HANDLE(a)	(a.addr)
#define RXV_IMAGE(a)		(a.image)
#define RXV_GOB(a)		(a.addr)

Oldes, without changes given by andreas, you will corrupt the executable's 
memory and will *eventually* have a memory acces failure.

post your current version so we might find a little typo...

That was a definitive bug.

add a little newline there...

RL_Print("%d, %s\n", i, list[i]);

Ok, so simple question, how to return a block with one string inside?

obviously, you need to have set the first value of the block to a 
string.

This contains a minimal extension that returns a block containing 
a string.

Yes, give me a minute. I'll check a full A110 on Windows.

Interesting. Seems to make a difference whether I run a script which 
imports the extension or import the extension from the command line.

same thing, command-line would last longer before a crash, but ultimately, 
it would still fail... only further down the execution.

does allocating a struct/union on the stack create an implicit pointer 
(which points to the stack rather than heap) when used?

A simple `a.index = 0` is missing.

we could propose a usefull macro to not get bitten by  this  again:

something like:

#define NEW_RXIARG(v) RXIARG v; v.index=0

CLEARS(&a);

RXIARG a;
CLEARS(&a);

is the most sensible way. but then you'll better #include <string.h> 
(at least until some include woes in the hostkit headers are straightened 
out).

which is why a standard macro which takes care of that would be nice... 
in the same idea than the RL_MAKE_BLOCK (which I know isn't exactly 
the same since its just a wrapper around the RL vector)

Ah.. thanks.. so I made another small step in learning extensions:) 
Thanks. I'm sure I will have more questions later. But I must work 
now for a while.

When I have command like:
	test: command [str [string!]]
And on C side I need a pointer to this string, how to get it?

is your file a .c or .cpp  ?

ah, I did a little bit of searching and I think I understand... 

try using RL_PRINT() instead.

its strange that I've never had a problem with this though cause 
I use RL_Print in all my files, but I realize that it might just 
be luck, that in my setup I don't have the linking problem you are 
having.

and replace:
#define RL_PRINT(a,b)               RL->print(a,b)

with:
#define RL_PRINT(a,b)               RL->print(a,__VA_ARGS__)

can't test it right now, but that should work.

__VA_ARGS__ can only appear in the expansion of a C99 variadic macro
Never mind... I really have to do real work now :)

Actually it's:

../src/include/reb-lib.h:270:49: warning: __VA_ARGS__ can only appear 
in the expansion of a C99 variadic macro

wand.c:74:43: macro "RL_PRINT" passed 3 arguments, but takes just 
2
wand.c: In function `RX_Call':

wand.c:74: error: `RL_PRINT' undeclared (first use in this function)

But I have a solution, this works:
RL->print("%d %s\n", n,item);

Extensions are quite additive as I'm getting closer to a real work 
:) I'm sure I will have other questions, but must force myself to 
stop now.