AltME groups: search

Carl, by STDIO, I just mean the basics for writing pipe-and-filter 
apps. e.g. 

  while [data: input] [print data]


I just found http://www.rebol.net/cgi-bin/r3blog.r?view=0282so maybe 
we're close and just need to write up some examples and a doc page 
explaining limitations on different OSs. Gabriele has some things 
in his power-mezz package as well (chain, filter, pipe), which are 
worth keeping in mind. The blurring of lines between in-process and 
inter-process, and piping is where we need design direction from 
above. 


A REBOL way (passing around blocks of dialected data, i.e., messages) 
vital, but we also need gateways to other mechanisms. The REBOL way 
is critical because it reduces or eimilates external dependencies 
and provides a model for gatewys to emulate.

It looks like the Win32 binary is from 2006 and the main daemon is 
~200K.

One: yes, it would still be a problem even as an external extension. 
Two: we would have to do it as a host-embedded extension if we want 
to use this for the core intertask messaging solution, and that is 
the same as linking the code in.

Yes, that's what I mean. We who distribute it, and passed along to 
those who use our work.

I'm willing to pay that price, but I can only speak for me. :-)

Gregg - to be honest - 200KB? Total bloat. I work in enterprise sphere 
for 15 years, and never heard of something like "spread". In fact 
- noone in enterprise sphere cares. Guys, really - let's have clean 
 and mean REBOL solution, the REBOL way. Then we can interop with 
other systems, as the need arises. Let's not adhere to pseudo standards, 
because they have some juicy website ...

Anyway, I guess most apps have some type of licensing that is viewable 
from the app, and you can just include the line in there.

Brian, I do commercial app development in REBOL as well as in-house.


Petr, I'm not here to defend Spread. I mentioned it because when 
I looked at it before, it was something I marked to remember because 
it wasn't too large or complex and didn't try to do too much (compared 
to, say, AMQP). 

I only played with it breifly, I didn't put it into production. 

I want a REBOL solution too. :-)

I have worked in enterprise sphere for 35 years and never heard of 
something like "REBOL" from anybody that I've every worked with.


Let's make sure that we don't change this and keep REBOL clean and 
mean and to itself.

Yes. For many, merits include robustness, reliability, documentation, 
support and availability of skilled people.

..and of course, its ability to integrate with what they already 
have.

Maybe we should make a list of candidate libraries and decide what 
is the best fit to our as yet unstated needs?

Well - I don't mind. It is not about if I ever heard about it. It 
is all about - how much interoperability with outer world do we get? 
E.g. - can you talk to most Windows or Linux (Unix) apps with it, 
like Amiga Arexx? Can you talk to SAP, Orcle APPs, WebSphere, SharePoint, 
etc. enterprise apps? In fact and once again - -I expect having extensions 
to as many systems as possible, but we surely also know, how picky 
is Carl about what is going to be included into standard distro, 
and 50KB is considered being a big addition :-)

what I would like to see for IPC mechanism is some port/events mechanism. 
And then, on top of that, the way to go is the Uniserve like engine. 
Python has Medusa - simply put - multiprotocol engine, to communicate 
with external world ...

Graham IMO almost every IPC need somewhat daemon runing. The library 
"could" be linked in extension and try to open communication with 
daemon, when it doesnt find the daemon let start its own (first process 
trying to comunicate), what is nice in Spread it combines P2P and 
multicast and members/group policy, and of course single/multi machines.

Petr, Peter doesn't talk nonsense IMO. There are a lot of things 
REBOL could use for interop. SOAP would be a protocol, not an extension. 
REBOL IPC may be a protocol too. MQ et al would be for specific systems, 
not general IPC. I'm all for a Uniserve model.


Historically, memory mapped files were one of the most efficient 
IPC mechanisms on Windows, but if a more portable approach can be 
written based on named pipes, I can live with that. Sockets are great 
as long as you don't run into firewall issues (even locally), and 
might encourage us to think in terms of protocols and messages.


As Pavel said, and one of the main design points, is the need for 
a daemon. Obviously memory mapped files aren't going to work across 
the net.

Pavel, I pulled out my old QNX manual and it lists Messages, Ports, 
and Exeptions as their three IPC mechanisms. Messages can be sent 
on "virtual circuits" which have a node ID in addition to the task 
ID to send to; no implementation details given. Ports were like named 
pipes it seems, with 16 being numbered and reserved by the OS.

So, there is the possibility then of two ipc mechanisms .. one for 
local ipc, and for one distributed ipc ...

If we want to pursue IPC chat, make suggestions, and get Carl involved 
(at least get some his thoughts), we should do it in a different 
group or somewhere else. Should we do it on AltME or somewhere else?

a good thing is to make a wiki open discussion about it.  collect 
ideas, wants, needs, gotchas, known tools... whatever.  when Carl 
addresses this, he will have a document to review and analyse.

Agreed, with the goal of gathering information, though I'm also a 
bit hesitant as he may have strong feelings and ideas already. 

I'm good with a wiki though. Where?

just notes and questions.  IIRC we talked about this at some point.

The daemon question is the primer for both threading and processes. 
The "what are we talking to?" question. I'll add that to the wiki.

all the IPC I've done with rebol where TCP socket based, some per 
connection, others with persistent and fault-detection wrappers similar 
to ip packets.

One enterprise service I did used EDI formatted information and allowed 
queued message sending and receiving, with order detection and re-sending 
on time-outs or connection errors.


funny thing is that I had to throttle my REBOL app because it parsed/replied 
messages faster than the remote service could handle :-)

I just built r3.exe from A102 and was able to load in the extensions 
I produced from A101 ...

jocko, I had assumed you had recompiled them....


you must recompile them for A101 and up, many enums and offsets have 
changed, so they don't correspond anymore.

The external sample extension works partly (apart from several string 
and word functions, like t-word-map)I understand that I have to recompile, 
but  it's not so easy to find the proper headers. Furthermore, I 
am not sure that the old make-ext.r script which generates the init_block 
is still usable. It would be useful to actualize the external sample 
extension. By the way, where is the page giving the main changes 
from a101 (changes from char * to REBSER*, t-word-map etc) ? I am 
not able to find it out.

The necessary headers are in src/include/: reb-c.h, reb-ext.h, and 
reb-ext-lib.h

And I fear that some parts of the extension docs are currently outdated.

My personal summary for the extensions API:

- A100 is basically the same as the previous extensions-only API 
release (A77).

- A101 adds map_word, word_of_string, words_of_object, get_field, 
set_field

- A102 adds make_image and gc_protect, renames the constants used 
for series_info from RXI_INFO_* to RXI_SER_* and adds RXI_SER_DATA

Andreas: thanks? With A100+, it seems that we need some other headers: 
ext-types.h, reb-defs.h (also present in src/include ). However, 
although they compile, it seems that they cannot be opened under 
r3 ( access error: cannot open %xxx.dll ... reason: none). I suspect 
the absence of a given #define flag, but not easy to debug. Anyway, 
I think that it would be useful  to clarify. If anybody already succeded, 
please give us the set of files and the config used. If not, could 
somebody do the test ?

thhanks, I will try. But my problem is not here. I have two concerns 
:  for the extensions dealing with strings (not the simple example 
where one reverse the order of chars), the compatibility is no more 
achieved (because of changes in the representation or processing 
of strings ?) Then I have to compile using the new headers. And when 
I do so, the extension is no more loadable (even a simple one like 
your example)

And as demonstrated: it compiles _and_ loads just fine.

I added a second file, sample2.c, to demonstrate working with strings 
in A102: https://gist.github.com/bc820cc3eb301c79c1ef#file_sample2.c

Also compiles and loads fine for me.

Well, I had the same problem that some months ago :  

 in my application, I must compile in c++, and there is a difference 
 in labelling the function calls in c and c++.  
The simplest solution is to modify a line in reb-ext-lib.h:  
#define RXIEXT _declspec(dllexport) to
#define extern "C" RXIEXT _declspec(dllexport) 

One shoud definitly insert in this header a condition like  
#ifdef _cplusplus   
 #define extern "C" RXIEXT _declspec(dllexport)

http://www.rebol.com/r3/docs/concepts/extensions-making.html#section-17
says it's out of date and I'm really having trouble including words 
as symbols in a result block of an extension command. 

It works fine with an *INIT_BLOCK defined as


 const char *init_block = "REBOL [\nTitle: {Power Management}\nName: 
 power\nType: extension\nExports: [power-status]\n]\n"

     "lib-boot:           does [map-words words] ;-- Is that a good idea?\n"

     "words:             [ac-line battery remains of high low critical 
     charging]\n"
	    "map-words:    command [words [block!]]\n"
	    "power-status: command []\n"
;

if after IMPORT %power.dll I call SYSTEM/MODULES/POWER/LIB-BOOT.


Is there a way to have IMPORT automatically execute the LIB-BOOT 
code with a simple extension not included into the host code with 
host-kit?

Straight from the samples, since I have about zero knowledge of C 
and only am exploring how extensions work.

Ouch! Now that problem above has a really simple solution: Just putting 
the module init code in the CONST CHAR *INIT_BLOCK works great:

	const char *init_block =
	    "REBOL [\n"
	        "Title:   {Power Management}\n"
	        "Name:    power\n"
	        "Type:    extension\n"
	        "Exports: [power-status capture-screen]\n"
	    "]\n"

     "words:          [ac-line battery remains of high low critical charging]\n"
	    "map-words:      command [words [block!]]\n"

     "power-status:   command [{Returns ac-line and battery status information.}]\n"

     "capture-screen: command [{Returns screenshot as an image.}]\n"
	    "map-words words\n";


The "magic" is the last line, where MAP_WORDS WORDS is evaluated 
ofter loading the module, that command inits the dll with the word 
ids used by the extension.

I have built a simple R3 extension for ODBC database access. Although 
more work needs to be done in the unicode area and configurable rowset 
max-rows retrieval as well as catching some GC-related bugs, basic 
functionality like selects, inserts, updates and statement parameters 
is there and working for most major types like LOGIC!, INTEGER!, 
DECIMAL!, TIME! and STRING! I have to test with more databases / 
odbc database drivers. 


I have, however, major problems in working with date values. I just 
don't manage to retrieve date values passed to a command or to return 
a proper date value. So, has anybody succeeded in working with date 
values and probably knows how to create, access and calculate them? 
Sadly, I've found no example code related to date values and there 
isn't much documentation too. Any info is greatly appreciated!

I would say it's absolutely necessary to find the definitions of 
the date values. Reverse engineering can work, but may hide subtle 
bugs, and take much longer before you're sure about the correctness 
of the implementation.

I've looked in DevBase, but haven't asked questions there yet. I've 
scanned all the extensions documentation on DocBase and the header 
files in the host-lib/extension packages. 


I didn't even manage to read a date value in a simple command like 
   TEST-DATE: COMMAND [DATE [DATE!]]      About every member n the 
REBOL value c-union type seemed to only contain zero values.

I have no idea on how date values are stored in C, all that the docs 
say is that date values are 32 bit encoded date and time zone values, 
so I mainly tried with value.int32a but tried other members too. 
I have no idea about how the encoding is done and - as Anton said 
- I really don't want to reverse engineer it.

But I suppose if you need it badly enough and can't find how to move 
dates then you parse and transform between the string formats.

On the rebol side, should just need a MOLD and a LOAD / TO-DATE...

Using external extensions with a104, I get an error when using these 
series macros :

RL_SERIES_INFO(ser, RXI_SER_TAIL) ,  RL_GET_CHAR(ser, idx) , and 
RL_MAKE_STRING. Did someone noticed this ?

Hmm, I somehow don't manage to compile my odbc extension with a109, 
on import I get the following error

>> import %../lib/odbc.dll
** Script error: make-scheme has no value
** Where: do -apply- make catch case case -apply- apply import
** Near: do body obj


I do understand that it's related to MAKE-SCHEME moving into SYS 
context,. Do I have to import SYS into my extension module's context 
first and, if so, how? NEEDS: [SYS] doesn't work, my extension is 
already declared with TYPE: MODULE OPTIONS: [EXTENSION DELAY], trying 
to access MAKE-SCHEME directly thru SYS/MAKE-SCHEME makes the intepreter 
crash.

Both bugs 1723 and 1729 have been fixed, for alpha 110. The next 
release should fix your problems, Christian.

We have been discussing adding a header option to tell the system 
to put off loading an embedded extension until after the mezzanines 
have loaded (or have that be the default, and have an option to load 
before the mezzanines). I haven't looked over the startup code yet 
to see how to do this, but it shouldn't be too hard. I'm waiting 
for Carl to decide on the option (he seems to be the namer of that 
kind of thing right now, I'm just an implementor).

Thanks, Brian, for the status update, that's good news in more than 
one way. I con stop trying and just expect it to work next time. 
.'J

I don't see the ODBC source, only link to dll and docs.

With the sample2.c example I have problem with undefined RXI_SERIES_INFO 
and RXI_GET_CHAR

i think you could also replace RXI_GET_CHAR by first obtaining a 
pointer using RL_SERIES(ser, RXI_SER_DATA) and then just indexing 
into this directly

I did several extensions for windows, and you can find the source 
code and the dlls here: http://www.colineau.fr/rebol/R3_extensions.html
. This code may not be up to date, but recently I have compiled new 
versions. If necessary, I can upload them.

funny, I intended also to try interfacing with ImageMagick (maybe 
the ImageMagick Com + component.), as I have a need for another project.

Concerning the extensions dll's and source files, I think that they 
are no more compatible. I should check and put the latest versions.

pages updated with the new versions (binaries and sources) compatibles 
a110 :
http://www.colineau.fr/rebol/R3_extensions.html

if you download the OpenGL demo, there is a complete working setup. 
 it has a makefile and a visual studio 2008 setup .  it should help 
you out.

I'm completely C/C++ newbie, so I would like to know, how to make 
the extension, if I have precompiled DLL.
For example, In R2 I can simply download the DLL like this one:
http://zlib.net/zlib125-dll.zip

And simply do:

zlib.dll: load/library %zlib1.dll

zlib-version: make routine! [ return: [string!] ] zlib.dll "zlibVersion"
zlib-version
;== "1.2.5"


Is there someone who can write a simple tutorial - C extension with 
the zlib-version command?

Then run make, and you should see something like the following:
$ make

gcc -DTO_WIN32 -I ../src/include -I include -mdll -L . -lzlib1 -o 
zlib.dll zlib.c

Now run the accompanying zlib.r, and you should get:
$ r3-a110-3-1 -q zlib.r
1.2.5

stdio.h is a leftover and not needed.

And isn't the conversion to a REBOL string quite complicated?

Yes and no.

developing an extension to wrap MagickWand is a huge work, because 
of the large number of functions, and data formats. It's better, 
in this case, to develop a full dedicated application in c, or, at 
least a dll with higher level functions ... wihch is approximately 
the case of ImageMagickObject. In other words, I am not convinced 
of the interest to program in Rebol if you want to use the full set 
of low level image processing functions.

yes, its VERY usefull... one doesn't want to have to compile applications 
and stuff... AFAIK there are binding to other languages which use 
the full IM suite.  

but yes its a lot of work.

Have you installed ImageMagick , and ImageMagickObject ?

With this version, all the inputs and outputs are files. I'm waiting 
for more specs before loading directly into memory. It's possible 
to convert many file formats (including tiff) into may others.

btw.. what is difference between simple CALL with args and using 
the ImageMagickObject?

could you check again the installation of ImageMagickObject by doing 
regsvr32 ImageMagickObject.dll (without /c and /s, you will have 
the messages indicating that everything is ok)

That's also a question for me ...

Part of the answer is that using it in a cpp exe will allow to execute 
these functions without te burden of executing a c process while 
masking the dos window (which is a huge challenge here, and I do'nt 
know why). In the case of Rebol, yes, the interest is not so evident.

though it does require usually thread safety and might require GC 
protection depending on what you are doing.

AFAIK we can't currently unload extensions.  


The extension interface actually is designed to support it, but right 
now, I think its never called and there is no way to trigger it from 
the REBOL code AFAIK.

Oldes, without changes given by andreas, you will corrupt the executable's 
memory and will *eventually* have a memory acces failure.

and of course `for (i=0; i<number_options; ++i)` as well

the reason I use c and n  for counters ;-)

The remaining bug will be somewhere in the interaction betweein RXIARG, 
arg.series and RL_SET_VALUE.

darn, ignore the above. I think I'm tired... I continued searching 
the C code, and posted too quickly.

And then you've got to properly construct the block in b :)

And the string?

Maxim, the setting of the string value of the block is actually my 
problem and that's why I'm asking. I can even return block with empty 
strings:) So I should extend my queastion to return block with non 
emptz string:)

Ok, confirmed that it works on both Linux and Windows A110.

you have to make sure to initialise all stack- and heap-allocated 
memory

When I have command like:
	test: command [str [string!]]
And on C side I need a pointer to this string, how to get it?

That's not working, the function I'm trying to call is:

char **MagickQueryConfigureOptions(const char *pattern, unsigned 
long *number_options)
And I would like to use REBOL input for the *pattern.

ah, I did a little bit of searching and I think I understand... 

try using RL_PRINT() instead.

and replace:
#define RL_PRINT(a,b)               RL->print(a,b)

with:
#define RL_PRINT(a,b)               RL->print(a,__VA_ARGS__)

can't test it right now, but that should work.

yes, once you understand that just about everything obeys the same 
basic pattern, it gets quite easy and addictive, since you get results 
pretty quickly  :-)

When I have integer (from REBOL side) and want to use it as a pointer, 
what I must do?

And, what would you do: (1.) Save wand pointers on REBOL side, or 
(2.) only on C side and provide just IDs (index) to this pointers 
(which is probably safer but would require dynamic array on the C 
side)?

So with Cyphre's help I have this function:
char* rebser_to_utf8(REBSER* series) {
    char *uf8str;
    REBCHR* str;
    REBINT result = RL_GET_STRING(series, 0 , (void**)&str);
        
    if (result > 0){
        //unicode string
        int iLen = wcslen(str);
        int oLen = iLen *  sizeof(REBCHR);
        uf8str = malloc(oLen);

        int result = WideCharToMultiByte(CP_UTF8, 0, str, iLen, uf8str, oLen, 
        0, 0);
        if (result == 0) {
            int err = GetLastError();
            RL->print("err: %d\n", err);
        }
    } else if (result < 0) {
        //bytes string (ascii or latin-1)
        uf8str = malloc(strlen((char *)str));
        strcpy(uf8str, (char *)str);
    }
    return uf8str;
}

and I can than use:
..
            char *filename = rebser_to_utf8(RXA_SERIES(frm, 1));
            status=MagickReadImage(current_wand, filename);
            free(filename);
            if (status == MagickFalse) {
                ThrowWandException(current_wand);
            }
            return RXR_TRUE;

because I don't know handle. I'm learning and there are no examples 
yet

REBOL doesn't have pointers, it has references, and it doesn't have 
addresses. So the only way you could legitimately get a pointer is 
to return it from a command. But you don't want to have any way to 
construct an illegitimate pointer in REBOL and pass it to a command 
because that would be a much worse security and stability problem 
than just having commands at all, and treating pointers as integers 
lets you do that. So there is the handle! type to store pointers. 
A handle! is an immediate value that is the size of a pointer, but 
that you can't convert directly to or from any other value, or even 
mold it to see its contents. When you return a pointer from a command 
you set the value to the handle! type. Then that handle! will be 
usable when passed back to other commands in the same extension, 
and maybe even when passed to other extensions, depending on address 
space issues. Handles are also used to store function pointers in 
R3, and other opaque system values like library addresses.

And on that subject, what is the current state of string conversion 
in extensions? When last I checked it was really poor, all sorts 
of missing macros.

what you can do, is wrap your command within a function in the extension's 
module and expose that function.  this allows you to control the 
return type of the command better and handle it within the comfort 
of rebol.

Updated my ImageMagick extension and doc. It is now possible to exchange 
image data between Rebol and ImageMagick without using disk files, 
thanks to the "mpr:" (memory program register) format which allows 
in-memory save and recover.

http://www.colineau.fr/rebol/R3_extensions.html(refresh the page 
in your web navigator)

I have a minute again.. so next step in my lesson...  Lets have:
typedef enum
{
  UndefinedResolution,
  PixelsPerInchResolution,
  PixelsPerCentimeterResolution
} ResolutionType;

and function like:  SomeFunc(const ResolutionType units);

What's the best way how to provide the units from the REBOL side 
to the extension's command?

on the other side I could just do range test and use the integer/variable 
from REBOL side.

I have this one so far:
int unit = RXA_INT64(frm, 1);
if(unit >= 0 && unit < 3) {
	SomeFunc(unit);
	return RXR_TRUE;
}
return RXR_FALSE;

and in REBOL just:
SomeFunc 2
or
PixelsPerCentimeterResolution: 2
SomeFunc PixelsPerCentimeterResolution
or using dialect

That is expected ;-) The numbers you'll get are fairly random. You 
would have to get the word-number of a supplied word and compare 
it to the word-number of the allowed words. If they match, act accordingly.

I never got RL_FIND_WORD to work, but I stopped trying around A102 
I guess and forgotten about it since then.

But I think that the easiest way is to define the enum constants 
during the extension init like:
    export PIXELS-PER-INCH-RESOLUTION: 1
    export PIXELS-PER-CENTIMETER-RESOLUTION: 2
    protect/words [
    	PIXELS-PER-INCH-RESOLUTION
    	PIXELS-PER-CENTIMETER-RESOLUTION
    ]
and then just use:
    SomeFunc PIXELS-PER-CENTIMETER-RESOLUTION
or
    SomeFunc 1

with the correct int bound check in the command so only valid numbers 
are processed.  I really like what is possible with R3.. I should 
start real coding in it soon.

Guys, I put the comment about 'temp hack' couple of months ago when 
I discussed this issue with Carl. Not sure but the init-words trick 
will probably stay as is because it works fine that way. Ofcourse 
I may be wrong if there is some better method and Carl decides differently.