AltME groups: search

In theory that kind of thing is supposed to be handled by R3's device/port 
model, which also will handle synchronization with R3's tasks. In 
practice noone has made a user-defined device yet, though I'm not 
quite sure why. Device extensions have been promised for a while 
now, but not yet delivered.

Maxim did a bit of research into a non-task-safe workaround to create 
callbacks.

As mentioned a few times in the past, I don't think that user-defined 
devices are at all possible, with the current hostkit. And yes, I'd 
love to be proven wrong :)

Ok, since we are not sure, I will take a deeper look into the code 
and try to ask Carl.

a little question... how will the gui be able to call rebol functions?

has anyone tried to build R3 host as a dll, and use it within R2 
 as a routine?

I have a client which might need R3 parsing but needs to use the 
R2 run-time...

we are talking an expert PARSE user, and so far , this would be his 
main reason to move to R3, but then all his prior code isn't upgraded 
so its a bit daunting to just plunge right in the middle of a project.

maybe but doing so on several thousand line parse rules is, well, 
more work than probably trying the R3 as a dll  :-D

You are thinking about scale relative to the wrong number. It's not 
a matter of how many lines the parse rules have, it's a matter of 
how many additions there are in R3 parse, and there aren't that many. 
Make the compiler once, and it will compile quickly. It could even 
do so at runtime.

R3 Host-Kit A100 has been uploaded to www.rebol.com.  This release 
can build CORE or VIEW (with externalized graphics lib.)


However, although the graphics lib is there, the DRAW commands are 
still in the process of conversion (Cyphre will be working on that 
part).


It's also an example (and a fairly simple one to learn from) of how 
to build a host-based extension module, including a few scripts that 
show how to process the module source to embed it in the boot.

The host kit comes with all files necessary to build itself.  In 
theory, all you need to do is type "make" and it will produce R3/View 
(as r3.exe).  Also, make core will produce a core exe, w/o graphics.

The code includes some new host source files, for example to handle 
windowing and the related events.


Although this release comes with the AGG sources needed to do the 
build, note that these sources are pre-GNU versions, so are a bit 
older.


In addition, I suspect Maxim or someone may want to try creating 
an OpenGL version.


Soon, we plan to take a long look at making PAIR! into a float-based 
point, to allow better control with graphics. This somewhat non-trivial 
due to the assumptions in code that PAIR! is integer only with truncation 
for math operations.

k - I was able to build the a96 one - will give this one a go too

Any timeline for a Linux libr3.so?

If not, someone could write one, if you release a Linux hostkit as 
well. If you do, why not release a single cross-platform package?

A single source base which automatically builds correctly for the 
platform the build tool is invoked on.

A few things on that...

I used CMake, which is a cross-plattform tool that automatically 
generates "project" files for a variety of targets (such as GNU Make, 
Eclipse project files, Visual Studio project files). It works great 
on Linux, Win32, OSX and is used by several high-profile projects. 
But I'm well aware that third-party tools are most likely no-go for 
REBOL, so I'm fine with all that, and can easily look into it, as 
long as there' s at least a Linux hostkit available.

So, what header files do you find a need to generate above?

http://203.97.234.193:8080/secure/project/ViewProject.jspa?pid=10020 
... I've added a hostkit tracker to my Jira instance.

CMake Input is a "CMakeLists.txt" file, which is written in "just 
another Makefile language".

Can't sensibly try without e.g. a OSX dylib.

( the current version of Os/2 is ecomworkstation  and is positioned 
as a secure enterprise client environemnt )

CMake provides the benefit that we don't have to reinvent the wheel 
for now. It has, for example, a nice installer for Win32 and can 
automatically generate Visual Studio project files from the above.

well, if you are considering a makefile generator, at least cmake 
is somewhat of a standard

CMake takes the above as input and generates e.g. a Makefile as output.

With CMake the workflow of a user who wants to build the hostkit 
on Win32 would be as follows:
- Download and install a toolchain (e.g. MingW or MSVC)
- Download and install CMake
- Download the R3 Hostkit

- Generate a R3 build script for your preferred toolchain using CMake. 
E.g. use CMake to generate a GNU make Makefile for MingW
- Build the R3 Hostkit (using the generated build script)

Without CMake, those steps would be the same, except for the extra 
installation of CMake and the generation of a local build script. 
But it would also either limit the user to a single toolchain (such 
as MingW, currently) or you would need to replicate some of the functionality 
of CMake in your make-build.r

there's a GUI front end for cmake too, though I guess one could be 
made for REBOL just as easily - this lets you resolve env var issues 
and other things

It built for me just fine but I had to modify the makefile to use 
cp lib\r3lib.dll .
instead of 
copy lib\r3lib.dll . 
inside a mingw cmd window.

Do we have a test files to run to check if all areas of R3 that is 
implemented is working?

After adding MinGW\bin and msys\1.0\bin to the windows command path 
and also copying rebol.exe to msys\1.0\bin folder, I was able to 
build it from a normal command window without modifying the makefile.

Carl: in any case, I don't care much about CMake. Just shipping the 
hostkit with your make-build.r instead of a pre-built makefile would 
most likely do just fine for getting an integrated cross-platrform 
build working. (Plus, of course, bundling the libraries and sources 
for all platforms in the hostkit.)

And just in case: _THANKS_ a million for the new hostkit code drop, 
Carl (and everyone else who helped make it happen)!

we now have a domain ( thanks Andreas ) for the tracker http://jira.rebolsource.net:8080/secure/Dashboard.jspa

and now is probably as good a time as any to move this discussion 
to !REBOL3 Host Kit

I agree - for R3 to gain traction, mobile support should be a priority.

Jira: Why a new tracker? Isn't CureCode still be used? No information 
fragmentation please.

Priorities: Before getting new platforms up & running I see feature 
completion to be more critical. Otherwise we have a moving target 
that takes to much effort to keep in sync.

Robert, if you read above, Carl did not want to make a public release 
unless someone gathered all the comments ... and Andreas & I volunteered.

The curecode tracker does not have a hostkit project ... so I used 
my new Jira tracker.  Simple as that.

Most of the host kit problems are R3 problems as well. We have categories 
in R3's Curecode project, and we can add a Host-Kit category if needed 
(I will do so right now). Multiple trackers are not a good idea, 
in general. We'll see if they become a problem in this specific case.

Done. As with all new versions added, be careful to set that field 
when you write up a new ticket. But we should be careful with that, 
because in general we haven't had prereleases. We may end up with 
problems reported for a100 that will be fixed in a100 :)

The "current hostkit drop" is not a release, it's a prerelease.

Carl calls it a release, so I consider it a release. And yes, should 
be in R3 Host Kit

I too vote for an  A101 numbering the next time a package is released... 
its going to be MUCH easier to follow for everyone.   IIRC the A97 
created this strange pre-release condition too and it had to be specified 
each time we talked about what we downloaded.

Here are two functions I created to handle strings a bit simpler:

continuing Request for object lookup within host/extensions here....



Here is a proposal (using example C code) for OBJECT access within 
extensions  this uses the EXACT same API as commands, and with just 
a little bit of work on the extensions API by Carl and a single hook 
within the r3core, we could allow callbacks within objects.


the advantage is that we re-use the same safe sand box as commands, 
and don't require to do much coding to enable it, AFAICT.


since object lookup is performed on demand, we can very rapidly supply 
an object to the command, and it doesn't cause any heavy-handed conversion 
at each command call if the object isn't needed.  more fine-grained 
control could be added so we don't need to frame the whole object, 
but it would be usefull as-is already.

RXIEXT int RX_Call(int cmd, RXIFRM *frm) {
	i64 i;
	
	// objects are frames, exactly like command arguments.
	RXIFRM *obj;
	

 // return values are also frames (with one argument), for type-checking 
 purposes.
	RXIFRM *rval;
	
	if (cmd == 1) {
		// ask the core for a frame which represents object.
		// attributes are returned in order they appear in object
		//

  // the command frame doesn't include the object frame, only a handle 
  to the

  // object which the core can understand. we then ask for the object 
  frame on-demand.
		obj = RXA_OBJ(frm, 1);
		
		switch (RXA_TYPE(obj, 1)) {
			case RXT_INTEGER:
	            i = RXA_INT64(obj, 1);
				break;
			
			// we could build a frame for supplying arguments to the eval
			case RXT_FUNCTION:
				rval = RXA_EVAL(obj, 1);
				if (RXA_TYPE(rval, 1) == RXT_INTEGER){
	            	i = RXA_INT64(rval, 1);
				} else {
					// return error
				}
				break;
			
			// wrong type
			default:
				// return error
				break;
		}
		
		// do something with the value
		my-native-function(i);

	}
}

so what do you guys think?

This idea was borne out of a previous code snippet I was building 
which actually used accessors   :-)


as I was writting it up, it occured to me that I was providing a 
different syntax for the same problem.

another approach would be to access a single attribute at a time, 
but anytime data is exchanged, I think re-using the current frame 
code is ideal... 


it already solves all of the type and memory marshalling, separates 
the external data from the core, solving the issues with the GC and 
stuff like that.


as long as the API copies data when its setting values back in the 
core (which I guess its already doing) then I see no point in inventing 
a second API just for objects.

IMHO, we cannot prevent marshalling without exposing a lot more of 
the core into the extensions, and/or making extensions unsecure (able 
to peek directly within the core datasets).

There are a number of bad links in the extensions-embedded.html document 
... are they just docs yet to written? Or has the doc been moved 
destroying the links?

the idea of handles is for an extension to give a key to a script, 
and make sure the script can't play with it.   by key, I mean an 
arbitrary value which makes sense for the extension (pointer, index, 
value, etc).

Maxim: let's see if I can get you a Reb_Get_Object(obj, word) function. 
That would work ok, right?

yes that would be great.  the important detail is the need for this 
accessor to be able to get objects too, so that we can browse a whole 
object structure.

re: "GC recycling interruptions"  if we can browse the objects directly 
from extensions a lot of the need to "bake" command access within 
blocks and execute via do-commands is alleviated in the first place.

Yep, I think it is a reasonable request.  The trick would be that 
you'd add any field names to the WORDs block of your ext module.

sure.


would there be a way to inspect an object to build something like 
in native C ?:

foreach words-of object [
	; ... do something ...
]

as an example, we could just represent a draw structure using primitives 
as objects (which can include several AGG primitives) and call AGG 
native code directly from the object tree.

something like 
draw context [
	prim: 'cross-box
	offset: 10x10
	size: 20x20
	thickness: 1
	border: black
	cross-color: red 
]


would draw a box and two cross lines, without the need to reduce 
this at every refresh when its refreshed and data comes from external 
manipulators:

draw compose [
	pen (prim/border)
	fill-pen none 
	line-width (prim/thickness)
	line-pattern none
	box  (prim/offset) (prim/offset + prim/size - 1x1)
	pen (prim/cross-color)		
	line (prim/offset) (prim/offset + prim/size - 1x1)

 line (prim/offset  + (prim/size * 1x0)) (prim/offset  + (prim/size 
 * 0x1))
]

For example, many graphical objects will be cached in their reduced 
blocks, so it's not a GC problem.

but the first one becomes a persistent object and doesn't need to 
reduce over and over.  


when the structure becomes somethings which goes beyond 50kb of draw 
source,  reducing over and over hits the GC very fast.

Well... I find that most coders don't pay close attention to their 
garbage side effects, but in a GUI sysetm, it's wise to do so.

Again, it really depends... because the DRAW block is constructed, 
objects are not that good of a substitute, because their only benefit 
is named fields, which in a constructed domain offer no clear advantage.

also note that I gave this example as a Draw, since we can all quickly 
see what is happening, but the above applies to many other datasets 
which are used libs, and for which objects are much more user-friendly 
than objects.


think of C libs which use deep struct trees. I'd rather keep the 
user interface object related.

yes... but they also allow context on the REBOL source side.


note that with the example above using objects, I don't need to go 
thru a draw block on the native side.  I could inspect the object 
and fire AGG (or other lib ;-) commands directly

as you say, its a case by case issue, but in most of my larger projects, 
i end up using object for a variety of management reasons.

Yes, but there is a problem in your example above in that one is 
an apple and the other an orange.

An object does not represent sequence, it represents state. A DRAW 
block represents sequence. They can be as long as you need.

So, for the above DRAW with object example to be useful, it would 
require a sequence of objects, so you're back to a block.

It's funny, I go back and forth a lot on my own designs in regard 
to object vs block.

For the human side, such as providing a style sheet containing graphics 
attributes, object is the winner. However, as that style sheet is 
processed, it is flattened into a sequence of commands sent to the 
AGG rendering engine.


Now, it's probably possible to change our API into AGG to use objects, 
and that's probably find, but I'm not sure that it's really any more 
efficient.

true, but objects can be nested. and a single small object, like 
in the above, may actually represent MANY draw commands.
 

for example... a single block of text strings... may actually represent 
all the items of a text list.  parsing that list to fit things within 
bounds. 

re-creating the whole AGG block as you scroll the list, forces you 
to possibly generate a few hundred draw items in a block.


but you have to build that block using intepreted code, which only 
ends up being an intermediate in order to pass the visuals to the 
rendering.


with an object, constructing that visual can all be handled on the 
native side and will save a lot of work on the interpreter and the 
GC.

the object will contain a few parameters and it represents the list... 
but rendering it will all be managed in the native side.

though, as I said, this is not specifically Draw related... if I 
want to represent a node structure which has properties, methods 
as well as data, doing so using blocks is impossible to manage.

The above block of texts example is puzzling to me. Here's why... 
normally, the display is constructed from nested layers of GOBs, 
each with offsets. So, scrolling a specific area of the display does 
not require any reconstruction of the blocks.

It seems to me that the such a method result will be a program that 
is many times the size and complexity... and hence cost and brittleness.

Of course, I may not be understanding precisely your method.

I want to use R3 in a project that means I will need to manage hundreds 
of thousands of "things".


If I can shove all of the heavy lifting into the native side of things, 
then I use REBOL for what its good at, high-level control.  but the 
datasets might still be huge.

At the core, a CPU executes a sequence of instructions, not an object.

liquid uses state to control processing.  each node is like a mini 
kernel only aware of itself.  by linking stuff together and with 
messaging, all the nodes cooperate.


right now, liquid is able to completely control a complete GUI forking 
of only the refresh of data which actually changes.  but i cannot 
implement some liquid's low-level code within extensions because 
they can't look up the nodes.

steeve, right, because in R2 there is other way to make a 100% AGG 
interface.

nested draw blocks may be slower in R3, but I see a benefit with 
Maxim's way. It allows easy composing and inheritance of transformations 
in the sub-blocks, like skewing, rotation, translation, scaling.

liquid is at a lower level than gobs (its not strictly gfx related) 
and I need object access for liquid (and other things).

off the topic, 

Gobs are not specifically related to graphics if they are not rendered 
(showed)

Gobs could be used to implement some efficient data structures, like 
linked list or tree.

As far I tested, dealing with structures of gobs is faster than with 
standard objets.

A really cool feature is that when a gob is append as a child to 
a gob, it's removed from its current parent automaticly.

I need to find a group to discuss the PAIR changes happening. Suggestions 
anyone?

I guess. Kind of general group tho. Was hoping we had a graphics 
or datatypes group.

Looks like it should be possible to call Java libraries from a C 
extension using the Java native interface ...

I guess we have to prove we're not a joke ...

Jocko, I was able to compile your tts example under visual studio 
express 2010 but with a lot of difficulties :(  
Size of the dll was 19.5 kbs vs your example which is 8.5kb

Maxim: Get and set of object fields has been added to the extension 
API in A101. So, great things can happen now?


But... before A101 is released I need a decent test for extensions, 
and Robert will be coordinating that project.

Good to hear that the ban on directly importing extensions from URLs 
is still in effect. It's a good rule.

It's good, really. It means that you can secure a sandbox and still 
allow extensions, just by prohibiting writing to disk.

Here's another test 
download  http://www.compkarori.com/r3extend/img.zip
unpack it
run the r3.exe

secure [ extension allow ]
import %3d.dll
test

and it shows a 3d figure you can rotate with the mouse

Ok, so there's a new R3 sitting here on the shelf drying out waiting 
for some better testing.

To be released once we have a volunteer to write some tests for it.

I presume curecode has a view filter too

We need a way to test extensions with each release.


Tests need to be able to handle normal extensions as well as embedded 
extensions (because the difference is only a few #ifdef statements.)

There are a few "dimensions" to test the basic mechanisms:

1. some commands (in a named extension module)
2. commands that use and verify all possible argument datatypes
3. commands that return all different datatypes
4. defining and setting words (variables)
5. accessing values in a block

6. accessing values in an object (new feature, but this is one reason 
why we need the test)

I'm sure there are a few other misc items.


We think that the entire tests can be generated with a REBOL script, 
and save a lot of typing. However, that's not a requirement.

This is a simple to use test environment that can automatically run 
thousands of tests.

And, of course the host-kit that needs to be tested needs to be released. 
IMO we will release a RC to get feedback on the extension part.

Pekr, yes... RT uses a simple testing framework.

The concept of "just upload it" does not work.  Here's why:


1. Parts of the API may not work at all, because of course, they 
are not tested!


2. Parts may work incorrectly... and therefore are just going to 
confuse most developers, and they'll get frustrated.


3. Because the code is not tested, it floods CureCode with tickets, 
most of which should not even be posted there.


4. Without a testing methodology, there will be no consistent test 
suite over time... just a bunch of hacked together code pieces from 
various developers.