AltME groups: search

Hi Doc, did you plan to integrate some Open CL programming acces 
to Red in any future ? Here is a summary of kernel programming with 
OpenCL - and to me this seems accessible to Red, some day : http://www.manning.com/scarpino2/ch04sample.pdf
 for a larger picture summary of the beast here is the link to the 
book I referred to :  http://www.manning.com/scarpino2/(this is 
the Manning's publiaher deal of the day ,,, that's why I talk abotu 
this now). May be just a new binding and some extensions are required 
- but I would like to know more about the actual modifs required 
- when a small time is affordable for you to answer ?

OpenCL: yes, I have that in mind too, but it's most probably for 
a later version of Red (> v1.0). Also I have other plans to give 
Red access to GPGPU, by directly having a GPU machine code assembler 
and a dialect on top of it. It would be mainly for compiler internal 
use, but such dialect could be exposed for user code if some are 
willing to implement some fast GPU-powered routines.

If you're doubting about such approach, read this article:

 http://www.geeks3d.com/20110317/low-level-gpu-programming-the-future-of-game-development-on-pc/

Nice Doc, Thanks for taking time to answer. I'll "follow the guide" 
as we say here ! Have a nice day. Now I'll go reading  your link.

OpenCL is certainly a quicker way to access it. My plan for direct 
GPU access is not only motivated by speed gains, but also by reducing 
complexity, having a simple, lightweight access to GPGPU capabilities.

Kaj: I've pushed a fix for ARM that should improve the stability, 
can you do some quick tests to see if some of the issues are fixed?

I've added preliminary support for 64-bits integers to the SQLite 
binding. They're represented by a float! so they can be passed around, 
but they're hard to compute with

The compiler is using two ways to encode the fact that a function 
is used as a callback:

- through the CDECL attribute presence (destined to be called by 
external code)

- through the internal CALLBACK flag that is assigned to all functions 
that get their pointer passed as argument (get-word! syntax).

Normally a function can't be used before it's defined. How could 
I pass a pointer to a function that's not defined yet?

I will push the fix for callbacks correct prolog/epilog generation 
in a minute.

(A bad copy/paste earlier changed the position of the first fix before 
I commited it, libc init is fine).

Yep, seemed to be a separate issue, like 0MQ

Kaj: for debugging the GTK issue, we need a minimalistic app. Could 
you re-implement the hello-GTK-world using only direct GTK calls 
and not going through typed functions? We could at least see if it's 
related to GTK calls or not.

Pekr: still a lot of details to work on before that, as always before 
a major release.

That also means, we'll probably postpone floats hardfp ABI support 
for a later release (probably 0.3.1).

Wrinting a new blog entry is part of the todo-list. ;-)

There already was a minimal GTK program with direct GTK calls: goodbye-cruel-GTK-world

Elapsed time is reporting a weird number in Fibonacci and Mandelbrot. 
I suppose that's due to running ARMEL code on an ARMHF platform

I am in a hurry, I want to report them before release :-)

That's a domain we can perfectly address with Red dialect + Red/System 
low-level access abilities.

Get Cyphre to do a modern View like engine, and we will be kings 
- just believe me :-)

I am in a hurry, I want to report them before release

 drive faster so you will be home before you are out of gas.  "But 
 yes, it's the domain I always wanted to target with REBOL" the domain 
 you want to target is the world isn't it?


I am studying the Red and Red/system sources these days to get more 
of a feeling what is going on.

Not exactly, I read rsc.r as if system-dialect is not set then start 
with the compiler.r script. (after that or else the rest of the script 
follows) but then is has some do other scripts and a modular part 
and the other scripts are modular too. It look all very sound bu 
somewhere <there must be/I expect> a first action or function call.

rsc: context

 does contain "fail-try "Driver" [main]" that looks like a starting 
 point but it is within the context. So in my mind that does not get 
 triggered.

I've found a bug in Red on ARM: the polymorphic dispatching isn't 
working.

RPi slowness: I've noticed it too...Even overclocked, at 900MHz, 
the UI still feel sluggish...Amiga 500 was having a fast UI running 
at 7MHz, so no excuse for those bloated UI stacks...

There's something deeply wrong in the way most so-called "modern" 
OS/desktops are designed. My 133MHz Bebox with a poor PCI video card 
was able to provide a very responsive UI, even under heavy load.


The RPi feels like dying as soon as you launch the lightweight (no 
kidding!) web browser Midori...Of course, the browsing is very slow...I'm 
quite disappointed by that and it's not the hardware fault, the RPi 
is a great platform, but the software stack sucks a lot.

At least in the official Raspbian you already have NetSurf, a completely 
custom browser ported from RISC OS

Doc, quick license question: Was the BSL chosen because it allows 
you to distribute a binary without requiring that you distribute 
the license, unlike MIT and almost all other open source licenses? 
Would it be a problem if you incorporated Apache licensed code, which 
doesn't distinguish between source or binaries in this? You probably 
wouldn't have to actually include the license with the product, only 
in a web site or help file somewhere...

The problem is that everyone compiling a Red program would have to 
include the licence with it when distributing

It doesn't say that the license must be included with the work, it 
just says that "You must give any other recipients of the Work or 
Derivative Works a copy of this License", it doesn't say how.

There are App Store apps that include a reference to the license 
in their store descriptions, without including them in their apps. 
It can be done.

If I put a link on my web site then I'm giving it. I don't have to 
include it, just give it. With BSD and MIT licenses you have to include 
it though.

When you distribute your REBOL program now, you just dump the script 
somewhere, on a web site or in an email or in AltME. To match that 
with Red, you don't want to have to give a licence at all

The length ot the text that you have to manage is immaterial. It 
doesn't matter if you have to manage a few lines of notice or a page 
of the full licence; you just don't want to have to think about it 
for every little progam

The Apache licence was written for a complete web server, the GPL 
is used for complete kernels

It's comparable with the runtime code that GCC compiles into a program 
binary

That has a special linking exception, actually.

Is there something like the BSL that has Apache's patent grants? 
Because (given that I'm in the US) patents are a bigger issue for 
me.

There is no such thing as "not having to provide a license". If a 
license is not provided, then the code cannot be (legally) used.

BrianH asked: "Was the BSL chosen because it allows you to distribute 
a binary without requiring that you distribute the license, unlike 
MIT and almost all other open source licenses?"


With the sole 3-clause BSD, users would need to provide a copy of 
the license with each compiled program, because of the runtime code 
included. So to avoid that, we chose to publish the runtime parts 
of Red and Red/System, under the more permissive BSL terms. The binaries 
that include the compilers (currently that would mean encapped version 
of Red and Red/System compilers) are still under BSD.


When Red will be self-hosted, it will include a JIT-compiler that 
will need to be released under also BSL or similar terms.

Gabriele, it's not about the code not having a licence, it's about 
not having to provide it. If the licence doesn't require physically 
including it, it's implied

Then there's software that doesn't even need to be bought. SQLite 
is released as public domain, so it doesn't need a licence nor a 
purchase, excepting countries that outlawed the concept of public 
domain

Doc, would you consider it sufficient to have a license that doesn't 
require that the license be included with (like MIT code) or distributed 
with (like BSD code) the product? The Apache license only requires 
that the license be given to the recipient - it hoesn't specify how 
- and it doesn't even require a copyright reference be included unless 
the product is distributed in source form. I'm just trying to determine 
the extent that you'll be able to include Apache-licensed code in 
Red, or how much this factor matters to you.

I'm perfectly fine with the current BSD/BSL licensing model we use 
for Red. I don't see the need for any change there for now.


I'm not sure what you mean precisely with "product". I don't see 
why third-parties redistributing the current Red compilers would 
have problem distributing a copy of the BSD license with them. For 
end-user binaries, users have no obligations to give a copyright 
reference or distribute a copy of the license.


If you are wondering about including possible future R3 code parts 
under APL-2 in Red codebase, I see no problem with that so far (as 
long as they are separate files or modules, we don't want to start 
having several licenses per file).

Brian, where do you get that the Apache licence doesn't require a 
copyright reference be included unless the product is distributed 
in source form?

As in my opinion, GPLv2-only libraries are a hostile mess I'd avoid 
in any case. Luckily I don't think I have come across any.

Quite a few media codecs are under GPL. If you would use things such 
as FFMPEG, the licence of the codec plug-ins would leak into Red 
through the plug-in framework. I haven't checked, but since those 
codecs have a long history, it's more likely that they're GPL 2 than 
3

Also, when you try to build an operating system with Red, you'd get 
into GPL 2 territory in kernel space, and you'd have a problem with 
the many GPL 2 drivers. The media codecs and some networking protocols 
mirror that situation in user space

Brian, as you noted before, "You must give any other recipients of 
the Work or Derivative Works a copy of this License" does not make 
a distinction between source and binary forms. That means that if 
you compile Apache into a Red program, you need to give a copy of 
the licence when you give the program to someone

Work" shall mean the work of authorship, whether in Source or Object 
form, made available under the License, as indicated by a copyright 
notice that is included in or attached to the work (an example is 
provided in the Appendix below)."

It was a bit painful commit anyway, as the internal API is not yet 
fully stabilized and to do it right, I would need to fully define 
the public runtime API first, but that would delay the 0.3.0 way 
too much, so we'll deal with that later.


What I mean by "public runtime API", is the Red API exposed to Red/System 
and to other host languages loading Red as a library. I'm still uncertain 
if such public API can be just some internal exposed or will need 
a thin layer of wrappers to make it handier (and in some cases, safer) 
to use. 


The R3 extension isolation model is too strong for my taste and makes 
the extensions harder to write than they should. I'm also uncertain 
if this model was stricly motivated by providing the safest possible 
interface with the core or, if the willing to keep the core internals 
as secret as possible was also playing a big part in this model choice.


Once the `dyn-lib-emitter` branch merged, I plan to study the Lua 
(and others) public API, to see if and how we can do better for Red. 
I already have a rough idea of how it should look like, I just need 
to refine it.

Red/System extensions will usually be compiled together with the 
Red runtime, so they will be much more flexible. For interfacing 
with precompiled binaries, a more stable interface would be needed

Public API stability: right, that's a good point in favor of a set 
of wrappers on top of current Red runtime API.

A stack-oriented API might be a good candidate for that...

For example, I've noted the alias! issue before. As long as all code 
is compiled together, alias numbers are a good interface, like symbol 
IDs are assigned at runtime in REBOL. But when precompiled code needs 
to communicate they become useless, hence why the R3 interface makes 
efforts to map symbols to known numbers

Part of the motivation of the R3 extension/host API was to isolate 
the extensions and hosts from changes in the underlying data model, 
which makes hosts and extensions really resilient to upgrades of 
R3. It also was designed to let you have a consistent internal execution 
model even in cases where the host has a completely different process/thread 
model.

I think it went a little too far at times, especially the lack of 
marshallers for immediate values that are more than 64 bits internally. 
I've frequently wanted to supplement it with marshallers for the 
other datatypes in R3, particularly the date, time and money types.

The big win is the command dispatch model though, because it basically 
lets you get dispatch to JIT-compiled functions for free. The dispatch 
function can manage changes between execution models completely without 
R3 even noticing. Lua has similar separation, though since it doesn't 
have to support anywhere near as many datatypes it can get away with 
a stack-based interface.

However, the way that the R3 script loader works, you could make 
an extension that has both Red and Rebol scripts in the module data, 
especially if Red uses a similar script-in-a-block embedding method,

Hi Kaj on my macbook:

Last login: Fri Oct 19 07:21:16 on 
ttys001

 MacBook-van-Arnold-160:~ Arnold$ 
 /Users/Arnold/Downloads/Red\(/System\)\ 
 Testing-dc1b702068063b65/Darwin/Red/hello 
 ; exit;
Hello, world!
§±ÖÁµ, ºÌüµ!
`O}Y, NLu
Dobrý den sv
te
logout

[Proces voltooid]

 /Users/Arnold/Downloads/Red\(/System\)\ Testing-dc1b702068063b65/Darwin/Red/Fibonacci 
 ; exit;

MacBook-van-Arnold-160:~ Arnold$ /Users/Arnold/Downloads/Red\(/System\)\ 
Testing-dc1b702068063b65/Darwin/Red/Fibonacci ; exit;
Fibonacci 35: 9227465
logout

[Proces voltooid]


MacBook-van-Arnold-160:~ Arnold$ /Users/Arnold/Downloads/Red\(/System\)\ 
Testing-dc1b702068063b65/Darwin/Red/empty ; exit;
logout

[Proces voltooid]
I took the programs from the Red tree under Darwin

Do you need a print?

I need one HTC sensation, press of a shooter, one button press to 
get jpeg into my email or facebook. Just throw your workflow to the 
trashcan, you can do better nowadays :-)

Good to know, but we'll need to make a last testing pass once the 
merge of 0.3.0 done.

Oh, the particular quality of the R3 extension dispatch model that 
makes it well-suited to JIT compiler implementation is that a command 
function contains an indirect reference to the dispatch function, 
and an index integer. When the command is called, the runtime calls 
the dispatch function and passes the integer and a marshalled stack 
frame. For a JIT compiler dispatch function, the index of the command 
can be an index into an array of function pointers or something like 
that, and the dispatch function can just pass the stack frame to 
the appropriate function, then return the results. This means that 
the hard part of JIT compiling - getting the regular runtime to call 
the created functions - is something that you essentially get for 
free with the existing command mechanism.


You could also use the dispatch function to marshall arguments into 
another runtime with a different call model. You could, for instance, 
have a dispatch function that pushes the contents of a marshalled 
stack frame onto a Lua stack and calls Lua functions. Or you could 
do something similar for LLVM functions, or ActiveScripting languages, 
or V8, or ODBC queries, or even Red's JIT.


This all depends on having a good marshalling model in the first 
place that can handle the datatypes you need to support, and it would 
also help if there was a good task-safe callback mechanism that actually 
works (R3's needs a bit of work at the moment). Still, the principle 
is sound.

I get a new error on GTK on Linux when it tries to load the Red-48x48.png 
logo:

It's helpful to make a conceptual distinction between the host interface 
and the extension interface, even though for R3 they are currently 
related to each other and share a lot of the same code.


For the host interface, the host is the OS (more or less) and provides 
an execution environment that the R3 runtime runs on like a program 
(this is all metaphorical, but I'm sure you get it). The OS in this 
case could be something like Windows, Linux, some microkernel, whatever, 
or it could be an application or application plugin like Eclipse, 
Visual Studio, Notepad++, Excel, Firefox, whatever.


For the extension interface, R3 is the OS, the extension-embedded 
module is the program that runs on the OS, and that program calls 
the extension's native code like a library. The program source is 
returned by the extension's RX_Init function, and that program then 
wraps the native library code. The module source is loaded like a 
normal script (slightly hacked after loading to make it a better 
wrapper), so the script could be embedded in binary data along with 
non-Rebol stuff just like with normal scripts. You could even have 
Red and Rebol scripts in the same file (if they use the same embedding 
method) so you the data the init function returns can be like a Red/Rebol 
fat binary, metaphorically.


Given this, Red could either be (or compile) a host for R3; or it 
could be (or compile) a runtime library that implements the same 
host interface as r3lib, making it a drop-in replacement for R3; 
or it could be (or compile) an extension that R3 is a client of, 
returning R3 code that calls calls the compiled Red code; or it could 
be an alternate extension container, for extensions that return both 
Red and R3 code from the same init function, which would call the 
Red code returned, which would in turn call the same native code. 
The two languages could be integrated at any point in the stack, 
along with other languages.

It would be much better to have a real IDL for the command descriptions

Most extension module scripts are currently IDL-like, but that is 
only because they aren't (and don't need to be) very ambitious in 
their system integration because they just export a bunch of functions. 
Any native implementation of a port scheme, native dialect, or other 
system enhancement would need Rebol code to integrate that enhancement. 
Doing that in Rebol code is what allows the actual native interface 
to be that simple. It is also what would allow Red wrapper code (which 
could be returned from the same RX_Init function in the same string) 
to use the same native code unchanged, even though Red's runtime 
model is likely to require different integration code.

That has been a fabulously successful strategy for Microsoft

Except that in a careless extension, you wouldn't be able to execute 
the module script in Red

Not if you split off the interface into a proper IDL

Ah, but the interface is too simple to need an IDL - make command! 
will do. The extra stuff is for system integration, which is only 
needed when you are doing port schemes, dialects, anything that you 
wouldn't expect to be cross-language compatible anyways, unless you 
explicitly implement a compatible system model. If you're just exporting 
functions then you can implement a simple IDL just by interpreting 
the (cooincidentally the same) module spec code with a very limited 
IDL dialect processor if no Red script wrapper is found.

Btw, you can't run applications for Linux on Unix without a compatibility 
library either. Bringing MS into it is just an insult.

For that matter, I expect to make my own Rebol spinoff language that 
will follow a completely different system model than either Red or 
Rebol, and the only reason to do so is because those other languages 
don't cover that situation (otherwise I would have been more active 
in Red so far). Being different justifies their existence; interoperating 
with each other justifies their cooperation :)

Assuming that Red will be compiled (even JIT), the actual semantics 
will be different even if the outside behavior will appear to be 
similar enough that it won't matter for most people, hence the "fake 
it" phrase. It would be a disservice to us if we got a compiler, 
which has definite if minimal disadvantges over an interpreter, without 
getting the advantages of a compiler such as a practical optimizer. 
The behavior of R3 and Red could be quite similar to an outside observer 
that doesn't look closely, but they would require different optimization 
strategies to get the most efficient code. In that way I don't expect 
them to be compatible - they would likely be even less compatible 
than R2 and R3. But that's not really a problem :)

Semantics will be very close, probably 99% close, as I've moved from 
a statically typed system to an hybrid static/dynamic typing system.

So, if all goes well, you'll have the full REBOL model + optimizations. 
There are many ways to optimize and on many different abstraction 
levels. Yet, some atomic operations will remain "unoptimized", like 
series atomic manipulations (maybe we'll figure out a way to optimize 
them too in the future).

Lexical binding will be added when we'll add contexts (in a few weeks), 
along with dynamic binding (yes, you'll be able to have fun with 
BIND again ;-)).

In a couple of days, I'll release the first Red alpha with a blog 
entry to describe it, you'll be able to get a better picture of what's 
in Red already and where it's heading.

Being simple

 is probably the most important feature I would require from a module 
 system.

There are still some parts that I could use a little help with though, 
like resolving import cycles. And there are some bugs and/or weaknesses 
in the native code that I've had to work around in my patches collection.

The "being simple" part was mostly driven by Carl. I tried to make 
it simple, but it took some doing to make it even more simple for 
Carl. There's a lot of subtle code in there to make it simple enough 
for Carl to want to use it.

Resolving import cycles

: we've solved that in Red/System compiler by having a simple "included 
files" list and it stops inclusion if already done once. But we do 
that at compile-time, so it's easy.

A module system is where you would find a lot of places where the 
internals would be different, due to the compilation thing, but the 
external behavior could appear to be the same. There are a lot of 
nice tricks in R3's module system, and I've gone through a lot of 
trouble to make it possible to have the module system be handled 
by a preprocessor that doesn't have to execute any of the code in 
the modukle bodies to resolve dependencies. It was designed with 
preprocessors like prebol or Ladislav's include in mind. And all 
of that is in the specifications of how modules are declared, not 
in the actual mechanism that implements it, so the concepts would 
be portable to Red.

Certainly, but AOT compilation would be a big plus. Also, in Red 
we need to modularize the compilation process itself, so  we can 
do incremental AOT compilation on multi-files projects instead of 
having to rebuild everything (include runtime libraries) each time. 
We need to, somehow, match those compilation units with the higher-level 
module system (it's not the only option, but probably the most simple).

Multi-scripts would be mostly a packaging method. It could even help 
AOT compiled scripts at the script distribution phase. I'm thinking 
of install-time-compiled scripts, for instance. It would also help 
with mixed-Rebol-Red projects like what I described above for R3 
extensions. Also, it could help for data files in some cases, or 
metadata embedded in other files.

Well, the R3 module system is designed to be statically resolvable, 
so the same method would work for resolving Red modules in a project.

With compressed scripts, you can either have raw compressed data, 
or binary! syntax compressed data, after the header and an optional 
trailing newline. If you have raw binary data and a length header 
then it is only decompressed until the end of the length (with DECOMPRESS/part). 
The option of binary! syntax is useful for block-embedded scripts 
or scripts posted in a text environment, and it doesn't really combine 
well with the length header so that is ignored in this case; one 
of them had to take precedence (until I get TRANSCODE/part) so I 
picked compression. It is more likely that the length header and 
raw compressed data would be combined, anyways, For compressed scripts, 
the checksum applies to the decompressed binary data.

install-time-compiled scripts
 We'll support that option too.


Statically resolvable module system: that's a very useful feature 
to have, not only for compilation, but for auto-documentation generation 
too.

You should take upcoming red|rebin format also into account. That 
might be a preferable method sometimes rather than compressed scripts. 
Red will pre-compile most of the scripts content, but we might keep 
some "code" blocks in source form for reflection (when used by the 
script and inferred by the compiler).

I wanted to publish my redbin specs earlier, but stopped when I read 
that R3 might use a copyleft license.

An asymetric permissive licensing model between Red and R3 might 
prevent me from publishing info about what we'll do next, to prevent 
the features from being implemented first with a copyleft license 
attached. It would just be unfair to Red, so I would have to be cautious 
about that.

Don't underestimate the value of treating the header and the rest 
of the script differently. It lets you put a lot of static information 
in the header without even looking in the sctipt data at all.

Right, the header is a great source of metadata.

Given that he's getting advice from Rosen, something like Apache 
seems likely. He has already shown a certain scepticism for the need 
for copyleft in this case. One can hope, at least.

I threw in a hint for a new license, the URL or Unified REBOL License, 
for the best of both worlds.

Doc-strings are now allowed in Red/System function spec:

https://github.com/dockimbel/Red/commit/b7f46eafe75b43f43d7cb282d4415e89c2858a5e


So, who's going to write a short and nice REBOL script now to extract 
them (extracting also the context where they are defined)? :-) Ideally, 
those info should be collected into a simple nested block structure 
that can be easily processed for generating formatted text output, 
HTML, PDF (using pdf-maker lib), ... (take make-doc.r script as example 
of parser/emitter separation, and plugable emitters).

You're welcome! I use Cheyenne too in my business and I didn't donate 
for it before, so I feel guilty a bit, and do it for Red :)

isn't there a Red interpreter in there?

Also, we only have ~200 unit tests for Red so far, so I'm unsure 
how stable is the current Red codebase. In a few weeks, we should 
have enough features to port QuickTest to Red, then we should be 
able to have an exhaustive test coverage.