AltME groups: search

I have pushed a commit yesterday night that reimplements almost fully 
namespaces support in Red/System. It is now cleaner, fixes a lot 
of issues and it is allows faster compilation of apps that heavily 
rely on namespaces like Red's runtime code. For example, the demo 
Red script now compiles 20% faster.


Please test well your current scripts to spot eventual regressions 
caused by this change.

james_nak: When you run the Red tests, their output is logged to 
Red/quick-test/quick-test.log. Would you mind taking a look to see 
if any error message was logged.

If I use a foreach on a c-string, how can I tell when I am at the 
last character?

tail? stringname doesn't seem to work. Maybe I need to use length 
to keep track of where I am!?

As I understand, there is no foreach in the current version of Red/System 
and no c-strings in the current version of Red.

The best loop to use in Red/System is until:

str: "1234567"
until [
  print str/1
  str: str + 1
   str/1 = null-byte
]

Will print the characters in a c-string ( as would print str).

Hmm, OK, I was using Red so probably it is a string! , but there 
seems to be a foreach.

In such case, you should not use FOREACH, but an alternative. FOREACH 
doesn't handle a series offset, so you can't test for a position 
in the series.

Doc, now that REBOL is Open Source, I ask which are the differencies 
between REBOL3 and RED to create a different language ? Can't you 
propose your view and have it merged into REBOL3 ?

Giuseppe: no, R3 is an interpreter based on C, Red is a compiler 
based on Red/System. I hardly can see how they could be "merged".

You will be involved in a 2 year commitment to make something that 
it will be quite close to another product with very few differencies.

RED/System is a "different" low level language. It is nice.

I see a great future for it.

RED is only a REBOL clone.

Keep in mind that R3's concurrency model isn't really set yet. That 
is an area where you can make a lot of impact.

Doc, lets suppose I am a customer:

I need a veicle with 4 wheels and and engine.

my vote is already a 100% for Red :)

Yes but do you need a family car or a Formula1 car? Or a truck?

Giuseppe - let's really be fair to Doc here! You are asking him to 
give-up on what was really a strong decision for him, which even 
has influenced his life. Doc moved from France to Montenegro, in 
order to work on Red. And all that would not be necessary, if Carl 
would not let us in the water. So really - just let Doc alone for 
few moments, to think about all new consequences ...

BrianH: I think Doc's answer was to Giuseppe claiming that Red is 
just a REBOl clone ...

Arnold. We are talking about minor differences like a car with 2 
doors instead of 4 and electric glasses in place of manual ones. 
The comparison doesn't suit here. If BOTH languages are "Very Close" 
(read above), which are the advantages ?

Giuseppe: you seem to not understand the difference between a interpreter 
and a compiler.

But I want to make the DEVIL ADVOCATE.

When RED will be mature. In which scenario should a developer choose 
it instead of REBOL3 ?

It is a question that must be answered. It is the core of your project, 
it is the motivation for your "customers".

GiuseppeC: Red/System is a language to build other languages using 
a similar syntax as REBOL, one of which is Red. R3 is based on C. 
There is no way for R3 to tap directly inline into C's performance, 
while Red will be able to. I think this is quite a feat that might 
make Red much more flexible than R3. You also get encapping right 
out of the box with the current compiler. I can't come up with an 
appropriate car analogy.

Red/System also offers a nicely compact and clean toolchain, which 
R3 doesn't even focus on. As far as I know, not many people like 
the current C toolchains.

That means that we have total 100% control over how and where Red 
can build. We don't depend on the quality of the compiler for a platform.

I can see and huge difference between C and RED/System which give 
to the latter a great appeal to the developers.

Henrik speaks wise words. For the past decade, we've been occupied 
with maintaining the GNU C/C++ toolchain in Syllable. If it hadn't 
been so problematic, we could have spent that time developing the 
operating system itself, and the project might have been in a much 
better position now

So the difference between Red and R3 is between a car that takes 
you there, and a car that breaks down before reaching your destination

It's a fair question, but the answer depends on insight

Well Guiseppe, you may have to think of a second hand car ;) 

You can drive with it, it can be the car you wanted but some details 
as its color and chairheating can differ from what you had in mind. 
It is up to you to buy it anyway. By buying second hand you can have 
a more powerful engine than buying a new car.

It may be time to consider REBOL an idea, a good one and one that 
now needs to have its true wings in the form of Red.

Does R3 need a console? It would be the interpreter made using Red.

Kaj - not sure it will be a console, but something like that, just 
not an interpreter, but more a JIT compiler?

Red will be a JIT compiler, but you could still write an interpreter 
on top of it. Might even be useful, for example for platforms that 
block JIT compilation

I posted in Sublime Text's forum in regard to the lexing needs that 
we might need for good Red support. The author hasn't answered yet, 
but maybe if others add to the thread, it'll keep it near the top 
and show there's interest in the idea. I suppose even if ST doesn't 
make its lexer pluggable, we could just make the built-in lexer do 
as little as possible by including no tmLanguage file for Red and 
delegating any syntax coloring/scope processing to a native library 
that's part of a Sublime Text package for the Red language. 

http://www.sublimetext.com/forum/viewtopic.php?f=2&t=9870

I think Red's USP is, "the performance of C with the elegance of 
REBOL" ... which is attractive to many folks like myself who woundln't 
otherwise venture near a "compiled" language (given the usual ease 
of use trade-off).

Nenad, could you describe a little the structure that's built up 
by the lexer? Are you intending to (at some point) allow for some 
sort of AST-like (if this is not what's generated already) structure 
to be passed back in along with some way of describing the start/end 
of a modified region in order to reduce the parsing that would need 
to be done if the lexer was being called relatively frequently when 
editing a large source string?

AdrianS: the output of the lexer is nested blocks of Red values, 
same as REBOL with its own lexer (LOAD). The AST is not stored anywhere, 
AST nodes are created and consumed on the fly during the compilation. 
So the closest thing to an AST you can get currently is the output 
of the lexer.


For the needs of a code editor, maybe you could just invoke it on 
the currently edited line (though you would need to deal with unmatched 
opening/closing delimiters). I haven't yet though how I will achieve 
it in Red IDE.

It's late to talk about merging. It's been 2 years, Red is getting 
mature very soon aspecially after the Red/System part is almost done. 
Don't make any distraction to slow it down. ... Besides, R3 is not 
open yet (where is the code?), Red and Doc are all we got now. Doc's 
got the whole plan for Red, which is a good plan. We donate for that 
plan, not for a merge plan. The latest blog article in Red-lang.org 
made me realize that we might have a complete and mature Red in one 
year.

Jerry: that sounds like a realistic deadline to reach 1.0 release, 
as long as I can keep working full time on Red in 2013. Though, Red 
should be fully usable in a couple of months, all features would 
not be there, it won't run at full speed, but it will be enough to 
be able to build almost any app.

Doc, how about Red becoming the Rebcode part of R3? IMO that would 
be a nice addition.

Cyphre's code also needed to be JITed first. I don't see a fundamental 
difference

Pekr: the difference between AOT and JIT compilation is much thiner 
than you think. Just load Red/System compiler code to your R2 app, 
pass it any source code at runtime, use the link?: no option and 
you get compiled code and related data in form of binary! values...and 
voilà! :-) The rest is same as for Cyphre's JIT, you need a way to 
call native code in memory, something that is hardly possible in 
R2, but maybe Cyphre found a hole to achieve it anyway.

something that is hardly possible in R2
 - not a problem

Ladislav: I was thinking about an internal only solution, I know 
it's possible to do it by using a tiny C code in a shared lib.   
        If you've figured out a way to do it without any external 
dependency, I would be glad to learn how you did it.

I know it's possible to do it by using a tiny C code in a shared 
lib.
 - does that look like a problem?

It breaks the "fit all in one binary" REBOL philosophy, requires 
to compile it for each platform and maintain OS-specific code...Not 
a problem per se, but IMHO a sub-optimal solution, that's why I was 
interested in a possibly purely "internal" solution, in case I would 
have missed it.

It's available but not documented as it is only used by the compiler 
internally for now. You can add it to any of the target definition 
block in %config.r for testing (or create a %custom-targets.r file 
instead). It will put the compiler in an "incremental" mode (it can 
compile incrementally as many source file as you want). Once compilation 
has finished, no file will be generated and compiler state will not 
be reset. You can then inspect the result of the compilation from 
console using:

>> probe system-dialect/compiler/job
>> probe emitter/symbols
>> probe emitter/code-buf
>> probe emitter/data-buf
>> probe system-dialect/compiler/imports


You can basically get most of these data in logs when compiling using 
-v 9 option.

People overestimate what linking means. It really is as simple as 
a pointer, the same thing that it means in a linked list.

Kaj: that's right. If you want addresses to be resolved, we need 
to add a in-memory linker. It will basically just resolve all the 
addresses. But that would not work as is with the imports that expect 
to be statically linked. So, a custom dynamic loader would be required 
(same requirements as for implementing LOAD/LIBRARY and MAKE ROUTINE!).

Not a big deal to implement though, just time-consuming.

I know what linking is; I wrote a relocating loader in Atari 8-bit 
assembler

I don't mean external symbols that are for the operating system to 
resolve, I mean the internal linking that the Red/System linker, 
presumably %red-system/linker.r, does inside a Red/System compile, 
possibly consisting of multiple sources

To put it concretely, the AVR backend needs to generate a memory 
image, without external symbols because there is no operating system, 
but with all internal addresses resolved. Can that memory image be 
constructed by appending data-buf and code-buf?

As you will notice, it uses a minimal C-based kernel and some imports 
are linked to it using syscalls.

Yes, that's a kernel for a Arduino Uno board.

I would like to get rid of the compiled C part, but never found the 
time to recode it in Red/System. It would also needs some addition 
to Red/System, like interruption handling (already planned) and other 
non-planned features, like a way to initialize RAM/SRAM from Flash 
memory (basically, it needs to copy the firmware data section from 
ROM to RAM), or initialize properly the timer clock (which should 
be doable with the hardware I/O support I've planned already).

The kernel does not use pure C, but Processing, a C/C++ hybrid.

Well, I do have something, but it's messy, buggy and incomplete. 
I can send you the AVR8 backend if you want to play with it, I don't 
want to publish it until it gets a stable and correct support for 
basic datatypes.

In Red/System, can I assign a value to a variable of different type? 
say    a: 1 a: "1"

Nope, Red/System is statically typed, you can never change the type 
of a variable. You can just do type casting to convert the variable's 
value to a compatible type.

Maybe I will take a shot at the tool. Outline would be handy and 
specs.

I guess it's not a good question. After all, I am not good at memory 
management. I should wait for Doc's Doc on Memory Management.

Jerry: here is a quick overview:


Red values are stored contiguously in series slots (128-bit cells). 
Series buffers are allocated from large chunks of memory of type 
series-frame!. Series value in slots store just a head offset and 
a pointer to a node!. The node! is another pointer to the series 
buffer. So series buffer are indirectly accessed, allowing them to 
be moved in memory (for reallocating with bigger/smaller size or 
moved by GC).

A series buffer has header, with OFFSET and TAIL pointers that define 
respectively the begin and end of series slots. The OFFSET pointer 
allow to reserve space at head of the series for optimizing insertions 
at head. Series slots size can be 1 (binary/UTF-8/Latin-1), 2 (UCS-2), 
4 (UCS-4) or 16 (value!) bytes wide.

From ~Links group: "Could Red eventually become a contender for #6? 
 How strong will support for parallel processing be, eventually, 
in Red?"


#6: yes, that is one of the goals I want to achieve with Red. For 
parallel processing, the model I have in mind is the "parallel collections" 
from Scala. This means that when you are looping over a series, Red 
should be able to parallelize the loop code over n (CPU and/or GPGPU) 
cores at the cost for the user of only a change of the loop function 
name (in Scala, they use a "par." prefix for such functions). This 
requires that the compiler do a deep static analysis of the loop 
body to determine if it can be parallelized (e.g. iterations not 
dependent on results from previous ones). Now, if you also add SIMD 
support in the equation to leverage intra-core parallelism, you get 
a good picture of what I want to achieve. ;-)


So, I think a semi-assisted parallelization/vectorization of loops 
in Red is doable. To what extent and which final efficiency, I'm 
not sure before we build some prototypes.

To what extent and which final efficiency
 may be a key factor in your billionaire-worthiness ;)

Remind me to start a space ship building company when I get to that 
point. ;-)

Just a simple comparison.

DO you need all 56 datatypes, 159 natives. Development last years: 
R3 nil Red exponential just a comparison ;)

They aren't even in the same stage of development, as R3 is much 
more mature at this point. This is not a criticism though, as Red 
was years away from existing yet when R3 was at the stage Red is 
at now. There is still much to look forward to from both projects.

Hopefully in a way that allows the compiler to not include the ones 
that aren't used, at least for closed apps. 190 seems like a lot. 
By closed apps, I don;'t mean closed source, I mean apps that don't 
execute external scripts, like iOS apps.

Probably not as much of a problem for Red as it was for Delphi though 
:)

He should write a script, one that runs both under R3 and Red. :-)

We don't have system/words yet nor file I/O...looks like a challenge. 
;-)

Is it possible to echo it? Then you can pipe it to a file.

We should have file I/O in a few weeks anyway.

The completeness and stability of them will be ignored here, which 
is not fair to R3. But like I said, It's just a simple comparison. 
The purpose is to see the progress of Red, not to discredit R3.

Also, Red will add its own features, like specific datatypes, natives, 
actions,... For example, MOLD and FORM both have a /PART refinement.

I write a book for R3 instead of R2 because R3 supports Unicode. 
Without Unicode, R2 is useless in China.

Many years ago, I found REBOL 2 and liked it a lot, but back then 
REBOL didn't support Unicode, so it was useless in China/Taiwan. 
I wrote e-mail to Carl, but I got no feedback. So I decided to start 
a magazine column in China and Taiwan to introduce REBOL. My idea 
was to make readers love REBOL and felt the same pain (of no unicode 
support). I also kind of encouraged them to write e-mail to RT on 
the Unicode issue.

After a while, Carl said (in somewhere, blog maybe) that he didn't 
know why REBOL had many Chinese users, and they need Unicode. So 
he decided to support Unicode.

I remember that Unicode in R3 is mostly thanks to you. :-) No modern 
programming language can miss full Unicode support now, so it's a 
mandatory feature to have, anyway.

For those of us still dealing mostly in pure ASCII will there be 
a 1-byte per character string alternative?

Ashley: Red string! use 1-byte per character by default, so as long 
as you stick to ASCII, it takes the same storage space as C strings. 
As soon as you insert a non ASCII character, the string will automatically 
upgrade to the appropriate format. It's transparent for the Red user. 
Moreover, you'll be able to force string encodings back down to 2-bytes 
or 1-byte when possible.

So, I should have said "as soon as you insert a non Latin1 character".

Red should provide an UTF-8 codec. For national encodings, we would 
probably proceed by offering  on-demand online codecs for the most 
used ones. That could be a shared resource with R3.

Sorry if I missed the answer to this, but are you going to be doing 
a UTF8 binary parser for Red's source the way that R3 does for its 
source? Rather than a Unicode string parser, which processes the 
source after it's been through a codec?

BTW, we already have a UTF-8 binary parser in the Red compiler.

I do prefer, actually. LOAD being mezzanine and calling a separate 
parser lets you do a lot of nice tricks. The "mezzanine" might be 
native in Red, but the separation of concerns is still a value. YMMV 
of course.

Agreed. TRANSCODE is a rather unelegant name, though :)

Agreed. The weird set of options turned out to be essential though. 
Every combination of options is used in LOAD at different points. 
We even need a /part option like that of DECOMPRESS/part, for the 
same reason.

Worse than being bad-sounding, it's a really general term being applied 
to a really specific operation. That name could have been used for 
a more general codec-based transformation process.

Should Red/System series be one-based? There are some discussion 
on it. Why not set a #pragma for it, so programmers can set it themselves?

I've become a low level programmer again, and I still want it to 
be one based

A pragma would be the worst of both worlds: not making a decision, 
like most other software out there

I agree that we should have only _one_ convention, else it will quickly 
become a nightmare when having to integrate 3rd-party code. We need 
to find some objective reasons for choosing it. 


For Red, I'm inclined to continue on the one-based convention that 
worked pretty well in R2 for many years (at least for me). I'm not 
very fond of the change in R3, introducing 0-based convention implicitly, 
it solves one problem (iterating over 0 index...I don't remember 
ever doing that), but introduces new ones (negative indexes point 
now to an IMHO, counter-intuitive position which will most probably 
lead to programming errors). For now, I prefer to stick to R2 way, 
until  we find a better solution (feel free to propose some on related 
github tickets or here). For example, we could decide to ban indexes 
<= 0 (not my favorite personal option though, but would solve simply 
the problem).


For Red/System, a 0-based convention might make more sense, but it 
would push us into the R3 issue I've mentioned above wrt indexes 
<= 0. Also, as a dialect of Red, it can use whatever convention best 
fits its purpose, but OTOH, having the same convention as Red would 
help. So, I'm really undecided for Red/System.


I think the whole issue boils down to decide about PICK behavior 
with <= 0 indexes, everything else should be able to fit in easily 
once that preliminary question is solved. It would be helpful if 
someone could put up everything related to this topic on a wiki page 
with all arguments sorted (there's a lot of them in R3 group posted 
a few weeks ago).

Off-by-one errors are everywhere in programming. Choosing between 
one-based and zero-based indexing shifts them to slightly different 
places, but they will still be there. As you said, I seldomly encounter 
a situation where there would be a strong preference for indexes 
to be zero based

Being a human myself, I don't find indices-as-ordinals ("one-based") 
particularly human friendly.