AltME groups: search

The most space efficient encoding I can come up with would be something 
like ~a-1 for 'a and ~A-2 for 'A. That would be cheap to evaluate 
for strict-equal? but expensive for equal?

A faster encoding would be to reserve a part of the integer identifier 
for the alias number, for example one byte. That would reduce the 
number of different symbols to 2^24 and the maximum number of aliases 
for one symbol to 256. That would only allow a word up to 8 characters 
to have all its aliases, but it would be cheap to evaluate for both 
strict-equal? and equal?

In REBOL, 'a and 'A are aliases of the same symbol. Red/System converts 
them to their integer identifier, right?


Symbols have two representations in Red compiler, one is at runtime 
(like in REBOL), the other is a compile-time, in the form of Red/System 
variables. In a very early version of the compiler, I was using integers 
(indexes in symbol table) instead of variables, but quickly realizef 
that it was obfuscating the generated Red/System code a lot, making 
it difficult to debug. Also, the integer approach had an additional 
runtime cost at it required to make an array access in order to retrieve 
the symbol value.


Currently, the Red/System ~<name> variables directly point to a word! 
value version, instead of a symbol! for simplicity and efficiency.

I have implemented a compile-time aliasing system for same words 
but different casing. It works fine so far and is cheap compared 
to other options (it requires a conversion table (symbol->alias) 
to be maintained during the compilation).

Aliases are already implemented in the symbol! type. Basically a 
word! relies on a symbol ID, which is an entry in the symbol table. 
Each entries in this table is a symbol! value that references the 
internal Red string! value and a possible alias ID (which is just 
another symbol ID).


Now, I just need to add alias handling in the equal? and strict-equal? 
natives when applied on words to make it work correctly.

Thanks Doc. This is good information to put in a doc somewhere, even 
if just as a reminder to formally doc it later.

This is one area where copying R3 as it is now would be a bad idea 
though. See http://issue.cc/r3/1834for details.

Brian: wrt '=, it's a typo, it should be ==.

I haven't implemented EQUIV? yet, I'll look at it when we'll have 
a complete IEEE-754 support (we are missing INFs and NaN handling 
in Red/System).

Anyway, having a prefix rather than a suffix is a possible option.

How do one know a rebol function is supported or not ?

I tried a simple FOR loop, but no result and the compiler is quiet.

How do one know a rebol function is supported or not ?

 Currently, only by looking in the source code. The compiler is lack 
 a lot of checks, so you need to get your Red code right for now.

The source code should be easily parse-able, so the list of functions, 
native, actions, ops could be extracted and pretty-printed as a web 
page. IIRC, someone tried to make such script but I didn't see any 
result yet.

Yes I wanted to give it a try for the doc scripts. But parse is not 
my expertise, and at the moment I am short in time as I can make 
extra hours at work. So everybody step in please and publish your 
baby-doc-scripts so we can all contribute little bits.

About the case-sensitivity,

What about to convert all the words into lowercase in compile time? 
Does it lead some unicode problems? What if a word is in Chinese, 
is there lower/upper cases in Chinese?

What about to convert all the words into lowercase in compile time?


Words values are not "compilable", they are data (words used as variables 
can be "compiled" to some extents). Converting all words into lowercase 
during compilation (including JIT-compilation for words constructed 
at runtime) would make you loose the ability to distinguish lower/upper-cased 
letters, leading to big issues and pitfalls in the language. For 
example: (form 'A) = "a" (beause 'A would get converted to 'a). Not 
an option.

Yes, that's what I meant. I phrased it that way because there was 
a big discussion where people were requesting that an option be added 
to objects to have them be case-sensitive, to distinguish based on 
case when mapping words to value slots, rather than the case-preserving 
default. We had to reject that proposal because there was no way 
to specify that option in the make object! syntax. The only way to 
do that in Rebol is to have a separate object-like datatype that 
has case-sensitive word mapping. The same proposal was made for maps, 
with the same results: a case-sensitive alternate type would be required. 
For both of those types, SELECT vs. SELECT/case could have some meaningful 
distinction, though we didn't get far enough for that to be an issue 
yet.

This is all old Rebol discussions, but Red would have similar issues 
with proposing such options because it was a matter of syntax.

Back to an older topic, hex syntax. If you had 16#abcdabcd translate 
to an integer!, it wouldn't have to be considered to be a conflict 
with #abcdabcd being an issue! value. It's just like {abcdabcd}, 
#{abcdabcd} and #abcdabcd are different now. There would be no reason 
to keep the hex syntax once the value is loaded, it could just be 
a regular integer. You could even keep the issue! type as a word 
type with some extra series-like operations supported, the way tuple! 
supports series-like operations without being a series.

Hex: your proposition is acceptable, but it makes hex literals writing 
still a bit more verbose than needed. We should be able to come up 
with a better solution that leads to just one additional character 
in order to write and identify hex literals (hence my # suffix proposition, 
with a base-16 default value).

(For comparison again, sorry) In R3, objects are in many ways like 
the tables in Lua, used for data purposes as well as for contexts, 
underlying several other datatypes or operations as well. Most contexts 
are declared using these other datatypes or functions that wrap objects; 
raw objects are more often used as data structures than as contexts. 
It might make sense to support case-sensitive objects as data structures. 
Nonetheless, I wasn't the one making the suggestion, and I'd have 
to do a bit of research to dig up who was requesting this.

I just had a very short look on R3 code yesterday's night, what striked 
me the most at first look is how both R3 and Red are struggling (in 
different ways) to avoid writing C code as much as possible. :-)

R3's source also avoids direct C code by doing a lot of the work 
in macros. With the macros, the C source of a native doesn't look 
that much different than mezzanine code.

Pekr, something like "The end of World and Red"?(no offense,  just 
a  joke Doc and John) :-)

I yesterday donated just 15EUR (just for a pizza, and didn't intend 
to mention it here), but if Nick will match it will be 2 pizzas :P

Janko, every bit counts. I've donated this month, but I will match 
December donations in a January donation, up to USD$200. So, donations 
in December get double matching up to USD$200, if Nick matches as 
well.

Red 0.3.1 released: http://www.red-lang.org/2012/12/red-v031-functions-support-added.html

- function support
- path get/set notation support
- refinement support for native functions
- expressions in parentheses compilation

- new datatypes: function!, paren!, path!, lit-path!, get-path!, 
set-path!

- new actions and natives: find, select, copy, reflect, type?, halt,...

- extended mold, form and comparison operators to all new and existing 
datatypes
- many new mezzanines functions

- modulo and remainder operators on floats implemented for ARM backend
- Quick-Test testing framework ported to Red
- a truckload of new unit tests
- many bugfixes
- about 200 new commits to the Github repository
- updated Red/System formal description document

Is http://www.red-lang.org/p/roadmap.htmlcurrent? You've been making 
so much progress, I'm guessing it's a little out of date.

To avoid having to use to-logic when using refinements to pick a 
value in a series. For example:

In REBOL:
    foo: func [/only][pick [1 2] only]
    foo

    ** Script Error: pick expected index argument of type: number logic 
    pair
    ** Near: pick [1 2] ref

In Red: it should return 2.

Note that PICK taking a logic! value is not yet implemented, it will 
be added in the next days.

This is a code pattern I use often, but always find it annoying to 
have to add a to-logic call each time in front of refinements.

Gregg: It will need to be updated a bit, right

Just specifiying a return type adds 68 bytes to a function. Why is 
that?

It's the Fibonacci function in the C library binding. I checked in 
a Red version. If you add return: to that it becomes 68 bytes bigger

Also, you won't find the source code of block literals in text format 
if you scan the binary, because they are stored as code and not data. 
That is the only way currently they can be stored in compiled binaries. 
Storing them as text would need a way to load them and then compile 
them at runtime (it will be possible in the future, but not right 
now).


Anyway, the probably best way to store all those series literals 
is to allow the use of a redbin format. We will have that too at 
some point.

Red/System Question: In Struct A, there is a menber which points 
to struct B. In Struct B, there is a member which is a function pointer, 
whose parameter is a pointer to Struct A. ... How can I do this in 
Red/System. This is not a fiction case. I am studying SQLite surce 
code, struct sqlite3_file and struct sqlite3_io_methods are exactly 
like this.

This any help?

Red/System []

a!: alias struct! [
  a1  [integer!]

  a2  [integer!]            ;; this is used to hold address of struct 
  b
  a3  [integer!]
]    

f!: alias function! [s [a!]]
f: func [s [a!]] [print [s/a1 lf]]

b!: alias struct! [
  b1 [f!]
]
  
a: declare a! 
b: declare b!


a/a1: 5
a/a2: as integer! b 
a/a3: 0
b/b1: as f! :f

foo: as f! b/b1           
foo a

Jerry: if I understand correctly, what you want to achieve is:

    Red/System [ ]

    structA!: alias struct! [
        p [structB!]
    ]

    structB!: alias struct! [
        fun [function! [a [structA!]]]
    ]


As is, it won't compile in Red/System as structB! is not yet defined 
when defining structA! (and the compile is making a single pass only, 
so can't look ahead). 


The workaround is to define `p` as integer! and use type casting 
when accessing it.


Also note that if you reverse the order of definitions and start 
by defining structB!, you could use the same workaround for `a` argument 
type (define it as integer!, then apply type casting when appropriate).

I have added a new function type today: routine!. It allows to write 
a Red/System function in a Red program. The compiler will marshal 
(or type-cast) the arguments back and forth automatically.

Here is the Fibonacci example rewritten as a routine:

Red [ ]

fibonacci: routine [
    n          [integer!]
    return: [integer!]
][
    either n < 2 [
        n
    ][
        (fibonacci n - 1) + (fibonacci n - 2)
    ]
]


The function body is Red/System code, so it will run at full Red/System 
speed.


Integer! and logic! values are converted automatically, other Red 
datatypes are passed boxed but type-casted as Red/System counterparts 
(as defined in the Red runtime). Hint: floats will be converted automatically 
too.

So, passing and processing a block! series would look like this:

Red [ ]

add-one: routine [
    blk       [block!]
    return: [block!]
    /local value tail int
][
    value: HEAD(blk)
    tail: TAIL(blk)
	
    while [value < tail][
        if TYPE(value) = TYPE_INTEGER [
                int: as red-integer! value
                int/value: int/value + 1
        ]
        value: value + 1
    ]
    RETURN(blk)
]


I haven't yet released the code, it needs a bit more work, it should 
be ready by tomorrow.


The purpose of routine! datatype is to provide access to ultra-fast 
and low-level code for Red program in a simple way. The design is 
not yet fully set in stone, so suggestions and comments are welcome.

That's cool, it's something like rebcode.

How about something like having a block of Red/System code inside 
Red code?

System [...some R/S code...]

There are no arrays in this bootstrap version of Red/System. So you 
have to manage this with a pointer:

Red/System []

s: declare struct! [
  c  [byte!]
  i1 [integer!]
  i2 [integer!]
  i3 [integer!]
]

; intialise the structure
s/c: #"a"
s/i1: 1
s/i2: 2
s/i3: 3

;create a pointer and point it at the first integer
i: as integer! s

i: i + 4                          ;; set i to address of first integer
ip: as pointer! [integer!] i

;; add the integers
sum: 0
count: 3
until [
  sum: sum + ip/value
  ip: ip + 1
  count: count - 1
  count = 0
]

print ["The sum of the integers is " sum lf]

There is a probable issue with value alignment as explained in http://static.red-lang.org/red-system-specs.html#section-4.7

I'm not sure how to handle "packed" stucts in Red/System.

That's cool, it's something like rebcode.

It has a broader range of usage and it's faster than rebcode.


How about something like having a block of Red/System code inside 
Red code?

System [...some R/S code...]


This is already implemented since a while in form of a Red compiler 
directive: #system [...]

To have a tightly packed semi-array of dynamcic length, use allocate 
and free

Doc, thanks very much for routine!. It's brilliant and exactly what 
I need to make the bindings available from Red. However, #define 
not being context aware is still a blocker

There's not much that can be done currently for #define, you can 
use #enum instead to workaround it in such cases. 


Anyway, we will need to rethink the preprocessor in Red/System v2, 
and if possible, remove it. It has been added at the beginning of 
Red/System because Red was not yet there to provide a higher-level 
macro system. Now that Red is alive, we could start thinking of a 
better alternative to the current Red/System preprocessor.

Doc, I'm very glad to see you're still working hard on Red!  I was 
concerned that the psychological effect of everyone paying attention 
to R3 source, plus your own curiousity about it, might derail you 
for a while.  I'm still eagerly awaiting every advance you make!

sounds cool :-) Lately I was wondering about the possible benefits 
of Red and R3. Difficult to judge, but could there be any overlapping 
ground, where twose two projects could cooperate? e.g your IDE for 
R3 to, simply a language would be a plugin, or - both projects want 
to address Android - could one bridge to JNI be used for both?

Yes, go Doc! I wish I was better at low-level programming so I could 
help a little. If there would be any examples of simple bindings 
or base of TCP that we could extend to different protocols I would 
try to participate a little.

Kaj, just a remark about Red and bindings: there is still  an additional 
feature to come that will allow to import both Red/System and external 
libs functions directly in Red in a declarative way. It will use 
almost the same syntax as #import in Red/System but will convert 
the datatypes automatically (in the same way R2 does with routines 
and structs). A struct! datatype would then be added also to Red. 
The routine! datatype primary purpose is provide a way to write ultra-fast 
code and enable system programming, even if it can be also used to 
wrap Red/System bindings. I'm still unsure which approach would work 
the best for building bindings to C libraries. I guess you'll be 
the first to find out. ;-)

Red/System Question: In a function, a local variable v1 is declared 
as struct! [ v2 [integer!]  ]. Once the function is called, v1 is 
in stack, v2 is in heap. When the function call is over, v1 is gone, 
but v2 is still in heap, right?

Doc, thanks for the heads-up. I'm also unsure which route to go, 
but I want to have the option to write Red/System only programs, 
so I guess I will usually build a Red binding on a Red/System binding

On the other hand, if a binding is mostly the #import section, the 
future format sounds more concise than writing all wrappers as routines

From Announce: Great Kaj! The optimizer should be able to do a pretty 
good job on simple cases like Fibonacci function. Still, you'll have 
the Red stack management overhead, which is currently unavoidable, 
but in the future, we might found ways to optimize it too. Actually, 
I have two options to reduce stack overhead: stack multiple openings/closing 
compression and inlining stack calls, but we'll see that in Red v2.

Red/System Questions:


I am translating a piece of code from C to Red/System, I don't know 
how to translate the following C code:


success = OSAtomicCompareAndSwapPtrBarrier(NULL, newzone,  (void 
* volatile *)&_zone_); // C

my problems are: 

1. In Red/Sys, _sqliteZone_ is a struct! (which is a pointer in Red/Sys), 
so I cannot use :_zone_ to get it's address.
2. "(void * volatile *)" in C => "as byte-ptr!" in Red/Sys ??

Thanks!

1) Is _zone_ a struct or a struct pointer? If it is a struct, then 
in Red/System, you just pass the struct! variable.

2) Correct, use "as byte-ptr!" in such case.

1. _zone_ is a struct pointer. I need to pass a pointer to a struct 
pointer as the 3rd parameter, that's why I try to use get-word here.

For 1., you can achieve it this way:

    s: declare struct! [n [integer!]]
    p: declare int-ptr!
    p/value: as-integer s

then `p` becomes a pointer on `s`.


I might extend get-word syntax to struct! and pointer! too, someone 
just needs to add a ticket to Github to remind me of that.

Ok, I see what you mean. Like in C, you can already write "thread-safe" 
code in Red/System by using only variables on stack. If you need 
concurrent access to global variables, usually, the OS are already 
providing some API for that (locks, semaphores, mutex,...). I haven't 
chosen yet how it will be implemented at Red/System level, there 
are different options. For example, it could be handled by the language 
directly using a similar construct as in Java:

    synchronize [ x: x + 1 ]


In my early design notes, I have researched only how to handle concurrency 
at Red level, I've left the underlying Red/System part as an "implementation 
detail". I plan to start working on it after I/O will be implemented.

I'm thinking about  the case where you want to implement operating 
system level code yourself. For example, I have been fixing many 
bugs in Syllable's PThreads implementation, but I would like to replace 
it with a Red/System implementation. PThreads adds quite a few constructs 
that Syllable's kernel primitives don't implement as such, so to 
add them you need to write extra concurrency code

For example, there are spinlock-like constructions in Syllable's 
PThreads implementation that can often prevent having to call a kernel 
mutex. This makes them much more efficient, but you need atomic operations 
on access flags and counters to implement the user-space spinlocks 
and mutex shells

Modern CPU usually provide a way to make such atomic operations (e.g. 
CAS-type  instructions), Red/System might provide access to them 
if it is required.

From a quick googling, I've found:


- x86: CMPXCHG (compare and exchange) is supported from 486. LOCK 
instruction seems to be available from PentiumPro only.


- ARM: Starting with the ARMv6 architecture ARM has the LDREX/STREX 
instructions that can be used to implement an atomic compare-exchange 
operation.

So it would bump Red's requirements a little, but I guess it's worth 
it

So how can I break/continue a loop now?

BREAK will probably be implemented before 2.0 (not sure for CONTINUE). 
So, you can't break from a loop currently. 


As a workaround, you can EXIT/RETURN instead. So if you write your 
loop in a separate function, you'll get similar effect as with BREAK.

I'm a bit late for the Xmas release, sorry...still fixing bugs...

I thought we already had a Xmas release :-)

Actually, it is not really a new release, as I did too many changes 
and it would need a week at least to stabilize and fully complete. 
But it will be a significant revision. ;-)

I've stressed the current Red runtime source code a bit so it keeps 
"tilting" as I try to make it work. :-)

Doc, will you spend some time making Red/Sys support loop BREAK/CONTINUE 
in the near future? or at least GOTO. Thanks. Programming without 
them is like drinking soup without a spoon ...

BREAK is on my todo list. I might add CONTINUE too. GOTO could be 
useful for implementing fast FSM, but having some specific feature 
for FSM support would probably be a cleaner option than GOTO.


I personally very rarely need BREAK/CONTINUE, their usage depends 
on your programming style and the way you implement algorithms.

BREAK in CASE/SWITCH statement is IMHO a very bad design choice leading 
to countless bugs. I'm glad Red and REBOL are doing it the right 
way.

Implementing that feature revealed a few deep bugs in the current 
Red runtime code, I hope to be able to finish fixing them by tonight.

Console support! I'm waiting for it impatiently!!

BREAK/CONTINUE: I had to use those a few times in REBOL, it would 
be nice to have CONTINUE in Red too.

Awesome Doc!  Didn't expect to see a console so early.

It's a full REPL, so it's an interpreter + console. I was thinking 
about adding an interpreter since this summer in order to fill the 
gaps until the JIT-compiler will be there. Even when Red will gain 
JIT-compilation ability, for dynamic code with no loops, interpreting 
the code will be faster. Also, on some platforms, JIT-compilation 
is not allowed, so the interpreter will be handy to workaround those 
limitations.

Now, you have a better picture about the "full-stack" approach of 
Red language. :-)

The console is currently just a Red script, once we get devices and 
ports, it will be replaced by a proper console:// port.

It still misses a lot of features, but I think it could be fairly 
complete (on par with Red compiler) in early January.

I guess all this means we have to think a bit more about how to benchmark 
things rather than simply profile code pieces directly in the console?

Right, if you bench from the console, currently, you're just measuring 
the speed of the interpreter. When the JIT will be there, loops from 
the console might get JIT-compiled, so it will be even less relevant 
to bench from the console. My plan since the beginning is to add 
a profiler to Red, so you'll be able to make precise comparisons 
whatever way the code is run.

Interpreter: that's the only way to make a REPL currently until JIT-compiler 
is there. I have thought about an alternative way using REBOL for 
the REPL and compiling the code in memory, then running it, but it 
would be a clunky solution (among other issues, would require an 
external DLL for being able to run native code from REBOL).

The interpreter will unlock a lot of features that would have needed 
to wait for the JIT-compiler, like DO/REDUCE/COMPOSE natives. These 
natives can be statically compiled as long as you pass a literal 
block to them, but anything else needs dynamic execution. I was refraining 
from adding them to Red compiler so far, because  they could work 
only partially. Now, I can implement them fully, thanks to the interpreter 
acting as a fallback.

Wow Doc! Congratulations on getting to this stage. What a great way 
to end the year.

Right Doc, thank you so much! I felt shivers going through my spine. 
I admit I also have Natasja on the couch, a cat on my lap, pears 
on the stove and John Lennon's Xmas song on the radio :-)

Kaj: that's a good way to enjoy Xmas!

well, still not clear, what does interpreter do. So it takes a Red 
code, compiles it, and runs it? That would be JIT, no?

Kaj, ssems like a relaxed evening :-)

Exactly, but JIT can't be implemented until the self-hosted stage 
is reached, so in the meantime, Red will use that interpreter as 
a fallback.

Also, for the pure console usage, the interpreter is probably a better 
option than the JIT-compiler.

Yeah, it was weird that it was called R# in 2005 and wasn't a .NET 
language (iirc).

Actually, it was a private joke at Softinnov to call it like that, 
it was meant to be a project code name, not the language final name.

Gregg: I wish too...but nobody contributed a better console for R3 
after hostkit release, I doubt it will be better for Red.

I think R3 being closed was a big part of that. I hope anyway.

And some smart person might be able to build a console that could 
be leveraged for both, or at least parts of it.

actually, r2 console was much better. r3 one was supposed to run 
as a real coneole, eg via com port, etc, but failed to deliver. Carl 
really tried to address it, I tried to help with googling some references, 
but Windows is weird in that regard ...

We were more focused on lower-level things. The console wasn't much 
of a priority, especially since we were intending to implement it 
in the R3-GUI, and that wasn't there yet.

brianh - try to locate related blog. carl asked for help - if you 
link view, he was not able to let the conole work in a real console 
way, or something lik that :-) Well, off topic here ...

For Red, if someone doesn't jump in and do it, Doc will have to. 
Then he has to decide how important it is to helping Red succeed. 
I think it is, and will also help set a foundation for a Red IDE. 
For that, I don't know if it will even help Doc if we write up things 
we like, or don't about various consoles. I think he has a good feel 
for what is needed.