AltME groups: search

That's not as hard as it sounds. There are only 3 API models in wide 
use: UTF-16, UTF-8, and no Unicode support at all. A given port of 
Red would only have to support one of those on a given platform.

So far, my short-list of encodings to support are UTF-8 and UTF-16LE. 
UTF-32 might be needed at some point in the future, but for now, 
I'm not aware of any system that uses it?


The Unicode standard by itself is not the problem (having just one 
encoding would have helped, though). The issue lies in different 
OSes supporting different encodings, so it makes the choice for an 
internal x-platform encoding hard. It's a matter of Red internal 
trade-offs, so I need to study the possible internal resources usage 
for each one and decide which one is the more appropriate. So far, 
I was inclined to support both UTF-8 and UTF-16LE fully, but I'm 
not sure yet that's the best choice. To avoid surprizing users with 
inconsistent string operation performances, I thought to give users 
explicit control over string format, if they need such control (by 
default, Red would handle all automatically internally). For example, 
on Windows::

    s: "hello"		;-- UTF-8 literal string

    print s		;-- string converted to UCS2 for printing through win32 
    API
    write %file s	;-- string converted back to UTF-8

    set-modes s 'encoding 'UTF-16 ;-- user deciding on format
or
    s/encoding: 'UTF-16

    print length? s	;-- Length? then runs in O(1), no surprize.



Supporting ANSI as internal encoding seems useless, being able to 
just export/import it should suffice.

BTW, Brian, IIRC, OS X relies on UTF-8 internally not UTF-16.

Be sure to not forget the difference between UTF-16 (variable-length 
encoding of all of Unicode) and UCS2 (fixed-length encoding of a 
subset of Unicode). Windows, Java and .NET support UTF-16 (barring 
the occasional buggy code that assumes fixed-length encoding). R3's 
current underlying implementation is UCS2, with its character set 
limitations, but its logical model is codepoint-series.

If you support different internal string encodings on a given platform, 
be sure to not give logical access to the underlying binary data 
to Red code. The get/set-modes model is good for that kind of thing. 
If the end developer knows that the string will be grabbed from something 
that provides UTF-8 and passed along to something that takes UTF-8, 
they might be better off choosing UTF-8 as an underlying encoding. 
However, that should just be a mode - their interaction with the 
string should follow the codepoint model. If the end developer will 
be working directly with encoded data, they should be working with 
binary! values.

Ah, but length is even O(n) for BMP characters in a UTF-16 string, 
because figuring out that there are only BMP characters in there 
is an O(n) operation. To be O(1) you'd have to mark some flag in 
the string when you add the characters in there in the first place.

Don't worry, I'm only nitpicking to make things better. There's a 
lot of buggy code out there that assumes UTF-16 is UCS2, so we're 
better off making that distinction right away :)

Brian: implementing an abstraction layer over string encodings is 
a trivial task. The intended datatype model is very similar to REBOL's 
one, even more since recentl,y as I moved to an hybrid dynamic/static 
type system. I'll commit the new code in a few days, so you'll see 
how close to REBOL it can be.


I hope that this hybrid model will help us get the best of both worlds.

Got my first real Red program working: 
    Red [] print 1
outputs:
    1


It doesn't look like much, but it validates the compiler + runtime 
from end to end, and at this point, it's really cool! FYI, the native 
PRINT here triggers a FORM (action) on the passed argument. No REDUCE 
yet (not implemented).

Congratulations Doc! This is a great day.

A milestone

Wooow.. Wooow... Did I hear the crying of a baby... I think this 
is the birthday of Red. The past 1.5 years are just pregnancy.

I can tell, it's gonna be a super star :-)

Pekr: right, from now on, you can expect daily progress on Red layer. 
I will push the new code soon, I still need to complete it a bit 
and clean it up.


Jerry: the baby looks nice, we'll just have to keep it away from 
junk food and it will grow up well. ;-)

Thanks Gerard! 

Jerry: Cool! I'll make a blog entry too.

Pekr: "I wonder what the strategy for Red is going to be in regards 
to GC ..."


Currently Red is using a per-OS thread mark-sweep-compacting GC. 
I intend to make it concurrent (maybe parallel) and incremental when 
(and if) possible. The collector is not yet fully implemented. It 
is also possible to easily make it generational if we need it. When 
we'll have big Red apps, we'll profile them and determine the best 
tuning options for the GC.


There's no silver bullet for memory management & GC, you need to 
adapt it to the language specific needs (and they may vary depending 
on the type of application: client-side, server-side, realtime, ...).

I am giving a speech in SDCC conference tomorrow. http://sdcc.csdn.net/speaker.html
My topic is Dart. I hope it will be Red next year when Red is mature 
enough. :-)

In a year, we should have passed v1.0 since a while. ;-)

Just want to have your advice on a issue I will have soon with Paypal 
and donations: there's a limit of 2500€ per year of reception allowed 
for EU accounts. It is possible to extend it by providing a lot of 
administrative documents to Paypal and even then it's at their discretion. 
Has I moved out of France, I won't be able to complete this requirements 
(and Montenegro is not supported by Paypal). So suggestions are welcome 
on how to keep receiving donations in a easy way for donators?

Personally, I plan on setting up a Bitcoin donation option

Do you know a safe trading place where bitcoins can be converted 
to real money? The ones I used to know have all closed after being 
hacked...

They're still hacked now and then, but there are ever more. Here's 
a market share list:

BitInstant is doing interesting things, giving out Bitcoins in stores 
in the US, Russia and Brazil, and claiming to introduce a Bitcoin 
Master card in a few weeks:

Well who is issuing them, with what authority and what fundamental 
value? I cannot see this, whereas the state has the authority of 
the land where we live and foods are produced to base money value. 
(Second life 'money' was a hype.)  But that is a discussion for outside 
this Red message group.

A recent story about PayPal, with advice: http://elliotjaystocks.com/blog/good-riddance-paypal/
The reply from the PayPal CEO: http://ndy.gd/JJgB

I do not bother to obtain bitcoins. Vague draining of computational 
powers of my tiny computer to compute some hashes(????) to unknown 
benefit to get by chance(???) ome 50 bitcoins of a limited(who controles 
this, I have serious doubts here too) supply of them. I am a real 
bitcoin sceptic, sorry. But again I think the Red group is not the 
place to discuss this matter, not that I have much more to say about 
this.

About the paypal issue I have the same doubts about paypal as well, 
for this 2500 Euro limit is (I do not get any gifts) really restricting, 
when going to a bank that cooperates with the paypal organisation, 
you can set up a business account where you are no longer bound by 
this limit but then 'you' need to become a business.


Yes you can just exchange your currency, but there was NO return 
prestation delivered for the original bitcoin.

Doc, a bit late congratulations for your first Red program! (I was 
offline for a week)

Doc, Services like Plimus or Fastspring probably have option to get 
you wire transfer or at least check (or some send you prepayed visa). 
Not eactly meant for your case, but if you don't find anything else. 
You could (I hope) make a "products" with them and then. I used them 
for similar transactions when nothing else could come into play. 
Paypal wasn' working here either and even now it's very clunky to 
retrieve the $$ from it.

I will write about Red Unicode support in a blog article this week.

BTW, I might also post soon a description of a redbin format (similar 
concept as Carl's rebin). The overhead and footprint of the Red boot 
script (loading all the words and their values) is not negligeable, 
so I may switch to redbin option sooner than later.

Rebin!!! I mean Redbin!!! Great! I wanted it when I was trying to 
implement a simple DBMS using REBOL.

I will push the current Red compiler & runtime code base tomorrow, 
still some more code cleaning to do. It will contain the Red boot 
script with all base definitions (currently actions, ops, a few natives 
and a few char! values) and a compiler front-end (similar to %rsc.r).


Don't expect too much, only MAKE has been fully implemented and FORM 
on integer! values only. PRINT is the only native currently.


This is not the first Red alpha, but it's a working base we can implement 
the alpha on (basically implementing actions and natives).

Can Red/System code be included yet, so the new platform can be sort 
of tested as a shell for Red/System programs?

Kaj: not yet, there are several ways to integrate Red/System in a 
Red program:


- inlined in Red code: it will be done using a #system compiler directive 
followed by a Red/System block of code. You should be able to use 
it pretty much anywhere. Trivial to implement.


- executed at runtime: using a DO/SYSTEM, but we need the Red/System 
compiler written in Red to be implemented first for that.


- through Red/System exposed functions to Red: this needs a native! 
definition in Red. It needs some intermediary code for supporting 
automatic marshalling/unmarshalling arguments passed between Red 
and Red/System. But it could be added quickly. Red's routine! type 
will use the same approach for mapping imports.

- by directly coding the native! in Red using something like:

    foo: make native! [[<spec>] [<Red/System body>]]

Lot of fun in perspective! ;-)

I will add math ops and basic block actions later today, so we can 
play with it a little. ;-)

@BrianH: as you can see from the source code, R3 actions are a strong 
inspiration. ;-)


I also plan to add some actions from Topaz, but Topaz is still higher 
level than Red currently.

Looks like you made some modifications in your existing code before 
updating and you ended up with merging conflicts (most probably you 
have merged two branches).


You should first do a checkout to the right branch, then issue a 
pull.

Two notes:


- when in root directory, where %red.r resides, is there a need to 
specify it in compilation process? do/args %red.r - or do we expect 
more compilers here?


- executable is spilled out into the root too. I would probably use 
the way compatible to red/system, and create /builds subdir?


Or is all above intentional to allow to compile/link from working 
dirs?

*Important*: Red will be distributed as a binary to end users and 
the effect of:

    red script.red
    

will be to compile and run it from memory directly. So the -o option 
will become mandatory in the future for generating an executable 
without running it. During the bootstrapping stage, it is complex 
to support that feature, but it will be implementd as soon as possible.

Could you please elaborate? Is red executable going to be kind of 
launcher for our scripts? Hence it compiles them before running it? 
Most probably not a solution for CGI, as the compilation stage will 
happen each time?

The main drawback right now to support direct Red script execution 
is REBOL inability to execute native code from memory. The workaround 
would be to provide a DLL to pass the memory pointer and call the 
Red code, but as I aim at a single Red binary, this option wouldn't 
be very satisfying.

I got math ops for integers working, but I think it's time we get 
a better macro system.

Kaj: what do you think of that? (for Red/System)

    #define MAX(a b) [either a > b [a][b]]

    print MAX(78 12)

outputs: 78

Here's a more complex example:
    
    #define MIN(a b) [ either a < b [a][b] ]
    #define MAX(a b) [ either a > b [ MIN(p/a p/b) ][p/b] ]

    p: "Hello"
    print MAX(2 1)

outputs: H

Nenad, thank you for the explanation. Good to know even the simple 
test works both ways, and I am a programmer not a tester ;)

No, I haven't yet tested the Red codebase on Linux, but I will in 
a few minutes.

Doc, did you know you have a possible theme song? It goes back to 
when I was a young lad but it still rocks:  "Red" by Sammy Hagar. 
http://www.youtube.com/watch?v=sfdy7tC-Us0&feature=fvwrel

If your `git clone`succeeded, `cd`inside the repo, and do a `git 
checkout origin/v0.3.0`

Math operations now supported in Red (integer! values only for now). 
For example:

    a: 123
    a: a + 2
    print a * 6 - a

will correctly output 625.

I like the new macro system, too. I've wanted that for a long time 
(in silence :-)

Doc - looks like you are having fun finally coming with Red :-), 
though it is clear it is a hard work :-)

I would very much like to have a console, but it is not doable at 
this point (unless someone has a bright idea?). We'll add one as 
soon as possible.

Pekr: I had fun too working on low-level with Red/System, but it 
takes a lot of energy while working at Red level is a lot more relaxing. 
;-)


But this is short holidays, once Red gets mature enough (I bet on 
a couple of months), we'll start working on Red/System v2 (rewritten 
in Red, with a new compiler & linker architecture). This will give 
up the opportunity to reboot Red/System, fix a few design decisions 
if required, extend it, and get a clean and lean new code base in 
Red. I plan to write some architectural specs about the target compiler 
before starting, so all contributors will be able provide me with 
feedback before we implement it. Trust me, you _will_ like the final 
compiler! ;-)

Ashley: that point is closer than you might think. Even at bootstrapping 
stage, once Red get enough Core features, you and any other reboler 
will be able to add new libraries or port existing one from REBOL 
(should be trivial in most cases). That's not limited to mezz code 
though, you'll be able to add bindings to pretty much anything, protocols, 
provide Red plugins for 3rd-party apps (browsers e.g.), even make 
a View clone if you want (that one could even be a community project). 
:-)

Kaj: I've answered in the tracker, should be just a couple of definitions 
missing (they are missing for Darwin too).

I will add the missing definitions for MacOSX platform in a few minutes.

For Git command-line users, a useful cheat sheet: 


http://jan-krueger.net/wordpress/wp-content/uploads/2007/09/git-cheat-sheet.pdf

Doc, you mentioned that the problem with Paypal is that they'll limit 
your total donation amount per year ... for now that works.  To avoid 
large wire fees for those who want to donate, perhaps you could set 
up a US bank account, and then wire money to your account in your 
country, after funds have collected.  If you need help, please let 
me know:  reverse "g r o  tod  z r a t i u g  ta  codmorf"

If you communicate clearly your financial needs, I'll do everything 
possible to help keep you funded.  An occassional update to your 
funding requirements would be appreciated.  I would appreciate being 
added to a regular email update, in case I can't keep up on AltMe. 
 I wonder if others who have donated on Paypal may be interested 
in an occassional update, so they can help when able/motivated.  
Your current progress is very motivating.  I hope that others who 
are watching feel the same way.

Has anyone here worked with OAuth and/or the etzy.com API in REBOL? 
 I'm currently looking through Christopher Ross-Gill's OAuth implementation 
for Twitter, and interested in writing a REBOL app using the etzy.com 
API.  I understand most of it, but don't get the "oauth_signature" 
spec.  Don't have much time for this, so would like to offer some 
$$$ for anyone able to instruct and or provide help implementing 
the etsy API.  Email reverse "g r o  tod  z r a t i u g  ta  codmorf"

Kaj: ok, I will have a deeper look in mmap syscall later, I'm currently 
debugging the OS X version.

Please remember we want to contribute but without a reasonable clue 
this is hard to do. (Besides the closed-source issue this is what 
Carl ran into when expecting others to contribute). Me and Github 
wil never become friends for example, I managed to get some source 
long ago but no clue how to update to the newest sources and github 
has informed me they do not wish to support their macosX tool for 
Leopard that I am running, nor remove the useless (?) check on OS 
number 10.5, and I am NOT updating my system and learning all the 
commandlines to upgrade etc is too much effort, (maybe someone can 
build a REBOL interface).

Kaj: I can reproduce a similar memory allocation error on OS X too, 
so I hope that fixing it there will fix it on Syllable too.

Syscalls: given how much a programming language has to call OS services, 
I don't see why we shouldn't try to avoid an unnecessary layer. But 
I'm pragmatic, if we get more troubles than benefits, we'll adopt 
a more classical approach.

Also, syscalls usage gives you a smaller executable (smaller import 
section).

NickA - it's impossible to setup a USA bank account without being 
there in person to do so.  I've tried ... you have to present your 
credentials in person.

Yes, it's a EU limitation.

No, I think Kaj produced a Curl binding for https

I did write a ftp, smtp and hylafax protocol for R3

Red should have a more "modern" set of schemes than the one provided 
by R2, though.

And imap, and a JDBC bridge

But they were just first passes waiting for Carl to have a look and 
give advice, which never came, so they never went past that stage

So, yes, there are a few of us waiting to contribute when Red reaches 
that point for us

I will provide the basic ones for Red: TCP, UDP, DNS, HTTP(S), SSL, 
SSH/SFTP, SMTP. Also, I would like to have a few more for remote 
or local storage builtin: Dropbox, MySQL, Postgresql. 


You are welcome to contribute other ones to that list (IMAP, SMTP, 
MongoDB, CouchDB would be nice additions).

What are your plans for a code repository?

I will probably host a server that will provide packaged and selected 
libraries for Red (with builtin support in Red for getting them). 
For sharing code, my plan was to built some support for that right 
into the IDE.

Doc - what are your plan re Uniserve? Will basic engine be present 
directly in Red, or a separate product?

Glacier allows files to be retrieved over a few hours

Uniserve: we'll see how it fits in the Red I/O model (not written 
in stone yet), we'll probably use a newer approach with sequential 
code instead of having to write callbacks.

So far, the plan was to use Actor abstraction, but I might decide 
to use a more lightweight approach (something like goroutines). The 
basic idea for the low layer is to have cheap concurrent threads 
of execution that are dispatched over a limited number of OS threads. 
On the upper layer, they might appear as task! or actor! values, 
that you could create in various ways (do/task, read/write on I/O, 
...).

I also have a few ideas about specializing CPU cores in order to 
avoid costly context switching, but I need to test that on paper 
first.

Gregg: thanks for that. I'm all ears if someone has a solid proposition 
for that. I'm even ready to relocate myself in a foreign country, 
if that can help. :-)

I've sent money to europe by bank wire, and it's very expensive. 
 Best if you can setup a US bank account when you're next visiting 
the USA ( if that is possible for non residents ), or an E*trade 
account.

hello is still out of memory on Syllable Server, at a slightly changed 
address

Please try again, pushed a fix a few seconds ago.

I'll tell Bas to schedule a Red presentation at Software Freedom 
Day coming week

Thanks Gregg, a lot more to come in the next weeks!

is there a list of any-function!s implemented?

Also, if someone is willing to make such a list right now and maintain 
it updated, that would be nice.

You just need to screen through %runtime/actions.reds and %runtime/natives.reds 
for changes. You could even make a short script to notify yourself 
when those files are modified using github's API. ;-)

The website needs an introduction/instruction section and a downloadlink 
to a zipped version of Red souces, so people can skip the github 
stuff.

+1 Arnold, although Github does provide a zip download link.

So - just link red-lang.org to the zip archive? Doesn't it have a 
static/non-changing link?

Kaj: I might make macros context-sensitive, but I don't see what 
more I can do. The point of #system is to write Red/System code within 
the 'red context, so naming should be done carefully to avoid conflicts. 
But with a few context wrappers, you should be able to solve that, 
no?

does it work like a preprocessor or what? Any example, of how is 
Red/System code inlined in the Red?

why not do it like a Rebcode? make red-system ...., or make native! 
.... etc.?

# char should not be used for anything else but - preprocessor. And 
if it works like a preprocessor, we should stop talking about the 
ability to use inlined Red/System code anywhere in the Red source 
code ... :-)

Arnold: Red source zip links are on github since the beginning. Maybe 
you can write such presentation/instruction section, submit it in 
a gist, put the link here and on the Red ML, discuss with others, 
and when it's ready send it to me so I can review and publish it. 
I need to advance on Red construction rapidly now, so the website 
is currently lower priority to me.

A what? A gist??

Guys, remember the project current version number: v0.2.6, this is 
far from a v1.0.0, don't expect the website to look like if the product 
was finished. Maybe I should add an old-fashion "Under Construction" 
yellow gif banner on top of it? ;-)