AltME groups: search

No, I will implement it when I'll need it, and I have a lot of other 
stuff to code for Android support before that.

I was hoping a little more could be done, but I'll have to postpone 
a lot of work

I told you I will have a look at it once the shared libs will be 
done, just wait a few days more. If it's critical to you, you might 
want to contribute the required conversion routines?

I could, but I know very little of Unicode, so there would be a lot 
of overhead in getting up to speed

I have no idea how long it will take you to finish the shared libraries. 
It has been a backburner project for a long time

Not very long, I just kept it postponed since almost a year now, 
and it's getting on my way for Android support since a while, so 
I've scheduled it since a few weeks to get it done just after the 
interpreter is finished (Exit/Return support).

The only "data out" support we need for now for building Red is the 
stdout support, and we have it since a while.

Also, there might be a cheap way to achieve the conversion in the 
meantime using wsprintf() or similar function.

Hmm, it might not be enough, so you might want to have a look and 
maybe wrap libiconv:
http://www.gnu.org/software/libiconv/

From a routine, if str is a red-string! pointer, this is the dispatch 
code you would need to use:

s: GET_BUFFER(str)
switch GET_UNIT(str) [
	Latin-1 [...conversion code...]
	UCS-2 [...conversion code...]
	UCS-4 [...conversion code...]
]

Beginning of internal string buffer is given by:

	string/rs-head str	(returns a byte-ptr!)

Sticking to Latin1 is not much use these days. Many data such as 
web sites is in Unicode. It would be fine if it worked like R2, as 
a transparent passthrough, but Red eats your Unicode and won't give 
it back from its internal format

I'd be happy to look at a UCS-2 to UTF-8 conversion function but 
I don't have the time to do it at the moment.

I see there are specialised platform specific print functions only 
for printing the internal format. They look like a base for the general 
purpose conversions, though

I've written a quick function that will take a Red char (UCS4) and 
output the equivalent UTF-8 as bytes stored in a struct!.


It can be used for the base of converting a Red sting to UTF-8. What 
is needed is to extract Red Char! s from the Red String, call the 
function and then appedn the UTF-8 to a c-string!

Yes, finally a C, that makes sense :-) Well, nothing against C, I 
am glad it is still around and going to stay ....

I've just committed a slightly improved version that retunrs a c-string! 
instead of  a structure.

For me the big issue of turning the function into the utf-8 string 
that Kaj's wants is "How to allocate a c-string! using the Red Memory 
Manager rather than malloc"

Any suggestions appreciated.

It would be best to do the conversions on the fly, that is why I 
want to wait for I/O get done to implement such conversion routines. 


Anyway, for doing it now, you need to allocate a new string, the 
best way to do it is:

    str: as red-string! stack/push*
    str/header: TYPE_STRING
    str/head: 0
    str/node:  alloc-bytes size


The new string! value will be put on stack, so any other call to 
a Red internal native or action might destroy it. Also, keep in mind 
that the GC is not there yet, so intensive I/O might quickly eat 
up all your RAM.

Oh, you meant a c-string!, not a string!, so it's even easier, just 
use: alloc-bytes size

Currently no, the freeing function requires a memory frame pointer 
in addition to the buffer pointer. It is meant for internal use only 
for now.

Here's how your main loop would look like for retrieving every codepoint 
from a string! value:

	head: string/rs-head str
	tail: string/rs-tail str
		
	s: GET_BUFFER(str)
	unit: GET_UNIT(s)
		
	while [head < tail][
		cp: switch unit [
			Latin1 [as-integer p/value]
			UCS-2  [(as-integer p/2) << 8 + p/1]
			UCS-4  [p4: as int-ptr! p p4/value]
		]
		...emit UTF-8 char...
		head: head + unit
	]

cp hold your codepoint as a 32-bit integer.

I should be able to turn this into a function for Kaj to include 
in his routine! where he needs UTF-8

Kaj is working on Linux and Syllable only. Also that API provides 
UTF-16 to UTF-8 support, but we need also UCS-4 to UTF-8 (UCS-2 being 
a subset of UTF-16).

Endo: I have submitted a report for false positive to AVIRA, I hope 
Red binaries will be whitelisted soon. It seems to be the last AV 
vendor producing false alams, according to virustotal online testing 
tool.

Kaj - You can find a rough and ready  red-string! to  c-string! function 
at:


https://github.com/PeterWAWood/Red-System-Libs/blob/master/UTF-8/string-c-string.reds


it #includes the UCS4 character to UTF8 convertor which you will 
need in the same directory as the string-c-string func.

who needs a GC/memory manager these days, just buy more RAM :-)

FYI, Bruno is working on a Zlib binding for Red/System:
https://github.com/be-red/Red/commits/zlib

I was in the impression that Red and Red/System scripts should be 
in a directory within the Red dir. Where the other (test)scriots 
are too.

I am now hesitant to put a new zip over the old one and lose my scripts 
(but maybe even that fear is vain).

Nenad did a lot of work on making it location independent. A few 
issues remain, for example that Red/System always gets compiled into 
the builds/ subdirectory, but the source can be anywhere

If you just unzip, there´s no reason your scripts would be lost, 
but you can make it a lot easier by not putting them in the Red folders

Not sure what mode of compiling Red/System you are thinking about, 
but if you compile via rsc.r, you can tell it where to write the 
binary by passing a `-o` command-line option.

Kaj: I think I've found a workaround for the semi-random compilation 
crash on Linux. It seems SELECT has some deep stability issues in 
R2.

But just think what a great test this will be for Red's SELECT when 
it is self-hosting. :-)

Actually, the SELECT expression where the compilation error occured 
was involving a word lookup in an object context, so those lookups 
are done, as far as we know, using internal hash tables, I wonder 
if the issue is not related to hash tables implementation bugs in 
R2. If I tried to put a PROBE between SELECT and the word lookup, 
the error was disappearing.

Question: does a Linux shared library need to call __libc_start_main() 
or can it assume safely that this will be done by the host app?

Nope, a lib must not this.

Sorry, that came out incoherent: a library does not need to call 
__libc_start_main.

I'm doing some changes in the exit sequence of Linux/Syllable executables. 
Currently, the Red main() passed to __libc_start_main was never returning 
as it was calling exit(). I'm changing that now to give the libc 
a chance to call its .fini routines and let it handle the exiting.

About the relative path, that was possible. Indeed. The Rebol console 
changed the path a few times so i needed to change it accordingly, 
but after that it was reusable.

./ is a relative path

And my Red is a couple of weeks old.

From a separate working directory?

I'll push a fix shortly.

Pushed. If you give it a try, please let me know if it works for 
you as well.

I´ll have a go tomorrow

Did you run the tests before pushing the commit Andreas?  I now have 
a few failing Red/System tests. I'm in the middle of writing some 
tests and need to get those commitable before I can take a look.

From a quick look your commit has changed the default for red/system 
so that executablles are now built in the current working directory 
rather than the builds directory.

(@Kaj your apostrophe is a Yen sign in my AltME on Mac)

Andreas: watch out for regressions, this is a sensitive feature as 
it affects everything else: test suite, my own local tests scripts, 
Red compilation toolchain, documentation on github homepage, etc...

Fix coming up in a second.

watching pic-emitter Commits section - Doc, you are a coding hurricane 
:-) I can smell first shared library generated by Red coming over 
the weekend :-)

Arnold, I´m on a new Linux installation set to dead keys, so it´s 
manipulating quotes and such. I guess I can´t keep up that experiment

So, what's the prospect for getting Red/System running on a Parallella?

Parallella: Red should be able to run on it from day one as we already 
support their system (Ubuntu/ARM). For the risc chips cluster (Epiphany), 
simply making a Red/System backend for them would not be enough to 
leverage the computing power. We would probably need some new abstractions 
at Red level, specific new natives in Red or even dialects to take 
real avantage of it. For example, we could have a MapReduce native 
support in Red that would   delegate execution to such kind of cluster 
(there are other competing hardware products in that area).

Pekr: you understand that building Red is not going from 0.0 to 1.0 
in one step? So, like in any other long ang complex building processes, 
you have intermediary building steps and states that are not necessarily 
representative of the final product. Think of a house in construction 
with scaffoldings, you don't point at the scaffoldings saying that 
the house will look horrible.

I've found a nice keyboard configuration that seems to do everything 
I want

Petr, there's already a red.exe in my downloads; just not a compiler, 
but the interpreter :-)

@Kaj, The Yen sign has disappeared here! Congratulations! :)

I would still like the default output to be deposited in the working 
directory, instead of the builds directory

 I think I agree with this. In the ideal situation you could specify 
 these things, maybe in a settings file or in a preference in a compilation 
 assisting program. (Started one once)  

The integration of rsc.r within red.r sounds as a logical step. (rsc.r 
Red System Compilation, not so bad and a clear difference from red.r 
for Red scripts)

@PeterWood Thank you again Peter! 

Peter answered a question on the Red mailing list (all subscribe!) 
about me calling a function in Red like f(10) where it should have 
been f 10 and I didnot get this. The (10) part was being processed 
as a paren! type so the value was calculated before giving to the 
function f as input.

DocKimbel: Parallella: Sounds like a very interesting challenge. 
:-)

Doc: "did you encapped Red compiler?" Nope, RedCompiler.exe is just 
calling the rsc.r or red.r according to the extension of given script 
file. It should be placed in the red-master directory.

I tried to make a encapped Red compiler but there are some problems 
with #include or other compiler directives.

Once we merge %red.r and %rsc.r, I will look into encapping the whole 
thing, so we'll be able to start distributing Red as a unique binary 
file.

Looking at http://www.red-lang.org/p/contributions.htmlit seems 
to me, that Red has more bindings, than for R2 + R3 combined :-) 
Maybe just an impression, as I miss such a conscise list on rebol.com 
website ....

I am not filling mem, that is only an extract of my code (a part 
of a search function).

Could be it is a compiler error, the message suggests it imho.

Have you previously done a while loop in Red/System like that? Try 
it a little simpler while [0 < len][mem: mem + 1 len: len - 1] to 
be sure it is not the combined statement that is confusing the compiler. 
(Just trying to help here until the real experts jump in)

Assignments can't be embedded, so you have to split them out into 
a separate expression

put all [] in a block too?

Good thing Red isn't called Green (as in combined blue and yellow)

The all [] was meant to be a test to see if Red/System can handle 
it?

That's a good functional programming idiom that REBOL and Red support. 
Without it, they wouldn't really be functional languages

Hey guys was just thinking.  Why not a RED group on linkedin.com? 
 Doc you should start one up.  I'll definatley join and think many 
here would.  Would be another outlet to get more discussion with 
those outside of the usual outlets.

Good idea. I would join from the start. But better maybe in a later 
stage of the development? When it is more appropriate to attrackt 
more outside momentum because there is even more Red to show to the 
world? After Red becomes really selfsupporting? It seems to me all 
the other channels are taking a lot of time for Nenad as is.

It's low maintemance - more a badge of recognition for the time being 
but since Doc updates the red-lang.lorg site may just as well put 
those posts in the linked group also.

Pekr: I think FB was a mistake, it's not the right place/tool where 
to have technical discussions (especially on deep topic like programming 
languages).

Kaj: I don't understand what you are talking about. If you cared 
about giving an example or any useful information I could look at 
or work on, your post would look a bit more constructive.

A very similar thing happens when I paste Latin-1 into Windows

Does READ create a red-string!?

You caught me while in the middle of work on pic-emitter, I switched 
to master branch without saving my changes, and now my local git 
repo is messed up (not sure why it happened). I can't test anything 
anymore before finding a way to fix it...

I think having comm lieutentants is a good idea.

yes, and as a contributor, you have been surely noticed :-) I am 
still somehow satisfied with how Red development goes, eg. even in 
comparison with R3. Nothing against R3, I wish Cyphre takes us to 
Android, but for general purpose needs, untill R3 gets protocols, 
multiline console, etc, stuff noticed on SO, it is no go for me. 
So, I keep my fingers crossed as far as Red development goes - it 
is nice to have a strong leader around ...

Kaj: why don't you open a ticket with a code example on github if 
something is broken or not working as advertised? I thought that 
was our process to get issues solved.

Andreas: that's what is supposed to be implemented, but now I have 
a doubt about it. Will need to check the code tomorrow.

Because you said it's not a priority and because I'm trying to ananlyse 
and fix things myself first. I'm just reporting my findings

Actually, I did one test that confirms Andreas' statement. The only 
way to get 8-bit data in is to compile a UTF-8 string literal that 
fits into Latin-1

I didn't see a claim anywhere, that you can compile Latin 1 sources.

Yes, and neither works, so there is no Latin-1 support at all, except 
in a corner case internally

The "internal corner case" is no Latin1 support either. It's purely 
Unicode support, with a more efficient internal storage.

Let's give Doc a couple days to recover from pic-emitter, and I'm 
sure he'll clarify things.

Eh, then why did Peter implement a lot of code for it?

It was announced last year that Unicode support was implemented in 
a week. What I found first is that Unicode support is useless, and 
now I've found that only ASCII is really supported

If Unicode support wouldn't be implemented, Peter would have a very 
hard time writing those transcoders.

Just a matter of discerning internal implementation details and user-visible 
external features.

Please, I'm not bashing Red, I'm trying to represent it truthfully. 
I'm trying to protect it from getting bashed. Why would I bash a 
project I spent the last two years of my life on?

I think this is just a misunderstanding. Kaj, would you be happy 
with a simple language change, for now, that descibes the current 
state of external unicode support?

We could say they are supported internally, but I/O is TBD. But I'm 
all for a simple text change that solves this for now.

Gregg: that's already what is claimed at the moment, but it seems 
to not be a very communicable message.

Kaj: I do take into account all observations, and especially from 
people that work on it and with it since the beginning. I just can't 
do miracles and implement in a night features that take weeks.

Looks like the newer Beagleboards are using regular ARM CPU (no more 
Cortex-M series), so Red should run just fine on them:

http://techcrunch.com/2013/04/23/the-beaglebone-black-is-a-new-single-board-computer-that-can-brew-beer/