World: r4wp

It would be best to do the conversions on the fly, that is why I 
want to wait for I/O get done to implement such conversion routines. 


Anyway, for doing it now, you need to allocate a new string, the 
best way to do it is:

    str: as red-string! stack/push*
    str/header: TYPE_STRING
    str/head: 0
    str/node:  alloc-bytes size


The new string! value will be put on stack, so any other call to 
a Red internal native or action might destroy it. Also, keep in mind 
that the GC is not there yet, so intensive I/O might quickly eat 
up all your RAM.

Oh, you meant a c-string!, not a string!, so it's even easier, just 
use: alloc-bytes size

Thanks.

Is there any easy way to free the c-string?

Currently no, the freeing function requires a memory frame pointer 
in addition to the buffer pointer. It is meant for internal use only 
for now.

Anyway, even freeing it won't help much as long as the GC doesn't 
do the cleanup.

Here's how your main loop would look like for retrieving every codepoint 
from a string! value:

	head: string/rs-head str
	tail: string/rs-tail str
		
	s: GET_BUFFER(str)
	unit: GET_UNIT(s)
		
	while [head < tail][
		cp: switch unit [
			Latin1 [as-integer p/value]
			UCS-2  [(as-integer p/2) << 8 + p/1]
			UCS-4  [p4: as int-ptr! p p4/value]
		]
		...emit UTF-8 char...
		head: head + unit
	]

Oops, you should replace 'head by 'p in the above code.

Many thanks.

cp hold your codepoint as a 32-bit integer.

I should be able to turn this into a function for Kaj to include 
in his routine! where he needs UTF-8

I guess that should be enough for his needs.

Fingers crossed :-)

why not to use native OS functions? At least on WIn there is: http://msdn.microsoft.com/en-us/library/windows/desktop/dd374085(v=vs.85).aspx

Kaj is working on Linux and Syllable only. Also that API provides 
UTF-16 to UTF-8 support, but we need also UCS-4 to UTF-8 (UCS-2 being 
a subset of UTF-16).

Some UCS related code for porting is here: http://public.googlecode.com/svn/trunk/UCSUTF.cpp

Endo: I have submitted a report for false positive to AVIRA, I hope 
Red binaries will be whitelisted soon. It seems to be the last AV 
vendor producing false alams, according to virustotal online testing 
tool.

Thank you, I'll check it later.

Kaj - You can find a rough and ready  red-string! to  c-string! function 
at:


https://github.com/PeterWAWood/Red-System-Libs/blob/master/UTF-8/string-c-string.reds


it #includes the UCS4 character to UTF8 convertor which you will 
need in the same directory as the string-c-string func.

The ucs4 -> utf8 char convertor:


https://github.com/PeterWAWood/Red-System-Libs/blob/master/UTF-8/ucs4-utf8.reds

I haven't really tested it as you can see from :


https://github.com/PeterWAWood/Red-System-Libs/blob/master/UTF-8/Tests/string-c-string-test.red

I'm not sure how it will cope with repeaated use as there is no way 
to release allocated c-strings under the Red memory manager.

Hope it helps.

who needs a GC/memory manager these days, just buy more RAM :-)

Peter, maybe you could user ALLOCATE function from Red/Sytem and 
let Kaj's code call FREE on UTF-8 buffers after usage?

I didn't think that it was possible to mix using the Red Memory Manager 
and C memory management in the same program. Is it safe to do so?

Yes, it is.

I have committed the change for the c-string to be allocated with 
Red/System ALLOCATE function.

FYI, Bruno is working on a Zlib binding for Red/System:
https://github.com/be-red/Red/commits/zlib

Much obliged, Peter. I can work that into my I/O routines

Really the thanks should go to Nenad. Without his help, I still be 
trying to work out how to do it.

I've add support for code points above the BMP.

add -> added

That is just called team work. :-)

Would be nice to be able to download the Red zip file and be able 
to compile scripts in another directory?

How do you mean? What you seem to ask is already possible

I was in the impression that Red and Red/System scripts should be 
in a directory within the Red dir. Where the other (test)scriots 
are too.

I am now hesitant to put a new zip over the old one and lose my scripts 
(but maybe even that fear is vain).

Nenad did a lot of work on making it location independent. A few 
issues remain, for example that Red/System always gets compiled into 
the builds/ subdirectory, but the source can be anywhere

If you just unzip, there´s no reason your scripts would be lost, 
but you can make it a lot easier by not putting them in the Red folders

Not sure what mode of compiling Red/System you are thinking about, 
but if you compile via rsc.r, you can tell it where to write the 
binary by passing a `-o` command-line option.

As in:
rebol2 -qsc rsc.r -o hello tests/hello.reds

I´m referring to the normal case of wanting to get the output in 
your working directory. Red does that now, but Red/System not without 
using that switch

You also need to give the full path to the source file

Actually, you also have to give the full path in the -o switch

I did some compiling today and found the resulting executable in 
the build directory.

Relative paths work for me.

Ishttps://github.com/dockimbel/Redthe official Github site?

Is https://github.com/dockimbel/Redthe official github site?

yes, it is ...

./ paths don´t work for me on rsc.r