World: r4wp
[#Red] Red language group
| older newer | first last |
| BrianH 4-Sep-2012 [1553x3] | IIRC Python 3 uses UCS4 internally for its Unicode strings, with all of the overhead that implies. UCS4 and UTF-32 are the same thing, both fixed-length. |
If you support different internal string encodings on a given platform, be sure to not give logical access to the underlying binary data to Red code. The get/set-modes model is good for that kind of thing. If the end developer knows that the string will be grabbed from something that provides UTF-8 and passed along to something that takes UTF-8, they might be better off choosing UTF-8 as an underlying encoding. However, that should just be a mode - their interaction with the string should follow the codepoint model. If the end developer will be working directly with encoded data, they should be working with binary! values. | |
Btw, in this code above:
s/encoding: 'UTF-16
print length? s ;-- Length? then runs in O(1), no surprize.
Length is not O(1) for UTF-16, it's O(n). Length is only O(1) for
the fixed-length encodings.
| |
| DocKimbel 4-Sep-2012 [1556x2] | Since Python 3.3, things have changed: http://www.python.org/dev/peps/pep-0393/ |
Brian: right, my claim is valid for BMP characters only. | |
| BrianH 4-Sep-2012 [1558] | Ah, but length is even O(n) for BMP characters in a UTF-16 string, because figuring out that there are only BMP characters in there is an O(n) operation. To be O(1) you'd have to mark some flag in the string when you add the characters in there in the first place. |
| DocKimbel 4-Sep-2012 [1559] | Ok, if you really want to be nitpicking, replace UTF-16 with UCS-2. ;-) |
| BrianH 4-Sep-2012 [1560x3] | If you are ensuring that only BMP characters are in there then you have UCS2, not UTF-16 :) |
Python 3.3 seems to finally be following the R3 model, good for them. Even better for them because it's actually implemented. | |
Don't worry, I'm only nitpicking to make things better. There's a lot of buggy code out there that assumes UTF-16 is UCS2, so we're better off making that distinction right away :) | |
| DocKimbel 4-Sep-2012 [1563] | Well, then I'm sure you'll be glad to write string unit tests for Red in order to ensure things are done in the proper way. ;-) |
| BrianH 4-Sep-2012 [1564] | Doc, pardon me because I don't know what the intended datatype model is for Red. Something like the REBOL datatype/action model could be used to implement the different underlying string encodings that you want in-memory support for. Each supported encoding would have its own set of action handlers, which would all have the same external interface. Swapping the encoding would be as simple as swapping the handler set. Resolving the difference at compile time could be similar to generic type instantiation, or C++ template generation. |
| Kaj 4-Sep-2012 [1565] | The datatype code that's committed so far seems to go that way |
| DocKimbel 4-Sep-2012 [1566x2] | Brian: implementing an abstraction layer over string encodings is a trivial task. The intended datatype model is very similar to REBOL's one, even more since recentl,y as I moved to an hybrid dynamic/static type system. I'll commit the new code in a few days, so you'll see how close to REBOL it can be. I hope that this hybrid model will help us get the best of both worlds. |
*recently, | |
| DocKimbel 5-Sep-2012 [1568] | Got my first real Red program working:
Red [] print 1
outputs:
1
It doesn't look like much, but it validates the compiler + runtime
from end to end, and at this point, it's really cool! FYI, the native
PRINT here triggers a FORM (action) on the passed argument. No REDUCE
yet (not implemented).
|
| james_nak 5-Sep-2012 [1569] | Congratulations Doc. That is so cool. Really! |
| DocKimbel 5-Sep-2012 [1570] | Thanks James! |
| Janko 5-Sep-2012 [1571] | Wow!!! awesome .. from print "hello worl" to print "<html>...</html>" to make webapps is not that long way, I was just afraid you would never get to RED, Awesome! |
| AdrianS 5-Sep-2012 [1572] | Yeah! Way to go! |
| Gregg 5-Sep-2012 [1573] | Congratulations Doc! This is a great day. |
| GrahamC 5-Sep-2012 [1574x2] | A milestone |
( kilometre in France ) | |
| Sunanda 5-Sep-2012 [1576] | +1 for Red :) |
| Henrik 6-Sep-2012 [1577] | Congratulations, doc. :-) |
| sqlab 6-Sep-2012 [1578] | This gives hope |
| Endo 6-Sep-2012 [1579] | Cool! Another 30€ for the good news :) |
| Jerry 6-Sep-2012 [1580x2] | Wooow.. Wooow... Did I hear the crying of a baby... I think this is the birthday of Red. The past 1.5 years are just pregnancy. |
:-) Can't wait to show Red to the Chinese people. | |
| Pekr 6-Sep-2012 [1582] | From now on, the child is going to grow day by day :-) |
| Jerry 6-Sep-2012 [1583] | I can tell, it's gonna be a super star :-) |
| Cyphre 6-Sep-2012 [1584] | Congrats, Doc! |
| DocKimbel 6-Sep-2012 [1585x2] | Endo: thank you! Thanks to all for your support! |
Pekr: right, from now on, you can expect daily progress on Red layer. I will push the new code soon, I still need to complete it a bit and clean it up. Jerry: the baby looks nice, we'll just have to keep it away from junk food and it will grow up well. ;-) | |
| PeterWood 6-Sep-2012 [1587] | Many congratulations Nenad. |
| DocKimbel 6-Sep-2012 [1588] | Thanks Peter! |
| Gerard 6-Sep-2012 [1589] | Félicitations Doc - I will also submit an additional donation to help the baby to grow further ... 50€ was sent to you Doc |
| Jerry 6-Sep-2012 [1590] | Doc, I announce this good news in Chinese http://weibo.com/rebol/ :-) |
| DocKimbel 6-Sep-2012 [1591] | Thanks Gerard! Jerry: Cool! I'll make a blog entry too. |
| DocKimbel 7-Sep-2012 [1592] | Pekr: "I wonder what the strategy for Red is going to be in regards to GC ..." Currently Red is using a per-OS thread mark-sweep-compacting GC. I intend to make it concurrent (maybe parallel) and incremental when (and if) possible. The collector is not yet fully implemented. It is also possible to easily make it generational if we need it. When we'll have big Red apps, we'll profile them and determine the best tuning options for the GC. There's no silver bullet for memory management & GC, you need to adapt it to the language specific needs (and they may vary depending on the type of application: client-side, server-side, realtime, ...). |
| Henrik 7-Sep-2012 [1593] | Will it be possible to swap it out, depending on needs? |
| DocKimbel 7-Sep-2012 [1594x2] | I plan to use an abstraction API for memory management, so that we could in future, implement different ones. |
Brian: I was wrong for OS X, it uses UTF-16 internally according to http://en.wikipedia.org/wiki/UTF-16 | |
| Jerry 7-Sep-2012 [1596] | I am giving a speech in SDCC conference tomorrow. http://sdcc.csdn.net/speaker.html My topic is Dart. I hope it will be Red next year when Red is mature enough. :-) |
| DocKimbel 8-Sep-2012 [1597x3] | In a year, we should have passed v1.0 since a while. ;-) |
Just want to have your advice on a issue I will have soon with Paypal and donations: there's a limit of 2500€ per year of reception allowed for EU accounts. It is possible to extend it by providing a lot of administrative documents to Paypal and even then it's at their discretion. Has I moved out of France, I won't be able to complete this requirements (and Montenegro is not supported by Paypal). So suggestions are welcome on how to keep receiving donations in a easy way for donators? | |
In the meantime, I would ask for donators making bigger than 100€ donations to use banking transfers instead of Paypal, so that I don't reach that limit this year. | |
| Kaj 8-Sep-2012 [1600] | Personally, I plan on setting up a Bitcoin donation option |
| DocKimbel 8-Sep-2012 [1601] | Do you know a safe trading place where bitcoins can be converted to real money? The ones I used to know have all closed after being hacked... |
| Kaj 8-Sep-2012 [1602] | They're still hacked now and then, but there are ever more. Here's a market share list: |
| older newer | first last |