World: r3wp

right now the GC is very cumbersome. it waits for it to have 3-5MB 
before working. and it can take a noticeable amount of time to do 
when there is a lot of ram.  I've had it freeze for a second in some 
apps.

 - what exactly does the GC have in common with the "Deduplicate issue"?

I demonstrated above, that, in fact, nothing.

But that doesn't  mean that deallocating immediately will be any 
more efficient; likely it won't.

This is all just pretending, if, what is needed, is a kind of incremental/generational/whichever 
other GC variant, then no "Deduplicate" can help with that

We don't need DEDUPLICATE to help with the GC. He was suggesting 
that having it be native would help reduce the pressure on the GC 
when used for other reasons instead of a mezzanine version. I don't 
think it will by much.

He needs DEDUPLICATE for his own code. The GC also needs work, but 
that is another issue :)

if we implement deduplicate as a mezz, we are juggling data which 
invariably tampers the GC.  doing this native, helps to prevent the 
GC from working to hard.


the problem is not how long/fast the allocation/deallocation is... 
its the fact that cramming data for the GC to manage, will make the 
GC trigger longer/more often.

But you would need to deallocate every time the function is called, 
instead of just when the GC is called. This is often slower.

Did you read Carl's note to the subject?

The subject of the set function implementation, or the GC implementation 
and how it compares to direct deallocation? If the latter then no.

I meant the note more to Max, and it was about the set function

Ah, cool :)

The GC is not a slow approach to the garbage collection. The main 
problem is, that it is "unpredictable", and possibly producing delays, 
when other processing stops. (but that does not mean, that immediate 
collection would be faster)

yes I did.

the "stop the world" approach is disturbing for user interface, which 
might need a different type of GC...

Also, multitasking could require a different kind of GC. Any thoughts 
on this, Ladislav?

just adding a generational system to the GC would help a lot.  I've 
read that some systems also use reference counting and mark and sweep 
together to provide better performance on data which is highly subject 
to volatility.

though I guess it requires a bit more structured code than rebol 
to properly predict what is expected to be volatile.

re DEDUPLICATE ... it's not just GUI code that would benefit, DB 
code often needs to apply this on the result set. "buffer: unique 
buffer" is a common idiom, which can be problematic with very large 
datasets.

You still need an intermediate dupe even for DEDUPLICATE. This just 
makes it so the argument is modified, in case there are multiple 
references to it.

it's not just GUI code that would benefit, DB code often needs to 
apply this on the result set. 

buffer: unique buffer" is a common idiom, which can be problematic 
with very large datasets" - that is exactly where I was trying to 
explain it was just a superstition - buffer: unique buffer is as 
memory hungry as you can get any Deduplicate is just pretentding 
it does not happen, when, in fact, that is not true

Does DEDUPLICATE *have to* create a new series. How inefficient would 
it be to SORT the series then REMOVE duplicates starting from the 
TAIL. Inefficient as a mezz, but as native?

Deduplicate does not have to use auxiliary data, if the goal is to 
use an inefficient algorithm. But, in that case, there's no need 
to have it as a native.

even if done without aux data, it will still be MUCH faster to do 
in native.

No

But, as Ashley pointed out, if there's no need to keep the original 
order, it can be done in place easily

Lad, just looping without doing anything is slow in REBOL.

That does not prove your point

the average test is that things done in extensions are at least 10 
times faster, and Carl has shown a few examples which where 30 x 
faster.  really Lad, there is no comparison.

Still missing the argument

you can elaborate if you want... I'm just passing thru, in between 
two jobs in the renovations... will be back later.

To find out what is wrong, just write an "in place" version of Deduplicate 
in Rebol, divide the time needed to deduplicate a 300 element series 
by 30, and compare to the algorithm (in Rebol again) allowed to use 
auxiliary data.

You shall find out, that the fact, that the algorithm is native, 
does not help.

Or, to make it even easier, just use an "in place deduplicate" written 
in Rebol, divide the time to deduplicate a 300 element series by 
30, and compare to the time Unique takes (Unique uses aux data, i.e. 
a more efficient algorithm)

You shall find out, that the difference made by an inappropriate 
algorithm is so huge, that even as a native it would be too slow 
compared to an efficient algorithm written in Rebol

yes, I was not comparing two different algorithms... but the same 
algo done native vs interpreted.

I am curious who do you think would agree to write that nonsense

You talk about is so much that someone could write an extension in 
the same time and give a real prove:) What I can say, using additional 
serie is a big speed enancement. At least it was when I was doing 
colorizer.

You talk about is so much that someone could write an extension in 
the same time and give a real prove:) What I can say, using additional 
serie is a big speed enancement.

 - actually, it has been proven already, just look at the performance 
 of the UNIQUE, etc. functions

That is why I do not understand, why it is so hard to understand.

ALIAS function removed in the next version; see http://issue.cc/r3/1163
http://issue.cc/r3/1164http://issue.cc/r3/1165and http://issue.cc/r3/1835
for details. Also, http://issue.cc/r3/1212dismissed as unnecessary 
now.

R3 still uses case aliasing internally for word case insensitivity, 
but there is no other aliasing.

unfortunately, in the extensions, case insensitivity doesn't work.

(unless its been fixed in the very latest versions)

Do you mean in object contexts, or the words block?

Either way, report it.

word blocks for sure, didn't test on objects.

I will, next time I play on the hostkit.  right now I coudnt' provide 
example code which demonstrates it.

A report like "Words are not case insensitive in extension word blocks" 
would help a lot. Carl has been bug fixing lately, and that includes 
extension bugs.

ok... will add that to begin.