World: r4wp

and I can't afford to have a hole in the list

 Brian, could you give us some short code cases where this was a problem 
 for you? This would really help.

pick ser idx + off

Just erroring out on index 0 is ann improvement.

 That's my intention for Red until we get a consensus on a better 
 overall solution.

We won't get consensus on this.

We won't if we stick to only R2/R3 options. Fortunately, there are 
other ones, maybe we'll find more.

Sorry, I didn't mean off in the sense of false, I meant short for 
an offset. Any place where you have computed indexes can have a computation 
that turns out to be less than 1, especially if your base position 
is offset.

Error out and add PICKZ, POKEZ while at it. That would at least give 
us a base to work with.

The problem with keeping most of R2's behaviour much longer is that 
switching to something different will only get harder.

Agreed.

Remember, R3's behavior isn't done either. There's a standing consistency 
bug, for instance. If we come to a consensus, R3 is likely to adopt 
it too.

So if, for example, we were to switch to 0-based indices-as-offsets 
complemented with an ordinal! type, it's better to do this sooner 
rather than later.

BrianH: CC for this consistency bug?

http://issue.cc/r3/609

Thanks.

I wouldn't even mind if PICKZ/POKEZ were the actions and PICK/POKE 
were the native wrapper functions. Not as pretty for port scheme 
implementors though.

Brian: in `pick ser idx + off`, how often do you need `idx + off` 
to give you both positive and negative results from the same expression?

Usually, the number I'm passing to PICK or POKE is computed elsewhere. 
That was a one-line example for simplicity.

I would just like to know if it's an issue (the 0 gap) you hit once 
in your lifetime or if it's something people encounter from time 
ot time or even often (depending on the coding style).

(just curious)

We should write a short-list of possible options that would solve 
the whole issue and see if we can get a large consensus on one of 
them. Anyone kind enough to extract the different options we've discussed 
and put them somewhere online with the main pros/cons?

I use computed indexes for computed lookup lists, such as for precomputed 
intermediate results of computations, translation tables, etc. If 
the computation uses signed numbers, you have to do an offset base 
position to get the results from the positions less than 1. Having 
a hole slows down the computation because it has to be handled in 
mezzanine code. PICKZ/POKEZ would actually be better for most of 
these situations because the computations work better with 0-based 
numbers (modulus, for instance). It's pretty common in code that 
actually *needs* to use PICK/POKE on series.

I've found that aside from computed indexes, the only times I use 
PICK/POKE on series are for negative/0 indexes, or indexes greater 
than 10. For the rest the ordinal functions are faster.

Of course port schemes and datatypes actually need to implement PICK/POKE, 
but at least for datatypes the implementation is usually native so 
the hole is less expensive to implement.

It's pretty common in code that actually *needs* to use PICK/POKE 
on series.


That's the sticking point. We can categorise the uses for PICK/POKE:

(a) with large positive literals
(b) with literals -1, -2
(c) with computed indices and series in "head position"
(d) with computed indices and an offset series

More categories welcome.

Having a hole slows down the computation because it has to be handled 
in mezzanine code.


Have you ever had to actually write such "0 gap" workaround in your 
code, or is Carl the only one that had ever been bitten by this isssue?

I think it's more common to rewrite the code to something different, 
where possible, than to try and workaround the 0 gap.

Anyone who does computed indices from offset base positions would 
run into that issue. However, given that this only tends to happen 
in heavy-math code, I wouldn't be surprised if Ladislav and I would 
be caught by it more often than anyone else in the REBOL community, 
even Carl.

Maybe the other Saphiron guys too.

It will also happen in data-heavy code.

E.g. pure-REBOL database implementations an such.

I use that pattern in structure-tricky code, like those databases 
Andreas mentioned.

I'm pretty sure I ran across this problem when trying to implement 
a triple store ages ago (and ended up switching to a different implementation 
approach).

I'm not as much of a math head as Ladislav, so I can't always do 
the algebraic transformations necessary to switch to unsigned arithmetic.

Strangely enough, Red's native code compiler means that I'm more 
likely to use it for this than I would REBOL. Mathematical computation 
isn't REBOL's strong suit.

Brian: that's also the kind of usage where Red should give you the 
best performances, as math expressions can be highly optimized.

Yup. But that doesn't mean that I can switch to unsigned arithmetic. 
For that matter, sometimes switching to unsigned arithmetic means 
mostly indices at the top and bottom of a large range, skipping the 
middle (basically what you'd have to do with the Python model). By 
"large" I mean allocating a 2GB series or more and only using the 
100MB or so at both ends. Not always a good idea.

If you don't want to be hit by the pick 0, you can stay at head position 
of the series cannot you?

You can stay at the beginning of the series if you add the base offset 
to every calculated index reference. Bad idea in REBOL, too slow, 
but maybe Red's optimizer will be smart enough to undo that calculation. 
How many algebraic transformations will the optimizer be able to 
handle?

For that matter, in R3 (where I don't get caught by the 0 hole for 
PICK/POKE, just for AT) I have to do an offset-forward-by-one reference 
to avoid having to add 1 to every calculated index reference. Doesn't 
help in R2 though.

Hard to answer that now, but as the optimizing compiler will have 
a plugin architecture, it won't be limited, in the sense that you'll 
be able to add your own specific code optimizations, that if generic 
enough, could make its way into the official   repo. If the optimization 
processing time is short enough, it can be used for the JIT, else, 
it will be used only in for AOT compilations.

In that case, it would help if PICKZ/POKEZ were the actions because 
they don't have the 0 hole, and we can add PICK/POKE optimizations 
that apply an algebraic transformation to change the call to PICKZ/POKEZ. 
Implementing the hole in the action has overhead that the optimizer 
wouldn't be able to undo. It would be able to undo the hole implementation 
in a wrapper function though by inlining the code then optimizing 
it away. That would make up for the problems caused by putting the 
0 hole in there in the first place.

Though a type inference action-to-native transformation would help 
too :)

For PICKZ/POKEZ, we'll see if they still make sense once we come 
up with a global solution.

Most computed index situations use modulus, which generates 0-based 
numbers. If there's no 0 hole in PICK/POKE then you can do PICK/POKE 
NEXT, as long as this doesn't have side effects the way it does for 
port schemes or weird copy-on-write stuff; in that case PICKZ/POKEZ 
would be more of a convenience. If there's a 0 hole in PICK/POKE, 
you absolutely need a version of the functions without that hole, 
and having the extra advantage of being 0-based would be even better, 
an extra justification for their existence.

Keep in mind though that another occasion when you use computed indices 
with PICK/POKE is when you do stuff like s/(x) or s/(x): val, since 
path references for series should be using PICK/POKE internally.

One occasion where you use computed indexes is when you do C-style 
fake multidimensional array access. As with C, however, the math 
tends to be 0-based because of the modulus, so a 0 hole really hurts 
here. You can do these calculations much more easily if your base 
series position is 2 (or plus 1 * (dimension - 1) for each dimension) 
because the 0's go back one.

(sorry, there's two standard ways to pluralize index and I don't 
use either consistently)

Doc, Will you publish a Red/System standard function guide, listing 
all the functions, such as: print, print-line, length?, size?, zero?, 
form-type, ...

- "the most sane way to make a decision here is to come up with use 
cases": +1
- We are unlikely reach consensus: +1
- Have I ever been bitten by R2's behavior? No.


Great discussion on this. I greatly appreciate Andreas's points about 
common (-1, -2) cases, and his gist example. I'm good with throwing 
an error where silent incorrect processing would be worse. The concrete 
examples will be the best way to drive this, IMO. By concrete, I 
mean real-world, "this is code I actually wrote to solve a problem" 
stuff. If you can't post code for legal reasons, a generalized adaptation 
is almost as good. Make the most common uses behave correctly, even 
if there are cases they don't handle, and show how the other cases 
*can* be handled, even if it means slower mezz code. Then see if 
there's a better way to support them.