AltME groups: search

For Red/System, a 0-based convention might make more sense, but it 
would push us into the R3 issue I've mentioned above wrt indexes 
<= 0.


With indices-as-offsets ("0-based"), there really is no issue with 
indices <= 0.

As for R3, it did not really introduce "0-based convention implicitly", 
it still is firmly "1-based" in as far as the first element in a 
series can be accessed using index 1.


When you want indices-as-ordinals, you really need to decide: (a) 
is the ordinal "zeroth" meaningful, and if so, what it means; (b) 
are negative indices meaningful, and if so, what they mean.


R3 went with the choices of (a) having meaningful zeroth, defined 
as "the item in a series before the first item", and (b) allowing 
negative indices, having index -1 as the immediate predecessor of 
index 0.


R2 went with the choice of (a) not having a meaningful zeroth, but 
instead of erroring out, functions (pick) & syntax (paths) accepting 
indices are lenient: passing an index of 0 always returns NONE. For 
(b), R2 allows negative indices and defines -1 as the immediate predecessor 
of 1.

Andreas: thanks for the good sum up.


R3: agreed that index 1 is still the first element in a series, but 
index 0 is allowed and there is this ticket #613 that clearly aims 
at introducing 0-based indexing in R3...so my guessing was these 
different changes or wishes were inter-related. http://curecode.org/rebol3/ticket.rsp?id=613


R2: I would have really prefered that index 0 raises an error than 
returning none.

There is also the option proposed by Gabriele to consider: an ordinal! 
datatype (...-2th, -1th, 1st, 2nd, 3rd, 4th,...).


It could solve the whole thing, but I see two cons about this option: 

1) negative ordinals look odd, I don't even know if they can be read 
in engllish?

2) code would be more verbose as it will need conversions (to/from 
ordinals) in many places.


In addition to the pros, making  a difference between an integer 
and an ordinal might help improve code readability.

The problem with no meaningful index 0 is that potentially meaningful 
index values are no longer isomorphic to integers. And as REBOL has 
no actual datatype for indices, all we can compute with are integers 
while relying on a correspondence of those integers to indices.


If you only ever compute indices for series positioned at the head, 
you get a nice correspondence of integers to indices, because meaningful 
indices for this series correspond to the positive integers.


But if you also want to compute indices for series positioned elsewhere, 
this nice integer-to-index correspondence breaks down as you suddenly 
have an undefined "gap" for the integer 0, whereas negative integers 
and positive integers are fine.

Andreas: do you have a short code example involving index 0 in computation? 
I don't remember ever having issues with index 0 and I use series 
with offsets a lot! Though, Ladislav claims he and Carl did encounter 
such issue at least once...the use cases for this issues remain a 
mystery well kept by Ladislav. ;-)

So sorry, I don't have a particular example at hand, but I can easily 
imagine it coming up with e.g. forall or forskip and trying to access 
previous values in an iteration.

Though, Ladislav claims he and Carl did encounter such issue at least 
once

 - I am claiming that I have revealed a bug in Carl's code caused 
 by the fact that indices are not isomorphic to integers, i.e. they 
 "contain a gap". That is a totally different issue than whether indexing 
 should be 1-based or 0-based.

Ladislav: I think it is relevant to this topic as findind out if 
the index 0 gap is a real practical issue or not, could help decide 
about the indexing base.

Then: gap is a practical issue, causing bugs.

Have a look at the following illustration:
https://gist.github.com/5af73d4ecf93ac94680a

Yes, I can in this case: "unreasonable thinking" here is the fact 
that the "mathematical model" - in this case the numbering of positions 
in series differs substantially from the properties of the object 
it is modelling - in this case there is a difference between the 
"no-gap in the series" versus "gap in the mathematical model".

It is because the series values actually are a contiguous concept, 
but their respective indices are not a contiguous space of integers.

Numbering positions in a series is, in other words, characterized 
as "mathematically modelling 'positions' in a series". Your "inbetween 
positions" are something that does not exist in the series in fact.

Tail position is inbetween - actually not. You can write: INSERT 
TAIL SERIES #"a". You do not insert the character "inbetween", in 
fact.

tail position

 is actually a misnomer, in as far as it corresponds to no proper 
 index of a series. The special behaviour series seen in some functions 
 when operating on series in "tail position" would warrant "tail mode" 
 as a more sensible description of the state the series is in.

as it corresponds to no proper index of a series

 - I reserve the right to disagree. INDEX? TAIL gives some correspondence

Yes, it gives an index that can I'd consider "improper" for the series, 
in as far as the series does not contain a value at that position.

A rather meaningless correspondence, though.

(And an index that does not correspond to a value in the series, 
at this point in time.)

It is not meaningless as far as I am concerned. Reson: it would be 
meaningless only if you agreed that INDEX? should not yield a value 
for TAIL

http://www.rebol.com/r3/docs/concepts/series-traversing.html


The first position of the block is called its head. This is the position 
occupied by the word red. The last position of the block is called 
its tail. This is the position immediately after the last word in 
the block. If you were to draw a diagram of the block, it would look 
like this: [...] Notice that the tail is just past the end of the 
block.

Too bad the images are missing...

(I meant that as a reaction to Andreas' contrib above)

Or, in a special case.

also, yet another inconsistency (PICK help string):

Returns the value at the specified position in a series.

My thought behind the "tail position" complaint above was, that series, 
position, and index are already used rather exchangably, and lax 
in a few parts of REBOL's documentation. Adding "tail position" to 
the mix, only further contributes to that confusion.

If you realize that indices are one degree vectors.   A lot of this 
discussion becomes moot.   vectors of length 0 are considered impossible 
when considering only natural numbers (due to 0 divide).  This is 
why I consider R2's handling of indices proper.   


As such, any series "position" is not AT a value it is LOOKING AT 
a value (oriented in positive direction, starting at a point in space 
which is "0").   like extending your arm to grasp the nth thing in 
front of you.  


Tail are 0 length vectors (thus mathematically imposible), when we 
are beyond  the last item edge we are at the edge of space.   you 
cannot "take" the tail item, there is nothin in front of you, just 
as you cannot "take" the 0th item, there is no such thing, 0 is the 
origin of the vector).


when we consider series indices to be vectors, we see the natural 
relationship which Ladislav pointed with SKIP and other methods.


with vectors, things like COPY/PART make sense in the negative direction, 
just as well as in the positive direction.



In R3, this was changed to indices being OVER a value , with the 
first item requiring you to look down and then away for other values. 
 The issue is that index 0 is looking backwards... that doesn' map 
to any good reasoning.  In fact it creates many weird inconsitencies 
in the model, when you try to describe it.


R3's series changes seem like a kludge work-around to map non-vectorial 
infinite integer space to a bounded vectorial space. sacrificing 
model integrity in the process (while trying to ease its mathematical 
properties).  R3's series *may* be "easier to count in a loop"  but 
the values being used make no sense.   counting backwards requires 
us to manipulate the indice for it to "make sense", whereas before, 
counting backwards was the same as counting forward.  we where just 
LOOKING in the opposite direction (the vector's orientation is inversed).

Moreover, something like "the head position" is actually a "permanent 
characteristic", while "the tail position" is in a sense "ephemeral" 
as demonstrated by the following:

block: [a b c]
pos-a: head block
pos-b: next pos-a
pos-c: next pos-b

; now, POS-C is not tail
tail? POS-C ; == false
remove back tail block
tail? POS-C; == true
append block 'c
tail? pos-c ; false again

My note to Max's contribution:


- in REBOL, blocks of length 0 are not "impossible", that is, we 
have to use a nomenclature compatible with this fact

The issue is that index 0 is looking backwards... that doesn' map 
to any good reasoning.  In fact it creates many weird inconsitencies 
in the model, when you try to describe it.

 - it may not be a "weird inconsistency", but it is almost imposible 
 to describe to a newbie in a reasonable way

empty blocks are not impossible to describe.  but all functions will 
provide special cases to manage them (return another infinitely small 
block or none, or raise an error) because as such, they are vectorially 
equivalent to null.    an empty  block is just a starting place without 
any room to move.  when only looking forward, it is exactly the same 
as the tail (when you read my original post) and hence in actual 
code, tail [1 2 3] and  [ ]   are exactly the same, if only positive 
indices are being used.

Keeping R3's behaviour is certainly a possibility.

anyhow I always considered negative indices to be a bad idea.  I 
find SKIP and BACK convey much better meaning, because they are inherently 
directional (vectorial) by nature.  


IMHO  negative indices should have been implemented like in python, 
where they count from the tail, rather than "curren" position.   
they are MUCH more useful, they would be used daily by me (and most 
rebolers) in such a case.

To me, the description here http://www.rebol.com/docs/core23/rebolcore-6.html#section-1.1
is logical, but I'm not a scientist.

it can still change for R3... very little code uses negative indices. 
 


 its already incompatible with R2, so we might be better off finding 
 the proper incompatibility than trying to "wing it" in the name of 
 continuity.   same for Red, it is free to be better than its pre-decessors. 
    


python's  negative indices where a revelation when I used python. 
 It was one of the very few redeeming features I found it had.

I'd rather add an offset-based PICKZ than a PICK-BACK :)

From R3-alpha world:

Carl:	It is zero based for math computation reasons. If you think 
that is wrong, we should discuss it and correct it soon.	11-Oct-2007 
6:03:15 PM

Carl:	Rest of REBOL is one based for this reason: first = pick series 
1	11-Oct-2007 6:03:33 PM

Carl:	The solution we discussed, but have not yet implemented (but 
it is easy to do) is to add a PICKZ (or PICK0) function.	11-Oct-2007 
6:05:41 PM

BrianH	: Those math computation reasons are why I prefer zero based. 
I tend to use SKIP instead of AT for the same reason.	11-Oct-2007 
6:06:09 PM

BrianH	: Please add PICKZ and POKEZ - I liked them in rebcode.	11-Oct-2007 
6:06:46 PM

aha, still not done, I would need to find a name for the zero-based 
counterpart of POKE...

I had this idea when under the shower, so it is still a bit wet: 
DOWNPICK

Negative indexes applied from tail of series could be a good option, 
that would help replace the `back tail series` idiom.

Ladislav, REBOL doesn't have a naming convention that handles 0-based 
addressing. If you can come up with better names, hopefully just 
as short, go right ahead.

PICK0 or PICK-0 might do. Definitely not ELECT, since the kind of 
selection that is done in elections is not related to the type done 
when you "pick" something. In US English there is a colloquial term 
"pick out of a lineup", which relates to series in a way that noone 
would think of the term "elect" (unless they are convinced that all 
politicians are crooks). PICK and POKE are from Basic, old-school 
tech terms that are more closely related to assembly instructions 
than they are to any high-level operation.

A native is pretty much called directly, maybe a little argument 
marshalling but that's it. An action dispatches to a handler associated 
with the type of the first argument, one of a table of such handlers. 
The action itself doesn't do much - everything is done by the handler.

BrianH: just curious - why don't you like negative indices starting 
from the tail of the series? Some ppl expressed here, that it is 
nice feature in Python. Do you see any negative consequences with 
such a design?

TAIL position exists without pointing to any value

 - the way to imagine it is to think it as pointing to the END! value. 
 (this solves the mistery of what END! is in REBOL.) You can imagine 
 END! as being the same as the NUL character in a C string.

Re: python and negative indices... python does not have series positions, 
so they can afford doing that. a REBOL-like language simply can't. 
it's much simpler to just write "pick tail series -2" for eg.


perhaps one other way to look at this problem is to only have positive 
indices, and have REVERSE (or similar) return a value that is not 
a copy or modification of the series, but simply lets you look backwards.

the most sane way to make a decision here is to come up with use 
cases, probably. then we can intuitively see what is wrong.

The problem with R2 can be easily seen with:

>> b: skip [1 2 3 4 5 6 7 8 9] 4
== [5 6 7 8 9]
>> for i -4 5 1 [print [i pick b i]]
-4 1
-3 2
-2 3
-1 4
0 none
1 5
2 6
3 7
4 8
5 9


Now, you could also say that you should never iterate over a series 
using indices. But, then why have PICK at all?

I can tell you why I don't like the idea of continuing from the back 
of the series.

Most data is not in a cyclic group, it is finite so definitely at 
the beginning you do not expect it to continue from the tail.

And a PICK/BACK refinement is also not an option?

You do not necessarily have to iterate over the series. I look up 
some values in a table and retrieve or store data at that index of 
a series.

But than again I use subscripting b/(i) over 'pick, maybe I must 
go back to REBOL School  ;-)

the most sane way to make a decision here is to come up with use 
cases,


I can't agree more. What I am wondering is: have rebolers ever hit 
that specific issue (0 gap producing a calculation error) at least 
once? My feeling is that it is an extremely rare error case (but 
a nasty one) that could have been mitigated by just making PICK raise 
an error on 0.


Now, you could also say that you should never iterate over a series 
using indices. But, then why have PICK at all?


Right, IMHO, you shouldn't iterate over a series using indices, unless 
you have a good reason for it. In the general case, people should 
use "navigational" functions to walk through a series. PICK is ok 
as long as you don't PICK on 0.

perhaps one other way to look at this problem is to only have positive 
indices, and have REVERSE (or similar) return a value that is not 
a copy or modification of the series, but simply lets you look backwards.


At first look, the implementation overhead wouldn't be high for supporting 
such feature and that could solve our PICK issue, elegantly I think. 
My only interrogation about it would be: will we be able to easily 
reason about such "reversed" view of series once mixed with "normal" 
series in the middle of our code or would it confuse many people?


Anyway, I think this one should be in the short-list of the possible 
solutions.

Andreas: your gist example nicely shows the continuous vs discontinuous 
numbering. I won't dispute   that continuity is better than discontinuity, 
but the more I think about it, the more I am convinced that the R2 
trade-off is "better" (in the sense that it is more easily understandable 
by users).

R2:
    >> values: [A B C]
    >> pick at values 2 -1
    == A
    >> pick at values 3 -2
    == A

R3:
    >> values: [A B C]
    >> pick at values 2 0
   == A
   >> pick at values 3 -1
   == A

I think that what we are seeing here is the frontier between academic 
and practical design choices. I am all for following academic principles 
as long as they are not "too" detrimental to practical usage. I would 
draw the line at the point where most users would get lost. I believe 
that this is a dangerous pitfall in language design if you aim at 
a widespread use.

I did some investigation over the rebol.org library and a few other 
available R2 apps a while back, I think it's basically always /-1 
and /-2 for negative path indices.

IMO, as long as we are in a 1-based system, s/0 will be confusing 
to most users.

Pekr, the reason it works for Python is because their series are 
always referenced from the start, so negative indices/offsets would 
otherwise have no meaning. REBOL has offset references to series, 
so negative indices/offsets already have a meaning for us.

The observation that we are using "a 1-based system" alone doesn't 
help you in any way with that.

I must have missed the proposal of BASIS?, but the fact that it would 
be a function or variable implies that it would be used to detect 
a global setting, like system/options/binary-base. Global settings 
like that have proven to be a universally bad idea in practice. Local 
settings are better.

In a pure 1-based system, you have neither 0 nor negative indices.

Doc, it's not the "learning a new meaning", it's losing the meaning 
they already have. We need the meaning we have, we need to be able 
to reference offsets from before the current position.

Andreas, oh good, whew. In R3 we're already having to replace all 
system options that affect MOLD with options to the MOLD function 
itself. It would be a shame to have to do the same for all of the 
indexing functions. Nonetheless, that meaning of BASIS? doesn't make 
sense for the name either, so I don't like it.

Yes, but that's not how my brain see it by default, I need to make 
a conscious effort for that. Also, I might be tempted to then use 
0 to access the first item instead of 1...This is a pitfall where 
almost every new (and maybe some older too) will fall in.

If we do R2's behavior, make sure that PICKZ and POKEZ exist so I 
have something to use. They can call PICK and POKE internally. I 
need something that does computed indexes/offsets, and I can't afford 
to have a hole in the list (0 for R2), and I can't count on the port 
scheme supporting SKIP.

Just erroring out on index 0 is ann improvement.

Making "pick 1" and "pick -1" return the same element is an improvement.
R3's behaviour is an improvement.

R2's messy behaviour with a clean set of SKIP, PICKZ, POKEZ is an 
improvement.

and I can't afford to have a hole in the list

 Brian, could you give us some short code cases where this was a problem 
 for you? This would really help.

Just erroring out on index 0 is ann improvement.

 That's my intention for Red until we get a consensus on a better 
 overall solution.

Sorry, I didn't mean off in the sense of false, I meant short for 
an offset. Any place where you have computed indexes can have a computation 
that turns out to be less than 1, especially if your base position 
is offset.

Error out and add PICKZ, POKEZ while at it. That would at least give 
us a base to work with.

Remember, R3's behavior isn't done either. There's a standing consistency 
bug, for instance. If we come to a consensus, R3 is likely to adopt 
it too.

Usually, the number I'm passing to PICK or POKE is computed elsewhere. 
That was a one-line example for simplicity.

We should write a short-list of possible options that would solve 
the whole issue and see if we can get a large consensus on one of 
them. Anyone kind enough to extract the different options we've discussed 
and put them somewhere online with the main pros/cons?

I use computed indexes for computed lookup lists, such as for precomputed 
intermediate results of computations, translation tables, etc. If 
the computation uses signed numbers, you have to do an offset base 
position to get the results from the positions less than 1. Having 
a hole slows down the computation because it has to be handled in 
mezzanine code. PICKZ/POKEZ would actually be better for most of 
these situations because the computations work better with 0-based 
numbers (modulus, for instance). It's pretty common in code that 
actually *needs* to use PICK/POKE on series.

It's pretty common in code that actually *needs* to use PICK/POKE 
on series.


That's the sticking point. We can categorise the uses for PICK/POKE:

(a) with large positive literals
(b) with literals -1, -2
(c) with computed indices and series in "head position"
(d) with computed indices and an offset series

Having a hole slows down the computation because it has to be handled 
in mezzanine code.


Have you ever had to actually write such "0 gap" workaround in your 
code, or is Carl the only one that had ever been bitten by this isssue?

I'm pretty sure I ran across this problem when trying to implement 
a triple store ages ago (and ended up switching to a different implementation 
approach).

I'm not as much of a math head as Ladislav, so I can't always do 
the algebraic transformations necessary to switch to unsigned arithmetic.

Yup. But that doesn't mean that I can switch to unsigned arithmetic. 
For that matter, sometimes switching to unsigned arithmetic means 
mostly indices at the top and bottom of a large range, skipping the 
middle (basically what you'd have to do with the Python model). By 
"large" I mean allocating a 2GB series or more and only using the 
100MB or so at both ends. Not always a good idea.

Hard to answer that now, but as the optimizing compiler will have 
a plugin architecture, it won't be limited, in the sense that you'll 
be able to add your own specific code optimizations, that if generic 
enough, could make its way into the official   repo. If the optimization 
processing time is short enough, it can be used for the JIT, else, 
it will be used only in for AOT compilations.

In that case, it would help if PICKZ/POKEZ were the actions because 
they don't have the 0 hole, and we can add PICK/POKE optimizations 
that apply an algebraic transformation to change the call to PICKZ/POKEZ. 
Implementing the hole in the action has overhead that the optimizer 
wouldn't be able to undo. It would be able to undo the hole implementation 
in a wrapper function though by inlining the code then optimizing 
it away. That would make up for the problems caused by putting the 
0 hole in there in the first place.

Though a type inference action-to-native transformation would help 
too :)

For PICKZ/POKEZ, we'll see if they still make sense once we come 
up with a global solution.

Most computed index situations use modulus, which generates 0-based 
numbers. If there's no 0 hole in PICK/POKE then you can do PICK/POKE 
NEXT, as long as this doesn't have side effects the way it does for 
port schemes or weird copy-on-write stuff; in that case PICKZ/POKEZ 
would be more of a convenience. If there's a 0 hole in PICK/POKE, 
you absolutely need a version of the functions without that hole, 
and having the extra advantage of being 0-based would be even better, 
an extra justification for their existence.

One occasion where you use computed indexes is when you do C-style 
fake multidimensional array access. As with C, however, the math 
tends to be 0-based because of the modulus, so a 0 hole really hurts 
here. You can do these calculations much more easily if your base 
series position is 2 (or plus 1 * (dimension - 1) for each dimension) 
because the 0's go back one.

Doc, Will you publish a Red/System standard function guide, listing 
all the functions, such as: print, print-line, length?, size?, zero?, 
form-type, ...

- "the most sane way to make a decision here is to come up with use 
cases": +1
- We are unlikely reach consensus: +1
- Have I ever been bitten by R2's behavior? No.


Great discussion on this. I greatly appreciate Andreas's points about 
common (-1, -2) cases, and his gist example. I'm good with throwing 
an error where silent incorrect processing would be worse. The concrete 
examples will be the best way to drive this, IMO. By concrete, I 
mean real-world, "this is code I actually wrote to solve a problem" 
stuff. If you can't post code for legal reasons, a generalized adaptation 
is almost as good. Make the most common uses behave correctly, even 
if there are cases they don't handle, and show how the other cases 
*can* be handled, even if it means slower mezz code. Then see if 
there's a better way to support them.

Why not have a special function to handle the 0 gap problem? The 
rest of the world can use the normal 'pick and 'poke and if you need 
the special ones they are available and behave as expected in those 
cases.

If we are going to have a 0 hole, make it an error, not return none. 
A quiet error is worse than a loud one.

I think that what we are seeing here is the frontier between academic 
and practical design choices. I am all for following academic principles 
as long as they are not 

too" detrimental to practical usage. I would draw the line at the 
point where most users would get lost. I believe that this is a dangerous 
pitfall in language design if you aim at a widespread use."

It's more of a "which is more useful" versus "which is less confusing 
for newbies". Best to find a balance.

As someone who is at say level 5 on the btiffin rebol scale what 
I am about to suggest may be stupid - if so be gentle :) Could we 
not create a different structure other than a series (an array or 
a matrix etc) that is 0 based and index driven and works like in 
most other languages for those who need that kind of usage?  Or could 
we introduce for series an index manipulation system such as this:s1/index: 
4, add s3/index 2, subtract s2/index 2, inc s3/index, dec s2/index, 
++ s1/index, s1/index: s1/index + off etc.

On a lighter note, Red seems to have something to do with basketball:

Ladislav, REBOL doesn't have a naming convention that handles 0-based 
addressing.

 - actually, REBOL does have a couple of general naming conventions, 
 which, therefore, apply th the 0-based indexing as well

Negative indices being back from the tail of the series: please, 
no, never.

 - yes, this needs some analysis to find out what may be wrong about 
 that idea. The starting point may be that the series tail is "ephemeral" 
 in a sense as I noted, while the head is not ephemeral at all.

Indeed, any modification of a series changing its length, in the 
middle of a loop over a "reversed" view of that series, would most 
probably provoke errors.

It's just a dialect for going in the opposite direction

I don't buy the "no right" argument. Romans had subtraction without 
0. It was a bad idea, but it was possible.

Ladislav, we don't need analysis on negative indices being back from 
the tail of the series. The concept only makes sense if you are only 
able to refer to a series from its head, thus negative indexes don't 
really have a meaning if you see the series as linear, so you decide 
to give negative indexes a meaning by looking at the series as a 
loop where the end wraps around to the beginning and vice-versa. 
If you are able to refer to a series frome a base position of somewhere 
other than at its head then you already have a meaning for negative 
indexes, and don't need to make up another.

Note that with R3-style bounds checking, the none value is considered 
to be roughly the same as a hole in the data. PICK past either end 
returns none. That means that PICK 0 returning none is basically 
a statement that there is a hole in the middle of the series that 
is just before the current position, and moves along as the series 
position moves along. Now, try to come up with a way to explain to 
newbies that this phantom hole in a series makes sense, or is a good 
idea.

At least triggering an error is easier to explain, it means that 
0 doesn't exist, like we're programming in Roman. It's (slightly) 
better than the 0 index existing but referring to a hole in the series 
(R2-style behavior).