AltME groups: search

Pekr, putting a trace before demo and it doesn't crash.  Very odd.

I tried the chat then demo trick under linux and Rebol crashed with 
a segmentation fault. Normally, the demo gives a script error. I 
also tried on Mac OS X but that was okay - the demo gave a script 
error.

and other issues http://twitter.com/rebol3

I think i will change again some habits in the way I code.


It seems that the try/except idiom has better perfs than EITHER if 
you use it  that way:
>> try/except [...frequent case...][...rare case...]

Try this and give me your results...

f1: funct [v][either zero? v [0][1 / v]]
f2: func [v][try/except [1 / v][0]]

print dt [loop 100000 [f1 1]]
print dt [loop 100000 [f1 0]]

print dt [loop 100000 [f2 1]]
print dt [loop 100000 [f2 0]]

Wow, I never realized what overhead TRY/except has, but that overhead 
seems to be fixed. As long as you can safely run until an error without 
having to roll back changes, that could be a good strategy. Sticking 
to the R3 safety model (generating errors before it matters with 
functions like ASSERT) you can make code that runs faster in the 
default case, and safely always.

Or you can look at the source of DECODE-CGI in R2 and make something 
useful of it for your purposes. Nothing built into R3 though.

And that parse rule would work in R2 or R3. We can't R3-optimize 
it yet with the change rule since change rule paren hasn't been implemented 
yet.

I built my own decoder for the query-string and it works pretty good. 
 But I was wondering what the /only refinement is for  construct. 
 This looks new in R3.

They probably had to break away and do some projects to pay some 
bills.

CONSTRUCT by default translates the built-in logic words to logic 
values, unset values to none, lit-words to words, and lit-paths to 
paths. This makes it easier to use it to handle headers. CONSTRUCT/only 
doesn't do any of those things - it only constructs the object with 
the data provided.

Thanks for the kind words, Brian,

But I am not sure we can yet agree on the issue involved. I've added 
my response to your detailed analysis:
  http://www.curecode.org/rebol3/ticket.rsp?id=1506

Without the ability to catch _all_ errors, REBOL is vulnerable to 
sudden, unexpected and unhandlable crashes.

Great analysis Brian. And I think Sunanda's point about arbitrary 
code is the key element for me. Some things we can address with language 
"Don do that", but error handling like this is very important to 
get right, and explain clearly (with examples of how to write robust 
apps).

I've given it some more thought and think I have combined two issues 
that could better be dealt with separately:

1. TRY and ATTEMPT as ways of protecting my code:

Generally, the current behavior is surprising but in most cases acceptable.

It is rare that something as stupid as attempt [exit] would make 
it to production status. So, in many cases  no problem. [perhaps 
an odd problem if my app is generating code (eg a genetic algorithm). 
But knowing the problem, that could be guarded against]. 

2. TRY and ATTEMPT for protecting arbitrary code

As Gregg suggests, that needs a careful, systems-level design to 
get right and to make robust. REBOL3 offers some help (SECURITY policies, 
ATTEMPT, TRY, CALL) but isolating code into a safe sandbox, and recovering 
from any possible error/lapse/attack is not something the language 
alone can provide.


Still, making TRY and ATTEMPT fully predictable [succeed or return 
an ERROR] would be most welcome!

The trick, Sunanda, is that the code that you want ATTEMPT to catch 
isn't actually erroneous. EXIT itself never generates an error - 
it basically is like a fancy goto that has to be caught by a handler. 
However, the handler can trigger an error (the one that you see) 
if the goto supposedly isn't handled; actually, all such goto functions 
are always handled and never cause an error directly, it is the handlers 
that cause the error.


The problem is really one of perception: Code that you think is erroneous 
(but isn't) is not triggering errors (and shouldn't), and the code 
that is triggering the error is not actually inside the code block 
that you are passing to TRY or ATTEMPT. Perceptual problems are tricky 
- either you have to change your perceptions, or rewrite the basic 
semantics of the language. Which is easier in this case?

But 
  exit
_is_ an error on its own.

It is hard to see why ATTEMPTing an error "is never an error on its 
own", especially as trying it by itself in a console does create 
an error.  
  exit  ;; REBOL says is an error
  attempt [exit]  ;; Brian and I disagree if this is an error

it creates a horribly inconsistent mental model.

You could create a trivia/gotcha quiz arond this. Which of these 
stop with an error?
  attempt [throw]
  attempt [throw 0]
  attempt [throw/break]
  attempt [throw/break 0]
  attempt [throw/name]
  attempt [throw/name 0 0]
  attempt [throw/name 0 'x]

And the first two THROW/name examples are wrong too, because of argument 
compatibility.

THROW 0 is always correct in and of itself. It's not THROW's fault 
you called it outside of a CATCH, it's yours.

No, but it works the same, just faster. Almost all functions that 
had a [throw] attribute in R2 (or needed one, like ATTEMPT) were 
converted to natives in R3. We're still waiting for the new equivalent 
of the [throw] attribute to be added to R3 - right now USE, OBJECT 
and CONTEXT are the only built-in functions that still need it, afaik.

Note that the ticket was marked as a problem, not dismissed (yet). 
The problem was explained in the comments, and more thoroughly now 
in my more recent reply there.

Actually, the first is an error too. It turns out that the way BREAK, 
EXIT, RETURN, THROW, HALT and QUIT work is by throwing pseudo-errors 
that are technically instances of the error! type, but not really 
errors. The ERROR? function disables that throwing for all error! 
values, even the fake ones. It's an error in ERROR?.

From my perspective, attempt [break] is working correctly. ATTEMPT 
has nothing to do with the BREAK, it simply passes it to the outer 
context (if you can say that) and then the BREAK doesn't hold up, 
because it's not inside a function. Seems simple to me.

>> a: does [attempt [exit] 1]
>> a
>>

Looks fine and dandy.

Once, and it did.

Twice, and it did.

The question (only for Carl) is whether task-local runtime contextual 
information can be made available to BREAK, EXIT, RETURN and THROW 
to let them know that they are going to be handled by the console, 
which would let them throw the error that the console should be throwing, 
on behalf of the console. Basically, letting them be more polite. 
The problem is that it is *really* likely (only Carl would know) 
that this would slow down all function! and closure! calls, all loops, 
and every call to DO (even indirectly, as in IF) by quite a bit. 
It could make R3 a lot slower (maybe several times slower).

I'm not sure what you would expect here. Wrapping break and continue 
in attempt has no effect, because we know there are no errors inside 
the attempt block and the context outside is correct for break and 
continue. The attempt has absolutely no effect here.

I'm curious now: How do those functions know they are inside loops 
and functions?

And in REBOL there is no way to statically determine whether an exception 
will be handled without tracing the code. You could in theory determine 
it at runtime (from the internals), but that would have overhead. 
The question is how much overhead.

ATTEMPT and TRY push a handler for exceptions with codes over 100 
on the task-local handler stack. Those exceptions are generated by 
DO MAKE error! (or code called by natives with the same effect).

Thanks for the repeated attempts to explain, Brian!

For me, the work around is:
    error? try [....]
That seems to work in all cases I have tried.


[Ironically, 99% of all the REBOL code I have ever written uses ERROR? 
TRY --  ATTEMPT did not exist when I started REBOL, and my fingers 
have learned to type the idiom. It's only with R3 that I'ved tried 
to update my error handling.....Seems I should stick with my older 
ways for a while...:)]

The gist of it: I think most control transfer functions should be 
unbound globally, and only bound in code executing within their respective 
control transfer handler.

For example, BREAK should be unbound globally, and bound only within 
loop bodies.

That means, an erroneous use of e.g. BREAK would simply result in 
a "has no value" error, just as the use of any other unset word. 
And this error would of course be nicely caught by ATTEMPT.

And I also think there's an easy way to fix things that they work 
as you expect

I also think it is reasonably straightforward to implement and has 
no significant impact on performance. But I don't have the time to 
write down why I think this, at the moment.

Sunanda, that error? try [....] code you keep mentioning doesn't 
work in all cases you mention. The problem is this:
>> error? break
== true

This is an error (bug#771 and bug#862): It should not return true, 
it should break and then later on trigger an error for an unhandled 
break. To illustrate why:
>> loop 1 [error? break 'buggy]
== buggy

That should have returned unset, not continued. The BREAK is supposed 
to halt execution and resume it somewhere else, either after the 
enclosing loop or at the top level. The same error is affecting your 
error? try [....] code, which is not behaving correctly. 


Your problem, Sunanda, is that you are thinking that ATTEMPT is not 
catching bad code (bugs) that it is supposed to catch - this is not 
strictly true. ATTEMPT isn't supposed to catch bugs, it is supposed 
to catch triggered errors. And a lot of logic bugs don't trigger 
errors, because the code itself is technically correct (what REBOL 
can check for), but not in accordance with the programmer's intentions 
(which REBOL can't know).


Now in this case the logic bug is that you have forgotten that the 
purpose of the BREAK, CONTINUE, THROW, EXIT and RETURN functions 
is escape from the normal flow of the code ("structured" gotos, aka 
non-local returns, exceptions, ...) - you remembered this about HALT 
and QUIT - and didn't realize that the error you are trying to catch 
with ATTEMPT is not in the code that you have wrapped in the ATTEMPT: 
It is not bad code. The error is triggered by the code on the other 
end, the handler, which is not wrapped in the ATTEMPT at all.

So you need to realize that this code is technically correct:
>> attempt [break]
** Throw error: no loop to break
but doesn't do what you thought it should.

And this code is absolutely incorrect:
>> error? try [break]
== true

but doesn't trigger an error because of a weird design quirk in R3 
that is arguably a bug, but not necessarily fixable (see bug#771 
and bug#862).

I think we need a way to pass that block so that 'a and 'b can evaluate 
to none

It is clear why it setting a and b to 1 but we need to have something 
a bit safer.

We want that behavior and much code depends on it. You can add the 
none explicitly if you need that value.

And CONSTRUCT doesn't evaluate anything, so safety is not an issue.

Still we need to check it and we could simply have another switch 
that could fix construct to be even more superior.

But let's you access the elements as if they are an object and can 
already exist in a block style spec that can be passed to context.

On the other hand, you can APPEND to objects and maps in R3, so you 
don't need the block at all.

And appending to objects doesn't chain assignments.

Paul, creating your own func is the way to go for that. And emails 
can be very complex. REBOL doesn't try to validate against the true 
grammar, it just looks for @.

cc#1509 -- Thanks Brian for consolidating a whole handful of bug 
reports [and/or misunderstandings].
That should help focus the core RT crew into clarifying / fixing.

I think that RETURN, EXIT and BREAK, CONTINUE should be only available 
in their respective contexts (functions, and loops). CATCH/THROW 
will need special treatment. QUIT, Q, HALT can be left as is.

This said, and leaving the discussion of improved semantics aside, 
I think that the implementation of improved error causation as described 
by Brian in bug#1506 would be very worthwile and should be pursued.

Andreas, I still think that issue should be resolved by introducing 
another function since the cases where we would need such assurance 
is minimal in comparision to the overall application of attempt. 
 Why add more overhead to 98 precent of the cases when you can adapt 
to the 2 percent with another function or just implementing your 
own check and parsing.

Andreas, I don't have a problem with that solution in principle. 
It's just that it wouldn't work, and wouldn't be task-safe. The handlers 
for those functions would be task-local, the code blocks not. Plus 
it would break code that uses code block references rather than nested 
blocks, code that uses those functions through function values, and 
any function with the [throw] attribute (which we will be getting 
back in R3 with different syntax), and all of those exist in R3 mezzanine 
code. Plus there's all the extra BIND/copy overhead added to every 
call to loop functions, startup code, etc., and don't think that 
you won't notice that because that can double the memory usage and 
executiion time, at least.


The solution I proposed in the ticket comments is to have DO, CATCH 
and the loops set a task-local flag in the interpreter state when 
the relevant functions become valid, and unset it when they become 
invalid, then have the functions check the flag at runtime before 
they do their work (which they could because they're all native). 
This would be task-safe, only add a byte of task-local memory overhead, 
plus the execution overhead of setting and getting bits in that byte 
in a task-local way. It's the execution overhead that we don't know 
about, whether it would be too much. It would certainly be less than 
your proposal though.

Carl is the authority on subtle implementation overhead, but for 
gross implementation overhead anyone can tell by just using the profiling 
tools and extraploating. And what you are proposing is definitely 
in the gross overhead category.

However, CATCH/name and THROW/name would need the additional memory 
overhead of a single block of words per task in the dynamic solution 
to store the currently handled names.

It might be hard to believe, but R3 has gotten so efficient that 
BIND/copy overhead is really noticeable now in comparison. In R2 
there were mezzanine loop functions like FORALL and FORSKIP that 
people often avoided using in favor of natives, even reorganizing 
their algorithms to allow using different loop functions like FOREACH 
or WHILE. Now that all loop functions in R3 are native speed, the 
FORALL and FORSKIP functions are preferred over FOREACH or FOR sometimes 
because FOREACH and FOR have BIND/copy overhead, and FORALL and FORSKIP 
don't. The functions without the BIND/copy overhead are much faster, 
particularly for small datasets and large amounts of code.

It's funny: While regular R3 code looks a lot like regular R2 code, 
optimized code looks a lot different because the balance of what 
is fast and what isn't has shifted. At least regular R3 code looks 
a lot more like optimized R3 code than regular R2 code looks like 
optimized R2 code. This is because we have been focusing on making 
the common, naive code patterns more optimized in R3, so that people 
don't have to do as much hand-optimization. The goal is to make it 
so that only writers of mezzanine and library code need to hand-optimize, 
and regular app developers can just use the optimized code without 
worrying about such things.

And if you re-read that, you will notice that it's precisely what 
you later describe.

Btw, non-local code blocks are a common optimization trick in mezzanine 
code, one which shows up a lot in Carl's code. It's probably the 
reason why REBOL supports the concept in the first place. And I've 
written code in REPLACE that uses the BREAK function as a function 
value, though I haven't checked whether other people use this trick 
:)

It seems so. I've asked Carl to look at those tickets and chime in, 
so we'll see what he thinks.

If you want to see how weird really optimized R3 code can get, take 
a look at the source of LOAD and IMPORT - they are probably the most 
heavily optimized mezzanine functions. For the most part the rest 
of the mezzanine code is written for readability and maintainability, 
and the language optimized to make readable code fast. It's a good 
tradeoff :)

IMPORT uses code blocks as a way of reusing duplicate code, though 
it might not be affected either. And REPLACE would be affected because 
'break wouldn't be bound at the point it is used: Being in a function 
isn't enough, it's outside of the loop. BREAK is used to break out 
of loops, not functions.

That means that the BIND/copy overhead for BREAK and CONTINUE would 
happen at every call to a loop function, not just FOR, FOREACH and 
REPEAT. And 'break and 'continue would become keywords rather than 
function names, unable to be used for loop-local variables.

LOOP, WHILE, FORALL and FORSKIP don't currently have BIND/copy overhead. 
Which is why they are used a lot in R3 :)

I don't have the time now to provide examples, I'm afraid, must run 
an errand. Try it yourself and see what you find out :)

REMOVE-EACH and MAP-EACH already have the BIND/copy overhead though.

Only verified it for FOREACH yesterday and assumed that the other 
loop functions that need binding would also copy.

But all the loop functions that take a word argument have BIND/copy 
overhead, and the rest don't.

Already proposed in CureCode, and the R2 2.7.7 version does that 
already.

Andreas: what you ask for has been discussed extensively when R3 
was started, by me, Ladislav and Carl. There are a number of disadvantages 
to that, starting from the fact that you need to bind/copy a lot 
more than you do now (eg. inside CATCH). It would also, unfortunately, 
not work in cases like this:

Andreas: "I think that RETURN, EXIT and BREAK, CONTINUE should be 
only available in their respective contexts (functions, and loops)." 
- that is what I preferred too, but Carl did not like it (binding 
takes time).

Nevertheless, at least for Return and Exist this does not look like 
an issue, since binding is likely to occur anyway in these cases

and, the speed difference for Return and Exit may still exist, but 
only if the respective function does not have any parameter

I think that the implementation of improved error causation as described 
by Brian in bug#1506 would be very worthwile and should be pursued.

 - again, this is a thing I discussed with Carl quite long ago, but 
 it seems, that Carl disliked the fact, that the implementation would 
 have to be more complicated, than it currently is, while the effect 
 Sunanda would like to achieve is probably achievable even now

We've been able to find out a lot from stuff posted so far and the 
scientific method though :)

I think a lot of the problems would be solved by just letting developers 
plug in their own recovery code into the console handler, and that 
would be more efficient too.

(and, you omitted Try)

Ladislav: actually, I agreed with you.  RETURN and EXIT should be 
definitionally scoped!

I'm not saying that. I'm just asking if there is any new development 
in error handling between A97 and A98.

And also, because it was a hot topic above... with many of the key 
people chatting on it. So, good time to get it finalized.

The example just below that sentence with definitional scoping: I 
agree on that.


I ran into exactly this issue just last night in an attempt to wrap 
some function bodies in a primitive callstack mechanism (R2 code) 
so I could debug it easier. The debugging mechanism kept crashing 
there, because it ran straight through the first 'return and "returned" 
twice.


R3 of course doesn't need function scoped return for that reason, 
since I can already see the stack trace on an error, but for other 
reasons (correctness?).

Yes, but word, :word, and 'word are taken.

Just so that we can do some dynamic building of blocks with set-words 
and pass it to construct and not worry about a set-word getting assigned 
a value from the chain.

Paul, just iterate through the block before you construct it and 
whenever a set-word! is followed by another set-word! insert a none 
in between.

Write such a function and offer it to be included as mezzanine.

Chris, I had an issue where I was putting data into a block dynamically 
and then had a skip pattern over it so that every other word was 
turned into a set-word and then passed to construct.

Paul, set-word!s are syntax for SET. [a: b: 2] is syntactical short-hand 
for [SET 'a SET 'b 2]. SET returns sets a word to a value and returns 
that value. If you don't want to pass the result of (SET 'b 2) as 
argument to SET 'a, then just don't do it.

Great, then write /as-is as a mezzanine for now and propose it to 
be included.

And now I would ask you why add that function to R3 when you already 
pass a spec to contruct in the first place?

So if I went to falcon or Lua and said do that and they did they 
could say they could handle such code safer than REBOL could.

let's say you have a service oreient protocal that is sending unencrypted 
blocks of data that is getting sent into contstruct on the other 
end.   All I have to do is capture it midstream alter the block and 
forward on and I can cause your code to behave differently.

Paul, you don't create services that carelessly evaluate data that 
is potentially manipulated by an attacker. That's a security hole 
and bad coding. That's one of the good things about dialects.

Agreed, and that's what this is - a dialect - as Rebol's sop is chaining...

yes, we do. with PARSE and dialects.

And that's what you should do too, Paul.

And no, it's not "all of us", as none of us have those specific needs.

I vote against the proposed change to CONSTRUCT. I believe the need 
is better served with a mezz. If it becomes popular and useful, then 
consider making it part of the standard distro.

Carl, if you make RETURN and EXIT definitionally scoped, it would 
help if the throw: attribute would disable that, so functions like 
USE would work properly. Also, please keep BREAK, CONTINUE and THROW 
dynamically scoped - most of the functions that they break out of 
don't do definition, so making them do so would add unwanted BIND/copy 
overhead.

Note to the "Should we catch unwinds?" question in the http://www.rebol.com/r3/notes/errors.html
article - the subject of the CureCode #1506 is, that it is (or should 
be) possible for the interpreter to find out, whether the unwind 
would cause a "No catch for throw" type error, and, that Try should 
catch such an "unhandled" unwind.

I wrote a few comments to the blog, explaining the advantages of 
the definitionally scoped RETURN and EXIT proposal. The optional 
function attribute we would need would not change them to dynamically 
scoped, it would just tell the function to not redefine RETURN and 
EXIT within, allowing them to keep their existing bindings. No dynamic 
scoping of those functions would be needed at all.

(Continuing from "Rebol School".)


I've thought about TRY/EXCEPT too, since it popped up a few days 
ago. My thoughts are more about design. Why do we have TRY? Why not 
make /EXCEPT a refinement of DO? DO can do any type, TRY only works 
on blocks. If you wanna do a script on the net, and it can go wrong, 
we have to write:

try/except [do <url>] [<handle error>]

With /EXCEPT on DO, it could be:

do/except <url> [<handle error>]


My point is: is it good to have all these words in the language? 
It may add depth, but it also add confusion and complexity. Maybe 
the depth could still be there with fewer words, and the language 
would be easier?

I don't miss a simplification of DO/TRY as much as structures for 
sequential tests that may or may not fail. REBOL doesn't have anything 
here, so you have to roll your own. Say you're trying to connect 
somewhere in a protocol and one of 50 things can go wrong, so you 
have to state 50 tests, 50 error messages, 50 exit routes. That's 
a lot of lines of almost identical code.

There may be another concern with this. How do we get the error in 
the except block, so we can handle it? A common way is:

if error? result: try [1 / 0] [probe result]

This doesn't work:
(Remember to clear result, if you did the above first.)

result: try/except [1 / 0] [probe result]

And having /EXCEPT on DO is the same problem.