AltME groups: search

Examining the core-tests suite results I wonder, which alternative 
do you prefer:

a:

>> for i 1 3 1 [print i if i = 1 [i: 2]]
1
2
3
== none

b:

>> for i 1 3 1 [print i if i = 1 [i: 2]]
1
3
== none

it's like a loop

actually that's a construct I miss....

eg. in dbase we had

do loop [ ............ ] endloop or something like that.


but you could exit anyway inside that block to restart the loop like 
this

do loop [ .... loop .... ] endloop

we only have break which exists a loop

need a break^2

Can be made even more elegant, since FOR is a mezzanine

CC being a function, would that mean RETURN or BREAK could not be 
used?
eg.
	myfunc: func [][
		for n 1 5 1 [
			cc [
				if mycondition [return]
			]
		]
	]
(until converted to native?)

A different idea, instantly causing any R2 cycle to "understand continue":

cc: func [
	{convert a cycle body to "understand" CONTINUE}
	body [block!]
] [
	compose/only [catch/name (body) 'continue]
]

continue: func [[throw]] [throw/name none 'continue]

; usage:
for n 1 5 1 cc [
	if n < 3 [continue]
	print n
]

Any name improvements (the 'cc name looks a bit unnatural)

Very understandable, but a little bit too long for my taste

I can't think of a better name. The reason is because it shouldn't 
be there at all. I think all the rebol loop constructs should be 
adjusted to use CC by default.

Well, there's a wish...

On the balance of things, I think it would make things better; code 
more readable and more consistent with R3 code, so I'm for it.
A better name for CC is definitely needed, though.

Well, that just means that the name of a feature is really part of 
the feature.

That you think it's not implemented in any loop construct (even under 
a different name) ?

Well... I think we should still aim high and make a wish.

How about a C-AWARE name, it looks a bit shorter, would it sill be 
acceptable?

CC makes me think of "Call with Continuation", but I agree that MAKE-CONTINUABLE 
is a little long. Just CONTINUABLE? Without 'continue in the name 
somehow, even if abbreviated, you lost the connection to the keyword.

Ladislav, your request to make HALT catchable was in CureCode already 
as #1520. Please add your concerns in a comment there. That ticket 
needs a use case to catch HALT in addition to everything else.

There is no need to worry about the name CC - it won't be used. A 
CATCH option is more likely.

CONTINUE is more likely to be added as a native in R2 than it is 
as a hack. We already have BREAK, so the mechanism is there.

yes... which is why putting a catch/all when executing code in a 
sandbox (I often use CONTEXT [ ]   to do so)  is very usefull to 
prevent the outer script from halting or quiting because of some 
externally loaded code..

An R3 suggestion.....

-- Ability to execute HALT, QUIT etc should be controlled by SECURE

-- DO should have a refinement DO/SECURE ... meaning the DOne thing 
cannot change any SECURE settings
That would be a big step towards safer sandboxing.
-

maybe we could start thinking about using a new word for what we 
are talking about extending CATCH to do... its starting to look like 
a long list of groceries for a single word.


reading Andreas' notes on 'CATCH I agree that it should only manage 
throws.  

/all  should mean catch any named or unnamed throw


/quit /unwind and other proposals should fall into another function.

CATCH is a sandbox already, just not a very good one. SANDBOX would 
need to be mezzanine because it would need to provide wrapper functions 
as well. And we need the subtle differences in the various CATCH 
options for different circumtannces.

Ah, a SANDBOX function sounds like a good idea.

brian  I don't agree... all of that whould be a separate function. 
 its a different level of failure control.

I don't care if it is a separate function, I am just using those 
names for the concept. There are practical reasons to put these in 
CATCH, but it's not strictly necessary. But SANDBOX would *have to 
be mezzanine* to be able to do its job, so we need native functions 
or options for it to call.

But they're not better placed elsewhere. CATCH already has the code 
in it to do this task. Putting it elsewhere would mean moving that 
code to an internal function that CATCH calls. Not a bad thing, really, 
but that is the practical reason. Also, we don't want to polute the 
namepace in lib with too many predefined words that could be better 
used for other functions.

Maxim, the reason that SANDBOX would need to be mezz is because there's 
more to sandboxing than catching stuff. You also have to set up a 
context full of safe functions for the sandboxed code to use, including 
wrapper functions. That is mostly mezz work. We don't want to waste 
the word SANDBOX on anything less than that.

but the mezz code won't be using my version since it's also definint 
the same word... for example (algo code, not explicitely working 
r3 code):

; pre-mezz exported
my-func: does [print "a"]

mezz: mezz-module [ 
	my-func: does [print "b"]
	print-my-func: does [my-func]
]

this is what I expect to happen:

>> mezz/print-my-func
== "b"

but I want(need)
== "a"


now I'm not toally clear on how the mezz is being defined, (a do 
block, a module, a contex, whatever) but it woudn't make a difference, 
AFAIK.

DO/SECURE ... thanks for the clarification. Brian.


What could be quite useful then, would be DO/SECURE having a parameter 
block of SECURE settings. But it can only tighten existing SECURE 
settings, not loosen them.


The DONE thing would execute under the tighter policies; and they 
would revert when it exited/returned/quit/crashed etc.

That would be a very useful tool for creating sandboxes.

I agree sunanda, restrict file reading to a specific directory when 
loading application plugins, for example..

Explicit override of a module/extension loaded after the standard 
mezz.

of a -> in a

SECURE is a mezz in R3, so DO/secure would need to be implemented 
in the intrinsic DO*. But It might be better to add as a SECURE option 
instead.

maybe the policy system on which secure is run could use a stack... 
so that we could do secure/push  secure/pop style pairs.

All this might mean making a MAKE-TASK* intrinsic similar to MAKE-MODULE*, 
but we can't know that yet.

yeah... but something like:

secure/do [... policies ...] [

	... code ...

]


looks like the best way... though the current native security system 
doesn't support this natively.   the set-policies/get-policies do 
not operate within a stack frame but within a global heap.

Formal proposals go in CureCode (eventually, after discussion in 
!REBOL3 or on a blog or wiki page).

anyhow... its just that more groups is faster than really big ones, 
so separating the use and proposal discussions seem to be a good 
thing in my mind.

I would say, that this looks as a terminological misunderstanding 
to me.

I explain it for RETURN, but the case of THROW is similar. Normally, 
when you use RETURN in REBOL, it is a global function that does something. 
I demonstrated, that it is possible to have a similar construct (possibly 
even using the same name), which could work as "local" in the sense, 
that it would be assigned to a locally-bound 'return word. Such a 
"local RETURN" would, in fact, be able to "jump many levels up", 
not just one level, as the current global RETURN does, since it would 
be tied to its "function of origin".

>> a: [ b [ c [ 1 ]]]
== [b [c [1]]]
>> d: 'b
== b
>> e: 'c
== c

using a, d, and e, how would I get the value "1" ?

There may be better ways:
   first do to-path reduce ['a get 'd get 'e]

>> a/:d/:e/1
== 1

is there a quick way to determine whether a string is enbased without 
debasing it?

Well, enbase.base 2 is easy:
     string: enbase/base "cffdf" 2
     "01" = sort unique join "10" string
And that is easily extendable to the other bases

But you might get some false positives -- say a base 16 number happened 
to be all 0s and 1s.

>> base64: charset [#"A" - #"Z" #"a" - #"z" #"0" - #"9" #"+" #"/"]
== make bitset! #{
000000000088FF03FEFFFF07FEFFFF0700000000000000000000000000000000
}

>> parse "c3VyZS4=" [any [4 base64] [3 base64 #"=" | 2 base64 2 #"="]]
== true

you may want to compare this to actual debasing to see which is faster 
on a big string. debase being native it might actually be faster.

(though, the allocation and copy of data could be significant enough 
on a big string for something like this to be worth it...)

well... enbase just converts binary (8-bit) data to a form that is 
ascii printable. it does not say anything about what the 8-bit data 
contains.

it's easier to use the parse instead:
>> parse/case "A" ["a" (print 1) | "A" (print 2)]
2

But I consider it as a big limitation of the SWITCH function.

switch: func [
    "Selects a choice and evaluates what follows it." 
    value "Value to search for." 
    cases [block!] "Block of cases to search." 
    /default case "Default case if no others are found."
][
    either value: select cases value [do value] [
        either default [do case] [none]]
]

replace SELECT by SELECT/CASE

the later switch doesn't work like a select anymore.. it finds and 
then skips to a block, which allows multiple keys per block.  its 
very usefull.

I use it almost exclusively in my stuff, although I've been using 
case a lot more lately.

Any thoughts on what should happen if you modify the cond-block of 
a WHILE while executing the WHILE?

Take this code for example:

    a: 0
    b: [a < 5]  ;; cond-block of WHILE
    c: [        ;; body-block of WHILE
         print a
         a: a + 1
         insert b false   ;;  modify the cond-block
         ]

    a while b c

R3 ... acts as if the code were:
     a while copy b c
  so the modification to b is ignored

R2/View and R2/Core Crash with a corrupt datatype 192 (or 64)


None of them seem to allow inflight modifcation of the cond-block.

I have a suspicion that the answer would differ based on whether 
the modification is performed in the cond block vs. the body block, 
but if what you describe is true then that would make sense. I'm 
going to run some tests.

First proposal (by Sunanda, copied from some other group): SECURE/do 
block!


SECURE/do would take a block of code to do in a different set of 
security constraints, and then return the security constraints to 
what they were before upon the return of the function. If the security 
constraints are more relaxed then the user would have to be asked, 
or whatever the security settings are. If they are more constrained, 
the user won't have to be asked. And the user won't be asked upon 
returning the constraints to their previous setting when SECURE/do 
returns - this is the whole point to the proposal.


SECURE/do would need to be task-local because its scope would be 
calling and returning from a function, a task-local operation.

brian, the only way I can see this is if the executed code didn't 
catch something and we unwind past the SECURE and it never gets a 
chance to restore the state.

thus, SECURE should (always?)  wrap the execution within what ever 
becomes the sandbox.

alternatively, it could a feature of the sandbox model...

SANDBOX/secure

I prefer this later idea.

Don't know, it's been a while. I can't implement this today - it's 
a busy week.

ok, so maybe the system/state/policies could have a companion which 
is a stack of the current policies at each step of secure/do.

The problem with making running state secure so that functions won't 
be installed when the security restores is that it is much trickier 
to secure task-local state than it is to do for the stuff in lib 
and sys. We should consider doing this as a MAKE task! extension 
instead.

though the MAKE task! is appealing its also very disruptive, since 
it implies a lot more than just securing access to devices.


but I am not sure what you mean by "so that functions won't be installed 
when the security restores"

We can secure lib and sys because we are smart mezzanine developers, 
and they can stay secured. But the user contexts are task-local and 
isolated, so they would need to be secured again for each task. And 
any script you run in a user context affects subsequent scripts run 
in that same context. So if you run a script with increased security, 
it can install functions in the user context that it is running in 
that wouldn't affect the already-secured lib code, but *would* affect 
code that is run in the same user context after SECURE/do returns. 
So if those functions are called later, they will run with full permissions. 
This is called escalation.

But SECURE doesn't protect from those kind of security issues anyways, 
so this may be not a problem that we have to worry about here. However, 
normally the security prompt will protect against this kind of thing, 
and the whole point of SECURE/do is to get rid of that prompt.

Full sandboxing would require running in a separate task, calling 
a script by IMPORT/no-lib/no-user/isolate instead of DO, and providing 
a whole set of safe words and wrapper functions for the script to 
use. But many aspects of the existing system are designed with this 
in mind.

I think we don't need a separate task for sandboxing... since this 
opens up new different concurency issues.

Total sandboxing might require SECURE/LAUNCH rather than SECURE/DO. 
But that's a whole other set of trade-offs.

since as part of the new task, there is an implicit "clone-user-context" 
step at some point...   maybe that could be something we can actually 
use manually within a script?

Um, no, there is no indication that there will be a "clone-user-context" 
step as part of making a new task. There is a "clone the user context 
from the lib" step in starting all user scripts, but no "clone a 
user context from another user context" step.

Everything said about the concurrency model so far indicates that 
it will not be the fork/exec model. Instead it will be the "create 
a new user context from scratch and populate its task-local values 
from the shared stuff" model.

I think that user-context cloning should be considered as an option. 
 even without tasks.  it might help in providing a proper sand-box. 
  though I can see some things that it currently doesn't cover on 
its own, it would allow the basis for a strong model.

something like:

ctx: clone-user-ctx


ctx/my-func data ; (this becomes shared, so you decide how this is 
managed, copy if required)

what I need:  ctx/some-value

ctx: none  ; recycle it.

yes... but "it's own" should allow a clone of the "current" one.

A user context by default only contains a reference to system. Everything 
else comes from lib. Cloning lib is more effective.

shared stuff can be tampered.

when its protected, then it can't be used in all circumstances.


a perfect clone of the user allows tampering, without the danger, 
since it can't affect you anyways.

A standard sandbox lib could be provided that contains a safe version 
of the standard lib that R3 comes with. Then you could derive your 
own safe lib from that one, to add your own predefined functions. 
No clone of the user context would be necessary. And for that matter, 
you should assume that cloning the user context is impossible because 
of cyclic references.

actually, cyclic references is one of the specific reasons for a 
clone-user-context native ! ;-)

The SANDBOX function wouldn't then need to do anything custom to 
the user context that the script it is running is using. All it would 
need to do is create a safe system object, create an object that 
references that object using the 'system word, and set the lib and 
security settings to those provided as arguments to the SANDBOX function.

A different lib than the one that regular scripts use, of course.

#1521 would be better to do as a separate function (perhaps RECOVER).

I would be just thrilled to see something like PROBE/SOLVE 

I have trouble "understanding" how to read rebol code.  I'm getting 
there, but still I make mistakes.  I would love to see a "step by 
step" breakdown of some rebol code.


I'll give you a good metaphor:   Remember Algebra, with parethasis 
and the steps you took to solve a problem:

( (a * 2) + (b / 3) ) / 5   Then you substituted for your variables 
step by step and solved the problem.  I'd love to have rebol do something 
like that.  So instead of one line like PROBE, you could get several 
lines, that show how the function was evaluated to arrive at the 
final result.  Maybe TRACE does this?


All in all, I'd like more debugging tools, or at least some expanded 
documentation on how to debug rebol code faster!  Thanks

Pat, remember that you can replace the function building mezz code 
like funct and func.   though its not for novice users, since you 
do have the source when you 'SOURCE these builders, it can be quite 
easy to tweak them so they do a few things more...  like add a little 
break point at the end of each func and probe all the collected words, 
in FUNCT.


with a global word you could control if this tracing occurs, just 
by setting it to true or false, dynamically.

wow, that is a very nice document, I hope it makes its way to the 
official r3 documentation when an "official" R3 specification is 
finally written, after all the tweaks to the language.

Keep in mind that only *one* of those proposals for the new model 
of function behavior would be done. I am in favor of either "Definitional 
return with an option to not define RETURN and EXIT, dynamic return 
as a fallback" or "Definitional return with an option to not define 
RETURN and EXIT, no dynamic return". How about you?

Just tweaked it a little - something wasn't a disadvantage, it was 
just part of the proposal.

right now..  my brain would choose this way:

my-brain [ ]

hour: 12h35AM
options: [
	"Dynamic return with optional transparency"
	"Definitional return only"

 "Dynamic-return-only functions vs. option of definitional-return-only 
 functions"
	"Dynamic return with a definitional return option"

 "Definitional return with an option to not define RETURN and EXIT, 
 dynamic return as a fallback"

 "Definitional return with an option to not define RETURN and EXIT, 
 no dynamic return"
]

either hour > 00:00:00 & hour < 06:00:00 [
	select options random length? options
][

 write altme://!REBOL3 Proposals/answer  read http://www.rebol.net/wiki/Exceptions
]

Let's just make it a standing question then :)

If a function can't be rewritten so it doesn't use inner functions, 
you have no recourse.

What's that supposed to mean?

Making definitional return optional means that this option would 
need to be specified in the code that ''calls'' the functions that 
would benefit from it, rather than the functions themselves. This 
means that the option would need to be specified a lot.

What is that supposed to mean?


With optional definitional-return-only functions I would simply define 
a mezzanine USE with a definitional return, at which point it no 
longer swallows dynamic return. A caller of USE then does not have 
to worry about this at all.

you have no recourse

 is a polite way of saying "you are out of luck". At least regular 
 programmers would be out of luck there - I'm sure someone like Ladislav 
 could come up with an arcane workaround, or you could give up on 
 RETURN and EXIT and use another escape function instead, like THROW. 
 But I assume that you know what I meant by "recourse", and want the 
 point explained.


Pardon me, that question needs some background info. The return models 
are being used to deal with a basic problem of functions catching 
RETURN and EXIT when you don't want them to. This is the case with 
many mezzanine control functions which take and execute a block of 
code. We have been putting requests for new mezzanine control functions 
on hold for quite a while because they can't currently be made to 
pass through RETURN and EXIT, but USE and COLLECT got through before 
we started that, and the restriction is lifted now.


Let's use USE to illustrate, ignoring for the moment that USE *can* 
be rewritten so it doesn't use an inner function.

use: func [
    "Defines words local to a block."
    vars [block! word!] "Local word(s) to the block"
    body [block!] "Block to evaluate"
][
    apply make closure! reduce [to block! vars copy/deep body] []
]


USE uses an inner function to create a binding for its words (the 
closure!). For the dynamic return we have now, the inner function 
catches returns from the body, but even if it didn't the USE function 
itself would catch those returns as well.


One proposal to solve this would be to switch to definitional return, 
which means that RETURN and EXIT would be redefined in the code block 
of a function to return directly to that function, not any intermediate 
function in the call chain. This would solve returns being caught 
by the USE function itself, because the body of code that contains 
the 'return or 'exit words is not physically in the code of the USE 
function, it is only referenced by word. However, that block of code 
is used by the inner function as its code block, so the inner function 
would redefine those words and catch the returns.


If your function uses inner functions like USE does, and can't be 
rewritten to not use them, and you are using definitional return 
without the option to turn it off, then the inner function will localize 
RETURN and EXIT every time.


As a caveat, I wrote that phrase before I came up with the workaround 
in the next section of using direct references to the RETURN and 
EXIT function values instead of referring to them by name, which 
avoids the rebinding issues because no words are involved. See the 
code in http://curecode.org/rebol3/ticket.rsp?id=637to see what 
that workaround makes your code look like.

USE uses the body of the affected code *as* the body of the inner 
function, so it *is* a good example of this pattern, if you ignore 
the fact that USE in particular can be rewritten so it doesn't use 
an inner function.

Keep in mind that we use USE to illustrate because it is simple and 
shows a lot of the problems we are trying to solve in its existing 
code. USE is being used as a standin for the many other functions 
that would be affected by these issues. The fact that USE can be 
rewritten doesn't mean that the other functions can also be rewritten.

All you have to do is construct a correct inner function, instead 
of an erroneous one.

Your example does not. It improperly rebinds a definitional return.

So even the FOREACH workaround would be enough to prompt a change 
now. For that matter, the most recent module system rewrite was specifically 
for that reason. The new module system doesn't do anything that the 
pre-108 rewrite didn't do, but it is easier to use. Easy by default 
is the top priority.

So the real disadvantage in the "inner function" paragraph is that 
you can not write a function which does not rebind RETURN.

So assume that in this case "not possible" could in some cases mean 
"not possible to get accepted as a mezzanine because it's too Scheme-like" 
or "not possible for a regular programmer to come up with a workaround 
like this", because both limitations have real examples that have 
already manifest.