AltME groups: search

CONTEXT catching RETURN and EXIT is error #520.

We want the separate TRY in order to distinguish between errors and 
unwinds.

maybe we could start thinking about using a new word for what we 
are talking about extending CATCH to do... its starting to look like 
a long list of groceries for a single word.


reading Andreas' notes on 'CATCH I agree that it should only manage 
throws.  

/all  should mean catch any named or unnamed throw


/quit /unwind and other proposals should fall into another function.

CATCH is a sandbox already, just not a very good one. SANDBOX would 
need to be mezzanine because it would need to provide wrapper functions 
as well. And we need the subtle differences in the various CATCH 
options for different circumtannces.

Then we could get rid of CATCH/quit and move it to SANDBOX and also 
move the desired CATCH/all to SANDBOX.

well, catch/all for should be just to catch any and all throws... 
no?

CATCH/all would also catch BREAK, CONTINUE, and dynamic RETURN and 
EXIT.

SANDBOX would need to be mezzanine. CATCH/quit and CATCH/all need 
to be native, whatever functions they are in. That means that SANDBOX 
would use CATCH/all, but not be able to implement it.

Sandboxing would require replacing the DO* and MAKE-MODULE* intrinsics 
for the sandboxed code as well. However, *you* would need the non-sandboxed 
mezz functions to *implement* the sandboxed functions.

DO/SECURE ... thanks for the clarification. Brian.


What could be quite useful then, would be DO/SECURE having a parameter 
block of SECURE settings. But it can only tighten existing SECURE 
settings, not loosen them.


The DONE thing would execute under the tighter policies; and they 
would revert when it exited/returned/quit/crashed etc.

That would be a very useful tool for creating sandboxes.

we'll see when it gets done... and I'll be the first in line to test 
it   ;-)

The only way to get around SECURE's user request thing securely is 
to do it within SECURE, and restore it before the initial call to 
SECURE returns.

The increasingly off-topic nature of this conversation makes me wish 
more and more that this was R3 chat. Then we could move messages.

Core is for issues that aren't as R3-specific as our conversation. 
We should be discussing CATCH/all and SECURE/do in the !REBOL3 group.

anyhow... its just that more groups is faster than really big ones, 
so separating the use and proposal discussions seem to be a good 
thing in my mind.

Brian, in

http://www.curecode.org/rebol3/ticket.rsp?id=1744&cursor=2

you wrote: "We already have local THROW and RETURN"


If we do, I must have missed them, knowing only the global variants. 
Can you point me to them?

>> a: [ b [ c [ 1 ]]]
== [b [c [1]]]
>> d: 'b
== b
>> e: 'c
== c

using a, d, and e, how would I get the value "1" ?

Well, enbase.base 2 is easy:
     string: enbase/base "cffdf" 2
     "01" = sort unique join "10" string
And that is easily extendable to the other bases

But you might get some false positives -- say a base 16 number happened 
to be all 0s and 1s.

I think that might be OK. It's just that I need to process some fairly 
large strings 50-100 kb each and it should really happen in near 
real-time, if possible.

Good luck!

For raw speed, experiment with dropping the SORT -- and try DIFFERENCE 
- it may be faster for large strings.
    "" = difference "01"  join "10" string

Curiously, this works in R2:
    x: charset ["0" "1"]

    find x string     ;; returns TRUE for all strings made up of just 
    "0"s and/or "1"s

It does not work in R3 -- which may be deliberate and sensible.

You were focusing on localiity of where the code was written, and 
I was talking of locality in the code that the flow of execution 
goes through at runtime. For instance, #1744 makes it difficult for 
non-local-definition code to do man-in-the-middle attacks or spoofing, 
making it useful for secure mezzanine control flow functions. But 
#1518 prevents you from being able to pass THROW/name through unknown 
code at all, making it useless for making mezzanine control flow 
functions at all. Fixing #1518 is what we do to make #1743 possible, 
and once #1520 is implemented then the arms race will be over, everything 
else could be mezzanine or user-defined.

Henrik, i suspect that debasing may be faster than checking eg. with 
parse. are we talking base64 here or just base 2 and 16?

(though, the allocation and copy of data could be significant enough 
on a big string for something like this to be worth it...)

If I need case sensitive SWITCH and still want to use string values, 
is there any other way than using PARSE/case instead?

switch is native, isn't it? and has no /case.

switch: func [
    "Selects a choice and evaluates what follows it." 
    value "Value to search for." 
    cases [block!] "Block of cases to search." 
    /default case "Default case if no others are found."
][
    either value: select cases value [do value] [
        either default [do case] [none]]
]

replace SELECT by SELECT/CASE

and, use the [throw] function attribute, when in R2

the later switch doesn't work like a select anymore.. it finds and 
then skips to a block, which allows multiple keys per block.  its 
very usefull.

Any thoughts on what should happen if you modify the cond-block of 
a WHILE while executing the WHILE?

Take this code for example:

    a: 0
    b: [a < 5]  ;; cond-block of WHILE
    c: [        ;; body-block of WHILE
         print a
         a: a + 1
         insert b false   ;;  modify the cond-block
         ]

    a while b c

R3 ... acts as if the code were:
     a while copy b c
  so the modification to b is ignored

R2/View and R2/Core Crash with a corrupt datatype 192 (or 64)


None of them seem to allow inflight modifcation of the cond-block.

The trick of course is that modification of the INSERT and APPEND 
variety can result in reallocating the block, and native code wouldn't 
necessarily notice the change since it would still be referring to 
the old version. It's not quite the same thing with REMOVE and CHANGE 
style modifications.

Mean WHILE.....Thanks for debugging my example, Brian. I got confused 
and reported the wrong thing.


Yes -- on R3 if is possible to modify the cond-block and that modification 
will be respected. That''s how I hoped it would be.


It is not possible to _replace_ the cond-block by reassigning its 
word. But that's understandable and probably exactly what we'd all 
want:

   cb: [true cb: [n < 3] true]       ;; cond-block that overwrites itself
    n: 0

    while cb [n: n + 1 print [n cb]]   ;; WHILE that uses original cond-block

err: 'cb and overwrite

we need to make distinction between overwriting a variable, and overwriting 
a block

Perhaps WHILE could also accept a word! for its condition parameter, 
and when it does, evaluates it to the block it is expected to reference 
before continuing to evaluate as usual.
Eg.
	while 'cb [...]

What should this function be called, and what options does it need?

change-in: funct [key value] [
    either a: find block key [change next a :value][
        repend block [key :value]
    ]
]


It basically implements the a/key: value operation of maps, but for 
series.

I suppose you had FUNCT [BLOCK KEY VALUE] in mind for the above, 
Brian? If so, I think I like MERGE much, but every now and then someone 
has to suggest ALTER

Yes, of course, Gregg, UPDATE only working on ports seems indeed 
too limited for such a nice word. 


BTW, Brian, is the above behaviour to return after the change for 
updates and at the insertion for inserts intended?

>> update: :change-in
>> attributes: []
== []
>> update attributes 'color red
== [color 255.0.0]
>> update attributes 'color blue
== [] ; or rather [color 0.0.255]

On the other side, to answer your question, it's related to SELECT 
and should probably support most, if not all, of SELECT's refinements, 
too. That's a bit of a stretch, because for ports UPDATE probably 
needs to be very fast and can't be thwarted by too complicated refinement 
handling?

And not Rebol!

Hmm ... http://www.rebol.org/view-script.r?script=crypt.r


I changed the strength to 256, and algorithm to rijndael .. and encrypted 
a file, but get an out of memory when I try to decrypt it :(

Well, I used the encryption key as a string instead and encrypted 
to 256 bits using Rijndael and successfully decrypted with Rebol. 
 But aescrypt was not able to decrypt the file :(

If you have counters block and want to increment it, is there some 
better solution than this one?
	b: ["a" 1 "b" 1]
	change f: find/tail b "a" f/1 + 1
	probe b

don't know if any of these are better(and in which sense better):

change n: next find b "b" n/1 + 1

b/("a"): b/("a")  + 1
...

This does it without using a temporary word....and it should work 
even if the counter name is not amenable to Cyphre's path notation 
(ie you are using something more exotic that strings for counter 
ids, or are using an older version of /Core).
   b:  next find/skip head b "a" 2 b b/1: b/1 + 1
Just remember to reset ....
   b: head b
....once in a while:)

b/("a"): b/("a")  + 1
is the winner as it's a little bit faster and shorter.

EXISTS? creates a port from the file! argument, then queries that.
DELETE doesn't work with ports.

I guess it is the creation of a port or QUERY which is able to handle 
backslashes, and DELETE cannot.

ntfs. And yes, I normalise now, just wanted to know, why it's different 
and if it's correct. (because I've spent some time to figure out 
this exists?/delete difference).

There is another way to put a directory together with a file than 
'join, and it handles more cases with the forward/back-slash on its 
own:
>> Dir: %foo/bar
== %foo/bar
>> File: %test  
== %test
>> Dir/:File    
== %foo/bar/test
>> Dir: %foo/bar/
== %foo/bar/
>> Dir/:File     
== %foo/bar/test
>> File: %\test  
== %/test
>> Dir/:File   
== %foo/bar/test

ROUND under R2 was a mezzanine, and was designed so the result would 
be an integer when possible, for use as a loop counter. It's no longer 
a mezzanine under R3. I don't remember if it was mentioned that it 
was an intentional change.

And we don't need to use integers for loop counters anymore.

What is the best way to form decimal and avoid the E notation?
>> form .02
== "2E-2"

EXTRACT works well for that, and maybe MAP-EACH too if you do nested 
blocks.

There are some databases that are RAM-based already and have SQL.

For that matter, RebDB is SQL-like and RAM-based, iirc.

my queries are currently taking 2 seconds and I need subsecond performance

It's a GPL app to do drug prescribing and drug interactions .. I 
am just using their database

It looks like you are using the iterative approach anyways, on the 
results of the SQL queries. If you cache the data in RAM you can 
do iterative stuff with FIND and loops, or even maps in R3 if you 
want real speed.

It's late here, and if see more than a dozen of lines, I could die

so at present I think I generate all possible combinations and check 
to see if they're in the same set and if not do the sql query

Would you kind enough and give us the expecting output from the input. 
I can't read all your lines currently.

drug1 [ id1 id2 id3 .. ] drug2 [ id1 id5 id6 .. ] drug3 [ id4 id7 
.. ]
-->

you'll see the output of the original script which produces html 
and not json

And what about purchasing a faster computer ? 

(I'm in a deep shit right know, I don't understand what he's talking 
about)

Store sorted in a and sort the query before find.

Sort the elements on insert and keep them sorted. Then sort the value 
you are comparing to before looking for it.

very odd ... I changed to using blocks instead of sql, and the query 
increased from 1.7 s to 7 seconds!

Graham, you can do your own indices using hash!, but that might be 
pushing your complexity budget. Try it and see.

This group is for proposing new stuff for REBOL 3, and then discussing 
these proposals. In general these proposals will be about changes 
to R3 itself, but many may end up being third-party extensions instead. 
Don't be shy!

(And keep things civil) :)

First proposal (by Sunanda, copied from some other group): SECURE/do 
block!


SECURE/do would take a block of code to do in a different set of 
security constraints, and then return the security constraints to 
what they were before upon the return of the function. If the security 
constraints are more relaxed then the user would have to be asked, 
or whatever the security settings are. If they are more constrained, 
the user won't have to be asked. And the user won't be asked upon 
returning the constraints to their previous setting when SECURE/do 
returns - this is the whole point to the proposal.


SECURE/do would need to be task-local because its scope would be 
calling and returning from a function, a task-local operation.

Thanks Brian.....But credit yourself and Maxim for refining it to 
something that may be implementable.

brian, the only way I can see this is if the executed code didn't 
catch something and we unwind past the SECURE and it never gets a 
chance to restore the state.

Maxim and I made some tweaks to Sunanda's proposal. Those tweaks 
were discussed in Core, but here are the basics:

- Making this an option to SECURE rather than DO is simpler (DO is 
hybrid, SECURE is full mezz) and more secure (only SECURE would need 
access to the internal stuff).

- Doing this as an explicit stack (SECURE/push and SECURE/pop) would 
not allow us to pop without asking the user for permission.

The problem with making running state secure so that functions won't 
be installed when the security restores is that it is much trickier 
to secure task-local state than it is to do for the stuff in lib 
and sys. We should consider doing this as a MAKE task! extension 
instead.

We can secure lib and sys because we are smart mezzanine developers, 
and they can stay secured. But the user contexts are task-local and 
isolated, so they would need to be secured again for each task. And 
any script you run in a user context affects subsequent scripts run 
in that same context. So if you run a script with increased security, 
it can install functions in the user context that it is running in 
that wouldn't affect the already-secured lib code, but *would* affect 
code that is run in the same user context after SECURE/do returns. 
So if those functions are called later, they will run with full permissions. 
This is called escalation.

But SECURE doesn't protect from those kind of security issues anyways, 
so this may be not a problem that we have to worry about here. However, 
normally the security prompt will protect against this kind of thing, 
and the whole point of SECURE/do is to get rid of that prompt.

Full sandboxing would require running in a separate task, calling 
a script by IMPORT/no-lib/no-user/isolate instead of DO, and providing 
a whole set of safe words and wrapper functions for the script to 
use. But many aspects of the existing system are designed with this 
in mind.

so you clone user context, run stuff inside of it, then pull out 
some expected data changes, and destroy that clone explicitely.

Everything said about the concurrency model so far indicates that 
it will not be the fork/exec model. Instead it will be the "create 
a new user context from scratch and populate its task-local values 
from the shared stuff" model.

And shared stuff you don't want modified will be made read-only. 
There will likely be some kind of message passing and/or synchronization 
to share modifiable data between tasks and coordinate efforts.

All you need to do to sandbox some code is isolate it and give it 
its own lib and user contexts.

I'm not talking about fast, I am talking about easy and practical. 
 the user-context already exists so Cloning it should also be an 
option.

A standard sandbox lib could be provided that contains a safe version 
of the standard lib that R3 comes with. Then you could derive your 
own safe lib from that one, to add your own predefined functions. 
No clone of the user context would be necessary. And for that matter, 
you should assume that cloning the user context is impossible because 
of cyclic references.

The SANDBOX function wouldn't then need to do anything custom to 
the user context that the script it is running is using. All it would 
need to do is create a safe system object, create an object that 
references that object using the 'system word, and set the lib and 
security settings to those provided as arguments to the SANDBOX function.

#1743 added to have QUIT with no /now, #1744 added to have CATCH/name 
use EQUIV? comparison (considering contexts), and #1520 scaled back 
to be the CATCH/name true option, no other options added.

Also changed the comment in #1742 to match the new combined model 
of #1518, #1520, #1742, #1743 and #1744.

I would be just thrilled to see something like PROBE/SOLVE 

I have trouble "understanding" how to read rebol code.  I'm getting 
there, but still I make mistakes.  I would love to see a "step by 
step" breakdown of some rebol code.


I'll give you a good metaphor:   Remember Algebra, with parethasis 
and the steps you took to solve a problem:

( (a * 2) + (b / 3) ) / 5   Then you substituted for your variables 
step by step and solved the problem.  I'd love to have rebol do something 
like that.  So instead of one line like PROBE, you could get several 
lines, that show how the function was evaluated to arrive at the 
final result.  Maybe TRACE does this?


All in all, I'd like more debugging tools, or at least some expanded 
documentation on how to debug rebol code faster!  Thanks

Pat, remember that you can replace the function building mezz code 
like funct and func.   though its not for novice users, since you 
do have the source when you 'SOURCE these builders, it can be quite 
easy to tweak them so they do a few things more...  like add a little 
break point at the end of each func and probe all the collected words, 
in FUNCT.


with a global word you could control if this tracing occurs, just 
by setting it to true or false, dynamically.

Ladislav and I have been collecting the pros and cons of the various 
issues and proposals related to escape functions here: http://www.rebol.net/wiki/Exceptions

Keep in mind that only *one* of those proposals for the new model 
of function behavior would be done. I am in favor of either "Definitional 
return with an option to not define RETURN and EXIT, dynamic return 
as a fallback" or "Definitional return with an option to not define 
RETURN and EXIT, no dynamic return". How about you?

right now..  my brain would choose this way:

my-brain [ ]

hour: 12h35AM
options: [
	"Dynamic return with optional transparency"
	"Definitional return only"

 "Dynamic-return-only functions vs. option of definitional-return-only 
 functions"
	"Dynamic return with a definitional return option"

 "Definitional return with an option to not define RETURN and EXIT, 
 dynamic return as a fallback"

 "Definitional return with an option to not define RETURN and EXIT, 
 no dynamic return"
]

either hour > 00:00:00 & hour < 06:00:00 [
	select options random length? options
][

 write altme://!REBOL3 Proposals/answer  read http://www.rebol.net/wiki/Exceptions
]

and my-brain doesn have the same seed as rebol, so you can't guess 
my option right now  ;-)

Ah, I was going to lie down and watch TV on my computer in the other 
room :)

you have no recourse

 is a polite way of saying "you are out of luck". At least regular 
 programmers would be out of luck there - I'm sure someone like Ladislav 
 could come up with an arcane workaround, or you could give up on 
 RETURN and EXIT and use another escape function instead, like THROW. 
 But I assume that you know what I meant by "recourse", and want the 
 point explained.


Pardon me, that question needs some background info. The return models 
are being used to deal with a basic problem of functions catching 
RETURN and EXIT when you don't want them to. This is the case with 
many mezzanine control functions which take and execute a block of 
code. We have been putting requests for new mezzanine control functions 
on hold for quite a while because they can't currently be made to 
pass through RETURN and EXIT, but USE and COLLECT got through before 
we started that, and the restriction is lifted now.


Let's use USE to illustrate, ignoring for the moment that USE *can* 
be rewritten so it doesn't use an inner function.

use: func [
    "Defines words local to a block."
    vars [block! word!] "Local word(s) to the block"
    body [block!] "Block to evaluate"
][
    apply make closure! reduce [to block! vars copy/deep body] []
]


USE uses an inner function to create a binding for its words (the 
closure!). For the dynamic return we have now, the inner function 
catches returns from the body, but even if it didn't the USE function 
itself would catch those returns as well.


One proposal to solve this would be to switch to definitional return, 
which means that RETURN and EXIT would be redefined in the code block 
of a function to return directly to that function, not any intermediate 
function in the call chain. This would solve returns being caught 
by the USE function itself, because the body of code that contains 
the 'return or 'exit words is not physically in the code of the USE 
function, it is only referenced by word. However, that block of code 
is used by the inner function as its code block, so the inner function 
would redefine those words and catch the returns.


If your function uses inner functions like USE does, and can't be 
rewritten to not use them, and you are using definitional return 
without the option to turn it off, then the inner function will localize 
RETURN and EXIT every time.


As a caveat, I wrote that phrase before I came up with the workaround 
in the next section of using direct references to the RETURN and 
EXIT function values instead of referring to them by name, which 
avoids the rebinding issues because no words are involved. See the 
code in http://curecode.org/rebol3/ticket.rsp?id=637to see what 
that workaround makes your code look like.

Thanks for all the effort that went into that doc! I've skimmed it, 
but will have to find time to read it in depth and digest it.

Keep in mind that we use USE to illustrate because it is simple and 
shows a lot of the problems we are trying to solve in its existing 
code. USE is being used as a standin for the many other functions 
that would be affected by these issues. The fact that USE can be 
rewritten doesn't mean that the other functions can also be rewritten.

You grabbed that phrase that I was answering from the section where 
binding with definitional return was the only way to bind. There 
is no option in that section (and that fact is listed in the advantages). 
In that section, definitional is the only way to bind.

And even if you insist on that hypothesis, the claim is still wrong.

Sorry, inner functions are the easiest way to implement certain things. 
And we need to consider that too-complex functions or ones that use 
arcane workarounds are getting rejected for that reason alone nowadays. 
Most of the native changes in the last 3 releases of R3 have been 
to deal with situations like this. For these proposals, easy trumps 
possible.

The only reason definitional-only might be chosen is because it is 
easier to understand than the other models. That alone might trump 
the places where it still doesn't work. I hope not, because the "option 
to not rebind RETURN and EXIT" is really simple to specify and much 
easier to do than the workarounds that you have to do if you don't 
have that option, workarounds like your BIND + DO, especially when 
you consider 'self issues that prevent you from using objects for 
this kind of thing (hence the FOREACH).