World: r3wp

Since there are so many questions, here is a group for those who 
want help with R3's new module/script system. If you need help with 
LOAD, SAVE, IMPORT, DO-NEEDS, Needs header syntax, the new binding 
conventions or the new semantic model of scripts and modules in R3, 
ask here!

I will try to answer as many questions as I can, but feel free to 
chip in if you know the answers. Don't be shy about asking: The stuff 
asked here will help us tweak the docs so you won't need to ask in 
the future.

Or write them, for that matter. I lost track of where the docs were 
in the various wiki reorganizations. Right now the definitive docs 
on the subtleties of the module system (beyond the official docs 
Carl wrote) are still in the CureCode tickets related to them, and 
their comments. I'll try to track down a list of the relevant tickets 
and post them here.

Here's the proposal for compressed scripts/modules: http://curecode.org/rebol3/ticket.rsp?id=1466

And the original blog (complete with comment flame war): http://www.rebol.net/r3blogs/0274.html

So, what works?

Everything specified, as of alpha 97, even the 'export keyword Carl 
requested. Compressed scripts/modules have been submitted and are 
being reviewed now.

What doesn't work:

- Selective import is possible but needs mezzanine wrappers (and 
a model for their behavior).

- Delayed init is just a dream now; Carl hasn't even elaborated on 
what he means by that.

- System modules might need a little tweaking to the model (by Carl).

- We need better checking of extensions before loading (authenticode?).

- A module-aware version of prebol is still needed (it's on my list).

But we have scripts and regular or isolated modules, and named or 
mixin import - that's 6 import models altogether, all working.

Extensions contain a regular or isolated module in them that acts 
as the wrapper for them, so once loaded they are treated like any 
other module, seamlessly. Not sure if that embedded module could 
be compressed using the standard code - I doubt it because of the 
wrapping process.

Nope, the wrapping is done later than the decompression, so at least 
script-encoded compression would work. Binary-encoded compression 
probably won't work though - that needs to be checked.

Even script-in-a-block works :)

Here are the official docs (so far):

Loading modules: http://www.rebol.com/r3/docs/concepts/modules-loading.html

Defining modules: http://www.rebol.com/r3/docs/concepts/modules-defining.html

Special notes: http://www.rebol.com/r3/docs/concepts/modules-special.html

Other script stuff: http://www.rebol.com/r3/docs/concepts/scripts.html

These docs don't yet include the real subtleties of when to use IMPORT, 
when to use DO and when to use Needs. In particular, they don't say 
when *not* to use IMPORT directly and why, what the difference is 
between named and unnamed modules, or any kind of hints about overall 
program structure. This is partly because the docs were written (by 
Carl) before the module system was finished (by me) so that info 
simply wasn't known at the time, and partly because I could use some 
help with writing docs about those issues that don't read like a 
CS paper.

Oh, and the script docs are still written for R2, not R3 yet.

Delayed Modules - http://www.rebol.com/r3/notes/delayed-mods.html

Brian - could you describe the phases we run thru? I mean - first 
you "include" a module, extension, it (maybe) gets into module list 
(catalogue), but still not loaded ... and now - what happens next? 
Is there anything like loading a module, but still not binding it 
for e.g.? And if so, what advantage does such phase have? Maybe it 
would be handy to have a process diagram of how modules work ...

I can describe the phases we run through now, but not what Carl wants 
for delayed modules. If I knew that I would have implemented them 
months ago. Right now I'm at the asking for more details phase - 
see chat for details. The method of doing this with extensions built 
into the host seems to be straightforward, just one of two choices. 
It's not as clear yet for other modules.

Mixins have such a use case for delaying that I almost want to make 
that the default behavior.

Ladislav, right now the best documentation of mixins is the source 
of the DO-NEEDS and MAKE-MODULE functions in DevBase. There are also 
the CureCode tickets related to them. But there aren't that many 
docs for them, and the behavior might be yet be tweaked. If you have 
any questions ask here, and I will answer as I can. But I'm going 
to be busy this month, so patience would serve you well here.

The discussion in bug#1625 is turning into a fairly deep explanation 
of the R3 script and module bbinding models, the meaning of FUNCT 
and why we have it, and the inherent limits of REBOL that were why 
we did things the way they are now. Take a look, it could be an eye-opener 
to anyone interested in this group.

Holy gotcha's Batman! Take a look at bug#1628 and make your comments, 
we need to decide this soon!

wrt 1625... it should unbind even inline modules.

otherwise, they become almost (exactly?) the same as objects, so 
I don't see the point.

That's #1628. Agreed. And put your comments there in the ticket, 
so they will better affect the final decision.

1625 - Interesting; BrianH explains at length.

OK, now we finally have the requirements for delayed modules (and 
bug#1628). Time for me to work through the issues and implement them.

BrianH: I am not sure I like Carl's comment re 'what showing the 
non-exported functions too. It is not about 'what for me, but why 
should non-exported functions be available? In following Carl's example, 
I see the only advantage of exported vs non-exported function being 
a binding control, but I thought we get encapsulation too. Was is 
intended that way? For me there is very little advantage in calling 
'init-subsystem vs m/init-subsystem ....

m: import 'my-graphics
m/init-subsystem
m/set-resolution 1600x1200
draw drawing

except that "init-subsystem" requires no lookup (fast) while "m/init-subsystem" 
requires a lookup (slow)

I still want to have /protect ability though ...

Gab, you did prot -http as a module.  Should all protocols be modules?

actually, when i did prot-http.r there were no modules yet, there 
just was an assumption that we would get them.

anyway, yes, i would guess all protocols should be modules, unless 
they are trivial enough (though, having them as modules as well may 
help with how they are loaded etc.)

Ok, good to know for when we start our rewrites :)

All protocols should be modules, period.

All infrastructure code that is loaded from the host should also 
be in modules or extensions. Scripts are for users.

Pekr, I answered your question in the blog. But in short, exporting 
is about exporting, not about hiding what isn't exported. Exporting 
means adding a word to the (closest thing that R3 has to a) "global" 
context. Otherwise, the word is just available in its local context. 
Modules are there for code organization, not encapsulation. Encapsulation 
is a separate process, and only rarely needed.

Nonetheless, we will be making it easier on a module basis by supporting 
a 'hidden keyword, similar to the 'export keyword.

I wish the word "expose" had been used instead of export... it seems 
more preceise to me.

in that case, just overload the word 'show

Nope - they are exported. Unless explicitly hidden, all module words 
are exposed.

the fewer characters the better

no has fewer characters than yes, but they mean different things.

We don't want to overload the word 'show because there are SHOW functions 
in common use that get exported from modules, and these keywords 
get *removed* from the source during their processing. Also, exposed 
by default was a deliberate design choice.

As was not-exported-by-default.

There are 3 levels of visibility:

- Exported to system/exports, or directly into the requesting context 
if the module is a mixin. Note that it is the value that is exported, 
not the word.

- Visible (or exposed, or shown, if you prefer), but not exported 
(the default)

- Hidden, meaning it can't be bound beyond code that has been bound 
to it already, usually just the module.


There is no point to an 'expose or 'show keyword, because words are 
exposed/shown by default. Hiding words is generally only done for 
security.

From the loading modules doc: "...if a version number appears before 
any module name, it is assumed to be the REBOL system version.

  Needs: [3.0.2 mysql 2.3.0 http-server 1.0.1]

 Is there an explicit alternative? And how would you specify that 
 you need View or Command rather than Core?

And for checksums, would 
 it make sense to allow a keyword before the checksum, so you could 
 choose md5, sha1, or something else in the future? An unmarked binary 
 could still be sha1. I know it maps to the /check refinement on IMPORT 
 as well. I'm just thinking of implicit meaning versus long lifecycles.

Hidden words can't be exported, because the export process has to 
see the words to export their values. This means that the 'export 
and 'hidden keywords will conflict. I can resolve that conflict by 
having one take precedence over the other, but that just seems like 
a hidden gotcha. It seems to me that triggering an error if both 
keywords try to modify a word would be the best approach. What do 
you think?

I agree. Conflicts are fine when they make sense.

Gregg, for R3 the View, Core and Command stuff are considered to 
be capabilities, not builds. With the host kit, we are assuming dozens 
of different builds (afaik). If you want View capabilities, put its 
module in your Needs header. Yes, I'm aware that they aren't yet 
fully modularized, but that is the plan.

Great. I still wouldn't mind the option to have an explicit 'rebol 
key at the head though.

Disambiguation and all that.