A REBOL challenge - The Information World
[1/23] from: g::santilli::tiscalinet::it at: 3-Dec-2001 18:45
Hello all! A REBOL challenge [Permission is explicitly granted to publish this document on the REBOL Zine, on REBOLForces or on any other REBOL-related publication/site/whatever.] With this message, I'm going to launch a challenge. The goal is to create a peer-to-peer communication system as described below; implementations will be judged by the members of the REBOL mailing list, with regards to: * Code elegance and simplicity * Efficiency * Usability If my free time will permit, I'll partecipate to the challenge too; a time limit will be set in agreement with the participants. The resulting code is required to be freely distributable at least inside the REBOL mailing list; freely distributable software will be preferred over restricted software because to be useful the system has to be available to all the users of the Internet. [I'm quite sure Maarten will participate with a Rugby based implementation; I'd like to have someone doing a REBOL/IOS based implementation too...] If you are interested, please read the following document carefully; comments or requests for clarifications will be gratefully accepted. [Since this version can be considered a sort of draft, I would be vert grateful to those of you that will really be brave enough to read it all and offer me some comments.] The "Information World" I want to create a sort of "virtual world". I will call it the Information World (IW). This world is made of informations, from static data to "live" entities able to interact with the rest of the IW. The IW is based on three simple entities that the implementation has to represent. These are "Places", "Objects" and "Agents". Places A "Place" is a location in the IW. It can contain objects and agents, and it can be connected to other places. (The connection does not necessarily need to be a TCP/IP connection or something like that; you can think of it as a "road" going from one place to another. Also notice that if place A is connected to place B, automatically place B is connected to place A.) Agents can move from one place to another only using a connection (normal agents cannot create a new connection). The implementation of a place has to provide a way for agents to know what other objects and agents are present in the place. I.e. the agents will have to be able to query the place where they are to know what's there besides of them. Objects Objects are "things", such as repositories, containers, documents, and so on. An object can contain other objects; for example, a book shelf object might contain book objects. By themselves, objects cannot move from one place to another or interact with other objects or agents. Only agents are able to iteract with other agents or objects. The implementation should make it easy for objects to be moved from a place to another (by agents); also, agents must be able to query an object to know what actions it can perform on it (for example, a book shelf might provide a "search" action to allow an agent to search for a book, and so on). The implementation has to provide an easy way for users to create new objects. Also, objects can be cloned. Agents Agents are the most interesting entity of the IW. They are the key for the communication, since they are the only entity that can move from a place to another and interact with other entities. They can also carry objects, but there should be a limit on the size of the objects an agent can carry on a given connection. Implementations have to deal with security issues raised by the presence of agents. In particular, two categories of agents have to exist: "Residents" and "Tourists". The only resident agents available should be: * The "Road Builder", which creates connections with other places. It is the only entity that can create connections; of course it can destroy a connection too. * The "Sentinel". A sentinel can be placed on every connection to verify the identity of coming tourists and place restrictions on them or even disallow their entrance. Each sentinel should be in communication with the one on the other side (if present) to be able to identify coming tourists. (Identifying might just mean assigning a "trust level" or something like that.) * The "Mail Agent", which is responsible of sending (big) objects to other places. The mail agents can be asked by an agent to send an object to another mail agent on another place; it should also be possible for mail to be delivered to a specific agent in a place (passing thru the mail agent of that place). The implementation should provide an efficent way to transfer any amount of data. (This makes up for the limit of agents to carry objects; place-to-place connections should be optimized for the transport of tourists with rather small baggage, while mail agent-to-mail agent "connections" might be optimized for transferring big objects.) * The "Banker". A place might have a banker which holds important objects. Objects available in the place can be used (and cloned) by any agent, so a banker might be needed to restrict access to some objects. The implementation should not allow for a tourist to become a resident, unless the user explicitly wants it to (and he/she should be warned that this is a security issue). Residents (sentinels and mail agents in particular) should have a secure way to identify themselves. (There's no reason for a resident to become a tourist and to move from one place to another; anyway, if an implementation allows the user to do this, it should take care of any security issue that may rise.) Tourists can be of any kind and freely move from one place to another (provided they get permission from the sentinels) traveling the IW. The implementation should provide a way for the user to create new tourists. Tourists can be cloned (with or without their baggage :). Also, the user has to have a simple way to instruct a tourist to do something (such as "Go on and search for this book!"). But why? Because it looks like a lot of fun. :-) And also because our brain is just a network of rather simple things... ;) Being visionary today, ;-) Gabriele. -- Gabriele Santilli <[giesse--writeme--com]> - Amigan - REBOL programmer Amiga Group Italia sez. L'Aquila -- http://www.amyresource.it/AGI/
[2/23] from: rebol665:ifrance at: 3-Dec-2001 22:52
Go Gabrielle, Go ! I am totally supportive to your challenge, however I don't see that newbie like me can be a part of it. Nevertheless I will enjoy seeing you all gurus and Rebol jedi in a fair and loyal joust. Patrick
[3/23] from: james:mustard at: 4-Dec-2001 12:46
Cool Idea Gabrielle, kinda like the Daliworld virtual ocean project and the Aglet system. This could be a fun challenge and with the ability of rebol to redefine itself dynamically the ability of self modifying Agents becomes a lot simpler. James.
[4/23] from: james:mustard at: 4-Dec-2001 14:39
Hi Gabrielle, All. /. had an article linking to a P2P project at University of Bologna (Italy) called Anthill. Code is freely available and they have an example of one which uses agents to simulate a document sharing network. http://www.cs.unibo.it/projects/anthill/description.htm Gabriele Santilli wrote:
[5/23] from: rebolmikko:yaho:o at: 3-Dec-2001 21:48
Very interesting, one of the best proposements for creating virtual world where AI would emerge from the interaction of artificial objects, users and programmers... Have you thougth about what could be the main goals in this world? The goals are needed for evolution to work... 8) Cheers, Mikko Mäkelä
[6/23] from: chaz:innocent at: 3-Dec-2001 11:51
Since a Tourist can carry baggage, wouldn't he be a special case of a Mail Agent who can only carry his own objects?
[7/23] from: chaz:innocent at: 3-Dec-2001 12:34
A site like http://www.cornerhost.com/cvs/ might be a better forum for the jousting than on this list.
[8/23] from: james:mustard at: 5-Dec-2001 23:23
Hi Gabriele, On further consideration REBOL is perhaps not the safest choice of language for this sort of project as there are lots of security issues involved due to its self-modifying abilities. In theory it would be rather simple to create a P2P agent world with REBOL but in practice you would have to severely limit the freedoms of the agents involved - almost to the point of re-writing the language. Let me give you some scenarios as an example: An agent is sent from one zone to another - is scanned by a sentinel for ??? The agent is then activated - at which point it begins to execute itself (assuming some sort of rate timer). If the agent posesses a physical face the possibility of malicious code is a very real threat - an agent could bring down an entire node by just accessing self/parent-face/... etc and deleting nodes. Other more insidious behaviours could also be initiated where an agent overwrites global functions and replaces them with its own ones, eg an agent re-writes the parse function, or the show function, etc. These behaviours become almost impossible to prevent by a Sentinel as an agent could (due to the power of REBOL) just hide command strings in an encrypted format and then decompress and run them with a little remold, do etc.. The only way to guarentee that none of this behaviour (from what I know of REBOL so far) is to either NEVER let an agent become autonomous to the point of being allowed to execute, to use non-changable templates for all agents that are checksummed for validity (this limits AI learning etc), or to never open up your local nodes to the rest of the world which kind of defeats the purpose of the exercise. IMHO the only truly safe way to ensure proper security would be to have the environment shell in a separate language that cannot be modified by rebol agents directly except in predefined ways- thus limiting self expansion or remote modification of nodes. Just my 5c :) James
[9/23] from: rpgwriter:yaho:o at: 6-Dec-2001 8:39
--- James Marsden <[james--mustard--co--nz]> wrote:
> Hi Gabriele, > > On further consideration REBOL is perhaps not the > safest choice of language for this sort of project > as there are lots of security issues involved due > to its self-modifying abilities.
As best I can tell, given REBOL's usual security features, you can do it if you use /Pro (or /Command -- the essential ability is launching seperate processes) and run the agents as seperate processes with appropriately limited "sandboxes" and let them communicate with the main "place" process via TCP/IP (with some type of RPC mechanism). At least it seems that this would "keep them in their place". You could probably do it in /Core or /View with a seperate native program to handle launching that the REBOL could talk to via TCP/IP, but that limits portability (although its easier to port a simple C-based server that does nothing but launch agents, etc., than the whole enviromnent, so its not so bad). Chris Dicely
[10/23] from: petr:krenzelok:trz:cz at: 6-Dec-2001 19:11
Christopher Dicely wrote:
>You could probably do it in /Core or /View with >a seperate native program to handle launching
<<quoted lines omitted: 3>>>nothing but launch agents, etc., than the >whole enviromnent, so its not so bad).
eh, what is the problem us usin 'launch function provided in both Core and View by default? -pekr-
[11/23] from: james:mustard at: 7-Dec-2001 9:04
Hi Petr, Problems exist when you want agents to be able to interact with each other and their environment - it is the interaction itself that can be a security risk as stated earlier, hence the need for some level of shell which is not directly linked to the rebol language. James.
[12/23] from: g:santilli:tiscalinet:it at: 6-Dec-2001 17:59
Mikko Mäkelä wrote:
> Very interesting, one of the best proposements for > creating virtual world where AI would emerge from the > interaction of artificial objects, users and > programmers...
Indeed, that's what fascinates me more about it. :)
> Have you thougth about what could be the main goals in > this world? The goals are needed for evolution to > work... 8)
I'd say that each agent has its own goal, decided by the user that creates it. Evolution, as you say, would need some sort of selection, but I'm already scared of the complexity that a network of a million places populated by some hundred million agents could represent... ;-) We're not even close to a human brain, but the IW can grow with no particular limits (except for available hardware/bandwidth)... Regards, Gabriele. -- Gabriele Santilli <[giesse--writeme--com]> - Amigan - REBOL programmer Amiga Group Italia sez. L'Aquila -- http://www.amyresource.it/AGI/
[13/23] from: g:santilli:tiscalinet:it at: 6-Dec-2001 17:49
[Been busy this week... still have to catch up on emails...] chaz wrote:
> Since a Tourist can carry baggage, wouldn't he be a special case of a Mail > Agent who can only carry his own objects?
There are two differences: Mail Agents, being residents (i.e. not moving around) can be identified; this way when you receive an object you are able to know from where it is coming from, thus deciding if to trust or not its contents; Mail Agents do not "move carrying objects", they send out objects to other mail agents (e.g. opening a new TCP/IP connection or whatever mean the implementation provides): this way the process can be optimized for transferring a large amount of data (imagine sending out music, or even a video clip); "roads" (i.e. the connections between places) are instead optimized for frequently sending small data (tourists with limited baggage, which might be frequently travelling the whole IW). Of course an implementation might blur this distinction, but there are cases where it is really needed if you don't want to create serious performance troubles (such as a lot of useless traffic). Hope it clears it a bit, Gabriele. -- Gabriele Santilli <[giesse--writeme--com]> - Amigan - REBOL programmer Amiga Group Italia sez. L'Aquila -- http://www.amyresource.it/AGI/
[14/23] from: g:santilli:tiscalinet:it at: 7-Dec-2001 12:01
James Marsden wrote:
> On further consideration REBOL is perhaps not the safest choice of language > for this sort of project as there are lots of security issues involved due > to its self-modifying abilities.
If you take the SECURE function into consideration, you'll see that you can be pretty secure; also, the implementation might use a custom dialect for agents (especially tourists), so that the amount of risk can be reduced to almost zero. (I'm quite sure the IW can be made much more secure than Outlook, for example. ;)
> An agent is sent from one zone to another - is scanned by a sentinel for ???
The sentinel might try to find information from the sentinel on the other side. If the sentinel on the other side says: "This agent was written by my user, so you can trust it if you trust my user", it may decide to let the sentinel in and give it permission to access some of the objects in the bank, or give it a certificate that allows it to be identified by the other residents and tourists. The implementation will need either to address all the issues, or simply run agents in a very restricted environment, where they cannot access the filesystem etc.
> eg an agent re-writes the parse function, or the show function, etc. These > behaviours become almost impossible to prevent by a Sentinel as an agent
On the contrary, protecting the system and unsetting uprotect might be enough. Or even don't let the agents access system/words, or resetting system/words at each call (code for doing all of this has been posted to this list in the past).
> IMHO the only truly safe way to ensure proper security would be to have the > environment shell in a separate language that cannot be modified by rebol
I respectfully disagree. :) Regards, Gabriele. -- Gabriele Santilli <[giesse--writeme--com]> - Amigan - REBOL programmer Amiga Group Italia sez. L'Aquila -- http://www.amyresource.it/AGI/
[15/23] from: g:santilli:tiscalinet:it at: 7-Dec-2001 12:14
James Marsden wrote:
> Problems exist when you want agents to be able to interact with each other > and their environment - it is the interaction itself that can be a security
It higly depends on the implementation IMHO. I've yet to see someone deciding to participate, tough. :) (Please note that it is not as difficult as it might seem. You don't need to be a guru to participate! :) Regards, Gabriele. -- Gabriele Santilli <[giesse--writeme--com]> - Amigan - REBOL programmer Amiga Group Italia sez. L'Aquila -- http://www.amyresource.it/AGI/
[16/23] from: james:mustard at: 8-Dec-2001 1:53
Heh - about time I memorized the REBOL dictionary ;) never had to use protect or secure before - time for a little reading! James
[17/23] from: greggirwin:mindspring at: 7-Dec-2001 10:44
Hi Gabriele, << I've yet to see someone deciding to participate, tough. >> I want to, but I have these annoying work projects getting in my way at the moment. :( One of the areas that interests me is self organizing systems. Regarding use of 'protect: Can you protect an entire context, or the words *in* a context? I haven't figured out how to do that yet. --Gregg
[18/23] from: rpgwriter:ya:hoo at: 7-Dec-2001 11:34
--- Petr Krenzelok <[petr--krenzelok--trz--cz]> wrote:
> Christopher Dicely wrote: > >
<<quoted lines omitted: 8>>> eh, what is the problem us usin 'launch function > provided in both Core and View by default?
Having not had occasion to use it in my experiments, I was working on a vague recollection that launching external processes was a /Pro feature. The manual is murky on what 'launch does or what its argument is, but I guess that might work. Which of course would make it easier. My point, mainly, though is that REBOL seems to be fine for this, from a security point of view, as long as you (among other things) run the agents in processes seperate from anything they aren't allowed to mess with. The rest seems like it should be able to be handled using REBOL's usual security features. Chris Dicely
[19/23] from: rpgwriter:y:ahoo at: 7-Dec-2001 11:37
--- James Marsden <[james--mustard--co--nz]> wrote:
> Hi Petr, > Problems exist when you want agents to be able to
<<quoted lines omitted: 4>>> level of shell which is not > directly linked to the rebol language.
I'm not sure why REBOL, as a language, is inadequate to this. Shouldn't the usual security features of REBOL, combined with having the agents *not* running in the same process as the "place" code, be able to provide adequate security? What is missing that you would gain through any other arrangement? Chris Dicely
[20/23] from: larry:ecotope at: 7-Dec-2001 12:19
> Regarding use of 'protect: Can you protect an entire context, or the words > *in* a context? I haven't figured out how to do that yet. >
PROTECT basically adds a protected flag to a word value. A REBOL value of datatype word includes a pointer to its context. The use of PROTECT will protect the word in that context. The argument to PROTECT can be either a word or a block of words. For contexts created by the function USE, which creates a local context for execution of a block of code, the use of PROTECT is straightforward:
>> use [a] [a: 1 protect 'a loc-a: does ['a]]
We have created a local context with the word 'a' being the only local variable. In the code block we create a global function LOC-A which returns the local word 'a', so that we can see what's happening. The word 'a' is both private and (after the PROTECT 'a) protected.
>> a: 52
== 52 The global word 'a' is set to 52.
>> get loc-a
== 1 We can access the value of the local 'a'. But it is protected from change.
>> set loc-a 2
** Script Error: Word a is protected, cannot modify ** Near: set loc-a 2 But because we have the word, we can use the UNPROTECT function on it.
>> unprotect loc-a >> set loc-a 2
>> get loc-a
== 2 For local contexts created by MAKE FUNCTION! and MAKE OBJECT! the situation is more complex and the usefulness of PROTECT is limited. BTW In future, with REBOL/Core 3.0, we have been told that REBOL will have modules, which should alleviate most of the currently existing namespace issues. -Larry
[21/23] from: rpgwriter:yah:oo at: 7-Dec-2001 13:46
--- Gabriele Santilli <[g--santilli--tiscalinet--it]> wrote:
> [Been busy this week... still have to catch up on > emails...]
<<quoted lines omitted: 7>>> object you are able to know from where it is coming > from, thus deciding if to trust or not its contents;
Surely, if you use an digital signature of some kind, you can also identify tourists. It would seem to me this would be a desirable feature, so you can distinguish between trust levels of various tourists. Chris Dicely
[22/23] from: greggirwin:mindspring at: 7-Dec-2001 20:52
Thanks Larry, << For local contexts created by MAKE FUNCTION! and MAKE OBJECT! the situation is more complex and the usefulness of PROTECT is limited. >> Yeah, that's kind of what I surmised. << BTW In future, with REBOL/Core 3.0, we have been told that REBOL will have modules, which should alleviate most of the currently existing namespace issues. >> That would be great! --Gregg
[23/23] from: g:santilli:tiscalinet:it at: 8-Dec-2001 18:03
Hello Christopher! On 07-Dic-01, you wrote: CD> Surely, if you use an digital signature of some CD> kind, you can also identify tourists. It would About tourists, I can see some problems. Tourist are not expected to be immutable: they will at least change state while they are travelling. (For example, a tourist that is searching for something will have at least two states --- "searching" and found --- and will behave differently depending on the state.) This way, the signature CAN change while the tourist is travelling, thus making any attempt to identify them rather risky IMHO. I'd like to know if someone knows a way to identify tourists that would not be easily hacked by a malicious user. Regards, Gabriele. -- Gabriele Santilli <[giesse--writeme--com]> - Amigan - REBOL programmer Amiga Group Italia sez. L'Aquila -- http://www.amyresource.it/AGI/
- Quoted lines have been omitted from some messages.
View the message alone to see the lines that have been omitted