Library & smart docs => was:{Re: [REBOL] Re: Parsing comment}

Glad you steered the topic this way! ..if the project gets off the ground here's some of my recent thoughts about it.. Rebol Library contributors should be able to add comments and explanation to code source *in-line*. But how to do that without making a mess of the so? Too much of good think is not a good thing, especially when its Rebol! Too many comments would bloat the size, and worse make it hard to read. If comments are added only at the head or tail then it the larger size is not such an issue. That's like a wiki-blog where comments are 'beads' - a set of threaded files joined together in sequence. Vanilla creates snips named "comment-{snipname}" In-line comments could be brilliant though. Tutorials are what beginners lack - running commentary. So we need a way to manage the Library which accomplishes several things: 1. Keep it up to date and intelligently structured. 2. Indexed and searchable, returning useful results. 3. Open for contributors to add comments, examples using scripts, and links to others. 4. Open for contributors to suggest special in-line comments suitable for generating tutorial versions of the scripts 4. A review mechanism so the Librarian can review all contributions and include them easily, select and include the changes 5. Notify people what's new. ie: precise links when changes have been committed How to do this? Vanilla can do the content management issue right now. Andrew Martin's wiki can too I think, though I've not used it. Brett said he was working on CMS stuff too I think.

'Leo' can do this !! It handles prefix- and suffix- type comments, structural code/comment includes and {tada!} in-line comments. I have been learning to use Leo better over past few days. Leo is a powerful tool to organize and manage source changes with comments and flexible cross linking/categorizing. I just added Rebol to Leo's list of language types. Now when I 'saves' a project several things happen: 1. Leo saves everything as a single ".leo" file This is a legit XML file which can be processed by anything. The XML markup includes syntax so when Leo opens these xml files the tree is rendered correctly with all metadata, cross-links and content intact. Any nodes marked for source are also saved out automatically in their native formats. They can be edited externally. The Leo file will include them correctly and update itself:-) 2. Node in the tree marked with "@file somename" will be saved as an external file. 3. Nodes marked "@language rebol" are saved as ".r" rebol files. Leo's own markup are converted to comments with ";" Other ['--language'] types include Actionscript, Java, C, C++, HTML, Python, PHP. 4. There are a small number of other Leo directives for managing comments, includes and special features. @ignore does not export. So a page of notes within a node can be left out of a the rebol file. @first lets one specify the bangpath statement 5. If you don't want any trace of the Leo markup, that's easy to do with command 'RemoveSentinels' Voila! 100% pure clean sweet short REBOL scripts. 6. Leo lets one reuse the same content in several ways. 'Clones' are aliases used within the outline tree to make it easy to have the same content under different category headings in different places. Combined with in-line insertions it's very powerful. Since Leo files are 100% XML, on can use XSLT to process them. For example the same source saved in a Leo outline file could be used to generate rebol, Easy-VID, HTML, colorized stylesheet layout HTML, and even an entire linked Library site. A good use of XSLT would be to generate HTML from Leo of rebol source code so that in-line tutorial comments can be "folded/unfold" with a click like XML does in MSIE. Another is sophisticated dynamic stylesheets are easy to do because simple user requests can trigger changes. For example user search would 'highlight' and 'expand' results within code. The decision to use REBOL vs. XSLT is an open one. One can mix and match them even. I have only just got into XSLT. I was scared and confused and suspicious of it before. I am starting to see that it has much in common with REBOL. REBOL is always shortest and sweetest it seems. But XSLT has the advantages of immediate inter-operability with a rich set of tools, docs, and tutorials. For content already in XML, XSLT makes pretty good sense. Some say it's too cluttered and complex. XSLT is very versatile. It does not even have to be used with XML. For example, XSLT could be used to convert a Leo REBOL Library tree into MakedocPro or Easy-VID files. I don't have enough experience to know which is easier for parsing and exporting a big XML source input file like that. LIBRARY - 5 solutions A. KISS-XP Forget all this. Just add some comments to the source. Ask people to add in-line notes. Librarian reviews and commits via simple expedient FTP/HTTP/REBOL tools http://www.rebol.com/docs.html B. VANILLA Vanilla can do all the above too. It's a 100% rebol web-based solution. Vanilla has a simple elegant architecture with custom scripts, comments and in-lining now. Con: It's Rebol but hacking around takes time because severe lack of docs. http://www.langreiter.com/space/Vanilla C. LEO Use Leo now. And 'publish' to multiple formats including Vanilla for web site presence. Librarian needs to try Leo and learn it. Leo benefits from openSource community and has strong interoperability. Needs Python skills to customize. http://personalpages.tds.net/~edream/front.html D. EASY-VID LIBRARY Build a version for the Library in 100% Rebol, adapting some of Leo's good ideas. Easy-VID sort of does this now. It has the wonderful and rather unique ability to execute code with a single mouse click, right in the middle of text. It's an incredibly persuasive demo of Rebol itself. Two problems I see in Easy-Vid based solution is that it is not web friendly and enabling.disabling so that needs to be managed. http://www.reboltech.com/library/html/easy-vid.html E. REBOL APPLICATION FRAMEWORK Robert's new 'Rebol Application Framework I need to read the white paper carefully and catch up on the RFC thread. To understand what it is and if its relevant here. Anyone? http://www.robertmuench.de/rebol-framework.html ./Jason

Hi Jason, VidWiki uses some of the easy vid code. The vidwiki client views dynamically (CGI) generated layouts, and the layouts can contain instructions to view layouts within the layouts. I did manage to convert several of the RT Howtos to this format with some modifications. But there is a limitation as to what you can do, and it doesn't scale well for larger pages. However, I guess some people have found it useful. At last count there have been over 8300 accesses to the script :) -- Graham Chiu

Thanks! I had looked some time ago, but forgot about it. What does not scale? btw, google does not find it, but finds your post which includes a rebsite url http://www.rebol.org/userlist/archive/504/648.html Whatever/whomever/however, the new improved Rebol Library Project [RLP] needs an effective strategy to make rebsites visible to google. Just a script which automatically parses a rebsite and generates a google-friendly html with <meta> tags would do it. Perhaps that's an easy add-on to %WidWiki.r Is /IOS also always invisible to google, or is there an option to enable google-vision? Have you tried Vanilla? Could be nice to connect VidWiki to Vanilla, adding 'browse', 'view', 'publish', 'set' and 'get' functions to both. ./Jason

Hmm. I thought that there was not supposed to be a direct link anywhere to that archive to prevent spiders getting to it ... as email addresses are visible on those pages.

I think that all dynamically created sites tend to be invisible to search engines. Zope is an example. I'll have to think about the latter to see how easy that might be. But as the vidwiki.r came from wiki.r which originally generated html, I guess it shouldn't be that hard :)

IOS is invisible as clients have to be authenticated. Communications is also encrypted.

No. I've seen it though.

-- Graham Chiu

Well, spiders don't get to sites if they observe robots.txt, which spam address harvesters most certainly don't.

Not exactly. Search engines have avoided URLs with query strings for a long time, but how would they distinguish static from dynamic content? They cannot, it's all bits to them (and us, for that matter ;-). -- c

We had a discussion before about the rebol.org archive, and Jeff removed all links to the archive. To get the address, you have to use the link on the RT rebsite. So even a spam address harvesting robot should not get there. But I guess google had it in it's database prior to that happening.

In practise they tend to be invisible. Here's an article from Paul Graham making that point http://www.paulgraham.com/mistakes.html and in my Zope site, I have this in my robots.txt to make me invisible! User-agent: * Disallow: /Shopping/ # This is an infinite virtual URL space -- Graham Chiu

It is also possible that people added a link to this page within their own website. This way google will access it while harvesting those people's sites. It's hard to keep a useful page from being referenced from somewhere else ;) Regards, Laurent -- Laurent Giroud