AltME groups: search

Having said that, I think Bo has written a utility to spider a website

the example Mike liked is:

    view layout build/with [
        box 600x600 effect [
            draw [
                spider [
                    size 600x600
                    ; offset 100x100
                    pen black
                    scale 4
                    ; scale [0 150 300 450 600]
                    categories [

                        "Category 1" "Category 2" "Category 3" "Category 4"

                        "Category 5" "Category 6" "Category 7" "Category 8"
                    ]
                    directions
                    pen red
                    data [100 200 300 400 500 600 700 800]
                    pen blue
                    data [100 100 100 100 100 100 100 100]
                ]
            ]
        ]
    ] [spider: :spider*]

(it builds a DRAW block creating a Spider graph)

** Script Error: spider* has no value
** Where: build
** Near: spider: :spider*

Spider is: http://www.fm.tul.cz/~ladislav/rebol/spider.r

>> do http://www.fm.tul.cz/~ladislav/rebol/spider.r
connecting to: www.fm.tul.cz
Script: "Spider" (8-Sep-2006/16:40:58+2:00)
** Script Error: lfunc has no value
** Near: spider*: lfunc [
    [catch] 

    description [block!] "a block containing a description of a chart"
] [] [
    current-...
>>

it wants to find the file in my local cache before the remote one 
and crashes:

>> do http://www.fm.tul.cz/~ladislav/rebol/include.r
connecting to: www.fm.tul.cz
Script: "Include" (5-Jan-2006/14:31:42+1:00)
>> append include-path http://www.fm.tul.cz/~ladislav/rebol/
== [%. %/Volumes/c/rebol/ http://www.fm.tul.cz/~ladislav/rebol/]
>> include %spider.r
connecting to: www.fm.tul.cz
** Access Error: Cannot open /Volumes/c/rebol/spider.r
** Where: include-script
** Near: found: load/all target 
if header
>>

Sorry, if I'm being a little difficult :-)

back to the example: what exactly does Spider do? I don't think it's 
very clear from the example what's going on. there could be hundreds 
of lines of code hidden there.

you need to have a look at http://www.fm.tul.cz/~ladislav/rebol/spider.r
. It generates the spider graph as you could see, if you succeeded 
to run the code

I saw it. The draw block would then be evaluated from the spider 
code and then inserted at that place?

yes, you can see the result of it if you do:

probe build/with [
        box 600x600 effect [
            draw [
                spider [
                    size 600x600
                    ; offset 100x100
                    pen black
                    scale 4
                    ; scale [0 150 300 450 600]
                    categories [

                        "Category 1" "Category 2" "Category 3" "Category 4"

                        "Category 5" "Category 6" "Category 7" "Category 8"
                    ]
                    directions
                    pen red
                    data [100 200 300 400 500 600 700 800]
                    pen blue
                    data [100 100 100 100 100 100 100 100]
                ]
            ]
        ]
    ] [spider: :spider*]

Well, how would you do the

view layout build/with [
        box 600x600 effect [
            draw [
                spider [
                    size 600x600
                    ; offset 100x100
                    pen black
                    scale 4
                    ; scale [0 150 300 450 600]
                    categories [

                        "Category 1" "Category 2" "Category 3" "Category 4"

                        "Category 5" "Category 6" "Category 7" "Category 8"
                    ]
                    directions
                    pen red
                    data [100 200 300 400 500 600 700 800]
                    pen blue
                    data [100 100 100 100 100 100 100 100]
                ]
            ]
        ]
    ] [spider: :spider*]

example without using BUILD?

by looking at it, SPIDER looks like a regular function to me with 
the following block as input. I guess it can't be, because then the 
layout block could simply be done by enclosing spider [...] in ()'s 
and compose/deep the whole thing.

another example: if you look at the SPIDER implementation, you will 
see it is implemented using BUILD too. I bet it would be much harder 
with compose, because there *are* parens in the implementation.

For example when I use the word 'spider, I am pretty sure, it cannot 
clash with any "native" DRAW command, because spider isn't native 
DRAW command. Does that make sense?

Nice idea, Graham..... But the REBOL script might be a spider/inder 
like RIX; and so expects a HTML page.
We can't second guess cases like that.

yes you can ... because a spider will identify itself

petr, why not just including a simple html links somewhere in the 
page? you can even hide it with js, so that a user does not see the 
links, but a spider will.

Javascript only links will stop pretty much every spider.....So you 
won't appear in many search engines at all.

You will also be breaking a lot of people's assistive technologies 
(like screen readers for visually disabled people).

Best to have a site that is fully navigable via plain HTML links. 
Then add the clever stuff on top of that.
Google's advice:

http://www.google.com/support/webmasters/bin/answer.py?answer=35769
http://www.google.com/support/webmasters/
***

A high score here usually means you have  a person-friendly and spider 
friendly website:
http://www.cynthiasays.com/

If you can get to them from the root, then they are fair game, unless


.....they have a rel=nofollow......We have that on a few simply because 
they duplicate content (eg viewing a script, viewing a script in 
color, downloading a script


....Mailing list -- best to index either the individual posts (http://www.rebol.org/ml-display-message.r) 
or the complete threads (http://www.rebol.org/ml-display-thread.r) 
but not both.


....you may get a __lot__ of duplication when spidering the AltME 
archive as every post has a URL, but we display in batches of 50.....So 
perhaps only spider URLs like
http://www.rebol.org/aga-display-posts.r?post=r3wp291xNNNN
when NNNN is 1, 51, 101, 151, etc....


....I think You already have indexed the ML as on REBOL.,net and 
Carl's latest 300 AltME messages, eg
   http://mail.rebol.net/cgi-bin/mail-list.r?msg=45305
   http://host4.altme.com/altweb/rebol3/chat771.html

It would be better _not_ to have index those; it just creates duplicates 
once you have indexed the equivalents on REBOL.org (especially as 
the AltMe last 300 goes out of date so quickly).

Tell me what is unclear there!

Spider graph.

Nice spider :-) http://www.engadget.com/2010/11/26/land-crawler-extreme-robot-carries-175-pounds-of-human/

A spider web, where everyone in the organisation is controlled by 
one head