AltME groups: search

Also, the link to the list of license tags from the [license] link 
of a script is broken.  You have an extra slash in there.

Ammon -- I can rename existing scripts. It's a moderator-only function.
If you want it done, please let me know which ones.

Maxim -- thanks....the diff function code is an embarrassing collection 
of random hacks. I'm hoping someone else will write a better version 
that could be made public :-)

Thanks Max -- I'll take a look at making that an option on upload.

sunanda COOL  :-)  and promote it on rebol week, on the ML and in 
"announce"  here too, really I think its a big deal/feature.


 I woundn't be surprised that most rebol.org users don't even know 
 the diff was there  ;-)

sundanda:  its cool for script owners.   It adds value in suddenly 
being more than just a storage space... I can even check my code 
as I submit it.

for me anyways, it adds a feature that I can really use ... an online, 
simple VCS.


I can make sure that a new release not only is shared, but also free 
of a set of bugs.  I can suddenly say.. what did I change... and 
more easily comment on it, for example.

Thanks for the script -- and for the problem report.

Looks to me like the script, as uploaded, contained non-UTF8 characters, 
and they are being treated as multi-byte characters.

REBOL, and REBOL.org can really only handle ASCII....Unicode and 
such like is for R3.


If you email (preferably in a ZIP to prevent email software from 
chewing up the code) the original script, I'll take a look.

In this way, the library accidentally supports utf-8 in the sense 
that if you upload utf-8 and display it in a browser set to display 
utf-8 everything will be displayed properly.

Supporting utf-8 will require a lot of changes ..... though probably 
not quite as many as moving to R3.

If most of it is currently ascii, would it not just be a case of 
adding a few filters?

Peter beat me to it, thanks.

Sorry Oldes, the Library does not support utf-8, despite my confused 
suggestion that it did.

Because we use a charset of UTF-8 in the browser header, it is _possible_ 
that we can more-or-less handle scripts with 2+ byte UTF-8 codings 
in REBOL strings! But that's not been tested.


Good point, Chris -- we already have such a filter, but it is not 
used to turn back conributions.

We've had a similar problem before (I've just checked the source 
code, and it's prompted my memory).


To solve it, we analyse the script for various extended ascii chars 
and then perform some messing around on HTTP content-type headers.


It's messy, and it's worked up til now......But obviously, we need 
some more analysing and messing around for this script.

Results of a tiny bit of debugging on the ascii chars problem:
-- problem seems to be at the input stage:

     -- if you have exended ascii characters (top bit set, like the 1/4 
     used in the script) what we get from the webserver is bad (extra, 
     unexpected extended ascii chars)

    -- only download is (visibly) affected, although the extra extended 
    ascii chars are present in the text streams

     -- though there is some REBOL mezz code (decode-cgi) that may be 
     doing something I do not understand

    -- I can replicate the problem with both Apache and Xitami which 
    suggests the problem may be in REBOL rather than a given server.


-- the quick fix would be to add accept-charset="ISO-8859-1" to the 
<form ....> or <textarea ....>

    -- but that stops all extended ascii, including the ones we want. 
    So we won't do that.

-- the slower fix has yet to emerge from the available options.

At the moment, I'd be worried about standarising the Library on utf-8 
as the effect of multibyte characters would have during script and 
mail processing is not understood. It could well be that the system 
handles multibyte characters without a hitch but nobody knows yet.


I have started to write some scripts to try to help move to a consistent 
character encoding of the Library data but, due to time constraints, 
I have been very slow.

Anton, REBOL.org uses 2.5.6.4.1

The obvious bad file is the one Scott added recently:
http://www.rebol.org/view-script.r?script=ascii-math.r

If you view it with that URL, all looks good.

If you click the [Download script] link you'll see many spurious 
high-ascii chars in the source.

Those high ascii _are_ actually in the source. But where they came 
from is a mystery.

Maxim, REBOL.org emits a header

   <meta http-equiv="Content-Type" content="text/html;charset=utf-8">

Yeah, I know we aren't utf-8 -- but experiment has shown that's the 
moste acceptable charset.

Not sure what you are saying we could put in <head> -- can you be 
more specific.

there is a specific charset for western -iso, which ensure the extra 
127 bytes are correct.


<meta http-equiv="content-type" content="text/html; charset=ISO-8859-1">

I think the root of the problem is that when the Library system was 
first written, no account was taken of character encoding. As a result, 
not only is the data encoded as it was when originally submitted 
but the method of encoding is not even known.


Whatever charset is specified in the http header is not going to 
be correct for all scripts and messages. Using charset=utf8 seems 
to cause the least problems. Though for example, it will cause many 
ISO-8859-1 "high bit" characters to be incorrectly displayed.

No actual stats. Just from feel:
* Scripts -- very few
* Posts on the ML -- a few dozen
* AltME archive -- no idea

Thanks Gabriele -- that's a clear explanation, and has helped me 
work out what is going on.


Anton and Gabriele -- I have tried changing the charset we emit on 
the download to say UTF-8. But that makes little difference. As both 
of you note, once the file has been saved then (without a MAC-type 
resource fork) there is no obvious indication of the encoding. And 
several editors I have tried get it wrong -- thus "revealing" the 
extra ASCII chars.


Not sure what the solution is other than to de-UTF-8 files on download.

Anton -- not yet run a crawl to check for other scripts with high 
ascii chars.

Which editors?

I think most editors these days allow manually changing the encoding, 
so developers who notice strange characters can just change it themselves.

Maybe it would be helpful to add a rebol.org library script header 
advertising the encoding (when it is known, and when not).

I don't recommend 'de-UTF-8'ing files on download - that's just going 
to confuse things more, especially when the file is view-script.r'd 
as utf-8 just beforehand.

It seems the responsibility lies with the clients to interpret encodings 
properly. As we move to a unicode world, software assuming 8-bit 
encodings are some ASCII encoding should drop off. But until the 
transition is complete, there's not much we can do about client software 
guessing wrong like that, except stating the encoding in the script 
header, in the web page that provides the download link, and by helping 
confused newbies.

If the offending downloaded script is executed in Rebol/Core, the 
extra ASCII chars are also present in the executed code.  The script 
defines ½ to be 0.5. If "help ½" is typed into the console, the result 
is "Found these words:   ½              decimal!  0.5". However, 
if the script is executed in Rebol/View, the result is "½ is a decimal 
of value: 0.5". It seems that View handles it correctly, while Core 
doesn't.

Thanks guys.
Other scripts with the same problem.....there are a couple. 

About 10% of all scripts have at least one extended ASCII char....But 
most of them are acceptable in LATIN-1 code page / charset (eg copyright 
symbol, some accented letters). It's just a very few scripts that 
use 1/4 and similar symbols that cause the problem.


What other editors? Windows NOTEPAD is one example of a common one 
that gets this wrong.

Of the various editors / word processors I have immediately to hand:

-- credit.exe -- [my usual editor] shows incorrect chars, and has 
no option to switch to UTF-8

-- open office writer -- works fine if you take the UTF-8 option 
when asked
-- ms word -- claims file is corrupt
-- word perfect -- makes a complete mess

-- R2/View's built in editor ( editor %/c/path to my local copy//ascii-math.r) 
-- shows incorrect chars

Ok, so there are some editors which don't support unicode, don't 
guess encoding correctly, or can change encoding only with difficulty.

How about this suggestion; if a rebol.org script is known to be UTF-8, 
then an additional link should appear:

[Download as ASCII] download-a-script?script-name=ascii-math.r&encoded-as=8-bit-ascii
which transcodes a UTF-8 file to ASCII.
Just have to get a conversion function in place for this to work.

Sunanda: given that R2 uses the host current code page, I think the 
best way would be for the user to convert the script after downloading 
it. On Linux or Mac for eg, UTF-8 is perfect for Core scripts as 
the terminal is UTF-8. On Windows or for View scripts, you'll get 
the host code page displayed anyway, so the user has to do the conversion. 
A tool to do that automatically would be nice (I have the code, it 
will be released soon, but you may need to wait a couple weeks more).

All these troubles go away with R3... but I think it would be nice 
if R2 recognized UTF-8 and converted it on the fly; we could add 
a BOM at the beginning to make that easier.

Yes, use of BOM has its own troubles. I don't think it's a good idea.

Thanks everyone.

I think our first step is to add a warning to any download for scripts 
that contain UTF-8 chars.

So, for that I need a function:

     utf-8?: func [data [string!] [ ...]   ; returns true or false [and 
     perhaps "not sure" in ambiguous cases]

I've done the easy part :-)
Can anyone help with the difficult "..." part ?


It is not as simple as just looking for ASCII > 128 .... some high 
ASCII is acceptable as part of, say, ISO 8859-1

I have a function which finds utf-8 multi byte character sequences 
in a string. Given the code ranges for mulit-byte characters, it 
would be rare to find such a sequence accidentally.

It's about 65 lines so rather than post it here I will email you 
a copy.

I understand what you mean, and obviously the definition of "legacy" 
is a bit fuzzy.

Using Peter's code (thanks again!), I've made two changes to the 
download-a-script link:


1. if we find UTF-8 chars in a script, we download it with the HTTP 
content type charset=utf-8


But that probably makes no practical difference.  A downloaded script 
will be saved by the browser, and then opened by a text editor. The 
text editor is unlikey to be passed the charset setting. So:


2. Scripts with UTF-8 encoding are downloaded with a few lines of 
comment at their top. The comment explains the possible problem.


Thanks to all for the comments and help with getting things this 
far.

sunanda:  I have a feature proposal for you  :-)


it would be nice to be able to supply a single picture to link with 
the scripts. this image (jpg, png, gif) would have hefty size limitation 
and I think only one image per script should be enough, but having 
this alongside the various listings of the application and within 
searches, new scripts, etc would be really cool.


sometimes, if you see a thumbnail (ui grab, console example, logo, 
output gfx, whatever), it will help raise people's curiosity.  this 
could probably benefit quite a few scripts, which are possibly overlooked.


having a simple search filter of scripts with pics, could also help 
people to quickly find usefull things at a glance.


what do you think?  it could start out really simple, and slowly 
thumbnails could creep into various listings of scripts.

re:pictures. Some but not all scripts also have docs, so that might 
be a good place to add them.For those that don't,a clickable small 
thumbnail?

Max, I assumed you meant have the pic on the main page for the script, 
eg for liquid.r you'd see a thumbnail here:
   http://www.rebol.org/view-script.r?script=liquid.r

That's a nice idea, though there are some technical CSS issues......For 
example, the actual script is displayed in a <pre> block. That means 
images may not float where you'd expect them. It'll take some experimentation 
to find the best way to do it.

....Maybe a better slot for a thumbnail would be in the LHS menu, 
just under the <Script Library Home> link. That would keep it out 
of the flow of the page.
Please suggest better ideas :-)

There are two file versions in the library, one for viewing, one 
for downloading. Did you use the one
from http://www.rebol.org/download-a-script.r?script-name=....r
Maybe the other ones have problems.

Mike

I checked your library example from the I'm new group producing errors.

There is probably a weakness, as the script does not regard comment 
lines.
A short enhancement would be
   
parse-ini-file: func [
    file-name [file!]
   /local ini-block
    current-section
    parsed-line
    section-name
][
 ini-block: copy []
    current-section: copy []
    foreach ini-line read/lines file-name [
		if #";" <> first ini-line [ ; do not process comment lines
			section-name: ini-line
			error? try [section-name: first load/all ini-line]
			either any [
				error? try [block? section-name]
				not block? section-name
			][
				parsed-line: parse/all ini-line "="
				append last current-section parsed-line/1
				append last current-section parsed-line/2
			][
				append ini-block current-section
				current-section: copy []
				append current-section form section-name
				append/only current-section copy []
			] ;; either
		]
    ] ;; for
 append ini-block current-section
 return to-hash ini-block
 ]

Thanks for your attenton to my questions. 

It seems I stumbled across the need for support of comments in ini 
files. I was also trying to cut & paste from the viewing version 
of some scripts, rather than the download version as I did not realise 
there was a destinction.  It is also possiable that my cut & pastes 
were not complete perhaps, as the scripts that would not run like 
that before seem to work ok now.


I notice that a lot of things show anomolous behaviour when used 
by inexperienced users who lack confidence. It is like they know 
who they can play tricks on, & who won't stand for it.

Cut'n'paste ought to work. Though the download format is safer as 
it has been less processed (no escaping of embedded HTML codes etc).

I can sympathise -- I generally notice _any_ new device I buy (from 
computers to wrist watches) won't work for the first half hour or 
so, But as soon as I start to think that either it's defective, or 
I am very stupid, then it starts to play fair.

Animism is still a good first attempt at understanding the universe 
:-)

yes, in the script description above the start of the script, under 
the :


     [View in color]   [View discussion [1 post]]    [License]   [Download 
     script]    [History]  [Other scripts by: moliad]



menu... just a list of uploaded images with a one line caption when 
you click on it, showing the full image. obviously, you should impose 
strinct size limits, enforcing proper compression.  and potentially 
a maximum size.

Just replied to a Feedback message sent to REBOL.org about the Script 
Library:
 
> When will the current script library "die"?

[The questioner suggests that R3 incompatibilities may mean we simply 
purge all R2 scripts to create an R3-only Library]

My response......

Thanks for the question. The simple answer is: I hope never.

***


Coincidentally, I have just started an exercise in seeing how many 
of my contributions to the Script Library will port with few or no 
changes.  The results are encouraging so far:
   http://www.rebol.org/art-display-article.r?article=j26z


I know some of my scripts will be obsoleted by R3. They can stay 
in the Library as R2-only resources.  I am hoping many of the rest 
will be code compatible between R2 and R3, so they will work either 
way.

***

Ladislav has also commented on his early porting experience:
  http://www.rebol.org/art-display-article.r?article=w24v

There is some discussion of the issue on the REBOL3 AltME world:
  http://www.rebol.org/aga-display-posts.r?post=r3wp453x15753

***

The Library has a flexible tagging method for scripts, see:
   http://www.rebol.org/st-topic-index.r

We can very easily add tags for (say)
   R3-ready
   R2-only


Or whatever seems the best set to help partition the scripts into 
R2, R3, or both.

Hope that helps a little!

I'd noticed that change -- it's a good one, thanks.

As Carl mentioned in Announcemements:

www.rebol.org is being moved by the ISP to a faster server.  I'm 
not sure they understand what's really involved, so it is very likely 
that the site is going to break.


I'm just going to set the site to read-only, as I am not sure they 
will be able to backup and copy everything while we are adding ML 
posts, scripts etc.  I hope normal service wiill be resumed asap!

REBOL.org died last night because:

 -- the ISP copied the entire site in preparation for a server migration

 -- that doubled our disk usage ..... and took us over our disk space 
 quota
 -- so all file writes failed

 -- but they failed by writing zero byte length files. result data 
 loss and corruption of indexes
We are now restored and back up and running.


The server migration is still to come. As Carl says, it may  break 
things in other ways.

uh? .. so you lost all data or a part of it or none?

We lost no data.


But we lost the site for a little while.....Little things like this 
built up:
    -- you try to login
    -- we update your last-active time

    -- but that file write completely trashed your user-profile record
Or:
  -- we upload some more posts from this AltME world

  -- as we write them into our index data structures, it corrupts the 
  index, thus making the whole
     AltME archive unavailable.

Corrupt-on-write is a hard mode to recover from :-)

huh..  writes failing by zeroing files is a thing that can cause 
quite few problems .. haven't been aware of that

Just tried logging on without success ... is this due to the above 
problems or do I require a password reset? (I've got a couple of 
useful scripts I finaly want to upload).

I checked the user list and "Ashley" isn't displayed. This could 
be or one of three reasons:
1. You requested privacy when you first joined.
2. You used another name.
3. You membership expired


If I remember correctly memberships can expire if you haven't added 
a script to the library and haven't logged in to the site in the 
last six months.

REBOL.org just migrated from the world's slowest single server to 
a small load balanced cluster.
Sadly, we're not superfast, but its early days yet.


One cosmetic problem: we're showing the date as 6 hours out. Highly 
trained techies are working to resolve that. Other than that, it 
was a smooth move, mainly handled by the ISP.

just noticed that downloading scripts seems to add a .txt extension 
after the .r   so you get  script.r.txt

I just tried  a script download under Windows:

-- when I selected "save to disk" it was saved as a *.r file as I';d 
expect
-- when I selected "open with notepad" it got named *.r.txt"

Google changes -- do you mean the whol /cgiwrap/ stuff in the middle 
of a URL?

here is an error I got trying to go to the http-tools.r script...

Not Found


The requested URL /cgi-bin/cgiwrap/rebol/view-script.r was not found 
on this server.


Additionally, a 404 Not Found error was encountered while trying 
to use an ErrorDocument to handle the request.
Apache Server at www.rebol.org Port 80

Thanks -- I see that too......We need a tweak on the URL rewriting.

Is there a way to provide page numbers into the altme messages?  
It's only got newer and older to allow pagination.

Also, can we have a smarter parser for obfuscating email addresses: 

bugs	mysql://[root-:-localhost]/bugs1

The email obsfucation is based on Andrew's  code. Improvements are 
welcome!
http://www.rebol.org/view-script.r?script=uri.r


Meanwhile, we do not obsfucate if you are logged on. So log on for 
a better browsing experience.

there are a lot of things like that to take into account... so i 
think it's better to start collecting ideas.

Italy is always a good idea.

I meant to do a call for papers/presentation abstracts earlier, but 
have been busy, the time is still good.

I should mention... I was wondering if we should have DevCon every-other-year... 
  A year is such a short period of time!

(We need a 1000 day year. That would be much better.)

If we are going to stretch it beyond a year, I personally wouldn't 
want to stretch it beyond 18 months...

A collaborative meeting with worldwide attendance.

if possible, also bring a bit better video and audio recording equipment. 
I liked last years videos, but the quality was poor

the quicktime server was extremly null I thinl it could be good to 
start to seek  for a better technology since to have to research 
one 2 days before the conf  ;)

also, while the stream camera would mainly point to the screen, the 
recording camera would point mainly to the speaker, since with minimal 
editing you can just make a screenshot of the slides and insert them 
in the video at the appropriate places.

about the place, here where i live would be cheaper (and we have 
some good winemakes in this region too ;), but it's not very easy 
to reach (the closest airport is Pescara, but it's a small airport; 
the closest international airport is Rome)

I do not like Milan from a turist perspective, no mountains, hills 
, no lake , no see, nothing but grey buildings, of course some nice 
churches butof those there are many in Italy.

Being so that we need lot of brain exercises during the conferences, 
would you prefer a cahotic metropolitan city that would take you 
anyway a whole week to visit or a nice quite relaxing place with 
sea view, good food (that is almost anywhere in Italy) ?

It is already a time to do a call for papers/presentations to build 
up publicity, and to do a press release, which gives plenty of time 
for it to google up.

If it is decided to do, then first press release could be one for 
call for papers, not naming exact location, only the name, then when 
a location decided, a second press release. (just wanting to throw 
some ideas from last years experiences)

The classes or scheduled things should be with plenty of time between, 
maybe alternate between a lecture and then a discussion activity 
so people have a longer time to "network".  And a big clock on the 
wall to help manage time...;-)

A bad thing about the hotel last year is that the conference room 
was not close to the lobby.  A better setup might be a good conference 
room, and a secondary room for "networking" / labbing (or just a 
room with an open door adjacent) with the conference room near the 
lobby so the flow of REBOLers is always near it, and the schedule 
posted there with any announcements.

I think we should try to get the conference to be hosted in a university, 
and exponsored by the university. Plus is that you may attract some 
students, also costs will lower, and the university campus will be 
nicer.

as someone who tried to watch the proceedings of the last devcon, 
it was not so much the video feed that was of value but the slide 
content, audio and perhaps the role altme played in proviiding feedback/questions 
from those not present , perhaps a tool could be built to facilitate 
this process ?.

University: i know a lot of people at the faculty of engineering 
at the university of L'Aquila. the closest airport is still Rome 
though.

I have a very good friend of mine who is winemaker. Maybe we can 
arrange an event and have some wine tasting.

maybe we've found a place; it's near Milan though. i'll let you know 
:)

will have to give it a miss this year...

I will be in France end Oct though...

is there a list of good places to stay in that area?

we're going to have all the info available soon. i'd like to have 
a trip in milan organized so that we can visit the city etc. If anyone 
is landing in Rome, we could also visit it too.

I looks like a boot

IMO it's a very nice time to go to Italy. And Milano is a very nice 
city.

Gabriele - where in Italy would it take place? The thing is - should 
we travel by car across the Europe, or will it be possible to to 
taky a flight? Will it be near enough of any airport?

Ammon, I seriously doubt I have the money to pay for a trip to Italy. 
 its a trip worth a few thousand dollars.

Michael: form result should go to email automatically. you can send 
me a mail manually if you want. if there's a problem for enough people 
with the email form, i'll look into setting up a cgi.

Graham: i've been thinking a lot about how to make it so that 1) 
it could be done in a couple mins 2) could be accessed by anyone, 
even without rebol or plugin. this seemed like the simplest and most 
accessible way. which shows that the web is too complicated.

petr: Milano is closer to the place where the devcon is. from Roma 
you have to travel across half Italy. so, if you have a limited time, 
Milano is best. while if you have enough time and want to take the 
chance to visit italy, you may prefer Roma.

Milano is more a "business" kind of city. Roma is probably more interesting 
from a tourist point of view. but it really depends. Milano can be 
very interesting too.

i think we should consider Roma only if we manage to get an arganized 
group there. that would mean a lot of fun.

btw, everyone feel free to submit the form even if you are not sure 
about your answers. it's just getting a mail to me so if you change 
idea you can just send me another one or send me a note manually.

Accessed without Rebol ?  I wonder if you *really* believe someone 
who does not have rebol installed would be coming to a devcon!

you may be on a pc without rebol. but - why not? ;)