AltME groups: search

maybe use Git as the basis for a new chat system

the number at the end gives the impression that its a world to speak 
about that version of rebol.

the R3 chat architecture is VERY nice... it just needs a GUI client.

Well, I'd imagine that this is a major problem for Altme ... so Carl 
will want to fix it.

I just tried sending Gregg a PM on REBOL-gate.

REBOL-gate is a tiny world that runs on the same server as REBOL3.
The PM took under a second.
That suggests world-size scaling is the issue to focus on.

Yes, Graham, my point is that I don't think this will be a priority 
for RT to address unless a hosted client has a big world.

I've just asked Carl to help in maintaining Altme in another comm 
channel... hopefully he'll see that and reply to me in a not to distant 
future.

IMO the fix is simple. Use a database as backend. Never understood 
why this step is so hard to do. Even with 1 Mio. messages you won't 
feel a slow down.

Interesting, why use a DB and add bloat, why not just write just 
enough code to keep it fast?
For example, loading in messages in groups of 100....

I love AltMe, but it has things that annoy me as well. Since we don't 
know exactly why it doesn't scale (yes, we can guess), I don't think 
just making the back end a DB is a guaranteed fix, or perhaps even 
that simple. And I don't think it will happen. I fear the R3 chat/rebdev/??? 
model will scale even worse than AltMe as I think the msg db is a 
single file and still a naive implementation designed for small-scale 
use.


Like Graham, I've thought that something like Git--a system designed 
for distributed, differenced, repositories of information--might 
be used. I haven't seen it done however.

What ar our options?

1) Live with AltMe, and roll worlds when they get too slow.
2) Find an alternative.
3) Implement our own.

Is #1 painful enough to drive us to #2 or #3?

actually the r3 chat system *client* is still pretty simple... but 
the way messages are managed via the server seems to be much more 
flexible.  every msg and file is its own single item.  they are downloaded 
as items, not as groups of items.


so actually, all that needs to be done to make r3 chat scale is to 
improve how it stores its messages on disk and on RAM.   


the fact that the server supports moderation, threads, user levels, 
files, and things like item re-classification is very nice.


really, all it needs is a gui client.... that's If the server is 
able to respond quickly to requests.

Better to start a new world than to stay in this one and see personal 
forks like Graham started:-)

And we are probably all lazy dogs :-) I like how we speculate, yet 
noone has ever asked Carl. Actually - I did, and it resulted in a 
fix of expired worlds. And it is always the same and the same. Carl 
once told me - tell me what the problem is, and I will look into 
it. So - anyone to ask Carl for open-sourcing AltME? :-)

so it would be hard to open source it now if it is continuing to 
provide a revenue stream

You may recall a recent tweet of his finding a lot of Amiga relics 
in his basement .. so he's going to sell them

I guess if he starts to refer to his basement as a cellar we will 
know :)

I think that someone from RM Assest who is now close to Carl would 
have to ask Carl, about the possiblity of open-sourcing AltME, but 
- AltME is used even by some commercial entities, so I don't know 
if it is likely to happen. As for me, I prefer Carl working on R3. 
And I can't see the situation being dramatic, nor do I agree with 
the launch of the new world. In such a case, I might consider not 
being present. I am used to be here, and apart from personal messages, 
there is not any significant slowdown or outages lately. We were 
in much worse situation in the past imo ...

How about ... we create a backup of this world somewhere, and prune 
all the rarely used groups from here?

I seem to have missed the details on how exactly AltME gets slow 
- it feels OK to me, these days. No real lag when submitting a message.

Maybe I should setup altme on another PC and see whether the slowdown 
is a local effect ... ie see how long it takes for a message to appear 
on the other instance

I think it's a combination of AltMe and the server it's running on.

I want to put a second world on my computer / world server and in 
the past I spent a fair bit of time resetting worlds due to them 
becoming unreachable.

You should give it a try. Whatever was done to this world, it was 
good :) Let's hope it stays that way for a while.

if Carl is just doing a TAR while the altme server is running, it's 
not strange that one or two files don't get backed up correctly. 
a quick workarourd is to keep more than just the last backup, as 
the chance of a file missing from two backups is pretty small.


of course, ideally one would be using ZFS and backing up from a snapshot...

It doesn't help that the client can't detect the issue and save the 
old data somehow. Once you sync a corrupted group, your local store 
is gone (unless you're backing them up locally ;-).

is it just me or does this mean that moderation and message alteration 
is actually possible... 

it would only require a few changes to the server and client so they 
support this feature.

Is there a problem with AltME at all? All this that has happened 
seems to not be directly related to AltME, but to the server and 
the backup scheme.

Henrik, the problem is that the server overwrites the client even 
when the server is wrong (corruption on the server side). It would 
be nice if it recognized the condition and recovered files from the 
clients.


The other problem is: can you think of a backup scheme to avoid this? 
(Note, we're speculating on the base of guesses, but still.)

Backup scheme: I suppose the only way is really some kind of snapshot, 
or by having AltME server pack a single file once an hour, that a 
backup system then can grab and stow somewhere.

Okay, so.... I just had to reinstall AltME on linux, so here's some 
things I noticed:

*It took me a little over 4 minutes to resync a little over 60 MB 
of data (checked net connection for amount transferred)

*SQLite and Advocacy both got stuck at the end, re-sycing over and 
over until I restarted AltME

*I clicked through everything because I didn't know there was a "Mark 
all messages as read" button, until I re-selected my custom dividers...

Thank you for noticing me of a button to mark all messages read :-)

I just had a VERY strange problem in altme... its the second time 
this week and its never happened before...


I stopped receiving notifications from altme, but it was still linked 
in, and not reporting Internet busy....

I just did a logout/login and saw that 7 groups had new messages 
in them  !!!

yes, it happens here too. I think that there might be "internet busy" 
for a while, but then altme reconnects, but does not receive new 
messages (just speculation, not observation)

the strange thing is that pn both occasions I remember hearing a 
new message "ping" and there being nothing new... which is a bit 
strange...

Maxim -- I see that regularly....It's the remnants of the old resync 
bug.

Luckily, logging off and back in seems to resync correctly -- not 
like the old days when getting groups to sync properly was capricious.

And sometimes, you'll see the All Groups heading in red, but no group 
heading below it is in red. It's like a phantom group has just received 
a message from Jim Phelps.

Good advice, Petr!
Max, Petr -- I've just sent you each a test PM. Did you get them?

Strange ... I saw red from both of you.

Did my name float to the top of the list, or was it buried a long 
way down?

I believe subsequent posts will be red ... try to send a ping to 
me ....

it seems I still have my private messages, not sure if they'll disappear 
next time someone writes to me :) I have a backup anyway.

no but I should not risk it untill I have my own backup,  I use it 
as a  sort of  distributed note pad

Tomc, wise.


I use Qtask since I know the notes will be there, or Qtask's Scratchpad.

Every time a computer crashes, or my celll phone resets, I'm SOOOOOOOOOOO 
happy.

I'm actually getting a new bug.  I see teh "All groups" RED, but 
I have read everything.  Started a couple of days ago.s

I did a quick test here, which failed, but is hardly full proof.

I've lost all my PM in this AltMe world too...I'm glad I did a full 
backup of this world files when someone first mentioned the issue. 
Can't AltMe admins restore them from backups?

I think we need to create a dedicated group for Custom Gob Renderers... 
there will be a lot of discussion and support or HW bug tracking 
about it too.

something like REBOL3! low-level gfx  could be ok,

note this isn't about discussing how to use view or a gui.


this really is for talking about rendering, graphic pipelines, agg 
C code, gfx engines, CGRs, stuff like that.

I thought we had a Graphics group at one point, but maybe not.

Gab has a complete backup of the world, and thought it might be possible 
to merge old and new data, but only RT can say for sure.

AltME uses a write/append which for some reason on Linux means: if 
you cannot append, truncate the entire file to zero.

I am a Windows user, I lost PMs too ...

Make a backup of your old persona's PM file then, before logging 
in.

Well.. we have a policy on our servers of *never* accessing any world's 
internal files for any reason. So, I had no idea something happened.

I don't know how many groups we lost, but the world got a lot faster 
when it happened.

you must chat a lot ;)

I can restore from tar any chats... but not so sure about merging... 
would need to think about that one a bit.

I suppose I could write a script to do a merge.

+1 from me (even though I have a full backup).

In AltME, your local sync'd copy is supposed to be your backup.  
I did not know it would get deleted on a server problem. "Was not 
supposed to work that way."

AltME uses a write/append which for some reason on Linux means: if 
you cannot append, truncate the entire file to zero.


For performance reasons (blame the early optimization guys), the 
default setting for many file systems on Linux is to write metadata 
independently from data, which means that it often gets written first. 
The result is that if writing the data fails (eg. power failure, 
kernel panic, ...), the files end up 0 length.


This can be disabled by setting the filesystem to always write data 
and metadata at the same time (at the price of write performance).

Could one of our world masters please create a "!REBOL3 Proposals" 
group and put it under the R3 divider? Thanks.

Cancel that gripe.....I think it just took a while to resync and 
update the list.

Renaming groups should have no effect on either REBOL.org's or REBOL.net's 
links (both use the persistent group number rather than its name).


Taking a group from [web-public] to not web-public.....REBOL.net 
deletes the posts (that does break links). REBOL.org simply stops 
publishing newer posts. So all posts orginally [web-public] stay 
that way.

it would be funny if it was a "." post.

Who knows?
Maybe., eventually, a name check on REBOLweek.

Bummer, I thought something like a free breast enlargement in Romania 
:-)

I manually removed altme from my Ubuntu netbook today as 1.1.29 would 
neither connect nor uninstall  -  is it worth trying again with a 
more recent release, or should I just use this ole windoze box (with 
1.2.25) and spare myself  the aggravation?

Likely because there's too much difference with 1.2.x, and 1.2.x 
for Linux was always officially just a beta, which they didn't automatically 
update

I was using 1.1.29 up until a few weeks ago when it stopped connecting, 
and have been using 1.2.25 since

AltME broken for me....


Background: this morning my Windows computer froze and I had to cycle 
the power on it. (AltME was running, but nothing to suggest that 
AltME was the cause of the problem).


After a reboot, AltME would not start -- complained there was a file 
it could not read.


No worries....I deleted the entire REBOL3 folder, and restarted AltME.


After it had downloaded 40+ meg of messages, all should have been 
back to normal. But the resync process got itself stuck, continually 
resyncing the SQLite group.


I have since tried downloading the whole world again; and I have 
uninstalled AltME completely and started again from scratch. Same 
problem.


Now, if I restart AltME, it looks okay at first. But, if I click 
into the SQLite group (or if someone posts a message to that group) 
the endless resync starts up again.


I am not seeing the problem on a different machine where I have not 
reinstalled AltME.

This is like the opposite of the ancient resync bug :)

***


I suspect the SQLite folder is somehow corrupt on the server; but 
the corruption only triggers trouble on a fresh install.....If so, 
we will all see it eventually :(

Can someone copy this to whoever maintains AltME? Thanks!

Peter Wood has sent me his SQLite file (439.set). Thanks Peter!

Copying that over the one in the /chat folder seems to have cured 
the problem.


So it looks like the server has a corrupt version that it sends out 
on a fresh install.

I've sent a Feedback to REBOL.com.

Well... at this point... given that the more time goes on the harder 
it will be to fix this world, why don't we just make a new one?

A new world has several attractions.

But if the problems lie in the codebase and/or the backup strategy, 
then they will recur..

I posted an Altme issue over at Accounts this AM - as I relented 
and put  a 1.2+ (the page said 0.42 ??? )  on my Ubuntu netbook - 
it now connects - but then rejects my account - which is working 
from this Windows box 1.2.25   Must be me.  3-letter word for dolt 
... let's see ... "usr"

If I could reset it HERE, that would not be an issue.  As I say I 
can both count to 3 and recite the alphabet ;-)  Whyever is it not 
in Settings by now?  There must be a good reason ...

Well, at 25 I could still remember library of congress call numbers 
for books I had read ...  now anything I don't use on a weekly basis 
is GONE unless it is from the period 1957 to 1965

But who could forget a 3-character password?

No, I'm not a world master (he said grinding his teeth)

In German, but that carries the notion of world champion. More benevolent 
than a master

'tre'  would be a good  three-char password

Haven't had a Japanese girlfriend yet, so I'm less fluent in Japanese...

That 150,000th [web-public] post took a little longer than I anticipated 
last week (it seems we all stopped posting web-public once the possibility 
was mentioned).


But it has happened now -- congratulations to BrianH and his comment 
on R3's break/return:
    http://www.rebol.org/aga-display-posts.r?post=r3wp824x176


No word yet on whether he'll be flying to Romania to treat himself 
to the prize Kaj wanted :)
    http://www.rebol.org/aga-display-posts.r?post=r3wp151x5092

I noticed something interesting: When you are reading a red group, 
the column in the left hand side is colored yellow for unread posts. 
When switching group, there is for me a waiting period of about 2-3 
seconds, for large groups before the new group appears.


However before the group switches, the coloration disappears, suggesting 
that a redraw of the list takes place before the switch. If this 
redraw was removed, switching columns might take much less time, 
perhaps half as long.

Bug:

The old inconsistent bug of failing to copy the text of a message 
by simple right-click prevented me from copying Oldes' message with 
the long referrer link and cookie data in it, so I selected the message 
text first before right-click. That worked, but then all messages 
in the display began disappearing (looks like a shared para scroll 
bug). Even switching groups. Right now I'm looking at !AltME with 
several blank messages - only the dates appear normally.

It's been a while since I updated the exe manually. I thought the 
auto-update was taking care of that.

If by fine you mean that you first have to go to the target paste 
area (for example a browser address bar), clear it, then to AltME 
and copy the text, and then back to the paste area and use the middle 
mouse button to paste. Any deviation from this procedure will break 
it in weird ways

I opened a new document in my usual text editor to paste the entire 
message text into it, so there was no other field involved.

you can split whatever.. but I know, it's not a solution, I don't 
know the correct answer.

I tried to create a zip archive, but it is about the same size, so 
no help. IIRC there was some AltME version, which allowed to use 
bigger sizes? I think it would be good to have it as a command-line 
option for the server startup. It is my private world, and I would 
allow 20MB easily. It is not first time I got file bigger than 10MB 
I needed to post ....

You are guru, it works. But I did it on the client side. Isn't it 
a bit of a security limit, if user can set-it himself?

as an ordinary user. But I am a world master ...

Kaj:  That's a symptom of linux, not of AltME.  See https://bbs.archlinux.org/viewtopic.php?id=79812

(AltME uses the X clipboard, the same one used when highlighting 
text)

It's a symptom of both. Other Linux apps also act horribly, but not 
that horrible

Petr, it's a big security hole, because the AltME server uses a multitude 
of the file size in memory during the upload, which can easily crash 
the server and at least overloads the swap space

Quite a while back I was confused about the clipboard in Kubuntu. 
I investigated heavily until I understood what was happening, that 
there were two clipboards and how they are used. I think I did a 
small bit of tweaking to Kubuntu settings, but basically it was just 
learning how it worked and practising a bit that cured my confusion. 
I got used to it and don't have much trouble with it in other apps. 
AltME is the exception - it just doesn't copy and paste consistently 
for some reason.

Windows doesn't work consistently, period, but that's just a force 
of nature ;-)

Welcome to the freedom of being reborn. Forget your past and start 
fresh. I'm just a few weeks old now myself.

It would also be a good place to discuss using REBOL to generate 
apps in Java (often easier than writing it directly in my experience).

for example, the host-kit and extensions group are the best reference 
for someone wanting to get up to speed.  it has discussions and a 
few web links on further on-line resources which some of us have 
posted.