AltME groups: search

...but INCLUDE (in essence) is just a combination of PREBOL and DO, 
so nothing really new

I think he will accept ideas as long as they are well tested and 
the code is up to his standards.

but the philosphy must also align.  and that is usually where he 
is harder to convince.

I need to evaluate our phone provider CSV data for one year. And 
they mix cell phone and normal phone on the invoice, so I need to 
go down and download details. And as it is pread across per month 
files, I want to do simple counter in REBOL. With read/lines ... 
and subsequent foreach line data [ ... parse line "," .....] REBOL 
is really cool to do such data manipulations ...

If you make an object with images, blocks and a lot of data and then 
set the object to none, and the data inside is not referenced elsewhere, 
will everything else inside be properly deallocated?

Peter, here's some R3 fun. To do the initial block allocation in 
R3:
    a: make object! 5
    foreach w [b c d e f] [extend a w make block! 1000000]
or maybe this way:
    a: object [b: c: d: e: f:]
    foreach w a [set w make block! 1000000]
To clear your way:
    foreach w a [set w none]
Of course Steeve's method works too, and better :)

henrik: the GC doesn't deallocate on demand... you can have several 
hundred megs of RAM allocated... and sundendly the RAM goes down. 
 and some things do not cause the rebol executables to shrink.


images, in my experience cause memory leaks, where they never get 
de-allocated from RAM under some circumstances.


in theory if you remove all references to a context, its content 
is supposed to deallocate, but when passing anything through view, 
all bets are off.

I've had 400MB pictures stay stuck in RAM, with absolutely no GC 
happening, not even going through view... just loading the image, 
without assigning it... its stuck  :-(

the easiest and FASTEST object clear:  

foreach itm next obj [set in obj itm none]

that's quite a problem, but is it possible to allocate space and 
load new image data into that location?

Diffferent, but related......memory may be deallocated within REBOL's 
sphere of influence one the items have no more references; but it 
is not (it seems) dellocated back to the opsys until the REBOL thread 
closes.


That can create several problems ..... eg an application that uses 
a huge peak memory load at startup, and would like to hand it back 
for the next 8 hours of running. It's all in the arena of efficient 
memory management.

There is a difference between loading image using 'load function 
and loading using 'load-image function (which is used by VID). The 
second one stares data in cache.

And I see no problem with the first one:
>> recycle stats
== 4384079
>> x: load %/f/IMG_0783.jpg
== make image! [1024x768 #{
4E707A59767C5D7172565F5A4D4E464E4A3F554D4258504558494254453E
4F40394E3D364A373148332E4A35304F3A35483E34...
>> recycle stats
== 7531902
>> x: none
== none
>> recycle stats
== 4384079
>> load %/f/IMG_0783.jpg
== make image! [1024x768 #{
4E707A59767C5D7172565F5A4D4E464E4A3F554D4258504558494254453E
4F40394E3D364A373148332E4A35304F3A35483E34...
>> recycle stats
== 7531950
>> recycle stats
== 4384127

oldes, about the load ... its not totally obvious why sometimes the 
images stay stuck in ram... but in this case it was huge png images. 
 and I did every recycle trick I new of... I tried solving this for 
2 hours... but nothing worked... not even allocating other stuff, 
and freeing it, to make sure the GC really did do a cleanup (in case 
is was count based).

but i need to build the software first, and right now.. that depends 
(again) on R3... I will be using OpenGL to build the output image 
setups, then dumping the info to a monstrous image magic process, 
which in the end might actually process more raw image data than 
my HD actually can contain.  I expect render times in number of days. 
 (4 pictures on a much smaller canvas took more than an hour).

the processing will be spread out on a farm and reintegrated, as 
tiles render-out.

so actual details will vary on time and budget when the final step 
of the project is green-lit... not this year for sure.

like any serious art project... it takes time to build up the hype 
around it too, and possibly get subsidies to pay for part of it.

I need to pay the RAID tower, the hours I'll pour on programming, 
photography, img tagging and output image synthesis qualities, etc. 
 publicity and find a few places to show off the end-results... which 
sometimes has to be paid in advance...  


life is so simple when you have no interests... sometimes I'd like 
to be that way for just a few days a year... hehehe.

Sunanda: if an app uses a huge peak at startup and then does not 
use that memory anymore, that memory just gets swapped to disk. So, 
it's not really in the way of other applications.

Gabriele: that may be true in an ideal world, but not in a Windows 
one.


I tried the test code below in R2 on a freshly booted machine.....I'd 
start a REBOL session, paste the code in,ánd wait until it gave me 
the timing. Then start another session (leaving the old one active) 
and repeat.


I only needed five console sessions to exhaust all physical memory, 
and have Windows behaving in an unstable manner. Killing the console 
sessions returned things to almost normal.


Basically, if this code modeled an application's start-up memory 
usage, it would be unwise to run it under Windows:

rebol []
    t: now/time/precise
    b: copy ["a"]
    attempt [
        forever [
          append b b
          append b join last b last b
          ]
     ]
     print length? b
     foreach a b [clear a]
     clear b
     recycle
     print now/time/precise - t
     halt

and using swap still takes up ram, it prevents other apps from using 
swap, so its a bad idea on any platform to occupy several hundred 
megs of unneeded ram.

on windows, the detail is that even the OS uses swapable RAM by default. 
 there is registry setting to prevent this, and it makes a hell of 
a big difference in windows itself.  when the machine is under load, 
the OS stays responsive, as opposed to becoming a hog.

In Windows you want virtual memory to be turned on, but you can set 
it to not use a swap file if you like, without touching the registry. 
Virtual memory is used to support memory-mapped files and speeds 
program loading and use (which is done with memory-mapping). The 
computer I am on right now has virtual memory and no swap file since 
it has an SSD, and it runs fine.

You better be sure you have enough RAM though, and for some purposes 
you might need 64bit Windows to even support enough RAM.

In my experience, even with 2GB of ram on a machine with xp and one 
or two software, you will get OS out of memory errors if you turn 
off swap files.  this, with 1GB physical RAM still available... I've 
tried it a few times before and after a short period I had to reboot 
my machine.   The kernel is built in such a way that it expects it 
and it just can't really cope without it.

increase it and it will disapear... and don't let the OS auto-increase 
the size.... that leads to other out-of memory conditions when your 
disks are near full.

However, I tend to not get out-of-memory errors on this computer, 
which has 1GB of RAM and no swap file. This is because I am careful 
about which programs I use (no IE, no Firefox).

has anyone been successful in compressing data in php and decompressing 
it again in R2? I see I asked such a question 7 years ago on the 
mailing list, but no solution was found back then. :-)

I was nagging you about the actors+message passing rebol lib I was 
playing with..  Now I have a real thing running with it .. I will 
post source code on blog soon. It's not that exciting video .. just 
some text changing :) but I hope it will help in getting a picture 
when I post the code.


On left is a Linux server with work dispatcher on  right are two 
workers on windows. There can be as many workers on many computers 
or none and workers can drop out at any time without work being undone/"halfdone". 


It uses message passing and actors for everything.. and comunication 
works like at tuple spaces so you get autobalancing. It's not something 
that special, you could do something roughly the same with http server 
app probably, although it would be a little more messy.

I needed this setup because I need to make screenshots of websites 
in browsers for websites in site assistant and you can't make a screenshot 
of IE browser on linux. I don't intend to have any windows servers 
running but I will have workers on mine and computers of colegues 
running on background and doing it's job silently. Thats why it's 
all made so that there is nothing wrong even if no workers are avalable 
for bigger amounts of time

and they can get on and off work as they please :) .. I call this 
"casual workers system" :)

I haven't seen it but heard of doc's task manager, and he uses multiple 
worker processes in cheyenne. I am certain those two (or one if it's 
the same) are much faster and robust. I am mostly experimenting here 
on how a "message passing with actors" system would or could look 
in REBOL , so I am not focused on implementation, but for now more 
on look of code.

I will post the code shortly so you can try it, but I am about to 
do big changes to it.. if actors will remain rebol objects then I 
will make whole lib more OO, refine the methods etc.. I have another 
option where actors would become just functions (this would make 
them more lightweight (which is important at actor systems) and it 
would enable some cool things, but I am not 100% if it will all work.

RPCs and message passing are quite different things and whole code 
logic works differently .. async message passing is " send and pray 
" (quote by Erlang inventor) model while RPC are "request response" 
.. I have been folowing erlang and message passing for so long but 
I wanted to see how it really is in practice thats why I started 
making this actor-net rebol library

two processes ping ponging to eachother 10 times with actor-net would 
look something like this now

pinger: make actor [ act-match: [ 

    [ 'pong count addr ] [ ~send addr compose [ ping (count) (get-my-addr) 
    ] ] 
    [ ''bye ] [ print "ok, bye" ]
]
ponger: make actor [ act-match: [ [ 'ping count addr ] [ 

        ~send addr compose ( either lesser? count 10 [ [ pong (count) (get-my-addr) 
        ] ] [ [ bye ] ] )
    ] 
]


and you would add the third actor that would act only once to "lend 
them the ball" to start them


server: make once-actor [ act-once: [ ~send get-actor 1 compose [ 
ping 0 (get-actor 2) ] ] ]


But the library is still quite messy .. especially with adresses 
of actors, global vars, I have a big list of things now to improve 
after I made that thing on the screencast

I need to view various files in a browser and if I use 'browse, then 
this blocks the UI until the browser appears with the file.  I just 
tried with a xmlrpc server accepting the url and it works fine.

the xmlrpc-exe returns immediately ( with an error though - no matter 
) and the page pops up.

(the previous version sources , examples and 2 more blog posts about 
actor-net can be found here http://itmmetelko.com/blog/2009/06/08/playing-with-making-an-actor-like-distributed-system-in-rebol-3/
)

what I mean is that I use xmlrpc to start up the browser and bring 
up the page I want

I have to admit I don't get what borwser (you mean like IE FF?) has 
to do with xmlrpc in this case ... and I.. aha this is a GUI app 
you you don't mean that xmlrpc-exec returns imediately (without waiting 
for result -- thats why I ask how you get the resut then) .. but 
that GUI doesn't block while waiting for it

Yes, that's correct.  My gui doesn't block while I wait for FF to 
start up and load the page I want.

This is pretty cool ... I am sitting in front of my laptop, and on 
the other side of the room is my 46" LCD screen with my media PC 
attached.  I installed the rebxr xmlrpc server on it .., and I have 
cheyenne installed on the laptop.  I click on a file in my application, 
it gets downloaded to the www directory in cheyenne, and then I send 
a command to the xmlrpc server to browse to that file so that it 
displays on the big screen but is served up by the laptop's cheyenne 
server.

Question : 
I have a script like this :

ctx: context [
	markup: "my script with <%val1%> and <%val2%>"
	val1: "Hello"
	val2: "Bye"
	build: does [build-markup markup]
]

It produce this result :
>> ctx/build

== {my script with ***ERROR no-value in: val1 and ***ERROR no-value 
in: val2}


Its because 'build-markup use 'parse and 'parse looks its word in 
the global context.

Is there any binding tricks to solve this ?

Smart Guys... What's the trick to using a variable to hold an email 
address when using "Send"
Send/header  some-email-address Some-text myheader
I always get a failed email because  the "To:"  word is not set.
If you directly type in an email address such as
Send/header  [confused-:-rebol-:-com] Some-text myheader

It works. I've tried composing, reducing, scrubbing and washing and 
the stains still don't come out, if you know what I mean.

There must be some trick. (And I know you know)

just had a look at the SEND source and from what I saw it is highly 
unlikely that it does not work Moreover, I am actually using the 
SEND function from time to time and no such problem occurrred.

Henrik and Ladislav, 

OK, stupid me. I was using a word name that I shouldn't have. Thanks!

Didier, assuming you can't preface the markup fields with the context 
name, e.g. 

  markup: "my script with <%ctx/val1%> and <%ctx/val2%>"


either manually or programmatically, hacking build-markup may be 
best. e.g., add a /with refinement that takes the object and bind 
to that.

DideC.
even this does not really use markup, it works at least
ctx: context [
	markup: ["my script with" val1 "and" val2]
	val1: "Hello"
	val2: "Bye"
	build: does [build-markup form reduce markup]
]

hm .. yes .. I basically don't explain actors and message passing 
.. so one should not know erlang exactly but this general concept 
.. it's very simple basically .. but I looked at wikipedia and it 
wasn't nicely "graphically" explained .. maybe I should write one 
post about what actors+message passing are

But with next rewrite I think actor-net will already become nicer 
and generally consistent .. I now have some sense what works and 
what not ..

Apple grand central seems very interesting to me too .. I saw it 
when someone posted link here .. I was thinking that I would try 
to do taht model in rebol too .. I am interested in concurrency and 
this

(I don't know a lot about it, only it seemed minimal and clear in 
concept which I liked)

the current changes I'm adding to liquid is the ability to use bidirectional 
messaging... you can now supply arguments to the process, and it 
will use the params within as an index key to match recurring processing

BrianH .. I am not sure yet .. but maybe I will be able to change 
this system so that actors will be rebol functions instead of objects 
.. that would mean more lighweigh and by that more performant probably 
( you could run more of them, create and destroy more of them faster 
)

and if it will be possible at all, you would gain something that 
now isn't possible .. making state machines very elegantly with them 
like you can with erlang's actors

Your articles have made my to-think-about list, so we'll see how 
mch cleaner and more efficient we can make your model :)

very cool ! .. because none of you gurus ever responded to my 3 posts 
about actor-net I was begining to think I am pushing all this into 
totally stupid direction and I don't even see it (I am not CS educated) 
, now that I know I can chase this with a little more certanty .. 
so this is definately a positive push for me

This has been a really busy week for R3 semantics and changes - don't 
take it personally.

Meijeru, Fork and Ladislav have really been stirring things up. We've 
had four alpha builds this week.

I was focusing more on how to make actors look natural and elegant 
with rebol so I wasn't focusing at the implementation at all so far.. 
only so it worked so I could test further .. that's why I called 
all this "playing with actor library"

and we know how fast it is

And the services model too :)

yes, when I started it I was thinking that it could serve as some 
food for thought on R3 concur model .. because Pekr and others mentioned 
erlang a lot, so I wanted to see how erlang would look in rebol at 
all

I was interested in erlang before it got popular .. but it's code 
always looked very ugly and complex .. then I got one e-book about 
it .. and in book where I knew what what I was looking it seemed 
very very clean in concepts

and it has slow IO .. slower than even other dynamic langs .. another 
minus of it is it's limitation at strings .. so people use binaries 
for strings so code looks like this <<"somestring">> + <<"sadasd">> 
.. quite horrible .. if it werent for these strings I would use it 
for something a while back but strings are the base of webapps and 
writing <<"">> all the time seems horrible

yes :) .. I still think it's code concepts are very nice and clean 
but it's hard to see them when you look at code

and it's concurrency model seems to be one of rare that really work 
well and people can think in it .. that is the geniois of erlang 
probably

I've been looking at ropes lately - it's a kind of functional string 
structure built on trees of immutable substrings. Someone did an 
article about it recently. I think we can implement this in R3 as 
a user-defined string type. Ropes could be useful for increasing 
performance and lowering memory overhead for R3 multitasking and 
large string processing.

hm.. this would be very good to have .. and it sounds very much like 
data structures that Clojure has ... I think it calls them persistend 
data structures , like list, map, etc .. and the same as you described 
here. You have many versions of a data structure but they share the 
unchanged parts.. this highly benefits functional programming style 
.. you get "new" data structure each time, but without the penalty 
of copying it each time , and is cruicial in concurrency which is 
in focus in clojure, because then different threads dont's share 
and "corrupt" one to another the data structure , but each can have 
it's own revision , etc .. 


I am not that good in this complex stuff, but it might be very intersting 
to you becuase you will know much better what's he talking about 
and how can that be implemented .. they made some of his data structures 
in Factor , so they ren't impossible to make in other languages.. 
I don't remember exacty where I read about this

http://www.blackpepper.co.uk/black-pepper-blog/QCon-Persistent-Data-Structures-and-Managed-References.html

This is where persistent data structures come in. Based on the work 
of Phil Bagwell, Rich Hickey described how he used bit partitioned 
hash tries to make efficient "copies" of data structures, and this 
forms the basis of data storage within Clojure. Essentially all data 
is stored in a tree and when one makes a copy with a small change, 
one can create a tree with a new root and only the path to the changed 
item needs to be copied and modified. The rest of the tree's branches 
remain precisely the same. This significantly reduces the amount 
of copying that is required and makes multiple "versions" of a data 
structure entirely practical.

and if you want to do for example efficient message passing concur. 
it's also cruicial to have this sort of data.. because message is 
always a copy .. if you are using message passing for distr. compuring 
- only between computers then you have to copy anyway so it's no 
penalty .. but if you use message passing for concurrency /paralel 
execution on one computer then copying data for messages each time 
will have a high penalty , but it wouldn't with ropes for example

To use it, double click on the blank canvas to create nodes, and 
double click on nodes to edit them.


Drag arrows from one node to another by click and drag from the left 
bottom corner.

Control-L to load a new map

Control-S to save a new map

Graham.. very very cool .. it seems that you have pockets full of 
magic rebol tools :) .. This graph editor is really nice and works 
smooth

I have updated the drawing again since I missed some connections 
. I was thinking .. once the "looks" of actor net is solid we should 
really move network stuff to uniserve to get the stable and fast 
network base.. it's one 1 function now so it shouldn't be too hard. 
Then I could actually recommend others to use it.

call/output does not work, when console is not being run? I tried 
to convert some character sets, prepared short script, and run it 
by double-clicking .r file. No result was shown and rebol process 
is hang in memory. It was enough to put one print statement so that 
console showed up, and it was OK then. But that is weird, I wanted 
it to run without REBOL console showing up ...

and wanting to do it async while write/binary/append in the async 
handler

It interfered with the garbage collector and was one of the reasons 
to start R3 development iirc (the whole port subsystem, actually).

Maarten ... when did that become common knowledge??  I had endless 
issues trying to upload files async, and it only worked by turning 
off GC  .... but of course killed my program with huge memory use 
:(

cut and paste works poorly in windows :(

and Mac ...

I think you should have 'some and not 'any as there should always 
be at least one space to be inserted.

you should solve it by parse and parse only - that is the challenge 
:-)

there's no edge condition. What if the name would not begin with 
a capital letter? I would parse all string and instead of 'next I 
would use 'trim, which would tream initial space, in case first letter 
is capital :-) But if it is the rule, that the first letter is always 
being a capital, then your solution is absolutly correct ...

I guess I could always make it a View app instead, and wait on  a 
port ... and use a timer to trigger the other stuff I need to do.

I once used lock file - you keep file openeed, and you try to delete 
that lock from the other app. If you can delete it, so create lock 
file, if not, another instance is already running. But - that works 
only under Windows, I've heard that under Linux, you can delete file, 
even if another app is using it. Dunno if true ...

You can also use TCP ports, but if you open some port, firewall will 
step-in and ask for approval ...

so, open a file and keep it open?

under Windows, if you open file in one process, another one can't 
delete it. Easy enough. Prevents the case, where process does not 
remove the lock, but crashes. Then you would be in locked situation. 
But if app crashes, you can remove the log. Under linux, dunno how 
to do it. One chance is e.g. the need for process to update timestamp, 
and if timestamp is not updated, then app most probably crashed, 
so you can start another instance ...

One way:

On startup:
-- check for your timestamp file

-- If it does not exist or  (it exists and timestamp is over 2 minutes 
in past), proceed to run

-- otherwise, wait 65 seconds.  Test if timestamp has changed: yes-halt; 
no-proceed

While running:

-- write the  timestamp file at least once a minute with an updated 
time

On clean closedown:
-- delete the timestamp file.

Drawbacks:

-- application could take over a minute to restart if immediately 
restarted after a crash.

-- manual deletion of timestamp file can lead to multiple instances 
running (you can minimise this by re-reading file and aborting if 
timestamp is not the last one you set)
-- all those writes of the file.

Sunanda,  does your timestamp file mean a file with the timestamp 
as content or just the date and time of the file?

I have many times seen, that the timestamp of a file under windows 
does not change, although there is always data added to the file.

AFAIK, due to multi-platform issues and various platforms not supporting 
it the ALT-key has never been utilized for the view event engine, 
hence it's probably not possible to detect the ALT-key. There's no 
/ALT refinement for the EVENT! datatype like there is /SHIFT and 
/CONTROL, so most likely there's also no control sequence for the 
ALT-key for /CORE to work with, too.

In R2 View, Alt is not supported. If you look at the event datatype, 
there are fields only for control and shift.

And why is that such a problem?

I mean, it's a solvable problem. IN could theoretically be enhanced 
to also accept paths, but, iIrc, Carl wasn't keen on the idea because 
he wanted IN to remain simple and fast. He said something like "it's 
for words".

GET and SET work with paths in R3. Perhaps as you describe could 
be added as well, to handle the IN function for paths. And functions 
wouldn't be traced through.

So you have to check for 'to in object/path first, and so on. I didn't 
say it would be a fast mezzanine :(

The way this kind of thing is resolved in R3 is through liberal use 
of the ASSERT/type function, and not representing XML as objects.

I guess there's a lot of advantages to just using blocks ... as it 
is now, some elements in my object are blocks and some are objects 
which makes it very messy

The R3 GUI is structured as a mix of objects, maps, blocks and gobs, 
but it is very consistent and not messy.