AltME groups: search

It may be fairer and more accurate to say REBOL has some subtle optimizations 
in its memory allocation.

Watch, for example, the memory drop here.....you'll see that as the 
block 'x grows, REBOL allocates space in anticipation, so a single 
insert into 'x does not always change the stats.
x: copy []
st: system/stats
loop 500 [
    print [(st - system/stats) length? x]
    append x length? x
    recycle
   ]

There is memory allocation optimisation both for value slots (as 
seen in the growing block of integers example above), and elsewhere.

So a single allocation is not enough to deriive the underlying algorithm.

And a binary is just a type of string, so yes, the value slot contains 
a pointer to the actual data.

Storing a bunch of integers in a binary should be more efficient, 
only one value slot used, and each integer takes only 32bits.

no negatives..  and a MAX of 32 bits is more than enough for the 
largest number

We must do this in our functions which should allow retrieving, changing 
and inserting the value.

Yes, there is lost performance. Obviously, you must go via your access 
functions each time you store and retrieve.

a value in the dictionary could be a page of rebol code (as a string) 
 .. and it could be represented with a single bit  (or in this case.. 
3 bytes)

the values never change.. they may be deleted... and their index 
re-valued.. or the dictionary may be appended..

i would rather have it not fixed, and grow as needed

2 bytes would be plenty to start.. but would quickly grow to 3.. 
and very slowly (if ever.. but possible) grow to 4 bytes..

dict: ["Rebol"  "Carl"  "isa"]    are common..  should  be  represented 
as [1  2  3 ]  .. and dict:["This is a rare string.. probably only 
used once"]   shoud be  [ ZZZ ]

Im going to store the entire dict in memory as a hash!, and my code 
as a separate block of 3 byte "symbols"

Compaction makes everything more complex. You are writing more and 
more code to implement that.

Hashes, lists and blocks can contain any type of value. The values 
are not affected by the container type.

If you just need to save large arrays of integers, you can use format 
used in AS3:
 The AS3 Integer can be encoded into between 1 and 5 bytes.

    *

      if the integer is between 0×00 and 0x7F then only one byte (representing 
      the integer)
    *
      if between 0×80 and 0x3FFF then 2 bytes :
          o
            (i & 0x7F) | 0×80
          o
            (i » 7)
    *
      if between 0×4000 and 0x1FFFFF then 3 bytes :
          o
            (i & 0x7F) | 0×80
          o
            (i » 7) | 0×80
          o
            (i » 14)
    *
      if between 0×200000 and 0xFFFFFFF then 4 bytes :
          o
            (i & 0x7F) | 0×80
          o
            (i » 7) | 0×80
          o
            (i » 14) | 0×80
          o
            (i » 21)
    *
      if more or equal than 0×10000000 :
          o
            (i & 0x7F) | 0×80
          o
            (i » 7) | 0×80
          o
            (i » 14) | 0×80
          o
            (i » 21) | 0×80
          o
            (i » 28)

That's interesting, using a bit in each byte to "escape" and open 
up the next, more significant byte. So it's a variable-length encoding 
scheme. (I guess, kind of like UTF8)

But yes, I think this adds complexity and would slow down the access 
routines.

yeah... I think the conclusion is ... don't worry about the number 
of bytes (and thus mem) when using plain integers with my index block, 
as it's much smaller than the dictionary anyways.. and unless Im 
shown otherwise, the crawling of it (find/ foreach, append) should 
be about as fast as any other method, right?

It should be faster and simpler to just use integers. When you want 
to cut down the size of your 2 million integers (=30MB), you can 
then look at implementing 3-byte integers packed in a binary.

and that could be reduced further.. 

dict: [

{fetch: func [index][do pick dict index]}

{integer-to-3-byte-binary: func [integer [integer!]][
	struct/int: integer
	copy/part third struct 3
]}

]

do pick dict 1
fetch 2
my-bin: integer-to-3-byte-binary 2000000

yes... you can make it in R2 using index: [1 2 3]  and for R3 you 
just replace:  index: make vector! [integer! 24 [1 2 3]]

Terminology question;  I know I could probably RTFM, but sometimes

Ask A Friendly Human is more fun.  What is the correct terminology 
for the global

REBOL context.  I'm describing (or trying to at least) the parse 
dialect "copy" versus

the REBOL "copy".  Is there a one word term for the "no context" 
context?  Or is
the REBOL global namespace
 good enough (and not too confusing to new rebols)?

and with greater speed but less accuracy:
>> load form [1 2 [3 4] 5 6]

and now I should of course have read what Graham posted...

Yes, and I wouldn't recommend the LOAD FORM way, it looks too brittle.

what if the record size is 2, consists of a binary and an object 
and you want to sort on a value in the object?

and the block checks out fine. it is properly arranged.

found the bug. (and I need more coffee)

It's mezanine, so I'll try to figure out some fix and post it.

some things are not implemented yet and there some small problems 
but I hope I can solve them

The reason that regex compilers for REBOL are rare is that parse 
is more powerful than regex, and most people who start trying to 
implement regex usually learn enough about parse during the course 
of doing so that they switch to using parse instead :)

Still, if you want help, a tester or a second opinion, post your 
code on the Parse group and we will optimize it for you.

It's funny, there's no better optimizer than the members of this 
world trying to show off and one-up each other :)

I love this community. One of my favorite things is the ML threads 
that optimize code and bring out different perspectives on design.

in 98% of cases I agree with what Brian just said about Parse being 
more powerfull than Regexp.  but in those 2% regexp is SO much more 
powerfull, that it still has its place.   now some of you will ask 
me to say when or why, (I know how we as rebolers think and like 
to challenge each other ;-)  but I cannot give an exact example, 
but having had to implement a few complex systems in both, I remember 
a few times in parse when I'd remember how a single char would replace 
2-3 lines of parse "tricks".

if we could add a conditional within the dialect of parse directly 
(without using tricks, which I know many will be tempted to demonstrate, 
and many which I already know about) then parse itself would have 
another level of expressiveness IMHO.

for example, not only the type and shape of data, but its actual 
value?  have I loaded enough of this, for this rule to qualify.  
is a specific attribute set to a mandatory value?  there are many 
such examples.

again, I know most patterns CAN be described using parse, but in 
many occasions, what could have been a simple parens with a decision 
and 2 or 3 very simple rules, ended up being a complex tree of tens 
or more rules, which have non obvious interdenpendencies and things 
like left entry recursions (I hope I make sense here) which are, 
well, slow(er) and hard to map in one's mind.

well, I guess the best way would be to use parens within the parse 
block as a means to return if we should continue in this rule, (and 
maybe even based on type, how many items to skip!).

hum... you are asking my mind to shift from cgi and web site writing 
to parse rule generation.. hehe I'm a bit deep in the construction 
of Revault right now... with about 10 files opened and mapped in 
my mind ;-)

I'm working on reducing memory consumption on my little database 
and was wondering if stats is reading out the total memory usage 
correctly or if Windows XP's job list is. I can do a script that 
gradually eats up 100 MB memory and then the memory is recycled, 
when I ask for it. 'stats then prints about 15 MB used, which is 
fine, but the job list reads out about 100 MB still used and it stays 
there. Right now it reads about 104.656 KB used, while stats prints 
15588191 bytes. This is in a stopped console. Recycling more doesn't 
help.


I've even seen the job list memory jump up 10-20 MB once when recycling. 
Which one is reading out the correct number?

btiffin and sqlab, it turns out that you both suggested the right 
thing. I must have a lot of ports being used already on my computer. 
Since you both thought this is what was wrong. I just kept on trying 
port numbers until finally...it worked! Thank you both very much! 
 Thanks to all of you that helped me, this day has ended pretty good! 
 Having endured the earlier aggravation, the good feels even better 
than usual.


So many ports being open does make me wonder why, however. That seems 
a little dangerous to me.

Louis, here's a windows tool that will let you see what ports are 
open and what application is listening on the ports.

http://www.nirsoft.net/utils/cports.html

Ammon and Oldes, thanks. I'll check them out.

87 ports open---most of them by REBOL.  The script was working, but 
I didn't realize it because no window was opening.  This was the 
result of working too many hours without enough sleep. It just doesn't 
pay.  We get more done in the long run if we stop and play or sleep 
when we should. Thanks again, you guys, for all the help; very much 
appreciated!

Start the server first, then the client, and file transfer should 
be automatic.

Play around with a very simple test, where you do it manually between 
2 REBOL sessions. This way you get a feel for it and can made your 
program to work.

I just tried these examples, and they works. To send more files, 
you should loop the server from the wait to the write, both included.

Geomol, thanks!  I'll experiment with your examples, and report back. 
But it will be awhile, as I'm out of time right now.

as current path-thru function is not working with queries and port 
numbers, and it's not part of core at all, what do you say about 
using this imporved version (which I'm already using for couple of 
years):

path-thru: func [
    "Return a path relative to the disk cache."
    url /local purl
][
    if file? url [return url]
    if not all [purl: decode-url url purl/host] [return none]

    if all [string? purl/target find purl/target #"?"] [replace purl/target 
    "?" "_query_"]
    rejoin [
        view-root/public slash
        purl/host

        either none? purl/port-id [""] [join "_atport_" purl/port-id] slash
        any [purl/path ""] any [purl/target ""]
    ]
]

so:

>> path-thru http://us.maps2.yimg.com/us.png.maps.yimg.com/png?v=3.52&t=m&x=3&y=0&z=16

== %/D/view/public/us.maps2.yimg.com/us.png.maps.yimg.com/png_query_v=3.52&t=m&x=3&y=0&z=16

And it must be so easy to fix it... it looks that it would be enough 
to increase the Y-size ot the face which is used to cound the size 
of the text link (so it do not breaks)

How many rebols have written language localization routines?

I'm toggling back and forth between external heaps and in-code strings
I've got a RebGUI widget...


lang-text {en "This is the english" fr "C'est francais" it "Don't 
know any italian"}

meaning a translator will have to get dirty in code edits (or send 
to coder) or use

text (lang "SomekindaKey")

where lang is some func that having read some file, selects the string 
key by lang type...
lang-type being buried somewhere in locale*


How often is a REBOL translator a non-programmer?  I find external 
text to be a pain when
coding.  But...it lets non-coders help with translations.


In particular, I only have about 10 or so screens that could be translated. 
 Ashley's

builtin localization nicely handles all the GUI stuff.  I'm leaning 
toward in-code strings.

and the preprocessor can create a nice .catalog file for you

A key consideration is whether you want the translation(s) to be 
static (compile-time) or dynamic (run-time). Advantages and disadvantages 
to both approaches.

Yeah, I toggle back and forth, but they'll be static this time.  
:)

Is this simpler and the same effect?
    act: func [a b][add a b]
    for a 1 5 1 [for b 1 5 1 [act a b]]

hm, not exactly what I want because this is simplified, I have more 
variables than just 'a and 'b, but still useful.

mine works with both http and https

same version as used on the detective and published on my site

the function is rather complex and must be used many times, so I 
wrote it outside the block.

and the function helps me to find out whether certain conditions 
for a database entry is true or false

or a wrapper, an 'f-without-c and 'f-with-c.

and then returns a path with a refinement. 'do gets the path, done.

After dealing with strings for so many years with various languages, 
I would say that TRIM should be default with any reading/writing 
functions, and when you don't want something trimmed then use a function.

a: "  my untrimmed string   "
write no-trim a

as an example, I NEVER use parse without the /all refinement... its 
just so damaging... lengths become all screwed up and trying to separate 
clearly your tokens in the input strings becomes much harder.

some things cannot be "undone" and such behavior unless it is switcheable 
is dangerous... I've had many problems with memory useage since Carl 
switch the object model so that it copies all series at each new 
instance... the old way was simple to copy... but now, its almost 
impossible to share data amongst peer instances. 


I know why he did it... but I think more explicit documentation where 
the feature was causing some unexpected effects for newbies would 
have been a better solution. and we still have many of the string 
sharing side effects in View anyways... so it didn't explicitely 
fix the main issue in the first place!

what is the difference between copy and copy/deep?  I cannot seem 
to find any simple examples...

icarii, copy/deep copies series inside blocks, where copy only copies 
the blocks and series inside are referenced, like in the original 
block.

tested and working fine here now :)  i generally use copy/deep with 
complex series but never stopped to wonder why..

something like the following would be useful for new people:
;given the following series
a: [a b c [d e f]]
;perform copy
b: copy a
probe b
>> [a b c [d e f]]
;b contains [a b c [{and references a/4}]]
;this can be seen by appending to a/4 and checking b again
append a/4 'g
probe b
>> [a b c [d e f g]]

;now on copy/deep an exact copy with no referencing of internal series 
is made
c: copy/deep a
append a/4 'h
probe c
>> [a b c [d e f g]]
;c does not have h but b does.
probe b
>> [a b c [d e f g h]]
append a 'i
;and neither b nor c will contain 'i as it is in the outer series
probe c
>> [a b c [d e f g]]
probe b
>> [a b c [d e f g h]]

Should we prepare for a revision of the REBOL wikibook, when R3 is 
out? Either change directly or make a now book copying over, what 
is ok and adding new?

Thanks Gregg -- I was looking for the definite list of file modes: 
world-write etc.
A bit of extra Googling got me to here:
http://www.rebol.com/docs/core25.html#sect1.1.

It would have been easier with some SEO on the .com and .net sites.

guys, do you have recursive directory read function? Simply put - 
what I need for our kiosk is:

 - script running in the background, window-less
- script checks for new drive to appear periodically
- then it reads specific directory, e.g. %/e/kiosk-update

- then it reads files, and stores them to target dir, not carring 
about adding new dir/file, simply overwriting it

It is kind of one-sided sync :-)

I can't find anything usefull on rebol.org ....

Petr, I'm not sure what you want. Obviously %c/ is a dir, and obviously 
it is at the root, so you have to access it as %/c/. This is called 
platform independent file paths. It's the same for all platforms.

Brock, I was confused about reading %/ and getting %c/ instead of 
%/c/, that is all. I did not regarded %/ a root, I thought it is 
just a helper, as %. is .... that dot surely is not real part of 
filesystem, is it?

I resolved it for myself by defining root: %/ and joining it with 
the result of the read, works .... new problem for me is how to set 
attributes of copied directory. Not sure why do I receive error. 
It seems like that dir would be locked or so, but not sure why set-modes 
fail. Is set-modes supposed to work with directories?

ok, thanks a lot ... it is a pity, I don't need it in fact, but found 
that possibility in docs, tried it, and it nicely works for files 
....

and is there a way to create directory not using make-dir, directly 
setting such attributes during creation process, not later?

And I somehow wanted to have 'disk available in my copy-dir :-) Well, 
I can easily solve it via drive: disk and use drive global word in 
my copy-dir func. It is just that I was thinking if I can somehow 
magically bind to it :-)

That will bind report and item to the local foreach item.

Well actually it binds print and item  to the local item.

ah .... I tried to look at it from the perspective of report function, 
and was stuck ....

I have weak USB port, and my flash disconnects from time to time. 
I am testing my copy-dir small script, and during disconnection it 
failed. I had following code there:

attempt [
  data: read/binary source
  write/binary target data
]


Shouldn't attempt catch the error, even if I am in the middle of 
copying of file?

I've been a little curious about attempt...
if not error? set/any 'value try :value [get/any 'value]

Are there any conditions where the [get/any 'value] could fail and 
as it is outside the try cause an interpreter error trap?  I don't 
know.

inside foreach and accessible somewhere else == global. the proper 
way to do it is a function argument. you can bind, but that does 
not seem clean to me (argument is clean).

and to me it looks like you have the argument already.

I'm wanting to send a bunch of huge files to my son. I used this 
command awhile back to convert the files to text, then used compress 
to greatly shrink their size. Unfortunately I accidentally erased 
the source file for my script, and now can't remember the name of 
the command.

Graham, sorry for the delay in communucating. The electricity went 
off here, and with it the Internet.

in headers (and in the construct function) both word lookup and function 
evaluation are disabled, but set-word (assignment) is not.

I see what you mean. The actuality seems to contradict those documents.

>> obj: construct [ n: none d: delete ]
>> probe obj
make object! [
    n: none
    d: 'delete
]


It seems that only potentially dangerous evaluation is prevented, 
and not all evaluation.


Scant Evaluation: A minimal form of evaluation used for headers and 
other data blocks that do not allow any level of deep evaluation.
  Perhaps the evaluation of none is not considered "deep."

Pekr, this is what I was looking for (I think): http://www.rebol.net/cookbook/recipes/0048.html


This seems to be related to what you were trying to tell me and I 
just didn't realize it.  Anyway thanks for your help!

And i don't think that rebol supports octet stream as content type.

Jerry, I found this:


The CONSTRUCT function will perform evaluation on the words TRUE, 
FALSE, NONE, ON, and OFF to produce their expected values. Literal 
words and paths will also be evaluated to produce their respective 
words and paths. For example:


    obj: construct [
        a: true
        b: none
        c: 'word
    ]


The obj/a value would be logical TRUE, obj/b would be NONE, and obj/c 
would be WORD.

file:///C:/SDK/docs/changes.html

Louis, For TRUE, FALSE, NONE, I can understand ( however, it's not 
consistent ). But ... even ON and OFF? Why not YES and NO. ...

Nothing should be evaluated in CONSTRUCT, except SET-WORD!, which 
is an inconsistency.


NONE, TRUE, FALSE, ON, and OFF are not SET-WORD!, but they are evaluated 
in CONSTRUCT, which is another inconsistency in an inconsistency.


YES and NO are not evaluated in CONSTRUCT, which is yet another inconsistency 
in another inconsistency in an inconsistency.

Inconsistency is no good.

Jerry, this is a "feature" of construct. the words NONE, TRUE and 
FALSE are converted to the respective values. it is done to allow 
construct to work correctly when /all is not used with mold.

Gabriele so why TRUE and ON are evaluated, while YES is not?

OK, I copied rebol to the same directory as the script, and now this 
works:

C:\.Alkitab\rebol.exe ftp-backup.r -s

give it a daily end time and you're away laughing :)