AltME groups: search

Adding an option to a function that changes its behavior makes it 
harder to learn than a seperate function. The only thing you have 
to remember with a new function is the name. You have to do refinement 
processing in your head too, remember :)

Of course it takes a lot of work to make a function simple to learn 
and use.

I guess it's a bit wordy.

You forgot the copy, but that is a good alternative.

dirize: func [file [file! url!] /off /local l][
	l: last file
	either l = #"/" [
		all [off remove back tail file]
	][
		any [off append file %/]
	]
	file
]
that should be a little bit faster

still a bit buggy but throwing it out there to play with.

I agree with Anton. IMHO, one should not build refinements that invert 
the behavior of a function. Refinements should extend a function's 
existing behavior, similar to what COPY vs. COPY/DEEP does.

maybe I understood Brian wrong.. I thought in current situatuion 
you need to call exists? somepath and dir? somepath to know that 
something exists and is a directory (which also means two query calls 
I suppose)

will R3 have a way to define custom infix words?

Janko, yes, the current situation is exactly that; to know that a 
directory exists, you need to call exists? and dir?, which causes 
two QUERY calls.

Re: /off - it's not that different from 'trace or 'new-line.  It 
switches a mode, albeit using a refinement instead of a value.

Dirized is a state that 'dirize alters.

In R2, there are only two 'un verbs: 'unset and 'unprotect.  'undirize 
seems contrived (yep, dirize is contrived too, but necessary for 
a state that has no other name).

Fair point about new-line. But does 'undirize seem more contrived 
than 'dirize/off ? The prior seems more like English to me, the second 
is more "implementationish". I understand the desire to prevent another 
word in the global namespace, but I don't think 'undirize is going 
to collide with anything a user is likely to want to use .. !

Could also be that 'dirize has a permanent second logic! arg. It's 
not so much about namespace as language space.

This is a particular case.  I can see the useful ness from a mezzanine 
standpoint of having a function that does both add the "/" and subtracts 
the "/".  Because in the case of looping we can easily homegrow our 
own need there that would be more efficient.  But I agree the name 
of dirize is not so elegant.

There's a big difference between an inverting refinment and a logic! 
parameter: default behavior. 


I'm all for a better name. Even better than that, a convention. Adding 
"ize" (dirize) or "ify" (blockify) isn't a great solution, but there 
is some basis for them (compartmentalize, normalize, scarify, terrify). 


TO-* and AS-* have specific meanings, and are core funcs. What should 
the standard derivation be for this kind of behavior?

Including antonyms (for lack of a better term).

Janko: "will R3 have a way to define custom infix words?" To my knowledge, 
no.

My original name for undirize was fileize, but that seemed even more 
contrived. The only advantage "undirize" has is that if you know 
what dirize does, it's not difficult to figure out what undirize 
does. The dirize function is only still called that for historical 
reasons, but we are trying to not just change the names of functions 
that act the same as R2 unless there is a really good reason for 
it. We prefer to only break compatibility for semantic reasons, not 
naming.

That said, I would like to have the BIND? function also assigned 
to the word BOUND? by default, or perhaps BINDING? or CONTEXT? given 
its behavior. Just a preference.

Vector! is more like a typed array. There is nothing in R3 like list! 
or hash!.

I found hash! a very useful datatype, I still don't get why it has 
to be removed. Map! looks less flexible because you have to conform 
to the key/value data model and it doesn't seem possible to navigate 
in a map! like in a hash!. Why can't we have both hash! and map! 
in R3?

Exists a way to change user-agent with R3-alpha? I try system/schemes/http/user-agent: 
"Mozilla/4.0" but I get ** Script error: cannot access user-agent 
in path system/schemes/http/user-agent:

hum, or you can pass a header block to the write function as is:
>> write [ url!  [ User-Agent: "TOTO" ... ]  #{...data...}]

but it's bugy, you have to add yourself some missing header properties 
in the block to generate a valid request.

like Content-Type: "application/x-www-form-urlencoded; charset=utf-8"

I liked list and hash and did use them a lot.  List was was buggy 
though.

it's a little incoherent to use sort on lists

I used list! once in list-view, but found that it had too much overhead, 
when needing to use it as a block, so it had to be converted.

There are plans to extend FOREACH to map! and object!, and MAP could 
make sense too I suppose, but no plans for that. What did you use 
hash! for that wasn't keyed search, Doc, and what advantages did 
it give you over block! aside from a different datatype?

It only becomes duplicate values if the value is not a reference 
type (numbers, characters, etc.).

So, if I understand correctly, map! offers a superset of hash! features 
(without giving up anything)?

You give up position and persistent ordering, and in theory you also 
give up duplicate keys though there is a bug ticket about that.

SELECT works on object! and map! but FIND doesn't. PICK and POKE 
work, but they take keys rather than indexes. There are some (at 
this point undocumented) limits on what can be used as keys too (at 
least you can't use a block as a key in practice), but that may be 
a bug.

Pavel, the problem is that we can't hash complex structures (for 
now) so there is no advantage to using them as keys. I want a keys-of 
function though that returns a block of the keys that reference a 
value (by same? semantics for the value, perhaps).

The words-of function already returns *all* of the keys of a map!, 
just like it does for objects.

I agree, Doc, that sounds like a good plan. Map! storage and implementation 
is in theory more efficient than hash!, so having two of them shouldn't 
be a problem, as long as your wrapper functions keep track of the 
mirroring.

The only trick is that you remember that none means nothing, so don't 
bother with a none key. The theory of map! is either:

- Keys that are associated with none don't exist in the map!, so 
assigning none to a key makes it go away.

- All possible keys are in the map! but only the ones associated 
with something other than none are displayed.

There is no difference between these two theories in practice, and 
whether there is memory allocated for keys that you can't see is 
an implementation detail that is irrelevant to the use of map! (though 
there is usually not).

something related, in the past i made some tests to simulate hashs 
with integer keys in R2. I used a bitset as an index, mixed with 
block of blocks to store data.

my tests show that for 10000 records,  finding data is near as fast 
as with hashs. 
actually it's incomplete but you have the idea with this:

REBOL []
f: fast-dic: context [
	size: 100000

 hash: 128 - 1	;** hash size speed up the search, must be a power 
 of 2 - 1 (ie. 15, 31, 63, 127, 257 ...)
	master: copy/deep head insert/dup/only [] [] hash + 1
	index: make bitset! size
	flag: func [idx [integer!]][
		unless find index idx [
			insert index idx

   insert/only insert tail pick master idx and hash + 1 idx copy []
		]
	]
	flag?: func [idx [integer!]][find index idx]
	deflag: func [idx [integer!]][
		remove/part index idx
		remove/part find pick master idx and hash + 1 idx 2
	]
] 

t: now/time/precise
loop 10000 bind [flag random 99999] f
print now/time/precise - t
t: now/time/precise
loop 10000 bind [flag? random 99999] f
print now/time/precise - t

Just a question. Is there a way to rappresent  Unicode Characters 
inside a string with an escape sequence ?

^(hex characters)

. The console may not render the character properly if the font doesn't 
support it though - it may look like a space.

Yeah, vector! doesn't work yet. There are a lot of tickets for vector!, 
all deferred. That to-block bug is not there, but not unexpected.

Brian, I think we need to address the lack of list in R3.  Maybe 
a list like handling of block data that still returns a block.

I think that would be an excellent thing to address with a user-defined 
datatype.

People in general didn't use list!, because of the bugs and because 
block! was good enough for most uses. We haven't felt its lack for 
a year now. Most of the advantage of list! in regular code was handled 
by allowing insert and remove at the head of a block! to expand or 
contract the block, just like it does at the end, without a block 
copy. Just by making block! more efficient for inserts and removes, 
we have made list! even less necessary.

>> a: []
== []

>> dp [append a 1]
== make object! [
    timer: 32
    evals: 11
    eval-natives: 4
    eval-functions: 1
    series-made: 2
    series-freed: 1
    series-expanded: 1
    series-bytes: 464
    series-recycled: 0
    made-blocks: 1
    made-objects: 0
    recycles: 0
]


as you can see the block has been expanded, which means copied in 
another place

¨People in general didn't use list!, because of the bugs and because 
block! was good enough for most uses. We haven't felt its lack for 
a year now. Most of the advantage of list! in regular code was handled 
by allowing insert and remove at the head of a block! to expand or 
contract the block, just like it does at the end, without a block 
copy. Just by making block! more efficient for inserts and removes, 
we have made list! even less necessary.¨

>> dp [a: [] b: []]
== make object! [
    timer: 15
    evals: 12
    eval-natives: 3
    eval-functions: 1
    series-made: 1
    series-freed: 0
    series-expanded: 0
    series-bytes: 432
    series-recycled: 0
    made-blocks: 1
    made-objects: 0
    recycles: 0
]

>>

>> dp []
== make object! [
    timer: 14
    evals: 8
    eval-natives: 3
    eval-functions: 1
    series-made: 1
    series-freed: 0
    series-expanded: 0
    series-bytes: 432
    series-recycled: 0
    made-blocks: 1
    made-objects: 0
    recycles: 0
]

see, a block is always created by dp

If the block needs to be expanded because there isn't enough allocated 
the system does a block copy. If there *is* enough allocated, as 
when you preallocate using make block!, then the system doesn't have 
to do a block copy. That is R2 and R3.


What is new in R3 is that the "head" pointer of a block doesn't have 
to point to the beginning of  the allocated memory, just like the 
"tail" pointer in R2. This means a remove from the head of the block 
just shifts the pointer over one in R3, while in R2 you had to copy 
over the rest of the block contents to shift it towards the head 
of the allocated memory. Preallocated memory can also exist before 
the head of the block contents in R3. This means that there is no 
difference in overhead between inserts at the head or the tail of 
a block in R3.


In theory, inserts inside the block in the first half could be more 
efficient because you would only have to shift from the nearest end, 
not the tail. I don't know whether this optimization has been implemented.


Block operations in general could be faster because with no list! 
type we wouldn't have to special-case as much code, so we could make 
our code much faster through more aggressive optimization.


Btw, I submitted a tweak to DP to make it more accurate by subtracting 
its own overhead. It still has some variance though - have to tweak 
the native to fix that. Plus there is the extreme variance caused 
by Windows.

There is a proposal (looking likely to be implemented) to have FOREACH 
work on object! and map! types. The word list syntax would be restricted, 
but you could do your traversal that way. In the meanwhile you have 
WORDS-OF to get the keys in a block, VALUES-OF to get the values 
in a block, BODY-OF object! to get both in a block (map! proposed 
too) and TO-BLOCK of map! to get both in a block. It works, but the 
FOREACH proposal would create fewer intermediate blocks.

REBOL is a lower-level language than SQL. No set operations (for 
map!, ar least). Those kinds of SQL operations need to be done procedurally 
in REBOL, using block copies and such.

Well, we don't havee RIF yet so you'll have to stay curious for a 
while :(

hmmmm.... interesting behaviour of blocks....

Brian, does that mean that Carl had in main a possibly new feature 
asked for a while: 

RANGE :A reference which is a sub part of an existing serie. (supress 
the need of overheaded copy/part in our sripts)

I think it's not at all tricky to implement if the head of a serie 
is a logical offset now.

No, there are no plans to implement range. The head and tail of the 
series are attributes of the series, not the reference to the series. 
It's like the difference between the position attribute of a port 
and a series: For a series, the position is an attribute of the reference 
to the series, while for port, the position is an internal attribute. 
Still, a subseries reference could be implemented as a user-defined 
datatype as long as it is careful to make up for changes in the underlying 
series.

Rebin is intended to be a binary syntax for REBOL, like EBML for 
XML. All REBOL values (or maybe just the literal values) will be 
stored in a binary format. I suppose rebin would be an output format 
of MOLD and SAVE.

You would need a binary format to even have a way to store those 
references since REBOL syntax doesn't, even with MOLD/all.

No, just the value and the reference to the value with the position. 
The poisition isn't a reference, just an integer (internally). So 
you need the series, the length of the series, a datatype tag, the 
reference to the series, and the offset. The reference and offset 
would be part of the contents of anotther series. It would take less 
space in binary than in memory or REBOL syntax.

B: next A: "blabla"

A (contains a logical reference) --> (physical reference)-->"blabla" 
(value)

B (contains a second logical reference to the same physical reference) 
--> 

if "blabla" is expanded the physical reference is modified and "blabla***" 
is stored in another one place.
The references in A and B are not modified.
That why i said there is 2 indirections (2 references)

When surplus memory is preallocated for a block, is the block still 
initialised at the start of the memory or somewhere in the middle?

one more question: when a value is removed inside the block. Is always 
the tail which is moved, or the head can be also ? (depending of 
the nearest one)

Pavel, i'm talking about the new implementaion of blocks in R3 (it 
was addressed to Brian). There is no empty space in the middle of 
a block, never.

Yes Pekr, this has nothing to do with zero based indexing. Series 
are still one based:
>> b: [a b c d e]
== [a b c d e]

>> b/1
== a

>> b/0
== none

Pekr, [pick b 1] behaviour is not changed:

>> b: [a b c d e]
== [a b c d e]
>> b: skip b 3
== [d e]
>> b/0
== c
>> pick b 0
== c
>> b/1
== d
>> pick b 1
== d
>> b/-1
== b
>> pick b -1
== b

I have to think and come up with example, when you use index? something, 
to get your position, I think that then the change shifts the calculation 
in such a case...

Pekr, I don't know if it makes a difference, because INDEX? seems 
to be used only on positive values. Can INDEX? return negative values?

From what I see here, it can't, so it won't make a difference.

btw - one question - in above Rebolek's example - why when we reassing 
(reference) a block ('b in his example), index? refers to original 
block, whereas functions like 'length? behave locally to new referenced 
block? Just a question :-)

'b is a block! type, so no excuse here that it just "points to original 
block at certain position" applies here :-)

It's perfectly consistent and useful for subblocks. If INDEX? was 
local to the block in use, a lot of things wouldn't be possible.

Petr, I have been proposing that new PICKZ and POKEZ functions be 
added to do a 0-based PICK/POKE, instead of having vector! be 0-based. 
This would give us 0-based referencing abilities for all series, 
not just vectors, then we could make vectors 1-based like the rest. 
There are real advantages to 0-based indexing so it would be good 
to have it, but consistency is better here.


Carl was not proposing to make a change to PICK and POKE in his blog: 
he already (half) made the change. He was asking us in his blog if 
he should change it  *back* to the old, buggy R2 behavior. I think 
he should finish fixing PICK, POKE and AT instead.


Henrik, INDEX? returns a 1-based index from the *head* of the series 
- that's why it's always positive.

Is there a way to control/stop a task by the calling process or shall 
the called task check a semaphore set by the calling process?

Tasks are really up in the air, especially since threads are being 
considered to be a bad model nowadays in the multitasking community. 
It would be good for REBOL to start with a good model now, before 
it becomes too late to change later. Maybe green processes like Erlang 
:)

erlangs concurrency model, hot code loading, easy process communication, 
robust message passing 

and a nice database (let's call it rebnesia)...all ported to our 
fabulous rebol.
...when did you said you are ready with this brian? :-))

Erlang uses their own lightweight process implementation (like you 
could write in REBOL .... hint); remember that the first version 
were written in Prolog. The Erlang VM uses all cores on a machine. 
The "no shared memory" aprroach makes this easy.


R3 should be able to utilize multiple cores. Then with async networking 
and people finally understanding dialecting who needs tasks? Just 
roll your own.

He might mean "who needs threads?", because "tasks" aren't defined 
yet. As Erlang proves, there are advantages to having both green 
processes and full processes, especially if they are basically interchangeable 
as far as the code running in them is concerned. You *could* just 
go with full processes like R2 does ("Just roll your own."), but 
that wouldn't be as efficient as a hybrid green/full approach, particularly 
on platforms like Windows that have huge creation and switching overhead 
for full processes.

Having to roll our own would be very cumbersome. It might be simple 
for Maarten who has written threading engines for R2, but for us 
lesser ones, it would be handy, if it was built in. I consider not 
having it a showstopper in many cases. AFAIR, threading was the original 
reason for starting R3 development?

Hope we get a GUI for R3 chat soon ... I find it just too hard to 
read it in a console.

One formatting fix they need to make for RebDev is to put a blank 
line as the first input of each post.  Makes it a bit more readable.

Also, maybe add an R options to automatically have the message prefixed 
with a REPLY to #XXXX where x is the message number.

I'm still a noob to RebDev.

I thought Henrik's R3 GUI skills would have manufactured a GUI for 
RebDev by now.

R3's GUI is missing some necessary styles for a RebDev client, notably 
grids. Filling in the blanks needs more programmers, so we are focussing 
on getting the file management portions of DevBase integrated first 
so we can get other people involved in the process. Priorities :(

This /into option proposal is based on profile-driven *language design*. 
It's a new approach to REBOL.

I guess, I don't follow.  Maybe, I'm jumping to conclusions about 
what /into actually implies.  To me, /into implies that the original 
series passed to the function is what is modified and not a copy 
of it.

No, that's the /no-copy option - it's a completely different thing. 
See #1789.

So you mean something like:
rejoin/into [a b c] out
instead of:
append out rejoin [a b c]
? I think I like it.

No, those are modifier functions. We already have those, and the 
algorithm for implementing a modifier is completely different than 
a builder. Some functions (like DELINE) could be changed into modifiers 
with /no-copy and that wouuld make sense. It doesn't make sense when 
the builder is creating something based on a spec or template, which 
often has a completely different datatype than the result. For these 
functions an output buffer makes more sense.

(That was a reply to Paul)

Modifiers and builders are completely different classes of functions, 
with some overlap (see DELINE or REPLACE vs. REWORD). When the function 
is making a copy of the spec with some tweaks, a /no-copy option 
makes sense, or a completely separate function with complex mezzanines. 
When the builder function creates something really different than 
the spec (like ARRAY or READ), there is no point to a /no-copy refinement 
since there is no eqivalent modifier even in theory.

That is the difference between /no-copy (which turns a builder into 
a modifier), /copy (which turns a modifier into a builder), and /into 
(which makes a builder insert into a provided buffer instead of creating 
its own series).

henrik, the key is your supplying a buffer.

Paul, I think I follow that far. What happens to the buffer is what 
I don't get. When BrianH says it inserts, that implies a lot of overhead 
on manipulating that buffer to me.

BrianH, but are we not doubling in size the memory space for that 
function?  If were passing it a huge series to begin with - would 
we not end up with a buffer that is potentially as big or bigger 
than the original series if were inserting data?

If you *only* model after INSERT it minimizes the changes to the 
builder code. We aren't saving memory on a single call, we allowing 
careful reuse of memory on multiple calls. I've done the tests, the 
overall memory overhead goes down drastically.

Most of the time a function using build/into functions will preallocate 
a series, build some data into it, then return the series. Builder 
functions can use lower-level builder functions. You get a reduction 
in intermediate series creation overall.

Give me a moment and I'll show you an example, using variants of 
the COLLECT function.

; Here's a version of COLLECT without /into, a typical example of 
a builder.
collect: func [

 "Evaluates a block, storing values via KEEP function, and returns 
 block of collected values."
	body [block!] "Block to evaluate"
	/local output
][
	output: make block! 16
	do func [keep] body func [value /only] [
		apply :append [output :value none none only]
		:value
	]
	output
]

; Here's COLLECT with the /into option.
collect: func [

 "Evaluates a block, storing values via KEEP function, and returns 
 block of collected values."
	body [block!] "Block to evaluate"
	/into "Collect into a given series, rather than a new block"
	output [series!] "The series to output to"
][
	unless output [output: make block! 16]
	do func [keep] body func [value /only] [
		output: apply :insert [output :value none none only]
		:value
	]
	either into [output] [head output]
]


Note that the version with /into also lets you use other series types 
than just block!. This option added to REDUCE and COMPOSE would let 
you create parens and paths as well, even though REDUCE and COMPOSE 
can only take block! specs.

The /into option will be handled natively when added to natives, 
but handled by mezzanine code when added to mezzanines. It's a minimal 
change either way. Backports to R2 of native /into options might 
need mezzanine wrappers though.

New discovery: APPLY treats 'a and :a parameters like a, ignoring 
the special treatment that specifying parameters that way would otherwise 
give you. This is a great thing: Now you can write functions that 
function values without the code injection risk that those special 
parameters have.

Janko, to reanswer your question: "will R3 have a way to define custom 
infix words?"

Yes, at least custom words to refer to existing infix functions for 
now. In theory this new behavior could be used to make new op! functions, 
but there will likely not be a facility to do so until the plugin 
model is finalized, and the redirect-to-an-action functionality may 
remain even then.