AltME groups: search

Gabriele's need to delimit the rules with '| brings up another old 
mezzanine thought: rejoin/with. I created a separate mezz, called 
DELIMIT, but this is a fairly common need IME. Should we start another 
ML thread? :-)

Maybe this useful to someone


delimit: func [data /quoted /with separator  [string! char!]  /local 
result quote][

    result: copy {}

    unless with [separator: ","]

    quote: either quoted [:mold][func[x][x]]
    append result quote form first data

    foreach value next data [
      append result separator
      append result quote form value

    ]

]



delimit [1 2 3] ;== "1,2,3"

delimit/with [1 2 3] ":" ;== "1:2:3"

delimit/quoted [1 2 3] ;== {"1","2","3"}

delimit/quoted/with [1 2 3] "|" ;== {"1"|"2"|"3"}

I have a DELIMIT function that will do it, changing the series in 
place, with the exception of the trailing element. So the final result 
would look like this:

	append delimit/skip series new-value record-size new-value

The basic idea you want is this:

	series: skip series size 
	series: head forskip series size + 1 [insert/only series value]


My DELIMIT func also works with list! and any-string! types correctly, 
which simple code above doesn't account for (the +1 part is simplified).

as-utc: func [date] [
    if all [date/zone  0:00 <> date/zone] [
        date: add date negate date/zone
    ]
    date/zone: none
    if none? date/time [date/time: 0:0:0]
    date
]

to-ISO8601-date: func [
    "Converts a date! value to an ISO 8601 format string."
    date [date!] "The date to format"

    /T           {Use T to delimit time value, rather than a space}
    /no-zone     "Don't include the timezone"
    /local pad z
][
    pad: func [val /to len] [
        val: form val
        head insert/dup val #"0" ((any [len 2]) - length? val)
    ]

    rejoin [
        pad/to date/year 4 "-" pad date/month "-" pad date/day
        either T ["T"] [" "]

        either none? t: date/time ["00:00:00Z"] [   ;<< reusing 'T here!
            rejoin [

                pad t/hour ":" pad t/minute ":" pad round t/second
                either no-zone [""] [

                    either 0:00 = z: date/zone ["Z"] [  ;<< setting 'z here!
                        rejoin [
                            pick ["+" "-"] z/hour > 0
                            pad abs z/hour pad abs z/minute
                        ]
                    ]
                ]
            ]
        ]
    ]
]

delimit: func [
        "Insert a delimiter between series values."
        series [series!] "Series to delimit. Will be modified."
        value            "The delimiter to insert between items."

        /skip   ;<-- be sure to use system/words/skip in this func

            size [integer!] "The number of items between delimiters. Default 
            is 1."
    ][
        ; By default, delimiters go between each item.
        ; MAX catches zero and negative sizes.
        size: max 1 any [size 1]

        ; If we aren't going to insert any delimiters, just return the series.

        ; This check means FORSKIP should always give us a series result,
        ; rather than NONE, so we can safely inline HEAD with it.
        if size + 1 > length? series [return series]
        ; We don't want a delimiter at the beginning.
        series: system/words/skip series size

        ; Use size+n because we're inserting a delimiter on each pass,

        ; and need to skip over that as well. If we're inserting a

        ; series into a string, we have to skip the length of that

        ; series. i.e. the delimiter value is more than a single item
        ; we need to skip.
        size: size + any [

            all [list? series  0] ; lists behave differently; no need to skip 
            dlm.

            all [any-string? series  series? value  length? value]
            all [any-string? series  length? form value]
            1
        ]
        head forskip series size [insert/only series value]
    ]

fmt-for-Excel: func [blk dlm] [
	rejoin delimit collect fld [
		foreach val blk [
			val: form val
			replace/all val {"} {""}
			if find val #"," [val: rejoin [{"} val {"}]]
			fld: val
		]
	] dlm
]

fmt-for-Excel [{"A"} "B" "C,C" {"4,4"}] #","

it's not a bug - parse without a rule is meant for csv parsing, and 
quotes delimit a field. it's not as useful as it was intended to 
be, but it's intentional behavior. you need to provide your own rule 
if you don't want quotes to be parsed.

Tomc  -- This version means that I need to have the entire file read 
in as a string -- Not with Read/Lines -- Because the newline will 
the the "delimiter" within the string while the Read/Lines will delimit 
each newline to a separate string inside a block.  Do I have that 
right?

Janko: you can, in general, use string parsing for EDSLs. Use {..} 
to delimit your code blocks instead of [..], and a different set 
of lexical limitations will apply.

Also I can clearly understand, why Janko needs the delimiter in his 
dialect. It simplifies a lot when you can delimit values, where some 
of the values can be functions requiring arguments. Without the delimiter 
you must add pretty large complexity which will provide info, how 
many args require each funcion.

; A dialected version of this could be very flexible; allowing more
    ; than just fixed size groupings.

    ; This could also be done by adding a /SKIP refinement to INSERT.
    delimit: func [
        ;[throw catch]
        "Insert a delimiter between series values."
        series [series!] "Series to delimit. Will be modified."
        value            "The delimiter to insert between items."

        /skip   ;<-- be sure to use system/words/skip in this func

            size [integer!] "The number of items between delimiters. Default 
            is 1."
    ][

        ; Hmmm, I wonder if we could extend the function spec dialect

        ; to include constraints like this declaratively? And should

        ; we trap the arg like this, or just use MAX to make sure we

        ; have a positive value? I think I'll do the latter for now,
        ; but leave this here as a comment.
        ;if all [size not positive? size] [
        ;    throw make error! join [script invalid-arg] size
        ;]
        ; By default, delimiters go between each item.
        ; MAX catches zero and negative sizes.
        size: max 1 any [size 1]

        ; If we aren't going to insert any delimiters, just return the series.

        ; This check means FORSKIP should always give us a series result,
        ; rather than NONE, so we can safely inline HEAD with it.
        if size + 1 > length? series [return series]
        ; We don't want a delimiter at the beginning.
        series: system/words/skip series size

        ; Use size+n because we're inserting a delimiter on each pass,

        ; and need to skip over that as well. If we're inserting a

        ; series into a string, we have to skip the length of that

        ; series. i.e. the delimiter value is more than a single item
        ; we need to skip.
        size: size + any [

            all [list? series  0] ; lists behave differently; no need to skip 
            dlm.

            all [any-string? series  series? value  length? value]
            all [any-string? series  length? form value]
            1
        ]
        head forskip series size [insert/only series value]
    ]

make-csv: func [block] [rejoin delimit copy block #","]

Gabriele, thanks for the info about the rewrite rules. That's an 
interesting way to lay them out - a little more traditional that 
I've come to expect from REBOL, but that #==> is unlikely to be found 
in REBOL code so its use to delimit the parse rules should work nicely. 
I look forward to trying it out!

delimit [1 2 3] ;== "123"
delimit/with [1 2 3] "," ;== "1,2,3"
delimit/quoted 
[1 2 3] ;== {"1","2","3"}
delimit/quoted/with [1 2 3] "|" ;== {"1"|"2"|"3"}

npa: 
703
nxx: 938
delimit [npa nxx] ;== "npa nxx"
delimit/reduce [npa 
nxx] ;= "703938"
delimit/reduce/with [npa nxx] "-" ;== "703-938"
delimit/reduce/quoted/with 
[npa nxx] "|" ;== {"703"|"938"}

;; usage examples

delimit [1 2 3] ;== "123"
delimit/with [1 2 3] "," ;== "1,2,3"
delimit/quoted [1 2 3] ;== {"1","2","3"}
delimit/quoted/with [1 2 3] "|" ;== {"1"|"2"|"3"}

npa: 703
nxx: 938
delimit [npa nxx] ;== "npa nxx"
delimit/reduce [npa nxx] ;= "703938"
delimit/reduce/with [npa nxx] "-" ;== "703-938"
delimit/reduce/quoted/with [npa nxx] "|" ;== {"703"|"938"}

;; Here is my implementation, maybe in can be fine tuned.


delimit: func [data [block!] /reduce /quoted /with separator  [string! 
char!]  /local result quote][
  unless empty? data [
    result: copy {}
    unless with [separator: ""]
    quote: either quoted [
        func[x][rejoin [{"} x {"}]]
    ][
        func[x][x]
    ]

    insert result quote form either reduce [do first data][first data]
    foreach value next data [

    insert insert tail result separator quote form either reduce [do 
    value][value]
    ]
    result
  ]
]

;; New version does proper reduction


delimit: func [data [block!] /reduce /quoted /with separator  [string! 
char!]  /local result quote][
  if reduce [data: system/words/reduce data]
  unless empty? data [
    result: copy {}
    unless with [separator: ""]
    quote: either quoted [
        func[x][rejoin [{"} x {"}]]
    ][
        func[x][x]
    ]
    insert result quote form first data
    foreach value next data [
      insert insert tail result separator quote form value
    ]
    result
  ]
]

;; example

>> delimit/reduce/with [npa nxx square-root 4] " "  ;== "703 938 
2.0"

And DELIMIT

Do you want the added functionality of the DELIMIT above?

conjoin is cool, language root are like "delimit/with"

Ah yes, Jaime, I wanted to say "DELIMIT" lies to you when it is used 
without the /WITH refinement, because DELIMIT implies delimiters. 
If there aren't any delimiters, then it is lying.

So an idea I would like to explore is to split the functionality 
of DELIMIT into perhaps two functions. As this kind of operation 
is very common, I suggest that this will save lots of typing.

The description I have for DELIMIT is "Returns a string! constructed 
from the chained values in the block."

I should not that I am not attached to DELIMIT. It stated like with 
a different purpose, but now I see that is more generic than REJOIN. 
So I offer it after modifying it a bit.

Picking names is very hard. I kind of like to have all the functionality 
cram in one function, less words to remember. So I left the naming 
decision to the community, just hope the features of delimit are 
include in R3.

So I would drop the /WITH refinement and make it implicit in a DELIMIT 
or CONJOIN function... (maybe..)

1) rejoin/quoted [...]   ; <-- this is most similar to rejoin, or 
delimit without the /WITH

2) conjoin/quoted "," [...]    ; <-- this is like delimit/with/quoted 
 (and the non-optional "with" argument is specified first)

The second function, CONJOIN, is like your DELIMIT, except with WITH 
refinement implicit.

>> delimit [1 1 / 2 ]  ;
== "11/2"

>> delimit/reduce [1 1 / 2]  ;
== "10.5"

delimit [1 1 / 2] ;== "10.5"
delimit/literal [1 1 / 2] ;== "11/2"

Also, your implementation is slower than DELIMIT, by an order of 
magnitude.

>> time-block [conjoin "," []] 0.05  ;
== 4.953515625E-5

>> time-block [delimit/with [] ","] 0.05  ;
== 2.453125E-6

When I said I like conjoin, I meant the word "conjoin". I think it 
woud be a good name for the ONE function that would perform a useful 
subset of all of the tasks that have been specified here as part 
of the various functions suggested here, starting with Jaime's delimit, 
but with the delimiter mandatory. We already have a verson of delimit 
without the delimiter - it's called rejoin.

To me, delimit would intersperse the delimiter in the block but not 
join it. That what the word suggests, at least.

So the suggestion of globing all the DELIMIT features into the new 
function.

I am currently rewriting delimit and conjoin to implement a few of 
my own ideas for it. You do yours and we'll compare.

As far as I'm concerned, delimit and conjoin are separate concepts.

Like I said before, the word "delimit" doesn't imply joining, while 
"conjoin" does. I'm expecting to make my conjoin function use my 
delimit.

Strings are not the typical usage for me. I saw those functions posted 
above and none of them would work for me. I'm writing my own right 
now, for posting here. I have delimit so far, and am now working 
on conjoin.

As an example, here is delimit. I'll post conjoin soon.

delimit: func [
    "Put a value between the values in a series."
    data [series!] "The series to delimit"
    delimiter "The value to put into the series"
    /only "Inserts a series delimiter as a series."
    /copy "Change a copy of the series instead."
] [
    while either copy [
        copy: make data 2 * length? data
        either empty? data [[false]] [[
            copy: insert/only copy pick data 1
            not empty? data: next data
        ]]
    ] [
        copy: data
        [not empty? copy: next copy]
    ] either only [[
        copy: insert/only copy delimiter
    ]] [[
        copy: insert copy delimiter
    ]]
    head copy
]

Well, it will be a little hard to check against your delimit since 
that will be more comparable to my conjoin. Still, building the while 
statement this way is no slower than calling one of several different 
while statements depending on options, and it is a lot less redundant. 
Plus it's fun.

delimit: func [
    "Put a value between the values in a series."
    data [series!] "The series to delimit"
    delimiter "The value to put into the series"
    /only "Inserts a series delimiter as a series."
    /copy "Change a copy of the series instead."
] [
    while either copy [
        if empty? data [return make data 0]
        copy: make data 2 * length? data
        [
            copy: insert/only copy first data
            not empty? data: next data
        ]
    ] [
        copy: data
        [not empty? copy: next copy]
    ] pick [
        [copy: insert/only copy delimiter]
        [copy: insert copy delimiter]
    ] only
    head copy
]

Brian, delimit function: 

- For long-term readability, I would avoid reusing 'copy as a variable. 
I suggest 'result, even if it means using another word.

- I understand with the /copy refinement you are able to get more 
speed in creating the result block, but I think I would prefer just 
letting the user copy the data before passing to delimit. This would 
give a simpler implementation, easier to read again.

I don't wish to devalue your effort in getting this version - I did 
a similar thing optimizing conjoin - made it harder to read.

Brian, I think Jaime was making the same point, as I do above, about 
speed vs clarity, with regards to /copy. Some benchmarking is needed 
comparing:

- delimit/copy data    ; <--- delimit with /copy refinement implemented
- delimit copy data    ; <--- delimit without /copy refinement

Brian, I've read carefully through your conjoin (but haven't tested 
yet), and I like it, except for *one* thing - I would reverse the 
order of the data and delimiter arguments. (Actually, I'm searching 
now for a better word than "delimit". It doesn't quite seem right.)

Looking at your conjoin with the /only and /pad-only refinements, 
it seems that with the /only you are trying to recreate the delimit 
function, but not as usefully. I thought of using pad as a variable 
name, but "delimiter" was more appropriate since padding functions 
usually pad outside the data, not within it. Let me try to add you 
fixes to my version and see what I get.

delimit: func [
    "Put a value between the values in a series."
    data [series!] "The series to delimit"
    delimiter "The value to put into the series"
    /only "Inserts a series delimiter as a series."
    /copy "Change a copy of the series instead."
    /local
] [
    while either copy [
        if empty? data [return make data 0]
        local: make data 2 * length? data
        [
            local: insert/only local first data
            not empty? data: next data
        ]
    ] [
        local: data
        [not empty? local: next local]
    ] pick [
        [local: insert local delimiter]
        [local: insert/only local delimiter]
    ] none? only
    head local
]

conjoin: func [
    "Join the values in a block together with a delimiter."
    data [any-block!] "The series to join"
    delimiter "The value to put into the series"
    /only "Inserts a series delimiter as a series."
    /quoted "Puts string values in quotes."
    /local
] [
    if empty? data [return make data 0]

    local: tail either series? local: first data [copy local] [form :local]
    while [not empty? data: next data] either any-string? local [
        either quoted [
            local: insert tail insert head local {"} {"}

            [local: insert insert insert insert local delimiter {"} first data 
            {"}]
        ] [[local: insert insert local delimiter first data]]
    ] [pick [
        [local: insert insert local delimiter first data]
        [local: insert insert/only local delimiter first data]
    ] none? only]
    head local
]

The copy refinement on the delimit function should be faster than 
pre-copying the data because the copy is preallocated to size.

I did understand your point about the /only and /pad-only refinements, 
but I realized that my delimit function made the change unnecessary, 
since its behavior was exactly what you were getting at. Using your 
example:
>> delimit ["one" 2 [3]] '|
== ["one" | 2 | [3]]

Keeping the concepts distinct wasn't the only reason I made seperate 
"conjoin" and "delimit" functions - it's more efficient too.

Yes, please.

I think I lost sight of the overall picture when I added /only and 
/pad-only. (Reminds me of a similar thought process in another frenetic 
function creation a year or two ago (?)) I was not thinking of the 
functionality that DELIMIT covered when I was "designing" those refinements. 
So on further reflection, it looks to me like you are right, for 
CONJOIN, using INSERT rather than INSERT/ONLY on the DATA values 
is more useful.

delimit: func [
    "Put a value between the values in a series."
    data [series!] "The series to delimit"
    delimiter "The value to put into the series"
    /only "Inserts a series delimiter as a series."
    /copy "Change a copy of the series instead."
    /local
] [
    while either copy [
        if empty? data [return make data 0]
        local: make data 2 * length? data
        [
            local: insert/only local first data
            not empty? data: next data
        ]
    ] [
        local: data
        [not empty? local: next local]
    ] either only [
        [local: insert/only local delimiter]
    ] [[local: insert local delimiter]]
    head local
]

conjoin: func [
    "Join the values in a block together with a delimiter."
    data [any-block!] "The values to join"
    delimiter "The value to put in between the above values"
    /only "Inserts a series delimiter as a series."
    /quoted "Puts string values in quotes."
    /local
] [
    if empty? data [return make data 0]

    local: tail either series? local: first data [copy local] [form :local]
    while [not empty? data: next data] either any-string? local [
        either quoted [
            local: insert tail insert head local {"} {"}

            [local: insert insert insert insert local (delimiter) {"} first data 
            {"}]
        ] [[local: insert insert local (delimiter) first data]]
    ] [
        either only [

            [local: insert insert/only local (delimiter) first data]
        ] [[local: insert insert local (delimiter) first data]]
    ]
    head local
]

Brian, maybe delimiter needs to be parenthesised in DELIMIT as well 
?

You don't need to parenthesise delimiter in delimit because every 
reference to it is at the end of a block. Using parentheses is slower 
in REBOL, so why use them when you don't have to?

Wow. A lot of thought here. I've skimmed it (trying to catch up on 
things quickly), and will just post a couple quick thoughts.


* REJOIN is probably a good enough name to stick with; I don' t know 
that any of the others are that much more meaningful in the context 
of REBOL. 

* Changing JOIN's params would break a lot of code.


* I have a DELIMIT function as well, and like the name. Mine doesn't 
have /copy or /only refinements (it always uses insert/only), but 
it has /skip.

i did not follow the whole discussion, so dunno what exactly 'delimit 
does, but could we have also a 'pad function?

Gregg, I think Brian's last post with DELIMIT and CONJOIN are probably 
the best in this thread.

I would normally be on the side of dynamic break - it would be easier 
to teach, and the rest of REBOL follows that model. What would be 
the major advantage of lexical break in a non-compiled language? 
REBOL code blocks aren't really lexically associated with their control 
structures in the DO dialect, as my conjoin and delimit functions 
above demonstrate. This isn't rebcode you know.

For example, I think RobertS might be happier if a new special type 
of string which is delimited by ~{ and }~ was added to Rebol.

In the content, single braces } or single tildes ~ would not need 
any escaping unless they happened to be together so that they look 
like the ending delimiter.
Or maybe no escaping is possible/necessary in such a string.

But now I'm also thinking of the start and end unique key strings 
used to delimit email attachments...

Paul, that function has been proposed as DELIMIT - not as REJOIN 
because that function needs to be low-level - but hasn't been added 
yet since there is no consensus about the feature set. No consensus 
means it goes in the library rather than the mezzanines. If you can 
make a version that is so awesome that everyone will agree with your 
feature set, we'll add it :)

hmm, there is a 'delimit function ... it seems to be renamed to 'split 
....

There is a conceptual conflict between the treatment of splitting 
into parts by length and splitting by delimiter, that has the effect 
of limiting both sets of behavior. It would be better to put the 
delimiter splitting into a separate function called DELIMIT. This 
would allow the dialected variants of SPLIT and DELIMIT to develop 
separately without conflict, and make the SPLIT dialect easier to 
understand. Then you would have two relatively simple functions with 
a clear distinction between them.

Brian, DELIMIT means something very different to me. My DELIMIT func 
inserts delimiters.


I don't have a problem with using a 'skip keyword. As I said, the 
current behavior wasn't the original design and may be Carl's doing. 
Check with him on that.


There is a conceptual conflict between the treatment of splitting 
into parts by length and splitting by delimiter...

 -- I don't see that, but I'm biased. I'm always happy to see alternative 
 designs.

I checked a half-dozen online dictionaries to get that definition 
of DELIMIT. Perhaps you're checking different dictionaries.
The conceptual conflict:

- Dialected splitting has incompatible dialects, which makes both 
more limited than they should be.

- Splitting a block based on a delimiter of one of the types used 
by length-based splitting isn't allowed.

I think that the word for inserting delimiters the way your old DELIMIT 
function did is "intersperse", but we can do better than that.

DELIMIT sounds OK to me for inserting delimiters, with TOKENIZE probably 
its reverse

wouldn't DELIMIT/ENLIMIT make sense? We have already en/decloak, 
de/encode, de/enbase, en/deline ....

delimit:  set, mark, or draw the boundaries of something 

ENLIMIT doesn't make sense to me.

On the DELIMIT func name topic, my original suggestion, long ago, 
was to add /SKIP to INSERT, but that never went anywhere. What is 
the current feeling toward that (knowing that getting it added might 
be an obstacle).

It's currently associated with fixed length records. I'm not sure 
that's flexible enough for DELIMIT

Three reasons:

- The { and } are not chosen at random, they are the consequence 
of using those characters to delimit strings themselves.

- There aren't many characters or character sequences that can be 
optional without conflicting with other stuff in the grammar.

- Syntax processors with user-defined stuff in them are much slower 
than ones without them.