[Dialects] Questions about how to create dialects
Created this group to discuss over dialect programming / how to create them best.
Ok, here is my first question: I'm constantly facing the problem of optional parameters. 'keyword set name string! opt [set caption [string!]] opt [set size integer!] opt [set maxlength integer!] opt [set default string!] As parse walks from left to right through this line the order of parameters play a role. After giving the first INTEGER value you can't specify the 'caption part anymore. On the other hand, if you don't want to specify the 'caption but the 'default you have to provide an "" for 'caption because otherwise the second string! will be used as 'caption instead of 'default.
How to handle this "dillema" best?
is this typical to/thru first [a | b | c] problem?
The problem seem to be, that there are no keywords for your optional parameters, and therefore the order of parameters paly a role. There are 2 string! and 2 integer! parameters, and if order didn't play a role, there is no way, the parser can figure out, which one is which. You may wanna go something like this: 'keyword set name string! any ['caption set caption string! | 'size set size integer! | 'maxlength set maxlength integer! | 'default set default string!]
Or you could put your parameters in a block: 'keyword set name string! opt into [any ['caption set caption string! | 'size set size integer! | 'maxlength set maxlength integer! | 'default set default string!]]
Robert: This is a problem I'd love to see an answer to -- I encountered it last parsing tags, eg. <a> can have href= and title= in no specific order, and with no certainty of title=, resulting in (psuedo) ['a [href opt title | opt title href]]. Naturally, this exponentially multiplies: ['img [src alt opt width opt height opt title opt id opt class | ... ad infinitum ...]] -- I can't help but feel that I'm missing something...
Not the same problem, though still a challenge for dialects...
Not sure there is a good solution other than to require optional parameters to have keywords to identify them. Then you need to take only one pass at the input, and use defaults (or ignore) parameters that are not present. At the end of the pass, you can then see if the collection you have makes sense (maybe some optional parameters are needed if others are also present). Basically, its raw data validation....Boring, but necessary.
Without keywords: (integers: copy  strings: copy ) 'keyword set name string! any [ set string string! (append strings string) | set integer integer! (append integers integer) ] (set [size maxlength] integers set [caption default] strings) but less readable.
@Chris, I made a HTML3.2 dialect in REBOL at one point. It uses keywords for the optional parameters, and the order or parameters is irrelevant. You can find it, if you start REBOL/View desktop and goto http://home9.inet.tele.dk/johnn/index.r Not much documentation there, I'm afraid, but you can see a small example in the text file at the site. The good thing with the html32 dialect is, that it'll always produce 100% correct html code. If the input has error, you'll see a message.
Geomol: this is going the other way -- I'd like to establish that -- <a href="foo"> is valid and <a href="foo" href="foo"> is not. Scaling up, how else to know that both <img src="foo" alt="Foobar" width="10" height="20" class="icon" /> and <img class="icon" src="foo" width="10" height="20" alt="Foobar" /> are valid and effectively the same tag without providing every permutation?
I know this isn't a Rebol dialect, but the same issue is encountered...
In terms of a Rebol dialect, valid forms of <a> are: [a href "foo"] [a href "foo" title "Foobar"] [a title "Foobar" href "foo"]
Invalid examples: [a] [a href "foo" title "Foobar" href "foo"] [a title "Foobar"]
One way to solve the problem with required parameters is to use on/off flags. The rule for an anchor <a> will then be something like: anchor: [#"a" (href: off) space some ["href=" (either href [print "HREF already found!"] [href: on]) set arg chars | "title=" .....]] (if href = off [print "HREF required in anchor!"]) You can find this technique in the html32 dialect (I mentioned) for the src and alt parameters of the image tag <img>.
Ah, I see. Still, it doesn't seem most elegant. Another option would be to generate the permutations dynamically, but again, not so elegant...
Using keywords for parameters of course works but bloats the dialect, it's than mostly like HTML, XML only the opening/closing tags can be obmitted.
I like Vincent's approach to collect the things and than assign them in the order.
But I have an other problem with the non-keyword approach: 'keyword set name string! opt [set caption [string!]] opt [set size integer!] opt [set maxlength integer!] opt [set default string!] In the above example the only order I can specify is: string! integer! integer! string! Right? I can leave out some parts but I can't shuffel them. This isn't possible: string! string! integer! integer! Because PARSE walks through the rule once from left to right. So by passing the second string! the integer! rules have been already "eaten".
The next question is: How do I provide only the second integer!? I would like to use NONE for skipped paramters. How can I handle this. I tried to add word! and than provide none but this will be parsed into a word! but NONE? returns FALSE...
Chris, multiplying out the permutations isn't a solution. To much combinations. The idea of Geomol is right. You need to keep track. The thing is that parse doesn't has a backtracking (or parallel checking of rules). If we would have this feature you could write something like: once [any [... | ... | ...]]
long requested feature imo some of us regard as not needed ...
it would really make my life easier sometimes to be able to find first occurance of A or B or C ...I know it can be done by using proper grammar rules, but - not by me ... so we are letting parse to work in some cases only for gurus, which are good at hacking such solution. But I thought rebol should made simple things simple .... and I find mixture of "to first occurance of any value" as usefull ....
This can be done with some [...] and nested rules. But as said, if you want to handle it only ONCE you need to keep track of it yourself.
I am not sure I am able to easily write nested rules, nor beginners, while something like to/thru [a | b |c ] would be really easy to use, wouldn't it?
Well, otoh Carl is open to changes to Core imo, as he wants to concentrate upon Language ... so - why not suggest him any enhancement?
Look at %vt-index.r in the desktop-source. its only drawback is, the last value counts, instead of insisting on a 'once.
Robert: to remove parameters order, you can act like this : strings: copy  integers: copy  rule: ['keyword (clear strings clear integers) some [set p integer! (append integers p) | set p string! (append strings p)] (size: integers/1 maxlength integers/2 caption: strings/1 default: strings/2)]
It's the principle used by VID but by a function, not with 'parse (see grow-facets func in SVV) But it can't solve the datatype order : you can't passed Maxlength without Size ! But you can add another keyword to handle this case, like : ['maxlength set p integer! (maxlength: p)] It's what VID does : tuple! can be color or font color (depends the style you use). To be sure it's font color, you have the 'font-color keyword.
Dide, ok thanks. How can I specify the "empty" parameters? I would like to use NONE so that the variables can be tested with NONE?
Sometimes do'ing the result work: >> a: 'none == none >> type? a == word! >> a: do a == none >> type? a == none!
you parse a block, get 'none and want to handle it as none! ? Use it as word and [ if word = 'none [..] ] . Or use [ get word ]
Dide, how to handle the "can't pass 'maxlength without 'size" problem best? I mean how to specify no-maxlength? Providing a 0 or a -1? I would like to use 'none for all those values, but 'none can't be destinquished if it's a none-integer or none-string.
I could use STRING! and INTEGER! as empty values.
For my ML dialect (for easily creating HTML and XML), I didn't bother with error processing.
That's because it's designed for experts to use. And the experts aren't going to be making simple errors.
For other applications, I use different dialects, like eText for simple text -> HTML layout.
Something to consider, when you have cases like Chris's href examples above, is the separation of syntax and semantics. Also, for Robert's examples, remember that most languages have very strict rules and a lot of "markers" to make parsing easier (at the expense of making things harder to write). VID is a great example of a language that works hard to be flexible.
Look at your dialect from the top down too, as a user will. Write example code in it and think about how you expect it to behave.
And remember, LANGUAGE DESIGN IS HARD.
Nah, it took Carl merely twenty years to come up with REBOL
My goal is to create the dialect in a way that it's easy to use for non-techies. Yes, it's hard but hey, otherwise it would be no challange.
Robert: you don't need to have any knowledge of PARSE to design a dialect. The only thing you need is to know how the dialect should look and work.
e.g. I am not implementing all my dialects using PARSE
Robert: Why not settings the words to none! just when you clear the block :
strings: copy  integers: copy  rule: ['keyword (clear strings clear integers size: maxlength: caption: default: none ) some [set p integer! (append integers p) | set p string! (append strings p)] (size: integers/1 maxlength integers/2 caption: strings/1 default: strings/2)]
then, none! is default unless you specify Size and, maybe, Maxlength.
Ups, it's even not needed : if Integers is empty, then "size: empty/1 --> none!"