AltME groups: search

I think that's where string parsing comes in, and where having rules 
for REBOL datatypes would ease the pain.

How about another way: integrate datatypes in string-parser. Basically 
a  load/next and check for type.
Then we can write (note i parse a string): 
parse "1 a , #2" [ integer! word! "," issue! ]

Quick question for the parse experts. If I have a parse rule that 
looks like this:

parse spec [
	any [
		set arg string! (...) | set arg tuple! (...) | ...
	]
]

How would I add a rule like:

	set arg paren! (reduce arg)


that always occurred prior to the any rule and evaluated parenthesized 
expressions (i.e. I want parenthesized expressions to be reduced 
to a REBOL value that can be handled by the remainder of the parse 
rule).

My brain is still asleep.  How to go thru a document and add <strong> 
</strong> around every word that is in capitals and is more than 
a few characters long?

pattern search on capitals, mark, copy to space, mark, count length 
of copy, if long, insert at mark2, and then at mark1, continue ??

that won't work because file is just text and not a block.

search for a series of capitalised words and strong them

Actually I would like to add a parse problem to the weeklyblog and 
get people to submit answers :)

And give a prize for the shortest answer

shortest .. I mean the least number of words, and operators - not 
in length

Seriously though, three charsets and two temporary variables, there's 
got to be a more efficient way.

and "g" is none

because " a: 5 capitals any capitals b:" stops at "g" and friends.

More importantly, it fails at "g" and friends, backtracks and proceeds 
to the next alternate action, some alpha.

No, that would break out of the enclosing all loop. The end skip 
will always fail and proceed to the next alternate.

Of course mine doesn't handle words with apostrophes or hyphens in 
them either. Easy fix though, just add ' and - to the capitals charset.

And do what?

A person is writing a text file.  It has headings which are denoted 
by caps, and terminating in ":".

To use the simpler of the CS terms:


Parse is a rule-based, recursive-descent string and structure parser 
with backtracking. It is not a parser generator (like Lex/Yacc) or 
compiler (like most regex engines) - the engine follows the rules 
directly. Since Parse is recursive-descent it can handle patterns 
that regular expressions wouldn't be able to. Since Parse backtracks 
it can handle patterns that ordinary recursive-descent parsers can't.


Basically, it puts the text and structure processing abilities of 
Perl 5 to shame, let alone those of the lesser regex engines.


In theory, Perl 6 has caught up with REBOL, but Perl 6 only exists 
in theory for now. By the time it becomes actual REBOL should surpass 
it (especially if I have anything to say about it).

It's pretty easy to demonstrate patterns that regular expressions 
can't handle. It's only somewhat difficult to demonstrate patterns 
that can't be handled by a recursive descent parser without backtracking 
or unlimited lookahead.


I have never run into a pattern that can't be handled by Parse in 
theory - its only limits are in implementation (available memory 
and recursion depth). I am not qualified to describe its limits. 
Still, you have to be careful about how you write the rules or they 
will trip you up.

Volker, it's more like it can do what a compiler-compiler can do 
without needing to compile :)

And backtracking is about the same as unlimited lookahead, but more 
powerful.

Thanks Brian, but as is the theme with questions I ask, I don't ask 
for myself, but rather that the "world" can learn what "we" know. 
 So perhaps you should add your 2 cents to Henriks, and Tom's in 
a public forum of the Wikibook.

the compiling is no big argument  as compiler-compilers are for compiled 
languages anyway ;) the point is, you can mix a grammar and actions 
for semantics easy.

Reichart, I figured as much (hence the "dry" comment). I'll look 
over the Wikibook and see if I can help.

i guess that depends on the coco. the point is, a bnf by default, 
and code inside therules, instead of putting things in vars andprocess 
later. IMHO.

Volker, I've used a lot of compiler-compilers before and reviewed 
many more, and unlimited lookup or backtracking are rare.

then the advantages of parse are beeing like a compiler-compiler 
and habving unlimited lookup etc?

and you can use parse to tokenize first?

Two rounds of parsing, one for tokenizing and one to parse? Interesting. 
That would work if you don't have control over the source syntax 
- otherwise load works pretty well for simple languages.

Thats where i got the idea: tokenize first and use block-parser :)

My next personal project is to go through the XML/XSL/REST specs 
and create exactly that. I already have an efficient structure, I 
just need to fill out the semantics to support the complete logical 
model of XML.

Still, "run away" is a common and sensible reaction to XML.

I'm a bit stuck because this parse stop after the first iteration. 
 Can anyone give me a hint as to why it stops after one line.

Here is some code:

data: read to-file Readfile

print length? data
224921


d: parse/all data [thru QuoteStr copy Note to QuoteStr thru QuoteStr 
thru quotestr

    copy Category to QuoteStr thru QuoteStr thru quotestr copy Flag to 
    QuoteStr
    thru newline (print index? data)]
1
== false


Data contains hundreds of "memos" in a csv file with three fields: 

 Memo, Category and Flag ("0"|"1")  all fileds are enclosed in quotes 
 and separated by commas.
  

It would be real simple if the Memo field didn't contain double quoted 
words; then 
parse data none
would even work; but alas many memos contain other "words".
It would even be simple if the memos didn't contain commas, then
parse data "," or parse/all data ","
would work; but alas many memos contain commas in the body.

I'm pretty sure that you are right in that I have to loop throught 
the "Data".  That was my big stumbling block and the rest is just 
logic to figure out.  Thanks a bunch.

Not sure - I remember seeing it in others' parse rules, so I just 
put it there and it worked  '^^
Take it out and see what happens lol

Hi BrianH;

  Yes I did try that and the problem was that even though I specified 
  the "," as the delimiter, it came across an embedded quote #"^"" 
  and split the input at the quote.  Rebol Shouldn't have split it 
  up that way, to my understanding.  I will post some simple data to 
  test.

Tomc:  Do I understand that :word would be like "get word" and needed 
in a parse sentence but you can just use the shortcut 'word' most 
everywhere else?

The colon before the word prevents the interpreter from evaluating 
active values like functions and parens. It's a safety thing.

and that would be get 'word  not get word

Thanks Tomc and BrianH.  I'll chew on it for a while.  Meanwhile 
I'm working on building some test data for the first problem.

That's interesting. Parens and paths used to be active - oh yeah, 
that was changeda while ago. Still, there are some value types that 
are active (function types, lit-path, lit-word) and if you think 
you will get one of these you should disable their evaluation by 
referencing them with a get-word, unless you intend them to be evaluated.

both parens and paths changed between  View 1.2.1 and 1.2.5, actually.

DideC: Thanks.  I've copied and pasted it for review and added it 
to my local public library.  This script should be useful especially 
with the html help page.   Documentation on a script is very rare 
and much appreciated.


Graham: Did a search using  "librarian" and search term of  "sql 
cvs" and didn't come up with anything.  Although, I think we've got 
it covered now anyhow.

What I was trying to do above is to look for the macro text preceded 
by a space or newline, and ending in a space or newline.

and then replace in situ.

at  the macros and expansons single tokens

HTML/XML entities begin with & and end with ; for just this reason. 
What kind of text? can you give us an example?

Heart: Heart regular rate and rhythm, no rubs, murmurs, or gallops 
noted. 

A: Abdomen:  soft, nontender, no mass, no hernia, no guarding, no 
rebound tenderness, no ascites, non obese 
Hbp Hypertension (high blood pressure) #401.9.  
Ii Type II Diabetes #250.00

I would have to intercept the keyboard handler to do this .. so I 
want to try and just do the replacement after he's finished typing.

hmm I am in the bussiness of sharing biological information and I 
got to say  please strongly consider  creating  an ontology if one 
does not exist already

and there's the proprietary MEDCIN

and the macros should  also be part of that  ontology

actually  the file will be saved in a database and loaded when the 
program starts

and the ontologically controlled programs are very expensive due 
to licensing fees

The AMA charge to use their codes, the American College of Pathologists 
charge to use their SMOMED-CT codes .. and so it goes on.

that the macro-expansoion fioe needs to self check for incidental 
occurances of  a "macro" in an "expansion" and protect against

And it won't have the problem you mention Tomc.

so, basically you created a single parse rule from the macro list 
and then parsed the text in one go.

I am glad to see someone else using here and there ;)

Gabriele and I have worked extensively on such submissions.

Yes.  So, somehow I need to force the area field to recognise them 
as newlines and reformat the screen.

how "local" are variables that are set during a parse? I was looking 
at Geomol's postscript.r and looked at:

coords: [err:
	(pos: none) [set pos pair! | set x number! set y number!] (
		either pos [
			append output compose [(pos/x) " " (pos/y) " lineto^/"]
		][
			append output compose [(x) " " (y) " lineto^/"]
		]
	)
]

yes, I try to print the variable and it just returns none.

actually, there is a difference between my code and this, which may 
be causing it:


I need to loop the block with 'any. I suspect the contents is lost 
after the first run.

And to answer your question, the variables are just regular rebol 
words, so they are as local as you make them.

and how looks the code you parse?

if the parse is inside function and you set pos in the function as 
a local - it will be local

where 'image is always first and the remaining items may come in 
random order

there is no pair and no numbers - the pos must be none

and what exactly do you want?

it works the exact opposite :-) Only the outer 'txt is set, and I 
can't reach the variable inside the block

the brackets would make it a "real" rule, wouldn't it? it would be 
possible to replace the rule with a variable and have the rule block 
placed elsewhere in your code

and it makes the parse scalable, so I can add options later

I tried doing myself small template "engine", which will simply look-up 
for some marks, and replace values accordingly. I decided to look 
for the end of the marks and my friend suggested me, that I should 
name even ending marks, to be clear there is not an error. My parse 
looks like this:

I now can create simply a func, which will accept mark name, and 
do some code-block accordingly - sql query, simple replace of value, 
whatever (well, it will not work for cases like img tags, so it is 
not as flexible as full html parser in temple for e.g., but hey, 
it is meant being simple)

... but should not be simpler, so I wonder - so far, as you can see, 
mark-x is not finished, so it is ignored. How to catch this case 
properly and eventually generate error, send email, write to log, 
whatever?

Maarten - now looking into build-markup - sorry, it is just strange 
was of doing things .... noone will place rebol code into template, 
that will not work ... btw - the code is 'done? What happens if someone 
uploads template with its own code? I want presentation and code 
separation.

I looked into rsp some time ago, and I liked it, especially as it 
was complete, with session support etc., but later on I found shlik.org 
being unavailable ...

Hey, locals and arguments (practically the same thing in REBOL) are 
the most important difference between closures and plain blocks. 
The difference is significant but Peters' background with Smalltalk 
made him miss it - Smalltalk "blocks" look like REBOL blocks but 
act like functions.

No, the main point is, easy definitions of code and referencing the 
original context. Rebol-blocks do that.

The highlights he mentions is: lexically-scoped, code and data,  
freely mix computations in

That scoping is the difference between a closure and doing a "string" 
here.

REBOL blocks don't reference a context, but they may contain words 
that reference a context. Still, this distinction makes no difference 
to the argument that Peters was making - REBOL text processing is 
more powerful than regex and easier to use. It would be easier to 
replicate REBOL-style parsing in Python using closures and generators 
anyway (Peters' real subject), since that is the closest Python gets 
to Icon-style backtracking.

I would like the functionality, when parsing things like TeX. There 
the greek letter gamma is called gamma, and the same in capital is 
called Gamma. Now I have to invent the word capgamma or something.

If it were a safe and easy thing to change, I can see some value 
in it as an option but, since words--and REBOL--are case insensitive, 
I'm inclined to live with things as they are, and use string parsing 
if case sensitivity is needed. I think it's Oldes or Rebolek that 
sometimes requests the ability to parse non-loadable strings, using 
percentage values as an example. I think loading percentages would 
be awesome, but then there are other values we might want to load 
as well; where do you draw the line? I'm waiting to see what R3 holds 
with custom datatypes and such.

And didn't you suggest that values throwing errors could be coerced 
to string! or another type? e.g. add an /any refinement to load, 
and any value in the string that can't be loaded would become a string 
(or maybe you could say you want them to be tags for easy identification).

I think, load/next can be used to handle invalid datatypes now:
>> b: {1 2 3 'x' ,}
== "1 2 3 'x' ,"
>> while [v: load/next b not empty? second v][probe v b: v/2]
[1 " 2 3 'x' ,"]
[2 " 3 'x' ,"]
[3 " 'x' ,"]
['x' " ,"]
** Syntax Error: Invalid word -- ,
** Near: (line 1) ,

Just add some hadler to convert the invalid datatype to something 
else what is loadable and then parse as a block

Gabriele, yes it works with strings. But I have words! Thing is, 
I parse the string input from the user and produce words in an internal 
format. Then I parse those words for the final output, which can 
be different formats. I would expect parse/case to be case-sensitive, 
when parsing words, but parse/case is only for strings, therefore 
my suggestion.

And there is some parse example how to deal with recursions while 
parsing strings? If you parse block, it's easy detect, what is string! 
and what is other type, but if you need to parse string, it's not 
so easy to detect for example strings like {some text {other "text"}}

Here's an idea to toss into the mix:

I am thinking of a new notation for strings using underscore (eg. 
 _"hello"_  ) in a parse block, which allows to specify whether they 
are delimited by whitespace or not. This would allow you to enable/disable 
the necessity for delimiters per-string. eg:

parse input [

 _"house"_   ; a complete word surrounded both sides by whitespace

 _"hous"   ;  this would match "house", "housing", "housed" or even 
 "housopoly" etc.. but left side must be whitespace

 "ad"_ ; this would match "ad", "fad", "glad" and right side must 
 be whitespace
]

But this would need string datatype to change.

On the other hand, I could just set underscore _ to a charset of 
whitespace, then use that with parse/all eg:

	_: charset " ^-^/"

parse/all input [
	[ _ "house" _ ]
]


though that wouldn't be as comfortable. Maybe I can create parse 
rules from a simpler dialect which understands the underscore _.
Just an idea...

Anton, Andrew had defined white space patterns in his patterns.r 
script which seems usable then you can use [ ws* "house" ws*] or 
other combinations as needed without underscore.  Andrew's solution 
for this and a lot of other things have given me some good mileage 
over the past few years.   WS*: [some WS]   and WS?: [any WS].   
 It makes for clean parse scripts clear once you adopt it.

I think either approach above can work well. I like the "look" of 
the underscore, and have done similar things with standard function 
names. For SOME, ANY, and OPT, the tag chars I prefer are +, *, and 
? resepctively; which are EBNF standard.

Oh yes, I've seen Andrew's patterns.r. I was just musing how to make 
it more concise without even using a short word like WS.  Actually 
the use case which sparked this idea was more of a "regex-level" 
pattern matcher, just a simple pattern matcher where the user writes 
the pattern to match filenames and to match strings appearing in 
file contents.

I also have a naming convention I've been playing with for a while, 
where parse rule words have an "=" at the end (e.g. date=) and parse 
variables--values set during the parse process--have it at the beginning 
(e.g. =date). The idea is that it's sort of a cross between BNF syntax 
for production rules and set-word/get-word syntax; the goal being 
to easily distinguish parse-related words. By using the same word 
for a rule and an associated variable, with the equal sign at the 
head or tail, respectively, it also makes it easier to keep track 
of what gets set where, when you have a lot of rules.

simple and clean, good idea!

so many years of reboling (since core 1.2) , and still parse remains 
largely untaimed by myself.

That's a ~very~ good example, Oldes... it should be put in the docs 
somewhere (if it isn't already.)  I didn't understand how get-words 
and set-words worked in parse, either, before..

I've got following PARSE problem:


I've got string - "<good tag><bad tag><other tag><good tag>" and 
I want to keep "good tag" and "<>" in other tags change to let's 
say "X" (I need to change it to HTML entities but that doesn't matter 
now). So result will look like: "<good tag>Xbad tagXXother tagX<good 
tag>"


I'm working on it for last few hours but still not found sollution. 
Is there any?

I'll probable replace everything and then just revert the "good tag" 
back. It's not very elegant, but...

<, and >  ?