World: r3wp

again, the reason is that it is impossible to come up with a general 
solution, and that this is a UI issue and NOT a language issue. unless 
you throw in AI, "language" for a computer cannot be the same as 
"language" for a human.

so, either we have AI, and the computer can figure out what a human 
means when reading the text the human wrote (but it'll have to ask 
questions most of the times! like when a human is talking to another 
human), or it can only be a specilized thing (eg. a date field that 
tries to be "smart" - but that's just a big set of rules)

I think that your answer is too abstract :-) All that guys wanted 
was - any known rebol datatype should be recognised, and the rest 
should be assigned under junk! or similiarly named datatype. What 
is AI about that? :-)

load-all: func [
	s [string!]
	/with limit [string!]
] [
	s: either with [
		parse/all  s limit
	] [
		parse s none
	]
	forall s [
		s/1: any [
			attempt [load s/1]
		] [
			s/1
		]
	]
	s
]


But you have to decide, what you wish as a delimiter or as part of 
item.

there was a bug
load-all: func [
	s [string!]
	/with limit [string!]
] [
	s: either with [
		parse/all  s limit
	] [
		parse s none
	]
	forall s [
		s/1: any [
			attempt [load s/1]
			s/1
		]
	]
	s
]

Gabriele; no sorry; I guess I obfuscated my point too much with the 
money! example.  I don't want REBOL to figure out all the possible 
human combinations of money.  I want REBOL to load a value that we 
humans commonly use as a format of money, without a syntax error. 
  The make phase would create a foreign data value and keep going. 
  Let the gurus that write general applications worry about the math 
operations on junk and account for it.  But let a professor load 
his book and use the uber powerful series operations of REBOL for 
ad hoc analysis.   No AI involved here, just a feature of the language 
that "if a piece of source does not match one of the lexically strict 
57 datatypes, it floats in a REBOL block as foreign".    And foreign 
is similar but not quite exactly the same as quoting the text as 
string!, without need of quoting the text.   I see great power in 
this feature.


Excuse the poor example;   I misdirected my intention when I mentioned 
that arg1 had a good hint, my mistake (but I would if I had brains, 
include this hint as a subfield of foreign! data, just because, why 
not, the current lexical parser already made a guess).  Our good 
Mr Hawley has a plan for a LOAD refinement /else with a type spec 
that looks very promising, but then it leads to being able to distinguish 
actual quoted input from coerced input, so I still lean toward foreign.


My plan (and I place burden of foreign! management for CODE blocks 
to the gurus) would let normal people load any data and run with 
it.  Even it they only use 4 or 5 REBOL functions at the console, 
the potential userbase becomes orders of magnitude greater than what 
we have today.   That it turn opens up a few that will write concise 
clear time lasting code that could analyze books or the sum of human 
knowledge using our favourite thinking environment.  No?

for me the key point lies not in the fact that we can already make 
mezz func which simulates the foreign! handling with a function such 
as ASSIMILATE.


the difference lies in the fact that if the native function can load 
invalid data, then it should simply because of the the fantastical 
speed its able to convert string data into rebol literal values of 
a variety of types.


hooking up interpreter driven code within the handling of that will 
slow it down.  using the parse with next refinement works... but 
its nowhere near as fast, for loading, say 300MB of scientific data. 
 which is an actual case I had to deal with.  just doing a replace/all 
on that file took 30Minutes.  assimilate with foreign! handling  
would have taken about 5-10 seconds, and the code would have also 
been easier to write over all.

Brian and Pekr, you can easily define where your foreign! value (garbage) 
would start, but you cannot define in a *general* way where it should 
end. You can only define that on specific cases and with specific 
rules. That's Gabriele's point. 


For example, let's see how an invalid input could be parsed using 
a theoretical foreign! type (represented below as quoted string for 
easier reading).

Test input string to parse: "123 abc 4c5.6"

Possible alternative outputs from a relaxed LOAD:

[123 abc "4c5.6"]
[123 abc 4 "c5.6"]
[123 abc 4 c5.6]
[123 abc 4 c 5.6]
[123 abc "4c" 5.6]
...

Question: which one is the correct output that LOAD should produce?

Answer: you can't know until you ask me what I wanted to express.


You cannot expect from LOAD to know how to handle invalid data (and 
to determine where it ends), the best thing LOAD can do is to stop 
on syntax error. If you have hints or rules on how should foreign 
be parsed, then write some parse rules and use string! parsing.


LOAD/else is a nice REP, but again it can only work on specific data. 
You cannot feed it with *any* data and expect it to work.

usuall rebol separators is the ovbious reply... spaces.  we all agree 
that we don't expect the system to behave magically.

for my part I just know how many times, this would have simplified 
parsing of mechanically generated data.

oh my... it could be simple, just load as junk! until you found a 
valid separator (blank, newline or tab) then continue to load.
What's the problem ????

Those who are saying endless that it's better to throw an error have 
never been in that case...

And if it can not be included in core as a new datatype! (constructed 
by to block!) then it's useless. I don't want it as a mezzanine because 
it would be slow.

at least junked tokens could be formated as a string! it would be 
more usefull than thorwing an error and stoping

to block! could return a false status if there are junks but we don't 
want useless errors anymore...

Doc, the point to LOAD/else is that it requires a parameter that 
is the answer to the question of what the calling code wants to happen 
at that point. This solves your concern, and is the major difference 
relative to all of the foreign! proposals.,

I don't know whether /else is the right name for that option, but 
at least it's short.

Load/foreign ?

More like load/on-error.

It would require you to provide a handler, or perhaps a word that 
denotes a stock handling strategy.

Like load/on-error "10,000,000.00" 'string.

Or 'skip.

by defaut it  should be a string conversion with a charset or a string 
with delimiters

By default, it will be an error. The default will be when the /on-error 
option is not used.

Otherwise you will be *required* to specify a strategy for dealing 
with the data, perhaps as a function or code block if you like.

you know what i mean, i'm talking about a default conversion when 
the option is selected

There will be no default - the parameter will be required.

It's either specify the option or don't.

like for parse.
default use: parse data none
with specific separators: parse date "..."

You are passing a parameter there - it's not the same thing. If we 
allow a none parameter (which we shouldn't), it should mean 'skip.

However, there should be a 'local parameter for localized numbers 
and dates.

we don't need to skeep, the only skiped chararcters are the separators

*skip

Yeah, you should definitely skip separators. Use load/on-error data 
'skip if you are expecting separators, as REBOL doesn't have them.

Except for whitespace, of course.

see the sqlab code, it's exactly how it should behave

his load-all, we don't need more fonctionnalities

No, that won't work. It will break on nested structures.

The whole point oif LOAD is handling the whole data syntax of REBOL.

oif -> of

i  talking about the functionnalities not about his implementation

So am I. That functionality won't work, for reasons Doc most recently 
explained.

we just need a list of separators when a wrong loadable value is 
encountered to continue the loading

reasons Doc exposed are false

We have a list of separators: Whitespace. Anything else requires 
special handling that will need to be specified. The only problem 
with Doc's comment is not getting that part of the LOAD/else proposal.

anyway if your objective is to build a mezzanine it's useless, it 
will be slow, i will continue with my own pre-parsing process

LOAD is a mezzanine, but it calls natives. My goal is to make LOAD 
useful.

Please contine to do your preparsing. I was doing this on btiffin's 
suggestion, and he seems to be afraid of PARSE :)

Umm, yes and no on the fear.  Yes fear kept me from holding a lecture 
on the subject, but I usually PARSE from reading the pretty print 
code.  That's how my locate utility works.  But I'm not concerned 
with us.  I'm concerned with construction bosses and non-tech professors 
having access to a programming language and learning maybe one or 
two tricks a week.  I'm also on the side of the gurus in terms of 
correctness and concise coding, but I'd like to see REBOL ,the system, 
that out of the box would be a robust battle tank.


Add taint to the fuel, it would still function; perhaps not gracefully, 
but the big guns would still fire.  Today, the slighest spoonful 
of sugar and our tank dies on the field, no movement, no guns.


foreign! and Steeve's suggestion of scan till whitespace (and yes, 
some source code would load as almost all completely foreign! gibberish 
if a quote was out of place, but so, we can take that and fix it). 
 But at least REBOL wouldn't die; the data/code would be loaded and 
inspectable.  And yes, this could lead to the odd rare catastrophic 
failure, but we get that potential with "clean" datatype! scripts 
too.  I think the slight increased risk is worth the new group of 
users this could attract.

Sorry if I misrepresented your position Brian, I was trying to be 
humorous. I've just put in a request to tweak TRANSCODE's handling 
of commas. If that request goes through, I will be able to add the 
/else option to LOAD. With a few more tweaks I can increase the speed, 
make more of the process native, etc. This is not preventing REBOL 
from dying - it is just allowing you to provide a defibrillator :)

In case you wondered about the origin of CTRL-ALT-DELETE - this is 
actually kinda funny http://www.flixxy.com/computer-history-ctrl-alt-del.htm