World: r3wp

if you don't know the final size... it's usually better to let rebol 
do its magic. but, i'd suggest timing the code and deciding which 
is really going to perform better, as there are many variables involved.

gabriele, doing "magic", isn't that degrading performance in the 
long run, if he has to perform the operation thousands of times?

Thanks, I think with the speed that's been mentioned compared to 
my old recursions in c# I'm worrying about nothing.

Biggest customer i've got has 250,000 x 20 rows x 15 tables

Gabriele, what I'm mostly concerned about is how memory is eaten 
up over time, because GC isn't done well enough. Do you know anything 
about what will be done here in R3? Will the GC be documented?

steveT, if you have the memory to fit that amount of data, then REBOL 
will process it without problems.

cool, I think I used to suffer a little because if your try to open 
a recordSet in visual studio it creates a natural bottle-neck because 
some idiots might have bound controls - and it has to handle that. 
I had to do all my processing of data on a 'SQL disconnected basis' 
 know what I mean by that?

I think from what everyone says once you get data into REBOL it's 
wonderful. I just wanted some options for pulling the data in.

for those cases:  Learn PARSE. I haven't fully learned it yet, but 
the things it can do is just.. wow :-)

I will. Will there be something standard in R3 to say pull a csv 
file in to blocks?

PARSE in R2 already does that to some extent.

an example of that would be much appreciated ? I need to print this 
thread out and digest what everyones suggested. Thanks so much for 
all the help.

I don't think much time will be spent on building special import 
functions unless its for media formats, which would really require 
performance.

read/lines is a start to turning CSV into blocks -- you get one block 
entry per record.

right , thanks I'll have a go with that. I'll save all these snipets 
and perhaps we could build a database how-to to put on the new users 
docs.

about PARSE: http://en.wikibooks.org/wiki/REBOL_Programming/Language_Features/Parse

thanks for that Henrik I'll have a look! Sorry i need to sign-off 
for a while (wifes picking up new car) please carry the thread on 
without me I'll print it later - thanks one again.

I found Brett Handley's parse tutorial a big help: http://www.codeconscious.com/rebol/parse-tutorial.html

Thanks Peter

I too found Brett's parse tutorial very helpful.

Hi Henrik, I read your blog about Natives vs. Actions vs. Mezzanines, 
thanks it explains the speed differences.

I regularly import CSV files into REBOL.

The only trick is that I insist they be tab-delimited rather than 
comma-delimited, and that none of the strings contain tabs. That 
way, we can be sure that tab occurs only as a field delimiter.
The code then is
   data-block: read /lines %csv-data-file.txt
   foreach record data-block [
        record: parse/all record to-string tab
        ....code to handle the 'record block
    ]


If you need to handle actual comma separated where strings can also 
contain commas, you'll need to tweak the parse.

Thanks so much Sunanda, a couple of my customers receive stock update 
files as csv's (I don't think we could get them to send tsv's ) but 
that will fill a requirement brilliantly. If I decide to move any 
more complex apps to Rebol I may need to wait till I can get R3/command, 
I believe that will give me odbc (for excel and access) imports etc.

I've learnt an awful lot over the past week! But I think one of the 
most profound lessons I've learnt,

I quote from one of the WIki guides on the language


 This is perhaps the most important single piece of advice when learning 
 REBOL for the first time: 

    forget the past. Forget what you 
 already know about programming.

    REBOL is different. If you know 
 other programming languages such as C or BASIC, and you expect to 
 program REBOL the same way, 

    then what you will get as a result 
 is C or BASIC. Nothing better. We've seen this happen many times. 
 courtesy of Wikibooks




Every function or data routine I've written this week, I have stepped 
back from and thought "that's the c# way of doing it!"


Rebol opens up wonderful ways of approaching software tasks - I think 
I'm just entering my second week of de-programming.

Just posted this to my blog http://swt1962.spaces.live.com;-)

BTW SteveT, the CLR doesn't interpret its bytecodes, ever - it compiles 
them to native code and then executes the native code. Interpretation 
isn't what makes .NET and C# slow.

Carl once told me that he has found the best REBOL programmers to 
be former assembly language programmers.

Henrik: if you know the size, preallocate. if you don't, in my experience 
it's preallocation that degrades performance. but... don't listen 
to me, just time your code. :)

Sunanda: parse works with comma delimited if there is a comma in 
the fields, as long as the fields are delimited by " when this happens.

>> parse/all {one,two,"three with a , inside",four} ","
== ["one" "two" "three with a , inside" "four"]

Thanks Gabriele -- that may work for SteveT.

However, there are some anomalies in the way parse handles RFC4180 
formatted CSV files. Particularly if there are embedded double quotes 
in a field.


For my purposes, I find tab delimited files are better -- though 
still not always perfect:

    >> x: parse/all {"a"^- ","","} to-string tab

    == ["a" { ","","}]   ;; 2nd field should be {,",}  -- result is close 
    enough that it can be tidied into that
 
   >> y: parse/all {"a", ","","} ","
    == ["a" { "} "" ""]    ;; 2nd field mashed irrecoverable!

the best solution is always to create your own rule, that fits your 
source files perfectly. the built in one helps in simple cases, but 
gets in the way more often than not (imho). it's handy for new users 
though, because they can parse cvs right away, and worry about the 
details only later on when they've learnt parse better :)

Hi Gabriele, that's a good point, I've read the above articles about 
parse and there's an awful lot to learn to use it effectivly in REBOL.

Hi Henrik, yes I know it can be done and you can create your own 
styles or extensions, but new 'Winows' programmers will be very put 
off by that. they shouldn't have to override the VID to do expected 
things liek I mentioned.

exactly. this problem will go away in VID3 for R3. VID is very sparse 
with features like this and you've hit that wall now, a wall I've 
been climbing for years by doing things like LIST-VIEW and vid-field.r. 
:-)

;-)

a problem is that a lot of these internals are not documented and 
so they are very hard to figure out. I think it's safe to say this 
is the biggest sore point about REBOL, one we're anxious to not only 
fix but to turn completely around on and become a leader with.

I know some of my old colleagues would have a tick list before they 
would attempt any language/system and i think VID fails badly there. 
I can cope with it cos I'm so into Rebol, but I know others would 
find say Python with vxWidgets would tick more of their boxes!  Sorry 
for swearing (Python) ;-)

your points are perfectly valid :-)

BTW Henrik - when do you sleep ?

i thought I was bad!

I'm usually a night owl. I went to bed at 3 am and I got up at 11 
am.

which is almost an hour ago here

You in Denmark ??

yes

Are a lot of the other guys in the US? It's interesting to see where 
were all spread - hink Ashley is anitpadean Tes?

I think we're mostly US and European, but we have a few easterners 
as well.

Cool, is Ashley re-working rebGUI for VID 3 or will it be redundant. 
I've never bean totally happy with a third party widgets (like Python 
+ vxWidgets) or Java + Mattisse etc

The accessors to the widgets need to be developed by the language 
developers normally IMO

I had a go with Ruby and vxWidgerts - it really put me off