World: r3wp

It gets easier in R3, where the behavior of some standard functions 
has been changed to be more in keeping with the none-means-missing-data 
principle. For now you have pick, find, select, and all of the conditionals.

In R3 you add the ordinals (first, second, ...) and the map! type 
to that collection.

Cool , in SQL et al , null can be useful for finding where say a 
JOIN has no corresponding RIGHT side.

Hi all, is there an EASY way of changing this :-

	[ fld1 fld2 fld3 fld4
	  fld1 fld2 fld3 fld4 ]
into this

	[ [fld1 fld2 fld3 fld4]
	  [ fld1 fld2 fld3 fld4 ] ]

Thanks
Steve

If you are happy to copy the data to a new structure....

    newblock: copy []

    foreach [a b c d][ fld1 fld2 fld3 fld4 fld1 fld2 fld3 fld4 ] [
      append/only newblock reduce [a b c d]
      ]

Thanks Sunanda, it's cos some of the data (lists) I pick up may be 
in form one or form two. But either may be quite large lists - obiously 
if it's in memory doubling it may be a problem!

if you are willing to, it may be possible to manipulate the block 
itself, but I think it's slower. otherwise make sure you allocate 
the necessary space for newblock. that makes garbage collection easier 
for REBOL.

newblock: make block! divide length? oldblock 4

is that faster than Sunandas foreach?

4 being the number of elements in each row ( So that must be known!) 
If you don't know how many elements (columns) could that be achieved 
using the newl-line?  ?

sorry new-line?

it's a supplement to sunanda's example. if you use newblock: copy 
[] you are correctly creating a new block, but every time you then 
append to that block, REBOL has to spend a little time allocating 
new space for the block as it grows. if you preallocate space with 
say:

newblock: make block! 10000


REBOL won't have to do that. This means that when REBOL must garbage 
collect or when you use 'recycle, it knows that there's one big block 
there that it can just remove. Easier for REBOL. Having more blocks 
inside that block makes the case a bit more complex, but since each 
block inside is only created and manipulated once, there may not 
be an issue.

my way was just a simple way of allocating exactly the space you 
need. if you can get the length of the input block and the number 
of items you want to split it in, then you can correctly determine 
the necessary length of the output block.

How fast is REBOL in processing this (I know it's processor/memory 
dependent) but could you manage say 100,000 rows ?

in REBOL it's generally a good idea to reuse blocks, if you are doing 
buffers. make your buffer global:

buffer: make binary! 4096  ; 4 kb buffer

loop [
  insert buffer *some stuff that needs to be buffered*

  *use buffer for something*

  clear buffer
]

100000 is easy

but... it's a good idea to study some things here about which loop 
functions are faster than others.

great - I don't think anything I would pull thru would contain more 
than 100,000  rows x 15 cols

Thanks Henrik.....For large blocks, you can help REBOL's memory managemeny 
by pre-allocating the needed space. So, applying Henrik's optimisation:

    oldblock: copy [ fld1 fld2 fld3 fld4 fld1 fld2 fld3 fld4 ]
    newblock: make block! (length? oldblock) / 4

    foreach [a b c d] oldblock [
       append/only newblock reduce [a b c d]
      ]

some functions can give simple code, but are slower, because they 
are mezzanines based on other loops.

in sunanda's example, APPEND is actually a mezzanine. if you wanted 
a tiny speed up, INSERT/ONLY TAIL is a little faster than APPEND/ONLY

You could do an easy timing test by creating some test data:
    loop 1000000 [append oldblock random/secure "abcdefghijklm"]

....My computer took about a second to reformat the test data. Fast 
enough?

Brilliant, thanks Sunanda. you guys answer questiions faster than 
I can ask them ;-)

you can explore this yourself by studying the source for various 
loop functions.


 http://www.hmkdesign.dk/rebol/files/385cc7420409fa08f768a537a52ce4d8-76.html

That blog post might be of help.

be aware that pretty code is not always fast, even in REBOL. :-)

Thanks Henrik for the append vs insert optimization too!

Thanks Henrik, I was a bit confused by looking thru the core and 
dictionary functions - the examples don't really show advantages/disadvantages 
for real world apps

one thing that amazes me is that PARSE is very fast, despite being 
one of the most complicated functions to understand in REBOL. if 
you learn how to use it, it can perform some quite amazing tasks.

and once again, we'll be turning this upside down for REBOL3. :-) 
there are a lot of new functions to aid in the creation and manipulation 
of blocks with simple code that also performs very well.

Yes I've only used it for my licence creator applet - I't's one of 
the core items I need to master. At the moment I'm trying to get 
blocks/series to fit into recursions I have created in legacy apps

I don't mind re-learning these for R3, I thnk I need to learn R2 
to master R3

sorry this wireless keyboard has a stutter ;-)

So the immediate thing I've picked up is as you use append Rebol 
has to re-form the block on each iteration! Is that right?

generally about speed: I rarely worry about REBOL/Core speed. I made 
a database once that was mainly about in memory block manipulation 
in REBOL. On an old 500 Mhz Celeron laptop, I couldn't saturate it, 
when throwing queries at it from 3 fast PCs over a 100 MBit LAN. 
I think it was about 100-1000 times faster than a MySQL database, 
but of course with the price that it's in-memory and that it had 
very few features.

be careful about wording, as 'reform is a function in REBOL. :-) 
all it has to do, is allocate more space. Just imagine a cramped 
desk and you want to put some things on the desk. Which is faster? 
Is it to remove one item at a time from the desk to place a new one 
there, or just swipe the desk clean in one go in order to free all 
desk space immediately?

The processing seem's amazing compared to say one of my c# apps.

yes, and they say interpreted languages are slow, right? :-)

Yep sorry Henrik, I meant allocate more space

Well really VB and C# are interpreted as well

the code for a c# app is compiled into an intermediary psuedo code 
then interpreted by the CLR (Common language runtime)

generally you just want to reduce the cases where you need to allocate 
space. if you reduce the number of cases and that reduces the number 
of garbage collections needed and there is a greater guarantee that 
all the allocated space will be freed up. REBOL is good, but the 
garbage collector isn't super intelligent so it helps to simplify 
its work.

I was going to ask how efficient the GC was in Rebol - it's one thing 
that is good in c'# compared to c++

how it works exactly is unfortunately undocumented. Only Carl knows, 
so everything we know is based on observations and tests.

when your re-painting screens and such you have to force GC or the 
display struggles

View is a bit different, because it hogs memory quite badly. You 
can do a few limited things there to speed up display, but not much 
in terms of helping GC.

Video processing on R3 looks amazing

the difference in performance on R2 vs. R3 in graphics is also quite 
amazing, mostly thanks to much reduced memory requirements to create 
and manage graphics.

Using the alpha channel allows for some great transparencies and 
other effects

btw, about your append question... no, append does not need to allocate 
memory at each call. each series has some space free, and when that's 
up, a bigger space is allocated, usually with 2x increments (until 
a certain size, then it's linear). however, the whole series needs 
to be copied when that happens. so, if you append 1000 times, it 
may happen just 2 or 3 times, but still, if you know the final size 
already, you can save a lot by just preallocating it.