AltME groups: search

Yes, should go into AltMe. And much, much faster display of large 
groups, without message limit. Virtual tables is the trick...

hash tables for such a big set are the only way to go... they will 
be magnitudes faster on access.

but... filled up....

b: make hash! 5000010
m: stats

loop 5000000 [append b copy random "1234567890" append b random 10000000]
print stats - m
== 188430448

here its half the space.


a ha!  depending on the string input... hash tables can actually 
be smaller...   :-)

append works on hash tables.  in fact they are exactly the same as 
if you where using blocks, except that the internal representation 
is different than what you look at through code.

but janko... if you test it, you will that hash tables are   extremely 
faster at retrieving data...  the larger the set the bigger the difference. 
 

on millions of records indexed with strings , it could be hundreds 
or thousands of times faster  :-)

Transfering memory based hash! (map! in R3) datatype into disk based 
shema automatically keeping the hash table computation and lookup 
hidden from user gives you a RIF. Holly grail of all rebollers :) 
long long time promissed, still waiting to be done. Anyway hash tables 
are always usually unsorted, when necessary to search in usually 
some type of additional index is used (B-tree for example), for simple 
information if the key is in the set, bitmap vectors are used with 
advantage, when the set is really big (and bitmap vector doesn fit 
into memory) comressed bitmap may be used and usually bitwise operations 
on those vectors are much quicker than on uncompressed. 

Thisi is why it should be used for bitset! datatype anyway. The number 
of byte aligned (BBC,Packbit,RLE)od word aligned (WAH) schemes exists. 
 It is used in very large datasets when index also resides in disk 
file. Once again bitwise operation may be much quickier even in memory 
on those schemes.

Never mind. Using some tables here.

Brian,  parse uses tables (at least last time I checked on compiler 
construction ;-) so it shoud be a easy to do.


If I have at/deep or so, I can creat a new head and build a stack 
of traversed parse rules. Actually that would represent the table. 
Which means I could do this today. And I can wite at/deep by using 
some recursion.

Thanks!

here is an example of how we could use the unit system if it where 
implemented.


ANY unit moniker could be used, and in fact the switch tables below, 
could be context-specific and dialect-configured, adding to the DSL 
power of  REBOL... 


ex: dm could be decimeter or deutchmark (the later should be DM anyways, 
no?) , based on context, the convertion units could mean either or... 
with a default convertion table explicitely defined.

a simple func could let us append or change the conversion tables 
used throughout the system.  


Imagine if the VID would use units directly.  you set your locale 
(or get it from OS) and see values as they should be for your locality.


unit-convert: func [
	in "unit value to convert"
	to-moniker "to what unit type to you want to convert to"
][
	; note: incompatible types set 'IN to value none
	in/value: switch to-moniker any [
		switch in/moniker [
			{mm} [
				[
					{m} [in / 1000]
					{dm} [in / 100]
					{cm} [ in / 10]
					{'} [in * 0.0032808399]
					{"} [in  * 0.0393700787]
				]
			]
			{"} [
				[
					{m} [ in * 0.0254]
				]
			]
		]
		; empty result set, switch on nothing
		[]
	]
	in/moniker: to-moniker
]


unit-convert 100mm "m"
== 0.1m

unit-convert 100" "m"
== 2.54m

unit-convert 2mm  {'}
== 0.0787401575'

but i read in rebol.com that a gui for rebol chat will be proposed 
!!! normaly in february 2009 "Resume work on the R3 GUI. The primary 
focus needs to be on styles for lists and tables." (carryover)."

This could be an advantage - there are many languages that support 
capitalization as a concept, but many that don't. The ones that don't 
have more characters than the ones that do (I'm guessing). This means 
that we could use smaller tables/code to do the capitalization in 
LOWERCASE and UPPERCASE - valuable space savings for a tiny language 
like REBOL.

its like programming cells directly. no rows, no columns, no tables.

I build my datasets as named associations rather than relations. 
  there is no relation in the data tables.  a completely generic 
and the most powerfull data model.

actually it's more high level. you get loops and mapping of blocks 
and objects to tables, etc.

A html dialect that builds tables and such would be used outside 
of the template.. which only does a simple substitution

The point would be, were tables done correctly in RebGUI, that the 
pretty print formatting would come at cell rendering time rather 
than as input to the table. In the work I've been doing, Cyphre changed 
table for me so that it would allow sorting on strings that contain 
numbers.

james, I have no such issues, but I guess it depends on how intensely 
the database is used. maybe write a separate test script that performs 
db access and reads some curecode tables into the browser. if it 
doesn't work, when curecode fails, then the db connection must be 
bad.

Guys, thanks for your input. I did log off, thinking that perhaps 
not doing so might be causing the issue. This morning, the same "page 
unavailable" occured. I checked the log and there is an RSP script 
 error in the head.rsp file:

 ** Script Error : say expected data argument of type: string none 
	** Where: rsp-script 
	** Near:  [projects/2: say projects/2 

Then a separate entry 

Request = make object! [ ...
 referring to the index.rsp file.


I can in fact run my test page which has a mysql test  that reads 
the curecode tables within it without any issues.


I have that work-around of restarting the service so I'm cool. I 
was just wondering if anyone else had that same behavior.

I'm also going to test it from another machine today.


Interesting. Reaching the server from another machine worked. Then 
when I went back to the server machine and tried curecode, it also 
worked.  

I'll do some more tests and let you know. Thanks.

Thanks Doc.    P.S. I re-ran an install of CureCode today and it 
is failing on the sql file (build-db.sql)  used to define the data 
base and tables.  I found it and moved it but now I am failing on 
webapp setup.

http://curecode.org/dl/upgrade-096-0910.zip


I am not sure it applies to 0.9.8, you'll need to check if the new 
SQL  tables are already there or not.

The upgrading script should have created all required tables.

it did not. there was a large number of errors, since there were 
no checks if the tables were actually existing or not and I couldn't 
tell whether the upgrade script actually finished.

As I said, the script is for v0.9.6, for your v0.9.8, you should 
have checked manually which tables are already created. You should 
then comment the lines in the upgrade script to match your configuration.

I have used such an approach in one of our apps.For example I have 
three tables, that depending on the app state, show different columns 
and data. So, when app state switches this will trigger the change 
of the tables.


And other example is changing the language just where you are. No 
need to re-enter the current GUI etc. There will be just a change-lang 
XY event send.

Robert and I are discussing field persistence, i.e. tieing fields 
directly to database tables in a layout. Going to be a bit about 
our conclusions in the R3 GUI specs soon.

The question is: How to get from GUI to a DB and back without cluttering 
the VID code, or having to code to much. The idea is to collect the 
GUI elements belonging to one record and than auto-create tables 
and columns. So, people can concentrate on the app code and get the 
75% always necessary database code for free.

Henrik i see, but there should be the less possible specifications 
of the DB inside the styles themself.


Each panel should only (see) a flat VIEW of its tied tables (more 
easy to manage). 

Then each VIEW (like in SQL) could have a shema specification (somewhere 
else in the code) which desrcibes all the referential constraints 
between tables of the DB.

So far there is no high level widet like flexible tables  treeview 
etc...

Screenshots will be available when there is something to show. Worked 
out a few bugs regarding tab navigation yesterday and there is work 
being done on the basis for a grid system that will be used for tables, 
dropdowns, menus, etc. Someone has been added to the list of developers, 
as I don't have much time right now to do GUI work. He may speak 
up if he wants to. :-)

persistent: If you want to store a form you need to get the relevant 
information from your styles. For example: which ID was selected 
in a drop-down box, sort-order of tables etc.

Yes, I think tables are key here (tbl did spanning long before HTML 
I believe :-). Do H and V panels help that much, e.g. to reduce clutter? 
I imagine the team talked about that, and whether just a TABLE would 
be enough.

I would like that the finished doc style supports images, tables 
and enough features to allow direct rendering of MakeDoc documents. 
This should simplify creating a documentation system, where we don't 
need to rely on browsers, when using browser-less platforms.

Bolek is working full time on styles, particularly tables.

The plan is to do styles on priority with needs with the SCRUM tool, 
that's why tables are being worked on right now. Also continual design 
reviews are necessary to make sure we won't run into design dead-ends. 
That's why there may be regressions.

tables - nice, and good to know ...

Ultimately a scroller and an item to scroll has many subtleties that 
you don't notice at first, such as step size, whether you want smooth 
or non-smooth scrolling, and the structure of the item to scroll, 
and whether you want separate behaviors for vertical and horizontal 
scrolling. Then there is also placement of the scroller. Do you want 
the scroller to automatically "sense" what direction it has to scroll 
in?


All this means that the scroller should treat each such case as a 
special case and you can't ask a style developer to meddle with attachment 
code inside the scroller style to deal with this issue. It's better 
and simpler to have a "dumb" scroller that will do your bidding for 
your style.


It "works" in Carl's demo, because he only has 1 or 2 cases to work 
with, but it doesn't really work that well. He never implemented 
tables, image-pans, icon lists, chat lists, maps, browser windows, 
etc.

And now also - back to point 5, away from politics :-)


- New resizing model. Will API change too? Or is is just internal 
change, so I don't need to care about it, apart from knowing, that 
in some cases, resizing model will be more efficient?


- Is RMA building any commercial app using R3 GUI right now? Because 
I still might miss something, but style-wise I find it difficult 
to imagine, how it could be used. (Tables, lists, tree, area, tabs 
missing or buggy?)

rebol[
	file: %carte-db-joueur.r
]


db-joueur: make object! [
	table: 't_joueurs
	oo: make joueur []
	
	
	list: func[
		o[object!]
		/local
		fields-in-file
		where
		test-nom
		test-prenom
	][
		fields-in-file: select my-db/tables :table
		where: copy []

  if all[(not o/id = 0) (not none? o/id) (not empty? o/id)]  [append 
  where compose [id = (o/id)]]

  if all[(not none? o/nom) (not empty? o/nom)] [append where compose 
  [find nom (o/nom)]]

  if all[(not none? o/prenom) (not empty? o/prenom)] [append where 
  compose [find prenom (o/prenom)]]

   return self/find-where fields-in-file compose/deep [all [(where)]]
	]
	
	find-obj: func[
		o[object!]
		/local
		fields-in-file
		where
	][
		fields-in-file: select my-db/tables :table
		obj: none
		where: copy []
		
		foreach field fields-in-file [
			if not none? o/:field [
				append where compose/deep [(field) = (o/:field)]
			]
		]

  return self/find-where fields-in-file compose/deep [all [(where)]]
	]
	
	find-where: func [
		fields-in-file [block!]
		where[block!]
		/local
		return-list
		data
		obj
	][
		return-list: copy []
		data: copy []	
		obj: none
		either not empty? where [
			data: (db-select/where * :table :where)
		][
			data: (db-select * :table )
		]
		do compose/deep [
			foreach [(fields-in-file)]  [(data)] [
				obj: make oo []
				set obj reduce [(fields-in-file)]
				append return-list obj
			]
		]
		return return-list

	]
	
	insert: func[
		o[object!]
	][
		db-insert :table compose [next (skip get o 1)]
	]
	
	update: func[
		o[object!]
	][
		do compose/deep [db-update/where (table) 
			[(skip select my-db/tables :table 1)]
			[(skip get o 1)]
			compose[id = (o/id)]
		]
	]
	
	delete: func[
		o[object!]
	][
		db-delete/where :table compose[id = (o/id)]
	]

	commit: func[
	][
		db-commit :table
	]
]

Add a few bad points:

4) We would have to throw all of the current code away and start 
over.

5) The semantics of REBOL have very little in common with Java, so 
we would be very limited in what Java libraries we could use.

6) The JVM is much worse at dynamic languages than its competitors, 
such as the CLR.
7) SLOW.
This might tip the tables in the other direction.

brianH  " increased language consistency" if this where true, then 
we couldn't store object in blocks without evaluating them by default 
either. 


IMHO maps are just hash tables, just like a block.... storage, not 
vector jump tables.   but that might just me my skewed view of the 
world.

SORT isn't stable in R3, but UNIQUE uses hash tables, not SORT.

Any info about the algorithms used to construct and perform lookup 
on symbol's tables in R3 ?
People gave some hints back in the day.
But I can't remember who.

The problem with R3 right now is that it isn't yet compiled in 64-bits 
we still have the 1.6GB RAM limit for a process which is the biggest 
issue right now.   I have blown that limit a few times already, so 
it makes things a bit more complex and it doesn't allow me to fully 
optimize speed by using more pre-generated tables and unfolded state 
rules.

Our datasets are huge and we optimise for performance by unfolding 
and indexing a lot of stuff into rules... for example instead of 
parsing by a list of words, I parse by a hierarchical tree of characters. 
 its much faster since the speed is linear to the length of the word 
instead of to the number of items in the table. i.e.  the typical 
 O*n   vs.   O*O*n  type of scenario .  just switching to parse already 
was 10 times faster than using  hash! tables and using find on them.... 


In the end, we had a 100 time speed improvement from before parse 
to compiled parse datasets.  this means going from 30 minutes to 
less than 20 seconds....but this comes at a huge cost in RAM... a 
400MB Overhead to be precise.

I don't care, if my C compiler is closed source or not, because it 
just works. I also shouldn't care, if my COBOL compiler and interpreter 
is closed source or not, but I actually do, because the company behind 
can't figure out to make graphical tables (called GRIDs) the correct 
way, so my COBOL programs doesn't work as intended, and I have to 
create work-arounds. Years ago, I didn't care, if REBOL was closed 
source or open, but later I did, because I couldn't finish projects, 
I made with it.


Any future language, I would use, I don't care if it's closed or 
open, if it delivers, what it promices. If it doesn't, it's another 
case.

It's better for the config to be higher level in the master build 
tables which are defined in REBOL.  Those tables define more than 
just GCC compile time symbols, but also code/data within the boot 
scripts used by the interpreter.

This is the old debate of table-based configuration systems versus 
detection-based configuration systems. I understand that tables are 
less work at first, and so that's what everyone did in the old days, 
but exactly because environments are so complex these days, most 
people have moved to automatic detection

The goal of the current R3 build automation method is to save my 
time.


Currently, the platform table is only ~10 lines of REBOL, so it is 
difficult to beat using any other method.  And, even with the detection 
approach you mention, you need still tables.


However, that being said, if you want to create and test such a detection-based 
method and confirm it works over a range of OSes I would be happy 
to use it!  It must handle Linux, Syllable, BSD, OS X, Solaris, Windows, 
AmigaOS, Haiku, QNX, and various others, and also work for systems 
ten years old or more.

I'm open to any idea like this... as long as it saves *me* time.

Maxim <If an object has length, why can't I pick it?>
Maybe they are like SQL tables.....
A table has a length, so this is valid

   select count(*) from ....   -- to get the length of the table (ie 
   number of rows)

But, in the absence of an ORDER BY the rows do not have a user-accessible 
sequence, so

  select first from ....   -- is not valid syntax, nor (without an 
  ORDER BY) is it even meaningful

(Sorry, I know you did not want that debate)

Judging by the tables, it looks like passing and returning structs 
in C is better done in internal code than in exported functions. 
There are too many exceptions and fallbacks.

though in rebol not having type for vars and () for function arg 
submition, make difficult to identify var content  distinguish var 
from function ... Maybe then  a syntaxique item like th e $ in php 
for vars or the @ in ruby for tables would be an interesting point... 
Sorry I try to think about rebol syntaxe structure like someone seing 
it for the first time

The `atexit` symbol is present in both the dynamic symbol and relocation 
tables of the ELF binary.

Not sure if the Windows symbol tables work like I said, so you'll 
have to try