AltME groups: search

Here's a version that works in R3, tested against your example code:
>> a: deline read clipboard://
== {a,      b ,"c","d1
d2",a ""quote"",",",}

>> use [x] [collect [parse/all a [some [[{"} copy x [to {"} any [{""} 
to {"}]] {"} (keep replace/all x {""} {"}) | copy x [to "," | to 
end] (keep x)] ["," | end]]]]]
== ["a" "      b " "c" "d1^/d2" {a ""quote""} "," ""]


But it didn't work in R2, leading to an endless loop. So here's the 
version refactored for R2 that also works in R3

>> use [value x] [collect [value: [{"} copy x [to {"} any [{""} to 
{"}]] {"} (keep replace/all any [x ""] {""} {"}) | copy x [to "," 
| to end] (keep any [x ""])] parse/all a [value any ["," value]]]]
== ["a" "      b " "c" "d1^/d2" {a ""quote""} "," ""]


Note that if you get the b like "b" then it isn't CSV compatible, 
nor is it if you escape the {""} in values that aren't themselves 
escaped by quotes. However, you aren't supposed to allow newlines 
in values that aren't surrounded by quotes, so you can't do READ/lines 
and parse line by line, you have to parse the whole file.

I copied Ashley's example data into a file and checked against several 
commercial CSV loaders, including Excel and Access. Same results 
as the parsers above.

My func handles 100% of the CSV standard - http://tools.ietf.org/html/rfc4180
- at least for a single line. To really parse CSV you need a full-file 
parser, because you have to consider that newlines in values surrounded 
by quotes are counted as part of the value, but if the value is not 
surrounded completely by quotes (including leading and trailing spaces) 
then newlines are treated as record separators.

load-csv: func [
    "Load and parse a delimited text file."
    source [file! string!]
    /with
        delimiter
    /local lines
][
    if not with [delimiter: ","]

    lines: either file? source [read/lines source] [parse/all source 
    "^/"]
    remove-each line lines [empty? line]
    if empty? lines [return copy []]
    head forall lines [
        change/only lines parse/all first lines delimiter
    ]
]

I'm working on a fully standards-compliant full-file LOAD-CSV - actually 
two, one for R2 and one for R3. Need them both for work. For now 
I'm reading the entire file into memory before parsing it, but I 
hope to eventually make the reading incremental so there's more room 
in memory for the results.

It doesn't do conversion from string (or even from binary with LOAD-CSV/binary). 
This doesn't have a /part option but that is a good idea, especially 
since you can't just READ/lines a CSV file because it treats newlines 
differently depending on whether the value is in quotes or not. If 
you want to load incrementally (and can break up the lines yourself, 
for now) then LOAD-CSV supports the standard /into option.

since you can't just READ/lines a CSV file
 - yes, mine does that, and that's no good.

I believe the PE file format has sections that are padded, so it 
may be that the internal padding is different, as are the code segments, 
but the resulting EXE is the same size. I suppose you could do a 
binary diff to see.

I started using 2.7.6.

It's great to be able to "call" external apps without the dos window 
popping up. I have alot of use for this. 

I noticed, however, that when calling wmplayer in windows the file 
will play in the background which I also find appealing but there 
are times I might want wmplayer to open. Is there a way to force 
this with call?

Little bit more playing around, to see if +s was taking ... doesn't 
seem so.
rebol []
probe system/script
change-dir %..
list-dir
ask "waiting"

c:\tools\rebol\appdata\rebview +s testing.r

...
    words: none
]
waiting

** Script Error: list-dir expected dir argument of type: file url 
unset
** Near: list-dir
ask "waiting"
>>

The ask is evaluated before the list-dir error     And if  list-dir 
%.  is used, it lists the dir, without a security box.

Gregg, That looks far more intricate than I need but I will definitly 
keep it in mind.

I used a simple method...

first i used system/version to see what version of rebol is running 
so I can see if its Windows , Mac or Linux.

2nd- If its Windows I use 'Call/output "ver" win_ver' to get the 
windows version. I parse out what I need from the output. I just 
need to know if its vista as it has some file structure differences 
that screw up my "install".

Thanks for hte help...

It's been broken for so long, I no longer have the computer where 
I originally wrote it back in 2000 and posted it to the mailing list. 
It's still in my head though, so I'll collect it by Tuesday evening 
and start working. In brief:

- I don't know about installation on platforms other than Windows. 
Someone else will have to chime in here. Too ambitious for 2.7.7.

- Windows 2000+ support is still broken. Multi-user support is broken 
(same thing, really).

- Folder usage was mostly fixed in 2.6.3, but the registry is still 
misused. Registry migration will be needed.

- Non-admin installation should be possible, including user-specific 
file associations.

- Installation was broken altogether in the 2.7.x series - it doesn't 
work at all, not even to 2.6.3 levels.

- No-install usage of View needs better support. This means UI support 
too, if need be. VIew should be able to be a portable app.

Keep in mind that portable app usage of directories is completely 
different than installed usage, and needs to be.

That is the portable app method, which can't set file associations.

most of the time, I can guarantee you, that users feel lost, once 
they are supposed to navigate to their docs in file-manager ...

File placement is easy, and mostly handled by 2.6.3 - the real trick 
is registry usage.

Right. File placement is a matter of chosing the default folders 
of the installer, as it does now. You can change those choices.

Pekr, it's not abot where the user can pt their user scripts, it's 
about the desktop files and such. File associations manage the user 
script issue.

BrianH - but when user script is run, REBOL changes current dir to 
that of the script placement, no? And the script might require to 
create some file at that destination. I know that temp files will 
go to some deep cache dir, but what if script wants to write to the 
dir of the script placement?

For R2, all that matters for install is the View desktop and file 
associations.

>> help run
USAGE:
    RUN file /as suffix

DESCRIPTION:
     Runs the system application associated with a file.
     RUN is a native value.

ARGUMENTS:

     file -- The file to open (file, URL) or command to run (string). 
     (Type: file url string)

REFINEMENTS:
     /as
         suffix -- (Type: string file)
>> run "screen.png"
** Script Error: Feature not available in this REBOL
** Near: run "screen.png"

but it's a separate .r file or a new rebol exe ?

It's a separate .r file, but its functions can be swapped into the 
SDK source and a new .exe built. And that is my task for the evening: 
to swap in the more R2-compatible functions in R2/Forward into the 
mezzanine source.

I loaded the new REBOL/View 2.7.7.3.1 1-Jan-2010 down and tried to 
run it without installing.

It just writes a bypass-install file with true in it, but it does 
not run.

However starting it with the option --noinstall still works as expected.

I note cd no longer requires a file type in R3 .. .can that be ported 
back to r2 console?

Graham, the file management functions are still on the list to be 
included. I just didn't have the time for 2.7.7, so they'll be in 
2.7.8.

so I can point to or find entry 34672 quickly in a debug session 
dumped to a file.

>> help "thru"
Found these words:
   do-thru         function! Do a net file from the disk cache.

   exists-thru?    function! Checks if a file is in the disk cache. 
   Returns: no...

   launch-thru     function! Launch a net file from the disk cache.
   load-thru       function! Load a net file from the disk cache.

   path-thru       function! Return a path relative to the disk cache.

   read-thru       function! Read a net file from thru the disk cache. 
   Returns ...

ACCUMULATE didn't make the cut for R3 - it will be in the /Plus library. 
An R2 version is in R2/Forward, near the end of the file.

Posted mezzanine changes for 2.7.8, ported from R2/Forward 2.100.80.0:

- Added COLLECT-WORDS, RESOLVE, SINGLE?, IMMEDIATE!, INTERNAL!, INVALID-UTF?,
  CD, MORE, and the convenience words LS, PWD, RM and MKDIR.

- Removed buggy binary! support from ASCII? and LATIN1?, as done 
in 2.100.60.


See mezz-control.r #6763, mezz-file.r #6776, mezz-series.r #6772, 
mezz-string.r

#6773 and mezz-reflect.r #6771 for the relevant changes. Details 
in R3's docs.


Note: The APPEND and REMOLD rewrites are too awkward to incorporate 
without a

native APPLY function. UNBIND hasn't been written yet (hoping for 
a native).

Yes, for file associations and the View desktop. R2 View, like AltME, 
is designed for Win9x behavior right now. It doesn't work well with 
Win2k and above.

Work will be moving ahead soon on the R2 installer. Please post any 
information you have about installation issues in modern Windows 
versions (or links to sites with such), or about equivalent file 
location issues for other supported platforms. This will affect placement 
of the program, user-specific placement of the viewtop files, and 
(on Windows) file associations. Windows needs 4 installation profiles 
- let's hope that things can stay simpler on other platforms.

I prefer it always runs as standalone console, even at first time. 
If user wants then he/she can INSTALL or starts DESKTOP.

So I can send rebol.exe to anyone and tell him just drag & drop that 
script file onto that exe file.

my point is that install is very annoying for people who develop 
because it creates a dependency in the interpreter which cannot be 
rooted out.


I am not against the installer, i'm just against it being forced 
upon me, when the whole platform itself has no requirement for it


I have not started REBOL without it requiring an external file or 
socket in what 10 years... so really what is the point.  it just 
makes EVERYTHING complicated, like telling clients they have to fuck 
around with adding -qs all the time in their batch code, etc, etc.

oh and did you know that in vista (and possibly windows 7)  the names 
for windows paths are translated in the explorer but remain english 
in the file structure?


such that you can actually have the same folder twice within an explorer 
view  :-)   yep, great engineering.

the way I see it, the moment there is a rebol.r file in the same 
folder as the rebol.exe, no install should ever be required, whatever 
the rebol.r file contains.

the rebol.r file should operate before the sandbox is activated. 
 it needs access to the secure command, so it can prevent the sand 
box from ever starting.


If it can be made that simple, then install-away all you want.   
:-D

folder aliasing is new vista, and its a very stupid idea.  using 
the same path you end up at one file for read another for write? 
 wtf!

installation would simply be the act of running rebol.exe without 
a rebol.r file and generating a default rebol.r file in your current 
setup.

no, rebol is an interpreter, it can write to any file... for my own 
application to write within my own install dir is perfectly fine. 
 its not going to affect any other application.

the windows file organization is so screwed up in EVERY conceivable 
way that it will never be fixed.

the way I see it, applications should be FORCED into using specific 
paths for specific things.  then you could enforce proper file access 
etiquette.

the root problem for all of this lies in the archaic tree structure 
file systems.

a file should be able to have several paths (a bit like symbolic 
links in unix)  each one with different properties based on app context.

but some files Are not user files... like the rebol.r file, for example. 
 it should define where the user files are though.

ICarii, AltME is currently a Win95 app (it puts its writable data 
files in the same directory as its program file) and it's written 
in R2.

in any case, I am happy we see eye to eye in the role of the rebol.r 
file.

Andreas, corresponding concepts:
- The place where the program is installed/located.

- The place where the installed program version is listed, so you 
can know whether an update is needed.

- The place where user files, particularly the view-root and desktop 
files, needs to go.

- The place where machine-specific user settings should go (console 
layout on Windows).

- File associations, where possible. Not assuming that these are 
possible on platforms other than Windows.


Where these go depends on the installation profile. Yes, not installing 
at all is one of those profiles.

Adrian, "so it executes"  I meant installing rebol in the OS via 
file extensions so it always uses the -qs arguments over and above 
those it gets from the explorer.


also can shortcuts have arguments in win7?  this was removed in vista.

File associations. A setting saying which version is installed. A 
setting saying where the program is installed.

Maxim, see this about passing arguments to apps started by file association:


http://stackoverflow.com/questions/444388/how-can-i-pass-command-line-arguments-via-file-association-in-vista-64


Re. shortcuts having arguments, yes they can - Vista shortcuts could 
as well, though.

and yes... you have to manually change the command-line arguments, 
but then the OS' editors break up in what I can't remember, I think 
rebol disapeared from the "installed applications" list, although 
the file association still worked.

is the read-only file property bug fixed in win7

look in the URL-parser context within the prot-utils.r file.

that is where the url decoding occurs.

Some of what is coming in 2.7.8:

- Bug fixes and enhancements to improve Cheyenne, and other apps 
that have to do similar stuff.

- Some native fixes for non-Windows platforms, particularly Linux.

- Environment variable stuff: GET-ENV expansion on Windows, SET-ENV, 
LIST-ENV

- Function fixes: RUN enabled, LIST-REG/values, possibly TO-LOCAL-FILE

- R2/Forward: FUNCT/extern, LAST?, COLLECT-WORDS, EXTRACT fixes, 
ASCII? fixes, LATIN1? fixes, INVALID-UTF?, CD, LS, MORE, PWD, RM

- (Still pending) Natives: ASSERT, APPLY, RESOLVE, FOREACH set-word 
support

At least for Windows so far. Things that didn't make it in this version:
- The TO-LOCAL-FILE fix for Windows UNC paths
- FOREACH set-word support
- ASSERT, APPLY and RESOLVE are still mezzanine

But as a bonus, we got SELECT and FIND on objects, and that is better 
than any of those others :)

one indicator: IE 8 (which is not my default now since I moved to 
K-mleon) runs VIEW 2.7.8 as a right-click option using RUN - but 
download with IE8 and run from Explorer file view correctly triggers 
a request to authorize due to no "valid" digital signature for MS 
- yet this MS alert fails to trigger when run after download using 
K-meleon browser (yet no issue after K-meleon downloads CORE 2.7.8 
)

The request-to-authorize thing is managed by an extended attribute 
in one of the other forks of the file on NTFS, and not on FAT. Have 
you tried Chrome or Firefox?

Have you tried downloading the zip file containing the exe instead 
of the exe directly? http://www.rebol.com/downloads/v278/rebol-view-278-3-1.zip

On Mac 2.7.8.2.5:

>> ls
** Script Error: path has no value
** Where: ls
** Near: switch type?/word :path [
    unset! [] 
    file! [change-dir path] 
    string! [change-dir to-rebol-file path] 
    w...

>> source speed?
test
test
... (fills console)
test
test
<code>

The callbacks documentation is incomplete, and refers to the host-ext-test.c 
file for examples, but this isn't in the current host kit

Has it ever been discussed that R3 should be able to do IMPORT %file 
and know to look for %file.so on POSIX systems and %file.dll on Windows?

thing is you can import  %file   for     %file.r   too.

You can import 'file for %file.r (or whatever system/options/default-suffix 
is), but when specifying a file you are assumed to know what name 
it is. And you can use %.rx if you want a cross-platform file extension 
for your extension.

Operating systems may react badly to that, because they may expect 
their standard file extension. R3 extensions are OS dynamic libraries 
and are expected to be registered that way in the system

Then the program has a problem with portability due to different 
file extensions on different operating systems

Also, look at system/options/file-types. You can find the file extensions 
that are loaded as R3 extensions, and then go through them, adding 
them to your file's base name to find your file. Or you can add a 
one or two line platform-specific wrapper for your module code.

file-types contains "%.so extension" so we're halfway there, but 
IMPORT doesn't act on it

You don't want IMPORT to do automatic lookup for more than one file 
extension because it's a potential security hole. For that matter, 
you should know if you're loading an extension instead of a module, 
because extensions could break the rules that REBOL source modules 
can't. And system/options/default-suffix can be set by your program. 
For that matter, the host code for LOAD-EXTENSION could translate 
.rx to .so on platforms that require .so and won't load .rx - that 
is a platform-specific restriction.

Not .dll - it would only be required on platforms with a restriction 
on filenames that can be loaded as libraries. On windows there are 
dozens of different suffixes that are all basically DLLs, including 
.exe, .ocx, .dpl, .cpl, etc. Not many platforms restrict the file 
suffixes of libraries.

Can Syllable load dynamic libraries with arbitrary file suffixes?

Those operating systems shouldn't add %.rx to file-types then - the 
code that sets that is in the host. Some platforms are less cross-platform 
than others.

Yes, and literally the only platform distinction is the file extension

I am fine with explicitly specifying an extension, just not with 
a forced file extension. How about IMPORT/EXTENSION ?

Yes, I have a problem with that. It's way too much work for making 
a file extension dynamic, and it breaks most of my use cases

There are many platforms that use the same file extensions, so you'll 
have to find a way to put the BSD .so files in another directory 
from the Linux .so files. The .rx extension just adds Windows and 
OSX to that.

LOAD-EXTENSION doesn't do lookup beyond the current directory, so 
if you want to have it use some library path you have to add that 
in the host code, or %rebol.r. Either way you can add extensions 
and modules programmatically to the R3 runtime before the script 
you're running starts. If you want your scripts to be cross-platform 
then you probably shouldn't use a platform-specific file extension 
for the files, but if they are loaded or delayed in the host code 
or %rebol.r then it wouldn't matter.

True. You would use a different file for Linux, though it could also 
be called %zlib.rx :)

But I want to make clear that the file is DLL, not SO.

Have them download a zip file containing the Windows binaries. That's 
how SQLite does it.

Kaj: No need. You can import the extensions by filename. We want 
the initial import of an extension to be really clear, distinct from 
the import of a module. If you say that LOAD-EXTENSION looks up the 
file in the system library path, that shoul;d be all the searching 
we would need.

if people are distributing their files in a single dir, they will 
have .so, .dll  in the same dir.


can't we just build a little .r file which imports the proper lib, 
no?

You won't have 4 .dll's and 10 .so's. You will have _one_ file that 
suits your platform.

I want to be able to setup a R3 config file that says:

extensions-dir:  %/some/path/


and then R3 will only ever load extensions from there.    the extensions 
can do whatever they want, but we have a controled point of entry 
within REBOL.

afaik, systems allow paths on dlopen.    rebol would simply always 
ask for libs with paths... 


in fact, on windows, that is how you are supposed to do it, probing 
the registry first to get proper paths for libs.

Roughly summarising: an extension search path (preferred) + os loader 
search (fallback) + a cross-platform mechanism to load extensions 
via abstract names.


Could be as simple as reusing system/options/module-paths and load-extension, 
allowing the latter to take a word! parameter and also search the 
former. The possible extensions themselves are already in place (system/options/file-types). 
Loading is already done via dlopen, so loading a lib only by name 
(w/o any path component) will use the OS loader's search mechanisms.


And alternative would be to add a system/options/extension-paths 
(block!) option, and a dedicated loading primitive, say, import-extension, 
which searches the extension-paths and tries all possible platform-specific 
extensions (as per system/options/file-types).

The idea of a "safe" extension is tenuous at best as it is. Don't 
add the possibility of having some system-specific libraries polluting 
the file name space. For instance, I would consider one of the minimum 
requirements of a system-installed "safe" extension to be that it 
be guaranteed delayable.

The extension file itself is not cross-platform (in most cases, at 
least), therefore using a standard suffix is extremely appropriate.

prudent to stick with the standard .so extension

 - I'm having a little trouble understanding this, but maybe that 
 comes from too much experience with platforms where it has been demonstrated 
 that there are advantages to distinguishing between different dynamic 
 library types by using different file suffixes. The "standard .so 
 extension" means that these differences get put somewhere else in 
 the filename. But I can see your point with libraries like System.Data.Sqlite 
 that support more than one calling convention in the same file, so 
 that the R3 extension API would just be added to a system dynamic 
 library that is otherwise meant to be called by non-REBOL code - 
 incorporating its own wrapper extension.

Remember, on many Linuxes you just want to load libc, but the actual 
file is something like libc.so.6 or something (I'm paraphrasing here). 
Having LOAD-EXTENSION translate .rx is no different from that. You 
can even have a filename map on your platform if you like.

There is no further translation happening on Linux from libc.so to 
libc.so.6 as that is just a symlink in the file system

The actual suffixes tried are by default taken from system/options/file-types.

You can try them in the order the suffixes appear in system/options/file-types. 
That way your code doesn't have to special-case per platform; let 
the host code special-case it instead. Just in case .rx isn't supported 
on a platform, you might consider searching for 'extension and backtracking 
to get the file! suffixes.

>> find/reverse/tail find system/options/file-types 'extension word!
== [%.rx %.dll extension]

Yes, Brian, as I stated above I'm already using system/options/file-types. 
Quoting myself:

The actual suffixes tried are by default taken from system/options/file-types.

The problem is, I'm not sure where it happens in the source-code 
file as the output doesn't tell you. But can't I use the (.text+0x867) 
stuff to locate the usage in the source file?

I have a very strange effect: The init_block gets a c filename attached. 
The .r file that is used to generate the .h header file of course 
doesn't inlcude it. And the generated init_block numbers don't include 
it too.

Ok, so I found out that the last number wasn't 0. When generating 
the header file I get this as last line:

	116, 105, 97, 108, 105, 122, 101, 100, 34, 10,

};

This is the line makeing problems. When I change it to:

	116, 105, 97, 108, 105, 122, 101, 100, 34, 10, 0

};

it works.

For example, take the zmqext extension specification:
https://github.com/earl/r3-zmq/blob/master/zmqext.r3

This is the header file generated by above generator script:
https://gist.github.com/e22a35049ae56324004a

Module path searching only works with non-file! module names. I.e.
import 'foo

will search the system/options/module-paths for a file name %foo.reb. 
This search is a _very_ useful feature which we shouldn't rid ourselves 
of by stupid wars over which suffix to use.


As a module author, I want to use a suffix which I know will make 
using the module easiest for my target audience.


As a module distributor, I also want to make it easiest for my target 
audience to use the modules _and_ I need to enforce consistency within 
my distribution.


The only one who does not really care about the suffix is the actual 
user. The user just wants to use third-party modules without having 
to homogenise their suffixes first. But the burden of choosing a 
sensible suffix is currently imposed on the user. This does not make 
much sense.

I would like to see some support for automatic logging of events.


Rebol currently has two "hooks" to allow "Front-end" code to execute 
automatically in the REBOL.r and USER.r files.

I would like to see another "hook" to allow "Back-end" code to execute 
automatically whenever Rebol stops its current evaluations for any 
reason such as encountering the HALT, QUIT, or any error which causes 
Rebol to stop evaluation.  


This could allow for some kind of automatic logging of  QUIT, HALT 
or ERROR as well as capturing the error messages.  It would be nice 
to allow capturing of any other information such as the script name, 
computer name, current timestamp, and where in the script the evaluation 
stopped, and how it stopped.


I would use this to log all of my scripts that is invoked (put the 
code in REBOL.r file) and then the termination conditions defined 
so they could be triggered automatically.  With such a function, 
I could even have certain selected scripts send an automatic SMS 
messages to my phone if certain scripts errored out etc.



I'd like to propose a new function called WHEN that can be used to 
trigger code at a terminating event.

WHEN  block  /all

For any specified terminating condition, evaluates what follows it.

Arguments:

block [block!] - Block of terminating events (conditions followed 
by values)
Refinements:

/all - Evaluate all terminating conditions (do not stop at first 
true terminating condition)


The WHEN command would be used to define code to be evaluated whenever 
a termination condition of any kind occurred in which continued evaluation 
will stop.

Examples:
WHEN/ALL [

 HALT?	[ print "This script will HALT now" ]  ; code to be evaluated 
 before the HALT is evaluated

 QUIT?	[ print "This script will QUIT now" ]   ; code to evaluate 
 befoe the QUIT is evaluated 

 ERR?	[ print "An error was discovered"  ]   ;code to evaluate after 
 an error is captured, but before the message is printed
	END?	[
		  print "Script about to end"
		  if  now/time  >  23:00:00  print "Stopped after 11 pm"

  ]                                                              ; 
   code to evaluate whenever the current evaluation has stopped for 
  any reason
	]


Note:  HALT?, QUIT?, ERR?, END? are all logic values which become 
true depending upon how evaluations will be stopped.

END? will always be true and one of the other terminating conditions 
will always be true.

What do you think?

prefix? and suffix? just return true if a series starts with the 
same items in the same order as the second one.  
the second argument is the prefix to compare

so you can easily do:

unless suffix? file-name %.png [append file-name %.png]

In the next release of glass, in the TO DO file, I list a few measurse 
to increase performance and decrease ram use by concentrating on 
using as much persistent data than throw away recycling.  but that 
means a complete overhaul of liquid and glob (the rendering engine 
over liquid).