• Home
  • Script library
  • AltME Archive
  • Mailing list
  • Articles Index
  • Site search
 

AltME groups: search

Help · search scripts · search articles · search mailing list

results summary

worldhits
r4wp4382
r3wp44224
total:48606

results window for this page: [start: 46501 end: 46600]

world-name: r3wp

Group: Parse ... Discussion of PARSE dialect [web-public]
Ladislav:
15-Nov-2011
Regarding CASE and backtracking: it is not a problem when the effect 
of the keyword is limited to the nearest enclosing block.
BrianH:
15-Nov-2011
Backtracking often happens within blocks too, but yes, that does 
limit the scope of the problems caused (it doesn't eliminate the 
problem, it just limits its scope). Mode operations also don't interact 
well with flow control operations like OPT, NOT and AND. What would 
NOT CASE mean if CASE has effect on subsequent code without being 
tied to it? As a comparison, NOT CASE "a" has a much clearer meaning.
Ladislav:
15-Nov-2011
Hmm, to not complicate matters and hoping that it is the simpler 
variant I modified the CASE/NO-CASE proposal to use the

    CASE RULE

and

    NO-CASE RULE


syntax, since it really looks like simpler to implement than other 
possible alternatives.
Endo:
1-Dec-2011
I want to keep the digits and remove all the rest,

t: "abc56xyz" parse/all t [some [digit (prin "d") | x: (prin "." 
remove x)]] print head t

this do the work but never finish. If I add a "skip" to the second 
part the result is "b56y".
How do I do?
Endo:
1-Dec-2011
Doc: Thank you. I tried to do that way (advancing the series position) 
but couldn't. I may add some more things so I wish to do it by parse 
instead of other ways. And want to learn parse more :)
Thanks for all!
Ashley:
1-Dec-2011
Anyone written anything to parse csv into an import-friendly stream?

Something like:

a,      b ,"c","d1
d2",a ""quote"",",",

a|b|c|d1^/d2|a "quote"|,|


(I'm trying to load CSV files dumped from Excel into SQLite and SQL 
Server ... these changes will be in the next version of my SQLite 
driver)
Endo:
1-Dec-2011
Geomol: It would be nice if trim/with supports charsets.

And also I would love if I have "trace/parse" just like trace/net, 
which gives info about parse steps instead of all trace output.
Hmm I should add this to wish list I think :)
Gregg:
1-Dec-2011
Ashley, not sure exactly what you're after. I use simple LOAD-CSV 
and BUILD-DLM-STR funcs to convert each direction.
BrianH:
2-Dec-2011
Here's the R2 version of TO-CSV and TO-ISO-DATE (Excel compatible):

to-iso-date: funct/with [
	"Convert a date to ISO format (Excel-compatible subset)"
	date [date!] /utc "Convert zoned time to UTC time"
] [

 if utc [date: date + date/zone date/zone: none] ; Excel doesn't support 
 the Z suffix
	either date/time [ajoin [

  p0 date/year 4 "-" p0 date/month 2 "-" p0 date/day 2 " "  ; or T

  p0 date/hour 2 ":" p0 date/minute 2 ":" p0 date/second 2  ; or offsets
	]] [ajoin [
		p0 date/year 4 "-" p0 date/month 2 "-" p0 date/day 2
	]]
] [
	p0: func [what len] [ ; Function to left-pad a value with 0
		head insert/dup what: form :what "0" len - length? what
	]
]

to-csv: funct/with [
	"Convert a block of values to a CSV-formatted line in a string."
	[catch]
	data [block!] "Block of values"
] [
	output: make block! 2 * length? data
	unless empty? data [append output format-field first+ data]

 foreach x data [append append output "," format-field get/any 'x]
	to-string output
] [
	format-field: func [x [any-type!]] [case [
		none? get/any 'x [""]

  any-string? get/any 'x [ajoin [{"} replace/all copy x {"} {""} {"}]]
		get/any 'x = #"^"" [{""""}]
		char? get/any 'x [ajoin [{"} x {"}]]
		scalar? get/any 'x [form x]
		date? get/any 'x [to-iso-date x]

  any [any-word? get/any 'x any-path? get/any 'x binary? get/any 'x] 
  [
			ajoin [{"} replace/all to-string :x {"} {""} {"}]
		]
		'else [throw-error 'script 'invalid-arg get/any 'x]
	]]
]


There is likely a faster way to do these. I have R3 variants of these 
too.
BrianH:
2-Dec-2011
Here's a version that works in R3, tested against your example code:
>> a: deline read clipboard://
== {a,      b ,"c","d1
d2",a ""quote"",",",}

>> use [x] [collect [parse/all a [some [[{"} copy x [to {"} any [{""} 
to {"}]] {"} (keep replace/all x {""} {"}) | copy x [to "," | to 
end] (keep x)] ["," | end]]]]]
== ["a" "      b " "c" "d1^/d2" {a ""quote""} "," ""]


But it didn't work in R2, leading to an endless loop. So here's the 
version refactored for R2 that also works in R3

>> use [value x] [collect [value: [{"} copy x [to {"} any [{""} to 
{"}]] {"} (keep replace/all any [x ""] {""} {"}) | copy x [to "," 
| to end] (keep any [x ""])] parse/all a [value any ["," value]]]]
== ["a" "      b " "c" "d1^/d2" {a ""quote""} "," ""]


Note that if you get the b like "b" then it isn't CSV compatible, 
nor is it if you escape the {""} in values that aren't themselves 
escaped by quotes. However, you aren't supposed to allow newlines 
in values that aren't surrounded by quotes, so you can't do READ/lines 
and parse line by line, you have to parse the whole file.
BrianH:
2-Dec-2011
I copied Ashley's example data into a file and checked against several 
commercial CSV loaders, including Excel and Access. Same results 
as the parsers above.
Endo:
2-Dec-2011
BrianH: I tested parsing csv (R2 version) there is just a little 
problem with space between coma and quote:


parse-csv: func [a][ use [value x] [collect [value: [{"} copy x [to 
{"} any [{""} to {"}]] {"} (keep replace/all any [x ""] {""} {"}) 
| copy x [to "," | to end] (keep any [x ""])] parse/all a [value 
any ["," value]]]]]

parse-csv {"a,b", "c,d"}  ;there is space after coma
== ["a,b" { "c} {d"}]   ;wrong result.


I know it is a problem on CSV input, but I think you can easily fix 
it and then parse-csv function will be perfect.
BrianH:
2-Dec-2011
If there is a space after the comma and before the ", the " is part 
of the value. The " character is only used as a delimiter if it is 
directly next to the comma.
BrianH:
2-Dec-2011
My func handles 100% of the CSV standard - http://tools.ietf.org/html/rfc4180
- at least for a single line. To really parse CSV you need a full-file 
parser, because you have to consider that newlines in values surrounded 
by quotes are counted as part of the value, but if the value is not 
surrounded completely by quotes (including leading and trailing spaces) 
then newlines are treated as record separators.
BrianH:
2-Dec-2011
For the purposes of discussion I'll put the CSV data inside {}, so 
you can see the ends, and the results in a block of line blocks.

This: { "a" }
should result in this: [[{ "a" }]]

This: { "a
b" }
should result in this: [[{ "a}] [{b" }]]

This: {"a
b"}
should result in this: [[{a
b}]]

This: {"a ""b"" c"}
should result in this: [[{a "b" c}]]

This: {a ""b"" c}
should result in this: [[{a ""b"" c}]]

This: {"a", "b"}
should result in this: [["a" { "b"}]]
Gregg:
2-Dec-2011
load-csv: func [
    "Load and parse a delimited text file."
    source [file! string!]
    /with
        delimiter
    /local lines
][
    if not with [delimiter: ","]

    lines: either file? source [read/lines source] [parse/all source 
    "^/"]
    remove-each line lines [empty? line]
    if empty? lines [return copy []]
    head forall lines [
        change/only lines parse/all first lines delimiter
    ]
]
Gregg:
2-Dec-2011
I did head down the path of trying to handle all the things REBOL 
does wrong with quoted fields and such, but I have always found a 
way to avoid dealing with it.
Ashley:
2-Dec-2011
load-csv fails to deal with these 3 simple (and for me, common) cases:

1,"a
b"
2,"a""b"
3,

>> load-csv %test.csv
== [["1" "a"] [{b"}] ["2" "a" "b"] ["3"]]

I've reverted to an in situ brute force approach:

c: make function! [data /local s] [
		all [find data "|" exit]
		s: false
		repeat i length? trim data [
			switch pick data i [
				#"^""	[s: complement s]
				#","	[all [not s poke data i #"|"]]
				#"^/"	[all [s poke data i #" "]]
			]
		]
		remove-each char data [char = #"^""]

  all [#"|" = last data insert tail data #"|"]	; only required if we're 
  going to parse the data
		parse/all data "|^/"
]

which has 4 minor limitations:

1) the data can't contain the delimter you're going to use ("|" in 
my case)

2) it replaces quoted returns with another character (" " in my code)

3) it removes all quote (") characters (to allow SQLite .import and 
parse/all to function correctly)
4) Individual values are not trimmed (e.g.c "a ,b" -> ["a " "b"])


If you can live with these limitations then the big benefit is that 
you can omit the last two lines and have a string that is import 
friendly for SQLite (or SQL Server) ... this is especially important 
when dealing with large (100MB+) CSV files! ;)
BrianH:
2-Dec-2011
I'm working on a fully standards-compliant full-file LOAD-CSV - actually 
two, one for R2 and one for R3. Need them both for work. For now 
I'm reading the entire file into memory before parsing it, but I 
hope to eventually make the reading incremental so there's more room 
in memory for the results.
BrianH:
3-Dec-2011
I'm putting LOAD-CSV in the %rebol.r of my dbtools, treating it like 
a mezzanine. That's why I need R2 and R3 versions, because they use 
the same %rebol.r with mostly the same functions. My version is a 
little more forgiving than the RFC above, allowing quotes to appear 
in non-quoted values. I'm making sure that it is exactly as forgiving 
on load as Excel, Access and SQL Server, resulting in exactly the 
same data, spaces and all, because my REBOL scripts at work are drop-in 
replacements for office automation processes. If anything, I don't 
want the loader to do value conversion because those other tools 
have been a bit too presumptuous about that, converting things to 
numbers that weren't meant to be. It's better to do the conversion 
explicitly, based on what you know is supposed to go in that column.
BrianH:
3-Dec-2011
Here's the R2 version, though I haven't promoted the emitter to an 
option yet:

load-csv: funct [

 "Load and parse CSV-style delimited data. Returns a block of blocks."
	[catch]
	source [file! url! string! binary!]
	/binary "Don't convert the data to string (if it isn't already)"
	/with "Use another delimiter than comma"
	delimiter [char! string! binary!]
	/into "Insert into a given block, rather than make a new one"
	output [block!] "Block returned at position after the insert"
] [
	; Read the source if necessary
	if any [file? source url? source] [throw-on-error [
		source: either binary [read/binary source] [read source]
	]]
	unless binary [source: as-string source] ; No line conversion
	; Use either a string or binary value emitter
	emit: either binary? source [:as-binary] [:as-string]
	; Set up the delimiter
	unless with [delimiter: #","]

 valchar: remove/part charset [#"^(00)" - #"^(FF)"] join crlf delimiter
	; Prep output and local vars
	unless into [output: make block! 1]
	line: [] val: make string! 0
	; Parse rules
	value: [
		; Value surrounded in quotes
		{"} (clear val) x: to {"} y: (insert/part tail val x y)
		any [{"} x: {"} to {"} y: (insert/part tail val x y)]
		{"} (insert tail line emit copy val) |
		; Raw value
		x: any valchar y: (insert tail line emit copy/part x y)
	]
	; as-string because R2 doesn't parse binary that well
	parse/all as-string source [any [
		end break |
		(line: make block! length? line)
		value any ["," value] [crlf | cr | lf | end]
		(output: insert/only output line)
	]]
	also either into [output] [head output]
		(source: output: line: val: x: y: none) ; Free the locals
]


All my tests pass, though they're not comprehensive; maybe you'll 
come up with more. Should I add support for making the row delimiter 
an option too?
BrianH:
3-Dec-2011
The R3 version will be simpler and faster because of the PARSE changes 
and better binary handling. However, url handling might be trickier 
because READ/string is ignored by all schemes at the moment.
BrianH:
4-Dec-2011
The one above misses one of the Excel-like bad data handling patterns. 
Plus, I've added a few features, like multi-load, more option error 
checking , and R3 versions. I'll post them on REBOL.org today.
BrianH:
5-Dec-2011
Making the end-of-line delimiter an option turned out to be really 
tricky, too tricky to be worth it. The code and time overhead from 
just processing the option itself was pretty significant. It would 
be a better idea to make that kind of thing into a separate function 
which requires the delimiters to be specified, or a generator that 
takes a set of delimiters and generates a function to handle that 
specific set.
Henrik:
5-Dec-2011
Well, now, Brian, this looks very convenient. :-) I happen to be 
needing a better CSV parser, than the one I have here, but it needs 
to not convert cell values away from string, and I also need to parse 
partially, or N number of lines. Is this possible with this one?
BrianH:
5-Dec-2011
It doesn't do conversion from string (or even from binary with LOAD-CSV/binary). 
This doesn't have a /part option but that is a good idea, especially 
since you can't just READ/lines a CSV file because it treats newlines 
differently depending on whether the value is in quotes or not. If 
you want to load incrementally (and can break up the lines yourself, 
for now) then LOAD-CSV supports the standard /into option.
Henrik:
5-Dec-2011
since you can't just READ/lines a CSV file
 - yes, mine does that, and that's no good.
BrianH:
5-Dec-2011
Yes, that is a possibility, but there yet. Resuming would be a problem 
because you'd have to either save a continuation position or reparse. 
Maybe something like LOAD/next would work here, preferably like the 
way R3's LOAD/next was before it was removed in favor of TRANSCODE/next. 
Making the /into option work with /next and /part would be interesting.
Henrik:
5-Dec-2011
I don't really need anything but having the ability to parse the 
first 100 lines of a file and doing that many times, so I don't care 
so much about continuation. This is for real-time previews of large 
CSV files (> 10000 lines).
BrianH:
5-Dec-2011
The latter makes chaining of the data to other functions easier, 
but requires a variable to hold the continuation; however, you usually 
use a variable for that anyway. The former makes it easier to chain 
both values (and looks nicer to R2 fans), but the only function you 
normally chain both values to is SET, so that's of limited value.
BrianH:
5-Dec-2011
The main problem with /part is that the current code reads the whole 
file into memory before parsing, and the parsing itself has miniscule 
overhead compared to the file overhead. Really doing /part properly 
might require incremental file reading, to the extent that that works 
(how well does it work for the http scheme on R3?).
BrianH:
5-Dec-2011
LOAD has the same problem on R2 and R3. The continuation returned 
would be an offset reference to the entire data in the file, at the 
position after the part parsed.
BrianH:
6-Dec-2011
http://www.rebol.org/view-script.r?script=csv-tools.rupdated, with 
the new LOAD-CSV /part option.

The LOAD-CSV /part option takes two parameters:
- count: The maximum number of decoded lines you want returned.

- after: A word that will be set to the position of the data after 
the decoded portion, or none.


If you are loading from a file or url then the entire data is read, 
and after is set to a position in the read data. If you are converting 
from binary then in R2 after is set an offset of an as-string alias 
of the binary, and in R3 after is set to an offset of the original 
binary. R3 does binary conversion on a per-value basis to avoid having 
to allocate a huge chunk of memory for a temporary, and R2 just does 
string aliasing for the same reason. Be careful to expect that if 
you are passing the value assigned to after to anything else than 
LOAD-CSV (which can handle it either way).
BrianH:
6-Dec-2011
I was a little concerned about making /part take two parameters, 
since it doesn't anywhere else, but the only time you need that continuation 
value is when you do /part, and you almost always need it then. Oh 
well, I hope it isn't too confusing :)
BrianH:
6-Dec-2011
This pass-by-word convention is a little too C-like for my tastes. 
If only we had multivalue return without overhead, like Lua and Go.
BrianH:
7-Dec-2011
I considered making a /strict option to make it trigger errors in 
that case, but then reread the RFC and checked the behavior again, 
and realized that noone took the spec that strictly. Most tools either 
behave exactly the same as my LOAD-CSV (because that's how Excel 
behaves), or completely fail when there are any quotes in the file, 
like PARSE data "," and PARSE/all data ",".
BrianH:
7-Dec-2011
The RFC is fairly loose and incomplete documentation of the observed 
behavior of most CSV handling tools. Excel's behavior is the real 
defacto standard, for better or worse.
Pekr:
8-Dec-2011
BrianH: one of my guys returned from some MS training, and he pointed 
me out to LogParser. It seems even some guys at MS are kind of dialecting 
:-) It looks like SQL, and you can query logs, and do some nice stuff 
around them ....

http://technet.microsoft.com/en-us/library/ee692659.aspx
BrianH:
18-Dec-2011
Yeah, blocks for cells are so far outside the data model of everything 
else that uses CSV files that TO-CSV was written to assume that you 
forgot to put an explicit translation to a string or binary in there 
(MOLD, FORM, TO-BINARY), or more likely that the block got in there 
by accident. Same goes for functions and a few other types.
BrianH:
18-Dec-2011
As for that TO-ISO-DATE behavior, yes, it's a bug. Surprised I didn't 
know that you can't use /hour, /minute and /second on date! values 
with times in them in R2. It can be fixed by changing the date/hour 
to date/time/hour, etc. I'll update the script on REBOL.org.
BrianH:
18-Dec-2011
Having to put an explicit conversion from blocks, parens, objects, 
maps, errors, function types, structs, routines and handles, reminds 
you that you would need to explicitly convert them back when you 
LOAD-CSV. Or more often, triggers valuable errors that tell you that 
unexpected data made it in to your output.
GrahamC:
18-Dec-2011
dunno if it's faster but to left pad days and months, I add 100 to 
the value and then do a next, followed by a form ie. regarding you 
p0 function
BrianH:
18-Dec-2011
It's worth timing. I'll try both, in R2 and R3.
GrahamC:
19-Dec-2011
and the outcome was?
GrahamC:
20-Dec-2011
Yeah, generally math is faster than using logic.  And old Forth trick.
BrianH:
20-Dec-2011
Be careful, if you don't quote string values then the character set 
of your values can't include cr, lf or your delimiter. It requires 
so many changes that it would be more efficient to add new formatter 
functions to the associated FUNCT/with object, then duplicate the 
code in TO-CSV that calls the formatter. Like this:

to-csv: funct/with [
	"Convert a block of values to a CSV-formatted line in a string."
	data [block!] "Block of values"

 /with "Specify field delimiter (preferably char, or length of 1)"
	delimiter [char! string! binary!] {Default ","}
	; Empty delimiter, " or CR or LF may lead to corrupt data
	/no-quote "Don't quote values (limits the characters supported)"
] [
	output: make block! 2 * length? data
	delimiter: either with [to-string delimiter] [","]
	either no-quote [
		unless empty? data [append output format-field-nq first+ data]

  foreach x data [append append output delimiter format-field-nq :x]
	] [
		unless empty? data [append output format-field first+ data]
		foreach x data [append append output delimiter format-field :x]
	]
	to-string output
] [
	format-field: func [x [any-type!] /local qr] [

  ; Parse rule to put double-quotes around a string, escaping any inside

  qr: [return [insert {"} any [change {"} {""} | skip] insert {"}]]
		case [
			none? :x [""]
			any-string? :x [parse copy x qr]
			:x = #"^(22)" [{""""}]
			char? :x [ajoin [{"} x {"}]]
			money? :x [find/tail form x "$"]
			scalar? :x [form x]
			date? :x [to-iso-date x]

   any [any-word? :x binary? :x any-path? :x] [parse to-string :x qr]
			'else [cause-error 'script 'expect-set reduce [

    [any-string! any-word! any-path! binary! scalar! date!] type? :x
			]]
		]
	]
	format-field-nq: func [x [any-type!]] [
		case [
			none? :x [""]
			any-string? :x [x]
			money? :x [find/tail form x "$"]
			scalar? :x [form x]
			date? :x [to-iso-date x]
			any [any-word? :x binary? :x any-path? :x] [to-string :x]
			'else [cause-error 'script 'expect-set reduce [

    [any-string! any-word! any-path! binary! scalar! date!] type? :x
			]]
		]
	]
]


If you want to add error checking to make sure the data won't be 
corrupted, you'll have to pass in the delimiter to format-field-nq 
and trigger an error if it, cr or lf are found in the field data.
BrianH:
20-Dec-2011
Nope, that's a bug in the R2 version only. Change this:
			:x = #"^(22)" [{""""}]
to this:
			:x == #"^(22)" [{""""}]

Another incompatibility between R2 and R3 that I forgot :(
I'll update the script on REBOL.org.
BrianH:
20-Dec-2011
Weirdly enough, = and =? return true in that case in R2, but only 
== returns false; false is what I would expect for =? at least.
BrianH:
20-Dec-2011
Note that that was a first-round mockup of the R3 version, Endo. 
If you want to make an R2 version, download the latest script and 
edit it similarly.
Endo:
20-Dec-2011
Native formats runs well if you export from one SQL server and import 
from other.
Endo:
20-Dec-2011
The biggest problem would be the different datatypes for different 
versions of SQL Server, if there is no good documentation for the 
native format. But BCP does the job quite well. I CALL it when necessary 
and try to FIND if any error output. 

There is XML format files as well, easier to understand but no functional 
differencies betwenn non-XML format files.
Rebolek:
5-Jan-2012
Endo, I will try to find newest version and let you know. But do 
not expect it to translate every regular expession.
Group: Core ... Discuss core issues [web-public]
Geocaching:
17-Mar-2011
???
>>  my-code-a: [a: [] append a 'x]
== [a: [] append a 'x]
>> do my-code-a
== [x]
>> do my-code-a
== [x x]
>> a: []
== []
>> a
== []
>> head a
== []
>> do my-code-a
== [x x x]
>> a
== [x x x]


what is the logic behind this? How could a be empty after a: [] and 
be filled will three 'x after just one call to do my-code-a
Ladislav:
17-Mar-2011
(or, be careful, and use the copy [...] construct)
Rebolek:
17-Mar-2011
The first A is defined inside MY-CODE-A block. And because you do 
not use copy [], it's not rewritten every time you call MY-CODE-A 
block.
Geocaching:
17-Mar-2011
It looks to me like everytime you call my-code-a, you assign the 
block defined in my-code-a to the variable a which is globally accessible. 
When you write a: [] outside my-code-a, you assigne another block 
to a... a is a pointer and you swith the adresse it is pointed to.

>> my-code-a: [a: [] append a 'x]
== [a: [] append a 'x]
>> do my-code-a
== [x]
>> a
== [x]
>> a: copy []
== []
>> append a 'y
== [y]
>> a
== [y]
>> do my-code-a
== [x x]
>> a
== [x x]
BrianH:
17-Mar-2011
Consistency between OS platforms. On platforms other than Windows, 
/Core doesn't need X or any other kind of GUI, and so doesn't require 
the libraries (good for headless servers). When there is no GUI, 
there is no clipboard. The Windows version of R2 just inherited this.
BrianH:
19-Mar-2011
Yeah, I remember that one from a couple years ago. The arrow words 
are special cased, and in R2 the tag type is recognized with higher 
priority unless there is a space afterwards.
BrianH:
19-Mar-2011
I already did this for word syntax for R3, and the process led to 
some bug reports.
Dockimbel:
19-Mar-2011
I've tested both with INSERT and WRITE-IO, same result. But I've 
used CALL/OUTPUT on the test CGI script to simulate a call from a 
webserver.
Andreas:
19-Mar-2011
and the print/prin thing obviously won't help with CALL/output crashing 
or freezing
Andreas:
19-Mar-2011
and with:
system/ports/output/state/with: "^/"
BrianH:
19-Mar-2011
Ok, weird. I don't know if this is the case, but IMO the --cgi option 
should cause the output port to be in text mode and to convert ^/ 
to crlf, network standard. And then WRITE-IO should not do any conversions, 
and output in binary mode regardless of the text mode of the output 
port.
Andreas:
19-Mar-2011
and write-io with 2.7.8 on linux actually running as cgi works fine 
as well
PeterWood:
20-Mar-2011
This is the console output from the command line pgm:

C:\REBOLServicesTest>cr
)haracter 13 is enclosed in the parentheses (


I then checked that the command line pgm could be successfully called 
with the following two lines of Ruby:

	puts %x{cr}
	print %x{cr}.dump

Which gave the following output:
C:\REBOLServicesTest>ruby call_test.rb
)haracter 13 is enclosed in the parentheses (
Character 13 is enclosed in the parentheses (\r)

I then called the command line pgm from a REBOL Console session:

>> call/console "cr"
Character 13 is enclosed in the parentheses (
)== 0
>> print to-binary {Character 13 is enclosed in the parentheses (
{    )}
#{
43686172616374657220313320697320656E636C6F73656420696E2074686520
706172656E74686573657320280A29
}
>> buffer: make string! 256
== ""
>> call/output "cr" buffer
== 0
>> probe buffer
Character 13 is enclosed in the parentheses (^/)
== "Character 13 is enclosed in the parentheses (^/)"
>> print to-binary buffer
#{
43686172616374657220313320697320656E636C6F73656420696E2074686520
706172656E74686573657320280A29
}


As you can see both call/console and call/output turned the 0x0D 
into a 0x0A.
Dockimbel:
21-Mar-2011
I concur, it's a CALL issue and not a --cgi one. I did more tests 
with my own CALL/OUTPUT implementation and it doesn't show any newline 
alteration in the binary CGI output.
Geomol:
25-Mar-2011
That must have changed in later version. I tested such things deeply 
1-2 years ago and wrote a document, that I sent to Carl. Back then 
I noticed:

>> 2.2#
** Syntax error: Invalid "integer" -- "2.2#"


, and I suggested, it should be an invalid decimal, not invalid integer. 
Today I get:

>> 2.2#
== #.2#

There are many such situations.
Andreas:
27-Mar-2011
And, of course:
>> apply :foo [1 false 3]
== [1 none none]
PeterWood:
1-Apr-2011
RSA is not really designed to encrypt large chunks of data. You'd 
be better of using AES (or  Rijndael as it used to be known as is 
still called in REBOL).


RSA is better used for exchanging passwords and "signing" documents.
GrahamC:
1-Apr-2011
RSA is significantly slower than symmetric key encryption algorithms, 
and a single encryption or decryption operation can only process 
an amount of data up to the size of the RSA key. For encrypting or 
decrypting large amounts of data RSA is usually used in combination 
with symmetric key algorithms or secure checksums as follows:
GrahamC:
1-Apr-2011
so I would use AES to encrypt the data,and then use RSA to encrypt 
the AES encryption key I guess
PeterWood:
1-Apr-2011
Yes you would use AES to encrypt the data and then RSA to encrypt 
and send somebody the encryption key.
PeterWood:
1-Apr-2011
In my experience, the issues of encrypting in REBOL and decrypting 
in something else usually involve either the chaining method or the 
padding used.
PeterWood:
1-Apr-2011
Also, from Wikipedia - AES has a fixed block size of 128 bits whereaas 
Rjindael can have a blocksize in any multiple of 32 between 128 and 
256 bits.
james_nak:
1-Apr-2011
Again, this might be a Graham question: I'm still working with that 
video encoder which uses http to communicate. They have a .cgi script 
which downloads the recorded video file from the internal SD card 
to the requester. My problem is the content I receive is somehow 
different than the files which I can download via a browser and of 
course will not play. I still using your http-tools to GET/POST. 
My initial thought was that  data returned is somehow being translated. 
Any thoughts?
MikeL:
4-Apr-2011
I am making a simple (I hope) worfkflow prototype and want to use 
REBOL objects which I can SAVE and LOAD.  A workflow object!  to 
have a node-map collection in it of simple nodes of the workflow 
graph.   Source ->A -> B -> SINK where the workflow knows about the 
next node and status. Externally there is UI to support the work 
part ... which is URL data on a given node.   Looks like it fits 
into Cheyenne RSP well - maybe zmq when I get a bit further along. 
  Save a flow in process as a .txt file using SAVE/ALL filename.txt 
work-flow-instance.   But no success with work-flow-instance: LOAD 
filename.txt        Do I have to BIND on LOAD to re-instantiate the 
object?
MikeL:
4-Apr-2011
OK I am just doing something stupid and will clear that up soon.
GrahamC:
12-Apr-2011
and what happens if you mold/all it?
onetom:
19-Apr-2011
just stumbled upon it yesterday too :) accidentally. i havent noticed 
that im not in bash and typed rebol
BrianH:
20-Apr-2011
Onetom, that error has been reported already and fixed in R2/Forward, 
but it hasn't made it into R2 yet. Here is the revised MAP-EACH:

map-each: func [

 "Evaluates a block for each value(s) in a series and returns them 
 as a block."
	[throw catch]

 'word [word! block!] "Word or block of words to set each time (local)"
	data [block!] "The series to traverse"
	body [block!] "Block to evaluate each time"
	/into "Collect into a given series, rather than a new block"

 output [any-block! any-string!] "The series to output to" ; Not image!
	/local init len x
][
	; Shortcut return for empty data
	either empty? data [any [output make block! 0]] [
		; BIND/copy word and body
		word: either block? word [
			if empty? word [throw make error! [script invalid-arg []]]

   copy/deep word  ; /deep because word is rebound before errors checked
		] [reduce [word]]
		word: use word reduce [word]
		body: bind/copy body first word
		; Build init code
		init: none
		parse word [any [word! | x: set-word! (
			unless init [init: make block! 4]
			; Add [x: at data index] to init, and remove from word
			insert insert insert tail init first x [at data] index? x
			remove x
		) :x | x: skip (

   throw make error! reduce ['script 'expect-set [word! set-word!] type? 
   first x]
		)]]
		len: length? word ; Can be zero now (for advanced code tricks)
		; Create the output series if not specified
		unless into [output: make block! divide length? data max 1 len]
		; Process the data (which is not empty at this point)

  until [ ; Note: output: insert/only output needed for list! output
			set word data  do init

   unless unset? set/any 'x do body [output: insert/only output :x]
			tail? data: skip data len
		]
		; Return the output and clean up memory references
		also either into [output] [head output] (
			set [word data body output init x] none
		)
	]
]
BrianH:
20-Apr-2011
The revised mezz-series.r (relative to the 2.7.8 SDK source) is in 
R3 chat #8008 - that's the number of the particular version, which 
you can see by going to #41 (R2/Mezzanines) and doing a LF command.
BrianH:
20-Apr-2011
The chat interface uses numbers as a deliberate design choice because 
it is easier to memorize and refer to a number than it is to a path 
or message ID. You can even write a message number in #8008 form 
in another message and it can be followed like a hyperlink to the 
message of that number. You can also do the hyperlink trich to CureCode 
tickets using the bug#539 form, which will take you to http://issue.cc/r3/539
(that R3 bug I mentioned above).
Geomol:
26-Apr-2011
REBOL also allow decimal! refinements like:

/1.2

but treat it like an integer refinement.


Some further thoughts: Can refinements be categorized in two groups, 
integer refinements and word refinements? If yes, then the refinement 
datatype maybe should be two different datatypes?
Geomol:
26-Apr-2011
I did this under OS X with R2:

>> write %/Volumes/NICOM/test.txt "Testing..."
(and the files was produced, then)
>> rename %/Volumes/NICOM/test.txt %/Volumes/NICOM/test2.txt
** Access Error: Cannot rename test.txt

So an error.
Henrik:
26-Apr-2011
it works, if I open the port and find the file by hand
Geomol:
26-Apr-2011
Reading the source for RENAME, it rename the file by changing its 
name in the directory file. And this doesn't work for some reason 
with USB sticks under OS X using R2.
Henrik:
26-Apr-2011
and yes, it is at the CHANGE on the port, that fails.
Geomol:
26-Apr-2011
Sure, but do you believe, refinements like /1, /1a and /1.2 are made 
on purpose or just a side effect on how it's implemented?
BrianH:
26-Apr-2011
There are a lot of values that can appear in paths that don't translate 
to refinements. Paths and refinements are only related in function 
call syntax; otherwise they are not related.
Geomol:
26-Apr-2011
I don't buy it, as I could just use #1, #1a and #1.2.
BrianH:
26-Apr-2011
We are still making tickets related to word and refinement inconsistencies 
for R3 (or at least I am, when I find bugs in the syntax while I'm 
trying to reverse engineer the syntax docs). While the numeric refinement 
issue is settled, there are other issues that haven't yet been discovered. 
Most of the syntax problems are related to scanner precedence. All 
of the word and path datatypes can be constructed with characters/contents 
that don't really scan the same way in literal syntax, but it is 
not really considered an error. Datatypes are meant primarily for 
in-memory use - their syntax is secondary, and in many cases the 
literal syntax only covers a subset of the possible values.
Geomol:
26-Apr-2011
The original design of REBOL has many many great ideas. It's just 
the implementation, that isn't good enough in many cases. With these 
new explanations, the whole thing just get more complex, which isn't 
good. My view is, that it's better to stick with a simple design 
and work on getting that implemented.
BrianH:
26-Apr-2011
The problem is that believing the original design to be simple is 
what got us into this mess. The original design was an overview, 
and the details were unspecified. The overview was simple and still 
is, but now we are looking at, documenting and refining the details.
Geomol:
26-Apr-2011
For a scanner (also called lexical analyser), I can recommend studying 
the UNIX command lex. The code produced might be a bit bigger in 
size, but it's fast and produce good result.
BrianH:
26-Apr-2011
I've used lex, and many other compiler generators over the years, 
and yes, it's helped with the REBOL syntax documentation discovery. 
However, it might help you to know that the REBOL lexical analyzers 
and parsers are hand-written, not generated. This is why TRANSCODE 
in R3 and LOAD in R2 are so fast, but it is also why it is so tricky 
to resolve syntactic precedence bugs quickly.
BrianH:
26-Apr-2011
It used to be generated, but Carl says it's faster. I don't doubt 
him, because I've used dozens of parser generators before and that 
always seems to be the case. Sometimes you can get faster generated 
parsers, but generated lexers are usually slower because they're 
table-driven rather than code-driven. The advantage to generated 
lexers is that they are easier to write for complex lexical rules; 
for simple lexical rules, it is usually worth hand-coding.
BrianH:
26-Apr-2011
One of the tricks when refining the details is to realize that there 
is a real runtime difference between recommending that people not 
do something, and prohibiting something. Every time we prohibit something 
it has runtime overhead to enforce that prohibition. So every recommendation 
needs documenting and explaining, but every prohibition needs justifying. 
There are situational tradeoffs that recommendations can resolve 
easier than prohibitions. This is why we have to be extra careful 
about this.
BrianH:
26-Apr-2011
Actually, that's still considered pretty simple. You still might 
need a DFA for some of the rules, but most of them can be recognized 
by hand-written code more efficiently. The problems are not caused 
by not using a generated lexer - even a generated lexer can have 
precedence errors. The real syntax bugs in R3 are there because noone 
has really gone through and figured out what they are, systematically; 
most of them are still undocumented. Recently, in my spare time, 
I've been trying to go through and document the syntax and ticket 
the bugs, so soon the limit will be developer time. (In R2, the bugs 
are there because the syntax is frozen for backwards compatibility.)
BrianH:
26-Apr-2011
As for the syntax-vs-memory data restrictions, it's another tradeoff. 
Regular REBOL syntax is much more limited than the full data model 
of REBOL, even if you include MOLD/all syntax, because the syntax 
was designed more for readability and writeability by humans. If 
we limit the data model to match the syntax, we limit our capabilities 
drastically. Limiting to the syntactic form only makes sense when 
you are serializing the data for storage or transport; in memory, 
it's unnecessary. A better solution is making a more comprehensive 
serialization format that doesn't have to be human readable - Rebin 
- and then using it when we need to serialize more of the in-memory 
data.
BrianH:
26-Apr-2011
C++ and Perl.
46501 / 4860612345...464465[466] 467468...483484485486487