Fixed (width) dialect
[1/3] from: al::bri::xtra::co::nz at: 27-Aug-2002 21:58
Here's the Fixed dialect I've been working on. It's a dialect for pulling
apart fixed width data files and translating the strings into appropriate
Rebol values. With the use of my Associate functions (included), the dialect
can even combine split up values. Here's an example of the dialect (with
comments), which I'm using to cut up a DOS/Windows 3.1 database that my
local high school uses:
Fields: [
"Family Name" 20 ; Column name is a string!, width of the field
in characters is a integer!.
"First Name" 25
"Preferred" 12
"Mail to Whom" 35
"Invoices?" logic! ; Converts "Y"/"N into logic value, no need
for size as it's automatic.
"Reports?" logic!
"Address" [25 20 20] ; multiple lines automatically converted to
one Rebol string!
"Telephone" 19 /with " -()" ; refinement to 'trim function, to
get rid of extraneous characters in 'phone number.
"Cellphone" 7 /with " -()"
"Nationality" 3
"Language" 2
skip 4
"Year" 2 integer! ; Converts integer text into integer value.
"Form" 4
skip 4
"Gender" char! ; Same for char! values.
"Birth" 8 date! ; and for date! values. The "8" specifies how
much text to use for the date.
"Enrollment" 5 issue!
skip ; Skip a single character
"First Started" 8 date!
"First Attended" 8 date!
skip 3 ; Skip several characters.
"Status" char!
"Magic Marker" 3 binary! ; Convert a binary value (key in BASIC
data).
skip 2
]
Andrew Martin
ICQ: 26227169 http://valley.150m.com/
-><-
-- Attached file included as plaintext by Listar --
-- File: Fixed.r
Rebol [
Name: 'Fixed
Title: "Fixed"
File: %Fixed.r
Author: "Andrew Martin"
eMail: [Al--Bri--xtra--co--nz]
Web: http://valley.150m.com
Date: 27/August/2002
Version: 1.0.0
Purpose: {Cuts up fixed width data file into Rebol values in an association.}
Category: [util 1]
]
Fixed: function [
{Cuts up fixed width data file into Rebol values in an association.}
Data [string! binary!] "The data file"
Dialect [block!] "The dialect controlling the cutting."
] [
Values Field Width Widths Value Type Previous With
] [
Values: make block! 200
parse Dialect [
any [
[
'skip (Width: 1) opt [set Width integer!] (
Data: next at Data Width
)
]
| [
set Field string! (Width: 1 Type: string!) [
[
set Width integer! set Type [
'issue! | 'integer! | 'date! | 'binary!
] (
Value: attempt [load trim to-string copy/part Data Width]
)
]
| [
set Width integer! opt [/with set With string!] (
;Value: trim to-string copy/part Data Width
Value: to-string copy/part Data Width
either With [
trim/with Value With
With: none
][
trim Value
]
)
]
| [
set Widths into [some integer!] (
Value: make string! 100
foreach Width Widths [
if not empty? Value [
append Value newline
]
append Value trim to-string copy/part Data Width
Data: next at Data Width
]
Width: 0
)
]
| [
'logic! (
Value: switch/default trim to-string copy/part Data Width [
"Y" [true]
"N" [none]
] [
none
]
)
]
| [
'char! (
Value: attempt [to-char trim to-string copy/part Data Width]
)
]
] (
Data: at Data 1 + Width
all [
series? Value
empty? Value
Value: none
]
Previous: associate? Values Field
either Previous: associate? Values Field [
either series? Previous [
append Previous Field
] [
associate-many Values Field Value
]
] [
associate Values Field Value
]
;print rejoin [Field ": " mold Value "."]
)
]
| [
set Value any-type! (
print rejoin ["Error: Couldn't understand: " mold Value "!"]
halt
)
]
]
end
]
reduce [Values Data]
]
-- Attached file included as plaintext by Listar --
-- File: Associate.r
Rebol [
Name: 'Associate
Title: "Associate"
File: %Associate.r
Author: "Andrew Martin"
eMail: [Al--Bri--xtra--co--nz]
Web: http://valley.150m.com
Date: 15/August/2002
Version: 1.2.0
Purpose: "Provides an associative memory store."
Category: [db util 4]
]
Associate?: function [
Association [series! port! bitset!]
Key [any-type!]
] [
Associated
] [
if found? Associated: select/only/skip Association :Key 2 [
first Associated
]
]
Associate: function [
Association [series! port! bitset!]
Key [any-type!]
Value [any-type!]
] [
Associated
] [
either found? Associated: find/only/skip Association :Key 2 [
either none? :Value [
remove/part Associated 2
] [
change/only next Associated :Value
]
] [
if not none? :Value [
repend Association [Key Value]
]
]
Association
]
Associate-Many: function [
Association [series! port! bitset!]
Key [any-type!]
Value [any-type!]
/Only "Appends a block value as a block."
] [
Associated
] [
if none? :Value [
return Associate Association :Key :Value
]
either found? Associated: Associate? Association :Key [
if not block? Associated [
Associated: reduce [Associated]
Associate Association :Key Associated
]
either Only [
insert/only tail Associated :Value
] [
insert tail Associated :Value
]
] [
associate Association :Key :Value
]
Association
]
Keys: func [Association [series! port! bitset!]][
extract Association 2
]
[2/3] from: greggirwin:mindspring at: 27-Aug-2002 17:55
Cool Andrew!
I had thought about something similar, as an old client of mine has lots of
COBOL data and some enticement would be good.
Since the logic! definition doesnt' take a size, it's limited to single char
values? Boolean/logic values are something that always give me a headache
because it seems like nobobdy could agree on what chars or strings to map
to. I think an old conversion function I wrote had about 8 different
identifiers for each state that I checked against. Going the other way, it
needed to know what you wanted to map the value to for export.
Thanks for posting that!
--Gregg
[3/3] from: al:bri:xtra at: 28-Aug-2002 17:15
Gregg wrote:
> Cool Andrew!
Thanks!
> Since the logic! definition doesnt' take a size, it's limited to single
char values? Boolean/logic values are something that always give me a
headache because it seems like nobobdy could agree on what chars or strings
to map to.
Currently, the DOS/Windows 3.1 application only use a single character to
store logic values, so I haven't implemented multicharacter versions. It's
fairly easy to add, I feel. Once you do so, please let me have a copy!
The latest version is attached (and I've sent a copy directly to you).
Andrew Martin
Super Cool Rebolutionary! :)
ICQ: 26227169 http://valley.150m.com/
-><-
-- Attached file included as plaintext by Listar --
-- File: Fixed.r
Rebol [
Name: 'Fixed
Title: "Fixed"
File: %Fixed.r
Author: "Andrew Martin"
eMail: [Al--Bri--xtra--co--nz]
Web: http://valley.150m.com
Date: 28/August/2002
Version: 1.1.0
Purpose: {Cuts up fixed width data file into Rebol values in an association.}
Category: [util 1]
]
Fixed: function [
{Cuts up fixed width data file into Rebol values in an association.}
Data [string! binary!] "The data file"
Dialect [block!] "The dialect controlling the cutting."
/Debug "Prints out the Fields and Values as they occur."
] [
Values Field Width Widths Value Type Previous With Line
] [
Values: make block! 200
parse Dialect [
any [
[
'skip (Width: 1) opt [set Width integer!] (
Data: next at Data Width
)
]
| [
set Field string! (Width: 1 Type: string!) [
[
[set Width integer! set Type 'binary!] (
Value: attempt [to-binary copy/part Data Width]
)
]
| [
[set Width integer! set Type ['issue! | 'integer! | 'date! | 'binary!]] (
Value: attempt [load trim to-string copy/part Data Width]
)
]
| [
[set Widths integer! 'char!] (
use [Block] [
Block: make block! Widths
loop Widths [
Value: trim to-string copy/part Data 1
Data: next Data
if not empty? Value [
append Block to-char Value
]
]
Value: Block
]
Width: 0
)
]
| [
['char!] (
Value: attempt [to-char trim to-string copy/part Data Width]
)
]
| [
['logic!] (
Value: switch/default trim to-string copy/part Data Width [
"Y" [true]
"N" [none]
] [
none
]
)
]
| [
[set Width integer! (With: none) opt [/with set With string!]] (
Value: to-string copy/part Data Width
either With [
trim/with Value With
][
trim Value
]
)
]
| [
[set Widths into [some integer!]] (
Value: make string! 100
foreach Width Widths [
Line: trim to-string copy/part Data Width
if not empty? Line [
append Line newline
]
Data: next at Data Width
append Value Line
]
Width: 0
)
]
] (
Data: at Data 1 + Width
Previous: associate? Values Field
either Previous: associate? Values Field [
either series? Previous [
append Previous Value
] [
if not empty? Value [
associate-many Values Field Value
]
]
] [
if any [
all [
series? Value
not empty? Value
]
not series? Value
] [
associate Values Field Value
]
]
if Debug [
print rejoin [Field ": " mold Value "."]
]
)
]
| [
set Value any-type! set Value1 any-type! (
print rejoin ["Error: Couldn't understand: " mold Value "&" mold Value1 "!"]
halt
)
]
]
end
]
reduce [Values Data]
]