Mailing List Archive: 49091 messages
  • Home
  • Script library
  • AltME Archive
  • Mailing list
  • Articles Index
  • Site search
 

Fixed (width) dialect

 [1/3] from: al::bri::xtra::co::nz at: 27-Aug-2002 21:58


Here's the Fixed dialect I've been working on. It's a dialect for pulling apart fixed width data files and translating the strings into appropriate Rebol values. With the use of my Associate functions (included), the dialect can even combine split up values. Here's an example of the dialect (with comments), which I'm using to cut up a DOS/Windows 3.1 database that my local high school uses: Fields: [ "Family Name" 20 ; Column name is a string!, width of the field in characters is a integer!. "First Name" 25 "Preferred" 12 "Mail to Whom" 35 "Invoices?" logic! ; Converts "Y"/"N into logic value, no need for size as it's automatic. "Reports?" logic! "Address" [25 20 20] ; multiple lines automatically converted to one Rebol string! "Telephone" 19 /with " -()" ; refinement to 'trim function, to get rid of extraneous characters in 'phone number. "Cellphone" 7 /with " -()" "Nationality" 3 "Language" 2 skip 4 "Year" 2 integer! ; Converts integer text into integer value. "Form" 4 skip 4 "Gender" char! ; Same for char! values. "Birth" 8 date! ; and for date! values. The "8" specifies how much text to use for the date. "Enrollment" 5 issue! skip ; Skip a single character "First Started" 8 date! "First Attended" 8 date! skip 3 ; Skip several characters. "Status" char! "Magic Marker" 3 binary! ; Convert a binary value (key in BASIC data). skip 2 ] Andrew Martin ICQ: 26227169 http://valley.150m.com/ -><- -- Attached file included as plaintext by Listar -- -- File: Fixed.r Rebol [ Name: 'Fixed Title: "Fixed" File: %Fixed.r Author: "Andrew Martin" eMail: [Al--Bri--xtra--co--nz] Web: http://valley.150m.com Date: 27/August/2002 Version: 1.0.0 Purpose: {Cuts up fixed width data file into Rebol values in an association.} Category: [util 1] ] Fixed: function [ {Cuts up fixed width data file into Rebol values in an association.} Data [string! binary!] "The data file" Dialect [block!] "The dialect controlling the cutting." ] [ Values Field Width Widths Value Type Previous With ] [ Values: make block! 200 parse Dialect [ any [ [ 'skip (Width: 1) opt [set Width integer!] ( Data: next at Data Width ) ] | [ set Field string! (Width: 1 Type: string!) [ [ set Width integer! set Type [ 'issue! | 'integer! | 'date! | 'binary! ] ( Value: attempt [load trim to-string copy/part Data Width] ) ] | [ set Width integer! opt [/with set With string!] ( ;Value: trim to-string copy/part Data Width Value: to-string copy/part Data Width either With [ trim/with Value With With: none ][ trim Value ] ) ] | [ set Widths into [some integer!] ( Value: make string! 100 foreach Width Widths [ if not empty? Value [ append Value newline ] append Value trim to-string copy/part Data Width Data: next at Data Width ] Width: 0 ) ] | [ 'logic! ( Value: switch/default trim to-string copy/part Data Width [ "Y" [true] "N" [none] ] [ none ] ) ] | [ 'char! ( Value: attempt [to-char trim to-string copy/part Data Width] ) ] ] ( Data: at Data 1 + Width all [ series? Value empty? Value Value: none ] Previous: associate? Values Field either Previous: associate? Values Field [ either series? Previous [ append Previous Field ] [ associate-many Values Field Value ] ] [ associate Values Field Value ] ;print rejoin [Field ": " mold Value "."] ) ] | [ set Value any-type! ( print rejoin ["Error: Couldn't understand: " mold Value "!"] halt ) ] ] end ] reduce [Values Data] ] -- Attached file included as plaintext by Listar -- -- File: Associate.r Rebol [ Name: 'Associate Title: "Associate" File: %Associate.r Author: "Andrew Martin" eMail: [Al--Bri--xtra--co--nz] Web: http://valley.150m.com Date: 15/August/2002 Version: 1.2.0 Purpose: "Provides an associative memory store." Category: [db util 4] ] Associate?: function [ Association [series! port! bitset!] Key [any-type!] ] [ Associated ] [ if found? Associated: select/only/skip Association :Key 2 [ first Associated ] ] Associate: function [ Association [series! port! bitset!] Key [any-type!] Value [any-type!] ] [ Associated ] [ either found? Associated: find/only/skip Association :Key 2 [ either none? :Value [ remove/part Associated 2 ] [ change/only next Associated :Value ] ] [ if not none? :Value [ repend Association [Key Value] ] ] Association ] Associate-Many: function [ Association [series! port! bitset!] Key [any-type!] Value [any-type!] /Only "Appends a block value as a block." ] [ Associated ] [ if none? :Value [ return Associate Association :Key :Value ] either found? Associated: Associate? Association :Key [ if not block? Associated [ Associated: reduce [Associated] Associate Association :Key Associated ] either Only [ insert/only tail Associated :Value ] [ insert tail Associated :Value ] ] [ associate Association :Key :Value ] Association ] Keys: func [Association [series! port! bitset!]][ extract Association 2 ]

 [2/3] from: greggirwin:mindspring at: 27-Aug-2002 17:55


Cool Andrew! I had thought about something similar, as an old client of mine has lots of COBOL data and some enticement would be good. Since the logic! definition doesnt' take a size, it's limited to single char values? Boolean/logic values are something that always give me a headache because it seems like nobobdy could agree on what chars or strings to map to. I think an old conversion function I wrote had about 8 different identifiers for each state that I checked against. Going the other way, it needed to know what you wanted to map the value to for export. Thanks for posting that! --Gregg

 [3/3] from: al:bri:xtra at: 28-Aug-2002 17:15


Gregg wrote:
> Cool Andrew!
Thanks!
> Since the logic! definition doesnt' take a size, it's limited to single
char values? Boolean/logic values are something that always give me a headache because it seems like nobobdy could agree on what chars or strings to map to. Currently, the DOS/Windows 3.1 application only use a single character to store logic values, so I haven't implemented multicharacter versions. It's fairly easy to add, I feel. Once you do so, please let me have a copy! The latest version is attached (and I've sent a copy directly to you). Andrew Martin Super Cool Rebolutionary! :) ICQ: 26227169 http://valley.150m.com/ -><- -- Attached file included as plaintext by Listar -- -- File: Fixed.r Rebol [ Name: 'Fixed Title: "Fixed" File: %Fixed.r Author: "Andrew Martin" eMail: [Al--Bri--xtra--co--nz] Web: http://valley.150m.com Date: 28/August/2002 Version: 1.1.0 Purpose: {Cuts up fixed width data file into Rebol values in an association.} Category: [util 1] ] Fixed: function [ {Cuts up fixed width data file into Rebol values in an association.} Data [string! binary!] "The data file" Dialect [block!] "The dialect controlling the cutting." /Debug "Prints out the Fields and Values as they occur." ] [ Values Field Width Widths Value Type Previous With Line ] [ Values: make block! 200 parse Dialect [ any [ [ 'skip (Width: 1) opt [set Width integer!] ( Data: next at Data Width ) ] | [ set Field string! (Width: 1 Type: string!) [ [ [set Width integer! set Type 'binary!] ( Value: attempt [to-binary copy/part Data Width] ) ] | [ [set Width integer! set Type ['issue! | 'integer! | 'date! | 'binary!]] ( Value: attempt [load trim to-string copy/part Data Width] ) ] | [ [set Widths integer! 'char!] ( use [Block] [ Block: make block! Widths loop Widths [ Value: trim to-string copy/part Data 1 Data: next Data if not empty? Value [ append Block to-char Value ] ] Value: Block ] Width: 0 ) ] | [ ['char!] ( Value: attempt [to-char trim to-string copy/part Data Width] ) ] | [ ['logic!] ( Value: switch/default trim to-string copy/part Data Width [ "Y" [true] "N" [none] ] [ none ] ) ] | [ [set Width integer! (With: none) opt [/with set With string!]] ( Value: to-string copy/part Data Width either With [ trim/with Value With ][ trim Value ] ) ] | [ [set Widths into [some integer!]] ( Value: make string! 100 foreach Width Widths [ Line: trim to-string copy/part Data Width if not empty? Line [ append Line newline ] Data: next at Data Width append Value Line ] Width: 0 ) ] ] ( Data: at Data 1 + Width Previous: associate? Values Field either Previous: associate? Values Field [ either series? Previous [ append Previous Value ] [ if not empty? Value [ associate-many Values Field Value ] ] ] [ if any [ all [ series? Value not empty? Value ] not series? Value ] [ associate Values Field Value ] ] if Debug [ print rejoin [Field ": " mold Value "."] ] ) ] | [ set Value any-type! set Value1 any-type! ( print rejoin ["Error: Couldn't understand: " mold Value "&" mold Value1 "!"] halt ) ] ] end ] reduce [Values Data] ]