AltME groups: search

When you import data using "data: read/binary {sometextfile}" you 
seem to get a string of hex values.  Ex:

probe 'data' of a file containg the word "Hello"  results in #{48656C6C6F} 
but if you 

probe first data it returns 72.  So when you probe the entire data 
stream it returns it in hexidecimal format
but when you probe each character it returns a decimal value.


At any rate how do you convert the characters in the variable 'data' 
back into ASCII values?  IOW, how do you convert the decimal value 
of 72 back into an "H" or the #{48656C6C6F} back into "Hello"?

Hi guys;
  Thanks for the input.


PeterWood & Anton:  I could have sworn that I tried both to-char 
and to-string.  It is the obvious answer, but I have been trying 
so many things in solving a parse problem that I missed it.  Now 
I remember, I did try them but at the time I complicated the character 
testing by using quotes and brackets and braces, or in the case of 
the hex string - not using the #{}.  Anyway, thanks for your time 
in answering. 


Gabriele:  As I was waking up this morning, I was thinking about 
modifying your CVS parser to make it work with (improperly) quoted 
strings.  That may be the simplier answer to my parsing problem.


MikeL: I started by not using the /binary but then the 'read' converts 
the #{0D0A} sequences to just #{0A} so I was going to try using the 
/binary option to preserve the original #{0D0A} and got sidetracked 
into changing the rest of the file back into a string.  Turns out 
that I will be going back to just using the 'read' without the /binary 
option and try modifying Gabriele's, CVS parser to handle improperly 
embedded quotes.

sint-to-hex: func [ smallint [integer!]][
    copy/part skip tail form to-hex smallint -2 2
]

rgb-to-hex: func [ c [tuple!]
    /local
][

    rejoin [ "#" sint-to-hex c/1 sint-to-hex c/2 sint-to-hex c/3 ]
]

to-hex 100
== #00000064

>> to-binary to-hex 100
== #{3030303030303634}

for going to-from hex... I don't have a solution.

maybe:
 skip form to-hex 232 6

A suggestion for an algorithm to produce a random seed from the time:


s: to integer! 2 ** 32 / 86400 * (to decimal! now/time/precise) - 
(2 ** 31)
s: enbase/base debase/base to-hex s 16 2
reverse s
s: to integer! to issue! enbase/base debase/base s 2 16
random/seed s

It does this:
1) convert now/time/precise to an integer using all possible bits
2) make a string of 0 and 1 from it
3) reverse the string
4) convert it back to an integer
5) And finally use it as a seed


Doing it this way, I hope to have a good span of possible start values 
for the random generator. Did I miss anything?

or easier for me to explain: you must search for the hex values in 
the file. work only in binary.

what I would do is get the offending chars from your file in a text 
editor and paste them in a separate file and save it in the same 
encoding as the original file. then read/binary it with rebol, and 
you can see the hex values directly.

to-hex <-> to-integer, except that's issue, not binary : (

i do this:
debase/base to-hex 256 16

>> debase/base skip  to-hex 256 4 16
== #{0100}

Steeve thanks  for the help, I think I can get bye with the debase/base 
to-hex 256 16

yes, made a string and then to-url. this helps if the caret is hex-encoded 
 %E5

$ ln hex-dump.r

ln: creating hard link `./hex-dump.r' to `hex-dump.r': File exists

-rw-r--r--  1 steves fusers  3657 Apr 16 14:38 hex-dump.r

I tried various HEX codes I found in google but none worked. Excel 
just printed strange chars instead of doing a line-break.

Izkata, it's a little tricky with R2, but this works for any integer:

>> enbase/base debase/base to-hex 546 16 2
== "00000000000000000000001000100010"

Because it was for R3 only, try this for R2:


cidr-match?: func [address [tuple!] network [tuple!] bits [integer!]] 
[

    address xor network and (to-tuple debase/base to-hex shift/left -1 
    32 - bits 16) = .0.
]

Don't know if it's faster than Andreas's, though

debase/base to-hex 255 16

>> debase/base to-hex 512 16
== #{00000200}

to-tuple debase/base skip to-hex 513 2 16

I don't know how it is stored in the DB, but it is bound to the DB 
as hex (base16) and stored as a 'Blob'.  As far as I'm aware, Ashley's 
driver will do this for you:

	sql ["INSERT INTO mytable VALUES (?)" #{ABCD}]

REBOL []

; r/3 = 'activate = left-click
; r/3 = 'activate = rt-click+menu-item-sel

hex: func [
    {Returns the base-10 value of a hexadecimal number.}
    value [integer! string! issue!] "A hexadecimal number"
][

    ; Convert to an issue first, so integers can also be translated.
    to integer! to issue! value
]

make-elements: func [name count type /local result][
    if not word? type [type: type?/word type]
    result: copy "^/"
    repeat i count [
        append result join name [i " [" type "]" newline]
    ]
    to block! result
]

NOTIFYICONDATA: make struct! compose [
    cbSize  [integer!]
    hwnd    [integer!]
    uId     [integer!]
    uFlags  [integer!]
    uCallBackMessage [integer!]
    hIcon   [integer!]
    (make-elements 'szTip 64 #"@")  ; CHAR
] none
NOTIFYICONDATA/cbSize: length? third NOTIFYICONDATA

;change at third NOTIFYICONDATA 25 "New ToolTip!"
;probe NOTIFYICONDATA
;halt

;constants required by Shell_NotifyIcon API call:
NIM_ADD:     hex 0
NIM_MODIFY:  hex 1
NIM_DELETE:  hex 2
NIF_MESSAGE: hex 1
NIF_ICON:    hex 2
NIF_TIP:     hex 4
WM_MOUSEMOVE:       hex 200
WM_LBUTTONDOWN:     hex 201  ;   'Button down
WM_LBUTTONUP:       hex 202  ;     'Button up
WM_LBUTTONDBLCLK:   hex 203  ; 'Double-click
WM_RBUTTONDOWN:     hex 204  ;   'Button down
WM_RBUTTONUP:       hex 205  ;     'Button up
WM_RBUTTONDBLCLK:   hex 206  ; 'Double-click


;Public Declare Function SetForegroundWindow Lib "user32" (ByVal 
hwnd As Long) As Long

lib: load/library %shell32.dll

Shell_NotifyIcon: make routine! compose/deep [
    dwMessage [integer!]
    pnid      [struct! [(NOTIFYICONDATA)]]
    return:   [integer!]
] lib "Shell_NotifyIconA"


my-hwnd?: does [second get-modes system/ports/system [window]]

set-tray-tooltip: func [struct string] [
    change at third struct 25 string
    struct
]



system-awake: func [port /local evt][
    if all [evt: pick port 1  (evt/1 = 'tray)] [
        status/text: mold evt
        show status
;         if any [
;             (evt/3 = 'activate)
;             all [(evt/3 = 'menu)  (evt/4 = 'desktop)]
;         ] [
;             if not desktop-loaded [
;                 link-exec-start-desktop/force
;             ]
;         ]
;         if all [(evt/3 = 'menu)  (evt/4 = 'quit)] [quit]
    ]
    false
]

system/ports/system/awake: :system-awake
append system/ports/wait-list system/ports/system

view layout [
    style button button 200
    button "Add Tray Menus" [
        set-modes system/ports/system compose/deep [
            tray: [
                add main [

                    help: (rejoin ["REBOL/Link" any [""]]) ; tooltip

                    menu: [test: "Test" desktop: "Start Desktop" bar quit: "Quit"]
                ]
                add other [
                    ;help: (rejoin ["REBOL/Link" any [""]])
                    menu: [test-2: "Test-2" bar quit-2: "Quit-2"]
                ]
            ]
        ]
    ]
    button "Remove Tray Main Menu" [
        set-modes system/ports/system [
            tray: [remove main]
        ]
    ]
    button "Remove Tray Other Menu" [
        set-modes system/ports/system [
            tray: [remove other]
        ]
    ]
    ;button "Change Tray Other Menu" [
    ;    set-modes system/ports/system [
    ;        tray: [
    ;            change other [
    ;                help: "New Help!"

    ;                menu: [test-3: "Test-3" bar quit-3: "Quit-3"]
    ;            ]
    ;        ]
    ;    ]
    ;]
    button "Modify Tooltip" [
        nid: make struct! NOTIFYICONDATA none
        nid/hwnd: my-hwnd?
        nid/uid: 1
        nid/cbSize: length? third nid
        nid/uFlags:  NIF_TIP  ; NIF_ICON
        ;nid/hIcon:
        ;nid/szTip:  "New ToolTip!^@"
        set-tray-tooltip nid "New ToolTip A!"
        ;print mold third nid
        res: Shell_NotifyIcon NIM_MODIFY nid
        print [res to logic! res]
    ]
    button "Modify Other Tooltip" [
        nid: make struct! NOTIFYICONDATA none
        nid/hwnd: my-hwnd?
        nid/uid: 2
        nid/cbSize: length? third nid
        nid/uFlags:  NIF_TIP  ; NIF_ICON
        ;nid/hIcon:
        ;nid/szTip:  "New ToolTip!^@"
        set-tray-tooltip nid "New ToolTip B!"
        ;print mold third nid
        res: Shell_NotifyIcon NIM_MODIFY nid
        print [res to logic! res]
    ]
    button "Unview" [unview]
    status: text 200
]



free lib

user32: context [
    dll: load/library %user32.dll

    HWND_BOTTOM:    1
    HWND_TOPMOST:   -1
    HWND_NOTOPMOST: -2
    HWND_TOP:        0
   
    SWP_SHOWWINDOW: to integer! #{0040} ;&H40
    SWP_NOSIZE:     to integer! #{0001} ;&H1
    SWP_NOMOVE:     to integer! #{0002} ;&H2
   

    GetActiveWindow: make routine! [return: [integer!]] dll "GetActiveWindow"
    SetWindowPos:    make routine! [
        hWnd [integer!]
        hWndInsertAfter [integer!]
        X [integer!]
        Y [integer!]
        cx [integer!]
        cy [integer!]
        uFlags [integer!]
        return: [integer!]
    ] dll "SetWindowPos"
   
    set 'arrange-window func [
        "Arrange window z-order."
        window [object!]
        mode [word!] "One of BOTTOM, NORMAL, TOP or TOP-MOST"
    /local
        hWnd
    ][
        window/changes: 'activate
        show window
        hWnd: GetActiveWindow

        SetWindowPos hWnd do select [bottom HWND_BOTTOM top HWND_TOP top-most 
        HWND_TOPMOST normal HWND_NOTOPMOST] mode

            0 0 0 0 to integer! to-hex SWP_SHOWWINDOW or SWP_NOSIZE or SWP_NOMOVE
       
    ]
    	SetWindowText: make routine! [
		handle			[integer!]
		Title			[string!]
		return:			[integer!]
	] dll "SetWindowTextA"
	
	set 'WindowTitle func [
		Title [string!] 
	] [
		SetWindowText get-modes system/ports/system 'window Title
	]	
]

R3 is a little wierd in its handling of large hex numbers - it displays 
them in scientific notation with limited precision instead of as 
integers.

That's not a hex number.
It's a REBOL pair.
For hex, try this:
  to-hex 333333333333

So to-hex is actually rather misleading.

Next, TO-HEX should probably not be removed, but it should return 
a string! instead.

A binary is only formatted with hex characters (if the binary-base 
is 16). A string would actually containthe hex characters themselves.

So to-hex would be shortcut for enbase/base to-binary num 16. Rather 
useless.

I expect that TO-HEX is a formatting function that is definitely 
not useless for web work, for example.

TO-HEX is a convenience function. Being lazy in a common situation 
is the whole point to convenience functions.

No bug. TO-HEX works as designed. Nothing to see here, move on people.

So something like:

>> d: make struct! [v [float]] none
>> i: to integer! (random 2 ** 32) - (2 ** 31) - 1
== -198913422
>> h: to-hex i
== #F424D272
>> change third d h
>> d/v
== 11532.55078125

why is it that those Hex error numbers got me all warm and fuzzy 
when I saw them?   :-)

At least the same syntax. not intended breaks like using $ for hex 
because orca will not need money! anyway.

we already use # in many datatypes, no? hex, binary, special notation 
of #[none], some possible binary conversion functions were suggested 
as 16#{} etc.

just one question about R3:

to convert a 32 bits integer to a binary serie i currently use that 
trick:
>>my-int: 150
>>debase/base skip tail to-hex my-int -8 16

it's quite inelegant and memory consuming (especially the to-hex 
function)
we need a more speed and compact function in R3.
I know another one trick using struct! but it's slow too.

To convert binaries to integers we don"t have such problem, cause 
[to integer! my-binary] works well and is short.

You don't need to "skip tail" :
>> debase/base to-hex 123456 16
== #{0001E240}

i need it cause my script works well with R3 alpha and R2 too.
to-hex returns longer strings in R3 than in R2

in know the trick with struct!, but my tests show that it's not faster 
than using [debase to-hex]

i don't know why but  it seems that it's not faster than the [debase 
to-hex] method

^(hex characters)

. The console may not render the character properly if the font doesn't 
support it though - it may look like a space.

Same as R2, but you can provide more hex characters.

Gabriele, I don't think, I got it wrong. From RFC 1738
http://rfc.sunsite.dk/rfc/rfc1738.html


The word "escape" is only used in the BNF in section 5. Escape is 
defined like:

escape         = "%" hex hex


That must be what you mean, when you gave the example of an url with 
the problem:
ftp://user%40email.com:[pass-:-ftp-:-domain-:-com]

When REBOL read that url, it convert %40 to the @ sign. Throughout 
RFC 1738, the word "encode" is used, when a character needs to be 
escaped. Like in this - taken from the RFC:


The user name (and password), if present, are followed by a
   commercial 
at-sign 
@". Within the user and password field, any ":",
   "@", or "/" must be encoded."


So "encoded" mean escaping the characters, right? I think, that's 
how I used the word in my comments here. If you think, I got it wrong, 
could you explain it to me then? I would like to get this right.

Peter, in response to the suggestions in your last message:

- issue! = binary! : not good, at least in R3. Perhaps issue! = to-hex 
binary!

- integer! = binary! : not good, at least in R3. Use integer! = to-integer 
binary!


Actually, anything-but-binary! = binary! is a bad idea in R3, since 
encodings aren't assumed. The TO whatever conversion actions are 
good for establishing what you intend the binary! to mean though, 
especially since extra bytes are ignored - this allows binary streams.

Because historically I am far from being alone, who finds current 
R2 working as being complete mess. How can anyone defend it? I can 
easily tell you, where it comes from. I am far from good at wrapping 
libraries, but I wrapped some functionality in the end. I can often 
see constants like:

0x0010

Which are for me equivalent to:

#0010  ; which I can get via to-hex or to-issue 16

#{0010} ; to-binary 16 = #{3136} .... and this is big WTF? Now at 
least R3 gets it correctly, although padded by full of zeros to form 
64  bit value


So, as you can see, my thought pattern came from not easy way of 
how to convert between integer and binary representation. I created 
my own crazy 2-binary function :-)


to-binary*: func [value [integer!]][load rejoin ["#{" to-string copy/part 
at to-hex value 7 2 "}"]]

So - stating above in regards to R2, my further thoughts went towards 
hex being a literal representation of binary values. But I forgot 
there is no hex! type, just an issue! type, which serves also other 
purpose. Now as to-binary 16 gives me desired value, I don't need 
to go via to-hex anymore ...

Datatype conversions: I think that once TO-HEX is removed for most 
datatypes the conversion issues of the TO-* set will be done. The 
rest will be handled by proper conversion functions, that we don't 
need to write immediately. We should probably wait on implementing 
those as natives until the APIs are worked out in REBOL versions, 
or plugin code. We can speed them up later once their behavior is 
agreed on.

so since we export some words and extension mezz are essentially 
unnamed modules... their content really is hidden and can't be extracted 
  :-)

although I guess scanning the extension with a hex editor will reveal 
the code inside!?

You are right about a hex editor seeing the REBOL source though, 
unless you do something weird to generate the source instead of referencing 
a literal string. However, remember that the code in memory is the 
result of executing the code in the module source, generating the 
in-memory data. That generation can be really elaborate if you like.

what's the best way to convert a hex string to a decimal value in 
R3?

One way is to start with an issue! rather than hex
    >> to-integer #100
    == 256

Maxim: "what's the best way to convert a hex string to a decimal 
value in R3?" - Try this:
>> pi
== 3.14159265358979
>> enbase/base to-binary pi 16
== "400921FB54442D18"
>> to-decimal debase/base "400921FB54442D18" 16
== 3.14159265358979
>> to-decimal debase/base enbase/base to-binary pi 16 16
== 3.14159265358979


You asked for the best way: No method that uses the issue! type for 
binary conversions could be considered the best way.

The problem is that while the scheme might not represent anything 
network-related, the standard for URI syntax is independent of network 
issues. And that standard is pretty strict about hex encoding, regardless 
of the scheme's internal rules. So schemes need to be hex-encoding-aware 
for their specs, whether they are network-related or not.

but I remember having the same issue a while back and traced it to 
the actual datatype always handling the hex values.

Um, no. The HTTP standard for basic authentication doesn't hex-encode 
the user or password fields. The browser (or in our case, http scheme) 
does.

Only the path is hex-encoded when passed to the server.

As parse rules go, it wouldn't be difficult. Try this:
>> hex: charset [#"0" - #"9" #"a" - #"f" #"A" - #"F"]
== make bitset! #{000000000000FFC07E0000007E}

>> parse a: "paul%40tretbase.com" [any [to "%" [b: skip copy x 2 
hex (b: change/part b to-char first debase/base x 16 3) :b | skip]]] 
a
== "[paul-:-tretbase-:-com]"

Now that is a modifying method, but it should be easy to adapt that 
to a copying method.

With the full change rule it would be this:

>> parse a: "paul%40tretbase.com" [any [to "%" [change [skip copy 
x 2 hex] (to-char first debase/base x 16) | skip]]] a

And I was wrong, the Python in your example was not operating on 
binaries at all, it was operating on integers that were specified 
in hex syntax, which is a completely different thing that REBOL has 
no support for at all. Not the same thing.

10 represents the number, but it can be stored in a variety of ways.


0x800 represents the number, just in another notation.  like 10  
is written   110   in binary  or   A   in hex

Pekr, you do realize that TO-INTEGER #{8000} is a conversion of an 
incomplete binary, an operation, right? And that 0x8000 is syntax 
for an integer value? REBOL doesn't have hex syntax for integers, 
or any default interpretation of binary values as being of a different 
datatype. Just like Python doesn't have syntax for binary values 
(unless I'm mistaken about that last bit).

>> ; R2 code converting integer -1 to 32-bit binary
>> debase/base to-hex -1 16
== #{FFFFFFFF}
; R3 code converting the said binary to integer
>> to integer! #{FFFFFFFF}
== 4294967295


As far as I am concerned, it looks incompatible to me, and I would 
prefer -1 to be the result of the conversion in R3

R3:

>> to-hex -1
== #FFFFFFFFFFFFFFFF

>> to-integer #{FFFFFFFFFFFFFFFF}
== -1

Max, you still can use digit as the first char. From changelog:

The representation of numeric values (invalid word forms) is still 
supported, so usage for HTML colors or hex numbers, and conversion 
to and from tuples and integers, still works.

the whole bitwise thing is pretty fucked up anyway. i tried to do 
a disk editor, a pic microcontroller HEX file processor, a custom 
serial communication protocol and in all cases i had to ping-pong 
between binary! issue! integer! and had to trim to the right bit/byte 
counts. it was a nightmare all the time.

>> x: [16#ffffff] 
== [#6#ffffff]

how can i specify an integer! in hex format?

debase/base "ffffff" 16  returns a binary! which i mostly can smear 
on my hair, since most operators just doesn't play nicely w it...

same problem again... i tried to use rebol for byte level processing 
and it's just not suitable for it.. :/

here is my ObjectID routine a'la mongodb.

wondering how much simpler could it be in r3?...  not that i could 
use r3 any time soon for production stuff, but i would love to, of 
course

  rejoin probe reduce [
    to-hex date-to-epoch now

    enbase/base copy/part checksum/method system/network/host 'md5 3 
    16
    skip to-hex access-os 'pid 4
    skip to-hex random/secure to-integer #ffffff 2
  ]

This code might be a better test: repeat i to-integer #7fffffffffffffff 
[if zero? i // 1'000'000 [recycle] to-hex i]

It should have less memory usage overall and if words are recycled 
then it won't run out. I'll run it now.

TO-HEX generates an issue!, which is a word type in R3. Yes, you 
can even bind them.

R2 way is to use to-hex
    to-hex 100
    == #00000064

>> to integer! binary: debase/base to-hex integer: 11 16
== 11

It's an incomplete list too. Here is a complete list of the control 
characters (not including the keywords or hex in parens):

>> print mold collect [for x 0 255 1 [if 4 < length? mold to-char 
x [keep to-char x]]]

[#"^@" #"^A" #"^B" #"^C" #"^D" #"^E" #"^F" #"^G" #"^H" #"^-" #"^/" 
#"^K" #"^L" #"^M" #"^N" #"^O" #"^P" #"^Q" #"^R" #"^S" #"^T" #"^U" 
#"^V" #"^W" #"^X" #"^Y" #"^Z" #"^[" #"^\" #"^]" #"^!" #"^_" #"^"" 
#"^^" #"^~"]

I was also thinking that you might want to extend the integer qualification 
you added for hex notation to the other integer sizes, when they 
are implemented.  ex: 

33L
33s
33b

etc

Right, that's something to consider too. But I've often found such 
notation a bit hard to read and remember. Also to consider: hex syntax 
opens a hole in the "no digit as first character in words" rule. 
I hoped that I wouldn't have to break that rule again for other literals.

Regarding hex form of integer. What if someone write:

ah: 42
foo: ah

Should foo be 42 or 10 (= ah hex)?
Maybe hex integers should require prefix zero in this case?

Also, I have an item in the todo list for "Hex values syntax checking". 
I guess I'll add some rules to avoid ambiguities as much as possible. 
For example, having to use uppercases only for hex letters would 
help. Allowing only 8 or 6 or 4 characters with prefix zeros when 
required should also help.

Is Red proper going to have this hex notation, as well?

I fear it is going to lead to problems if not all words are available 
to use in a context. Suppose there is an external source of data 
with named values that map naturally to a context. The ability to 
import that data easily will be broken if certain words are interpreted 
as hex values

The examples in 10.1 and 10.2 use hex numbers in lowercase

Hex lowercased: thanks for reporting, fixing that for next spec release.

I observe a strange behaviour at the compilation of this code
#import [
                "user32.dll" stdcall [
                               OemToChar: "OemToCharA" [

                                               in                            [c-string!] 

                                               out                        [c-string!] 
                                               return: [integer!]
                                               ]
                ]] 
error message :

Compiling tests/readConsole.reds ...
** User Error: Invalid hex literal: Char: "OemToCharA" [
            in      [c-string!

** Near: make error! reform ["Invalid hex literal:" copy/part s 40]

It seems to be linked to the names used here. Is it a parsing problem 
?

- Reimplemented bitset! as binary
- Added native function: COMPLEMENT
- Added native function: ROTATE
- Added native function: SHIFT
- Added << and >> operators to cortex.w
- Added hex form for characters, ^(00) - ^(FF)
- Added REFORM to rebol.w
- Added DETAB to rebol.w
- Added ENTAB to rebol.w
- New test
- Bugfixes

Both R2, R3

escape-uri: [#"%" 2 hex-digit]
email-char: complement union charset {%@:} termination-char
email-syntax: [
	[#":" | not #"<" email-char | escape-uri]
	any [email-char | escape-uri]
	#"@"
	any [email-char | escape-uri]
	termination
]

Corrected version, works with R2 and R3:

escape-uri: [#"%" 2 hex-digit]
email-char: complement union charset {%@:} termination-char
email-esc: [email-char | escape-uri]
email-syntax: [
	[
		#":" any [email-esc | #":" ] #"@" any [email-esc | #":" ]
		| not #"<" some email-esc #"@" any email-esc
	]
	termination
]