Mailing List Archive: 49091 messages
  • Home
  • Script library
  • AltME Archive
  • Mailing list
  • Articles Index
  • Site search
 

do %image.png

 [1/12] from: jasonic:panix at: 14-Nov-2001 14:47


do %image.png My objective is to implement a roundtrip toolkit with /Core and /View interfaces for embedding, extracting and running REBOL scripts embedded in images. Then the real fun can begin! I have made a little progess: A tiny hardcoded test is now working :-) buddha.png is a PNG image file with a REBOL script embedded in it. http://www.postcardscience/rebol/buddha.png I used TweakPNG (Win32only) to learn about PNG and inset my REBOL script. http://entropymine.com/jason/tweakpng/ By design I insert a special keyword "REBOLscript" after the "tEXt" chunk to help parsing. Also, a special REBOL end of script marker ";EOF" at the end of test script itself. The helps parsing the REBOL script end from other data or padding in the PNG file. It takes about 6 seconds to store 29kb image file as a string on a 500Mhz PIII Win98se Vaio laptop. Lots of features and funtions to implement. But first I need some basic advice: Q1. How can I improve the time to load PNG into a rebol string? Q2. What other approach do you suggest than the one I am following? thanks for any feedback - Jason ======================== REBOL [ File: %buddha.r] ;download or read from http://www.postcardscience/rebol/buddha.png buddha: read/binary %buddha.png b: "" for i 1 length? buddha 1 [append b to-char buddha/:i] ; 7 seconds for 29Kb image on 500MhzPIII ; head b ; make sure we are at the beginning when debugging in REBOL shell search-key: "tEXtREBOLscript" ; better than just looking for "REBOL" header-key: "REBOL " ; space after REBOL should mean it's the header tail-key: ";EOF" ; by design, this marks end of REBBOl script ; later probably will want to include #!path-to-rebol and find those first b: find next b search-key ; start of our tEXt chunk containing REBOL script b: find next b header-key ; start of the script start-mark: length? b ; how many bytes to end of PNG file bend: find next b tail-key ; end of script end-mark: length? bend ; how many bytes to chop off script-length: start-mark - end-mark ; how many bytes to use of 'script script: "" for i 1 script-length 1 [append script b/:i] ; chop the end off so we have pure script write %buddha-script.r script ; save it to file do %buddha-script.r ; test it!

 [2/12] from: ryanc:iesco-dms at: 14-Nov-2001 12:21


Should be even easier than that. Just insert something like this into it... REBOL[] confirm "This is a test." halt Just make sure you have the code enveloped in linefeeds. REBOL was designed to do this already. --Ryan Jason Cunliffe wrote:
> do %image.png > My objective is to implement a roundtrip toolkit with /Core and /View
<<quoted lines omitted: 46>>
> [rebol-request--rebol--com] with "unsubscribe" in the > subject, without the quotes.
-- Ryan Cole Programmer Analyst www.iesco-dms.com 707-468-5400 The contradiction so puzzling to the ordinary way of thinking comes from the fact that we have to use language to communicate our inner experience which in its very nature transcends lingistics. -D.T. Suzuki

 [3/12] from: jasonic:panix at: 14-Nov-2001 15:40


> buddha.png is a PNG image file with a REBOL script embedded in it. > http://www.postcardscience/rebol/buddha.png
oops.. sorry try: http://www.postcardscience.net/rebol/buddha.png ^^^ ./Jason

 [4/12] from: jasonic:panix at: 14-Nov-2001 15:37


Ryan Cole <[ryanc--iesco-dms--com]> wrote
> Should be even easier than that. Just insert something like this into
it...
> REBOL[] > confirm "This is a test." > halt > > Just make sure you have the code enveloped in linefeeds. REBOL was
designed to
> do this already.
Hi Ryyan I don't quite understand you. ..what "this" REBOL was design to do ? How will wrapping my embedded script in linefeeds help? How will I know I am at the end of my script and at some space in the middle of it? thanks - Jason

 [5/12] from: ryanc:iesco-dms at: 14-Nov-2001 14:29


Hey Jason, I was mistaken in that my last example only worked well in a "prefaced" situation. What you really want is this: garblyjunkbinaryfile [ REBOL [] confirm "This is a test." ] garblyjunkbinaryfile There is a problem I discovered, a bug probably. Seems that the null characters in the png file are disrupting REBOL's ability to trigger. Without nulls present, the embedding works fine. Example:
>> write/binary %test "^@^/[ rebol [] confirm {Works?} ]" >> do %test
** Syntax Error: Script is missing a REBOL header ** Where: do-boot ** Near: do %test
>> write/binary %test " ^/[ rebol [] confirm {Works?} ]" >> do %test
== true
>>
I sent a feedback, hopefully they get around to fixing it soon, --Ryan Jason Cunliffe wrote:
> "Ryan Cole" <[ryanc--iesco-dms--com]> wrote > > Should be even easier than that. Just insert something like this into
<<quoted lines omitted: 19>>
> [rebol-request--rebol--com] with "unsubscribe" in the > subject, without the quotes.
-- Ryan Cole Programmer Analyst www.iesco-dms.com 707-468-5400 The contradiction so puzzling to the ordinary way of thinking comes from the fact that we have to use language to communicate our inner experience which in its very nature transcends lingistics. -D.T. Suzuki

 [6/12] from: pwoodward:cncdsl at: 14-Nov-2001 17:40


Hey - have you checked out the PNG spec? It's pretty clear about how the file is arranged. http://www.w3.org/TR/REC-png Basically using a combination of read/part/skip you should be able to navigate thru the various chunks to get to the chunk you want. The first 4 bytes of a chunk indicate its length, and the next 4 indicate the chunk type. So - after reading the first 8 bytes of the file to verify it's a PNG, read the next 8 bytes and take a look at them to see if the next chunk is a "tEXt" chunk - if not use the length value as the skip in a read... You'll have to accumulate bytes read and what not though. But - it'd avoid you having to scan thru the file for an identifier. - Porter

 [7/12] from: jasonic::panix::com at: 15-Nov-2001 8:27


Thanks everyone... Ryan and Gabriele both suggested I put my script in a block, and Ryan noted a bug where that the outer block needs to be preceeded by "^/" or #{0A}. Unfortunately this strategy still does not work inside a binary file. Nice try ;-) Maybe next version ??
>> do %buddha_inablock.png
** Syntax Error: Script is missing a REBOL header ** Near: do %buddha_inablock.png Porter suggested using a more sophisticated use of "read", to avoid need to first load the entire image binary into a string as I do now. To recap, this took about 6 seconds for a 29Kb PNG image on 500Mhz PIII. Once loaded like this, it was very fast and flexible to find my script.
> Basically using a combination of read/part/skip you should be able to > navigate thru the various chunks to get to the chunk you want. The first
4
> bytes of a chunk indicate its length, and the next 4 indicate the chunk > type. So - after reading the first 8 bytes of the file to verify it's a > PNG, read the next 8 bytes and take a look at them to see if the next
chunk
> is a "tEXt" chunk - if not use the length value as the skip in a read... > You'll have to accumulate bytes read and what not though. But - it'd
avoid
> you having to scan thru the file for an identifier.
On first impressions, this does seem to be much faster hybrid approach. It should really help with larger images. Now I just need to become more skilled in my use of read/binary/part/skip :-) I think storing special PNG 'tEXt' chunk keyword like "REBOLscript" before the REBOL[] script block is a good strategy. Consistent with PNG spec, which anticipates common keyword descriptors [Title, description, ..], but also specialized and highly individual ones. The keyword naming scheme could be extended to allow multiple scripts in the same PNG file: REBOLscript0000 ,"REBOLscript0001", ... "REBOLscript9999" or maybe more poetic choices. For some uses this would be overkill, but at a minimum I can see argument for using 2 distinct scripts, a default 'Image-system.r' and a 'Image-script.r'. The first would be probably be much better defined, associated with security, and the the outer mechanics for transport, parse, edit, display functions. The second script would be freer according to whatever the application/context. There woudl be some filesize bloat, so hwere that was critical one might try comprssion and bundling. Imagine an image which is widely shared, among a group of system or individual people. Each of thse might want to have their own small script space inside. Much like VANILLA snip/dynasnip scheme, which is where I am going to use this initially. Instead of multiple scipts,it might be better to have lots of objects inside a snigle script. This wuold seem to afll inline faster with Vanilla's structuer, and also adapt nicely to xmlrpc programming. No reason that eveything has to be REBOL either, its #1 choice here. Python is a good candidate too. - Jason

 [8/12] from: ingo:2b1 at: 15-Nov-2001 10:18


Hi Jason, Ryan, Once upon a time Jason Cunliffe spoketh thus:
> "Ryan Cole" <[ryanc--iesco-dms--com]> wrote > > Should be even easier than that. Just insert something like this into
<<quoted lines omitted: 10>>
> I don't quite understand you. > ..what "this" REBOL was design to do ?
Rebol was designed to be run when imbedded into other data, e.g. you can run a script directly from an email, without having to clean it first. So, you can 'do %image.png if the script itself is stored in normal ASCII within the binary data, but it Won't work the way Ryan suggested, the way to go is, embed the whole script into a block, like this [REBOL [] print { You can save this email, e.g. to %mymail.r, and 'do %mymail.r in a REBOL session. } ]
> How will wrapping my embedded script in linefeeds help? > How will I know I am at the end of my script and at some space in the middle > of it?
(IIRC, the rules are: find newline followed be "REBOL" and a block --> starting from REBOL unto the end is a rebol script find newline followed by "[REBOL" and a block --> the script ends at the closing bracket I hope that helps, Ingo

 [9/12] from: jasonic:panix at: 15-Nov-2001 11:36


Hi Ingo
> Rebol was designed to be run when imbedded into other data, > e.g. you can run a script directly from an email, without > having to clean it first.
within ascii text yes.
> So, you can 'do %image.png if the script itself is stored in > normal ASCII within the binary data, but it Won't work the way
<<quoted lines omitted: 6>>
> } > ]
I tried that with and wihout linefeeds as well. No luck. AS far as I can see, REBOL *is* expecting to parse a REBOL block from wihtin ascii text. Are you saying REBOL "do" currently parses a binary automatically, and looks for text containing script in a block ? ^/[REBOL[]print "here's looking at you from inside image.png"] I would be surprised and very delighted if this is true now. Somebody show me an example please! Emanehiel it seems we must build it. But I hope an efficient implementation of this will become true in a version of REBOL soon: do/binary %image.png The /binary refinement is what seems to be missing. Because it is a special case, one would not want to slow down the default "do" behavior. It could reasonably be an implicit /binary, which is invoked as soon as do sees it is dealing with something that is not a .r file: <wishlist examples> do %somescript.r do %image.png ; runs a default implied "do/binary" refinement do/binary %image.png png-options png-options might load a config file "binary-png.r" similar to user.r for tuning the response of do/binary do/binary %tracks.midi midi-options would handle that format etc.. </wishlist>
> (IIRC, the rules are: > find newline followed be "REBOL" and a block > --> starting from REBOL unto the end is a rebol script > find newline followed by "[REBOL" and a block > --> the script ends at the closing bracket
demo please? does not work for me.. thanks -Jason

 [10/12] from: ryanc:iesco-dms at: 15-Nov-2001 9:37


<SNIP> > Are you saying REBOL "do" currently parses a binary automatically, and looks > for text containing script in a block ? > > ^/[REBOL[]print "here's looking at you from inside image.png"] > > I would be surprised and very delighted if this is true now. > Somebody show me an example please!
</SNIP> Yes this is true. Here is your example below. Unfortuneately I could not find any good binary candidates because they had nulls in the beginnings, so I just demonstrated at the console.
>> test: ""
== ""
>> for I 1 255 1 [ append test to-char i ] ; Must skip null because of bug.
== {^A^B^C^D^E^F^G^H^- ^K^L^M^N^O^P^Q^R^S^T^U^V^W^X^Y^Z^[^\^]^!^_ !"#$%&'()*+,-./0123456789:;<=>?@A BCDEFGHIJKLMNOPQRSTUVWXYZ[\]^^_`...
>> code: "^/[rebol[] confirm {test}]"
== "^/[rebol[] confirm {test}]"
>> append test code
== {^A^B^C^D^E^F^G^H^- ^K^L^M^N^O^P^Q^R^S^T^U^V^W^X^Y^Z^[^\^]^!^_ !"#$%&'()*+,-./0123456789:;<=>?@A BCDEFGHIJKLMNOPQRSTUVWXYZ[\]^^_`...
>> for I 0 255 1 [ append test to-char i ] ; Can have nulls after the code.
== {^A^B^C^D^E^F^G^H^- ^K^L^M^N^O^P^Q^R^S^T^U^V^W^X^Y^Z^[^\^]^!^_ !"#$%&'()*+,-./0123456789:;<=>?@A BCDEFGHIJKLMNOPQRSTUVWXYZ[\]^^_`...
>> write/binary %test test >> do %test
== true
>>
My recommendation is to just write a function that would works like 'do is supposed to. This way when RT fixes the bug you can just switch over to using 'do. --Ryan

 [11/12] from: ryanc:iesco-dms at: 15-Nov-2001 10:16


Jason Cunliffe wrote:
<SNIP> > <wishlist examples>
<<quoted lines omitted: 5>>
> do/binary %tracks.midi midi-options would handle that format etc.. > </wishlist
As my example demonstrated, it is just an issue with nulls in the binary, all other characters work fine. I would'nt think a /binary switch would be necessary, RT probably just needs to use a string functions that dont stop upon reaching null characters. Also, I haven't seen this in recent docs, but you can alternatively use <script> tags to embed REBOL. This is actually how I knew that it could be done. When trying to find docs on exactly how it worked, I ran across the block embedding trick in the Core PDF. Anyway, here is an example using script tags. test: "" for I 1 255 1 [ append test to-char i ] append test "^/<script>^/rebol[] confirm {test}^/</script>^/" for I 0 255 1 [ append test to-char i ] write/binary %test test do %test Note that the <script> tags are a bit pickier than the blocks, and must be surounded by linefeeds. Also, since the seemingly lack of docs on using tags, I am suspicious if RT is going to discontinue tag capability. --Ryan

 [12/12] from: jasonic::panix::com at: 15-Nov-2001 16:00

Re: do %image.png =>YES!!!


> > I would be surprised and very delighted if this is true now. > > Somebody show me an example please! > > </SNIP> > > Yes this is true. Here is your example below. Unfortuneately I could not
find
> any good binary candidates because they had nulls in the beginnings, so I
just
> demonstrated at the console.
Ryan: yes WOW thanks :-)) I feel incredibly enlightened ++REBOL has shot up many cool knotches in my universe. I have spent all this time trying doh level complexidity: read/binary %image.png ...append to-char ...find ...write ...read do WRONG!! Pass that envelope please ==> the answer is: KISS !!!! dimage: read %image.png dscript: find dimage "REBOL [" do dscript BINGO: forget read/binary just read %binary Does not even seem to need the line feeds or outer block, though may be good style. next-job: writing/inserting dscript into dbinary files... Jason: [happy reboler taliban left town gonna fly a kite tonight]

Notes
  • Quoted lines have been omitted from some messages.
    View the message alone to see the lines that have been omitted