Using %build-pack.r

Source Code by Carl Sassenrath, REBOL Technologies
Usage document by Brian Tiffin
28-Jun-2007

Contents:

1. Introduction to %build-pack.r

This script creates a single compressed REBOL program from a list of files. File can include binary data files such as images.

The output from build-pack.r is plain text so it can be opened in any text editor or sent as an email message.

2. Using %build-pack.r

build-pack.r is an example of a rebol.org repositoryHow-To. That means that it shows how to do something useful, but may require changes for it to do something useful for you.

build-pack.r is a tool to help you bundle up a bunch of files into a single executable REBOL file. In particular, this can let you include images and other data that a program may need, right in the program package.

2.1 The files list

You will need to change the list of files to include in the package you want to build.

The list is held in block. You will see, near line 28ish

files: [
    logo: %nyc.jpg
    %show.r
]

the example file list. You will need to change this for your own purposes, and this list has some special features. Data is included in the package by placing a REBOL variable name in front of the filename. From the list above, the logo: %nyc.jpg will include the image %nyc.jpg in the bundle and the %show.r script will be programmed to use the logo variable.

3. Building a test package

Before we change %build-pack.r, let's setup a test. From the code above, we will need two files:

1. %nyc.jpg
2. %show.r

As this is an example How-To, there is no %show.r program. So, lets build our own.

3.1 Setting it up

Starting from a working directory, lets say

/home/blue/rebol/test

for GNU/Linux, or maybe

C:\blue\rebol\test

for Windows™. Of it could be some longer filename such as

C:\Documents and Settings\blue\rebol\test

Just so you know, I use Debian GNU/Linux and my computer user name is blue (my friends all call me Bluey). If you become involved with the REBOL community you may see me online as btiffin as well. So the blue in the above example would be changed to your username.

If you don't use a Windows user name, it may be Administrator.

As a programmer, it may be easier to create a directory off of your C drive, just to get sane filenames, or be prepared to get used to the Windows™ directory structure.

3.2 Some assumptions

For the sanity of this document, I'll use

	home directory	/home/blue
	system shell	bash with a prompt of $
	rebol	REBOL/View 1.3.2 with a console prompt of >>

So lets start up REBOL/View and get to work. You may need to press the Console icon (looks like a computer terminal) on the left of REBOL/View desktop when it starts up. I have my REBOL/View configured to start in Console mode.

$ cd /home/blue
$ rebol
>> make-dir/deep %rebol/test/

3.3 A minimal %show.r

>> write %show.r {REBOL [] view layout [image (load logo)]}

that gives a %show.r of

REBOL [] view layout [image (load logo)]

That is all it takes. After we get a copy of %nyc.jpg, we can build the package. The above %show.r is an example of a minimal REBOL program. REBOL lets you build more information into a program, and it is a lot more professional and friendly if you get used to including at least a little documentation in your programs.

So let's do a slightly longer, but more complete %show.r instead.

3.4 A less minimal %show.r

You might want to use the REBOL/View editor for this

>> editor %show.r

or any editor you like, but save it to your working directory as show.r

REBOL [
    Title: "Show New York city"
    File: %show.r
    Author: "Brian Tiffin"
    Date: 28-Jun-2007
    Purpose: "An example program for build-pack.r"
]

;; This code assumes it has access to a variable called "logo"
;;  It does when included in the build-pack example %testpack.r
view layout [
   vh1 "New York city"
   image (load logo)
   btn "Close" [unview]
]

Once again, that's all it takes. This time we added a documented REBOL header, a title above the picture and a close button. I used the builtin REBOL/View editor function, but you could easily use any editor you like.

3.5 Getting %nyc.jpg

A lot of the How-To scripts include a small set of standard images. These images are stored on the rebol.com website. We can get a copy of the New York city picture with

>> write/binary %nyc.jpg read/binary http://www.rebol.com/view/nyc.jpg

So now we have %show.r and %nyc.jpg, let's run build-pack.r. We could actually do this right out of the library since we've set things up so we don't need to change the packaging script, but let's make a local copy.

>> write %build-pack.r read http://www.rebol.org/library/scripts/build-pack.r
>> do %build-pack.r

This will pop up a save file requestor, titled "Save file as:"

Let's use testpack.r as the output name. So answer the Save file as: with testpack.r and hit the Select button. (Under REBOL/View Linux, the button is labelled Select, it may be different for your operating system.)

Now let's run it.

>> do %testpack.r

And, unless something was typed wrong...not totally unlikely, I do it all the time...you should see a window with a nice picture of New York city, and a close button underneath.

4. What you can learn

The rest of this document is a kind of line by line analysis of the script. Explaining things as it goes. One of the nice things about REBOL is that programs can be pretty short for amount of functionality that is involved.

4.1 Filenames in REBOL

Note: REBOL uses % (percent) before a filename. This is a builtin datatype. The REBOL scanner uses the %nyc.jpg syntax to "know", deep down "know", that you mean the file nyc.jpg.

This may seem a little strange at first, but you'll be slinging filenames around using the percent syntax, before you know it.

4.2 system/options/binary-base

The REBOL system object has an internal variable that controls the numeric base for binary! molding and conversion. When you look at %testpack.r you will see that it start out with code: 64#{ which tells REBOL to expect a base 64 binary string. Base 64 lets things compress down further than base 16, the normal mode of hexadecimal operation.

Base 64, packs more binary! into the string version as it can use more symbols to represent values.

Common values for system/options/binary-base are 64, 16, and 2. The default of 16 is usually fine, unless you really want to shrink down included string forms of binary data.

Other library scripts, %bin-data.r and %bin-save.r use these features as well.

4.3 Defining functions

One of the first things this script does is define an error function. REBOL functions are defined with a header and a body. The header defines the parameters and the body is what executes. The REBOL/Core user manual explains this in detail in Chapter 9. REBOL functions can be highly self documenting, but the error function defined here is not. Not a bad thing per say, just not self documented in this case.

4.4 make things

One of the nice features of REBOL is make. Make, makes things. It can be used for simple preallocation of string space, all the way up to building complex objects In this script, the variable out is preallocated with 20,000 bytes of space. This is solely for performance. REBOL will allocate and garbage collect things on the fly, with no worries from the programmer's perspective.

4.5 right to left flow through assignment

Next this script initialises two variables, variable and main both to the special data value none. none is a special data value used throughout REBOL, and you'll get used to it quickly. It doesn't look like much but it enables all kinds of expressive power and shortcuts in source code.

4.6 foreach

Now we get to some meaty REBOL code. Put on the seat belt, we're going for a loop.

foreach file files starts a looping structure that sets the local variable file to every element of the files block. And this loop starts out with some what the? kind of REBOL code.

4.7 either

either is the REBOL version of if/then/else found in other programming languages. If the test is true evaluate the first block, and if it is not true then execute the second block. In this case, the script is testing to see if the file variable is actually a word, logo: in this sample, but it could be any word.

4.8 any-word?

any-word? tests a value for literal words, set-words, get-words and a few other things in the REBOL realm of words. In the case of this script it expects a set-word. A set-word is the REBOL way of assigning things to variables, or, a name followed immediately by a colon. (logo:) There is no "=" or "==" syntax in REBOL. REBOL uses set-words to umm, set words.

So, following into the either, if the value is an any-word, then set the variable, variable to the value of the value. Hmmm, value of the value? That is the special colon immediately followed by a name or a get-word. It means don't do the normal variable evaluation, but return the value of the variable instead.

4.9 get-word

:file tells REBOL that you want the unevaluated value of the word. As an example

print [1 2 3] will print out the values 1 2 3. If you said myvar: print REBOL would attempt to evaluate print and then set the result to myvar. But what if you wanted the 'function' that is print? That's where get-words come in.

myvar: :print returns the value that is print, not the results of print. So now you could write code like myvar [1 2 3] and it will print out the values 1 2 3.

In this loop, it lets REBOL test for things like logo: without triggering the assignment of logo. And then it goes a little further. to-word :file when file contains logo: returns the word logo without the colon. It converts the set-word to a word.

4.10 Careful when you are eyeballing code.

The either sets variable to a variable name or assumes you want to include a file. Sometimes you have to be careful looking at REBOL code, because indentation is only for human benefit, the computer doesn't care. In this example, there is no indentation for the else part of the condition test. It is on the same line. If :file is any-word, set variable, otherwise the false block, which starts on the same source line is evaluated, so when scanning this code quickly, the first indented bit is not the true case, but the false case.

4.11 exists?

So, assuming we want a file, the next line checks to see if the file exists. If not, error out and quit the program, this is treated as an unrecoverable error and you'll either have to update the files block or move the file around to the right directory. In this case %nyc.jpg and %show.r both have to exist in the current directory.

4.12 script?

We passed the file exists test, so now we test if the mentioned file is a REBOL script. REBOL scripts all start with a REBOL header block. It can be empty, but it has to be there.

script? tests a file to see if it has a header. Using either, a file has a header or it doesn't. If it does, we execute a really cool piece of REBOL code.

4.13 load/all

One of the more fascinating features of REBOL is the ability to load and save.

Try and experiment with these features. It takes a while to get used to, but load/all can be a very powerful programming technique. load converts things into a block of REBOL values. load/all loads an entire script into a block, with the header portion as the first element and the body (or whatever follows) as the next part(s).

4.14 none testing

Remember the right to left assignment of none we did a few lines back? If main is none that means we have not set any real value into it yet. The first script in the files block is stuffed into main. Any other scripts, main won't be none anymore, so this assignment will be skipped.

4.15 append

The string out that was preallocated before the loop started now has the contents of script appended to the end of it. Well, actually it is the molded value of the script, without any REBOL header information, that is the skip 2 part. The output package has it's own header information added, so the input script header is ignored.

4.16 mold

REBOL can produce a string form of values that REBOL can understand. Mold is the way to go for this. Strings are properly quoted, blocks are properly umm, blocked and binary values are prepped in a way that they can be read back in.

4.17 tricks for set-words

If the file is not a script, the code now checks to see if there was an any-word stashed away in variable. If there was, it is output along with a string colon ":" and when REBOL evaluates the package bundle, it will look to REBOL like a set-word, exactly what we want for the logo variable associated with the %nyc.jpg file. REBOL can build programs from internal REBOL values, or you can build code out of strings.

After the set-word trick, the associated file is treated as binary information and appended to the output bundle as a molded binary string.

The code then loops back to the foreach, sets the local file variable to the next item in the files block and all the tests and out appending are done again.

4.18 Testing for at least one script

If after the files loop, there is no main, %build-pack.r will display an error message and quit. No sense building a package if there is nothing to execute.

5. Compression

If there is a main, then the current output is compressed. A function built into REBOL is data compression. The variable out is replaced with a compressed version of itself.

5.1 Adding a header

The package now gets a REBOL header block.

5.1.1 reform

Reform is short for reduce form, which takes a block of information, reduces it to REBOL values and forms it into strings.

5.1.2 'REBOL literal

When you reduce a value, it is evaluated. The package bundle wants to start with the word REBOL. The expression 'REBOL tells REBOL to use the literal word REBOL, and to not try and evaluate it as a variable. This is another type of REBOL word, a lit-word, or literally the word that follows the apostrophe.

6. make and context and other goodness

Now we get to a pretty hairy expression. mold third make context main/2. The mold we've seen before, it creates a string that REBOL can read.

6 third

Third returns the third element of a series. We'll get back to this.

6 make

The make is going to create an object in the case, we'll come back to this too.

6 context

Context is a shortform for make object! [...stuff...], and we'll get back to this in a sec.

6 main/2

REBOL allows for path notation. You can get at object fields and other things, but in this case, it returns the second element of the main block. It turns out that the first element of main is the keyword REBOL and the second part is the header information we included in the %show.r program. Repeated here.

[
    Title: "Show New York city"
    File: %show.r
    Author: "Brian Tiffin"
    Date: 28-Jun-2007
    Purpose: "An example program for build-pack.r"
]

So, this last sequence makes an object! with context with the header fields, takes that object and adds a few fields

[
    Built: now
    Length: length? out
]

now being the current data and time, and length? returning the length of the compressed out variable, and creates a new object (the make before the context) so we get and extended object.

6.1 third of an object.

It turns out that the third field of an object in REBOL is kind of the source code for that object. REBOL has reflective properties for just about everything.

So the mold third make context main/2 creates source code of an extended header object that started out in the main script. Pheww...

This information is stashed away in the header variable along with the literal REBOL word.

6.2 repend

repend is short for reduce append. What follows is the actual code that the package bundled will execute when you run %testpack.r, or any of the other sophisticated world class REBOL code bundles you craft together in the future.

6.3 Another set-word trick

The bundle starts out with a blank line after the REBOL header block, (a reduced newline is a blank line when saved to a text file), followed by a set-word, code: followed by a bunch of compressed data/code/code/data stuff that we built into the out string. After this, is the self executing bundle will verify that nothing has gone wrong with itself by making sure the length of the code is what the %build-pack.r builder thought it was, and if everything checks out, decompresses all the scripts and data we stuffed in the bundle and then evaluates it using do. The script that this will execute first is the one that ended up in the main variable, the first script in our files list, or in this case the code that was in the %show.r file.

7. You made a bundle

So now you can build packages out of scripts that need data associated with them and deliver that package to the world as a single text editor friendly REBOL executable file. Feel the power.

8. Credits

File	Author
%build-pack.r	Carl Sassenrath
buil-pack Usage doc	Brian Tiffin

Credit also to the rebol.org library team for keeping the repository up and running.