Documentation for: lds-local.r

  Version: 0.0.8
  31-Jan-2006
  Sunanda
 

1. History

1.1. 2-Mar-2006 -- 0.0.8

• script-search request added to search the Script library for matching words.
• Some typos corrected.

1.2. 28-Jan-2006 -- 0.0.7

• rss-get-feed-data request added to supply the data available in the RSS feed.

1.3. 9-Feb-2005 -- 0.0.6

• Add request for the Mailing list search feature: get-rml-status
• Ditto for search-rml.

1.4. 8-Sep-2004 -- 0.0.5

• Package update requests added. You can now use LDS to write a client to upload and update packages.
• get-script-documentation added. You can now download any documentation for a script

1.5. 9-Jul-2004 -- 0.0.4

• latest option added to get-rml-message to get the latest message without having to know its message number.
• token support added for verified requests
• contribute-script request added -- you can add or update your scripts using LDS.

1.6. 28-Apr-2004 -- 0.0.3

Archive version requests added:

• get-archive-version
• list-archive-versions

Extra fields added to list-package-header to support installation folders and better download information

1.7. 19-Jan-2004 -- 0.0.1

Initial version

1.8. 20-Feb-2004 -- 0.0.2

Package handling requests added:

• get-package-file
• list-package-header

2. Overview

Library Data Services (LDS) is an interface to the REBOL.org website that lets any REBOL program retrieve data from the Library in a REBOL format. Example: get the list of Library members

   probe lds/send-server 'get-user-list []

     make object! [
     status: [0 0 "]
     data:
     make object! [
         user-list: ["gregg" "volker" "sunanda"]
     ]
   ]
 

What comes back is a REBOL object with just two fields:

• status - contains any error information if the request failed
• data - another object, containing the data you requested. The format of this depends on the request, and in some cases, the data available.

3. Uses

You could use this interface to:

• Download messages -- such as all messages from one poster -- into a personal message archive
• Write a few lines of code to perform more complex searches on the Library scripts
• Update the scripts you own

4. Levels of access

LDS defines three levels of access:

• Public
• Member
• Owner

4.1. Public

Broadly, any data you can retrieve via a web browser while not logged on:

• Lists of Library member names
• Download a script
• Download all or part of the Mailing List archive
• Find scripts that match various criteria

4.2. Member

Things you can do as a member of the Library:

• View extended member and script details
• Contribute a new script

4.3. Owner

Things you can do as the owner of a script:

• Update a script with a new version
• View a script's statistics

5. Installing Library Data Services

Run it directly from the REBOL.org server:

do http://www.rebol.org/library/public/lds-local.r

That's all it takes.

6. Using Library Data Services

6.1. Initialising the interface

Issue this call once before you call the service:

do http://www.rebol.org/library/public/lds-local.r

6.2. Calling the interface

All calls have the same format:

response-data: lds/send-server 'request [parameters]

• request: the request you want -- eg get-user-list or help
• parameters: any additional data needed for the request. Example,
• get-script requires a script name. If there are no parameters for a request, specify as none or []
• response-data is the reply back from the server. It is an object! with two operational fields:
  • status: a block with three error/status fields (see next section)
  • data: an object! or 'none. It will be 'none if there is an error, otherwise it will be an object that contain the results of your request.

6.3. Calling the interface with a token

A Library token is needed if you are doing something at the Member or Owner level.

Getting a token is explained later (see Library Tokens).

A call using a token looks like this:

response-data: lds/send-server/use-token 'request [parameters]

7. Status reply

LDS always replies with values in the Status field. It is a block with three parts:

• status/1-- major error code
• status/2-- further status information
• status/3-- short english text describing the error

7.1. Status/1

An integer! between 000 - 999. The values are divided into specific ranges:

• 000- the request worked
• 001 - 099- reserved
• 100 - 199 - a problem with the URL. Almost certainly a bug in lds/send-server
• 200 - 299 - a formatting problem with the parameter (eg you have supplied a string! for a request that expects an integer!)
• 300 - 399 - a contextual problem with the parameter (eg you have supplied the name of a script that does not exist)
• 400 - 499- communication problem.
• 500 - 799 - reserved
• 800 - 899 - authorisation error: the request requires a valid Library member name and password.
• 900 - 999 - internal problem: error in LDS. Congratulations! You've just crashed the server-side of LDS. Please tell me what you did!

7.2. Status/2

May contain context-sensitive information. For example, if the status/3 text is "script not found" , status/2 may contain the script name we were looking for. Not all requests support this.

7.3. Status/3

A short text, in english, describing the problem.

7.4. Examples

   probe lds/send-server 'list-updated-scripts ["zzzzz"]

   make object! [
       status: [200 0 "non-integer days"]
       data: none
   ]
  

(zzzzz should be an integer)

  probe lds/send-server 'get-script ["xxxx.r"]

  make object! [
       status: [300 0 "unknown script"]
       data: none
   ]
  

(xxxx.r is not a script in the Library)

  probe lds/send-server 'get-script ["sssss.r"]

  make object! [
       status: [400 0 "can't connect"]
       data: none
  ]
  

(You may be offline)

8. Library tokens

To use some LDS requests, you must be a member of the Library.

If you are not a member of the Library, you can join by going here:

http://www.rebol.org/cgi-bin/cgiwrap/rebol/signup.r

LDS needs you to supply a Library token on each call. This identifies who you are in much the same way that a cookie does when you visit the Library using the web interface.

8.1. Getting a token

Log on, and visit this page:

http://www.rebol.org/cgi-bin/cgiwrap/rebol/lds-library-tokens.r

Tell it how many tokens you want (one is normally enough -- see multiple tokens below for why you may want more than one) and click "send me tokens" .

The tokens will be emailed to your registered address.

8.2. Install the token

A token looks something like this:

    #{00C04C0913E76ECFF780DD73}
  

Save it as a text file called lds-library-token.txt in the folder from which you execute Library Data services, eg:

   write %lds-library-token.txt  mold  #{00C04C0913E76ECFF780DD73}
  

The token is now ready for use.

8.3. Using the token

Add the /use-token refinement to each lds/send-server request, eg:

Response-object: lds/send-server/user-token 'token-test ["test my token"]

• LDS will need security permission to both read and write %lds-library-token.txt -- the token is updated on each successful usage.
• You can add /use-token to any request, but it is only needed for requests that are marked as needing it.

8.4. Multiple tokens

If you use LDS from more than one location (perhaps on different machines) you need a different token for each location. You can have as many tokens as you want.

9. General Library requests

9.1. help

Returns a list of all LDS requests and some help on each of them -- a good way to see if you have an out-of-date copy of these notes.

• Parameters: none
• Response: request-list [block!]
  • Request list is a block with paired values.
  • The first is the name of a request
  • the second is an object with details of the request. Details include:
    • authorisation? -- true if request requires a userid and password
    • purpose -- what the request does
    • parameters -- what parameters the request needs
    • response -- what the response is

9.1.1. Example

 probe lds/send-server 'help none

  make object! [
   status: [0 0 "]
   data:
   make object! [
     request-list: [
       "get-rml-message"
       make object! [
           authorisation?: false
           purpose: {Retrieve a message from the REBOL Mailing list archive}
           parameters: "Msgid [integer!]"
           response: "Object containing message and control headers"
             ]
    "get-script"
    make object! [
  [snip]
  

9.2. list-users

Retrieve list of activated Library members

• Parameters: none
• Response: user-list []

("activated" means they have logged on at least once)

9.2.1. Example

  probe lds/send-server 'list-users []

  make object! [
      status: [0 0 "]
      data:
      make object! [
          user-list: ["volker" "gregg" "sunanda"]
      ]
  ]

  

10. Mailing List requests

Use these requests to read messages from the Mailing list archive

10.1. get-rml-message

Retrieve a message from the REBOL Mailing list archive

• Parameters: Msgid [integer! | string!]
• Response: Object! -- containing message and control headers

10.1.1. Example 1 -- get a single message

  probe res: lds/send-server 'get-rml-message [237859]
 

The fields in the data object for this request are:

• msgid -- Library Archive identifier for this message. (The Library has two formats for the message id: a number, which is broadly compatible with the message numbers in other archives of the ML, and a code that isn't. Either will work. These two requests return the same message:
  • probe res: lds/send-server 'get-rml-message [237859]
  • probe res: lds/send-server 'get-rml-message ["CKLQ"]
• author -- slightly munged email address
• msgdate -- local date and time of posting. Will usually be a REBOL date!, but may sometimes be a REBOL date! in a string. Use load form data/msgdate to coerce to date!
• subject -- subject line
• links -- msgids of: first-in-thread / next-in-thread / previous-in-thread
• a-links -- msgids of: first-for-author / next-for-author / previous-for-author
• thread -- If message is the first in a thread, msgids of: first-in-previous thread / first-in-next-thread
• content: -- message text

10.1.2. Example 2 -- get all messages for a given thread

  msgs: copy []
  res: lds/send-server 'get-rml-message [213498]
  forever  [
           append msgs res/data
           if res/data/links/3 = 0 [break] ;; no next-in-thread link
           res: lds/send-server 'get-rml-message reduce [res/data/links/3]
        ]
  print ["There were" length? msgs "message(s) in the thread"]

 

10.1.3. Example 3 -- get the latest message

  msgs: copy []
  res: lds/send-server 'get-rml-message ["latest"]

 

10.2. get-rml-status

Sends you back some status information about the Mailing list archive.

• Parameters: None
• Response: Object! -- containing various counts -- see example

10.2.1. Example

  probe res: lds/send-server 'get-rml-status []

  make object! [
     status: [0 0 "]
     data:
     make object! [
         counts: [241337 241337]
         threads: [8706]
         years: [1980 2000 2001 2002 2003 2004 2005 2024]
     ]
 

The fields in the data object for this request are:

• counts block
  • count/1 -- total messages in the Archive
  • count/2 -- total messages that are in the word index (ie that you can search on). This number may often be a little lower than count/1: indexing is an async process and lags behind at times. If it is significantly lower, then we may be rebuilding the word index. If so, please be patient
  • count/3 -- and higher: reserved. we may be adding them later, so don't assume that count/1 = last count
• threads block -- Thread information. Currently a block with just 1 number: the total number of threads. But we may add more numbers, so (as above) don't assume that thread/1 = last thread
• years block -- information about the years we have indexed. Use this with the search-rml request. Note that there maybe more years (as shown in the example) than you might expect. See search-rml for more details.

get-rml-status is most useful for getting the full span of years for a search-rml request....See immediately below.

10.3. search-rml

Sends you back the message ids for all threads that meet your search criteria

• Parameters: A string that has both the words on which you want to search and, optionally, the years you want to search -- see below
• Response: Object! -- containing various fields -- see below

10.3.1. Parameter

The parameter is a single quoted string. It should contain at least one word on which you want to search. It may also contain one or more years.

• If the string does not contain any years, the search will be on the subject lines of all posts
• If the string does contain one or more years, the search will be on the full text of the posts for those years

10.3.2. Examples of the search parameter:

• "help" -- searches for "help" in all subject lines
• "help 2005 2004" -- searches for "help" in the full text of all posts whose thread begin in 2004 or 2005
• "2001 bunny 2003" -- searches for the word "bunny" in all posts for 2001 and 2003 (yes, there is one result)
• "2000 white pink 2001 apple 2002 2003" -- searches for posts containing all the words "white pink apple" in the years 2000 through 2003 (and, yes, expect one result)

• "async version" -- searches the subject lines for the two words.

10.3.3. Response object

Contains both the message ids of the messages that matched your search, and some reflected back fields to confirm what has been searched.

An example is

  make object! [
     status: [0 0 "]
     data:
     make object! [
         target-words: ["apple" "pink" "white"]
         search-type: "posts"
         span: [2003 2002 2001 2000]
         msgids: [207276]
     ]
 ]
 

• target-words -- the words searched were "apple" AND "pink" AND "white". If your original request contained other words, they have been ignored as they are on the stop list
• search-type -- the search type was posts meaning we checked the whole of all the posts. The other possibility is subjects, meaning we searched just the subject lines.
• span -- we searched the years 2000 through 2003
• msgids -- we found one thread meeting the search criteria. 207276 is the message id of the first message in that thread

10.3.4. Searchable years

Using the search-rml request, you can search on all years for which threads exist. This includes some strange ones, such as 1980 and 2024. They exist because we take the date field in the original email's header as correct. and, obviously, in some cases they were not: the sender's ISP or server was badly set up.

We don't have much choice about that. The Mailing list archive started some years after the Mailing list, so we don't have accurate context information to correct the problem.

10.4. Examples

  probe lds/send-server 'search-rml ["async version"]
  make object! [
     status: [0 0 "]
     data:
     make object! [
         target-words: ["async" "version"]
         search-type: "subjects"
         msgids: [234122]
     ]
 ]
 

  probe  lds/send-server 'search-rml ["1960 1961 help me to get a good job 2004 2003"]

 make object! [
     status: [0 0 "]
     data:
     make object! [
         target-words: ["get" "good" "help" "job" "me"]
         search-type: "posts"
         span: [2004 2003]
         msgids: [230329 230451 ....snipped .....]
     ]
  ]
 

Note that the span contains only the years we could search (we don't have messages from the 1960s), and the target-words does not include the stop words "to" and "a".

11. Script requests

Use these requests to search the Script Library, and to download scripts.

11.1. get-script

Retrieve latest version of a script

• Parameters: script name [string!]
• Response: script [string!]

Returns the source of the named script

11.1.1. Example:

  probe lds/send-server 'get-script ["wumpus.r"]

  make object! [
     status: [0 0 "]
     data:
     make object! [
         script: {REBOL[
     Title:   "Hunt the Wumpus"
     Date:    20-Jul-2003
     File:^- %wumpus.r
     Purpose: "A REBOL version of a retro text game. Just for fun."
     Author:  "Gregg Irwin"
     EMail:   greggirwin@acm.org
     Version: 1.0.1
      Library: [
         level: 'intermediate
         platform: 'all
  [snip]

 

11.1.2. Note to script owners

If you own a script, and check its download count on the statistics page of REBOL.org: LDS get-script downloads are counted as "Total LDS downloads" .

11.2. list-scripts-by-filter

Returns a list of script names that match the search filter.

• Parameters: search filter and value, eg type-demo or domain-cgi
• Response: script-list [block]

11.2.1. Example

   probe lds/send-server 'list-scripts-by-filter [domain-xml]

  make object! [
      status: [0 0 "]
      data:
      make object! [
          script-list: ["json.r" "xmlgen.r" "rblxelparser.r"]
      ]
  ]
  

11.3. list-search-filters

Returns an object with all the possible filters that can be used with list-scripts-by-filter

• Parameters: none
• Response: object!

The response object has a series of blocks, each represents a possible set of filters, for example

type: [article demo dialect faq function ... ]

means you can filter on type-article type-demo etc

11.4. search-scripts

Returns a list of script names that contain the words you asked for

• Parameters: string containing the target words
• Response: script-list [block]

11.4.1. Example

    probe lds/send-server 'search-scripts ["decompress"]

  make object! [
      status: [0 0 "]
      data:
      make object! [
          script-list: ["compress.r" "autoextract.r"]
      ]
  ]
  

The script-list returned is in script relevance order: best matches first.

11.5. list-scripts-owned-by

Returns a list of scripts owned by a given Library member

• Parameters: owner-name [string!]
• Response: script-list [block!]

If the owner name is not recognised, the script-list is returned as an empty block.

11.5.1. Example -- get list of scripts owned by Sunanda

   probe lds/send-server 'list-scripts-owned-by ["sunanda"]

  make object! [
      status: [0 0 "]
      data:
      make object! [
         script-list: ["cookie-example.r" "oneliner-chess-board.r"
 "oneliner-monty-hall.r"]

 

11.6. list-updated-scripts

Get a list of new or updated scripts in the last [n] days

• Parameters: n [integer!]
• Response: script-list [block!]

Lets you find which scripts have been added or updated in the last n days. n is an integer: 1 or more

11.6.1. Example 1: what's new in the last 15 days?

  probe lds/send-server 'list-updated-scripts [15]

  make object! [
      status: [0 0 "]
      data:
      make object! [
          script-list: ["mtv.r" "mvt.r" "mvs.r"]
      ]
  ]
  

11.6.2. Example 2: Download new versions of scripts changed in the last 31 days

  res1: lds/send-server 'list-updated-scripts [31]
  foreach scr res1/data/script-list
  [
   res2: lds/send-server 'get-script reduce [scr]
   write to-file scr res2/data/script
  ]

 

12. Script Archive requests

Use these requests to get archived versions of scripts (each time a script owner updates a script, we put the previous version in the archive. In some cases, the previous versions remain available for download).

12.1. list-archive-versions

REBOL.org usually only shows the latest version of each script. Previous versions may still be available. This request returns a list of all publicly available versions of a script.

• Parameters: script-name
• Response: archive-details [block]

12.1.1. Example:

  probe lds/send-server 'list-archive-versions [demo1.r]

  make object! [
     status: [0 0 "]
     data:
     make object! [
         archive-versions: [
             [7 15-Mar-2004 "Fixed bug in header"]
             [12 28-Apr-2004 "Version 2.0 -- supports IOS"]
             ]
     ]
    ]
 

In the above example, there are two publicly-available versions of the script: 7 and 12. The version number is an internal number to the Library. It will usually be different to the owners' version number in the script itself.

The highest version number show will always be the current live version of the script. Lower numbered entries are back-versions of the script that the owner has left publicly available.

To retrieve an archive version of a script use the request get-archive-version.

12.2. get-archive-version

Retrieve an archived version of a script (see list-archive-versions for more details)

• Parameters: script-name/version-number [string] Separate the two parts with a forward slash, eg my-package/file1.txt
• Response: script -- compressed file containing the script

12.2.1. Examples:

Get archive version 17 for script demo1.r:

  probe res: lds/send-server 'get-archive-version ["demo1.r/17"]

  make object! [
     status: [0 0 "]
     data:
     make object! [
         script: #{
   789CC551C14AC34010BDEF574C03524592565191F46015EB410445C54BE86193
   snip
   ED800F8DA6768B8BCD39E0D9FE255DA877CE25EAB52D030000
   }
     ]
    ]

 

demo1.r has a publicly available version 17, so it has been returned:

Get archive version 14 for script demo1.r:

   probe res: lds/send-server 'get-archive-version ["demo1.r/14"]

   make object! [
     status: [301 "demo1.r" "not available"]
     data: none
     diagnostics: [41641 33710 0.11]
  ]

 

Version 14 of demo1.r is nor publicly available. If you need that exact version, contact the owner and ask them to make it available.

13. Update scripts requests

13.1. contribute-script

Use this request to either add a new script to the Library; or update an existing script that you own.

• Parameters: simple dialect -- see below
• Needs token?: Yes
• Response: complex object -- see below

13.1.1. Parameters

Are a dialect with these words: mode, note, and script. Mode and script must always be present. Note is optional.

• mode -- can be check or update . check means check that the script is okay, but do not actually add or update it. Use check to play around with this request without actually updating anything. Update means add (the new script) or update (the existing script) if all is well with it.
• note -- a note about why you are updating the script (eg "V2 adds extra video formats" )
• script -- the script itself

13.1.2. Example of the dialect:

note "changed error handling" mode check script {rebol [file: %zzz.r .....}

13.1.3. Examples of use

     scr: {rebol [] quit}
     response-object: lds/send-server/use-token 'contribute-script
     compose [mode "check" note "fix error handling" script (scr)]
  

This example does not send an acceptable script! This is what you'd get back:

   Probe response-object

   make object! [
   status: [210 30 "problems with script"]
    data:
     make object! [
     validation: ["file" "We need a File: field in the REBOL header."
     "title" "We need a Title: field in the REBOL header."
     "date" {The Date: field in the REBOL header needs to be a valid REBOL
 date. (Not none)}
     "purpose" "We need a Purpose: string in the REBOL header."]
      ]
      ]
  

13.2. get-script-documentation

Retrieve the documentation for a script, either as HTML or in its editable format

• Parameters: script name [string!] / format-wanted [string!]
  • format-wanted is either editable or html. If omitted or incorrect, editable is assumed/
• Response: documentation [compressed string!]

13.2.1. Examples:

  probe res: lds/send-server 'get-script-documentation ["lds-
 local.r/editable"]

 make object! [
     status: [0 0 "]
     data:
     make object! [
         documentation: #{
 789CED7D6977DBD6D9E077FF8A1BF6742C69448AE04E2675476BDF74ECC68D9D
 E4EDE8B8E7402428210601160045AB99FCF77996BB622129937E9BCE8CCE492C
 91C05D9EFBECDBFDC617B1BF08FED0C89369D36BBCFAE6CC7FF5CD43E7D5EBF0
 2EF5D32771E5E7BE7817A48FE134C8BE39836F04FD7C330B1FC534F2B38C5E85
 [snip]

  print decompress res/data/documentation

   [h2 Library Data Services
   [contents

   [asis
    Version: 0.0.5
    12-Sep-2004
    Sunanda
 [snip]

 

13.2.2. Note to script owners

If you own a script, and check its download count on the statistics page of REBOL.org: LDS get-script downloads are counted as "Total LDS downloads" .

14. Script owner requests

Use these requests to get information about a script that is only available to the script owner.

14.1. List-my-scripts

Returns the list of scripts that you own. Similar to list-scripts-owned-by, but you identify yourself by using a Library token.

• Parameters: none
• Needs token?: yes
• Response: list of scripts you own

14.1.1. Example

response-object: lds/send-server/use-token 'list-my-scripts []

14.2. get-script-stats

Returns owner statistics for the script -- only available if you own the script.

• Parameters: script-name
• Needs token?: yes
• Response: various counts and facts about the script

14.2.1. Example

response-object: lds/send-server/use-token 'get-script-stats [lds-local.r]

15. Package read requests

Use these requests to query or retrieve files from packages.

There is a also special LDS utility that makes working with packages easy:

do http://www.rebol.org/library/public/repack.r

15.1. get-package-header

Retrieve the header for a package -- i.e. the index to that package.

• Parameters: package-name [string!]
• Response: Object! -- containing details of all files in the package
  • data/user-notes - text about the package
  • data/package-file-list - one entry for each file or script in the package.

15.1.1. data/package-file-list fields

• file-name -- name of the file
• install-folder -- a relative folder name suggesting where this file should be installed.
• last-updated - date and time stamp of when the file was last uploaded to the package
• cgi-type -- file type as reported by the browser that uploaded the file. Will be "text/plain;library script" if the file is an existing script from the Library
• file-type -- one of three values
  • file -- it's a file uploaded as part of the package
  • script -- it's a script that is part of the Library, and has been included in this package
  • package -- it's another package -- see "file-type: package" below for more notes on this
• notes -- any text the owner has entered about the file
• checksum -- a checksum/secure on the file contents. Use this to either check you get a valid copy of the file when downloading, or to bypass downloading if the local copy has the same checksum.
• file-size -- the size of the file after it is downloaded and decompressed
• compressed-size -- the size of the file to download (ie the compressed size)

15.1.2. Possible errors.

The package may not exist, or may not be available. See http://www.rebol.org/cgi-bin/cgiwrap/rebol/boiler.r?display=lds-package-dow nloader-help.html for a list of the package error messages

15.1.3. Example -- header for package lds-demo1-package.r

  probe res: lds/send-server 'get-package-header ["lds-demo1-package.r"]

  make object! [
     status: [0 0 "]
     data:
     make object! [
         user-notes: {Run lds-demo1.r under REBOL/View and see if you can
 Spot The Developer!}
         package-file-list: [
             make object! [
                 file-name: "image1.jpg"
                 cgi-type: "image/pjpeg"
                 install-folder: "images/"
                 file-type: "file"
                 version: 0.0.4
                 last-updated: 17-Apr-2004/16:17:19.39+1:00
                 notes: "
                 checksum: #{4001620992AE7B6974F159A6D957571CD1558E36}
                 file-size: 17165
                 compressed-size: 12573
             ]
             make object! [
                 file-name: "image2.jpg"
                 cgi-type: "image/pjpeg"
                 install-folder: "images/"
                 file-type: "file"
                 version: 0.1.0
                 notes: "
                 checksum: #{B76028C7C1D1DFFF2584788BEC82B8F7A8B55C07}
                 file-size: 75276
                 compressed-size: 39052
             ]
 etc -- one entry per file

 

15.1.4. file-type:package

This means that the file is itself another package. That probably means that his package requires all the files from that package to be downloaded for a complete installation.

The package header for the current package does not include all the files for the included package. If you are writing a package installer or downloader you may need to take into account file-type:package. If you do, issues include:

• the included package may be flagged as not available for download
• the included package may itself include other packages
• there may be loops in the inclusions. Perhaps Package1 includes Package2, and Package2 includes Package1. This should not be interpreted as an error -- it may simple be that both packages require the other one.

15.2. get-package-file

Retrieve one file from a package

• Parameters: package-name/file-name [string!] Separate the two parts with a forward slash, eg my-package/file1.txt
• Response: Object! -- file containing the compressed file

15.2.1. Example -- file notes.txt package lds-demo1-package.r

  probe res probe res: lds/send-server 'get-package-file
 ["lds-demo1-package.r/notes.txt"]

  make object! [
     status: [0 0 "]
     data:
     make object! [
         file: #{
  789C7552D16ED3500C7D8F947F70DF3AD4A06D8C017B63A39B040C101DF0EC24
  6E7AD5C48E1CA7617F8FEF6DD6371429B24F7C8E8FEDE4599EDDB5A1DA83096C
  0CD5F26C89AD15D50B3898F46779F6BF67D38B81ED087EAE6FBF7F859A0ED44A
  snip
  074DF93BCF8F8B9881F70EA41B9DFEB6237F9567171F8A7B2A8BCBF3F3AB3CFB
  07C289D8E6C6020000
  }
     ]
  ]

 

15.2.2. Notes

The file as returned is compressed:

  decompress res/data/file
  {^M
  ^M
  Click to Start^M
  (alt-click to stop)^M
  ^M
  ^M
  ^M
  ^M
  Spot the REBOL developer!^M
 

To save it, and lose all those ^M thingies in a text file, always save the file as binary:

  write/binary %notes.txt decompress res/data/file
  print read %notes.txt

  Click to Start
  (alt-click to stop)

  

Saving the file as binary is always the correct thing to do: the file may be a JPG or executable, or some other non-text format.

16. Package update requests

16.1. Overview before we start

A package is a multi-file unit that makes up one contribution to the Library.

From a user's perspective, it is three things:

• A script in the Library with a type header that includes the magic word 'package. This script will be just a dummy -- more like an index card that a real script.
• Zero or more uploaded files that make up the package
• Zero or more existing scripts from the Library that are included in the package.

All the LDS requests for uploading or updating packages will need a Library token.

Again, as with the contribute-script request, there is a mode parameter on every request.

• mode check will not perform the update -- thus making some testing easier
• mode update will be required for an update

16.2. Transaction level support

To create a full and usable package, several requests (perhaps many) have to be issued and succeed. If the server or the client fail part-way through, there will be an incomplete or unusable package left on public view.

This is not so different from when a user is uploading files by hand to create a package -- their connection could fail at any point. I just mention in it case LDS clients want to get clever with some restart code.

LDS itself will not be offering any sort of journaling or rollback. It sees and actions individual requests.

16.2.1. Step 1: contribute-script

Use contribute-script request to upload a new script. Include 'package in the Library header 'type field.

After successfully contributing a new script, various automated mechanisms start telling the world the good news (email notices, RSS feed, "what's new" pages, etc)

So some quick off-the-mark people may be looking at the new package before we've uploaded all the files. Nothing much we can do about that. It's part of the way packages were implemented as a bolt-on.

16.2.2. Step 2: update-package-header

Use this request to:

• optionally, add some header information to the package
• set the package as unavailable so people cannot download it while it is incomplete

16.2.3. Step 3: upload-package-file

Use this request for each of your files that are part of the package.

16.2.4. Step 4: include-script-in-package

Use this request for any existing Library scripts that are part of the package.

16.2.5. Step 5: update-package-header again

Use this again to set the package as available . People can now download your new package.

16.3. Steps involved in updating a package

If you later need to change the package, use these steps:

16.3.1. Step 1: update-package-header

Set the package as unavailable to prevent people accidentally downloading an incomplete or inconsistent package

16.3.2. Step 2: upload-package-file

To add new files or update existing ones

16.3.3. Step 3: remove-file-from-package

To remove a file from the package

16.3.4. Step 4 include-script-in-package

To include additional Library scripts in the package

16.3.5. Step 5 remove-script-from-package

To remove a Library script from the package.

16.3.6. Step 6: update-package-header

Use this again to set the package as available . People can now download your updated package.

16.4. update-package-header

Use this request to add or update the header details of a package

• Parameters: simple dialect -- see below
• Needs token?: Yes
• Response: complex object -- see below

16.4.1. Parameters

Are a dialect with four words : mode, script-name and status and user-notes

• mode -- can be check or update . As per contribute-script
• script-name -- the name of the script. Must be present
• status -- sets the available status. Values are private or public
• user-notes -- optional notes about the package.

16.4.2. Response

A data object that normalises the dialect -- if you had any fields missing or with unacceptable values, you can see from the response what we assumed.

16.4.3. Examples

A full request looks something like:

    lds/send-server/use-token 'update-package-header
      [script-name "example-package.r"
       mode "update"
       status "private"
       user-notes  "Only works on MACs"
     ]
  

(As with all these requests, the order of the keywords in the dialect does not matter)

These following examples stand in for a lot of detailed exposition about error checking and what we do with bad requests.

The data section of the response is the internal, normalised dialect words -- so you can see how LDS-server has interpreted missing or bad data:

   probe lds/send-server/use-token 'update-package-header [
         script-name "example-package.r"
         mode "update"
        ]

  make object! [
     status: [300 0 "unknown script"]
     data:
     make object! [
         script-name: "example-package.r"
         mode: "update"
         status: none
         user-notes: none
     ]
    ]
 

Problem: no such script. Expect similar errors for:

• script is not a package (does not have a library header type: [... package ...])
• script is not owned by you (ie not owned by owner of the token)

   probe lds/send-server/use-token 'update-package-header [
         script-name "example-package.r"
         user-notes "zzz"
       ]

  make object! [
     status: [0 0 "]
     data:
     make object! [
         script-name: "example-package.r"
         mode: "check"
         status: none
         user-notes: "zzz"
     ]
   ]
 

Appears to have worked (got a 0 response), but the missing mode was taken as check -- so no changes made.

   lds/send-server/use-token 'update-package-header [
         script-name "example-package.r"
         mode "do it"
         user-notes "zzz"
        ]

  make object! [
     status: [0 0 "]
     data:
     make object! [
         script-name: "example-package.r"
         mode: "check"
         status: none
         user-notes: "zzz"
     ]
    ]

 

Mode is bad (not check or update ) -- treated as its safest value, check . Again, no update.

  probe lds/send-server/use-token 'update-package-header [
         script-name "example-package.r"
         mode "update"
         user-notes "zzz"
        ]

  make object! [
     status: [0 0 "]
     data:
     make object! [
         script-name: "example-package.r"
         mode: "update"
         status: none
         user-notes: "zzz"
     ]
  ]
 

This one worked. It updated the user notes, but did not change the status (because none was specified).

  lds/send-server/use-token 'update-package-header [
       script-name "example-package.r"
       mode "update"
       status "confused"
       ]

  make object! [
     status: [0 0 "]
     data:
     make object! [
         script-name: "example-package.r"
         mode: "update"
         status: "private"
         user-notes: none
     ]
   ]
  

This one worked, but may not have done what you expected. There was a status, but it was a bad value (neither private or public ) we normalised it to private that being the safest value. The user notes remain unchanged.

  probe lds/send-server/use-token 'update-package-header [
         script-name "example-package.r"
         mode "update"
         status "public"
         user-notes "
        ]

  make object! [
     status: [0 0 "]
     data:
     make object! [
         script-name: "example-package.r"
         mode: "update"
         status: "public"
         user-notes: "linux only"
     ]
  ]
 

Changes the status to public and sets the user notes to "

16.5. upload-package-file

Use this to add or update details of a file in the package

• Parameters: simple dialect -- see below
• Needs token?: Yes
• Response: complex object -- see below

16.5.1. Parameters

Are a dialect with these words: mode, script-name, file-name, install-folder, user-notes, version, contents, and checksum. Mode, script-name and file-name must always be present

• mode -- can be check or update . As per contribute-script.
• script-name -- name of the script. Must be present
• file-name -- the name of the file. Must be present
• install-folder -- folder into which the downloaded file will be saved. Must be a relative path if present (do not start with / . Must end with / . " is allowed to mean the default folder.
• contents -- the file itself as a compressed binary. Optional if you are changing the details (version, user-notes, install-folder) of a previously uploaded file
• checksum -- a checksum/secure of the pre-compressed file. Needed only if contents are present. Used to detect transmission errors in the upload.
• user-notes -- notes for the user about the file (eg "Not needed under Windows" )
• version -- a version number for the file (eg 0.0.3)

16.5.2. Examples

  probe lds/send-server/use-token 'upload-package-file
      [script-name "example-package.r"
       mode "update"
        file-name "image-1.jpg"
        install-folder "images/"
        user-notes "Photo of Fluffy Bunny"
        version "0.0.5"
        contents  #{...binary field...}
        checksum #checksum/secure of contents (to detect in-flight damage)
    ]
  

Uploads a new file (or completely replaces the details of an existing one)

   lds/send-server/use-token 'upload-package-file
      [script-name "example-package.r"
       mode "update"
        file-name "image-1.jpg"
        install-folder "my/images/"
        user-notes "Better Photo of Fluffy Bunny"
        version "1.0.5"
        ]
  

Will change the install-folder, user-notes and version

   lds/send-server/use-token 'upload-package-file
      [script-name "example-package.r"
       file-name "image-1.jpg"
       mode "update"
       user-notes "Even better Photo of Fluffy Bunny"
      ]
 

Will change just the user notes.

16.5.3. What you get back

Upload-package-file returns the normalised dialect, but the contents field is set to [omitted] -- no point sending back a 5meg file, or however large the file is.

16.6. include-script-in-package

Use this to include an existing Library script in the package. Note that you don't upload the script (after all, we already have it). You just tell us to make it part of the package.

• Parameters: simple dialect -- see below
• Needs token?: Yes
• Response: complex object -- see below

16.6.1. Parameters

Are a dialect with these words: mode, script-name, file-name, install-folder, library-script-name.

• mode -- can be check or update . As per contribute-script.
• script-name -- name of the script. Must be present
• file-name -- the name of the file. Must be present
• install-folder -- folder into which the downloaded library script will be saved. Must be a
• relative path if present (do not start with / . Must end with / . " is allowed to mean the default folder.
• Library-script-name -- the name of the script to be made part of the package (eg "convert- base.r" or values.r )

16.6.2. Example

 lds/send-server/use-token 'include-script-in-package
      [script-name "example-package.r"
       mode "update"
       library-script-name: "convert-base.r"
       install-folder "lib-scripts/"
      ]

  

16.7. Remove-file-from-package

Use this to remove a file you have uploaded to a package.

• Parameters: simple dialect -- see below
• Needs token?: Yes
• Response: complex object -- see below

16.7.1. Parameters

Are a dialect with these words: mode, script-name, file-name, and install-folder.

• mode -- can be check or update . As per contribute-script.
• script-name -- name of the script. Must be present
• file-name -- the name of the file. Must be present
• install-folder -- currently optional

16.7.2. Example

 lds/send-server/use-token 'remove-file-from-package
      [script-name "example-package.r"
       mode "update"
      install-folder: "
      file-name "image-1.jpg"
    ]

 

image-1.jpg will not be a part of the package after this request is run. (Note: this does not mean it was a part of the package before; but, if it was, it isn't now)

16.8. Remove-script-from-package

Use this to remove a Library script from the package.

• Parameters: simple dialect -- see below
• Needs token?: Yes
• Response: complex object -- see below

16.8.1. Parameters

Are a dialect with these words: mode, script-name, file-name, install-folder, library-script-name.

• mode -- can be check or update . As per contribute-script.
• script-name -- name of the script. Must be present
• library-script-name -- the name of the Library script to be removed
• install-folder -- installation folder to which the script was to be installed

16.8.2. Example

 lds/send-server/use-token 'remove-script-from-package
      [script-name "example-package.r"
       mode "update"
      install-folder: "
      Library-script-name "convert-base.r"
    ]
 

/convert-base.r will not be a part of the package after this request is run. (Note: this does not mean it was a part of the package before; but, if it was, it isn't now)

17. Other requests

17.1. rss-get-feed-data

Retrieves the raw data used to generate the Library's RSS feed.

You can subscribe to the Library's RSS feed by visiting get RSS feed. 

Alternatively, you can used this LDS request to retrieve the same data as a REBOL object. In addition, there are some extra fields not supplied to the RSS readers.

• Parameters: none
• Response:response object -- see below

17.1.1. response object

has these fields:

• generation-info -- an object with useful status details
• script-library -- a block with an object for each script included in the feed
• mailing-list-archive -- a block with an object for each mailing list thread included in the feed
• news -- a block with details of each "news" headline

17.1.2. response object/generation-info

Has these useful fields

• rss-test-mode -- should always be false.
• timestamp -- precise timestamp for when the RSS feed data was last generated
• supply? -- will be generated if your request has triggered the Library into generating a fresh set of RSS feed data; or cached if you are being supplied some slightly older data. Note that cached does not usually mean stale. Even cached data is likely to be very close to real-time accurate.
• data-range -- a block with two dates; showing the start and end date of the range for the RSS feed.
• timings -- for testing

17.1.3. response object/script-library

Is a block of objects, one per script included in the RSS feed data.

• script-name -- the name of the script
• link -- the URL to view it
• update-type -- updated or new
• entry-date -- when the script was added or last updated
• title -- from the script's title header
• purpose -- from the script's purpose header
• version -- from the script's version header
• owners -- a block with zero or more Library member's names; these are the members who own (and can thus update) the script
• change-note -- a note added when the owner last updated the script to explain why it is changing.

17.1.4. response object/mailing-list-archive

Is a block of objects, one per included in the RSS feed data.

• msgid -- the mailing list message id of the first message in the thread
• subject -- the subject line of the first message in the thread (note that it is possible for later messages in the same thread to have different subject lines).
• new-messages -- a block with two data items per message:
  • message date -- the date the message was entered into the ML archive
  • message id -- the message id of the message
  There will be an entry (date and message id) in the block for each message in the thread that is included in the RSS feed data:
  • if this block includes the msgid of the first message in the thread, then this is a new thread;
  • if it does not, then this is an old thread that has one or more new messages in it.

17.1.5. response object/news

Is a block of objects, one per news item included in the RSS feed data. News is the little snippets you can find on the top right of the REBOL.org home page.

• title -- headline
• link -- URL for further information
• date -- date news item added
• content -- text of the information paragraph.

17.1.6. Example

  probe lds/send-server 'rss-get-feed-data []
   

---

18. Token requests

Use these requests to work with your Library tokens.

18.1. token-test

Test if your Library token works:

• Parameters: any text
• Token required: yes
• Response: Object! -- your text back

If the token is valid, you get your text back. If not, you get an error message about the token

18.1.1. Example:

response-object: lds/send-server/use-token 'token-test ["any text"]

18.2. token-cancel

Cancels your current token. It will never work again.

• Parameters: none
• Token required: yes
• Response: Object! -- status

18.2.1. Example:

response-object: lds/send-server/use-token 'token-cancel []

19. Random points and answers to NAQs (never-asked questions)

19.1. Why not use LNS (Lightweight Network Services)?

LDS predates LNS by a year, so LNS simply wasn't a possibility.

It would be good to add an LNS interface to the Library. If any one wants to volunteer to help with that, please drop us a feedback message

19.2. Validation and verification

All validation and verification of requests is done on REBOL.org. Lds-local does no parameter checking -- it simply passes the parameters you supply to REBOL.org.

19.3. Sorting

The lists you get back from LDS are not usually not sorted. Any sequencing is purely accidental, and should not be relied on. LDS provides the data, it is your choice what order to handle or present it.

One exception is the ''search-scripts request. The scripts-list it returns is sorted according to the relevance of the results: best fit first.

19.4. Caching

Each time you call lds/send-server, it talks to the REBOL.org server. There is a case for caching results as many things do not change that often. It's on the upgrade list.

19.5. XML

I've been thinking of adding an XML refinement:

lds/send-server/xml 'help none

So what you get back is an XML page, not a REBOL object. Easy to do, but no point, unless anyone needs it.