• README

  • Welcome to the OpenBIOS tokenizer Toke. 
    
    This README provides you with a short description of the tokenizer and
    of how to use it.
    
    ---------------------------------------------------------------------
    
    Table of contents:
    
    1. What is the OpenBIOS tokenizer?
    2. What is required to build the OpenBIOS tokenizer?
    3. How to use the OpenBIOS tokenizer
    4. Toke's Special Features not described in IEEE 1275-1994
    5. Text Strings
    6. Contact
    
    ---------------------------------------------------------------------
    
    1. What is the OpenBIOS tokenizer?
    
       toke is a GPLed FCode tokenizer. It can tokenize (assemble) fcode
       source to bytecode files as described by the IEEE 1275-1994 standard.
       
       This program is compliant to IEEE 1275-1994.
       
       Bytecode files normally contain Open Firmware drivers or other 
       packages for use with an Open Firmware compliant system.
       
       
    2. What is required to build the OpenBIOS tokenizer?
    
       toke should build with any ANSI compliant C compiler.
       
       For building toke on certain platforms you might have to adjust the 
       Makefile. If so, please contact Stefan Reinauer <stepan@openbios.org>
       
       Build toke by just enter "make". To clean up an existing build, 
       use "make clean" or "make distclean"
     
       
    3. How to use the OpenBIOS tokenizer
    
       tokenize an fcode source file with 
    	toke [-v|--verbose] [-i|--ignore-errors] <file.4th> [<file2.4th>]
    	
       Get help with:
    	toke [-h|--help]
    
       The file extension will be changed to .fc. If no file extension 
       exists, .fc will be appended. If -i is specified, toke continues
       on errors, producing invalid fcode binaries.
    
       
    4. Toke's Special Features not described in IEEE 1275-1994
    
       In addition to the standard Open Firmware words, some additional
       commands have been added to make life easier and for compatibility
       to other, mostly commercial, fcode tokenizers. These are:
    
    4.1 NEXT-FCODE
    
       In tokenizer[ .. ]tokenizer context there is an additional control
       word besides emit-byte: next-fcode. It sets the increasing fcode 
       number used to emit new fcode words to the specified value. Thus 
       allowing sparse fcode numbering and overwriting previously defined
       fcode words.
       
       This could look like:
       -------------------------------------------
           \ next emitted fcode is 0x053, 2dup.
           tokenizer[ 053 next-fcode ]tokenizer
           \ now define the word.
           : 2dup over over ;
       -------------------------------------------
       
       Note: Toke is capable of creating fcode files that implement words
       defined by the IEEE 1275-1994 standard itself. This is not necessarily
       supported by every Open Firmware implementation.
     
    
    4.2 FCODE-VERSION2
    
       This word generates an FCode header using START1. It should be used 
       with FCODE-END.
    
       
    4.3 FCODE-END
    
       This word creates an END0 FCode and fixes up the FCode header 
       information. Use this with FCODE-VERSION2
    
    
    4.4 PCI-HEADER
    
       This word creates a PCI option ROM header. PCI-HEADER expects 3 
       values on the top of the stack when it is invoked: Vendor#, Device# 
       and Class-Code.
       
       These values are used for the according fields in the PCI Data
       structure that is emitted by PCI-HEADER.  To include these values
       put them on the stack with TOKENIZER[ and ]TOKENIZER.
    
       
    4.5 PCI-END / PCI-HEADER-END
    
       This word completes the PCI header created by PCI-HEADER. It fills out
       the image to a multiple of 512 bytes, and sets the missing values in 
       the PCI header (image-length, revision).
    
       Example:
       HEX
       TOKENIZER[ 1234 5678 ABCDEF ]TOKENIZER
       PCI-HEADER \ generate PCI option rom header
    	FCODE-VERSION2 \ generate FCode header (within PCI image)
            ...
            ...
    	FCODE-END \ terminate FCode in image
       PCI-END \ complete the PCI header and image.
       
       
    4.6 PCI-REVISION / SET-REV-LEVEL
    
       Use this word to change the revision field of the PCI header. Like 
       PCI-HEADER, PCI-REVISION takes it's value from the stack. You can
       write: TOKENIZER[ 23 ]TOKENIZER PCI-REVISION
    
       
    4.7 NOT-LAST-IMAGE
    
       Normally Toke assumes that a PCI image generated by PCI-HEADER is
       the only ROM image in the (EEP)ROM it will be written to. With 
       NOT-LAST-IMAGE Toke sets a flag in the PCI header that other images 
       will follow in the same ROM.
    
       
    4.8 FCODE-DATE
    
       FCODE-DATE creates an FCode string that contains the date of the 
       tokenization. This string could be used to create a device tree 
       property that can be inspected by  drivers, etc. to check when the 
       image was created. The format of the string emitted by FCODE-DATE 
       is MM/DD.YYYY
    
       
    4.9 FCODE-TIME
    
       FCODE-TIME creates an FCode string that contains the time of the 
       tokenization.  This string could be used to create a device tree 
       property that can be inspected by drivers, etc. to check when the
       image was created. The format of the string emitted by FCODE-TIME 
       is HH:MM:SS
    
       
    4.10 ENCODE-FILE
    
       encode a binary file. Use as: encode-file filename
    
     
    5. Text Strings
    
       From "Writing FCode":
       Escape Sequences in Text Strings
    
       -----------------------------------------------------------
       Syntax          Function
       -----------------------------------------------------------
       ""              quote (")
       "n              newline
       "r              carriage return
       "t              tab
       "f              formfeed
       "l              linefeed
       "b              backspace
       "!              bell
       "^x             control x, where x is any printable 
                       character
       "(hh hh)        Sequence of bytes, one byte for each pair
                       of hex digits hh . Non-hex characters will
                       be ignored 
    
       " followed by any other printable character not mentioned
       above is equivalent to that character.
    
       "( means to start parsing pairs of hexadecimal digits as
       one or more 8-bit characters in the range 0x00 through
       0xFF, delimited by a trailing ) and ignoring
       non-hexadecimal digits between pairs of hexadecimal
       digits. Both uppercase and lowercase hexadecimal digits are
       recognized. Since non-hex characters (such as space or
       comma) are ignored between "( and ), these characters make
       useful delimiters. (The "makearray" tool can be used in
       conjunction with this syntax to easily incorporate large
       binary data fields into any FCode Program.)
    
       Any characters thus recognized are appended to any previous
       text in the string being assembled. After the ) is
       recognized, text assembly continues until a trailing
       "<whitespace>.
    
       For example:
    
       " This is "(01 32,8e)abc"nA test! "!"! abcd""hijk"^bl"
                   ^^^^^^^^    ^         ^ ^      ^      ^
                    3 bytes    newline   2 bells  "      control b
    
    6. Contact
    
       Ideas, bug reports, patches, contributions are welcome.
       
            Stefan Reinauer <stepan@openbios.org>