1 Introduction
2 Ground Rules

Building a File System
3 File Systems
4 File Content Data Structure
5 Allocation Cluster Manager
6 Exceptions and Emancipation
7 Base Classes, Testing, and More
8 File Meta Data
9 Native File Class
10 Our File System
11 Allocation Table
12 File System Support Code
13 Initializing the File System
14 Contiguous Files
15 Rebuilding the File System
16 Native File System Support Methods
17 Lookups, Wildcards, and Unicode, Oh My
18 Finishing the File System Class

The Init Program
19 Hardware Abstraction and UOS Architecture
20 Init Command Mode
21 Using Our File System
22 Hardware and Device Lists
23 Fun with Stores: Partitions
24 Fun with Stores: RAID
25 Fun with Stores: RAM Disks
26 Init wrap-up

The Executive
27 Overview of The Executive
28 Starting the Kernel
29 The Kernel
30 Making a Store Bootable
31 The MMC
32 The HMC
33 Loading the components
34 Using the File Processor
35 Symbols and the SSC
36 The File Processor and Device Management
37 The File Processor and File System Management
38 Finishing Executive Startup

Users and Security
39 Introduction to Users and Security
40 More Fun With Stores: File Heaps
41 File Heaps, part 2
42 SysUAF
43 TUser
44 SysUAF API

Terminal I/O
45 Shells and UCL
46 UOS API, the Application Side
47 UOS API, the Executive Side
48 I/O Devices
49 Streams
50 Terminal Output Filters
51 The TTerminal Class
52 Handles
53 Putting it All Together
54 Getting Terminal Input
55 QIO
56 Cooking Terminal Input
57 Putting it all together, part 2
58 Quotas and I/O

UCL
59 UCL Basics
60 Symbol Substitution
61 Command execution
62 Command execution, part 2
63 Command Abbreviation
64 ASTs
65 Expressions, Part 1
66 Expressions, Part 2: Support code
67 Expressions, part 3: Parsing
68 SYS_GETJPIW and SYS_TRNLNM
69 Expressions, part 4: Evaluation

UCL Lexical Functions
70 PROCESS_SCAN
71 PROCESS_SCAN, Part 2
72 TProcess updates
73 Unicode revisted
74 Lexical functions: F$CONTEXT
75 Lexical functions: F$PID
76 Lexical Functions: F$CUNITS
77 Lexical Functions: F$CVSI and F$CVUI
78 UOS Date and Time Formatting
79 Lexical Functions: F$CVTIME
80 LIB_CVTIME
81 Date/Time Contexts
82 SYS_GETTIM, LIB_Get_Timestamp, SYS_ASCTIM, and LIB_SYS_ASCTIM
83 Lexical Functions: F$DELTA_TIME
84 Lexical functions: F$DEVICE
85 SYS_DEVICE_SCAN
86 Lexical functions: F$DIRECTORY
87 Lexical functions: F$EDIT and F$ELEMENT
88 Lexical functions: F$ENVIRONMENT
89 SYS_GETUAI
90 Lexical functions: F$EXTRACT and F$IDENTIFIER
91 LIB_FAO and LIB_FAOL
92 LIB_FAO and LIB_FAOL, part 2
93 Lexical functions: F$FAO
94 File Processing Structures
95 Lexical functions: F$FILE_ATTRIBUTES
96 SYS_DISPLAY
97 Lexical functions: F$GETDVI
98 Parse_GetDVI
99 GetDVI
100 GetDVI, part 2
101 GetDVI, part 3
102 Lexical functions: F$GETJPI

Glossary/Index


Download sources
Download binaries

Lexical functions - F$FAO

In the previous articles we discussed the operation of FAO and the implementation of LIB_FAO. In this article, we look at the UCL lexical function F$FAO, which allows scripts to access LIB_FAO. Much of the following definition comes from the FAO description in article 91. It is excerpted here for completeness. DCL limits F$FAO to 15 parameters. UCL essentially has no limit to the number of parameters that can be used. Here is the definition of the F$FAO function.

F$FAO converts character and numeric input to character strings. Note that while DCL generates ASCII strings, in UCL this function generates UTF-8 strings.

Format
F$FAO(control,argument{,...})

Return Value
A string containing formatted output created from the control argument and the optional arguments.

Arguments
control
Specifies the fixed text of the output string and formatting directives. The directives are described below.

argument{,...}

Specifies arguments that correspond to FAO directives in the control string. The arguments may be integer or string. The order of the arguments must correspond exactly with the order of the directives in the control string, even if some directives require multiple arguments.

Directives
The following directives are supported by F$FAO. String directives:
DirectiveDescription
!ASstring

Zero-filled numeric directives:
DirectiveDescription
!BBConvert a byte value to the ASCII representation of that value in base 2.
!BWConvert a word value to the ASCII representation of that value in base 2.
!BLConvert a longword value to the ASCII representation of that value in base 2.
!BQConvert a quadword value to the ASCII representation of that value in base 2.
!OBConvert a byte value to the ASCII representation of that value in base 8.
!OWConvert a word value to the ASCII representation of that valud in base 8.
!OLConvert a longword value to the ASCII representation of that valud in base 8.
!OQConvert a quadword value to the ASCII representation of that valud in base 8.
!XBConvert a byte value to the ASCII representation of that value in base 16.
!XWConvert a word value to the ASCII representation of that value in base 16.
!XLConvert a longword value to the ASCII representation of that value in base 16.
!XQConvert a quadword value to the ASCII representation of that value in base 16.
!ZBConvert a byte value to the ASCII representation of that value in base 10.
!ZWConvert a word value to the ASCII representation of that value in base 10.
!ZLConvert a longword value to the ASCII representation of that value in base 10.
!ZQConvert a quadword value to the ASCII representation of that value in base 10.

Blank-filled numeric directives:
DirectiveDescription
!UBConvert an unsigned byte value to the ASCII representation of that value in base 10.
!UWConvert an unsigned word value to the ASCII representation of that value in base 10.
!ULConvert an unsigned longword value to the ASCII representation of that value in base 10.
!UQConvert an unsigned quadword value to the ASCII representation of that value in base 10.
!SBConvert a signed byte value to the ASCII representation of that value in base 10.
!SWConvert a signed word value to the ASCII representation of that value in base 10.
!SLConvert a signed longword value to the ASCII representation of that value in base 10.
!SQConvert a signed quadword value to the ASCII representation of that value in base 10.

Other Directives:
DirectiveDescription
!/Inserts a new line (carriage return and linefeed). It takes no parameters.
!_Inserts a horizontal tab (ASCII 9). It takes no parameters.
!^Inserts a form feed. It takes no parameters.
!!Inserts an exclamation point. It takes no parameters
!%SInserts the letter S if the most recently converted numeric value is not 1. If the character before the directive is upper case, an upper case S is inserted, otherwise a lowercase s is inserted.
!%TInserts the system time. The parameter is the datetime stamp. If the parameter is 0, the current time is inserted.
!%USame as !UQ.
!%IConverts a UIC to the account name. If an invalid UIC is specified, the directive is treated as !UQ.
!%DInserts the system date and time. The parameter is the timestamp. If the parameter is 0, the current date/time is inserted.
!n%CConditional. See discussion of conditionals below.
!%EElse portion of conditional. See discussion of conditionals below.
!%FEnd of conditional. See discussion of conditionals below.
!n<See next directive.
!>The preceeding directive and this one are used together to define an output field that has a width of n. Within this field are displayed all directives between the !n< and !> directives. The field is blank-filled on the right to make it n characters wide if necessary. All directives within this field are left-justified and blank-filled. Note that these can be nested.
!n*cRepeats the character c in the output n times.
!-Reuse the most recently used parameter value.
!+Skip the next parameter value.

Conditionals
!%nC, !%E, and !%F are used together to insert values depending upon parameter values. This is primarily for use with plurals. The general format is:
!%nCa!%Eb!%F
If n matches the last parameter value, then a is inserted, otherwise b is inserted. Example:
!ZB !%1Cchild!%Echildren!%F
In this example, if the first parameter is 1, the output would be:
1 child
But if the first parameter is not 1, the output would be:
n children
where "n" is the value of the first parameter.

The following table illustrates how the directives interact with width and filling.
Directive TypeDefault output widthWhen explicit width is greater than defaultWhen explicit width is less than default
!BB8Right justify and blank fillResult truncated on left
!BW16Right justify and blank fillResult truncated on left
!BL32Right justify and blank fillResult truncated on left
!BQ64Right justify and blank fillResult truncated on left
!OB3Right justify and blank fillResult truncated on left
!OW6Right justify and blank fillResult truncated on left
!OL11Right justify and blank fillResult truncated on left
!OQ22Right justify and blank fillResult truncated on left
!HB2Right justify and blank fillResult truncated on left
!HW4Right justify and blank fillResult truncated on left
!HL8Right justify and blank fillResult truncated on left
!HQ16Right justify and blank fillResult truncated on left
Unsigned zero-filled decimalAs many characters as are necessaryRight justify and blank fillField completely filled with asterisks (*_
Signed or unsigned deciumalAs many characters as are necessaryRight justify and zero-filled
StringsAs many characters as in the stringLeft justify and blank fill to specified lengthTruncate on right

Example
$ X = F$FAO("NUMBER OF FILES: !SL",COUNT)
This would insert the numeric value of COUNT into the string at the point where "!SL" occurs. For instance, if COUNT was equal to 105, the resulting string would be "NUMBER OF FILES: 105".

        Function_FAO : begin
                           if( Missing_Parentheses( '(' ) ) then
                           begin
                               exit ;
                           end ;
                           if( Parse_FAO( Err, Context ) ) then
                           begin
                               exit ;
                           end ;
                           if( Missing_Parentheses( ')' ) ) then
                           begin
                               exit ;
                           end ;
                           S := Context ;
                       end ;
This code is added to the Function_Reference function.

function Parse_FAO( var Err : integer ; var Context : string ) : boolean ;

var Control, Count, Directive, S : string ;
    I : integer ;
    SL : TStringList ;

var A : array of int64 ;
    Array_S, Data : string ;

var IL : TInteger_List ;
    Integer_Value, P : int64 ;

begin // Parse_FAO
    Result := False ; // Assume success

    // Get control value...
    Control := Get_Parameter( Err, Context ) ;
    if( Err <> 0 ) then
    begin
        Result := True ; // Assume error
        exit ;
    end ;
This function first obtains the control string parameter. If there was an error, we exit.

    // Process control string to see what items are used...
    SL := TStringList.Create ;
    I := 1 ;
    while( I <= length( Control ) ) do
    begin
        if( Control[ I ] = '!' ) then
        begin
            if( copy( Control, I + 1, 1 ) = '!' ) then // !! construct
            begin
                I := I + 2 ;
                continue ;
            end ;
            if( copy( Control, I + 1, 1 ) = '/' ) then // !/ construct
            begin
                I := I + 2 ;
                continue ;
            end ;
            if( copy( Control, I + 1, 1 ) = '_' ) then // !_ construct
            begin
                I := I + 2 ;
                continue ;
            end ;
            if( copy( Control, I + 1, 1 ) = '^' ) then // !^ construct
            begin
                I := I + 2 ;
                continue ;
            end ;
            if( copy( Control, I + 1, 1 ) = '-' ) then // !- construct
            begin
                I := I + 2 ;
                continue ;
            end ;
            if( copy( Control, I + 1, 1 ) = '+' ) then // !+ construct
            begin
                I := I + 2 ;
                continue ;
            end ;
            if( copy( Control, I + 1, 1 ) = '>' ) then // !> construct
            begin
                I := I + 2 ;
                continue ;
            end ;
            Parse_Directive( I ) ;
            if( Err <> 0 ) then
            begin
                exit ;
            end ;

            if( Directive = '%U' ) then
            begin
                Add_To_SL( 'i' ) ;
            end ;
            if( Directive = '%D' ) then // System Date/Time
            begin
                Add_To_SL( 'i' ) ;
            end else
            if( Directive = '%I' ) then // Identifier for UIC
            begin
                Add_To_SL( 'i' ) ;
            end else
            if( Directive = '%T' ) then // System Time
            begin
                Add_To_SL( 'i' ) ;
            end else
            if( Directive = 'AC' ) then
            begin
                Err := UCL_INVDIR ;
                exit ;
            end else
            if( Directive = 'AS' ) then
            begin
                if( Control[ I - 1 ] = ')' ) then
                begin
                    Control[ I - 2 ] := 'D' ; // Convert AS) to AD)
                end else
                begin
                    Control[ I - 1 ] := 'D' ; // Convert AS to AD
                end ;
            end else
            if( Directive = 'AF' ) then
            begin
                Err := UCL_INVDIR ;
                exit ;
            end else
            if( Directive = 'AB' ) then
            begin
                Err := UCL_INVDIR ;
                exit ;
            end else
            if( Directive = 'AZ' ) then
            begin
                Err := UCL_INVDIR ;
                exit ;
            end ;
        end else
        begin
            inc( I ) ; // Regular character
        end ;
    end ;
DCL does very little checking on the parameters passed to the F$FAO function. This allows the user to use specifications that can result in unpredictable behavior, including access violation exceptions - for instance, if fewer parameters are provided than the number of directives specified. DCL passes on whatever directives are supplied, even if they are unsupported, such as !AF and !AZ, which also can cause unpredictable results. We want to be a little bit more careful in UCL. In addition, UCL symbols are untyped, whereas DCL can type symbols as string or integer. Thus, we have to be careful about what we pass to the LIB_FAOL function.

So, we parse through the control string, looking for directives, and then adding "s" (for string) or "i" (for integer) to the string list. I is the index into the control string. When we are done, the string list will tell us how many directives were provided and what types they expect. As we encounter directives, we check for unsupported ones and return an error if found. We will address the Parse_Directive and Add_To_SL functions in a bit.

Understand that this is not a guarantee that there won't be any errors, but it is much safer to use than DCL. I leave it as an exercise for the reader to come up with a scenario in which even UCL could have an unexpected error. Hint: one scenario has to do with the !- directive.

    // Get parameters...
    Data := '' ;
    I := 0 ;
    IL := TInteger_List.Create ; // List of offsets into Data

    S := Get_Token ;
    while( S = ',' ) do // For each parameter provided
    begin
        S := Get_Parameter( Err, Context, True ) ;
        if( Err <> 0 ) then
        begin
            IL.Free ;
            exit ;
        end ;
Now we process the parameter list. We loop for as long as we have parameters (until we reach a non-comma delimiter). I is used as the parameter index. In this loop we are going to build the parameter list to pass to LIB_FAOL. We will also build a block of string data, if any strings are provided.

        if( ( I < SL.Count ) and ( SL[ I ] = 's' ) ) then
        begin
            Add( length( S ) ) ;
            IL.Add( length( A ) ) ;
            Add( length( Data ) ) ;
            Data := Data + S ;
        end else
        begin
            if( not trystrtoint64( S, Integer_Value ) ) then
            begin
                Integer_Value := 0 ;
                P := length( S ) ;
                if( P > sizeof( Integer_Value ) ) then
                begin
                    P := sizeof( Integer_Value ) ;
                end ;
                move( PChar( S )[ 0 ], Integer_Value, P ) ;
            end ; // if( I < SL.Count )
            Add( Integer_Value ) ;
        end ;
        inc( I ) ;
        S := Get_Token ;
    end ; // while( S = ',' )
    Parser.Put_Token( S ) ;
If the next directive is a string, we add the string length to the FAO parameter list, add the length of the parameter list to the IL list, add the offset of the string in the Data string buffer, and add the string to the Data string (which is the block of string data).

If the directive is not a string, we see if the passed argument is an integer value. If not, we treat the first eight bytes as a binary integer (or less than eight bytes if the string is shorter than that). Either way, we add the integer value to the FAO parameter list.

Next we increment the parameter index and grab the next token. If it is a comma, the loop continues to the next parameter. Otherwise, we drop out of the loop and return the token.

    for I := 0 to IL.Count - 1 do
    begin
        A[ IL[ I ] ] := A[ IL[ I ] ] + int64( PChar( Data ) ) ;
    end ;
    IL.Free ;
Each time we append text to the Data string in the above loop, the string has to be resized to hold the additional text. This reallocation of memory can cause the location of the string text to move with each append. Thus, we could not add parameters that were pointers to string text, because by the time we were done with the loop, the address of the string may have changed, rendering the address we put on the FAO parameter list invalid. Thus, we go back through the parameter list, adding the final address of Data to the offset added in the loop. The IL list contains the offsets in the FAO parameter list of those string offsets, which allows us to "backpatch" them to the actual address of the corresponding string.

    // Build parameter list...
    setlength( S, sizeof( int64 ) ) ;
    setlength( Array_S, length( A ) * sizeof( int64 ) ) ;
    for I := 0 to length( A ) - 1 do
    begin
        move( A[ I ], PChar( Array_S )[ I * sizeof( int64 ) ], sizeof( int64 ) ) ;
    end ;

    // Execute the call...
    Context := FAOL( Control, int64( @PChar( Array_S )[ 0 ] ) ) ;
end ; // Parse_FAO
Variable A is a dynamic array, which we use since we don't know how many parameters we will need until we finish processing the F$FAO arguments. However, dynamic arrays are not stored as a consecutive series of bytes. Thus, we must convert from that format into a sequence of contiguous bytes. Array_S is a string which we use to accumulate this contiguous buffer of FAO parameters. First, we set the length of the string, then we loop through the dynamic array, and copy each integer value from the dynamic array to the parameter buffer.

With the string buffer and parameter list built, we now call FAOL to process the control string. We will address the FAOL function later in this article.

procedure Parse_Directive( var I : integer ) ;

var Starting, Ending, Width : integer ;

begin // Parse_Directive
    // Setup...
    Starting := I ;
    Ending := I ;
    Directive := '' ;
    Count := 1 ;

    // Handle width...
    Process_Width ;

    // Handle count...
    if( copy( Control, Ending + 1, 1 ) = '(' ) then // Count + width
    begin
        inc( Ending ) ;
        Count := Width ;
        Process_Width ;
        if( copy( Control, Ending + 1, 1 ) = '@' ) then
        begin
            Err := UCL_INVDIR ;
            exit ;
        end ;
        Directive := copy( Control, Ending + 1, 2 ) ;
        Ending := Ending + 2 ;
        if( copy( Control, Ending + 1, 1 ) = ')' ) then
        begin
            inc( Ending ) ;
        end ;
    end else
    begin
        if( copy( Control, Ending + 1, 1 ) = '@' ) then
        begin
            Err := UCL_INVDIR ;
            exit ;
        end ;
        Directive := copy( Control, Ending + 1, 2 ) ;
        Ending := Ending + 2 ;
    end ;

    // Default the width...
    if( copy( Directive, 1, 1 ) = '<' ) then
    begin
        setlength( Directive, 1 ) ;
        dec( Ending ) ;
    end else
    if( copy( Directive, 1, 1 ) = 'B' ) then // Binary
    begin
        Add_To_SL( 'i' ) ;
    end else
    if( copy( Directive, 1, 1 ) = 'X' ) then // Hexadecimal
    begin
        Add_To_SL( 'i' ) ;
    end else
    if( copy( Directive, 1, 1 ) = 'O' ) then // Octal
    begin
        Add_To_SL( 'i' ) ;
    end else
    if( copy( Directive, 1, 1 ) = 'Z' ) then // Zero-filled decimal
    begin
        Add_To_SL( 'i' ) ;
    end else
    if( copy( Directive, 1, 1 ) = 'S' ) then // Decimal
    begin
        Add_To_SL( 'i' ) ;
    end else
    if( copy( Directive, 1, 1 ) = 'A' ) then // String
    begin
        Add_To_SL( 's' ) ;
    end ;
    I := Ending + 1 ;
end ; // Parse_Directive
The local Parse_Directive function parses the next directive in the control string. This is a trimmed-down version of the function with the same name that we covered in the last article, so we won't cover it blow-by-blow. What is to be noted is that we add either "i" or "s", depending upon the directive, to the string list. We also catch any indirect references (@) and return an error if found. This is because Parse_FAO builds the parameter list itself and if we allowed an indirect reference to be passed to FAO would result in unpredictable behavior since it would be using a value as an address to a value.

      procedure Process_Width ;

      var I : integer ;

      begin
          if( copy( Control, Ending + 1, 1 ) = '#' ) then
          begin
              SL.Add( 'i' ) ;
              inc( Ending ) ;
          end else
          if( pos( copy( Control, Ending + 1, 1 ), '0123456789' ) > 0 ) then
          begin
              I := Ending ;
              inc( Ending ) ;
              while( pos( copy( Control, Ending + 1, 1 ), '0123456789' ) > 0 ) do
              begin
                  inc( Ending ) ;
              end ;
              Width := strtoint( copy( Control, I + 1, Ending - I ) ) ;
          end ;
      end ;
Parse_Directive uses the Process_Width local function to skip past the width. But we also save the width in case it is a repeat value.

procedure Add_To_SL( const S : string ) ;

begin
    while( Count > 0 ) do
    begin
        dec( Count ) ;
        SL.Add( S ) ;
    end ;
end ;


procedure Add( I : int64 ) ;

begin
    setlength( A, length( A ) + 1 ) ;
    // Append I to end of array A
    A[ length( A ) - 1 ] := I ;
end ;
The Add_To_SL function adds a value to the SL list. If the current directive has a count, we use that to repeatedly add the value to the list.
The Add function adds a value to the dynamic array of parameters. First, it extends the size of the array, then sets the last value.

function FAO( Control : string ; P1 : int64 = 0 ;
    P2 : int64 = 0 ; P3 : int64 = 0 ; P4 : int64 = 0 ; P5 : int64 = 0 ;
    P6 : int64 = 0 ; P7 : int64 = 0 ; P8 : int64 = 0 ; P9 : int64 = 0 ;
    P10 : int64 = 0 ; P11 : int64 = 0 ; P12 : int64 = 0 ; P13 : int64 = 0 ;
    P14 : int64 = 0 ; P15 : int64 = 0 ; P16 : int64 = 0 ; P17 : int64 = 0 ) : string ;

var OutLen : int64 ;
    SRB : TSRB ;

begin
    Set_String( Control, SRB ) ;
    setlength( Result, length( Control ) ) ;
    while( True ) do
    begin
        OutLen := length( Result ) ;
        LIB_FAO( int64( @SRB ), int64( @OutLen ), int64( Pchar( Result ) ), P1, P2,
            P3, P4, P5, P6, P7, P8, P9, P10, P11, P12, P13, P14, P15, P16, P17 ) ;
        if( OutLen < length( Result ) ) then // Result fit in our buffer
        begin
            Setlength( Result, OutLen ) ;
            exit ;
        end else
        begin
            setlength( Result, length( Result ) * 2 ) ;
        end ;
    end ;
end ;


function FAOL( Control : string ; Parameters : int64 = 0 ) : string ;

var OutLen : int64 ;
    SRB : TSRB ;

begin
    Set_String( Control, SRB ) ;
    setlength( Result, length( Control ) ) ;
    while( True ) do
    begin
        OutLen := length( Result ) ;
        LIB_FAOL( int64( @SRB ), int64( @OutLen ), int64( Pchar( Result ) ), Parameters ) ;
        if( OutLen < length( Result ) ) then // Result fit in our buffer
        begin
            Setlength( Result, OutLen ) ;
            exit ;
        end else
        begin
            setlength( Result, length( Result ) * 2 ) ;
        end ;
    end ;
end ;
We've added these two functions to PasStarlet. They are simply wrappers for the LIB_FAO and LIB_FAOL functions. They convert from a pascal string to a TSRB structure and pass that along to Starlet.

In both functions, we make a call the the LIB routine in a loop. We need to have a result buffer that is large enough for the processed string. However, we would have to implement the complete FAO service here to know what the length of the final result is. To avoid that, we start with a result buffer that is the length of the control string. If the result length is shorter than that, we're done and we can return the result and exit. Otherwise, we double the size of the result buffer and try again. This continues until the result size is smaller than the result buffer meaning that we have the whole result in a string. We could also have addded a new call that calculated the FAO result and returned the length, then use that to size the buffer before calling FAO. Or, we could have added another result address that FAO could write the actual length to. But we want starlet to be VMS-compatible (we might add some extensions in the future). We could have written FAO as a Pascal string function that simply returned the full string (LIB_FAO could call that instead of doing the work itself). However, it is important to realize that Starlet is a language-indepedent library, so we cannot expect Pascal-compatible strings from it. Our implementation should only require two calls in most cases, which would be the same amount of overhead if we added a "get size" function that we called before making the FAO call.

Now that we're done with the articles on FAO, in the next article, we'll look at the next lexical function.

 

Copyright © 2020 by Alan Conroy. This article may be copied in whole or in part as long as this copyright is included.