1 Introduction
2 Ground Rules

Building a File System
3 File Systems
4 File Content Data Structure
5 Allocation Cluster Manager
6 Exceptions and Emancipation
7 Base Classes, Testing, and More
8 File Meta Data
9 Native File Class
10 Our File System
11 Allocation Table
12 File System Support Code
13 Initializing the File System
14 Contiguous Files
15 Rebuilding the File System
16 Native File System Support Methods
17 Lookups, Wildcards, and Unicode, Oh My
18 Finishing the File System Class

The Init Program
19 Hardware Abstraction and UOS Architecture
20 Init Command Mode
21 Using Our File System
22 Hardware and Device Lists
23 Fun with Stores: Partitions
24 Fun with Stores: RAID
25 Fun with Stores: RAM Disks
26 Init wrap-up

The Executive
27 Overview of The Executive
28 Starting the Kernel
29 The Kernel
30 Making a Store Bootable
31 The MMC
32 The HMC
33 Loading the components
34 Using the File Processor
35 Symbols and the SSC
36 The File Processor and Device Management
37 The File Processor and File System Management
38 Finishing Executive Startup

Users and Security
39 Introduction to Users and Security
40 More Fun With Stores: File Heaps
41 File Heaps, part 2
42 SysUAF
43 TUser
44 SysUAF API

Terminal I/O
45 Shells and UCL
46 UOS API, the Application Side
47 UOS API, the Executive Side
48 I/O Devices
49 Streams
50 Terminal Output Filters
51 The TTerminal Class
52 Handles
53 Putting it All Together
54 Getting Terminal Input
55 QIO
56 Cooking Terminal Input
57 Putting it all together, part 2
58 Quotas and I/O

UCL
59 UCL Basics
60 Symbol Substitution
61 Command execution
62 Command execution, part 2
63 Command Abbreviation
64 ASTs
65 Expressions, Part 1
66 Expressions, Part 2: Support code
67 Expressions, part 3: Parsing
68 SYS_GETJPIW and SYS_TRNLNM
69 Expressions, part 4: Evaluation

UCL Lexical Functions
70 PROCESS_SCAN
71 PROCESS_SCAN, Part 2
72 TProcess updates
73 Unicode revisted
74 Lexical functions: F$CONTEXT
75 Lexical functions: F$PID
76 Lexical Functions: F$CUNITS
77 Lexical Functions: F$CVSI and F$CVUI
78 UOS Date and Time Formatting
79 Lexical Functions: F$CVTIME
80 LIB_CVTIME
81 Date/Time Contexts
82 SYS_GETTIM, LIB_Get_Timestamp, SYS_ASCTIM, and LIB_SYS_ASCTIM
83 Lexical Functions: F$DELTA_TIME
84 Lexical functions: F$DEVICE
85 SYS_DEVICE_SCAN
86 Lexical functions: F$DIRECTORY
87 Lexical functions: F$EDIT and F$ELEMENT
88 Lexical functions: F$ENVIRONMENT
89 SYS_GETUAI
90 Lexical functions: F$EXTRACT and F$IDENTIFIER
91 LIB_FAO and LIB_FAOL
92 LIB_FAO and LIB_FAOL, part 2
93 Lexical functions: F$FAO
94 File Processing Structures
95 Lexical functions: F$FILE_ATTRIBUTES
96 SYS_DISPLAY
97 Lexical functions: F$GETDVI
98 Parse_GetDVI
99 GetDVI
100 GetDVI, part 2
101 GetDVI, part 3
102 Lexical functions: F$GETJPI
103 GETJPI
104 Lexical functions: F$GETSYI
105 GETSYI
106 Lexical functions: F$INTEGER, F$LENGTH, F$LOCATE, and F$MATCH_WILD
107 Lexical function: F$PARSE
108 FILESCAN
109 SYS_PARSE
110 Lexical Functions: F$MODE, F$PRIVILEGE, and F$PROCESS
111 File Lookup Service
112 Lexical Functions: F$SEARCH
113 SYS_SEARCH
114 F$SETPRV and SYS_SETPRV
115 Lexical Functions: F$STRING, F$TIME, and F$TYPE
116 More on symbols
117 Lexical Functions: F$TRNLNM
118 SYS_TRNLNM, Part 2
119 Lexical functions: F$UNIQUE, F$USER, and F$VERIFY
120 Lexical functions: F$MESSAGE
121 TUOS_File_Wrapper
122 OPEN, CLOSE, and READ system services

UCL Commands
123 WRITE
124 Symbol assignment
125 The @ command
126 @ and EXIT
127 CRELNT system service
128 DELLNT system service
129 IF...THEN...ELSE
130 Comments, labels, and GOTO
131 GOSUB and RETURN
132 CALL, SUBROUTINE, and ENDSUBROUTINE
133 ON, SET {NO}ON, and error handling
134 INQUIRE
135 SYS_WRITE Service
136 OPEN
137 CLOSE
138 DELLNM system service
139 READ
140 Command Recall
141 RECALL
142 RUN
143 LIB_RUN
144 The Data Stream Interface
145 Preparing for execution
146 EOJ and LOGOUT
147 SYS_DELPROC and LIB_GET_FOREIGN

CUSPs and utilities
148 The I/O Queue
149 Timers
150 Logging in, part one
151 Logging in, part 2
152 System configuration
153 SET NODE utility
154 UUI
155 SETTERM utility
156 SETTERM utility, part 2
157 SETTERM utility, part 3
158 AUTHORIZE utility
159 AUTHORIZE utility, UI
160 AUTHORIZE utility, Access Restrictions
161 AUTHORIZE utility, Part 4
162 AUTHORIZE utility, Reporting
163 AUTHORIZE utility, Part 6
164 Authentication
165 Hashlib
166 Authenticate, Part 7
167 Logging in, part 3
168 DAY_OF_WEEK, CVT_FROM_INTERNAL_TIME, and SPAWN
169 DAY_OF_WEEK and CVT_FROM_INTERNAL_TIME
170 LIB_SPAWN
171 CREPRC
172 CREPRC, Part 2
173 COPY
174 COPY, part 2
175 COPY, part 3
176 COPY, part 4
177 LIB_Get_Default_File_Protection and LIB_Substitute_Wildcards
178 CREATESTREAM, STREAMNAME, and Set_Contiguous
179 Help Files
180 LBR Services
181 LBR Services, Part 2
182 LIBRARY utility
183 LIBRARY utility, Part 2
184 FS Services
185 FS Services, Part 2
186 Implementing Help
187 HELP
188 HELP, Part 2
189 DMG_Get_Key and LIB_Put_Formatted_Output
190 LIBRARY utility, Part 3
191 Shutting Down UOS
192 SHUTDOWN
193 WAIT
194 SETIMR
195 WAITFR and Scheduling
196 REPLY, OPCOM, and Mailboxes
197 REPLY utility
198 Mailboxes
199 BRKTHRU
200 OPCOM

Glossary/Index


Downloads

LIBRARY utility, Part 3

I had hoped that we would be starting on the next CUSP in this article, however while updating documentation and putting the final touches on the binary distribution, I discovered that I had left something out of the previous articles on LIBRARY and LBR services. No excuse - it just slipped through the cracks. So we will address the LIBRARY/LIST functionality before we move on. /LIST was in the user documentation, so we won't reproduce that here.

    // List...
    if( List ) then
    begin
        fillchar( Descriptors, sizeof( Descriptors ), sizeof( Descriptors ) ) ;
        Descriptors[ 0 ].MBO := $FFFF ;
        Descriptors[ 0 ].MBMO := -1 ;
        Descriptors[ 0 ].Item_Code := LBI_FREE ;
        Descriptors[ 0 ].Buffer_Length := sizeof( int64 ) ;
        Descriptors[ 0 ].Buffer_Address := int64( @Free_Space ) ;
        Descriptors[ 0 ].Return_Length_Address := int64( @TempL ) ;
        Descriptors[ 1 ].MBO := $FFFF ;
        Descriptors[ 1 ].MBMO := -1 ;
        Descriptors[ 1 ].Item_Code := LBI_SIZE ;
        Descriptors[ 1 ].Buffer_Length := sizeof( int64 ) ;
        Descriptors[ 1 ].Buffer_Address := int64( @Total_Space ) ;
        Descriptors[ 1 ].Return_Length_Address := int64( @TempL ) ;
        GETLBI( Context, int64( @Descriptors ) ) ;
        Put_Output( 'Directory of ' + Type_Name( Typ ) + ' library ' +
            trim( UUI.Get_Component_By_Name( 'input' ).Text ) + ' on ' +
            ASCTIM( 0 ) ) ;
        Put_Output( 'Used space: ' + inttostr( Total_Space - Free_Space ) +
            '  Free space: ' + inttostr( Free_Space ) ) ;
        Search_Context := 0 ;
        setlength( ModName, 1024 ) ;
        Set_String( ModName, SRB ) ;
        SL := TStringList.Create ;
        while( SRB.Length > 0 ) do
        begin
            Find( Context, '*', Search_Context, SRB ) ;
            S := ModName ;
            setlength( S, SRB.Length ) ;
            SL.Add( S ) ;
            SRB.Length := 1024 ;
        end ;
        SL.Sort ;
        Put_Output( 'Number of modules: ' + inttostr( SL.Count ) ) ;
        for I := 0 to SL.Count - 1 do
        begin
            Put_Output( SL[ I ] ) ;
        end ;
        SL.Free ;
        Find_Close( Context, Search_Context ) ;
    end ; // if( List )
This code is added near the end of the main Library code. If the /LIST switch was specified, we get information from the library via the GETLBI function for display. We will look at GETLBI later in this article. Two items we want are the total space in the library and the amount of free (unused) space. We write a heading with general information about the library. Included in this heading are the free and total space in the library. Type_Name returns the library type in text form and we cover that in a moment.

Next we loop through the modules in the library, building a list of them. Note that the order we get the modules back is unpredictable, thus we sort the list when done so that we can output the sorted list to the user. The Find function allows us to iterate through all modules until the returned length is 0. Finally, we clean up by freeing the list and calling Find_Close to free the search context. We will cover Find and Find_Close later in the article.

function Type_Name( Typ : integer ) : string ;

begin
    case Typ of
        LBR_C_TYP_HLP : Result := 'HELP' ;
        LBR_C_TYP_MLB : Result := 'MACRO' ;
        LBR_C_TYP_OBJ : Result := 'OBJECT' ;
        LBR_C_TYP_TXT : Result := 'TEXT' ;
        else Result := 'UNKNOWN' ;
    end ; // case Typ
end ;
This function merely converts a library type to a string representation. The reason why they are all uppercase is because VMS does the same.

function GetLBI( Context, List : int64 ) : int64 ;

begin
    Result := LBR_GetLBI( int64( @Context ), List ) ;
end ;


function Find( Context : int64 ; Name : string ; var Index : int64 ;
    var Res : TSRB ) : int64 ;

var SRB : TSRB ;

begin
    Set_String( Name, SRB ) ;
    Result := LBR_Find( int64( @Context ), int64( @SRB ), int64( @Index ), int64( @Res ) ) ;
end ;


function Find_Close( Context : int64 ; var Index : int64 ) : int64 ;

begin
    Result := LBR_Find_Close( int64( @Context ), int64( @Index ) ) ;
end ;
These wrapper functions are added to the LBRPas module. Like other functions in this unit, they provide a simplified interface to LBR services.


The three new LBR services called in the above code are not found in VMS. Apparently the VMS librarian grabs this information directly from the file without using an LBR service. We want to make sure that all UOS utilities (with only a couple of exceptions) derive all of their functionality from system services. That way a user can write their own utilities to do anything they want via documented services. Here is the user documentation for the three new LBR services.

LBR$Find
Find modules

LBR$Find iterates over modules in an open library.

Format:

LBR$Find context name index result
Returns:
Returns a condition code.
Arguments:
context
The address of a 64-bit context value returned by LBR$INI_CONTROL that indicates the library to iterate through.

name

The address of an SRB with the name of the module to lookup. This can be a wildcard specification, using asterisks (*) and question marks (?). This is ignored if the value pointed to by index is not 0.

index

The address of a 64-bit lookup context which is used to iterate over matching modules. If 0 is passed as the argument, an error is returned. The value pointed to must be 0 to retrieve the first matching module. The value is managed by the LBR routines and should not be modified by the caller, except to initially set it to 0.

result

The address of an SRB that points to the buffer to receive the name (and the length of the name) upon lookup. On call, the SRB length must indicate the maximum length of the buffer. On return, the length will be the length of the returned name. However, if the buffer is too small, only as much of the name that fits in the buffer will be returned. If no (more) matches are found, the length is set to 0.

Description

Returns the all module names matching the passed name. This call is designed to be called repeatedly to iterate over multiple modules that all match the specification. On the first call, index must point to a value of 0 and the name must not be null. LBR$Find will update the index value so that the next call will find the next match. On subsequent calls, the name is ignored. When no more matching modules are found, the result length will be set to 0. Note that the names are not returned in any specific order.

The typical usage will follow these steps:
Set index value to 0
Set result buffer length to the length of the buffer
while buffer length > 0
call LBR$Find
if buffer length > 0 then buffer contains the name of the next module, otherwise exit the loop

Condition Codes Returned:
LBR_ILLCTL Specified context is not valid.
LBR_LIBNOTOPN Specified library is not open.
UOSErr_Missing_Value No index address was provided.
SS_NORMAL Successful completion.

LBR$Find_Close
Clean up after finding modules

LBR$Find_Close cleans up the context used by LBR$Find, including releasing any allocated memory.

Format:

LBR$Find_Close context index

Returns:

Returns a condition code.

Arguments:
context

The address of a 64-bit context value returned by LBR$INI_CONTROL that indicates the library associated with the operation.

index

The address of a 64-bit lookup context passed to LBR$Find.

Description Closes the context used by LBR$Find, which is specified by the value pointed to by index. Note that after this is done, using the context value with LBR$Find will result in an error.

Condition Codes Returned:
LBR_ILLCTL Specified context is not valid.
LBR_LIBNOTOPN Specified library is not open.
UOSErr_Missing_Value No index address was provided.
SS_NORMAL Successful completion.

LBR$Get_LBI
Get library information

LBR$Get_LBI returns information about a library.

Format:

LBR$Get_LBI context list

Returns:

Returns a condition code.

Arguments:
context

The address of a 64-bit context value returned by LBR_INI_CONTROL that indicates the library to access.

list

The address of a list of descriptors which indicate what information to return. The valid items codes are listed below.

Descriptors:
Offset byteSize in bytesDescription
02Must be -1
22Item code
44Must be -1
88Result buffer address
168Result buffer size
248Result length

Description
Returns the requested information about a library. The following item codes are valid for the passed descriptor list:
ItemReturn typeInformation returned
LBI_FREEintegerTotal free space in library, in bytes.
LBI_SIZEintegerTotal size of the library, in bytes.

Condition Codes Returned:
LBR_ILLCTL Specified context is not valid.
LBR_LIBNOTOPN Specified library is not open.
LBR_NOTFOUND Specified module is not found in the library.
SS_BADPARAM An invalid item code was passed in a descriptor.
SS_NORMAL Successful completion.

Before we look at the source code, let's address one issue. You might think that passing a list of descriptors to return - at most - two items is overkill in LBR_GETLBI. But we will be adding more items in the future. These were simply the ones needed for the changes to the LIBRARY utility.

Also, you may recall that we plan to use the FS$ services in LBR rather than directly accessing the file system. That is still the plan, but I haven't had time to make those changes yet, so the following code doesn't use FS$ services.

function LBR_GetLBI( Context, List : int64 ) : int64 ;

var Buff : PChar ;
    C : LBR_Context ;
    Descriptor : TSYS_Descriptor ;
    Offset : integer ;
    Size, Value : int64 ;

begin
    // Set up...
    Result := 0 ;
    if( Context = 0 ) then
    begin
        Result := LBR_ILLCTL ;
        exit ;
    end ;
    Context := Pint64( Context )^ ;
    if( Context = 0 ) then
    begin
        Result := LBR_ILLCTL ;
        exit ;
    end ;
    C := LBR_Context( Context ) ;
    if( C.Sanity <> #3'LBR' ) then
    begin
        Result := LBR_ILLCTL ;
        exit ;
    end ;
    if( not C.Opened ) then
    begin
        Result := LBR_LIBNOTOPN ;
        exit ;
    end ;
    if( List = 0 ) then
    begin
        exit ;
    end ;
First we do the sanity checks that are done for most of the LBR services. Then we see if no list is passed, we exit immediately since there is no work to do.

    // Process the list...
    Buff := PAnsiChar( List ) ;
    Offset := 0 ;
    while( true ) do
    begin
        move( Buff[ Offset ], Descriptor, sizeof( Descriptor ) ) ;
        Offset := Offset + sizeof( Descriptor ) ;
        if(
            ( Descriptor.Buffer_Length = 0 )
            or
            ( Descriptor.Buffer_Address = 0 )
            or
            ( Descriptor.Return_Length_Address = 0 )
          ) then // End of list
        begin
            exit ;
        end ;
        if( Descriptor.Item_Code = LBI_SIZE ) then
        begin
            Value := C.FS.Store.Max_Storage ;
            move( Value, PChar( Descriptor.Buffer_Address )[ 0 ], sizeof( int64 ) ) ;
            if( Descriptor.Return_Length_Address <> 0 ) then
            begin
                Value := sizeof( int64 ) ;
                move( Value, PChar( Descriptor.Return_Length_Address )[ 0 ], sizeof( int64 ) ) ;
            end ;
        end else
        if( Descriptor.Item_Code = LBI_FREE ) then
        begin
            Value := C.FS.Store.SpaceAvail ;
            move( Value, PChar( Descriptor.Buffer_Address )[ 0 ], sizeof( int64 ) ) ;
            if( Descriptor.Return_Length_Address <> 0 ) then
            begin
                Value := sizeof( int64 ) ;
                move( Value, PChar( Descriptor.Return_Length_Address )[ 0 ], sizeof( int64 ) ) ;
            end ;
        end else
        begin
            Result := UOSErr_Bad_Parameter ;
            exit ;
        end ;
    end ;
end ; // LBR_Get_LBI
This is similar to other code we've covered in the past that deals with descriptor lists. We start with a pointer to the first item, read the descriptor, increment the pointer to the next item, and keep looping until we find a descriptor that is zeroes. Then depending upon the code for the item, we return the proper information from the library in the file system file. If an unrecognized item is found we return with an error.

function LBR_Find( Context, Name, Index, Res : int64 ) : int64 ;

var C : LBR_Context ;
    I : int64 ;
    P : pointer ;
    S : string ;

begin
    // Set up...
    Result := 0 ;
    if( Context = 0 ) then
    begin
        Result := LBR_ILLCTL ;
        exit ;
    end ;
    Context := Pint64( Context )^ ;
    if( Context = 0 ) then
    begin
        Result := LBR_ILLCTL ;
        exit ;
    end ;
    C := LBR_Context( Context ) ;
    if( C.Sanity <> #3'LBR' ) then
    begin
        Result := LBR_ILLCTL ;
        exit ;
    end ;
    if( not C.Opened ) then
    begin
        Result := LBR_LIBNOTOPN ;
        exit ;
    end ;
    if( Index = 0 ) then
    begin
        Result := UOSErr_Missing_Value ;
        exit ;
    end ;
    if( Res = 0 ) then
    begin
        exit ;
    end ;
    I := Pint64( Index )^ ;
    if( ( Name = 0 ) or ( I <> 0 ) ) then
    begin
        S := '' ;
    end else
    begin
        S := Get_String( PSRB( Name )^ ) ;
    end ;
Again, most of this matches what we've seen before. After the validation of the library context, we exit with an error if the passed index address is 0. Then if the result address (Res) is not provided, there is nothing to do since we can't return any data. Then we get the index value and the name. If name is not provided or the index value is non-zero, we set the name to a null value - otherwise we get the name. This is done for the purpose of the file system lookup, which requires a null name for iterative calls after the first.

    // Do the lookup and return the data...
    P := pointer( I ) ;
    S := C.FS.Lookup( PChar( inttostr( C.Typ ) + '\' + S ), P ) ;
    if( length( S ) > PSRB( Res )^.Length ) then
    begin
        setlength( S, PSRB( Res )^.Length ) ;
    end ;
    move( PChar( S )[ 0 ], PChar( PSRB( Res )^.Buffer )[ 0 ], length( S ) ) ;
    PSRB( Res )^.Length := length( S ) ;
    I := integer( P ) ;
    PInt64( Index )^ := I ;
end ; // LBR_Find
The file system requires a pointer for its context, so we convert the integer index to a pointer, call the file system to do the lookup, and then truncate the resulting name if it exceeds the length of the buffer. Then we return the name and its length. Finally, we convert the pointer (which may have been modified) back to an integer and write it to the passed address so it is ready for the next iterative call.

You may note that we prefix the name with the library type (C.Typ) and a backslash. This is because, if you recall, a library file can have multiple types of modules in it and they are stored in a directory that is the number of the type. This guarantees that we will be searching within the directory that has the modules of the type we specified when we called LBR_INI_Control. So, if we open a help library, it will search for help modules and not object modules, for instsance.

function LBR_Find_Close( Context, Index : int64 ) : int64 ;

var C : LBR_Context ;
    P : pointer ;

begin
    // Set up...
    Result := 0 ;
    if( Context = 0 ) then
    begin
        Result := LBR_ILLCTL ;
        exit ;
    end ;
    Context := Pint64( Context )^ ;
    if( Context = 0 ) then
    begin
        Result := LBR_ILLCTL ;
        exit ;
    end ;
    C := LBR_Context( Context ) ;
    if( C.Sanity <> #3'LBR' ) then
    begin
        Result := LBR_ILLCTL ;
        exit ;
    end ;
    if( not C.Opened ) then
    begin
        Result := LBR_LIBNOTOPN ;
        exit ;
    end ;
    if( Index = 0 ) then
    begin
        Result := UOSErr_Missing_Value ;
        exit ;
    end ;

    // Clear the context...
    Index := PInt64( Index )^ ;
    P := pointer( Index ) ;
    C.FS.Clear_Context( P ) ;
end ; // LBR_Find_Close
This function does the normal validation, and also checks that an index address is passed. Then it converts the integer to a point and calls the file system to clear the context. This will deallocate the context data and that context will no longer be valid.

In the next article, we will finally have look at the next CUSP.