1 Introduction
2 Ground Rules

Building a File System
3 File Systems
4 File Content Data Structure
5 Allocation Cluster Manager
6 Exceptions and Emancipation
7 Base Classes, Testing, and More
8 File Meta Data
9 Native File Class
10 Our File System
11 Allocation Table
12 File System Support Code
13 Initializing the File System
14 Contiguous Files
15 Rebuilding the File System
16 Native File System Support Methods
17 Lookups, Wildcards, and Unicode, Oh My
18 Finishing the File System Class

The Init Program
19 Hardware Abstraction and UOS Architecture
20 Init Command Mode
21 Using Our File System
22 Hardware and Device Lists
23 Fun with Stores: Partitions
24 Fun with Stores: RAID
25 Fun with Stores: RAM Disks
26 Init wrap-up

The Executive
27 Overview of The Executive
28 Starting the Kernel
29 The Kernel
30 Making a Store Bootable
31 The MMC
32 The HMC
33 Loading the components
34 Using the File Processor
35 Symbols and the SSC
36 The File Processor and Device Management
37 The File Processor and File System Management
38 Finishing Executive Startup

Users and Security
39 Introduction to Users and Security
40 More Fun With Stores: File Heaps
41 File Heaps, part 2
42 SysUAF
43 TUser
44 SysUAF API

Terminal I/O
45 Shells and UCL
46 UOS API, the Application Side
47 UOS API, the Executive Side
48 I/O Devices
49 Streams
50 Terminal Output Filters
51 The TTerminal Class
52 Handles
53 Putting it All Together
54 Getting Terminal Input
55 QIO
56 Cooking Terminal Input
57 Putting it all together, part 2
58 Quotas and I/O

UCL
59 UCL Basics
60 Symbol Substitution
61 UCL Command execution
62 UCL Command execution, part 2
63 UCL Command Abbreviation
64 ASTs
65 UCL Expressions, Part 1

Glossary/Index


Download sources
Download binaries

Putting it All Together

SYS_PUT

Now that we have the entire stack for terminal output from the executive, let's look at how an application makes the call to send output.
The LIB_Put_Output function in Starlet is used to send a string to the process' sys$output device (at this point, the terminal associated with the process).

procedure LIB_Put_Output( const S : string ) ;

var RAB : TRAB ;

begin
    fillchar( RAB, sizeof( RAB ), 0 ) ;
    RAB.RAB_Size := sizeof( RAB ) ;
    RAB.RAB_W_ISI := RH_SysOutput ;
    RAB.RAB_L_RBF := integer( PChar( S ) ) ;
    RAB.RAB_W_RSZ := length( S ) ;
    RMS.SYS_Put( RAB ) ;
end ;
We set up a RAB (Record Access Block) structure with a pointer to the string, the string length, and set RAB_W_ISI to the sys$output relocation handle (RH_SysOutput). Note that we pass the RAB structure to the RMS unit. RMS (Record Management Services) is used to cook data for files on stores. For terminals, RMS just passes the data to the TTerminal instance which uses the output filter to cook the data. Here's the SYS_Put function:

procedure SYS_Put( var RAB : TRAB ) ;

var Status : byte ;
    SysRequest : TFile_Request ;

begin
    // Stream output...
    fillchar( SysRequest, sizeof( SysRequest ), 0 ) ;
    SysRequest.Request.Subsystem :=  UOS_Subsystem_FIP ;
    SysRequest.Request.Request := UOS_FIP_SYS_Out ;
    SysRequest.Request.Length := sizeof( SysRequest.FRB ) ;
    SysRequest.Request.Status := integer( @Status ) ;
    SysRequest.FRB.Handle := RAB.RAB_W_ISI ;
    SysRequest.FRB.Buffer := RAB.RAB_L_RBF ;
    SysRequest.FRB.Length := RAB.RAB_W_RSZ ;
    SysRequest.FRB.Flags := RAB.RAB_L_ROP and TFRBF_RAB_Mask ;
    Call_To_Ring0( integer( @SysRequest ) ) ;
end ;
This function constructs a system request and calls to the Kernel. All FiP-related calls use the TFile_Request structure. The FRB (File Request Block) portion of structure contains everything necessary for file/device operations. Fairly typical of such structures are the file handle, a buffer and length, and some flags. File operation flags apply to both RMS (cooked) and raw I/O operations; the TFRBF_RAB_Mask value is used to select the relevant executive (raw) flags (i.e. removes any RMS flags), since the executive knows nothing about RMS.

We now turn to the TUOS_FiP.API method:

procedure TUOS_FiP.API( Request : int64 ; SReq : TSystem_Request ) ;

var Base, Offset : int64 ;
    BBase, BOffset : int64 ; // I/O buffer
    PID : TPID ;
    SysReqFRB : PFile_Request ;

begin
    PID := Kernel.PID ;
    case SReq.Request of
        UOS_FIP_SYS_Out:
            begin
                if( SReq.Length < sizeof( TFRB ) ) then
                begin
                    Set_Last_Error( Create_Error( UOSErr_Invalid_System_Request ) ) ;
                    exit ;
                end ;
                Offset := MMC.Lock_Pages( PID, Request, sizeof( TFile_Request ) ) ;
                try
                    Base := MMC.Map_Pages( PID, 0, Request, sizeof( TFile_Request ), MAM_Read or MAM_Lock ) ;
                    if( Base = 0 ) then // Couldn't map memory
                    begin
                        USC.Set_Process_Exception( PID, MMC.Last_Error ) ;
                        if( MMC.Last_Error = nil ) then
                        begin
                            Set_Last_Error( Create_Error( UOSErr_Memory_Address_Error ) ) ;
                        end ;
                        exit ;
                    end ;
                    SysReqFRB := PFile_Request( Base + Offset ) ;
                    BOffset := MMC.Lock_Pages( PID, SysReqFRB.FRB.Buffer, SysReqFRB.FRB.Length ) ;
                    try
                        BBase := MMC.Map_Pages( PID, 0, SysReqFRB.FRB.Buffer, SysReqFRB.FRB.Length, MAM_Read or MAM_Lock ) ;
                        if( BBase = 0 ) then // Couldn't map buffer memory
                        begin
                            USC.Set_Process_Exception( PID, MMC.Last_Error ) ;
                            if( MMC.Last_Error = nil ) then
                            begin
                                Set_Last_Error( Create_Error( UOSErr_Memory_Address_Error ) ) ;
                            end ;
                            exit ;
                        end ;
                        Write_File( SysReqFRB.FRB.Handle, SysReqFRB.FRB.Stream,
                            PAnsiChar( pointer( BBase + BOffset ) ),
                            SysReqFRB.FRB.Length, SysReqFRB.FRB.Flags ) ;
                        MMC.UnMap_Pages( 0, SysReqFRB.FRB.Buffer, SysReqFRB.FRB.Length ) ;
                    finally
                        MMC.Unlock_Pages( PID, SysReqFRB.FRB.Buffer, SysReqFRB.FRB.Length ) ;
                    end ;
                    MMC.UnMap_Pages( 0, Request, sizeof( TFile_Request ) ) ;
                finally
                    MMC.Unlock_Pages( PID, Request, sizeof( TFile_Request ) ) ;
                end ;
            end ; // UOS_FIP_SYS_Out
        //else // Error
    end ; // case SReq.Request
end ; // TUOS_FiP.API
This method is very similar to the API method in the SSC. In this case, the UOS_FIP_SYS_Out method is processed and then Write_File is called.

Before we look at the Write_File method, we need to take a detour and cover the TFiP_File class, which is the base class for all "file" classes used by the FiP component. In the previous article, we discussed the association of a TFiP_File (or descendent thereof) instance with handles (TResource instances) and devices. Now we will see what those classes look like.

TFiP_File

type TFiP_File = class( TUOS_File )
                     public // constructor...
                         constructor Create( Kernel : TUOS_Kernel ) ;
                         destructor Destroy ; override ;

                     private // Instance data...
                         _Kernel : TUOS_Kernel ;
                         _File : TUOS_File ;
                         _Handles : TList ;

                     public // API...
                         function Is_Class( Name : PChar ) : boolean ;
                             override ;

                         function Create_Stream( Name : int64 ) : longint ;
                             override ;

                         procedure Delete_Stream( Name : int64 ;
                             Index : longint ) ; override ;

                         function Max_Stream : longint ; override ;

                         function Stream_Name( Index : longint ) : int64 ;
                             override ;

                         function Get_Contiguous : boolean ; override ;

                         procedure Set_Contiguous( Value : boolean ) ; override ;

                         // I/O...
                         function Read( Stream : longint ;
                             Position : TStore_Address64 ;
                             Length : TStore_Size64 ; var Buff ) : TStore_Size64 ;
                             override ;

                         function Write( Stream : longint ;
                             Position : TStore_Address64 ;
                             Length : TStore_Size64 ; var Buff ) : TStore_Size64 ;
                             override ;

                	 	     function Get_Stream_Size( Stream : longint ) : TStore_Size64 ;
                             override ;

                	 	     procedure Set_Stream_Size( Stream : longint ;
                             Value : TStore_Size64 ) ;
                             override ;

                         function Get_File_Size : int64 ;
                             override ;

                         procedure Set_File_Size( Value : int64 ) ;
                             override ;

                         function Read_Only : boolean ;
                             override ;

                         function Write_Only : boolean ;
                             override ;

                         function XSpaceAvail : int64 ;
                             override ;

                         function Record_Size : int64 ; virtual ;

                         procedure Add_Handle( Handle : TResource ) ; virtual ;

                         procedure Remove_Handle( Handle : TResource ) ;
                             virtual ;

                         function Is_Store : boolean ; override ;
                 end ; // TFiP_File

// constructor and destructor...

constructor TFiP_File.Create( Kernel : TUOS_Kernel ) ;

begin
    inherited Create ;

    _Kernel := Kernel ;
    _Handles := TList.Create ;
end ;


destructor TFiP_File.Destroy ;

begin
    _Handles.Free ; // It is assumed that all handles have been closed before freeing this instance

    inherited Destroy ;
end ;


// API...

function TFiP_File.Is_Class( Name : PChar ) : boolean ;

var _N : string ;

begin
    _N := Name ;
    _N := lowercase( _N ) ;
    Result := ( ( _N = 'tfip_file' ) or ( _N = 'tfile' ) or ( _N = 'tuos_file' ) or ( _N = 'tcommon_com_interface' ) ) ;
end ;


function TFiP_File.Create_Stream( Name : int64 ) : longint ;

begin
    Result := _File.Create_Stream( Name ) ;
end ;


procedure TFiP_File.Delete_Stream( Name : int64 ; Index : longint ) ;

begin
    _File.Delete_Stream( Name, Index ) ;
end ;


function TFiP_File.Max_Stream : longint ;

begin
    Result := _File.Max_Stream ;
end ;


function TFiP_File.Stream_Name( Index : longint ) : int64 ;

begin
    Result := _File.Stream_Name( Index ) ;
end ;


function TFiP_File.Get_Contiguous : boolean ;

begin
    Result := _File.Get_Contiguous ;
end ;


procedure TFiP_File.Set_Contiguous( Value : boolean ) ;

begin
    _File.Set_Contiguous( Value ) ;
end ;


// I/O...

function TFiP_File.Read( Stream : longint ; Position : TStore_Address64 ;
   Length : TStore_Size64 ; var Buff ) : TStore_Size64 ;

begin
    Result := _File.Read( Stream, Position, Length, Buff ) ;
end ;


function TFiP_File.Write( Stream : longint ; Position : TStore_Address64 ;
   Length : TStore_Size64 ; var Buff ) : TStore_Size64 ;

begin
    Result := _File.Write( Stream, Position, Length, Buff ) ;
end ;


function TFiP_File.Get_Stream_Size( Stream : longint ) : TStore_Size64 ;

begin
    Result := _File.Get_Stream_Size( Stream ) ;
end ;


procedure TFiP_File.Set_Stream_Size( Stream : longint ;
   Value : TStore_Size64 ) ;

begin
    _File.Set_Stream_Size( Stream, Value ) ;
end ;


function TFiP_File.Get_File_Size : int64 ;

begin
    Result := _File.Get_Stream_Size( 0 ) ;
end ;


procedure TFiP_File.Set_File_Size( Value : int64 ) ;

begin
    _File.Set_Stream_Size( 0, Value ) ;
end ;


function TFiP_File.Read_Only : boolean ;

begin
    Result := _File.Read_Only ;
end ;


function TFiP_File.Write_Only : boolean ;

begin
    Result := _File.Write_Only ;
end ;


function TFiP_File.XSpaceAvail : int64 ;

begin
    Result := _File.XSpaceAvail ;
end ;


function TFiP_File.Record_Size : int64 ;

begin
    Result := 1 ;
end ;


procedure TFiP_File.Add_Handle( Handle : TResource ) ;

begin
    _Handles.Add( Handle ) ;
end ;


procedure TFiP_File.Remove_Handle( Handle : TResource ) ;

begin
    _Handles.Remove( Handle ) ;
end ;


function TFiP_File.Is_Store : boolean ;

begin
    Result := True ; // Default
end ;
This class is a thin veneer around a TUOS_File. However, it is not a descendent of the TUOS_Class - it simply contains an instance of one so that it can be used as an interface to a native File System file (we'll look at that in a later article). As a consequence, the methods don't do very much and no further explanation is necessary. The only other aspect of this is the Handles list to keep track of handles attached to the file instance.

Now we'll look at the TFip_Terminal_File class, which is a descendent of TFiP_File.

type TFiP_Terminal_File = class( TFiP_File )
                              private // Instance data...
                                  Terminal : TTerminal ;

                              public // API...
                                  function Is_Class( Name : PChar ) : boolean ;
                                      override ;

                                  function Create_Stream( Name : int64 ) : longint ;
                                      override ;

                                  // Delete stream...
                                  procedure Delete_Stream( Name : int64 ; Index : longint ) ;
                                      override ;

                                  // Stearm information...
                                  function Max_Stream : longint ; override ;

                                  function Stream_Name( Index : longint ) : int64 ;
                                      override ;

                                  // I/O...
                                  function Read( Stream : longint ;
                                      Position : TStore_Address64 ;
                                      Length : TStore_Size64 ; var Buff ) : TStore_Size64 ;
                                      override ;

                                  function Write( Stream : longint ;
                                      Position : TStore_Address64 ;
                                      Length : TStore_Size64 ; var Buff ) : TStore_Size64 ;
                                      override ;

                                  function Get_Stream_Size( Stream : longint ) : TStore_Size64 ;
                                      override ;

                                  procedure Set_Stream_Size( Stream : longint ;
                                      Value : TStore_Size64 ) ;
                                      override ;

                                  function Get_Contiguous : boolean ; override ;

                                  procedure Set_Contiguous( Value : boolean ) ;
                                      override ;

                                  function Get_File_Size : int64 ; override ;

                                  procedure Set_File_Size( Value : int64 ) ;
                                      override ;

                                  function Read_Only : boolean ; override ;

                                  function Write_Only : boolean ; override ;

                                  function XSpaceAvail : int64 ; override ;

                                  function Record_Size : int64 ; override ;

                                  function Is_Store : boolean ; override ;
                          end ; // TFiP_Device_File
This class simply overrides the various methods and adds a TTerminal instance.

function TFiP_Terminal_File.Is_Class( Name : PChar ) : boolean ;

var _N : string ;

begin
    _N := Name ;
    _N := lowercase( _N ) ;
    Result := ( _N = 'tfip_terminal_file' ) ;
    if( not Result ) then
    begin
        Result := inherited Is_Class( Name ) ;
    end ;
end ;
This is the standard Is_Class method that we've seen before.

function TFiP_Terminal_File.Create_Stream( Name : int64 ) : longint ;

begin
    Result := 0 ;
end ;


procedure TFiP_Terminal_File.Delete_Stream( Name : int64 ; Index : longint ) ;

begin
end ;


function TFiP_Terminal_File.Max_Stream : longint ;

begin
    Result := 0 ;
end ;


function TFiP_Terminal_File.Stream_Name( Index : longint ) : int64 ;

begin
    Result := 0 ;
end ;


function TFiP_Terminal_File.Get_Stream_Size( Stream : longint ) : TStore_Size64 ;

begin
    Result := 0 ;
end ;


procedure TFiP_Terminal_File.Set_Stream_Size( Stream : longint ;
    Value : TStore_Size64 ) ;

begin
end ;
Because only store files have streams, we ignore all stream-related methods. We return 0 for any functions.

function TFiP_Terminal_File.Get_Contiguous : boolean ;

begin
    Result := False ;
end ;


procedure TFiP_Terminal_File.Set_Contiguous( Value : boolean ) ;

begin
end ;


function TFiP_Terminal_File.Get_File_Size : int64 ;

begin
    Result := 0 ;
end ;


procedure TFiP_Terminal_File.Set_File_Size( Value : int64 ) ;

begin
end ;


function TFiP_Terminal_File.Read_Only : boolean ;

begin
    Result := Terminal.Stream.Read_Only ;
end ;


function TFiP_Terminal_File.Write_Only : boolean ;

begin
    Result := Terminal.Stream.Write_Only ;
end ;


function TFiP_Terminal_File.XSpaceAvail : int64 ;

begin
    Result := 0 ;
end ;


function TFiP_Terminal_File.Record_Size : int64 ;

begin
    Result := 0 ; // Not a record-formatted device
end ;


function TFiP_Terminal_File.Is_Store : boolean ;

begin
    Result := False ;
end ;
Many other methods don't apply to terminals either. As a consequence, these methods also do nothing, or return 0. Is_Store returns false, since terminals are not stores.

function TFiP_Terminal_File.Read( Stream : longint ;
    Position : TStore_Address64 ;
    Length : TStore_Size64 ; var Buff ) : TStore_Size64 ;

var UEC : TUnified_Exception ;

begin
    Result := Terminal.Read_Data( Buff, Length, UEC ) ;
    if( UEC <> nil ) then
    begin
        Set_Last_Error( UEC ) ;
    end ;
end ;


function TFiP_Terminal_File.Write( Stream : longint ;
    Position : TStore_Address64 ;
    Length : TStore_Size64 ; var Buff ) : TStore_Size64 ;

var UEC : TUnified_Exception ;

begin
    Result := Terminal.Write_Data( Buff, Length, UEC ) ;
    if( UEC <> nil ) then
    begin
        Set_Last_Error( UEC ) ;
    end ;
end ;
The Read and Write methods simply wrap the TTerminal read and write methods.

Write_File

Now let's return to the Write_File method of the FiP.

function TUOS_FiP.Write_File( Handle : THandle ; Stream : longint ;
    Buff : PAnsiChar ; Length : int64 ; Flags : integer ) : TUnified_Exception ;

var Len : TStore_Size64 ;
    Resource : TResource ;

begin
    Set_Last_Error( nil ) ;
    Handle := USC.Translate_Handle( Kernel.PID, Handle ) ;
    if( not USC.Valid_Handle( Kernel.PID, Handle ) ) then
    begin
        Result := Set_Last_Error( Create_Error( UOSErr_Invalid_Handle ) ) ;
        exit ;
    end ;
    Resource := TResource( Handle ) ;
    Len := Resource._File.Write( Stream, Resource.Position, Length, Buff[ 0 ] ) ;
    Result := Resource._File.Last_Error ;
    if( Resource._File.Is_Store ) then
    begin
        Resource.Position := Resource.Position + Len ;
    end ;
end ;
When writing to a file, we first translate the handle (we will talk about the Translate_Handle method in a moment). Next we check to see if the handle is valid and exit with an error if it isn't. Then we convert from a handle to a TResource instance. Next, we call the file's write method and set the current exception (which will be nil if there is no exception). Finally, if the file is a store we adjust the resources's position. This Position is the current context for store files. Note that the Write call uses the current position and passes the Length parameter. Write returns the length actually written, and that is what we add the current position*. Thus, a TResource provides a context. If the file has multiple accessors, each one has its own context - as would be expected. And this explains why we need the extra level of indirection provided by TResource. We will delve into multiple concurrent file access in a future article.

* We do this because the write operation may fail to write all characters to the file and we only want to update the context to the actual position - not what the position would have been if all the bytes were written.

function TUSC.Translate_Handle( PID : TPID ; Handle : THandle ) : THandle ;

var Process : TProcess ;

begin
    Result := Handle ;
    if( PID = 0 ) then
    begin
        Process := Get_Process( Kernel.PID ) ;
    end else
    begin
        Process := Get_Process( PID ) ;
    end ;
    if( ( Process = nil ) or ( Process._Handles = nil ) ) then
    begin
        exit ;
    end ;
    case Handle of
        RH_SysInput : Result := Process._Sys_Input ;
        RH_SysOutput : Result := Process._Sys_Output ;
        RH_SysError : Result := Process._Sys_Error ;
        RH_SysCommand : Result := Process._Sys_Command ;
    end ;
end ;
The purpose of Translate_Handle is to convert a redirection handle constant to the actual handle associated with a process. First we get the appropriate TProcess instance, and exit with an exception if it isn't valid. We only translate the relocation handles, so we switch based on the passed handle and return the corresponding actual handle. If the passed handle isn't one of the relocation handles, we return the passed handle. Thus, this is a safe method to call in all circumstances. But it doesn't validate a handle. If the passed handle is invalid, it will return that invalid handle. The next method will validate a handle.

function TUSC.Valid_Handle( PID : TPID ; Handle : THandle ) : boolean ;

var Process : TProcess ;

begin
    Result := False ;
    if( PID = 0 ) then
    begin
        Process := Get_Process( Kernel.PID ) ;
    end else
    begin
        Process := Get_Process( PID ) ;
    end ;
    if( ( Process = nil ) or ( Process._Handles = nil ) ) then
    begin
        Set_Error( UOS_User_Security_Error_Invalid_PID ) ;
        exit ;
    end ;
    Result := Process._Handles.ItemIndex( Handle ) <> -1 ;
end ;
Determining if a handle is valid for a process is a simple matter of checking to see if the handle is in the _Handles list.

Putting it All Together

The last several articles have covered the code involved in outputting characters to a terminal. It may seem like a lot of effort for a simple result, but we have set up the framework for a whole lot more than simply outputting characters to a terminal. We will expand upon this more in the future. The following is a conceptual diagram showing the basic UOS executive I/O architecture for two processes accessing the same file/device (one of which is accessing it through two different handles).

Now that we've covered all the classes involved, let's bring it all together and follow the execution through from the application to the terminal device.

  1. We start with our call to LIB_Put_Output.
  2. LIB_Put_Output fills a RAB structure and calls to Sys_Put in RMS.
  3. If this were a cooked store file, RMS would do some extra processing. But since this is a terminal, Sys_Put constructs a TFile_Requesst for the UOS_FIP_SYS_Out system call, and then calls to the executive through ring 0.
  4. The Kernel's API receives the call and passes it on to the FiP's API handler.
  5. The FiP calls Write_File, which gets the TResource instance for the passed (and translated) handle, and calls the associated file's Write method.
  6. In this case (writing the prompt from UCL to the terminal), the resource's associated file is a TFiP_Terminal_File instance. Its write method calls the terminal's Write_Data method.
  7. The terminal is a FiP TTerminal instance, whose Write_Data method calls its output filter's Write method.
  8. The filter adds the character(s) to the ring buffer and, assuming the device is ready to accept output, the Write method calls Write_To_Driver.
  9. Write_To_Driver verifies that there is something in the buffer and that the device is ready to accept output, and then calls the terminal's associated stream's Write_Data method.
  10. Finally, the stream's Write_Data method calls the driver's Output method, which writes the data to the device.

In the next article, we will begin a discussion of how to get input from a terminal.