The File Processor and Device Management

The File Processor is the component through which all file I/O happens. File management requires file system management, which requires device management. Since all three areas of management are intertwined, we include them all in the file processor instead of separate components. In this article we will address the methods related to device management. Here is the class definition:

type TUOS_FiP = class( TUOS_File_Processor )
                    private // Instance data...
                        _Device_Names : TUOS_Symbol_Table ;
                        _Devices : TDevice_List ;
                        _Virtual_Stores : longint ;

                        _Kernel : TUOS_Kernel ;
                        _SSC : TUOS_System_Services ;
                        _USC : TUOS_User_Security ;

                        Temp : string ;

                   private // Internal utility functions
                       function SSC : TUOS_System_Services ;
                       function USC : TUOS_User_Security ;

                       function Get_Device( const S : string ) : TDevice ;

                    public // API...
                        function Is_Class( Name : PChar ) : boolean ; override ;
                        procedure Set_Kernel( K : TUOS_Kernel ) ; override ;

                    public // Device Management...
                        function Device_Table : TUOS_Symbol_Table ; override ;
                        procedure Process_Store( Device : TDevice ; Index : integer ;
                            S : string ; Store : TCOM_Store64 ) ;
                        procedure Reload_Devices ; override ;
                        procedure Add_Store_Device( Store : TCOM_Store64 ;
                            RAMDisk : boolean ) ; override ;
                end ; // TUOS_FiP

Most of the instance data should be familiar to you after our previous articles about other executive components. The first new one, _Device_Names, is a symbol table that will hold the device names, following the UOS naming convention of <type><controller><unit> (for example DISKA0). The second, _Devices, is a list of device information. _Virtual_Stores is simply a count of the number of virtual stores. More on these later.

Here are the basic executive component methods:

// Internal utility functions...

function TUOS_FiP.SSC : TUOS_System_Services ;

begin
    if( _SSC = nil ) then
    begin
        if( _Kernel <> nil ) then
        begin
            _SSC := _Kernel.SSC ;
        end ;
    end ;
    Result := _SSC ;
end ;


function TUOS_FiP.USC : TUOS_User_Security ;

begin
    if( _USC = nil ) then
    begin
        if( _Kernel <> nil ) then
        begin
            _USC := _Kernel.USC ;
        end ;
    end ;
    Result := _USC ;
end ;


// API...

function TUOS_FiP.Is_Class( Name : PChar ) : boolean ;

var N : string ;

begin
    Result := False ;
    N := string( Name ) ;
    N := lowercase( N ) ;
    if( N = 'tuos_fip' ) then
    begin
        Result := True ;
    end ;
end ;


procedure TUOS_FiP.Set_Kernel( K : TUOS_Kernel ) ;

begin // TUOS_FiP.Set_Kernel
    _Kernel := K ;
    if( _Kernel <> nil ) then
    begin
        HAL := _Kernel.HAL ;
        __HMC := _Kernel.HMC ;
        if( __HMC <> nil ) then
        begin
{$WARNINGS OFF}
            SetMemoryManager( FIP_MM ) ;
{$WARNINGS ON}
        end ;
        Reload_Devices ;
    end ; // if( K <> nil )
end ; // TUOS_FiP.Set_Kernel


function TUOS_FiP.Device_Table : TUOS_Symbol_Table ;

begin
    if( _Device_Names = nil ) then
    begin
        _Device_Names := TSymbol_Table.Create ;
    end ;
    Result := _Device_Names ;
end ;

Again, most of this should look familiar from previous articles about other executive components. The last method shown is simply a deferred creation of the _Device_Names symbol table.

We need a way of looking up a device by name, which is what the Get_Device method does for us:

function TUOS_FiP.Get_Device( const S : string ) : TDevice ;

var Index : integer ;

begin
    Result := nil ;
    Index := _Devices.Indexof( S ) ;
    if( Index >= 0 ) then
    begin
        Result := _Devices.Get_Device( Index ) ;
    end ;
end ;

This function merely looks up the device name in the _Devices list and then returns the device instance associated with that class. If the device name is not in the list, the function returns nil.

Next we have the code to interface the FiP's memory manager with the executive's HMC.

var __HMC : TUOS_Heap_Manager ;

function HMC_NewGetMem( Size : Integer ) : Pointer ;

begin
    Result := __HMC.Getmem( Size ) ;
end ;


function HMC_NewFreeMem( P : Pointer ) : Integer ;

begin
    Result := 0 ;
    __HMC.Free( P ) ;
end ;


function HMC_NewReallocMem( P : Pointer ; Size : Integer ) : Pointer ;

begin
    Result := __HMC.Realloc( P, Size ) ;
end ;



{$WARNINGS OFF}
const FIP_MM : TMemoryManager = (
                                    GetMem : HMC_NewGetMem ;
                                    FreeMem : HMC_NewFreeMem ;
                                    ReallocMem : HMC_NewReallocMem ;
                                 ) ;
{$WARNINGS ON}


procedure TUOS_FiP.Set_Kernel( K : TUOS_Kernel ) ;

begin // TUOS_FiP.Set_Kernel
    _Kernel := K ;
    if( _Kernel <> nil ) then
    begin
        HAL := _Kernel.HAL ;
        __HMC := _Kernel.HMC ;
        if( __HMC <> nil ) then
        begin
{$WARNINGS OFF}
            SetMemoryManager( FIP_MM ) ;
{$WARNINGS ON}
        end ;
        Reload_Devices ;
    end ; // if( K <> nil )
end ; // TUOS_FiP.Set_Kernel

The Set_Kernel method does the familiar job of setting the _Kernal instance data, the HMC is obtained from the Kernel, and the local memory manager is set up. The only new item here is the call to Reload_Devices.

Here's the code for Reload_Devices:

procedure TUOS_FiP.Reload_Devices ;

var Device : TDevice ;
    Index : integer ; // HAL device index
    Info : TDevice_Info ;
    S, St : string ;
    Store : TCOM_Store64 ;

begin
    // (Re)Create device and device name table...
    if( _Device_Names <> nil ) then
    begin
        _Device_Names.Detach ;
        _Devices.Free ;
        MStores.Clear ;
        _Virtual_Stores := 0 ;
    end ;
    _Device_Names := TSymbol_Table.Create ;
    _Devices := TDevice_List.Create ;

    // Build device and device name tables...
    Index := 0 ;
    Info := HAL.Device( Index ) ;
    while( Info.Device_Type <> DT_Non_Existant ) do
    begin
        S := chr( 65 + Info.Controller ) + inttostr( Info.Device_Unit ) ;
        Device := TDevice.Create ;
        Device.Index := Index ;
        Device.Info := Info ;
        case Info.Device_Type of
            DT_Serial :
                begin
                    S := '_TERM' + S + ':' ;
                    _Device_Names.Set_Value( PChar( S ), '', 0 ) ;
                    _Devices.AddObject( S, Device ) ;
                end ; // DT_Serial
            DT_Store :
                begin
                    S := '_DISK' + S + ':' ;
                    _Devices.AddObject( S, Device ) ;
                    if( Info.Media_Present ) then
                    begin
                        Store := HAL.Store( Index ) ;
                        Process_Store( Device, Index, S, Store ) ;
                    end ; // if( Info.Media_Present )
                end ; // DT_Store
        end ; // case Info.Device_Type
        inc( Index ) ;
        Info := HAL.Device( Index ) ;
    end ; // while( Info.Device_Type <> DT_Non_Existant )

    // Determine RAID sets...
    for Index := 0 to MStores.Count - 1 do
    begin
        S := '_DISKZ' + inttostr( Index ) + ':' ;
        Device := TDevice.Create ;
        Device.Index := -1 ; // Not a HAL device
        Device.Info.Controller := 26 ;
        Device.Info.Device_Type := DT_Store ;
	Device.Info.Device_Unit := Index ;
        Store := MStores.Get( Index ) ;
        Process_Store( Device, -1, S, Store ) ;
    end ;

    // Special devices...
    Device := TDevice.Create ;
    Device.Index := -1 ; // Not a HAL device
    S := '_NULL:' ;
    _Device_Names.Set_Value( PChar( S ), '', 0 ) ;
    _Devices.AddObject( S, Device ) ;
end ; // TUOS_FiP.Relead_Devices

Normally, this function will only be called once, but it is written so that it can be called multiple times, if so desired. So the first thing we check is if _Device_Names is non-nil. If so, the routine has been called before, so we must clear the existing _Devices and _Device_Names via Free and Detach, respectively. We also have to clear the mirror stores via MStores.Clear since the RAID sets will be recalculated. Then we reset the number of virtual stores to 0.
The first order of business is to create the _Devices and _Device_Names class instances. Then we loop through the HAL's devices. This should look familiar - it is very similar to code in Init which does the same function. We skip non-existant devices, then construct the device name (in S). We also construct a TDevice instance for this device. We add the name to _Device_Names and the name/device to _Devices. Why do we add the device name to two different lists instead of only one? Because of two reasons: data decoupling and performance. The FiP relies on the _Devices list for device management. But other components (such as the SSC) needs a simple list of device names that it can look through. _Device_Names provides this list, which can be traversed quickly when someone wants to find a device name. Also, that way no other component needs to know any details about the _Devices list and so we avoid pathological coupling between the FiP and other components.
The device type determines what exact processing is to be done. In the case of a serial device, the processing is what we described above. In the case of a store, we also get a store instance (assuming that media is present) and save that in the current device instance. Then we process the store (more on that in a moment).
Next, we check against the RAID sets (if any) and handle them like any physical store returned to us by the HAL. Note that we don't handle RAM disks here. On executive startup, those are handled by the Kernel, and any after that are the consequence of a user request. In either case, our Add_Store_Device method is called to add the device to our device list.
Finally, we handle special devices. For now, the only special device we will deal with is the NULL device. This kind of device exists on most Operating Systems for, at very least, testing purposes. For example, on RSTS/E it was called NL:, and on MSDOS it was called NUL. Essentially, it is a device in name only. It has no associated physical or logical device associated with it. Output sent to it is ignored, and attempts to read from it result in errors (or EOF indicators). We will discuss the null device more in later articles.

Here is the code for the Process_Store method:

procedure Process_Store( Device : TDevice ; Index : integer ;
                            S : string ; Store : TCOM_Store64 ) ;

var Buffer : PChar ;
    E : TUnified_Exception ;
    GUID_Partition : TGUID_Partition ;
    N : string ;
    Partition : TPartition ;
    PIndex : integer ; // Partition index
    PInfo : TPartition_Info ;
    PStore : TCOM_Store64 ;
    RInfo : TRAID_Info ; // RAID information
    Size : longint ;
    St : string ;

begin
    Buffer := allocmem( Store.Min_Storage ) ;
    try
        Device.Store := Store ;
        RInfo := Get_RAID_Info( Store ) ;
        Device.RInfo := RInfo ;

        if( RInfo.RAID_Level = 1 ) then
        begin
            _Device_Names.Set_Value( PChar( S ), '', 0 ) ;
        end else
        begin
            PInfo := Partition_Info( Store ) ;
            if( PInfo.Partition_Count > 0 ) then
            begin
                if( PInfo.Partition_Type = PT_MBR ) then
                begin
                    Store.Read_Data( Buffer[ 0 ], 0, Store.Min_Storage, E ) ;
                    if( E <> nil ) then
                    begin
                        exit ;
                    end ;
                    setlength( S, length( S ) - 1 ) ;
                    St := S ;

                    // MBR...
                    for PIndex := 0 to 3 do
                    begin
                        move( Buffer[ 446 + PIndex * sizeof( Partition ) ], Partition, sizeof( Partition ) ) ;
                        if( Partition.Partition_Type <> 0 ) then // Not an unused entry
                        begin
                            if( Partition.Drive_Status <> 0 ) then // Active
                            begin
                                S := St + uppercase( Partition_Spec( PIndex ) ) + ':' ;
                                PStore := Get_Partition( Store, PIndex ) ;
                                RInfo := Get_RAID_Info( PStore ) ;
                                Device.RInfo := RInfo ;
                                Device := TDevice.Create ;
                                Device.Index := PIndex ;
                                Device.Parent_Index := Index ;
                                Device.Store := PStore ;
                                _Devices.AddObject( S, Device ) ;
                                if( RInfo.RAID_Level = 1 ) then // Partition is part of a RAID set
                                begin
                                    Device.RAID_Check := Check_Store( PStore ) ;
                                end else
                                begin
                                    _Device_Names.Set_Value( PChar( S ), '', 0 ) ; // Partition
                                end ;
                            end ; // if( Partition.Drive_Status <> 0 )
                        end ; // if( Partition.Partition_Type <> 0 )
                    end ; // for Index := 0 to 3
                end else
                begin
                    // GUID...
                    Size := PInfo.GPT_Header.Entry_Size * PInfo.GPT_Header.Partition_Count ;
                    Size := ( Size + Store.Min_Storage - 1 ) and
                        not ( Store.Min_Storage - 1 ) ;
                    Reallocmem( Buffer, Size ) ;
                    Store.Read_Data( Buffer[ 0 ],
                        PInfo.GPT_Header.Entries_LBA * Store.Min_Storage, Size, E ) ;
                    setlength( S, length( S ) - 1 ) ;
                    St := S ;

                    for PIndex := 0 to PInfo.GPT_Header.Partition_Count - 1 do
                    begin
                        move( Buffer[ Index * PInfo.GPT_Header.Entry_Size ], GUID_Partition,
                            sizeof( GUID_Partition ) ) ;//Info.GPT_Header.Entry_Size ) ;
                        if(
                            ( GUID_Partition.Partition_Type[ 0 ] <> 0 )
                            or
                            ( GUID_Partition.Partition_Type[ 1 ] <> 0 )
                          ) then // Used
                        begin
                            if( GUID_Partition.Attributes <> 0 ) then // Active
                            begin
                                setlength( N, sizeof( GUID_Partition.Name ) ) ;
                                move( GUID_Partition.Name, PChar( N )[ 0 ], length( N ) ) ;
                                N := Edit( N, 4 or 128 ) ;
                                setlength( N, length( N ) - 1 ) ;
                                Device := TDevice.Create ;
                                Device.Name := N ;
                                Device.Index := PIndex ;
                                Device.Parent_Index := Index ;
                                PStore := Get_Partition( Store, PIndex ) ;
                                Device.Store := PStore ;
                                if( PStore <> nil ) then
                                begin
                                    RInfo := Get_RAID_Info( PStore ) ;
                                    Device.RInfo := RInfo ;
                                    if( RInfo.RAID_Level = 1 ) then // Partition is part of a RAID set
                                    begin
                                        Device.RAID_Check := Check_Store( Store ) ;
                                    end else
                                    begin
                                        S := St + uppercase( Partition_Spec( PIndex ) ) + ':' ;
                                        _Device_Names.Set_Value( PChar( S ), '', 0 ) ; // Partition
                                        _Devices.AddObject( S, Device ) ;
                                    end ;
                                end ; // if( PStore <> nil )
                            end ; // if( GUID_Partition.Attributes <> 0 )
                        end ; // if
                    end ; // for Index := 0 to Info.GPT_Header.Partition_Count - 1
                end ; // if
            end else
            begin
                _Device_Names.Set_Value( PChar( S ), '', 0 ) ; // Normal store
            end ; // if( Info.Partition_Count > 0 )
        end ; // if( RInfo.RAID_Level = 1 )
    finally
        freemem( Buffer ) ;
    end ;
end ; // TUOS_FiP.Reload_Devices.Process_Store

This code is almost identical to the partition-handling code used in Init. We didn't cover it there because it was specific to the handling of the MBR and GPT partitioning standards and I didn't want to bog things down with detailed descriptions of those standards. You can go find the standards and read all about them if you wish. What we do in this routine is important to our implementation of the FiP, so I've included the code. But I will not describe it in gory detail. The first thing we do is set the Device instance's store to this store. Then we get the RAID information and store that in the Device instance as well. If the store is part of a RAID set, we remove the device from the _Device_Names list - UOS provides a separate device name that includes the whole set and we don't allow access to the individual members. In that case we are done processing this store.
If the store isn't a RAID member, we get the partition information. It is either an MBR or a GPT type of partition structure, and we process whichever one it is. Other than the differences inherent in the partition structure, we do the exact same thing in both cases: iterate through the partitions and create additional entries in the _Device_Names and _Devices lists. But we also check if the partition is part of a RAID set and handle that as well. For RAID members, we set the Device's RAID_Check instance data to the result of the call to Check_Store, which returns True if the store was successfully added to the RAID set.

Here is the code for Check_Store:

function Check_Store( Store : TCOM_Store64 ) : boolean ;

var RInfo : TRAID_Info ;
    MStore : TMirror_Store ;

begin
    Result := False ;
    if( Store = nil ) then
    begin
        exit ;
    end ;
    RInfo := Get_RAID_Info( Store ) ;
    if( RInfo.RAID_Level = 1 ) then // Found a mirror member
    begin
        MStore := MStores.Store_With_GUID( RInfo.RAID_GUID ) ;
        if( MStore = nil ) then
        begin
            MStore := TMirror_Store.Create ;
        end else
        if(
            (
              ( RInfo.Timestamp > MStore.Timestamp )
              and
              ( RInfo.Sequence <= MStore.Sequence )
            )
            or
            (
              ( RInfo.Timestamp < MStore.Timestamp )
              and
              ( RInfo.Sequence >= MStore.Sequence )
            )
          ) then // Some sort of mismatch
        begin
            Result := True ;
        end ;
        MStore.Add_Store( Store ) ;
    end ; // if( RInfo.RAID_Level = 1 )
end ; // Check_Store

This function simply does a consistency check between this store and the RAID set. If there is a mismatch (or the passed store is nil, or the RAID level isn't 1), the function returns False (meaning failure). Otherwise, we add the store to the RAID set.

In article 24 we discussed the TMirror_Manager class, an instance of which is published via the MStores variable. We've added a new method to it:

procedure TMirror_Manager.Clear ;

var Index : integer ;

begin
    for Index := Count - 1 downto 0 do
    begin
        Remove( Index ) ;
    end ;
end ;

The function simply iterates through the mirror sets and calls Remove on each one.

Next we need a way to add new virtual stores (such as RAMDisks and network disks). The following method supports this:

procedure TUOS_FiP.Add_Store_Device( Store : TCOM_Store64 ; RAMDisk : boolean ) ;

var Device : TDevice ;
    S : string ;

begin
    Set_Last_Error( nil ) ;
    if( Store = nil ) then
    begin
        exit ;
    end ;
    if( _Virtual_Stores = high( Device.Info.Device_Unit ) - 1 ) then
    begin
        Set_Last_Error( Create_Error( UOSErr_Exhausted_Resource ) ) ;
        exit ;
    end ;

    // Create device...
    Device := TDevice.Create ;
    Device.Index := -1 ; // Not a HAL device
    Device.RAM := RAMDisk ;
    Device.Info.Controller := 25 ;
    Device.Info.Device_Type := DT_Store ;
    Device.Info.Device_Unit := _Virtual_Stores ;
    S := '_DISKY' + inttostr( _Virtual_Stores ) + ':' ;
    inc( _Virtual_Stores ) ;
    _Devices.AddObject( S, Device ) ;
    Process_Store( Device, -1, S, Store ) ;
end ; // TUOS_FiP.Add_Store_Device

This method is called when a virtual device that is a store is being added. These devices are not recognized by the HAL because they are not physical devices in the normal sense. First we clear any errors. If nil is passed, we exit. Otherwise We check to see if there are any unit numbers remaining for virtual stores. There can be up to 65535 virtual stores in any run of UOS. If we are at the limit, we set an error and exit. Next we create a TDevice for the passed store. All virtual disks are considered to be controller 25 ("Y"). The unit will be a one-up number (_Virtual_Stores) that we increment on each call. Even if a virtual disk is disposed of later, its unit number is not recycled - the unit just keeps increasing. We add the device to _Devices and call Process_Store to handle any issues of partitioning or mirroring.

_Devices is an instance of the TDevice_List class, whose definition follows:

type TDevice_List = class( TStringList )
                        public // Constructors and destructors...
                            destructor Destroy ; override ;

                        public // API...
                            function Get_Device( Index : integer ) : TDevice ;
                    end ;

This class is a descendent of TStringList, which is a list of strings associated with object instances. The string for each item is the device name and the object is the TDevice instance associated with that device name.

Here are the methods:

// Constructors and destructors...

destructor TDevice_List.Destroy ;

var Loop : integer ;

begin
    for Loop := 0 to Count - 1 do
    begin
        TDevice( Objects[ Loop ] ).Free ;
        Objects[ Loop ] := nil ;
    end ;

    inherited Destroy ;
end ;


// API...

function TDevice_List.Get_Device( Index : integer ) : TDevice ;

begin
    Result := TDevice( Objects[ Index ] ) ;
end ;

The destructor loops through the devices and frees them. Get_Device simply provides a means of casting the generic pointer in the TStringList to a TDevice instance.

The definitions used by TDevice, and the class follow:

const Protection_Allow_User_Read = 1 ;
const Protection_Allow_User_Write = 2 ;
const Protection_Allow_User_Control = 4 ;
const Protection_Allow_User_All = 7 ;
const Protection_Allow_Group_Read = 8 ;
const Protection_Allow_Group_Write = 16 ;
const Protection_Allow_Group_Control = 32 ;
const Protection_Allow_Group_All = 56 ;
const Protection_Allow_World_Read = 64 ;
const Protection_Allow_World_Write = 128 ;
const Protection_Allow_World_Control = 256 ;
const Protection_Allow_World_All = 448 ;

type TOwnership_Type = ( OT_None, // Not owned
                         OT_User, // Owned by a user
                         OT_Process // Owned by a process
                       ) ;

type TOwnership = packed record
                      Ownership_Type : TOwnership_Type ;
                      Owner : cardinal ; // User or PID
                      Protections : cardinal ; // See Protection_*
                      //ACL
                  end ;

type TDevice = class
                   public // API...
                       // Identification...
                       Index : integer ; // HAL device index
                       Parent_Index : integer ;
                       Info : TDevice_Info ;
                       RInfo : TRAID_Info ; // RAID information
                       Store : TCOM_Store64 ; // Nil if not a store
                       Name : string ; // Named GPT partition

                       // For RAID...
                       RAID : integer ; // RAID level
                       RAID_Check : boolean ; // True if there is an inconsistency between RAID members

                       // For terminals...
                       Attached : cardinal ; // PID attached to terminal (0 if none)

                       // Other...
                       Owner : TOwnership ;

                       // For stores...
                       Mounted : boolean ; // True if mounted
                       FS : TUOS_File_System ; // File system for device (nil if non-structured store)
                       RAM : boolean ; // True if a RAM disk

                   public // Constructors and destructors...
                       constructor Create ;
               end ; // TDevice

// Constructors and destructors...

constructor TDevice.Create ;

begin
    inherited Create ;

    Index := -1 ;
    Parent_Index := -1 ;
    RAID := -1 ;
end ;

We will discuss protection codes and ownership in a future article. All instance data is zeroed when the class instance is created, so we set the values that need to be something other than 0 initially.

There are a couple of other device management methods that we will not cover until a later article because they fit into a larger discussion of users and security. In the next article, we will discuss the File System management portion of the FiP.