1 Introduction
2 Ground Rules

Building a File System
3 File Systems
4 File Content Data Structure
5 Allocation Cluster Manager
6 Exceptions and Emancipation
7 Base Classes, Testing, and More
8 File Meta Data
9 Native File Class
10 Our File System
11 Allocation Table
12 File System Support Code
13 Initializing the File System
14 Contiguous Files
15 Rebuilding the File System
16 Native File System Support Methods
17 Lookups, Wildcards, and Unicode, Oh My
18 Finishing the File System Class

The Init Program
19 Hardware Abstraction and UOS Architecture
20 Init Command Mode
21 Using Our File System
22 Hardware and Device Lists
23 Fun with Stores: Partitions
24 Fun with Stores: RAID
25 Fun with Stores: RAM Disks
26 Init wrap-up

The Executive
27 Overview of The Executive
28 Starting the Kernel
29 The Kernel
30 Making a Store Bootable
31 The MMC
32 The HMC
33 Loading the components
34 Using the File Processor
35 Symbols and the SSC
36 The File Processor and Device Management
37 The File Processor and File System Management
38 Finishing Executive Startup

Users and Security
39 Introduction to Users and Security
40 More Fun With Stores: File Heaps
41 File Heaps, part 2
42 SysUAF
43 TUser
44 SysUAF API

UCL
45 Shells and UCL

Glossary/Index


Download sources
Download binaries

Using Our File System

Before we get to the meat of this article, let's take a moment to discuss using a video terminal with Init. I was planning to address this at some point in the future, but to be honest, I had forgotten how annoying it was to deal with the non-video command line when deleting characters. Since I make a lot of mistakes while typing, this was a big deal. So, I added a command to toggle the console between video and hardcopy input modes. The command is "VIDEO". Here is the code in the main command loop:
if( Match( 'VIDEO', S, 1 ) ) then
begin
    Keyboard.Video := not Keyboard.Video ;
end else

Now we can type "VIDEO" and switch to video mode. But we can make this even better. Since the HAL knows about the hardware environment, it will know if the console is a display/keyboard item. Well, unless it is a serial port that is connected to some remote terminal. On a PC, the console isn't remote and the HAL knows it is a video device. So, we have added a video function to the TTerminal class that will return true if the terminal is known to be video. In the constructor of TUOS_Terminal, we will check for this and set ourselves to video mode if it returns true. So on a PC (or our simulator), we don't need to type VIDEO to be in video input mode. Note, we can still use the command to test non-video input.

Now let's get to the disk command processing.

function Do_Disk( S : string ) : boolean ;

var Dummy : integer ;
    S1 : string ;
    Prompt : boolean ;

begin
    Result := False ;
    Prompt := S = '' ;
    while( True ) do
    begin
        if( Prompt ) then
        begin
            Output( 'Disk> ' ) ;
            S := Input( '' ) ;
        end ;
        if( S = #3 ) then
        begin
            Result := True ;
            exit ;
        end ;
        if( ( S = ESC ) or ( S = #26 ) ) then
        begin
            exit ;
        end ;
        S := Edit( S, 4 or 8 or 32 or 128 ) ;
        if( S = '' ) then
        begin
            continue ;
        end ;
        Dummy := pos( ' ', S + ' ' ) ;
        S1 := copy( S, Dummy + 1, length( S ) ) ;
        S := copy( S, 1, Dummy - 1 ) ;
        if( ( S = '?' ) or Match( 'HELP', S, 1 ) ) then
        begin
            Output_Line( 'CLONE <disk1> <disk2> - Copy contents of disk1 to disk2' ) ;
            Output_Line( 'DIRECTORY <disk> n - List directory n on disk' ) ;
            Output_Line( 'HELP - Show this text' ) ;
            Output_Line( 'LIST - List all disks' ) ;
            Output_Line( 'INITIALIZE <disk> {/QUERY} - Initialize a new file system on disk' ) ;
            Output_Line( 'PARTITION <disk> - Manage partitions' ) ;
            Output_Line( 'RAID <type> - Manage RAID sets' ) ;
            Output_Line( 'REBUILD <disk> - Rebuild file structure on disk' ) ;
        end else
        if( Match( 'CLONE', S, 1 ) ) then
        begin
            if( Disk_Clone( S1 ) ) then
            begin
                exit ;
            end ;
        end else
        if( Match( 'INITIALIZE', S, 1 ) ) then
        begin
            if( Disk_Init( S1 ) ) then
            begin
                exit ;
            end ;
        end else
        if( Match( 'DIRECTORY', S, 1 ) ) then
        begin
            if( Disk_Directory( S1 ) ) then
            begin
                exit ;
            end ;
        end else
        if( Match( 'LIST', S, 1 ) ) then
        begin
            if( Disk_List( S1 ) ) then
            begin
                exit ;
            end ;
        end else
        if( Match( 'RAID', S, 2 ) ) then
        begin
            if( Disk_RAID( S1 ) ) then
            begin
                exit ;
            end ;
        end else
        if( Match( 'REBUILD', S, 2 ) ) then
        begin
            if( Disk_Rebuild( S1 ) ) then
            begin
                exit ;
            end ;
        end else
        if( Match( 'PARTITION', S, 1 ) ) then
        begin
            if( Disk_Partition( S1 ) ) then
            begin
                exit ;
            end ;
        end else
        if( S <> '' ) then
        begin
            Output_Line( 'Invalid Disk command' ) ;
        end ;
        if( not Prompt ) then
        begin
            exit ;
        end ;
    end ; // while( True )
end ; // Do_Disk

This routine follows the structure of the main input loop. We will talk about the partition and RAID options later.

The initialize option will create an initialized native file system on a disk.

function Disk_Init( Command : string ) : boolean ;

var Aborted : boolean ;
    Buffer : PChar ;
    Clustersize, Folder_Clustersize, Patterns : int64 ;
    Dummy, Flags : integer ;
    FS : TUOS_Native_File_System ;
    MStore : TUOS_Managed_Store ;
    AT_Offset, AT_Size, P : TStore_Address64 ;
    Notifier : TInit_Notifier ;
    Password : string ;
    Prompt : boolean ;
    Query : boolean ;
    Store : TCOM_Store64 ;
    S : string ;
    FSHeader : FS_Header ;
    UEC : TUnified_Exception ;
    Volume_Label : string ;

begin // Disk_Init
    Prompt := Command = '' ;
    Result := False ;
    while( true ) do
    begin
        // Query user for disk...
        if( Command = '' ) then
        begin
            Output( 'Disk init (DISKA0) > ' ) ;
            S := Input( 'DISKA0' ) ;
        end else
        begin
            S := Command ;
        end ;
        if( S = #3 ) then
        begin
            Result := True ;
            exit ;
        end ;
        if( ( S = ESC ) or ( S = #26 ) ) then
        begin
            exit ;
        end ;
        S := Edit( S, 4 or 8 or 32 or 128 ) ;
        if( S = '' ) then
        begin
            continue ;
        end ;
        if( ( S = '?' ) or Match( 'HELP', S, 1 ) ) then
        begin
            Output_Line( '<disk>{/QUERY} - initialize disk' ) ;
            Output_Line( 'HELP - Show this text' ) ;
            Output_Line( '' ) ;
        end else
        begin
            Query := pos( '/Q', S ) > 0 ;
            Dummy := pos( '/', S + '/' ) ;
            S := copy( S, 1, Dummy - 1 ) ;

            Store := Parse_Device( S ) ;
            if( Store <> nil ) then
            begin
                // Confirm operation...
                Buffer := allocmem( Store.Min_Storage ) ;
                Store.Read_Data( Buffer[ 0 ], 0, 24, UEC ) ;
                move( Buffer[ 16 ], P, 8 ) ; // Get store header address
                if( ( not odd( P ) ) and ( P > Store.Min_Storage ) ) then
                begin
                    Store.Read_Data( Buffer[ 0 ], P, sizeof( FSHeader ), UEC ) ;
                    move( Buffer[ 0 ], FSHeader, sizeof( FSHeader ) ) ;
                    if(
                         ( FSHeader.ID = -1 )
                         and
                         ( FSHeader.ID1 = 135 )
                         and
                         ( FSHeader.Version <= 10 )
                      ) then
                    begin
                        S := 'public' ;
                        if( ( FSHeader.Flags and UOSFSF_Private ) <> 0 ) then
                        begin
                            S := 'private' ;
                        end ;
                        Output_Line( 'This device appears to have an existing ' + S +
                            ' file system, labelled "' + FSHeader.Volume_Label + '"' ) ;
                    end ;
                end ;
                Output( 'Any existing data on the device will be lost.  Continue? <NO> ' ) ;
                S := Input( 'NO' ) ;
                if( copy( S, 1, 1 ) <> 'Y' ) then
                begin
                    Store.Free ;
                    exit ;
                end ;
                if( Query ) then
                begin
                    if( Do_Query ) then
                    begin
                        exit ;
                    end ;
                end else
                begin
                    Patterns := 4 ;
                    Volume_Label := 'System' ;
                    Password := '' ;
                    Clustersize := Store.Min_Storage ;
                    Folder_Clustersize := Clustersize ;
                    Flags := 0 ;
                    AT_Offset := 0 ;
                end ;

                // Initialize disk...
                MStore := TUOS_Managed_Store.Create ;
                MStore.Store := Store ;
                FS := TUOS_Native_File_System.Create ;
                FS.Store := MStore ;
                Notifier := TInit_Notifier.Create ;
                try
                    FS.Init( PChar( Volume_Label ), PChar( Password ),
                        Patterns, Clustersize, Folder_Clustersize, Flags, AT_Offset,
                        Notifier.Callback ) ;
                    Aborted := Notifier.Aborted ;
                finally
                    Notifier.Free ;
                end ;
                if( Patterns > 0 ) then
                begin
                    Output_Line( '' ) ;
                end ;
                Output_Line( '' ) ;
                S := '' ;
                if( FS.Last_Error <> nil ) then
                begin
                    S := 'File System initialization error: ' +
                        inttostr( FS.Last_Error.Get_Error ) + CRLF ;
                end ;
                FS.Free ;
                if( Aborted ) then
                begin
                    exit ;
                end ;
                if( S <> '' ) then
                begin
                    Output( S ) ;
                    continue ;
                end ;
                if( _Count = 0 ) then
                begin
                    Output_Line( 'No bad clusters found' ) ;
                end else
                if( _Count = 1 ) then
                begin
                    Output_Line( '1 bad cluster found' ) ;
                end else
                begin
                    Output_Line( inttostr( _Count ) + ' bad clusters found' ) ;
                end ;
            end ;
        end ;
        if( not Prompt ) then
        begin
            exit ;
        end ;
    end ; // while( True )
end ; // Disk_Init

The routine prompts for the disk to initialize, if it wasn't passed into the function. The Parse_Device function takes the disk specification and returns a matching source, if such device exists. Otherwise it displays an error and returns nil. Here is the Parse_Device function:
function Parse_Device( S : string ) : TCOM_Store64 ;

var Controller, Dummy, Partition, Index, _Unit : integer ;
    Info : TDevice_Info ;

begin
    Result := nil ;

    // Parse disk spec...
    if( ( copy( S, 1, 4 ) <> 'DISK' ) or ( length( S ) < 6 ) ) then
    begin
        Output_Line( 'Invalid disk' ) ;
        exit ;
    end ;
    Controller := ord( S[ 5 ] ) - 65 ;
    val( copy( S, 6, length( S ) ), _Unit, Dummy ) ;
    if( Dummy = 1 ) then
    begin
        Output_Line( 'Invalid disk' ) ;
        exit ;
    end ;
    S := copy( S, 7 + Dummy, length( S ) ) ;
    if( S = '' ) then
    begin
        Partition := 0 ;
    end else
    begin
        Partition := Partition_Index( S ) ;
    end ;
    if( ( Controller < 0 ) or ( _Unit < 0 ) or ( Partition < 0 ) or ( Controller > 25 ) ) then
    begin
        Output_Line( 'Invalid disk' ) ;
        exit ;
    end ;

    // Get store for device...
    Index := 0 ;
    Info := HAL.Device( Index ) ;
    while( Info.Device_Type <> DT_Unknown ) do
    begin
        if(
            ( Info.Device_Type = DT_Store )
            and
            ( Info.Controller = Controller )
            and
            ( Info.Device_Unit = _Unit )
          ) then
        begin
            Result := HAL.Store( Index ) ;
            exit ;
        end ;
        inc( Index ) ;
        Info := HAL.Device( Index ) ;
    end ; // while( Info.Device_Type <> DT_None )
    if( Result = nil ) then
    begin
        Output_Line( 'Device not found' ) ;
    end ;
end ;

First, the function validates that the specification is valid, and then loops through the devices in the HAL, looking for a match. If found, it gets the store instance from the HAL and returns it. The Device method in the HAL returns a record describing the device with the given index. Here it is:
type TDevice_Info = packed record
                        Device_Type : word ; // See DT_*
                        Controller : word ;
                        Device_Unit : word ;

                        Media_Present : boolean ;
                    end ;

The device type constants are as follows:

MnuemonicValueDescription
DT_Non_Existant0Invalid device index
DT_Unknown1Type not otherwise included here
DT_Serial2Serial (stream) device
DT_Store3Data store device

The function loops until a DT_Non_Existant device type was found, meaning that we hit the end of the devices known by the HAL. We only look at devices with type DT_Store.

When the device is specified by the user and looked up in the HAL, we can now initialize the file system on it. We default the various parameters, but if the user specifies /QUERY after the device specification, we will prompt the user for each item. The default is to perform 4 patterns. Writing these four patterns (as done by the file system Init method) meets the requirements of the DoD 5220.22-M standard for data sanitization. So, to sanitize a disk when we are done with it, we can just run three patterns on it. We want to default UOS to the most secure options in all respects, although we will allow the user to configure less secure systems. Having said that, we will not default to using a password on the store. The reason is that any disk access tools would reveal the data even if a password was set. The only way to truly secure the data would be to encrypt it as it is written out and decrypt it when read back. But that would slow down the disk access, which we don't want to do by default. We will provide a means of doing this, if the user wishes, but it will be done at the file processor component level. The volume label isn't essential, so we just default it to "System". We default the store and folder clustersizes to the disk sector size. We default the flags so that the store is considered "Public", and set the AT offset to 0 so that the Init method will put the allocation table in the center of the store. These defaults should work adequately for most purposes. But, again, the user can override any of them by using /QUERY, which calls the following local function.

function Do_Query : boolean ;

label AT_Offset_Prompt, Clustersize_Prompt, Folder_Clustersize_Prompt,
    Label_Prompt, Patterns_Prompt, Public_Prompt, Password_Prompt ;

begin
    // Get initialization parameters...
    Result := True ;
Patterns_Prompt:
    Output( 'Number of patterns: <3> ' ) ;
    S := Input_Number( 3, Patterns ) ;
    if( S = #3 ) then
    begin
        exit ;
    end ;
    if( ( S = ESC ) or ( S = #26 ) ) then
    begin
        Store.Free ;
        exit ;
    end ;
    if( S = '' ) then
    begin
        Output_Line( 'Invalid pattern count' ) ;
        goto Patterns_Prompt ;
    end ;

Label_Prompt:
    Output( 'Volume Label: <System> ' ) ;
    S := Input( 'System', False ) ;
    if( S = #3 ) then
    begin
        exit ;
    end ;
    if( ( S = ESC ) or ( S = #26 ) ) then
    begin
        goto Patterns_Prompt ;
    end ;
    Volume_Label := Edit( S, 4 or 8 or 128 ) ; // Trim leading/trailing spaces and control characters

Password_Prompt:
    Output( 'Volume Password: <none> ' ) ;
    Keyboard.Echo := False ;
    S := Input( '', False ) ;
    Keyboard.Echo := True ;
    Output_Line( '' ) ;
    if( S = #3 ) then
    begin
        exit ;
    end ;
    if( ( S = ESC ) or ( S = #26 ) ) then
    begin
        goto Patterns_Prompt ;
    end ;
    Password := Edit( S, 4 or 8 or 128 ) ; // Trim leading/trailing spaces and control characters

    if( Password <> '' ) then
    begin
        Output( 'Verify Password: ' ) ;
        Keyboard.Echo := False ;
        S := Input( '', False ) ;
        Keyboard.Echo := True ;
        Output_Line( '' ) ;
        if( S = #3 ) then
        begin
            exit ;
        end ;
        if( ( S = ESC ) or ( S = #26 ) ) then
        begin
            goto Password_Prompt ;
        end ;
        S := Edit( S, 4 or 8 or 128 ) ; // Trim leading/trailing spaces and control characters
        if( S <> Password ) then
        begin
            Output_Line( 'Passwords do not match' ) ;
            goto Password_Prompt ;
        end ;
    end ;

Clustersize_Prompt:
    Output( 'Disk Cluster Size: <' + inttostr( Store.Min_Storage ) + '> ' ) ;
    S := Input_Number( Store.Min_Storage, Clustersize ) ;
    if( S = #3 ) then
    begin
        exit ;
    end ;
    if( ( S = ESC ) or ( S = #26 ) ) then
    begin
        goto Label_Prompt ;
    end ;
    if( S = '' ) then
    begin
        Output_Line( 'Invalid cluster size' ) ;
        goto Clustersize_Prompt ;
    end ;
    if( Clustersize < Store.Min_Storage ) then
    begin
        Output_Line( 'Invalid cluster size' ) ;
        goto Clustersize_Prompt ;
    end ;
    if( ( Clustersize mod Store.Min_Storage ) <> 0 ) then
    begin
        Output_Line( 'Invalid cluster size' ) ;
        goto Clustersize_Prompt ;
    end ;

Folder_Clustersize_Prompt:
    Output( 'Folder Cluster Size: <' + inttostr( Clustersize ) + '> ' ) ;
    S := Input_Number( Clustersize, Folder_Clustersize ) ;
    if( S = #3 ) then
    begin
        exit ;
    end ;
    if( ( S = ESC ) or ( S = #26 ) ) then
    begin
        goto Clustersize_Prompt ;
    end ;
    if( S = '' ) then
    begin
        Output_Line( 'Invalid cluster size' ) ;
        goto Folder_Clustersize_Prompt ;
    end ;
    if( Folder_Clustersize < Clustersize ) then
    begin
        Output_Line( 'Invalid cluster size' ) ;
        goto Folder_Clustersize_Prompt ;
    end ;
    if( ( Folder_Clustersize mod Clustersize ) <> 0 ) then
    begin
        Output_Line( 'Invalid cluster size' ) ;
        goto Folder_Clustersize_Prompt ;
    end ;

Public_Prompt:
    Output( 'Public or Private: <PUBLIC> ' ) ;
    S := Input( 'PUBLIC' ) ;
    if( S = #3 ) then
    begin
        exit ;
    end ;
    if( ( S = ESC ) or ( S = #26 ) ) then
    begin
        goto Folder_Clustersize_Prompt ;
    end ;
    Flags := 0 ;
    if( copy( S, 1, 2 ) <> 'PU' ) then
    begin
        Flags := UOSFSF_Private ;
    end ;

AT_Offset_Prompt:
    AT_Offset := Store.Max_Storage div Clustersize ; // Number of clusters on store
    AT_Size := AT_Offset div 8 ; // Number of bytes for AT
    AT_Size := AT_Size div Clustersize ; // Number of clusters for AT
    AT_Offset := ( AT_Offset - AT_Size ) div 2 ; // Center of store
    AT_Offset := AT_Offset * Clustersize ;
    Output( 'SAT position: <' + inttostr( AT_Offset ) + '> ' ) ;
    S := Input_Number( AT_Offset, AT_Offset ) ;
    if( S = #3 ) then
    begin
        exit ;
    end ;
    if( ( S = ESC ) or ( S = #26 ) ) then
    begin
        goto Folder_Clustersize_Prompt ;
    end ;
    if( S = '' ) then
    begin
        Output_Line( 'Invalid position' ) ;
        goto Folder_Clustersize_Prompt ;
    end ;
    if(
        ( AT_Offset < Clustersize )
        or
        ( AT_Offset > Store.Max_Storage - AT_Size - Clustersize )
      ) then
    begin
        Output_Line( 'Invalid position' ) ;
        goto Folder_Clustersize_Prompt ;
    end ;
    if( ( AT_Offset mod Clustersize ) <> 0 ) then
    begin
        Output_Line( 'Invalid position' ) ;
        goto Folder_Clustersize_Prompt ;
    end ;
    Result := True ;
end ;

In the case of the password, we turn the echo off in order to hide the password from any prying eyes, and turn it back on afterwards.

Back to the Disk_Init method, we check for a valid file system on the store. If there is one, we tell the user that the store is already initialized. Either way, we remind the user that the operation will destroy any existing data on the disk and ask for confirmation before we perform the operation. We create a initialization notifier that is passed to the Init method so that we can display our progress during the initialization. Here is that class:

type TInit_Notifier = class
                          public
                              Aborted : boolean ;

                              procedure Callback( Sender : pointer ;
                                  Progress, Count : cardinal ;
                                  var Abort : boolean ) ;
                      end ;

procedure TInit_Notifier.Callback( Sender : pointer ; Progress, Count : cardinal ;
    var Abort : boolean ) ;

begin
    // Save values...
    _Progress := Progress ;
    _Count := Count ;

    // Notify user...
    case Progress of
        10,20,30,40,50,60,70,80,90 : Output( inttostr( Progress ) + '%' ) ;
        5,15,25,35,45,55,65,75,85,95 : Output( '.' ) ;
    end ;

    // Check for abort...
    if( Keyboard.Peek = #3 ) then
    begin
        Abort := True ;
        Aborted := True ;
    end ;
end ; // TInit_Notifier.Callback

The other Init method that uses the File System is the Rebuild command, handled by this function:

function Disk_Rebuild( S : string ) : boolean ;

var FS : TUOS_Native_File_System ;
    Prompt : boolean ;
    MStore : TUOS_Managed_Store ;
    Store : TCOM_Store64 ;

begin
    Result := False ;
    Prompt := S = '' ;
    while( True ) do
    begin
        if( Prompt ) then
        begin
            Output( 'Disk rebuild (DISKA0) > ' ) ;
            S := Input( 'DISKA0' ) ;
        end ;
        if( S = #3 ) then
        begin
            Result := True ;
            exit ;
        end ;
        if( ( S = ESC ) or ( S = #26 ) ) then
        begin
            exit ;
        end ;
        S := Edit( S, 4 or 8 or 32 or 128 ) ;

        if( ( S = '?' ) or Match( 'HELP', S, 1 ) ) then
        begin
            Output_Line( '<disk> - Rebuild file system on disk' ) ;
            Output_Line( 'HELP - Show this text' ) ;
            Output_Line( '' ) ;
        end else
        begin
            Store := Parse_Device( S ) ;
            if( Store <> nil ) then
            begin
                if( not Valid_File_System( Store ) ) then
                begin
                    Output_Line( 'Not a valid file system' ) ;
                end else
                if( Store.Read_Only ) then
                begin
                    Output_Line( 'Disk is read-only' ) ;
                end else
                begin
                    MStore := TUOS_Managed_Store.Create ;
                    MStore.Store := Store ;
                    FS := TUOS_Native_File_System.Create ;
                    FS.Store := MStore ;
                    try
                        if( FS.Rebuild ) then
                        begin
                            Output_Line( 'File system structure corruption found - run recovery process' ) ;
                        end else
                        begin
                            Output_Line( 'Structure rebuilt' ) ;
                        end ;
                    finally
                        FS.Free ;
                        MStore.Free ;
                    end ;
                end ;
            end ; // if( Store <> nil )
        end ;
        if( not Prompt ) then
        begin
            exit ;
        end ;
    end ; // while( True )
end ; // Disk_Rebuild

So, now we finally have made use of our file system in real code that accomplishes something useful for UOS. In fact, this is the first step to setting up an installation of UOS. In the next article, we will continue looking at the functions in Init.