1 Introduction
2 Ground Rules

Building a File System
3 File Systems
4 File Content Data Structure
5 Allocation Cluster Manager
6 Exceptions and Emancipation
7 Base Classes, Testing, and More
8 File Meta Data
9 Native File Class
10 Our File System
11 Allocation Table
12 File System Support Code
13 Initializing the File System
14 Contiguous Files
15 Rebuilding the File System
16 Native File System Support Methods
17 Lookups, Wildcards, and Unicode, Oh My
18 Finishing the File System Class

The Init Program
19 Hardware Abstraction and UOS Architecture
20 Init Command Mode
21 Using Our File System
22 Hardware and Device Lists
23 Fun with Stores: Partitions
24 Fun with Stores: RAID
25 Fun with Stores: RAM Disks
26 Init wrap-up

The Executive
27 Overview of The Executive
28 Starting the Kernel
29 The Kernel
30 Making a Store Bootable
31 The MMC
32 The HMC
33 Loading the components
34 Using the File Processor
35 Symbols and the SSC
36 The File Processor and Device Management
37 The File Processor and File System Management
38 Finishing Executive Startup

Users and Security
39 Introduction to Users and Security
40 More Fun With Stores: File Heaps
41 File Heaps, part 2
42 SysUAF
43 TUser
44 SysUAF API

UCL
45 Shells and UCL

Glossary/Index


Download sources
Download binaries

Exceptions and Emancipation

In this article, we will address three final issues that impact the implementation of our class.

Exceptions
UOS also needs to keep track of, and properly handle, exceptions. An exception in UOS simply indicates some sort of unexpected condition or error - it doesn't necessarily have anything to do with CPU exceptions or exception handling in languages such as C++. It is important for UOS to detect errors so that it doesn't attempt to continue along a path of operation after an error occurs. It is important to log and report these errors so that the user can take action - especially when those errors may indicate a failing hardward component. It may be tempting to use the exception handling provided by C++ or Delphi, but there are problems with that approach. Since UOS is composed of components that communicate with each other, and since each component could be compiled by any compiler, we cannot assume that exception handling is done in a compatible manner across all components. Especially since some of those implementations make use of specific capabilities of given hardware. Since one of the premises of UOS is to be able to run on a wide variety of hardware, we cannot assume specific hardware capabilities. Furthermore, the common exception handling of today is single-level, whereas UOS uses multi-level exceptions. This is not to say that there is anything wrong with catching exceptions within your code - simply don't raise an exceptions to be passed on to the calling code.

UOS exceptions are handled as objects. A given exception consists of text, a severity code, a facility code, and an error code. They can also be nested, or chained together. Let's say that our class calls a method in the store, which gets an error. It passes this to our class as an exception object. We then create our own exception object that indicates where the failure happened in our class, and attach the exception returned from the store to our exception, so that now it is two levels. Our caller may then take our exception, create their own exception and attach ours to it and then pass that three-level exception up the call stack. This can go on for many levels. There are two reasons for doing this. First, the caller of a component doesn't know the implementation of the component, so simply returning an exception from some other code called by the component means nothing to the calling code other than the operation failed. It doesn't know how it failed, only that it did. But, if the exception returned is specific to the component, then the calling code can make intelligent decisions about how to react to it. Second, the user can derive as much information about the problem from the exception chain as they desire. Perhaps they don't want to know more than that the file expansion failed. But perhaps they want to know it failed because the disk drive was full. The text for the exceptions is probably sufficient in most cases, but a facility and error code help provide an unmistakably clear way of identifying what the error is, and where it occured. Here is the definition of the exception class:


type tUnified_Exception = class
                            public
                                function Get_Facility : longint ; virtual ;
                                    stdcall ; abstract ;
                                function Severity : longint ; virtual ;
                                    stdcall ; abstract ;
                                function Error_Text( var Size, Typ : longint ) : PChar ;
                                    virtual ; stdcall ; abstract ;
                                function Get_Error : longint ; virtual ;
                                    stdcall ; abstract ;
                                function Get_Previous : tUnified_Exception ;
                                    virtual ; stdcall ; abstract ;
                                procedure Terminate ; virtual ;
                                    stdcall ; abstract ;
                        end ;

So, we will add a local instance variable that is tUnified_Exception. And then a Get_Error method is added that will return the last exception that occurred. If no exception has happened, the result of this method will be nil (NULL in C++). We will set the instance variable to nil when we start an API operation, and only set it when something goes wrong. We will create a descendant of tUnified_Exception that will be used in our class. Here is the code:

type tAllocated_Cluster_Manager_Unified_Exception = class( tUnified_Exception )
                                                        private
                                                            _Code : longint ;
                                                            _Text : string ;
                                                            _Previous : tUnified_Exception ;

                                                        public
                                                            function Get_Facility : longint ; override ;
                                                            function Get_Facility_Version : longint ;
                                                                override ;
                                                            function Error_Text( var Size, Typ : longint ) : PChar ;
                                                                override ;
                                                            function Get_Error : longint ; override ;
                                                            function Get_Previous : tUnified_Exception ;
                                                                override ;
                                                            procedure Terminate ; override ;
                                                    end ;


function tAllocated_Cluster_Manager_Unified_Exception.Get_Facility : longint ;

begin
    Result := 133 ;
end ;


function tAllocated_Cluster_Manager_Unified_Exception.Severity : longint ;

begin
    Result := UE_Error ;
end ;


function tAllocated_Cluster_Manager_Unified_Exception.Error_Text( var Size, Typ : longint ) : PChar ;

begin
    _Text := ERT( _Code ) ; 
    Size := length( _Text ) ;
    Typ := 0 ;
    Result := PChar( _Text ) ;
end ;


function tAllocated_Cluster_Manager_Unified_Exception.Get_Error : longint ;

begin
    Result := _Code ;
end ;


function tAllocated_Cluster_Manager_Unified_Exception.Get_Previous : tUnified_Exception ;

begin
    Result := _Previous ;
end ;


procedure tAllocated_Cluster_Manager_Unified_Exception.Terminate ;

begin
    Free ;
end ;



function Create_Exception( C : longint ; E : TUnified_Exception ) : tAllocated_Cluster_Manager_Unified_Exception ;

begin
    Result := tAllocated_Cluster_Manager_Unified_Exception.Create ;
    Result._Code := C ;
    Result._Previous := E ;
end ;

The Create_Exception function will construct an instance of our exception class and return it to us. We pass an error code and the chained exception (or nil, if none). Note that we don't assign the text until asked for it, to save us from extra overhead if the caller doesn't need it. The text is obtained from the ERT function, which is defined as:

const ACM_Error_Cannot_Allocate_Cluster = 1 ;
      ACM_Error_Corrupted_File_System_Structure = 2 ;
      ACM_Error_Data_Expansion_Failed = 3 ;
      ACM_Error_Data_Truncation_Failed = 4 ;

function ERT( C : longint ) : string ;

begin
    case C of
        ACM_Error_Cannot_Allocate_Cluster : Result := 'Cannot allocate cluster' ;
        ACM_Error_Corrupted_File_System_Structure : Result := 'Corrupt file system structure' ;
        ACM_Error_Data_Expansion_Failed : Result := 'Data expansion failed' ;
        ACM_Error_Data_Truncation_Failed : Result := 'Data truncation failed' ;
        else Result := 'Unknown error' ;
    end ;
end ;

We add instance data to hold our current exception object:

                                                 _Last_Error : TUnified_Exception ;

We also add an internal method to set it, and a public method for the caller to obtain it.
function TCOM_Allocation_Cluster_Manager64.Get_Last_Error : TUnified_Exception ;

begin
    Result := _Last_Error ;
end ;

Why would we need a routine to set the last error? Can't we just assign it directly. Yes, we can. But it turns out that there is a little bit more involved...

Emancipation
When the calling code asks for our last error, we return our exception instance (or nil) to them. But the next call to our class will either clear _Last_Error (setting it to nil), or assign a different exception if something goes wrong. What happens to the instance that we handed to the caller? In theory, we should leave it alone since the caller may be holding on to the instance for a while. For instance, in the case of an exception, the calling code may try some alternate means of accomplishing what it wants and, if that fails, then it may want to pass that instance on to its caller. Or it may simply decide that it is no longer needed and can be freed. On the other hand, if the calling code doesn't ask for our exception instance and we want to replace it with another one (or nil), then we need to free the old instance so we don't fill our heap with a bunch of obsolete exception instances. Since exception objects can be passed around between various components, it can become impossible to maintain the instance such that it persists as long as it is needed but goes away when no one wants it any more. We risk it being destructed in one place but then referenced in another, which will likely cause our code to abend (end abnormally). Or, we risk it remaining in memory until the next system reboot. To solve this problem, we emancipate our objects. This is done by telling the object how many references there are to the instance, and when those references go away. When the number of references reaches 0, the object can self-destruct. This is done through a reference count and the methods Attach and Detach. Thus, each time some code asks for an instance of our exception object, assuming it isn't nil, that code needs to attach to the object, and then call Detach when it no longer needs it. Here's how these methods look:

procedure tAllocated_Cluster_Manager_Unified_Exception.Attach ;

begin
    inc( _Reference_Count ) ;
end ;


procedure tAllocated_Cluster_Manager_Unified_Exception.Detach ;

begin
    dec( _Reference_Count ) ;
    if( _Reference_Count <=0 ) then
    begin
        Terminate ;
    end ;
end ;

And the new instance data:
           _Reference_Count : longint ;

So, this is the reason we have a Set_Exception method - because we need to handle attaching and detaching from our exception object. The routine looks like this:

procedure TCOM_Allocation_Cluster_Manager64.Set_Exception( E : TUnified_Exception ) ;

begin
    if( E <> nil ) then
    begin
        E.Attach ;
    end ;
    if( _Last_Error <> nil ) then
    begin
        _Last_Error.Detach ;
    end ;
    _Last_Error := E ;
end ;

Note that we attach to the passed exception object before we detach from the current one. This is just an added measure of safety in case the same object is assigned to us twice. It shouldn't happen, but better safe than sorry. Obviously, we don't call either method for a nil instance. The passed instance will be nil if we are clearing the last exception, and the current exception (_Last_Error) will be nil if we have no current exception. What this code does is it makes sure that we register our reference to the instance, via Attach, and release our reference to the current instance. If some other code has a reference to the instance, our Detach will not cause the instance to go away and the other code can still use it. This simple approach removes all the possible confusion about whether or not the instance is needed. In fact, we will use this for most of our classes. The only thing we have to keep in mind is that if we request an emancipated object from somewhere, that we attach to it immediately. Likewise, when we are done with it, we need to be sure to detach from it. Our allocation cluster manager class now needs to handle creating the exception instance when something goes wrong - and you can be sure that Murphy is correct: whatever can go wrong, will go wrong. Here is an example of the updated Pull method:
procedure TCOM_Allocation_Cluster_Manager64.Pull( AC : longint ) ;

var Original, P : TStore_Address64 ;

begin
    Buffer ; // Make sure buffer is allocated
    if( _Last_Error <> nil ) then
    begin
        exit ;
    end ;
    if( _Current_AC = AC ) then
    begin
        exit ; // Already have it
    end ;
    Original := AC ;

    if( ( _Current_AC >= 0 ) and ( AC > _Current_AC ) ) then // After our current AC
    begin
        AC := AC - _Current_AC - 1 ; // How much further to go
        P := Buffer^[ _Max_Index ] ;
    end else
    begin
        P := _Root ; // Start from the beginning
    end ;
    while( AC >= 0 ) do
    begin
        if( P = 0 ) then
        begin
            Set_Exception( Create_Exception( ACM_Error_Corrupted_File_System_Structure ), nil ) ;
            exit ;
        end ;
        dec( AC ) ;
        Read( P ) ;
        _Current_AC_Pointer := P ;
        P := Buffer^[ _Max_Index ] ;
        inc( Turns ) ;
    end ;
    _Current_AC := Original ;
end ;

The call to Buffer will try to allocate a buffer if it hasn't been allocated before. But this can fail if we run out of heap space, so we check to make sure that the Buffer method didn't create an exception. If it did, we exit immediately. As we are traversing the chain, if at any point we end up trying to read address 0 from the store, we know we have a problem. This indicates that we've run into the end of the allocation cluster chain before we reached the requested cluster index. This can have two possible causes: 1) a bug in the code, or 2) bad data on the store. Since we validated the data before calling Pull, this is almost certainly an issue of bad data on the store. Somehow the allocation chain for the file has been compromised. This is a definite case for stopping our processing of the file immediately rather than attempting to operate on bad data, which could cause further data corruption on the store. This exception is not chained, because there is no other exception instance associated with it. Of course, we must also check for the presence of an exception after each call to Pull. And whomever calls that routine needs to check for completion, and so on. This may seem like a lot of extra work, but it is the cost of writing reliable code. And for a class which is fundamental to the proper operation of UOS, it is doubly important for it to be rock-solid reliable.

Last Changes
We know that the file system will have information about the file size, so we can have the code that creates our class pass the size along with the root pointer, thus saving us from having to traverse the allocation cluster chain and prevent all the turns that we would otherwise do. So we add a new method:

procedure TCOM_Allocation_Cluster_Manager64.Set_Root_And_Size( R : TStore_Address64 ;
    S : TStore_Size64 ) ;

begin
    _Root := R ;
    _Size := S ;
    _Current_AC := -1 ;
end ;

Since we are told what the root and data size are, we can just set the values and clear the buffer cache. Of course, if the size we are given doesn't match the allocation clusters, all manner of possible problems could result - anywhere from corrupted data to abends.

There is another performance improvement we can give our class. In the Set_Size method, we know that, for expansion, _Buffer will always contain the last allocation cluster, so we can set _Current_AC to the proper value. This has the potential of preventing the need to traverse the entire chain from the root on the next request. We can do the same thing on truncation - but only if the truncation didn't require us to deallocate other allocation clusters. Once we move from the last allocation cluster, _Buffer will no longer match any of our allocation clusters and so we have to set _Current_AC to -1. How do we know if we are still on the last valid allocation cluster? We can tell by looking at the first pointer. If the first pointer is 0, we are no longer in a valid allocation cluster. Otherwise we are and we can set _Current_AC appropriately. Worst case scenarios would save us no turns. Best case scenarios would save us a lot of turns. Reality will probably average out to be a slight overall performance improvement. Since it is a couple simple lines of code, it is worth it. Here is how our method looks now:

procedure TCOM_Allocation_Cluster_Manager64.Set_Size( Value : TStore_Size64 ) ;

var I : longint ;
    Last, P : TStore_Address64 ;
    New_Size : TStore_Size64 ;
    List : TInteger64_List ;

begin
    Set_Exception( nil ) ;

    // Determine max cluster for new size...
    Value := ( Value + _Clustersize - 1 ) div _Clustersize ; // Clusters required for the specified size
    New_Size := Value * _Clustersize ; // Total new size, if we succeed
    if( New_Size = _Size ) then // No size change
    begin
        exit ;
    end ;

    // Update allocation clusters for the new size...
    Buffer ; // Make sure buffer is allocated
    if( _Last_Error <> nil ) then
    begin
        exit ;
    end ;
    Last := 0 ;
    if( _Root = 0 ) then // Nothing allocated yet
    begin
        _Root := Allocate_Allocation_Cluster ;
        _Current_AC := -1 ;
	if( _Last_Error <> nil ) then
	begin
	    exit ;
	end ;
    end ;
    if( New_Size > _Size ) then
    begin
        // Expanding...
        Pull( ( _Size - 1 ) div _Clustersize div _Max_Index ) ; // Get last allocation cluster
        if( _Last_Error <> nil ) then
        begin
            exit ;
        end ;
        Value := Value - _Current_AC * _Max_Index ;
        P := _Current_AC_Pointer ;
        while( Value > 0 ) do
        begin
            for I := 0 to _Max_Index - 1 do // For each pointer in this cluster
            begin
                if( Buffer^[ I ] = 0 ) then // Unallocated cluster
                begin
                    Buffer^[ I ] := _Store.Allocate( _Clustersize ) ;
                    if( Buffer^[ I ] = 0 ) then
                    begin
                        Set_Exception( Create_Exception( ACM_Error_Data_Expansion_Failed, _Store.Last_Error ) ) ;
                        exit ;
                    end ;
                end ;
                dec( Value ) ; // One less cluster to account for
                if( Value = 0 ) then
                begin
                    break ; // No need to process any more of the pointers in this allocation cluster
                end ;
            end ; // for I := 0 to Max_Index - 1
            if( ( Value > 0 ) and ( Buffer^[ _Max_Index ] = 0 ) ) then // Need to extend the allocation chain
            begin
                Buffer^[ _Max_Index ] := Allocate_Allocation_Cluster ;
                if( Buffer^[ _Max_Index ] = 0 ) then
                begin
                    Set_Exception( Create_Exception( ACM_Error_Data_Expansion_Failed, _Store.Last_Error ) ) ;
                    exit ;
                end ;
            end ;
            Write( P ) ;
            if( _Last_Error <> nil ) then
            begin
                exit ;
            end ;
            if( Value > 0 ) then
            begin
                P := Buffer^[ _Max_Index ] ;
                if( P <> 0 ) then
                begin
                    _Heap.Fill( integer( Buffer ), _Buffer_Size, 0 ) ;
                end ;
            end ;
        end ; // while( Value > 0 )
        _Current_AC := ( New_Size - 1 ) div _Clustersize div _Max_Index ;
        _Current_AC_Pointer := P ;
    end else
    begin
        // Truncating...
        List := TInteger64_List.Create ;
        try
            Pull( ( Value - 1 ) div _Max_Index ) ; // Get new last allocation cluster
            if( _Last_Error <> nil ) then
            begin
                exit ;
            end ;
            Value := Value - _Current_AC * _Max_Index ;
            P := _Current_AC_Pointer ;
            while( P <> 0 ) do // Until remaining allocation clusters are dealt with
            begin
                for I := 0 to _Max_Index - 1 do // For each pointer in this cluster
                begin
                    if( Buffer^[ I ] = 0 ) then // Unallocated cluster
                    begin
                        break ; // No need to go through the rest of the pointers once we hit a 0
                    end ;
                    if( Value > 0 ) then // Until we reach the
                    begin
                        dec( Value ) ;
                    end else
                    begin
                        List.Add( Buffer^[ I ] ) ; // Mark the cluster for release
                        Buffer^[ I ] := 0 ;
                    end ; // if( Value > 0 )
                end ; // for I := 0 to _Max_Index - 1
                Last := Buffer^[ _Max_Index ] ;
                if( Buffer^[ 0 ] = 0 ) then // This allocation cluster is now unused
                begin
                    Deallocate_Allocation_Cluster( P ) ; // Remove allocation cluster
                    if( _Last_Error <> nil ) then
                    begin
                        exit ;
                    end ;
                end else
                if( Value <= 0 ) then
                begin
                    Buffer^[ _Max_Index ] := 0 ; // No more allocation clusters are needed for this file
                    Write( P ) ; // Update on-store cluster immediately for safety
                    if( _Last_Error <> nil ) then
                    begin
                        exit ;
                    end ;
                end ;
                for I := 0 to List.Count - 1 do
                begin
                    if( _Store.Last_Error <> nil ) then
                    begin
                        Set_Exception( Create_Exception( ACM_Error_Data_Truncation_Failed, _Store.Last_Error ) ) ;
                        exit ;
                    end ;
                end ; // for I := 0 to List.Count - 1
                _Size := New_Size ;
		// Once we update even one of the allocation clusters, that means our new size is valid,
		// even if we haven't finished removing all other ACs
                List.Clear ;
                P := Last ;

                // Move to next allocation cluster in chain...
                if( P <> 0 ) then
                begin
                    inc( Turns ) ;
                    Read( P ) ;
                end ;
            end ; // while( P <> 0 )
        finally
            List.Free ;
        end ;
        if( Buffer^[ 0 ] = 0 ) then // This allocation cluster is now unused
        begin
            _Current_AC := -1 ; // Reset buffer cache state
        end else
        begin
            _Current_AC := ( New_Size - 1 ) div _Clustersize div _Max_Index ;
            _Current_AC_Pointer := Last ;
        end ;
    end ; // if( New_Size > _Size )
    _Size := New_Size ;
    if( New_Size = 0 ) then
    begin
        _Root := 0 ;
        _Current_AC := -1 ;
    end ;
end ; // TCOM_Allocation_Cluster_Manager64.Set_Size

That finishes up our class (finally!). In the next article, we will discuss the framework under which this class, and all others in UOS, operate.