1 Introduction
2 Ground Rules

Building a File System
3 File Systems
4 File Content Data Structure
5 Allocation Cluster Manager
6 Exceptions and Emancipation
7 Base Classes, Testing, and More
8 File Meta Data
9 Native File Class
10 Our File System
11 Allocation Table
12 File System Support Code
13 Initializing the File System
14 Contiguous Files
15 Rebuilding the File System
16 Native File System Support Methods
17 Lookups, Wildcards, and Unicode, Oh My
18 Finishing the File System Class

The Init Program
19 Hardware Abstraction and UOS Architecture
20 Init Command Mode
21 Using Our File System
22 Hardware and Device Lists
23 Fun with Stores: Partitions
24 Fun with Stores: RAID
25 Fun with Stores: RAM Disks
26 Init wrap-up

The Executive
27 Overview of The Executive
28 Starting the Kernel
29 The Kernel
30 Making a Store Bootable
31 The MMC
32 The HMC
33 Loading the components
34 Using the File Processor
35 Symbols and the SSC
36 The File Processor and Device Management
37 The File Processor and File System Management
38 Finishing Executive Startup

Users and Security
39 Introduction to Users and Security
40 More Fun With Stores: File Heaps
41 File Heaps, part 2
42 SysUAF
43 TUser
44 SysUAF API

Terminal I/O
45 Shells and UCL
46 UOS API, the Application Side
47 UOS API, the Executive Side
48 I/O Devices
49 Streams
50 Terminal Output Filters
51 The TTerminal Class
52 Handles
53 Putting it All Together
54 Getting Terminal Input
55 QIO
56 Cooking Terminal Input
57 Putting it all together, part 2
58 Quotas and I/O

UCL
59 UCL Basics
60 Symbol Substitution
61 UCL Command execution
62 UCL Command execution, part 2
63 UCL Command Abbreviation
64 ASTs
65 UCL Expressions, Part 1
66 UCL Expressions, Part 2: Support code
67 UCL Expressions, part 3: Parsing
68 SYS_GETJPIW and SYS_TRNLNM
69 UCL Expressions, part 4: Evaluation
70 PROCESS_SCAN

Glossary/Index


Download sources
Download binaries

Quotas and I/O

One last subject to discuss before moving on from this series of articles on I/O is that of quotas. We discussed the concept of quotas back in article 42. Now it is time to implement some of those quotas. Of concern right now are BIOLM (buffered I/O limit) and BYTLM (byte limit) - all of the I/O we've addressed so far qualifies as buffered I/O. The proper place to handle these quotas is in the Read_File and Write_File methods of the File Processor. Here is the updated Read_File method:

function TUOS_FiP.Read_File( Handle : THandle ; Stream : longint ;
    Length : int64 ; Flags : integer ; var IOSB : TIOSB ) : string ;

var Resource : TResource ;

begin
    // Setup...
    Set_Last_Error( nil ) ;
    Result := '' ;

    Resource := TResource( Handle ) ;
    setlength( Result, Length ) ;
    if( USC.Check_Quota( Kernel.PID, Quota_BIOLM, 1 ) ) then
    begin
        Set_Last_Error( Create_Error( UOSErr_Quota_Exceeded ) ) ;
        exit ;
    end ;
    IOSB.r_io_64.r_bcnt_32.l_bcnt := Resource._File.Read( Stream, Resource.Position, Length, PAnsiChar( Result )[ 0 ], Flags ) ;
    setlength( Result, IOSB.r_io_64.r_bcnt_32.l_bcnt ) ;
    if( USC.Check_Quota( Kernel.PID, Quota_BYTLM, IOSB.r_io_64.r_bcnt_32.l_bcnt ) ) then
    begin
        Set_Last_Error( Create_Error( UOSErr_Quota_Exceeded ) ) ;
    end ;
end ;
We've added two if statements that call the USC's Check_Quota method. The first one passed Quota_BIOLM with a value of 1. We will discuss Check_Quota below, but essentially we are telling the USC that we're doing one more buffered I/O operation. If the method returns true, then we've exceeded that quota and we will set an exception and exit.
The second if statement calls Check_Quota to tell the USC that we have read in a certain number of bytes. Again, a true result from that method means we've exceeded a quota and we set the exception.

function TUOS_FiP.Write_File( Handle : THandle ; Stream : longint ;
    Buff : PAnsiChar ; Length : int64 ; Flags : integer ) : TUnified_Exception ;

var Len : TStore_Size64 ;
    Resource : TResource ;
    Written : cardinal ;

begin
    Set_Last_Error( nil ) ;
    Handle := USC.Translate_Handle( Kernel.PID, Handle ) ;
    if( not USC.Valid_Handle( Kernel.PID, Handle ) ) then
    begin
        Result := Set_Last_Error( Create_Error( UOSErr_Invalid_Handle ) ) ;
        exit ;
    end ;
    if( USC.Check_Quota( Kernel.PID, Quota_BIOLM, 1 ) ) then
    begin
        Result := Set_Last_Error( Create_Error( UOSErr_Quota_Exceeded ) ) ;
        exit ;
    end ;
    Resource := TResource( Handle ) ;
    Len := Resource._File.Write( Stream, Resource.Position, Length, Buff[ 0 ] ) ;
    Result := Resource._File.Last_Error ;
    if( Resource._File.Is_Store ) then
    begin
        Resource.Position := Resource.Position + Len ;
    end ;
    if( USC.Check_Quota( Kernel.PID, Quota_BYTLM, Written ) ) then
    begin
        Result := Set_Last_Error( Create_Error( UOSErr_Quota_Exceeded ) ) ;
    end ;
end ;
The changes to the Write_File method are exactly the same as for the Read_File method. First we check the buffered I/O limit and then we check the byte limit. Again, we set an exception if we exceeded a quota. I should note here that the file Write method returns the number of bytes processed. For stores, this will be the same as the number of bytes actually written. However, in the case of a terminal, the number of characters processed is not the same number that are written (such as if Control O was used to turn off output). Because of this new difference, we've added another var parameter to our file Write methods so that the number of characters actually written can be returned.

Perhaps you may have considered that we're only reporting the number of characters sent to the Write_File or retrieved from the Read_File method. This doesn't include any flow-control characters that occurred during the operation. This is intentional - we don't want to charge flow control to the user's quota any more than we'd apply a quota to hardware flow control. But, then what about echoing characters? The incoming character would count as 1 and the outgoing echo would count as 1 (or more), but since the echo bypasses the Write_File method and goes directly to the terminal, we'd only count the one character that was input. This too is intentional - UOS will not charge echoed characters against the byte limit. And then there are the asynchronous characters such as control-T, Control-O, etc. Those don't count as input (unless the terminal is in binary mode), so they aren't charged against the quota. This too, is as we plan. Control-T results in a line of status that is charged against the quota, so the one-character uncharged input results in 70 or 80 charged output. Thus, not counting the Control T has miniscule effect - even if the user is typing it over and over. With Control-O, we won't charge that against the quota because we want to encourage people to discard unnecessary output if they don't need it. The other few cases are such minor overhead that there is no need to charge them either.

Now let's take a look at the Check_Quota method. But first, let's look at the constants that method uses:

// Quotas ...
const Quota_ASTLM = 1 ; // Async I/Os
const Quota_BIOLM = 2 ; // Buffered I/Os
const Quota_BYTLM = 3 ; // Buffered I/O bytes
const Quota_CPUTIM = 4 ; // CPU seconds per session
const Quota_DIOLM = 5 ; // Raw I/Os
const Quota_ENQLM = 6 ; // Lock queue
const Quota_FILLM = 7 ; // Open files
const Quota_MAXACCTJOBS = 8 ; // Max batch jobs
const Quota_MAXJOBS = 9 ; // Max jobs
const Quota_PGFLQUOTA = 10 ; // Max pages in page file
const Quota_TQELM = 11 ;
const Quota_WSEXTENT = 12 ; // Max Working Set size
const Quota_PRCLM = 13 ; // Subprocess limit
const Quota_THREADLM = 14 ; // Thread limit
These constants provide a means of referring to a specific quota.

function TUSC.Check_Quota( PID : TPID ; Index : integer ;
    Value : int64 ) : boolean ;

var Process : TProcess ;

begin
    Process := Get_Process( PID ) ;
    Result := Process.Check_Quota( Index, Value ) ;
end ;
This method simply obtains the process instance and passes the check on to the Check_Quota method of the TProcess.

function TProcess.Check_Quota( Index : integer ;
    Value : int64 ) : boolean ;

var User : TUser ;

begin
    Result := False ;
    User := Get_User( _User ) ;
    case Index of
        Quota_ASTLM :
            begin
                Usage.ASTLM := Usage.ASTLM + Value ;
                if( User <> nil ) then
                begin
                    User.Usage.ASTLM := User.Usage.ASTLM + Value ;
                    if( User.Quotas.ASTLM > 0 ) then
                    begin
                        Result := ( Usage.ASTLM > User.Quotas.ASTLM ) ;
                    end ;
                end ;
            end ;
First we get the user associated with the process, since that is where the quotas reside (they are set per user, although most of them apply to each process). Then we base our processing on the specified quota. The first case is for the ASTLM quota. Step 1 is to add the passed value to the process' ASTLM usage variable. If the user is not nil, we also add the value to the user's ASTLM usage variable. Note that it would be unusual for a process to not have an associated user, but there are times where that is true that we will discuss in a future article (usually related to the login process before the user has successfully logged in). In such a case, we still want to keep track of the usage, even if there is no user whose quota we can validate against. If there is a user, we then check our current usage value against the user's quota for ASTLM. If the quota is 0, that means there is no limit. Otherwise if the usage exceeds the quota, we return true.

        Quota_BIOLM :
            begin
                Usage.BIOLM := Usage.BIOLM + Value ;
                if( User <> nil ) then
                begin
                    User.Usage.BIOLM := User.Usage.BIOLM + Value ;
                    if( User.Quotas.BIOLM > 0 ) then
                    begin
                        Result := ( Usage.BIOLM > User.Quotas.BIOLM ) ;
                    end ;
                end ;
            end ;
        Quota_BYTLM :
            begin
                Usage.BYTLM := Usage.BYTLM + Value ;
                if( User <> nil ) then
                begin
                    User.Usage.BYTLM := User.Usage.BYTLM + Value ;
                    if( User.Quotas.BYTLM > 0 ) then
                    begin
                        Result := ( Usage.BYTLM > User.Quotas.BYTLM ) ;
                    end ;
                end ;
            end ;
        Quota_CPUTIM :
            begin
                Usage.CPUTIM := Usage.CPUTIM + Value ;
                if( User <> nil ) then
                begin
                    User.Usage.CPUTIM := User.Usage.CPUTIM + Value ;
                    if( User.Quotas.CPUTIM > 0 ) then
                    begin
                        Result := ( Usage.CPUTIM > User.Quotas.CPUTIM ) ;
                    end ;
                end ;
            end ;
        Quota_DIOLM :
            begin
                Usage.DIOLM := Usage.DIOLM + Value ;
                if( User <> nil ) then
                begin
                    User.Usage.DIOLM := User.Usage.DIOLM + Value ;
                    if( User.Quotas.DIOLM > 0 ) then
                    begin
                        Result := ( Usage.DIOLM > User.Quotas.DIOLM ) ;
                    end ;
                end ;
            end ;
        Quota_ENQLM :
            begin
                Usage.ENQLM := Usage.ENQLM + Value ;
                if( User <> nil ) then
                begin
                    User.Usage.ENQLM := User.Usage.ENQLM + Value ;
                    if( User.Quotas.ENQLM > 0 ) then
                    begin
                        Result := ( Usage.ENQLM > User.Quotas.ENQLM ) ;
                    end ;
                end ;
            end ;
        Quota_FILLM :
            begin
                Usage.FILLM := Usage.FILLM + Value ;
                if( User <> nil ) then
                begin
                    User.Usage.FILLM := User.Usage.FILLM + Value ;
                    if( User.Quotas.FILLM > 0 ) then
                    begin
                        Result := ( Usage.FILLM > User.Quotas.FILLM ) ;
                    end ;
                end ;
            end ;
        Quota_MAXACCTJOBS :
            begin
                if( User <> nil ) then
                begin
                    User.Usage.MAXACCTJOBS := User.Usage.MAXACCTJOBS + Value ;
                    if( User.Quotas.MAXACCTJOBS > 0 ) then
                    begin
                        Result := ( User.Usage.MAXACCTJOBS > User.Quotas.MAXACCTJOBS ) ;
                    end ;
                end ;
            end ;
        Quota_MAXJOBS :
            begin
                if( User <> nil ) then
                begin
                    User.Usage.MAXJOBS := User.Usage.MAXJOBS + Value ;
                    if( User.Quotas.MAXJOBS > 0 ) then
                    begin
                        Result := ( User.Usage.MAXJOBS > User.Quotas.MAXJOBS ) ;
                    end ;
                end ;
            end ;
        Quota_PGFLQUOTA :
            begin
                Usage.PGFLQUOTA := Usage.PGFLQUOTA + Value ;
                if( User <> nil ) then
                begin
                    User.Usage.PGFLQUOTA := User.Usage.PGFLQUOTA + Value ;
                    if( User.Quotas.PGFLQUOTA > 0 ) then
                    begin
                        Result := ( Usage.PGFLQUOTA > User.Quotas.PGFLQUOTA ) ;
                    end ;
                end ;
            end ;
        Quota_TQELM :
            begin
                Usage.TQELM := Usage.TQELM + Value ;
                if( User <> nil ) then
                begin
                    User.Usage.TQELM := User.Usage.TQELM + Value ;
                    if( User.Quotas.TQELM > 0 ) then
                    begin
                        Result := ( Usage.TQELM > User.Quotas.TQELM ) ;
                    end ;
                end ;
            end ;
        Quota_WSEXTENT :
            begin
                Usage.WSEXTENT := Usage.WSEXTENT + Value ;
                if( User <> nil ) then
                begin
                    User.Usage.WSEXTENT := User.Usage.WSEXTENT + Value ;
                    if( User.Quotas.WSEXTENT > 0 ) then
                    begin
                        Result := ( Usage.WSEXTENT > User.Quotas.WSEXTENT ) ;
                    end ;
                end ;
            end ;
        Quota_PRCLM :
            begin
                Usage.PRCLM := Usage.PRCLM + Value ;
                if( User <> nil ) then
                begin
                    User.Usage.PRCLM := User.Usage.PRCLM + Value ;
                    if( User.Quotas.PRCLM > 0 ) then
                    begin
                        Result := ( Usage.PRCLM > User.Quotas.PRCLM ) ;
                    end ;
                end ;
            end ;
        Quota_THREADLM :
            begin
                Usage.THREADLM := Usage.THREADLM + Value ;
                if( User <> nil ) then
                begin
                    User.Usage.THREADLM := User.Usage.THREADLM + Value ;
                    if( User.Quotas.THREADLM > 0 ) then
                    begin
                        Result := ( Usage.THREADLM > User.Quotas.THREADLM ) ;
                    end ;
                end ;
            end ;
    end ; // case Index
end ; // TProcess.Check_Quota
The remaining quotas are handled in the same manner, except for MAXJOBS and MAXACCTJOBS. Those quotas are not process-specific - they are limits across all processes for a given user. Therefore, the quota check is against User.Usage instead of the process Usage.

In the next set of articles, we will dive into the UCL.

 

Copyright © 2018 by Alan Conroy. This article may be copied in whole or in part as long as this copyright is included.