1 Introduction
2 Ground Rules

Building a File System
3 File Systems
4 File Content Data Structure
5 Allocation Cluster Manager
6 Exceptions and Emancipation
7 Base Classes, Testing, and More
8 File Meta Data
9 Native File Class
10 Our File System
11 Allocation Table
12 File System Support Code
13 Initializing the File System
14 Contiguous Files
15 Rebuilding the File System
16 Native File System Support Methods
17 Lookups, Wildcards, and Unicode, Oh My
18 Finishing the File System Class

The Init Program
19 Hardware Abstraction and UOS Architecture
20 Init Command Mode
21 Using Our File System
22 Hardware and Device Lists
23 Fun with Stores: Partitions
24 Fun with Stores: RAID
25 Fun with Stores: RAM Disks
26 Init wrap-up

The Executive
27 Overview of The Executive
28 Starting the Kernel
29 The Kernel
30 Making a Store Bootable
31 The MMC
32 The HMC
33 Loading the components
34 Using the File Processor
35 Symbols and the SSC
36 The File Processor and Device Management
37 The File Processor and File System Management
38 Finishing Executive Startup

Users and Security
39 Introduction to Users and Security
40 More Fun With Stores: File Heaps
41 File Heaps, part 2
42 SysUAF
43 TUser

Terminal I/O
45 Shells and UCL
46 UOS API, the Application Side
47 UOS API, the Executive Side
48 I/O Devices
49 Streams
50 Terminal Output Filters
51 The TTerminal Class
52 Handles
53 Putting it All Together
54 Getting Terminal Input
55 QIO
56 Cooking Terminal Input
57 Putting it all together, part 2
58 Quotas and I/O

59 UCL Basics
60 Symbol Substitution
61 Command execution
62 Command execution, part 2
63 Command Abbreviation
64 ASTs
65 Expressions, Part 1
66 Expressions, Part 2: Support code
67 Expressions, part 3: Parsing
69 Expressions, part 4: Evaluation

UCL Lexical Functions
72 TProcess updates
73 Unicode revisted
74 Lexical functions: F$CONTEXT
75 Lexical functions: F$PID
76 Lexical Functions: F$CUNITS
77 Lexical Functions: F$CVSI and F$CVUI
78 UOS Date and Time Formatting
79 Lexical Functions: F$CVTIME
81 Date/Time Contexts
83 Lexical Functions: F$DELTA_TIME
84 Lexical functions: F$DEVICE
86 Lexical functions: F$DIRECTORY
87 Lexical functions: F$EDIT and F$ELEMENT
88 Lexical functions: F$ENVIRONMENT
90 Lexical functions: F$EXTRACT and F$IDENTIFIER
92 LIB_FAO and LIB_FAOL, part 2
93 Lexical functions: F$FAO
94 File Processing Structures
95 Lexical functions: F$FILE_ATTRIBUTES
97 Lexical functions: F$GETDVI
98 Parse_GetDVI
99 GetDVI
100 GetDVI, part 2
101 GetDVI, part 3
102 Lexical functions: F$GETJPI


Download sources
Download binaries

Putting it all together, part 2

In the previous article, we examined the default terminal input filter. The filter pulls input from the type-ahead buffer and processes it in response to a read request. The purpose of the type-ahead buffer is to allow the terminal class instance to accumulate input while the program is doing processing and not yet ready to read input. But how does data get from the terminal into the buffer? Let's start with the Input Filter New_Character method.

procedure TDefault_Input_Filter.New_Character( Key : cardinal ) ;

var PID : TPID ;

    if( Key = _Control_T ) then
        if( ( Flags and ( TIFF_Binary or TIFF_NoControlT ) ) = 0 ) then
        begin // Not in binary mode and ^T status not disabled
            PID := Term.Device.IO_PID ;
            if( PID <> 0 ) then
                Term.Kernel.USC.Show_Status( PID, Term.Kernel.USC.Translate_Handle( PID, RH_SysOutput ) ) ;
            end ;
            exit ;
        end ;
    end ;
This method receives individual characters from the terminal and adds them to the type-ahead buffer. But before we do that, we need to handle certain special characters. The Read method that we discussed in the last article processes the input synchronously. That is, it only processes the input when the program requests it. However, some keys are handled immediately upon reception. In other words, they are processed asynchronously. The first case is the Control T status request. Control T can be disabled explicitly via the TIFF_NoControlT flag, or implicitly via the TIFF_Binary flag. In either case, we process the control T like any other input characters. But if we're not in binary input mode and the status line hasn't been disabled, we call the Show_Status method of the USC, which will display the status line on the terminal. We will discuss that routine in a future article. If we process the status line request, we exit the method without putting the Control T into the type-ahead buffer.

    if( Key = _Control_C ) then
        if( ( Flags and ( TIFF_Binary or TIFF_NoControlC ) ) = 0 ) then
        begin // Not in binary mode and ^C not disabled
            _Buffer.Clear ;
            Term.Output_Filter.Reset ;
            Term.Resume_Input ;
            if( ( Flags and TIFF_NoControl ) = 0 ) then
                Term.Output_Filter.Write( '^C', False ) ;
            end ;
            Term.Output_Filter.Write( CRLF, False ) ;
            Term.Kernel.Interrupt_Process( 0 ) ; // Interrupt current PID
            exit ;
        end ;
    end ;
The next asynchronous character we handle is Control C. Like Control T, there is a flag that can disable special Control C processing. If it is disabled or we are in binary input mode, the character is added to the buffer as normal input. Otherwise, we clear the buffer, reset the output filter, and let the terminal know that we can receive input again (since the buffer is now empty). We do this because the input may have been turned off due to the buffer being full. If TIFF_NoControl is not set, we will echo "^C" to the terminal. Regardless of that flag, we also output a CR and LF to move the terminal to the beginning of the next line. Then we call the Kernel's Interrupt_Process method. This will interrupt the execution of the current program. We will discuss that method in a future article.

    if( Key = _Control_Y ) then
        if( ( Flags and ( TIFF_Binary or TIFF_NoControlY ) ) = 0 ) then
        begin // Not in binary mode and ^Y not disabled
            _Buffer.Clear ;
            Term.Output_Filter.Reset ;
            Term.Resume_Input ;
            if( ( Flags and TIFF_NoControl ) = 0 ) then
                Term.Output_Filter.Write( '^Y', False ) ;
            end ;
            Term.Output_Filter.Write( CRLF, False ) ;
            Term.Kernel.Interrupt_Process( 1 ) ; // Interrupt current PID
            exit ;
        end ;
    end ;
Control Y is processed exactly the same as Control C with the following three exceptions: 1) we check the TIFF_NoControlY flag, 2) we output "^Y", and 3) we pass a different value to the Interrupt_Process method. Again, we will discuss Interrupt_Process later. Briefly, the difference in the parameter value indicates whether we are "requesting" a program interruption or "demanding" one.

    if( Key = _Control_O ) then
        if( ( Flags and TIFF_Binary ) = 0 ) then // Cooked input
            if( ( Term.Output_Filter.Flags and TOFF_Null ) = 0 ) then
                Term.Output_Filter.Write( '^O' + CRLF, False ) ;
                    Term.Output_Filter.Flags :=
                    Term.Output_Filter.Flags or TOFF_Null ;
            end else
                Term.Output_Filter.Flags :=
                    Term.Output_Filter.Flags and ( not TOFF_Null ) ;
            end ;
            exit ;
        end ;
    end ;
Control O is used to toggle output on and off. If we are in binary mode, the character is considered normal input. Otherwise, we toggle the output filter's TOFF_Null flag. If we are toggling output to the off state, we first output a "^O" (and a CR and LF) as a visual indicator that output has been turned off.

    if( Key = _Control_Q ) then
        if( ( Flags and TIFF_NoTermXON ) = 0 ) then
            Term.Output_Filter.Flags :=
                Term.Output_Filter.Flags and ( not TOFF_Paused ) ;
            exit ;
        end ;
    end ;
    if( Key = _Control_S ) then
        if( ( Flags and TIFF_NoTermXON ) = 0 ) then
            Term.Output_Filter.Flags :=
                Term.Output_Filter.Flags or TOFF_Paused ;
            exit ;
        end ;
    end ;
Control Q and Control S are used to unpause/pause output. Unlike Control O (which actually throws output away), Control S pauses it until Control Q is sent. Being in binary mode has no effect on these characters since they are used for flow-control. However, the TIFF_NoTermXON flag can be used to treat the characters as normal input. Other than that, we change the output pause state and exit.

    // Default buffer...
    if( _Buffer.Length + Term.Max_Record_Size >= _Buffer.Size ) then
        // Buffer is full...
        Term.Pause_Input ;
        TFiP_Stream( Term.Stream )._Device.Info.Status :=
            TFiP_Stream( Term.Stream )._Device.Info.Status or DS_Buffer_Overrun ;
        exit ;
    end ;
    if( _Buffer.Length >= _Buffer.Size - 4 * Term.Min_Record_Size ) then // Almost full
        Term.Pause_Input ;
    end ;
    _Buffer.Append( Key, Term.Min_Record_Size ) ;
end ; // TDefault_Input_Filter.New_Character
First we check to see if the buffer is full. If so, we tell the terminal to pause the input, set the buffer overrun flag and exit. Then we check if the buffer is close to being full. If so, we tell the terminal to pause the output, but we go ahead and add the data to the buffer. The reason for this check is that the control flow characters/signals may not reach the terminal before it sends another few characters. By pausing the terminal before the buffer is full will help ensure that no characters are lost due to the buffer completely filling up. Finally, we append the data to the buffer.

The next question is: who makes the call to the New_Character method? Let's jump over to the terminal and follow the process from there. When a key is pressed on the terminal, the terminal sends the character to the computer. This typically generates a hardware interrupt that the CPU responds to. The HAL handles interrupts and makes a call to the Kernel's interrupt routine.
Note that even on hardware which doesn't interrupt the CPU when a character is received, the HAL simulates this by polling for input when the Idle method is called. If input is waiting, the HAL calls the Kernel's interrupt routine as if it received a hardware interrupt. From the Kernel's perspective, it doesn't matter if there was a physical or simulated interrupt.

Back in article 47 we discussed setting up the handlers for calls across rings. We will add the following statement to that section of code:

__HAL.Hook_Interrupts( integer( @CB_Interrupts ) ) ;
This tells the HAL that any interrupts are to be redirected to the CB_Interrupts method.

procedure CB_Interrupts( Value : int64 ) ;

    Kernel.CB_Interrupt( Value ) ;
    Kernel.HAL.Idle ;
end ;
This callback routine simply passed the interrupt on to the Kernel instance's CB_Interrupt method. Then we call the HAL Idle to allow the HAL to handle any other issues that came up during the processing of the interrupt. On typical hardware, this will do nothing. But on some hardware, it may be necessary for this call to be made on a regular basis.

procedure TKernel.CB_Interrupt( Value : int64 ) ;

    if( ( Value and $FFFF ) = Interrupt_IO ) then // I/O interrupt
        FiP.Interrupt( Value shr 16 ) ;
        exit ;
    end ;
end ;
This method receives incoming interrupts. For now we are only dealing with I/O interrupts. We will discuss other interrupts in the future. The HAL packs the passed value with information that defines the kind of interrupt. The lowest 16 bits define the interrupt code, which is consistent across all hardware platforms, thanks to the HAL. The remaining bits are dependent upon the type of interrupt. I/O interrupts are handled by the File Processor, so we trim off the interrupt code and call the Interrupt method of the FiP.

procedure TUOS_FiP.Interrupt( Value : int64 ) ;

var Controller : word ;
    Device : TDevice ;
    C, I : integer ;
    Got_Input : boolean ;
    _Unit : word ;

    Value := Value shr 16 ; // Ignore software/HAL flag
    _Unit := Value and $FFFF ;
    Controller := ( Value shl 16 ) and $FFFF ;
    for I := 0 to _Devices.Count - 1 do
        Device := TDevice( _Devices.Objects[ I ] ) ;
        if( Device.Info.Index = Index ) then
            if( Device.Terminal <> nil ) then
                Got_Input := False ;
                if( Device.Terminal.Input_Filter.Has_Space( Device.Terminal.Min_Record_Size ) ) then
                    while( TFiP_Stream( Device.Stream ).Driver.Input( C ) ) do
                        Got_Input := True ;
                        Device.Terminal.Input_Filter.New_Character( C ) ;
                        if( not Device.Terminal.Input_Filter.Has_Space( Device.Terminal.Min_Record_Size ) ) then
                            break ;
                        end ;
                    end ;
                end ;
I/O interrupts communicate the specific device to the File Processor. Encoded in the value are the unit, controller, and device index. So we parse out the information and then loop through our devices, looking for a match. If the device is not found, we do nothing. This shouldn't happen, but if it does we ignore it (probably indicates a bug in the HAL). Once we've found the matching device, we check to see if it is a terminal. For now, terminals are the only I/O device that we are handling, but in the future there will be others.
Once we've identified the terminal that caused the interrupt, we clear a flag (Got_Input) indicating if we obtained any input from the terminal. If our type-ahead buffer has space for the character, we obtain the next character and insert it into the buffer via the New_Character method. We keep looping until the terminal (via the Input method) has no more input. It is important to understand that a given hardware interface may buffer several characters, or it may only buffer a single character. At this point we don't know what the hardware is, nor do we care. So we keep asking for characters until we are told that there are no more. This will work for all hardware, regardless of how much it buffers. As we loop, if the type-ahead buffer fills up, we stop requesting characters and exit the loop. If we've read any characters into the buffer, we set the Got_Input flag.

                if( TFiP_Stream( Device.Stream ).Driver.Pending_Input ) then
                    Device.Terminal.Pause_Input ;
                end ;
                if( Got_Input ) then
                    if( Device.IO_PID <> 0 ) then // A process blocked on this device
                        Kernel.USC.UnBlock( Device.IO_PID ) ;
                    end ;
                end ;
            end ;
            exit ;
        end ; // if
    end ; // for I := 0 to _Devices.Count - 1
end ; // TUOS_FiP.Interrupt
After the input loop, we check to see if there are any more characters pending. If there are, it means that our buffer is full and we could not accept any more characters. So we tell the terminal to pause sending us more input. When the buffer is less full we will unpause it, as we've discussed previously.
Finally, if the Got_Input flag is set, we check to see if a process has been blocked while waiting for input from this device. If so, we tell the USC to unblock that process so that it can process the input. Note that the process could be blocked for other reasons, so we only unblock it if it is specifically blocked for I/O from this device. A process will never be blocked for more than one thing at a time.

In the last few articles, we've examined various pieces of the terminal input stack. Much of this applies to input from any/all devices and we'll only review that which is different when we discuss other types of input. So, now we've got a complete chain of processing from the terminal sending a character to the computer, the HAL processing that and calling the Kernel's interrupt handler, which calls the FIP interrupt handler, which adds the characters to the type-ahead buffer where they wait until the program is ready to receive input. The program makes a system request to obtain the input, which comes into the Read method of the input filter, where the input in the type-ahead buffer is then processed and returned.

There are always compromises that we make when writing programs. One compromise I made with UOS and terminal I/O is putting it into the executive rather than handling it at the application level, in a ring 3 system service. Certainly, anyone can write their own I/O filters at the application level and set the terminal to binary mode to bypass the normal I/O filtering. So why not do that for the default filters? The reason has to do with context changes that most CPUs have to do when switching between different rings. This can be an extremely expensive operation (relatively speaking) and so we want to reduce the overhead from such switches. This can be especially problematic if a lot of characters are coming into the computer from multiple terminals. If the input filter was in the outer ring, each character that came in would require a context switch to ring 0 to handle the interrupt, then a context switch to the application to process the character, another switch to ring 0 to output the echo, and finally another switch back to the application - for each character! However, including the filter in the ring 0 code means that we cut the number of context changes in half. Custom filters will thus have more CPU overhead than the filters included in the executive (if they are echoing characters), although it can still be done when it must.

In the next article, we will wind up our series of I/O articles with an implementation of quota checking.


Copyright © 2018 by Alan Conroy. This article may be copied in whole or in part as long as this copyright is included.