1 Introduction
2 Ground Rules

Building a File System
3 File Systems
4 File Content Data Structure
5 Allocation Cluster Manager
6 Exceptions and Emancipation
7 Base Classes, Testing, and More
8 File Meta Data
9 Native File Class
10 Our File System
11 Allocation Table
12 File System Support Code
13 Initializing the File System
14 Contiguous Files
15 Rebuilding the File System
16 Native File System Support Methods
17 Lookups, Wildcards, and Unicode, Oh My
18 Finishing the File System Class

The Init Program
19 Hardware Abstraction and UOS Architecture
20 Init Command Mode
21 Using Our File System
22 Hardware and Device Lists
23 Fun with Stores: Partitions
24 Fun with Stores: RAID
25 Fun with Stores: RAM Disks
26 Init wrap-up

The Executive
27 Overview of The Executive
28 Starting the Kernel
29 The Kernel
30 Making a Store Bootable
31 The MMC
32 The HMC
33 Loading the components
34 Using the File Processor
35 Symbols and the SSC
36 The File Processor and Device Management
37 The File Processor and File System Management
38 Finishing Executive Startup

Users and Security
39 Introduction to Users and Security
40 More Fun With Stores: File Heaps
41 File Heaps, part 2
42 SysUAF
43 TUser
44 SysUAF API

UCL
45 Shells and UCL

Glossary/Index


Download sources
Download binaries

Shells and UCL

All operating systems come with some sort of user interface (otherwise they'd be useless in most cases). These interfaces have had many names over the years: "CCP" (Console Command Processor) for CP/M, "RTS" (Run Time System) for RSTS/E, "Command Line" on MSDOS, and "bash" for Unix/Linux, as but a few examples. On RSTS/E and Unix, there were several different interfaces available, and on Unix these interfaces were called "shells". Imagine the kernel/executive as the yolk and white of an egg, and the user interface as the shell that surrounds it. Getting access to the inside requires going through the shell. Hopefully, in terms of software, this doesn't require breaking the shell! The term "shell" has come to be a general name for Operating System interfaces - at least the textual ones. On Mac OS and Windows, the shell is graphical and is called the "Desktop".

In olden days, the shell was often built into the O/S. Even today, the windows shell is an integral part of the O/S - although not of the kernel. Frequently, the shell was a BASIC interpreter. This was the case with older versions of RSTS/E, for instance. This approach provides a powerful interface since the user could enter simple commands or write complex code to accomplish whatever task was at hand. This was also a popular approach for the original personal computers, including the TRS-80, Commodores, Apples, and Amigas, among others.

While modern shells aren't BASIC interpreters, the importance of being abile to do some sort of programming in the shell was recognized. Thus, shells like bash and MSDOS included the ability to use variables and conditionals in addition to the essential commands and programs. Likewise, the VMS shell (DCL) is an advanced programming interface. Sometimes shells are called "scripting engines", especially if they can "run" files containing code recognized by the shell. This code is often referred to as a "script". The line between a "script" and a "program" (especially interpreted ones) is somewhat fuzzy, and some scripts can be "compiled" into stand-alone programs. The main difference is that a script is usually customized for some other software platform, as opposed to being more "general-purpose".

Graphical User Interfaces (GUIs) tend to not include scripting - a scripting engine provides that capability. Textual interfaces are distinguished from GUIs by the name "command line" interfaces. Some even make a distinction between a "textual user interface" and a "command line interface". We won't make that distinction.

What is the nature of a shell, besides providing some rudimentary programming ability and being a way for the user to interact with the system? From the perspective of the average user, the shell is the operating system. But, a shell is simply a program. On UOS, the only difference from any other program is that a shell can be set as the program to run when no other programs are running. That is, a user is always running a program whether he realizes it or not. This might seem inefficient - if the user is doing nothing, why should a program be running and using system resources? But, in fact, while waiting for user input, the shell will be in an "input wait state", which uses no CPU or RAM. When the user runs another program, the shell exits and passes control to the program. When that program ends, the operating system automatically starts up the shell.

UCL
UOS has two user interfaces that are part of the overall operating system: UCL (UOS Command Language) and UDesk (UOS Desktop). UCL is the textual interface and scripting engine, and UDesk is a GUI. Discussion of UDesk will be in a far future article. We start with UCL, because although all graphics-capable devices are able to show text, not all textual devices can show graphics. UOS supports both. Therefore, we will start with the lowest common denominator - the text-only interface, UCL.

UCL is the name of the script as well as the name of the program that executes that script. The script is compatible with VMS's DCL. The basic features of UCL are:

  • Prompt user
  • System commands
  • Ability to run other programs
  • Programming/scripting
    • Variables
    • Functions
    • Flow control
On VMS, the RUN command was needed to execute a program. In UCL, any command entered by the user, and not recognized by UCL, is assumed to be a program name and the said program is executed. UCL also supports RUN, but that is necessary only when a program has the same name as a UCL command. When looking for a program not specified by RUN, the system CUSP folder (sys$system) is searched. For a RUN command, the user's path is used, and if that fails, then sys$system is used.

Although UCL serves as an interactive interface, it also can take commands from a file. Such files are called "command files" (files with an extension of ".com", by default). In fact, UCL can take input from a terminal, a file, or any other input device. Nor is this capability unique to UCL, or shells in general. All processes have certain logical I/O devices defined for them:

  • SYS$INPUT - non-command input
  • SYS$OUTPUT - Normal output
  • SYS$COMMAND - Command input
  • SYS$ERROR - Error output
By default, all of these logicals point to the terminal device. Thus, UCL takes input from the terminal and writes output to it. But when processing a command file, sys$command is pointed to a file, so that UCL takes commands from that file. Output still goes to the terminal. If UCL requires input from the user, it uses sys$input, which comes from the termainal. Of course, sys$input can be redirected to a file so that such input also comes from the file. This I/O redirection is used for other purposes as well.

Rings
Since the main purpose of UCL is to take input and act on it, we need to be able to read data from a device. However, unlike the executive, which can directly call routines in the FiP (and eventually the HAL), programs (including UCL) do not have direct access to the FiP or HAL. Most of the old 8-bit CPUs had no protection mechanisms that would guard against errant or malicious software. On those machines, a program could call directly into the kernel. Even so, the effort required to call a specific function directly would be difficult since there are multiple entry points which change each time a new version of the O/S is released. So, either there was a value stored at a known location that pointed to the API (Application Programming Interface) function, or the address contained a jump instruction that the application could call to. However, these methods require a fair number of bytes of code to use, so a more compact method was generally used to make O/S calls - the interrupt mechanism. Interrupts have different names on some systems (for instance, on the PDP-11 they were called "traps"), but the function is the same - a single instruction can cause the CPU to save the current location and jump to a handler associated with that interrupt. On a system with hardware protection, the interrupt also switched into a higher-privilege mode that allowed access to hardware features that were otherwise unavailable to the application. This allowed the O/S to validate the call and prevent certain operations (such as memory management or I/O operations) if the requesting application did not have sufficient privileges. On modern Intel CPUs, a mechanism called "call gates" is used to perform the same function as an interrupt. Our UOS simulator uses yet another mechanism. But the specifics are of little consequence until we get around to writing a HAL for a specific platform - so we won't worry about them here.

Not surprisingly, the hardware-controlled elevated abilities have different names and implementations on different CPUs. For instance, most models of the PDP-11 had 3 modes: user, supervisor, and kernel. Modern Intel CPUs have four protection levels: kernel, system services, O/S extensions, and application. Other systems have other models. In general terms, these levels of access are referred to as "rings". Usually ring 0 is the most powerful level and is reserved for the kernel (or the executive in UOS), and higher rings indicate lower levels of access. Usually each ring has a completely separate set of register values, memory mapping, etc. so that it is impossible for an outer ring to access the context of a more inner ring, but inner rings can always access the context of a more outer ring. Not only does this mechanism allow protection from malicious code, it also protects the system from failure because of an application bug. Many times, a program with an errant pointer would crash the MSDOS operating system, which didn't use a ring model (because the original CPUs it ran on didn't have hardware protection). The scheme works because the only way for code at one ring to jump to code in a more inner ring is to use whatever mechanism the CPU provides for this, which turns control over to the code in the inner ring at a specific location. The following diagram illustrates the ring approach:

UOS uses a four-ring protection model, as follows:

  • Ring 0: Executive
  • Ring 1: Device drivers
  • Ring 2: Services layer
  • Ring 3: User layer
UOS uses this model regardless of the hardware protection mechanism that is available. The logical ring is mapped to a physical ring by the HAL. There may be a one-to-one mapping between logical and physical rings, or the HAL may need to combine logical rings into a smaller set of physical rings. On a CPU with no hardware protection, all logical rings would map to physical ring 0 (the only one available). Physical ring 0 will always be the executive, and the outermost physical ring will always be for the user. How the remaining logical rings are mapped depends on how many physical rings there are on the platform. The following tables illustrate this.

One physical ring:
Logical Ring Physical Ring
0 0
1 0
2 0
3 0

Two physical rings:
Logical Ring Physical Ring
0 0
1 0
2 1
3 1

Three physical rings:
Logical Ring Physical Ring
0 0
1 1
2 2
3 2

Four, or more, physical rings:
Logical Ring Physical Ring
0 0
1 1
2 2
3 3*
* Whatever the outermost physical ring is.

Thus, for an application program to do anything with the hardware, it must request the operation through the UOS executive. And to make a request to the executive, it must make a call to the innermost ring. However, since the actual mechanism may differ between platforms, applications that wish to be platform-independent must use a custom library that is included with each copy of UOS, which uses the mechanism appropriate for the hardware. This library is called "Starlet", and we will discuss it more in the future.

One more issue for consideration is how we use the compiler that we are writing UOS in. All compilers include some form of interaction with the operating system - for I/O if for nothing else (a program that does no input or output and doesn't otherwise interact with the system is of no practical use). So, all compilers are implemented for a specific O/S. Since we are using a compiler that "targets" Windows, it won't work if we use the built-in file or heap support. Such an attempt would result in the program trying to send a request to Windows via the mechanism used by that O/S. But, since our programs will be running on UOS, they will have to send appropriate requests to UOS instead. We encountered this situation in the Init bootstrap and wrote a couple simple classes that simply made calls to the HAL rather than using certain built-in Pascal language features. At some future date, we will have a compiler that directly targets UOS, but in the meantime we will have to be careful to use Starlet to properly interact with the UOS executive. Note that this is not an issue that is unique to Delphi, or to Pascal. A Windows C++ compiler or Python interpreter are also tied to the Windows O/S. Even so-called "platform-independent" software such as C# and .net are tied to a given platform indirectly by the CLR (Common Language Runtime system). The CLR is simply a type of compiler, so it is subject to the same issue as any other compiler. In the case of an interpreter, the problem is simply moved from the compiled program code to the interpreter that runs it. In other words, you can't escape it - at some point a running program will need to interact with the O/S that it is running on. I will briefly acknowledge the existence of emulators, which map calls for one O/S to another. But the emulator has to be able to access the UOS API, so it doesn't solve the immediate problem. The subject of emulators is something that we will discuss in the future, because emulators do solve one class of problem - just not the one at hand.

Now that we've laid the conceptual foundation, in the next article, we can discuss the UOS API and implement a file class to make use of it.