Have you ever used SideKick before? How about the new DOS 5.0's DOSKEY? When you run those programs, they terminate (i.e. the DOS prompt C:\> comes back), but they stay resident and can become active again when you call their "hot key" (i.e. the Up and Down arrows in DOSKEY).

How did it all start? When IBM originally created its PC, it needed an operating system and a programming language for its new machine. So they went to Microsoft for a BASIC language interpreter. However, Microsoft said they would be pleased to provide IBM with both BASIC and an 8088 operating system.

The problem was that Microsoft didn't have an 8088 operating system either, and being a small company back then, they didn't have the resources to build one by IBM's deadline. So they found an even tinier company called Seattle Computer who had built a small operating system for the 8088 and 8086 microcomputers called 86-DOS. Microsoft bought it and everything seemed fine until we all began really using the new operating system. We wanted additional things from MS-DOS, like printing in the background or supporting serial port printers. Microsoft heard us and made a series of small changes and additions to the internals of MS-DOS which allowed extra "add-on" code to be "hung" onto the outside of MS-DOS and also provided for limited communication between DOS and the add-on code. Microsoft's idea was that these MS-DOS limitations would be "fixed" by writing some new DOS commands (in the above examples PRINT and MODE) which would make use of special undocumented aspects of their latest version operating system.

The problem was that MS-DOS, still really the old 86-DOS in disguise, was never built to support multitasking. A command like PRINT, which can continue to operate even after the user has begun doing something else, could very easily confuse it, since now there could be service requests coming into MS-DOS from two quite different places at once.

Since this began making the entire operating system rather fragile, Microsoft's grand plan was to retain all these special hooks and communications paths as a company secret. They figured that as long as they were the only ones to use these new MS-DOS secrets, things wouldn't get out of hand. But Microsoft must have underestimated the cleverness and determination of the development community. When the MODE command proudly stated: "Resident portion installed", programmers' eyebrows shot up! Resident portion? Hmmm. If the PRINT command could access the disk for file printing when we were messing around with a spreadsheet, why couldn't we do that?

Using debugging software, programmers probed into the very heart of Microsoft's own PRINT and MODE commands, unscrambling and deciphering the undocumented secrets of their operation. Using the same secrets Microsoft's own developers had used, they learned how to give their own programs similar capabilities.

What are these tricks and techniques? How do resident programs stay in memory? The main operating RAM memory of an IBM compatible PC is a contiguous block beginning at address zero (called the bottom of RAM memory) and extending upward for a maximum of 640K bytes (called the top of memory). Since the very bottom of this large contiguous block of RAM is reserved for system purposes, DOS is booted by copying it from disk into RAM just above this low-RAM region. Once DOS has been loaded, all the space above it, to the top of RAM, is available for applications. This location just above DOS is called the load point for application software. Accordingly, when a program is started, it is copied from disk into RAM starting at this load point, continuing upward until the entire program has been loaded. The memory available for the program's data and manipulation extends from the end of itself to the physical top of the RAM memory. Most standard programs have control of the computer until the user tells the program to terminate. After performing whatever cleanup and file saving might be required, the program simply informs DOS that it is all finished. Responding to this, DOS simply forgets all about the program and whatever data it might have had in RAM. It displays the DOS prompt and awaits another command.

Now consider the special case of the resident program. It begins innocently enough, just like any other application program. When its name is typed at the DOS prompt, DOS copies it from disk into the user's RAM memory starting at the same load point as any prior application software, then hands it control. But at this point, things start getting interesting! The resident program will generally display the fact that it is resident and show a "hot key" to activate it again. Then it does a few non-standard things to the system which keep the program from being ex-communicated after it terminates. Rather than simply saying "terminate me" to DOS, it says: "I am xxx bytes long. Now terminate me but don't overwrite me."

Upon receiving this request, DOS takes control from the program, moves the software load point upward by those xxx bytes, and only then returns the DOS prompt to the user. Since the load point has been moved up by "xxx" bytes, all subsequent programs will be loaded after the programs which asked to remain resident, thus leaving it undisturbed in the system's memory. Additionally, since it's the resident program which tells DOS how big it is, it is able to declare itself to be any length it chooses! This is how programs like SideKick, which are only 39K bytes long while on the disk, can have a 50K notepad file and take up 89K bytes while in RAM! It effectively deceives DOS about how big it is, so DOS obligingly leaves it space for itself and for however much working memory it wants.

So, we've now seen how programs are able to remain in memory after being officially terminated. How can they then "pop up"? A computer's actions are directed by its reading of instructions which are stored in its main memory. These instructions may be temporary, as in the case of programs loaded from disk into RAM, or permanent, as in the case of instructions permanently wired into ROM chips. In either case, each instruction has a unique address which is just the number of the byte it occupies in the machine's memory. We cause the computer to perform a given task by having it read a series of instructions which detail the task.

In a sense we call upon those instructions whenever we need to exert their intended function. Such a called instruction sequence generally has a clever instruction called return which is able to send the computer back to wherever it was called from. IBM compatible computers come equipped with a BIOS ROM which contains a host of very useful general purpose functions which return control to their caller as soon as they have performed their designated duty. This is exactly what that often mentioned BIOS ROM is all about! BIOS stands for "Basic Input Output System", and consists of a collection of standard functions for reading and writing to the system keyboard, screen, mass storage, and many other standard devices. Naturally, we must know the exact address of these functions in order to call them into service. By convention, the starting address of every built-in function in the IBM PC is located in a predefined vector table of addresses located at the very start of the computer's RAM memory. Every function is numbered simply by the location of its address in this table. This means that in order to call for any built-in IBM PC function we only need to know its standard function number. From this we're able to calculate the location in the table where that function's starting address can be found. There's even a clever computer instruction which lets you directly call upon a function whose address is stored at a specific location in this table. So in effect you can simply ask the computer to call "standard function number 7". In response to this function request the computer will look into the 7th slot in the table for the address of function number 7, and go there directly, returning to you when the function has been performed.

Even the electrical hardware of the computer is able to call upon these standard functions. In fact many of the standard functions in the ROM BIOS are designed to operate with the PC's hardware. For example, every time a keyboard key is either pressed or released the standard function number 9, whose address is naturally in location 9 of the Address Table, is called upon by the hardware. This function is called even if the software was busy doing something else at the time!

This ability of the hardware to interrupt whatever the software might be doing at the time is called a "hardware interrupt". Essentially the keyboard hardware says to the processor: "Attention CPU, remember where you are and what you are doing. Now go perform the function whose address is in slot number 9. When that function returns to you, go back to whatever is was you were doing." This is exactly how we are able to type ahead of our computers. Even if our spreadsheet is lost in thought recalculating our tax refund, we're able to type ahead of it! Each depression and release of a key steals a fraction of a moment, a slice of time, away from the spreadsheet's recalculation. The function invoked by the keyboard's hardware interrupt places the key into a type-ahead buffer which the spreadsheet will read from as soon as its good and ready. Then it returns control to the spreadsheet as if nothing at all had happened.

Now imagine what would happen if some program like our resident one were to point the pointer stored in the 9th table location to somewhere else, such as to itself, prior to issuing the Terminate and Stay Resident (TSR) request to DOS! Then the hardware interrupt occurring at every keyboard action would turn control over to this TSR resident program instead of to the normal built-in keyboard handling function! If the resident program wasn't interested in that particular keyboard action or keystroke, it could simply pass control along to the place where the pointer had been pointing before just as if it hadn't been eavesdropping at all! However, if the key action happened to be this resident program's assigned "hot key", it could retain control of the system as long as it wished, before passing it along as if nothing at all had happened!

That's all there is to it! As we've seen, the clever application of these various tricks and techniques can significantly expand the power and flexibility of MS-DOS operating system.