Tenex
The TOPS-20 operating system by DEC was the second proprietary OS for the PDP-10. It was preferred by most PDP-10 hackers over TOPS-10 (at least by those who were not ITS or WAITS partisans). TOPS-20 began in 1969 as Bolt, Beranek and Newman's TENEX operating system, using special paging hardware. The system is entirely unrelated to the similarily-named TOPS-10. more...
TENEX
In the 1960's BBN was involved in a number of LISP-based artificial intelligence projects for DARPA, many of which had very large (for the era) memory requirements. One solution to this problem was to add paging software to the LISP language, allowing to write out unused portions of memory to disk for later recall if needed. One such system had been developed for the PDP-1 at MIT by Dan Murphy before he joined BBN. Early DEC machines were based on an 18-bit word, allowing addresses to encode for a 262kword memory. The machines were based on expensive core memory and included nowhere near the required amount. The pager used the otherwise unused bits of the address to store a key into a table of blocks on a magnetic drum that acted as the pager's backing store, and the software would fetch the pages if needed and then re-write the address to point to the proper area of RAM.
In 1964 DEC announced the PDP-6. DEC was still heavily involved with MIT's AI Lab, and many feature requests from the LISP hackers were moved into this machine. BBN became interested in buying one for their AI work when they became available, but wanted DEC to add a hardware version of Murphy's pager directly into the system. With such an addition, every program on the system would have paging support invisibly, making it much easier to do any sort of programming on the machine. DEC was initially interested, but soon (1966) announced they were in fact dropping the PDP-6 and concentrating solely on their smaller 18-bit and new 16-bit lines. The PDP-6 was expensive and complex, and had not sold well for these reasons.
It wasn't long until it became clear that DEC was once again entering the 36-bit business with what would become the PDP-10. BBN started talks with DEC to get a paging subsystem in the new machine, then known by its CPU name, the KA-10. DEC was not terribly interested. One development of these talks was the inclusion of two dual memory areas, allowing all programs to be divided into a protected (exec in DEC-speak) and user side. Additionally, DEC was firm on keeping the cost of the machine as low as possible, including only 16k words of core and placing registers in RAM, resulting in a considerable performance hit.
BBN nevertheless went ahead with its purchase of several PDP-10s, and decided to build their own hardware pager. During this period a debate began on what operating system to run on the new machines. Strong arguments were made for the continued use of TOPS-10, in order to keep their existing software running with minimum effort. This would require a re-write of TOPS to support the paging system, and this seemed like a major problem. At the same time, TOPS did not support a number of features the developers wanted. In the end they decided to make a new system, but include an emulation library that would allow it to run existing TOPS-10 software with minor effort.
Read more at Wikipedia.org