So, now that we got SPU UNIX up and running on the service processor unit (see Convex C1 XP Service Processor power-on), it's time to do something with it.

The purpose of SPU UNIX is to serve as an environment to initialize the system (load microcode onto the job processor, or main CPU; initialize memory), as well as diagnose the hardware. Wil a lot of help from ex-Convex employees (a special thanks goes to Doug Hosking), I learned a lot about what can be done in this environment.

The two main utilities used in diagnosing a system are margin and dshell. The margin command is used to check or alter the clock speed and power supply voltages. Used by itself, it checks and prints out the current status:

(spu)> margin
clk: n (10.0Mhz)  ps1: n (+5.00)  ps2: n (+4.95)  ps3: n (+5.00)

But you can also use it to raise or lower power supply voltages and clock frequency slightly:

(spu)> margin -l clk
clk: l (9.0Mhz)  ps1: n (+5.00)  ps2: n (+4.95)  ps3: n (+5.00)

The dshell command is used to enter the diagnostic shell, where you can execute diagnostic tests. The following is a list of all tests (excluding those not applicable to my system):

        CONVEX DIAGNOSTIC TEST MENU
Dir.    Test            Rev.    Description
____    ____________    ____    ___________

/mnt/test
        cpu4000.t       1.30    CPU INSTRUCTION SET TESTS 
        cpu4010.t       1.20    CPU INSTRUCTION SET TESTS 
        cpu4030.t       1.6     CPU4030 Functional Test Program 
        cpu4040.t       1.4     VECTOR CONCURRENCY TESTS 
        dev4100.t       1.33    Xylogics 450/451 Controller Test Program 
        dev4110.t       1.33    Xylogics 450/451 Controller Test Program 
        dev4200.t       1.17    STC Tape Drive Test Program 
        dev4300.t       1.22    Systech Comm Test Program 
        dev4500.t       1.18    EXOS/201 Ethernet Controller Test
        io4000.t        1.21    IOP Functional Test Program
        mem4000.t       1.16    Main Memory Diagnostic Test 
        spu4000.t       1.10    FITS Kernel Tests 
        spu4100.t       1.7     File I/O and Seek Test 

A test can be started by typing test . A list of subtests can be obtained with test -s. A selection of subtests to run can then be made. Since I only have the SPU card installed in the main card cage, running the spu4000 tests seems to be in order. However, this falls over pretty quickly:

: test spu4000
Test 'spu4000.t'                        Sun Aug 30 20:12:35 1970

        Subtest 100    0:00:00   0:00:00   0:00:00   passed
        Subtest 105    0:00:00   0:00:00   0:00:00   failed

*****  Sun Aug 30 20:12:54 1970  *****
Test:    spu4000.t  1.10   Class: 1    Subtest: 105 1.4   Count: 1    Error: 0
Failed:  SPU RUN_ARM Circuitry

Expected RUN_ARM and RUN bits set   
Address: 00ffb840 Exp: 00000009 Act: 00000008 
scn_rp: Warning -- No board present in slot mcu

Test 'spu4000.t' failed
Elapsed time:   0:00:02

Well, Doug warned me that a lot of tests may require the MCU (memory control unit) and at least one MAU (memory array unit) to be installed. So, let's do a pwrdwn and install those cards. After this change, most of the SPU tests passed, except those that talk to the scan interfaces on the other (presently not installed) cards. Some of these tests also margin the power supplies low or high.

Now that we have the MCU and MAUs installed, we should also be able to run the mem4000 tests. Most of these pass (except the ones that require the job processor (main cpu) to be present), but the sizing test reports an error:

        Subtest 340    0:00:00   0:00:00   0:00:00   failed

*****  Fri Jan  2 16:33:08 1970  *****
Test:    mem4000.t  1.16   Class: 3    Subtest: 340 1.2   Count: 1    Error: 1
Failed:  Memory sizing test

Memory-Cop inconsistency on MAU 2
MAU size Expected 16 Mbytes, Actual 0 Mbytes

Expected:
MAU     Allocated Blocks

 2      00 01 02 03 04 05 06 07 

Actual:
MAU     Allocated Blocks

 2      

Test 'mem4000.t' failed
Elapsed time:   0:03:12

The COP is a small serial EEPROM on each board that stores configuration data. Apparently, the COP on this MAU tells the system there is no memory on the board, which is clearly wrong. Perhaps that EEPROM is broken?

Anyway, I took that MAU out, and re-ran the tests. All is fine now. Time to move on, let's add the IOP (I/O Processor) card, and plug the Multibus cards into the Multibus card cage. Now, we can run the io4000 tests.

        Subtest 202    0:00:00 

SPU_UNIX(109):Trap: Soft error 

  0:00:05   0:00:10   0:00:10   0:00:10   failed

*****  Sun Jan 17 16:31:32 1971  *****
Test:    io4000.t  1.21    Class: 2    Subtest: 202 1.1   Count: 1    Error: 0
Failed:  Main memory acess test

Ccu 7: 
Memory error @ 200000
Expected: 1    Actual: 0

*************** Test started Sun Jan 17 16:30:09 1971
*************** Test ended   Sun Jan 17 16:31:32 1971

Another memory related error. This one turned out to be relatively easy to fix... Memory needs to be initialized before running the io tests. So, we do an mminit -s (the -s tells mminit not to use the job processor), and rerun the io4000 tests. Now they all pass. It should now be possible to let the IO Processor (again based around a 68000) boot its own OS.

 

iosysload -livh /ioconfig iop

should do the trick. /ioconfig is the name of a text file that contains a definition of the installed I/O cards and their addresses. As a result, the heartbeat LED on the IOP starts blinking at a steady 1Hz, that's good. Doug told me that the row of 8 LEDs on the Systech MTI-1650 16-line serial port card should do a little knight-rider scrolling. They don't. Spent some time trying to figure out what was wrong, and finally found out that the cables to the external box with the actual RS232 ports was not plugged in. Plugged that in, and the lights started scrolling as they should. I also ran the tests for the tape controller (dev4200), disk controller (dev4100), serial card (dev4300) and LAN interface (dev4500). The LAN interface tests passed, the others failed on tests that actually needed the devices to be hooked up. So far, so good.

So far, progress has been really very encouraging, but we're running out of things we can try without the main CPU (job processor), so, let's add the main CPU cards (a few at a time, running the appropriate SPU tests after installing them). With all cards installed, we should be able to load the microcode onto the CPU:

(spu)> initall -c
System Initialization
clk: n (10.0Mhz)  ps1: n (+5.00)  ps2: n (+4.85)  ps3: n (+5.10)

Loading ASU control stores
Reading file /mnt/usr/ucode/base.fasu  Rev 1.7  
cs: Checksum error in /mnt/usr/ucode/base.fasu
Initialization Aborted

Drats, we have a corrupt file on the hard disk. So, to make a long story short, I opened the disk image in a hex editor, figured out where the directory entries and inodes were, figured out which blocks on the disk were part of the base.fasu file, re-read those blocks from the original SPU disk (using a slightly slower process, reading them one block at a time), and found a difference. I patched the new SPU disk with the corrected data, reran the initall, and:

(spu)> initall
System Initialization
clk: n (10.0Mhz)  ps1: n (+5.00)  ps2: n (+4.90)  ps3: n (+5.00)

Loading ASU control stores
Reading file /mnt/usr/ucode/base.fasu  Rev 1.7  0:01
Loading wcs and epcs    0:10
Verifying wcs and epcs  0:19
Loading VCU control stores
Reading file /mnt/usr/ucode/lsctl.lcs  Rev 2.12  0:01
Loading lcs             0:08
Verifying lcs           0:07
Reading file /mnt/usr/ucode/actl.acs  Rev 1.22  0:01
Loading acs             0:08
Verifying acs           0:07
Reading file /mnt/usr/ucode/mctl.mcs  Rev 1.17  0:01
Loading mcs             0:07
Verifying mcs           0:08


Memory interleave set to 4
Initializing memory   0:00:00 
Hard error reported by MCU
Multiple bit error
Address:  04003000   Syndrome: 77    MAU:  MAU4
Memory master: PCU


mminit: Unable to initialize main memory
Initialization Aborted

Ok, the microcode loads, but the main CPU does not like something about the memory on board MAU4. Let's take that one out.

Initializing memory   0:00:00 
Hard error reported by MCU
Nonexistent memory error
Address: 0400b010


mminit: Unable to initialize main memory
Initialization Aborted

Drats, more memory problems, but no indication which MAU(s) are involved. I won't bother you with everything I tried (no, the mem4000 tests did not help), I stumbled upon a combination of four MAUs that did work:

(spu)> initall
System Initialization
clk: n (10.0Mhz)  ps1: n (+5.05)  ps2: n (+4.95)  ps3: n (+5.00)

Loading ASU control stores
Reading file /mnt/usr/ucode/base.fasu  Rev 1.7  0:01
Loading wcs and epcs    0:10
Verifying wcs and epcs  0:19
Loading VCU control stores
Reading file /mnt/usr/ucode/lsctl.lcs  Rev 2.12  0:00
Loading lcs             0:08
Verifying lcs           0:07
Reading file /mnt/usr/ucode/actl.acs  Rev 1.22  0:00
Loading acs             0:08
Verifying acs           0:07
Reading file /mnt/usr/ucode/mctl.mcs  Rev 1.17  0:01
Loading mcs             0:07
Verifying mcs           0:08


Memory interleave set to 4
Initializing memory   0:00:00   0:00:03   0:00:03 

Main memory size:  67108864

MAU     Allocated Blocks

 0      00 01 02 03 04 05 06 07 
 1      00 01 02 03 04 05 06 07 
 2      00 01 02 03 04 05 06 07 
 3      00 01 02 03 04 05 06 07 
 4      
 5      
 6      
 7      

Initialization Complete

Ok, we have 64MB of memory to play with... Let's try to run all of mem4000 (including the tests that use the job processor):

        Subtest 601    0:00:00   0:00:05   0:00:08   0:00:08   failed

*****  Thu Jan  1 05:46:53 1970  *****
Test:    mem4000.t  1.16   Class: 6    Subtest: 601 1.4   Count: 1    Error: 1
Failed:  Vectorized address uniqueness test

Mode:    EDC disabled   
Address: 03265998   Exp: 00000000 03265998   Act: 00000000 83265998

MAU:     MAU 3
RAMS:    E0-017

Test 'mem4000.t' failed
Elapsed time:   0:03:00

That's a very useful error message. It indicates a single RAM chip (location E0-017 on MAU3) that's bad. I replaced that chip, and reran the tests. It pointed out another bad RAM chip. Replaced that too, and a third. After that, all mem4000 tests passed.

Let's try to test the CPU:

: test cpu4000
Test 'cpu4000.t'                        Thu Jan  1 05:37:48 1970

Run default options                                  : y

libtest:  SIGSEGV received, using SIGIOT to dump core

This one turned out to be another corrupt file on the disk. It crashes before it even starts the subtests. cpu4010 and cpu4030 did pass. I fixed the disk corruption using the find-the-blocks-in-these-files-and-reread-them routine, and got cpu4000 to get as far as the first vector test:

        Subtest 925    0:00:00   0:00:00   0:00:00   failed

*****  Thu Jan  1 05:38:55 1970  *****
Test:    cpu4000.t  1.30   Class: 20   Subtest: 925 1.30  Count: 1    Error: 0
Failed:  vl = 0/vec. inst. test, source: vlzero.s

Subtest Failure, JP halted

a0: 0007d0f0    s0: 00000000 00000010    t0: 00000000    pc:   0002ae08
a1: 00000100    s1: 00000000 00000000    t1: 00000000    ca:   0002ae08
a2: 0002acf8    s2: aaaaaaaa aaaaaaaa    t2: 00000000    psw:  02000000
a3: 00000000    s3: 00000000 00000000    t3: 00000000    jpcr: 01d77f08
a4: 00000000    s4: 00000000 00000000    t4: 02000002    upc:       bfd
a5: 0002adea    s5: 00000000 00000000    t5: 00000000
a6: 00000000    s6: 00000000 00000080    t6: 00000014
a7: 00000000    s7: 00000000 00000000    t7: 01d77f08
LCTL: 4c4c0012000a96      MCTL: 300014000a90          ACTL: 280020002696
STLCTL,VL: 17fff207,7f    STMCTL,VL: 20011207,7f      STACTL,VL: 3bc11207,7f
DATA BUS: 0000000000000000  MISC: 383d1
VM REG STATE: 40000       VM: 0000000000000000 0000000000000000
C0,ST0: 0101,10101  C1,ST1: 0101,10101  C2,ST2: 0101,10101  C3,ST3: 0101,10101


Test 'cpu4000.t' failed
Elapsed time:   0:02:32

The cpu4040 tests also failed, so there appears to be something wrong with the vector units. At Doug's suggestion, I reran these tests with the clock margined down to 9 MHz, but that did not help. I then checked the board revisions between the C1 XP and the C1 XL, and found that the vector cards (2 VPUs - vector processor unit and 1 VCU - vector control unit) had the same revision level. I put the cards out of the C1 Xl into the C1 XP, and...

cpu 4000 passes. cpu4040 is a very intensive, long-running vector test, but after 5 hours and 26 minutes, that passed as well. One thing left to do. On the SPU disk is a copy of the main Convex UNIX kernel (/mnt/os/vmunix). There is no root disk, so it will fall over, but let's see if we can boot that kernel:

(spu)> /mnt/os/boot
Booting Convex UNIX.  Type ^C to abort
System Initialization
clk: n (10.0Mhz)  ps1: n (+5.00)  ps2: n (+4.95)  ps3: n (+5.05)

Memory interleave set to 4
Initializing memory   0:00:00   0:00:04   0:00:04 

Main memory size:  67108864

MAU     Allocated Blocks

 0      00 01 02 03 04 05 06 07 
 1      00 01 02 03 04 05 06 07 
 2      00 01 02 03 04 05 06 07 
 3      00 01 02 03 04 05 06 07 
 4      
 5      
 6      
 7      

Initialization Complete
[SPU_@05:42:18] Errlog started: -l /mnt/errlog
[SPU_@05:42:19] 
[SPU_@05:42:22] errintd(1.13) started, options:
Loading vmunix
[SPU_@05:42:27] mm_sniff: sniff rate: 32.14 MB/day (1.99 days/pass)
vmunix: text: 749568  data: 77824  bss: 1409024  
[CPU_@05:42:47] Convex UNIX Version 7.1.0.10 Mon Mar 27 15:29:00 CST 1989
[CPU_@05:42:50] Device "DKC-001" (7/0/0x3f0 int 2)
[CPU_@05:42:50] Unit   "DKD-005" unit number 0
[CPU_@05:42:50] Unit   "DKD-005" unit number 1
[CPU_@05:42:50] Device "MTC-001" (7/0/0xc0 int 4)
[CPU_@05:42:50] Unit   "MTD-001" unit number 0
[CPU_@05:42:53] Device "ACM-001" (7/0/0x3c0 int 7)
[CPU_@05:42:56] Device "LAN-001" (7/1/0x4c0 int 1)
[CPU_@05:42:56] Unit   "ex" unit number 0
[CPU_@05:42:56] NX Firmware Version 5.5, Hardware Rev. 0.0
[CPU_@05:42:56] ex0: address = 08:00:14:12:92:32
[CCU7@05:42:56] da0: DrvR:Drive Not Ready [cyl=759 hd=18 sect=58 cnt=0]
[CPU_@05:42:57] WARNING: no swap space
[CCU7@05:42:57] da0: DrvR:Drive Not Ready [cyl=759 hd=18 sect=58 cnt=0]
[CPU_@05:42:58]         FATAL CONVEX UNIX ERROR: (rootmount cannot mount root)
[CPU_@05:42:58]         sp:     0b172f44        a1:     00000000
[CPU_@05:42:58]         a2:     00000000        a3:     0038bb70
[CPU_@05:42:58]         a4:     0038ab70        a5:     00000400
[CPU_@05:42:58]         ap:     0b172f58        fp:     0b172f44
[CPU_@05:42:58]         s0: ffffffff00000005    s1: 0000000040000001
[CPU_@05:42:59]         s2: 400000ff00000000    s3: 8000000000000010
[CPU_@05:42:59]         s4: 0000000000000000    s5: 0000000000000000
[CPU_@05:42:59]         s6: ffffffff00000005    s7: ffffffff00000005
[CPU_@05:42:59]         int. mask: 40000001
[CPU_@05:42:59] syncing disks... 
[CPU_@05:42:59] done
[CPU_@05:42:59] halting processor

/mnt/os/boot: 104 Quit - core dumped

There you go. This might well be the only working Convex C1 CPU on the planet...