The Maths/Physics SCAN

What is the Maths/Physics SCAN?

The Maths SuperComputer At Night initiative harnesses the power of many individual PCs to form a supercomputer capable of carrying out very large computational tasks such as model simulations, etc. If you can imagine a large computer containing not just one or two CPUs (Central Processing Unit, or processors) but 50, 100 or 200 of them plus a huge amount of memory, this is a very good approximation of the Maths supercomputer. Currently, all 36 of the HP dc7900 PCs in Huxley room 215 plus the 43 HP dc7800 PCs in 410 and 411 form part of this cluster outside of normal college hours - this gives us a total of 36 64-bit quad-core CPUs, 43 64-bit dual-core CPUs and 172 gigabytes of memory. As these computers would otherwise be idle at night and at weekends and during college holidays, this gives us many megaflops of raw processing power for no real cost.

The diagram below depicts SCAN 1 as it was in 2006. This SCAN has now been superseded by SCAN 3 and the servers presley and potomac have been replaced by the single potomac3 server while the PCs are now either HP dc7800 (Huxley 410 and 411) or HP dc7900 (Huxley 215). In addition, a HP blade server fitted with 10 blades each containing two 64-bit quad-core CPUs and 16 GB of memory has been added to SCAN 3. Otherwise SCAN 3 remains the same as SCAN 1 and the diagram is reproduced here for illustrative purposes.

How does it work?

These machines all have Windows XP Professional and those in Huxley 410 and 411 also have Red Hat Enterprise Linux Workstation 5 installed; these computers operate as normal PCs during the daytime and are either Windows-only or dual-boot, meaning they can be booted into either Windows or Linux. At the end of the day when the room is closed to students, the machines shut down automatically and then boot FreeBSD UNIX from a network boot server, running UNIX entirely in RAM (memory) and leaving the machine's own hard disk untouched.

Each system is essentially an autonomous node but they are all networked together and can communicate with each other and with the controller. So each system could be thought of as a CPU with its own memory attached and is linked to other CPUs and memories in the SCAN.

The computational work to be done is then downloaded onto the boot server (which is also the SCAN's central controller), which knows how to divide up the tasks involved and distribute them to each PC in the SCAN for processing. Output from the computations is sent back to the controller and written to disk files, etc in the usual way on the controller's disks, not the PC's disk.

Early in the morning, after the SCAN has worked all night, the cluster shuts down automatically and reboots back into Windows, ready for the room's reopening to students.

At what times is the SCAN operational?

The newly-introduced HP blade system is permanently available for SCAN work as it is not used for any other purposes such as teaching or for student open access - the system is physically accommodated in the ICT data centre and enjoys uninterruptible power and airconditioning.

As the SCAN is distributed over three different rooms with differing opening times, the number of CPUs available in the SCAN varies according to the day of the week and also, the time of day. From Monday to Thursday the SCAN is operational with 18 CPUs from 6 pm in 411 when the Maths library closes, from 8 pm in 410 with another 25 CPUs and from 11pm in 215 with the full complement of 79 machines; all three SCAN CPU groups then run until 6.50am the following morning. On Friday evenings, the SCAN groups in rooms 410 and 411 start up at the same times as for the other weekdays and continue to run over the weekend until 6.50am the following Monday. However, the PCs in room 215 continue to be available for student use throughout Saturday and Sunday between 6.50 am and 11 pm.

During the Easter, summer and Christmas vacations, rooms 410 and 215 are closed altogether from the end of term and the clusters there will be running full time as part of the SCAN over this period. We will get an awful lot of computing done! In addition, the 18 PCs in the library computing room (room 411) will join the SCAN whenever the library is closed.

If rooms 215 and 410 are closed, are there any Windows PCs I can use?

The library computing room (room 411, inside the Maths Library) houses 18 dual-boot PCs running Windows XP/Red Hat Enterprise Linux Workstation 5 which are available to all users during library opening hours. During the night and at weekends, however, these machines reboot into FreeBSD UNIX and join the SCAN. In addition, room 409 has a number of PC's running Windows XP which are available to postgraduates and year 4 undergraduates of the Mathematics department. Finally, the undergraduate common room 212 is home to 8 new HP PC's which dual-boot both Windows XP and RHEL WS5. These systems are accessible to all Maths users at any time.

These computers should satisfy the requirement for undergraduate computing facilities during college vacations but do let me know if these reduced general computing facilities cause you any undue hardship or inconvenience.

How powerful is the SCAN?

Percolation code written by a Maths Physics researcher and run on the Maths SCAN has broken several records that were previously set by a Cray MP3 supercomputer, completing simulations in a shorter time than this million+ dollar machine.

Future plans

There are a lot of Windows PCs sitting idle at night and at weekends in the department so we are looking at these hungrily, especially as the ongoing equipment renewal is putting increasingly more powerful computers onto people's desktops.

Older SCAN items:

March 24th, 2006: operating system upgraded to FreeBSD 6.0

March 1st, 2004: SCAN news update

July 21st, 2003: Introducing the SCAN