Using your ICNFS home directory

Deleting unwanted files and folders

You'll probably find a lot of things in your home directory that are no longer needed and can be deleted to free up space so go ahead and do it now! If you do accidentally delete something and then wish you hadn't, don't worry as ICT keep a backup of your home directory for up to 4 months and deleted files & folders can be restored if they are in the previous nightly backup. Contact the ICT Service desk if you want data restored to your ICNFS home directory.

If you have deleted a lot of stuff but still find you have used a lot of disk quota, there are two things that could be causing this:

• core files: on UNIX-like systems, when a program runs into difficulties and crashes or terminates, a 'core dump' file is created and saved in the directory you launched the program from. It will be named simply as 'core' and this file can be analysed to find out what went wrong with the program, or it could be sent to a software vendor for analysis or to aid in creating a more robust product. These files can often be quite large - several gigabytes in the case of some browsers - and as most users have no use for them, they can be safely deleted. You can find and list the type of any files named 'core' in your ICNFS home directory with the following command:

  find $SHOME -type f -name 'core' -exec file {}\;

  This will list all the files with the name 'core' in your ICNFS home directory along with its type.

  Do not be tempted to modify the command above to delete all files named core! There has long been a convention of reserving the filename 'core' for core dumps but stupidly, some recent commercial programs have ignored this and used this filename for important program files. Also, you are at liberty to name any of your own files 'core' so be careful! Only delete the files that the file utility reports as being a core file, as in the example above - if the file named 'core' is reported as being anything other than a core file, then leave it alone unless you know it can safely be deleted.

• hidden Mozilla files: Firefox and Thunderbird are programs created by the Mozilla Foundation which store their data in hidden folders in your home directory called .mozilla and .thunderbird respectively. (The period in front of the folder name hides the folder from normal directory listings and you may not even be aware of them). Unfortunately, if you use either Firefox or Thunderbird - or both - these hidden folders can grow to a huge size without you knowing! Also, even if you have switched from Firefox to using another browser such as Google Chrome, there may still be a lot of cached pages, downloads, bookmark files, etc left in the Firefox folder within your .mozilla folder. The same applies to Thunderbird which stores all of your email within the .thunderbird folder.

  To find out how much space is being used by your Mozilla Firefox folder (if you have one), type the following command:

  du -sh $HOME/.mozilla

  and you may see something like this example:

  andy@susie4:~> du -sh .mozilla/
  721M    .mozilla/

  which shows Firefox has stored 721 megabytes of data hidden within the .mozilla folder.

  You can check your Thunderbird folder in the same way:

  du -sh $HOME/.thunderbird

  and you might see something like this:

  andy@purdy32:~$ du -sh .thunderbird
  5.8G    .thunderbird

  Here, Thunderbird has stored 5.8 gigabytes of data in the .thunderbird folder.

  As you can see, the disk usage in both these examples is way above the default 400 MB quota allowance you have on your ICNFS home directory.

I have exceeded my quota and can't log in - what do I do?

If you have exceeded your ICNFS disk quota, you won't be able to log into a college Linux system using the graphical log-in screen. This is because graphical logins create or modify some hidden files in your ICNFS home directory but won't be able to do this if you have gone over quota and there is no room to add new files or change an existing file. To get around this you need to log in either using a console terminal or remotely via ssh as described below:

• console login: most Linux systems (and some other UNIX-like systems) will allow you to switch from graphical to text terminal mode simply by simultaneously pressing the 3 keys:

  ctrl-alt-F2

  This will switch the screen to a white on black terminal screen with a login prompt. You can now log in using your usual username and password and delete or move files and folders until you have brought your home directory usage down and below your ICNFS quota.

  When you have finished, you can log out and then switch back to graphical mode on most systems with:

  alt-F7

  or, on some Fedora Linux systems, you need to use:

  alt-F1

  Using ctrl-alt-F2 will work with all systems; 6 console terminals are actually available on most and these can be selected by using ctrl-alt and the function keys F1 to F6; some versions of Fedora and OpenSUSE are a bit different - with Fedora, ctrl-alt plus the keys F2 to F7 select the six text terminals while ctrl-alt-F1 will switch you back to graphical mode while OpenSUSE 13.x uses ctrl-alt-F5 to switch to graphical mode. (There are now dozens of Linux distributions each with their own quirks and too many to list here, so you may have to experiment to find out the key combinations for switching between graphical and terminal modes for your particular Linux. The combinations mentioned here will work with the popular Linuxes such as Ubuntu).

• ssh login: you can also log into most Linux and UNIX-like systems over the network using ssh and deal with disk usage issues on the command line. This is useful if you don't have access to a desktop Linux system or you are not in college; using ssh is described in detail on the ssh pages but please be aware that a ssh server is not installed on Ubuntu Linux desktop systems by default. Most Ubuntu systems in college will have this installed but if it is not installed on your own Ubuntu system, you can install and start it with:

  sudo apt-get -y install openssh-server

I need more storage space - what are my options?

If you need temporary storage for large files, you can log into one of the Maths compute cluster nodes and copy or move data from your ICNFS home directory to one or more of the scratch disks on the cluster. This should be regarded as an emergency option because the scratch disks are not backed up in any way and if a problem arises on a scratch disk, you could lose some or all of your files. Having said this, scratch disk failures are surprisingly rare and some users have kept files and folders on these for years with no problems.

A much better solution is to use the calculus storage server, one of the Maths large storage silos or the new clustor and clustor2 storage servers which not only offer much more storage space than ICNFS but do not have any quotas imposed. You can store as much data as you like on these limited only by the available storage space - about 10 terabytes each for calculus, silo1, silo2, silo3 and silo4, 30 TB (terabytes) for clustor and 60 TB on clustor2. To use either of the silo servers, you will need an account on it - simply ask for one - but you can create storage space for yourself on the calculus server while you will find you already have an empty home directory on the clustor and clustor2 servers even if it is not your default home directory on the Maths cluster.