High Performance Computing (HPC) is often used by scientist who have to run really big computations. Think folding proteins of analyzing astrophysical data. If you can split up your calculation in parallel or independent parts, then you should consider things like clustering or grid computing.
In general, there are four technologies to do parallel computation:
Clusters make use of network connections to utilize the processing power of multiple computers at the same time. Clusters assume low-latency, high-bandwith LAN networks. Some of the biggest supercomputers are clusters. A grid is kinda "cluster of clusters" and typically also works over WAN links. GPUs are designed to execute parallel computation, so they are often used as well.
For an overview of the most common grid software, please download my report:
For a student assignment, I created a simple .NET program that makes use of the Alchemi framework. It takes a MRI scan and does a few operations on the slides in parallel. You can download the program here (source code).