The application servers I recently discussed are available to any account holder, without any restrictions: log in, use any program, and consume as many resources as you want. We also run some private servers, restricted to certain subsets of users, which also have no usage limit. Even though this works fairly well in practice, with everyone playing nice and getting their work done, we sometimes encounter situations where a single person is in a hurry to get something finished, and tries to do too much. Overuse by one can cause performance degradation for everyone else and may even crash the server. By knowing what to watch for you can help avoid such a situation.
For computational workloads (and everything else, really) there are two different scarce resources on a computer--processor and memory. Running a typical task on a computer, such as your email program, doesn't use the processor continuously--it is in short, small bursts. As a result, many small tasks can share a processor and get work done.
In the case of large computational tasks, the processor is kept more or less continuously busy, and it has no time for any additional tasks. Since modern computers have multiple logical processors ("cores",) they can easily run a computational task and many other things at the same time. But if you try to run more large tasks than there are cores available, the computer will become unresponsive and difficult to use for the duration of those tasks. Any new tasks will be queued up, waiting for the active ones to finish.
In the case of memory, the behavior is slightly different. If active tasks occupy all available memory on a computer, new ones are not queued up, but end up using "virtual memory," which is on the computer's hard drives. The problem is that virtual memory is much slower than real memory, and any tasks relegated there will run at a fraction of the speed that they would otherwise. In addition, if there is enough pressure on the physical memory of the computer, it might become entirely unresponsive, or crash outright.
To avoid overwhelming our servers we ask you to limit your work to two simultaneous jobs per server. This policy applies to jobs that take several hours to complete. You may run many jobs that require a short time within reason--just don't queue up thousands of jobs at a time.
Since it is very difficult to monitor memory usage, we simply expect you to monitor your own jobs and avoid consuming more than half the memory on a given server (less if there are many others also running jobs).
Note that these rules are not strictly enforced as we can't watch the servers all the time looking for violators. If you notice anyone using too many resources and it is preventing you from doing work, notify us by email at email@example.com.April 18th, 2012
CLA Service Desk