Updated 2/2007

	        Common Sense Rules vs "The Tragedy of the Commons"


Common Sense: The internet and the servers are a shared resource.  Don't
hog the resources as you can slow down the internet for other people.  

The Tragedy of the Commons: For some, getting the forecast earlier can
make money.  For some, the effort of writing scripts that use more resources
than necessary is too high.

I am not as naive as before and "Tragedy" beats common sense.  The system
has gone from an honor system to a "big stick".  You abuse the system for 
any reason, and your IP address is turned off for some undefined time.  
It's a big stick because users often share IP addresses.  One user can
turn off the access for many others.




For people who believe in common sense, here are some rules.

(1) Always put a sleep command before checking for the 
	existance of a file again.

	If the MRF forecast has not been completed, don't check
	again without adding a sleep command.  There have been cases
	where 95+% of hits were created by one job that just 
	couldn't wait for the file.  Of course the rouge job
	was on a fast link and everybody else had to suffer
	with a slow server.
	

	Bad:
		loop:	check for forecast
			if forecast not available goto loop


	Good:
		loop:	check for forecast
			if forecast not available then
				sleep 10 minutes
				goto loop
			endif


2) Do not submit multiple jobs at one time, run them serially

	The servers do NOT have multiple CPUs.  Running 8 jobs
	(ftp2u/plots) at one time is not faster than running
	8 jobs serially.  Running 8 jobs at once just makes the
	server much slower for the interactive users.


	Bad:

			wwwgrab (...) &
			wwwgrab (...) &
			wwwgrab (...) &
			wwwgrab (...) &

	Good:

			wwwgrab (...)
			wwwgrab (...)
			wwwgrab (...)
			wwwgrab (...)


	Fact 1:
		Running running jobs sequentially can be faster
		as it minimizes disk head thrashing.

	Fact 2:
		When the number of tasks (load average) gets too large,
		jobs begin to use virtual memory which is much slower
		than physical memory.

	Fact 3: 
		When the number of tasks (load average) gets large,
		the interactive users will have a terrible time.
                They get angry and add tests to prevent people
                from overwhelming the system.