After much testing we discovered the root of our problem. It appears that a couple of Win2003 kernel i/o parameters that apply to the symmpi driver supplied from LSI where throttled to the max. The LSI driver installer sets these values to the max.
The parameters are located under:
[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\Symmpi\Parameters\Device]
The first parameter is "MaximumSGList" which defines the maximum I/O block size. The symmpi driver supports block sizes of upto 1MB on 32bit Win2003. This was set to the maximum value by the installer.
The other parameter was the "NumberOfRequests" which defines the maximum number of concurrent I/Os (Guaranteed) by Win2003. Windows Server 2003 guarantees a maximum of 32 concurrent I/Os active on a
particular SCSI bus. This value was set to 255 by the LSI installer. Needless to say this value was much too high for our environment.
After a few tests we moderated both values respectively to 256KB and 100 concurrent I/Os. I am wandering whether we should have kept the block size at 1MB but that is bridge we can cross in the next tests.