Saturday, August 08, 2009

LSF issues: RAM and swap

It were interesting workdays last week due to a lot of users began submit jobs to LSF. It was my first week after annual vacation :) And it was my first little experience of troubleshoting LSF issues.
One user submited calculation of big model in DYNAMO and sent to down two nodes of LSF with message "Out of memory". It were IBM x3550 with 32GB RAM and 8 GB of swap. Support suggested us to increase the swap memory and to try after that. I increased swap memory to 32 Gb by adding into swap files from filesystem () and we tested with user. He submited his job which consumed all available RAM and more than 20 Gb of swap.
But it successfully finished!
Other user could not finish his model after 4 or 5 hours of processing. And I did not suspect that problem was in unavailablee free space on his disk partition.
So I have got two things - at first check possible lack of RAM and swap and lack of disk space.

No comments: