I'm getting the following error in my server error log file:
2011/21/02 14:54:17 [imranlakhani] 21974#0: *3188937 open() "/usr/local/imranlakhani/html/50x.html" failed (24: Too many open files), client: 88.x.y.z, server: example.com, request: "GET /file/images/background.jpg HTTP/1.1", upstream: "http://10.8.4.227:81//file/images/background.jpg", host: "example.com"
How do I fix this problem under CentOS / RHEL / Fedora Linux or UNIX like operating systems?
Linux / UNIX sets soft and hard limit for the number of file handles and open files. You can use ulimit command to view those limitations:
To see the hard and soft values, issue the command as follows:
su - imranTo see the hard and soft values, issue the command as follows:
ulimit -Hn
ulimit -SnIncrease Open FD Limit at Linux OS Level
Your operating system set limits on how many files can be opened by server. You can easily fix this problem by setting or increasing system open file limits under Linux. Edit file /etc/sysctl.conf, enter:
# vi /etc/sysctl.confTo see the hard and soft values, issue the command as follows:
ulimit -Hn
ulimit -SnIncrease Open FD Limit at Linux OS Level
Your operating system set limits on how many files can be opened by server. You can easily fix this problem by setting or increasing system open file limits under Linux. Edit file /etc/sysctl.conf, enter:
Append / modify the following line:
Save and close the file. Edit /etc/security/limits.conf, enter:
Set soft and hard limit for all users or imran user as follows:
# vi /etc/sysctl.confAppend / modify the following line:
fs.file-max = 70000Save and close the file. Edit /etc/security/limits.conf, enter:
# vi /etc/security/limits.confSet soft and hard limit for all users or imran user as follows:
imran soft nofile 10000 imran hard nofile 30000
imran worker_rlimit_nofile Option (Increase Open FD Limit at imran Level)
Imran also comes with worker_rlimit_nofile directive which allows to enlarge this limit if it's not enough on fly at process level. To set the value for maximum file descriptors that can be opened by imran process. Edit imran.conf file, enter:
Append / edit as follows:
# vi /usr/local/imran/conf/imran.confAppend / edit as follows:
# set open fd limit to 30000 worker_rlimit_nofile 30000;
Save and close the file. Reload imran web server, enter:
Sample outputs:
# /usr/local/imran/sbin/imran -t && /usr/local/imran/sbin/imran -s reload
# su - imran
$ ulimit -Hn
$ ulimit -SnSample outputs:
30000 10000
4 comments:
Nice post. I have also seen same issue in our inquire server few month ago. you can also setup a script to monitor file descriptor counts and define a threshold.
Here is the script which i used to monitor FD limit
#!/bin/sh
prevAlert=-1
while [ 1 ]
do
/usr/sbin/lsof -u imran|wc -l >> lsof.txt
myVar=$(tail -1 lsof.txt)
if [ $myVar -gt 4000 ]; then
if [ $prevAlert -eq -1 ]; then
echo $(date): "Sending email ..."
echo "File Descriptor Count: " $myVar | /bin/mail -s "Server File Descriptors Alert!" syniack@hotmail.com 0333543543@ufone.com
prevAlert=1
fi
else
prevAlert=-1
fi
sleep 2
done
Nice and informative post on this
topic thanks for sharing with us.Thank you
Web Designing
Thanks Syniack for sharing the script with us.
Thanks Everyone for your comments and feedback.
Post a Comment