1. 리눅스 기본 로그 설명
$ ll /var/log
Log files from the system and various programs/services, especially login (/var/log/wtmp, which logs all logins and logouts into the system) and syslog (/var/log/messages, where all kernel and system program message are usually stored). Files in /var/log can often grow indefinitely, and may require cleaning at regular intervals. Something that is now normally managed via log rotation utilities such as 'logrotate'. This utility also allows for the automatic rotation compression, removal and mailing of log files. Logrotate can be set to handle a log file daily, weekly, monthly or when the log file gets to a certain size. Normally, logrotate runs as a daily cron job. This is a good place to start troubleshooting general technical problems.
*/var/log/messages – Contains global system messages, including the messages that are logged during system startup. There are several things that are logged in /var/log/messages including mail, cron, daemon, kern, auth, etc.
*/var/log/dmesg – Contains kernel ring buffer information. When the system boots up, it prints number of messages on the screen that displays information about the hardware devices that the kernel detects during boot process. These messages are available in kernel ring buffer and whenever the new message comes the old message gets overwritten. You can also view the content of this file using the dmesg command.
*/var/log/auth.log – Contains system authorization information, including user logins and authentication machinsm that were used.
*/var/log/boot.log – Contains information that are logged when the system boots
*/var/log/daemon.log – Contains information logged by the various background daemons that runs on the system
*/var/log/dpkg.log – Contains information that are logged when a package is installed or removed using dpkg command
*/var/log/kern.log – Contains information logged by the kernel. Helpful for you to troubleshoot a custom-built kernel.
*/var/log/lastlog – Displays the recent login information for all the users. This is not an ascii file. You should use lastlog command to view the content of this file.
*/var/log/maillog /var/log/mail.log – Contains the log information from the mail server that is running on the system. For example, sendmail logs information about all the sent items to this file
*/var/log/user.log – Contains information about all user level logs
*/var/log/Xorg.x.log – Log messages from the X
*/var/log/alternatives.log – Information by the update-alternatives are logged into this log file. On Ubuntu, update-alternatives maintains symbolic links determining default commands.
*/var/log/btmp – This file contains information about failed login attemps. Use the last command to view the btmp file. For example, “last -f /var/log/btmp | more”
*/var/log/cups – All printer and printing related log messages
*/var/log/anaconda.log – When you install Linux, all installation related messages are stored in this log file
*/var/log/yum.log – Contains information that are logged when a package is installed using yum
*/var/log/cron – Whenever cron daemon (or anacron) starts a cron job, it logs the information about the cron job in this file
*/var/log/secure – Contains information related to authentication and authorization privileges. For example, sshd logs all the messages here, including unsuccessful login.
*/var/log/wtmp or /var/log/utmp – Contains login records. Using wtmp you can find out who is logged into the system. who command uses this file to display the information.
*/var/log/faillog – Contains user failed login attemps. Use faillog command to display the content of this file. Apart from the above log files, /var/log directory may also contain the following sub-directories depending on the application that is running on your system.
*/var/log/httpd/ (or) /var/log/apache2 – Contains the apache web server access_log and error_log
*/var/log/lighttpd/ – Contains light HTTPD access_log and error_log
*/var/log/conman/ – Log files for ConMan client. conman connects remote consoles that are managed by conmand daemon.
*/var/log/mail/ – This subdirectory contains additional logs from your mail server. For example, sendmail stores the collected mail statistics in /var/log/mail/statistics file
*/var/log/prelink/ – prelink program modifies shared libraries and linked binaries to speed up the startup process. /var/log/prelink/prelink.log contains the information about the .so file that was modified by the prelink.
*/var/log/audit/ – Contains logs information stored by the Linux audit daemon (auditd).
*/var/log/setroubleshoot/ – SELinux uses setroubleshootd (SE Trouble Shoot Daemon) to notify about issues in the security context of files, and logs those information in this log file.
*/var/log/samba/ – Contains log information stored by samba, which is used to connect Windows to Linux.
*/var/log/sa/ – Contains the daily sar files that are collected by the sysstat package.
*/var/log/sssd/ – Use by system security services daemon that manage access to remote
2. kern.log 설정
RHEL(CentOS) 6.x 기준으로 dmesg 로그는 기본적으로 /var/log 디렉토리에 dmesg와 dmesg.old 로그가 남아 있게 된다.
dmesg.old 파일은 바로 직전 부팅 시점의 커널 링 버퍼 정보를 포함하고 있다. 재부팅을 하면 dmesg.old 파일은 dmesg 파일로 overwrite 된다.
따라서 그 전전 시점의 dmesg 정보는 확인을 할 수가 없다. 시스템의 문제 사항을 트러블슈팅 하기 위해서는 dmesg 정보가 필요할 수 있기 때문에 kern.log 를 활용하도록 한다.
kern.log는 기본적으로 로그가 생성되도록 설정되어 있지 않는데, 아래와 같이 로그로테이션까지 될 수 있도록 세팅하면 동작되게 된다.
1) dmesg 로깅 설정 1 line 추가
$ vi /etc/syslog.conf
kern.* /var/log/kern.log
2) dmesg 로그 로테이션 설정 1 line 추가
$ vi cat /etc/logrotate.d/syslog
/var/log/kern.log
3) rsyslog 데몬 restart
$ /etc/init.d/rsyslog restart
4) kern.log 생성 확인
$ cat /ver/log/kern.log
Jan 9 09:06:27 Kicksatrt-server kernel: Kernel logging (proc) stopped.
Jan 9 09:06:27 Kicksatrt-server kernel: imklog 5.8.10, log source = /proc/kmsg started.
-> kern.log가 생성된 것이 확인된다. kern.log는 dmesg에는 없는 timestamp 값이 있으므로 트러블 슈팅시 더 도움이 될 수 있다.
5) 로깅 테스트
터미널 창을 두개 더 띄우고 dmesg 와 kern.log 를 모니터링 한다.
♦ 터미널 창 2에서
$ tail -f /var/log/kern.log
♦ 터미널 창 3에서
$ while true; do dmesg -c; done
* dmesg -c 는 dmesg 내용을 지우므로 주의해서 사용 할 것.
-c Clear the ring buffer contents after printing.
내용을 지우고 싶지 않으면 실시간으로 확인하지 않고 dmesg를 수동으로 입력하여 확인하도록 한다.
♦ 터미널 창 1에서
$ echo "test message" > /dev/kmsg
터미널 창 1에서 위와 같이 test messgage 로그가 찍히도록 하면 dmesg와 kern.log가 동시에 찍히는 걸 확인할 수 있다.
* 참고
아래와 같이하면 /var/log/messages 로그에 임의로 로깅할 수 있다.
$ logger -t rsyslogd "test msg"
※ 참고 사항
* dmesg -c 로 지운다 해도 kern.log 의 내용은 그대로 있다. 즉 kern.log는 kernel messgae 내용을 발생 즉시 insert 만 한다.
* /var/log/dmesg 는 부팅시의 커널 메시지만 기록이 되고 시스템 운영 중의 로그는 기록이 되지 않는다.
따라서 dmesg.old 와 dmesg 파일은 분석에 큰 도움이 되지 못 될 수 있다.
* /var/log/kern.log는 시스템 운영 중 발생하는 커널 메시지를 즉각 반영하게 된다.