Agenda: exercises-log-management-swatch.txt

Network Management & Monitoring
Using Swatch

Notes:
------
* Commands preceded with "$" imply that you should execute the command as
  a general user - not as root.
* Commands preceded with "#" imply that you should be working as root.
* Commands with more specific command lines (e.g. "RTR-GW>" or "mysql>") 
  imply that you are executing commands on remote equipment, or within 
  another program.

Exercises
---------

0. Log in to your PC or open a terminal window as the root user:

        $ sudo bash

1. Let's enable logging of everything to a single file:

        # vi /etc/syslog-ng/syslog-ng.conf

        - Add this line at the end of the file:

destination everything {
  file("/var/log/everything"
    template("$DATE <$FACILITY.$PRIORITY> $HOST $MSG\n") template_escape(no)
  );
};
log { source(s_all); destination(everything); };

... this will enable logging of ALL messages to a single file, so that we
can run a monitoring script on the messages.

        - Now re-initialize Syslog:

        # service syslog-ng restart

2. Enable a daily automated script to truncate the log file so it doesn't
grow too big:

        # vi /etc/logrotate.d/everything
        
        - In the file add the following:

/var/log/everything {
  daily
  copytruncate
  rotate 1
  postrotate
        /etc/init.d/swatch restart
  endscript
}


2. Install swatch

        # apt-get install swatch

3. Create the file /etc/swatch.conf and add the following rules in the file:

        # vi /etc/swatch.conf

watchfor /PRIV_AUTH_PASS/
        mail=sysadm,subject=Enable mode entered
        threshold type=limit,count=1,seconds=3600

watchfor /CONFIG_I/
        mail=sysadm,subject=Router configuration
        threshold type=limit,count=1,seconds=3600

watchfor /LINK-3-UPDOWN/
        mail=sysadm,subject=Link state change
        threshold type=limit,count=1,seconds=3600

watchfor /SSH/
        mail=sysadm,subject=SSH connection
        threshold type=limit,count=1,seconds=3600

watchfor /ssh/
        mail=sysadm,subject=SSH connection
        threshold type=limit,count=1,seconds=3600


        Save the file and exit


4. Start swatch:

        # swatch -c /etc/swatch.conf --daemon

        Check that it is running:

        # ps ax | grep swatch

5. Log in to your router, and run some "config" commands (example below):

        # telnet 10.10.X.254            [where "X" is your router number]
        rtrX.ws.nsrc.org> enable
        Password: <password>
        rtrX.ws.nsrc.org# config terminal
        rtrX.ws.nsrc.org(config)# int FastEthernet0/0
        rtrX.ws.nsrc.org(config-int)# description Description Change for FastEthernet0/0 for Swatch
        rtrX.ws.nsrc.org(config-int)# ctrl-z
        rtrX.ws.nsrc.org# write memory
        rtrX.ws.nsrc.org# exit

6. Verify that you are receiving emails to the sysadmin user from Swatch

        $ su - sysadmn
        $ mutt -f /var/mail/sysadm