Syslog TACACS+ Rancid

ScreenHunter_02 Dec. 15 13.21

ScreenHunter_01 Dec. 15 13.21


1. Go to /etc/rc.conf
10.25.x.x -replace for your syslog server IP.
2. go to /etc/syslog.conf and comment everything  what was there originally.
Add your devices at the bottom
3. Create the log files in /var/log/
touch /var/log/RT01-MUN01.log
chmod 600 /var/log/RT01-MUN01.log
4. /etc/rc.d/syslogd restart
ps ax | grep syslogd
NMS# netstat -an | grep 514
udp4       0      0 *.1514                 *.*
udp4       0      0 *.514                  *.*
udp6       0      0 *.514                  *.*


To  archive the Syslog Logs and control the size of the logs use Newsyslog


600 -permissions to file

7-Number of days to store the archives

20M size of the archive

$W4D6 – $W4- Forth days of the week at 6Am


ScreenHunter_07 Dec. 15 14.28





Zbxlog is a standalone daemon written on Perl that receives all the syslog messages forwarded to it by syslog daemon (native syslogd doesn’t do it properly and has to be replaced by Rsyslogd) and put it into Zabbix’s mySQL DB so that you can see those messages in Zabbix WEB interface separately for every device and work with them in a more convenient way.

Zbxlog originally was written for Linux and has been adopted for FreeBSD and Zabbix v2.0.6


Original thread on Zabbix forum started by the developer with latest versions of Zbxlog can be found here:


1.  Checking Perl dependencies

Zbxlog is dependent on the following Perl modules:










First step to perform before the installation is to check those dependencies if they are installed or not.

Just issue the commands below one by one and notice if you have man pages for every module (case sensitive)

perldoc POSIX

perldoc NetAddr::IP::Util

perldoc IO::Socket6

perldoc IO::Socket::INET6

perldoc Time::Local

perldoc DBI

perldoc DBD::mysql

perldoc Data::Dumper


If not, you have to install modules that are missing by issuing cd /<path>/<module name>/ && make install clean

For FreeBSD 8.3 the paths for installation are below:

POSIX – usually installed with Perl itself, called ‘in core’









2. Making Zbxlog a service

Issue echozbxlog_enable=”YES” >> /etc/rc.conf so that Zbxlog starts automatically after every reload.


3. Copying files

Place the Zbxlog files onto FTP server and copy them into appropriate directories -> /usr/local/sbin/ && chmod 774

zbxlog.conf.example -> /usr/local/erc/zbxlog.conf.example && cp zbxlog.conf.example zbxlog.conf

zbxlog -> /usr/local/etc/rc.d/zbxlog && chmod 774 zbxlog

All the .pm modules from Zbxlog folder -> /usr/local/lib/Zbxlog/*.pm


Folder usr-local-www-zabbix2-include contains patched .php files along with patches, you can copy already patched files for Zabbix web interface or use patch utility if you prefer.

CScreenHistory.php -> /usr/local/www/zabbix2/include/classes/screens/ -> /usr/local/www/zabbix2/include/ -> /usr/local/www/zabbix2/include/

Don’t forget to back up original filese!!!

4. Configuring zbxlog.conf

Make sure that the following parameters are configured appropriately:







listen_port=1514 (Rsyslogd uses standard UDP 514 port and forwards all the messages to to UDP 1514)



db_name, db_user, and db_password should be the same as in /usr/local/etc/zabbix2/zabbix_server.conf


5. Replacing Syslogd with Rsyslogd

Disable system’s syslog server first issuing

echo syslogd_enable=”NO” >> /etc/rc.conf


Install Rsyslogd VERSION 5cd /usr/ports/sysutils/rsyslog5 && make install clean

Make it as a service echo rsyslogd_enable=”YES” >> /etc/rc.conf


Copy rsyslog.conf -> /usr/local/etc/rsyslog.conf


6. Rebooting the machine and testing the installation

Issue reboot and check that Zbxlog and Rsyslogd are up and running.

NMS# netstat -an | grep 514

udp4       0      0 *.1514                 *.*

udp4       0      0 *.514                  *.*

udp6       0      0 *.514                  *.*


NMS# ps ax|grep syslog

37947   2- S      0:29.85 /usr/local/sbin/rsyslogd -i /var/run/ –f


7. Creating Items in Zabbix WEB interface

Go to Zabbix web interface and create at least two items.

First one for syslog messages that DO NOT match any configured hosts in Zabbix web interface.

Configuration -> Hosts -> Items -> Create item in Zabbix server

ScreenHunter_04 Dec. 15 13.42


The second one has to be configured for every host configured in Zabbix web interface:

ScreenHunter_03 Dec. 15 13.42

As a result you should see the messages in Zabbix like the screenshot below.

ScreenHunter_05 Dec. 15 13.42


8. Troubleshooting Zbxlog

If there are no any messages in Zabbix it’s time to make sure that Zbxlog gets the syslog messages.

edit /usr/local/lib/Zbxlog/


my $DEBUG = 0;


my $DEBUG = 1;


Restart the service

/usr/local/etc/rc.d/zbxlog stop

/usr/local/etc/rc.d/zbxlog start


Have a look into



You should see one line per syslog message received

Read:remoteip= remote_host= buf=<118>Sep 26 15:15:49 kernel:

Sep 26 15:15:49 <syslog.err> SW04-MUN01 2258: Sep 26 15:15:49.658 mez: %LINEPROTO-5-UPDOWN:

Line protocol on Interface Vlan666, changed state to down


Don’t pay attention on

Prototype mismatch: sub Zbxlog::Controller::AF_INET6: none vs () at lib/Zbxlog/ line 23


If no line of this type appears in zbxlog.log, Zbxlog is not receiving anything.

netstat, tcpdump, iptraf are your friends:)


If you have this kind of lines but still nothing in Zabbix disable DEBUG in

edit /usr/local/lib/Zbxlog/


my $DEBUG = 0;


my $DEBUG = 1;


Restart the service

/usr/local/etc/rc.d/zbxlog stop

/usr/local/etc/rc.d/zbxlog start

Look into



In /var/log/zbxlog.log you should see messages sent to Zabbix, such as:

Zbxlog::Sender::Send item=$VAR1 = [

          ‘Zabbix server’,







          ‘from SW04-MUN01: 2261: Sep 26 15:15:49.658 mez: %LINEPROTO-5-UPDOWN:

Line protocol on Interface Vlan666, changed state to down’



Zbxlog::Sender::Send response=ZBXD.W…….{


        “info”:”Processed 1 Failed 0 Total 1 Seconds spent 0.000018″}


Check that response is success and Failed is 0. Otherwise, it means that Zabbix is rejecting this message. In this case, zabbix_server.log may contain useful error messages.


You can send a test messages from local machine using:

nc -w0 -u 514 “testing again”



RANCID is an utility (a bunch of Perl scripts) that monitors a router’s (or more generally a device’s) configuration, including software and hardware (cards, serial numbers, etc) and uses CVS (Concurrent Version System) to maintain history of changes.


RANCID has been installed in every location. Local RANCID pulls the configuration files from local Cisco devices only, i.e. routers, switches, and ASA’s. It can be accessible on every NMS server by http://<IP-address>/config/

London  http://<IP-Address/config/

New York http://<IP-Address>/config/

Munich http://<IP-Address>/config/


1. Installation & Configuration

Login as a root and issue cd /usr/ports/net-mgmt/rancid/ && make install clean

DON’T check “Use Subversion instead of CVS”

After the installation issue:

cp /usr/local/etc/rancid/rancid.conf.sample /usr/local/etc/rancid/rancid.conf

Open rancid.conf for editing and put there (search for that option in the file):


where XXX=MUNICH/LONDON/NEW_YORK – depends on location of installation.


XXX-devices – is the name of a group of devices, you will see that name in WEB interface after the whole installation is finished

ScreenHunter_06 Dec. 15 13.58

We need a separate user to run RANCID under that user’s account. Below is the listing of adduser process, use it as an example with the same username/groups/shell/etc:

NMS# adduser

Username: rancid

Full name: RANCID user

Uid (Leave empty for default):

Login group [rancid]: wheel

Login group is wheel. Invite rancid into other groups? []: www

Login class [default]:

Shell (sh csh tcsh bash rbash nologin) [sh]: nologin

Home directory [/home/rancid]:

Home directory permissions (Leave empty for default):

Use password-based authentication? [yes]:

Use an empty password? (yes/no) [no]:

Use a random password? (yes/no) [no]:

Enter password: Password

Enter password again: Password

Lock out the account after creation? [no]:

Username   : rancid

Password   : *****

Full Name  : rancid

Uid        : 1002

Class      :

Groups     : wheel www

Home       : /home/rancid

Home Mode  :

Shell      : /usr/sbin/nologin

Locked     : no

OK? (yes/no): yes

adduser: INFO: Successfully added (rancid) to the user database.

Add another user? (yes/no): no


RANCID installs under root account in the directory /usr/local/var/rancid/ so the owner has to be changed.

chown -R rancid:wheel /usr/local/var/


We have to create a file with usernames, passwords, and device IP addresses which we are going to pull the configs from. That file should be in the home directory of the user rancid.

cp /usr/local/share/rancid/cloginrc.sample /usr/home/rancid/.cloginrc

Comment or delete everything in that file and add the following:

(NEW_YORK  used as an example, TAB is a delimiter)

### NEW_YORK devices

## ASA

add user 10.x.x.x<–>backupasa

add password 10.x.x.x<——>Password<—->Password

add user 10.x.x.x<–>backupasa

add password 10.x.x.x<——>Password<—->Password


## routers & switches

add user *<—->backupconfig

add password *<>password

add method *<–>ssh


Then change the owner and permissions of that file:

chown rancid:wheel /usr/home/rancid/.cloginrc

chmod 640 /usr/home/rancid/.cloginrc


2. TACACS changes

We have to add backupconfig user with ability to issue show, dir, wr t, more, etc commands.

(see full config in /usr/local/etc/tac_plus.conf)

### Fro RANCID configs backup on routers ###

user = backupconfig


ASA has issues with some auth commands so I use a user with full access (had no time to troubleshoot that)

### Fro RANCID configs backup on ASA ###

user = backupasa


3. Test access to devices

A login ability to every device has to be tested before an addition to the list of monitored devices.

XXX-NMS# su -m rancid -c ‘/usr/local/libexec/rancid/clogin -f /usr/home/rancid/.cloginrc 10.x.x.x’

After issuing that command you should get Priv15 command line console access like XXX1-XXX01#

If not have a look .cloginrc file in /usr/home/rancid/ directory for typos or try to login with local SSH client using ssh –l backupconfig <IP> command


4. Creating CVS structure and list of devices

To create initial directory structure issue:

XXX-NMS# su -m rancid -c ‘/usr/local/libexec/rancid/rancid-cvs’

After that you will see a new folder in /usr/local/var/rancid/ called as group name in rancid.conf


Open for editing /usr/local/var/rancid/XXX-devices/router.db

Add one string per host (if you were able to login with ‘clogin’, if not add a device here with ‘down’ at the end and solve the problem with login)





and so on


5. Mailing reports about configuration files changes

Edit /etc/aliases


# For RANCID reports



where XXX=MUNICH/LONDON/NEW_YORK – depends on location of installation and group name in rancid.conf


Then issue newaliases in console to rebuild aliases DB.


6. Automation

Edit /etc/crontab

# RANCID configs backup utility


So that RANCID will run at 5:15am every day, pull all the config files, run diff and send a report if there is a difference in configs in comparison to the old ones.


7. Troubleshooting

Log files to see what’s going on with RANCID are here



Devices configuration files are here


where XXX=MUNICH/LONDON/NEW_YORK – depends on location of installation and group name in rancid.conf


To completely delete a host from list of devices do the following:

– delete the IP from router.db

– delete config files from /usr/local/var/rancid/XXX-devices/configs/ & /usr/local/var/rancid/CVS/XXX-devices/configs folders

– go to /usr/local/var/rancid/XXX-devices/configs/ and issue cvs update command

– check /usr/local/var/rancid/logs/ for any errors

– if you still see the ERROR

cvs diff: cannot find revision control file for configs/

go -> configs DIR and issue ‘cvs update‘ one more time


8. WEB interface for CVS tree

Issue cd /usr/ports/devel/viewvc && make install clean command.

DisableSVN support” option.

After the installation edit /usr/local/viewvc/viewvc.conf

Add cvs_roots = rancid: /usr/local/var/rancid/CVS


Edit Apache config file /usr/local/etc/apache22/httpd.conf

Add the alias:

Alias /config “/usr/local/viewvc/bin/cgi/”

  <Directory “/usr/local/viewvc/bin/cgi”>

    AddHandler cgi-script .cgi

    Options NONE +ExecCGI

    DirectoryIndex viewvc.cgi

    Order allow,deny

    Allow from all



Save and issue apachectl graceful

Check availability http://XXX-IP/config