Install Squid, webmin and sent data to Splunk with CentOS 7
This article explains you how to install Squid, webmin and sent data to Splunk with CentOS 7 Linux Operating System. Follow this steps to get started:
- Install CentOS 7 and Squid
- Install Webmin
- Install the Squid App in your Splunk server
- Install Linux Splunk Universal Forwarder
- Deploy the Splunk client for data input
1.- Install CentOS 7 and Squid
At the time of installing your Linux box, I personally prefer to install always the "Server GUI" version, not ideal for highly secure environment but easier to troubleshoot if you (like me) are not a Linux-guru (yet)
Right after you install it, start a SSH connection to your server and issue these commands:
yum -y update
yum -y install epel-release
yum -y install squid
yum -y update
yum clean all
Enable Squid to start at system boot:
systemctl enable squid
You can use the following command to control the Squid program
systemctl start squid
systemctl status squid
systemctl restart squid
systemctl stop squid
squid -v //**help options
squid -h //**view the version number
The main important location for Squid are:
- /var/log/squid/access.log ;the log of the proxy connections that we later on need to forward to Splunk
- /etc/squid/squid.conf ;the configuration file for the ACL lists allowed to use the Squid Proxy
To manage Squid through Webmin, first visit the webmin download page here (http://www.webmin.com/download.html), and then copy the shortcut for the RMP package to your SSH connection of your CentOS, and install it as follows:
wget http://prdownloads.sourceforge.net/webadmin/webmin-1.831-1.noarch.rpm
Then issue "ls" to ensure you are in the directly where the RPM has been downloaded
//**verify your location first
yum install webmin-1.831-1.noarch.rpm
For CentOS 7
To install webmin on CentOS 7, do as follows:
sudo vi /etc/yum.repos.d/webmin.repo
#enter the below in the above file:
[Webmin]
name=Webmin Distribution Neutral
#baseurl=http://download.webmin.com/download/yum
mirrorlist=http://download.webmin.com/download/yum/mirrorlist
enabled=1
wget http://www.webmin.com/jcameron-key.asc
sudo rpm --import jcameron-key.asc
sudo yum install webmin
After installation, visit
Configure the firewall
Before visiting the webmin page, we need to open the default port that it uses (10,000) on the firewall. To do that, perform as follows:
sudo firewall-cmd --set-default-zone=trusted
firewall-cmd --get-default-zone
sudo firewall-cmd --zone=trusted --add-port=10000/tcp --permanent
sudo firewall-cmd --reload
After that, visit the website https://192.168.0.127:10000 or whichever IP address you have on your CentOS
- Username: root
- Password: [your current root password]
Once Webmin opens, visit Servers > Squid Proxy Server and click to Initialize Cache:
After the cache has started, click on the "Access Control" button to start with your configuration of Squid:
Or you may find it easier to edit the /etc/squid/squid.conf file and add the ACL entries there
After this installation, we won't need the graphical interface anymore, therefore issue this command so that CentOS does not load it:
systemctl set-default multi-user.target
3.-Install the Squid App in your Splunk server
Go to your Splunk Enterprise server and download and install the "Splunk App for Splunk Enterprise"
Note, DO NOT INSTALL THIS ONE!!! === https://splunkbase.splunk.com/app/2965/
THIS ONE (great credits to Patrick Nordien from here) :) https://splunkbase.splunk.com/app/453/
If you haven't done so, setup a receiver on port 9997 (or any other one that is free) by adding this into the C:\Program Files\Splunk\etc\system\local\inputs.conf file
[splunktcp://9997]
connection_host = ip
After the addition RESTART SPLUNK by visiting website > System > Server Control > Restart Splunk
4.-Install Linux Splunk Universal Forwarder
For the client Splunk Universal Forwarder, we would actually have to open Firefox on the CentOS 7 machine, logon to your Splunik account and visit this link:
From where you would download the RPM package and save it in the downloads folder
To install it, we better use yum, as always, go to the location where you downloaded the file and install it
yum -y install splunkforwarder-7.0.2-03bbabbd5c0f-linux-2.6-x86_64.rpm
- If you don't have the GUI installed, you can copy the file using WinSCP
- You can also use the wget command to download the installation: wget -O splunkforwarder-7.3.0-657388c7a488-linux-2.6-x86_64.rpm 'https://www.splunk.com/bin/splunk/DownloadActivityServlet?architecture=x86_64&platform=linux&version=7.3.0&product=universalforwarder&filename=splunkforwarder-7.3.0-657388c7a488-linux-2.6-x86_64.rpm&wget=true'
Visit /opt/splunkforwarder/bin/ and run the following to agree to the license (you can also do "./splunk start --accept-license")
./splunk
Then add the forwarder server, specifying the IP address of you Splunk Enterprise server and the port you configured on the inputs.conf file of the Splunk server
./splunk add forward-server 192.168.0.214:9997
./splunk add monitor /var/log/squid/access.log
./splunk add monitor /var/log/squid/
./splunk restart
./splunk list forward-server
./splunk list monitor
./splunk enable boot-start //**this ensures Splunk runs at boot up
Ensure that port is open on the CentOS firewall (verify the correct zone is selected!)
firewall-cmd --get-default-zone
firewall-cmd --zone=trusted --add-port=9997/tcp --permanent
firewall-cmd --reload
firewall-cmd --runtime-to-permanet
firewall-cmd --list-all-zones //**verify that on your ACTIVE zone port 9997 is allowed
firewall-cmd --list-ports //**similar as above, will show you if 9997 is allowed
To verify that all configuration has been done correctly, visit the following files and DOUBLE and TRIPLE check that you have entered the right formatting:
inputs.conf
outputs.conf
If all goes well, you should be able to see that you Splunk Enterprise server show as active when you run the command below; if it shows as inactive make sure that you have configure your Splunk to receive data on port 9997 by visiting its web on the location Settings > Forwarding and Receiving > Receiving data
5.-Deploy the Splunk client for data input
Visit again /opt/splunkforwarder/bin/ and run the following:
./splunk set deploy-poll 192.168.0.214:8089
//**If you are asked to logon -session invalid- use the following default settings:
Username: admin
Password: changeme
This will create a new file called "deploymentclient.conf" under /opt/splunkforwarder/etc/system/local/ ;that file will be use to generate data input on your Splunk server. Now RESTART your Splunk Enterprise and visit Settings > Data Inputs > Forwarder inputs > Files and Directories and click on "New", if all goes well you would be able to see the deployment installation of your CentOS machine
On the "Select Source" section, type carefully the location of the Squid access log
Ensure that you search type of "squid" (added by the Squid App) and create a new index called "squid" to store the logs, then review one last time before submitting the new addition:
If all goes well, you should be able to see straight away some data on the Squid App dashboard, well done!
Finally, to change the logo of the Squid app, you can get the icon from here:
and customize for the App following these instructions:
where one of the key points is to add this stanza to the app.conf file in the /$SPLUNK_HOME/etc/apps/SplunkForSquid/static/
[install]
build = 2
Looks cool!
Troubleshooting
If when opening the Squid App you are presented with the error "Eventtype 'wineventlog-dns' does not exist or is disabled"; just go to Settings > Knowledge > Event Types and create it for the Squid App
Like this:
If you enjoyed reading this article about how to install Squid, webmin and sent data to Splunk with CentOS 7, you might like this other one too:
- Use rConfig to manage your Cisco switches and Cisco useful tips: https://www.nazaudy.com/use-rconfig-to-manage-your-cisco-switches-and-cisco-useful-tips
London 24 February 2018
References
A million thanks to Liptan Biswas for this great tutorial: https://hostpresto.com/community/tutorials/how-to-install-and-configure-squid-proxy-on-centos-7/
Nice job Shanker admin: https://www.thegeekdiary.com/5-useful-examples-of-firewall-cmd-command/
Useful resources from Splunk:
- http://docs.splunk.com/Documentation/Forwarder/7.0.2/Forwarder/Enableareceiver
- http://docs.splunk.com/Documentation/Forwarder/7.0.2/Forwarder/HowtoforwarddatatoSplunkEnterprise
And better useful resources from the forums (tried and tested):
Comments powered by CComment