Splunk Enterprise and VMware & NetApp monitoring
Let's say your have Splunk Enterprise and VMware & NetApp monitoring, and you want to configure Splunk to gather date from VMware and NetApp, what do you do? Let's investigate the procedure on this article
The steps are as follows:
- Setup the Splunk OVA for VMware
- Redirect logs on your vCenter to the DCN, and open firewall
- Redirect logs on your ESXi hosts to the DCN, and open firewall
- Install the "Splunk Add-on for VMware" on Splunk Enterprise
- Install the "Splunk App for VMware" on Splunk Enterprise
Update Feb 2022: This article is only for references purposes. these apps have reached the end of life and Splunk does not longer distribute them
1. Setup the Splunk OVA for VMware
Download this OVA on your vSphere and start it up:
- https://splunkbase.splunk.com/app/3216/ (at the time of writing the version of OVA they have online is 3.4.1)
After power up, logon with "root" and "changemenow", then run the DCN (Data Collection Node) network configuration utility
On my example I called the DCN "Splunk_Collector", and gave it an IP of 10.10.10.33, don't forget as well to add an entry on DNS of your "Splunk_Collector" IP, it will need it
And now that you are on DNS, ensure there is an entry as well for your Enterprise Splunk server, on my case I called it "heappsvr3" with the IP 10.10.10.130
Once all this is done, run the "dcn-splunk-config" to configure Splunk by following the wizard
- For your indexers enter: 10.10.10.130:8089
- For your license master enter: https://heappsvr3
Once you got the IP address configured, you can access the VM if you like by visiting https://10.10.10.33:8000 , but DO NOT change anything yet
passwd root //** use this command to change the default root password
2. Redirect logs on your vCenter to the DCN, and open firewall
Now we need to visit our vCenter (on my example I'm running the appliance, so it is a VCSA) and also the ESXi hosts, and configure all to send their logs to the Data Collector Node
If you are running VCSA 6.0 visit System Configuration > Nodes > Related Objects and find the VMware Syslog Service, and configure this service accordingly.
On my example I'm runing VCSA 6.5, so the SysLog configuration for this version are kept under the VAMI ui https://10.10.10.144:5480
Visit also your vCenter > Configure > Advanced Settings and set both the "config.log.outputToSyslog" and "config.log.outputToSyslog" to true
After modifying this configuration, you need to restart the VCSA or its vCenter service
Once your VCSA comes back online, visit System Configuration > Nodes > Manage > Firewall and white-list both IPs for your Splunk Enterprise and your Data Collector Node
3. Redirect logs on your ESXi hosts to the DCN, and open firewall
Now we need to do the same thing with the ESXi hosts; for these guys visit the Configure tab > Advanced System Settings and configure the setting "Syslog.global.logHost" to read tcp://DCN_IP_or_DNS:1514
Do this modification on all your hosts, and after that SSH to the hosts and run this command:
esxi system syslog reload
**//run this command to check that the port of the DCN is accesible:
nc -z 10.10.10.33 514
If SSH is disable on the ESXi host, enable it by visiting Configure > Security Profile > scroll down for services, and start the SSH service
Configure also the firewalls on the ESXi hosts
Enable the TCP data inputs in the Splunk Server
4. Install the "Splunk Add-on for VMware" on Splunk Enterprise
Next, download the VMware add-on for this site:
- https://splunkbase.splunk.com/app/3215/ , which at the time of writing is version 3.4.1
Unzip the contents of the add-on VMware, and copy them to the %SPLUNK_HOME%/etc/apps on your Enterprise server, then restart the Splunk service
After you logon to the Enterprise Server, you would see the new app icon on the left hand side
Before attempting to launch it, visit Settings > Access Control > Users and add to the admin the "splunk_vmware_admin" role
After adding the role to the admin user, open the "Add-on for VMware", and it should load with no configuration whatsoever
Click the "+" symbol to create a New Collection Node, and add the details of the OVA. Notice that you also have the "VMware add-on" on the actual Splunk OVA, but don't get confused and don't configure anything there yet
Add also the details of your vCenter until you see the configuration like below, with both the DCN and vCenter added. Then click on "Start Scheduler"
At this stage, you are ready to start the installation of the VMware App in your Splunk Enterprise
Configure the ESXi to use To configure remote syslog using TCP on port 514: esxcli system syslog config set --loghost='tcp://10.11.12.13:514'
5. Install the "Splunk App for VMware" on Splunk Enterrprise
On your Splunk Enterprise (also called your Search Header, if you only have one ), visit this page and download the "Splunk App for VMware":
- (at the time of writing this document version 3.4.1) https://splunkbase.splunk.com/app/725/
There is a great video tutorial to follow for the installation:
- https://www.youtube.com/watch?v=GgJUkh0eFH4 , but we'll do this our own way
1.First unzip the contents for the VMware App
Copy the contents into the %SPLUNK_HOME%/etc/apps folder on your Enterprise server:
Then restart the Splunk service on the server by visiting Settings > Server Control > Restart. Once you log back in again you'd be able to see the VMware App
Start the VMware App by accepting the default setup
Click on the app and follow "Continue to app setup page", then accept the default vales and click "Save"
NET APP
For the configuration of the NetApp, please use this tutorial:
If you have enjoyed this article about Splunk Enterprise and VMware & NetApp monitoring you might find this other one interesting too:
- Disk Monitoring with Splunk Enterprise https://www.nazaudy.com/disk-monitoring-with-splunk-enterprise
References
- https://kb.vmware.com/s/article/2003322
- https://docs.vmware.com/en/VMware-vSphere/6.0/com.vmware.vsphere.vcsa.doc/GUID-9633A961-A5C3-4658-B099-B81E0512DC21.html
- https://www.virtuallyghetto.com/2017/02/what-logs-do-i-get-when-i-enable-syslog-in-vcsa-6-5.html
Comments powered by CComment