+
Integrating AbuseIPDB with Splunk© - Automatically Detect and Report Bad IPs

AbuseIPDB provides a free API for reporting and checking IP addresses. Every day webmasters, system administrators, and other IT professionals use our API to report thousands of IP addresses engaging spamming, hacking, vulnerability scanning, and other malicious activity in real time.

Splunk © is the world’s first Data-to-Everything Platform. Now organizations no longer need to worry about where their data is coming from, and they are free to focus on the business outcomes that data can deliver. Innovators in IT, Security, IoT and business operations can now get a complete view of their business in real time, turn data into business outcomes, and embrace technologies that prepare them for a data-driven future.

In this tutorial, we will learn how to set up Splunk© so that attempted intrusions against your system are automatically blocked.

AbuseIPDB + Splunk© Integration Tutorial

Pre-Requisites - Before You Start This Tutorial
Set up Splunk© data flow
Setting up the data views
Optional - Steps for creating your own custom alert to report to AbuseIPDB
Debian Note
Troubleshooting

Pre-Requisites - Before You Start This Tutorial

Create an AbuseIPDB API v2 key

Before starting this tutorial, we assume that you have an account registered with AbuseIPDB, and have verified your domain and created an API v2 key. The API v2 is free to use, but you do have to create an account.

Download and set up Splunk©

Visit the Splunk© download page and select your Operating System for the proper download link.

Download our custom app from Splunk©.

Please visit our app download page and click the green download button.

Ensure that Splunk© Web is up and running

For information on starting Splunk© for the first time and launching Splunk© Web, visit their Documentation.

Install our app from file

Once you are on Splunk© Web, navigate to Apps > Manage Apps.
Click install app from file
Click choose file, and navigate to our uncompressed app
Click Upload
You should now see our App after returning to the Splunk© Web home

* The Report to AbuseIPDB for Splunk© app was developed and tested on Splunk© version 7.3.0 *
* The following steps describe setting up the Report to AbuseIPDB for Splunk app on a CentOS 7 server. If you are using a Debian based server please refer to the DEBIAN NOTE at the bottom of this document. *

Set up Splunk© data flow

1). Configure a Splunk® forwarder to monitor the /var/log directory.

Please visit the Splunk© forwarder doumentation page for detailed instructions on installing a Splunk© forwarder.

2). Configure a Splunk® receiver to accept data from the port you used in the forwarder setup.

Please visit the Splunk© receiver doumentation page for detailed instructions on installing a Splunk© forwarder.

3). Ensure that there is a data flow

On Splunk© Web, click on the Report to AbuseIPDB for Splunk app icon listed under your apps. Ensure that there is by clicking on the Data Summary button on the Report to AbuseIPDB for Splunk dashboard.

4). Add your AbuseIPDB API v2 key to the alert_actions.conf file

Either through the terminal or your favorite text editor, open the alert_actions.conf file located in $SPLUNK_HOME/etc/apps/abuseipdb/default/. $SPLUNK_HOME can vary depending on your operating system, but for linux servers, it is /opt/splunk/.

Locate the following line:

# Enter your AbuseIPDB APIv2 key below.

Under that line, there should be:

param.key =

Paste your API key to the right of the equal sign then save the file.

Filtering out your IP

5). Ensure Splunk© does not report your IP

In the same directory, open the file savedsearches.conf.

Locate the following line under the [Messages Log Report] stanza:

search = host=YOUR_HOST_NAME "SRC=*" SRC!="YOUR_IP_HERE”

Replace YOUR_HOST_NAME with the host name of your server, and YOUR_IP_HERE with your IP.

*You can exclude other IPs you would not like reported in the same manner by adding SRC!="YOUR_IP_HERE” again to this line. This can be useful if more than one person may need to access the Splunk© Web*

Locate the following line under the [Secure Log Report] stanza:

search = host=YOUR_HOST_NAME  source="/var/log/secure" ip="*" NOT "YOUR_IP_HERE”

Replace YOUR_HOST_NAME with the host name of your server, and YOUR_IP_HERE with your IP.

*You can exclude other IPs you would not like reported in the same manner by adding NOT “YOUR_IP_HERE” or ip!= “YOUR_IP_HERE” again to this line. This can be useful if more than one person may need to access the Splunk© Web*

Setting up the data views

6). Setup the chart for the messages alert

Move to the data/models directory.

Locate the following line in Messages_Alert.json:

"search": "host=YOUR_HOSTNAME \"SRC=*\" SRC!=\"YOUR_IP_HERE\"

Replace YOUR_HOST_NAME with the host name of your server, and YOUR_IP_HERE with your IP. You can exclude other IPs you would not like reported in the same manner by adding SRC!=\"YOUR_IP_HERE\”again to this line and save the file.

7). Setup the chart for the secure alert

In the same directory, locate the following line in Secure_Alert.json:

"search": "host=YOUR_HOST_NAME  source=\"/var/log/secure\" ip=\"*\" NOT \"YOUR_IP_HERE\""

Replace YOUR_HOST_NAME with the host name of your server, and YOUR_IP_HERE with your IP. You can exclude other IPs you would not like reported in the same manner by adding NOT “YOUR_IP_HERE” or ip!= “YOUR_IP_HERE” again to this line and save the file.

8). Restart Splunk©

Navigate to the /opt/splunk/bin directory in your terminal and restart splunk by entering:

./splunk restart

9). Make sure everything is working properly

Check the AbuseIPDB dashboard and verify that there is a data flow
Check that the alerts are triggering when they are supposed to:

The secure log alert triggers anytime there is an invalid/failed login attempt recorded to the secure log file.
The messages log alert triggers anytime the kernel blocks an incoming IP with the firewall

If your messages log does not currently have any port scanning data, there is not a SRC field and therefore no results for this search. The search and alert will begin working on their own once this data is introduced.

Both of these alerts will appear in the Activity>Triggered Alerts tab of Splunk® Web

Ensure that the report script is running properly by monitoring your AbuseIPDB report page

If successful, each time a log event occurs on either /var/log/messages or /var/log/secure there should be a report logged to your account on our website.
If reports are not showing up, but your alerts are triggering, make sure you added your API Key to the alert_action.conf file

Optional - Steps for creating your own custom alert to report to AbuseIPDB

10). Determine which log file you would like to monitor

In the Report to AbuseIPDB for Splunk search bar type: source="PATH_TO_YOUR_LOG_FILE", replacing PATH_TO_YOUR_LOG_FILE with the actual path to the log file you would like to monitor and run your search.

11). Check your Search Results

Check the list of Fields located on the left of your search results. If you see an ip or SRC field continue to step 12.

If you do not see one of these two fields you will need to extract your own by following these steps:

Click extract new fields
Select a sample event by clicking a line in your search results and then clicking next on the progress bar at the top of the page.
Click I prefer to write the Regular Expression myself
Enter the following regex and click preview:

 (?\<ip>\d+\.\d+\.\d+\.\d+)

This is for IPv4. If you need IPv6 you will need to modify it.

Set permissions to app, everyone can read and admin can write
Ensure that the extraction name is ip and then click finish

12). Set up the Alert

Find and click either the ip or SRC field and then click events with this field
In the search bar following the existing search type NOT "YOUR_IP_HERE" making sure to fill in your ip address and then run the search
Click Save As then select Alert
Enter a title and description of your choosing and enter an expiration that suits your needs
Set permissions to Shared in App
Set Alert type to Real Time
Make sure that the trigger alert when setting is per-result
Under trigger actions click add actions and then click Add to triggered alerts with a severity of your choosing
Click add actions again and click the report action which has the AbuseIPDB logo with a category and comment of your choosing - please make the comment relatable to the type of attack and click save

13). Modify the Script

In the directory $SPLUNK_HOME/etc/apps/abuseipdb/bin open the file report.py so that you can edit it
Locate the following line in the file:

 url = data["configuration"]["base_url"]

Insert the following block of code directly above it:

elif data["result"]["source"] == PATH_TO_YOUR_FILE:
    logfile = A_FILE_OF_YOUR_CHOOSING_TO_RECORD_SERVER_RESPONSE
    ip = data["result"]["NAME_OF_YOUR_IP_FIELD_IN_ALERT"]

Replace your file names before saving the code

Locate the following line:

if ip !=

Place your server IP in the first set of quotation marks, and your IP in the second

./splunk restart

Debian Note

Differences for Debian based servers

Due to a different file naming system you will need to update your file paths.

Anywhere that we use /var/log/secure your will need to replace it with /var/log/auth.log

Anywhere that we use /var/log/messages you will need to replace it with /var/log/syslog

Files where you will need to make these changes are:

/opt/splunk/etc/apps/abuseipdb/default/savedsearchs.conf
/opt/splunk/etc/apps/abuseipdb/default/data/models/Messages_Alert.json
/opt/splunk/etc/apps/abuseipdb/default/data/models/Secure_Alert.json
/opt/splunk/etc/apps/abuseipdb/bin/report.py

Troubleshooting

Data Flow Issues

During testing, we noticed that following the documentation for configuring a forwarder through CLI caused a “TailReader Error”. At this point, Splunk was not receiving data either. The issue was resolved by Going to Settings>Forwarding and Receiving and deleting both the forwarder and receiver.

API Key & Parameter Mistakes

If reports aren't being automatically transmitted to your account, check the custom log file with sudo less /opt/splunk/etc/apps/abuseipdb/bin/secureReport.log. You can verify that IPs are being properly banned by your jails, and check for cURL errors that could be causing your reports to fail.

API Limits & Throttling

By default, API usage limits are capped at 1,000 reports per day. These limits are increased to 3,000 for verified webmasters and 5,000 for contributors, which is highly recommended for Splunk© users (especially if you have Splunk© running on multiple servers reporting to the same API key).

We also throttle the same IP from being reported more than once every 15 minutes in order to avoid duplicate reports. Please keep this in mind when checking if your reports are getting through.

Splunk, Splunk>, Turn Data Into Doing, Data-to-Everything, and D2E are trademarks or registered trademarks of Splunk Inc. in the United States and other countries. © 2005-2020 Splunk Inc. All rights reserved.

Thanks for supporting AbuseIPDB! Do you have any feedback or suggestions about this tutorial? Please let us know!

+ Integrating AbuseIPDB with Splunk© - Automatically Detect and Report Bad IPs