Splunk© + AbuseIPDB
Integrating AbuseIPDB with Splunk© - Automatically Detect and Report Bad IPs

*NEW* The AbuseIPDB Splunk app has been recently updated to a different app, AbuseIPDB App, which can be found on Splunkbase. This new release is compatible with Splunk© Enterprise versions 9.0 and 9.1. For users of the previously existing Report to AbuseIPDB app, our new app will will allow you to perform the same reporting functionality, as well as an added integration with the AbuseIPDB APIv2's check and reports endpoints. The following documentation and setup tutorial will be intended for the newly-created AbuseIPDB app on Splunk©, specifically AbuseIPDB App V2.0.0.

AbuseIPDB provides a free API for reporting and checking IP addresses. Every day webmasters, system administrators, and other IT professionals use our API to report thousands of IP addresses engaging spamming, hacking, vulnerability scanning, and other malicious activity in real time.

Splunk © is the world’s first Data-to-Everything Platform. Now organizations no longer need to worry about where their data is coming from, and they are free to focus on the business outcomes that data can deliver. Innovators in IT, Security, IoT and business operations can now get a complete view of their business in real time, turn data into business outcomes, and embrace technologies that prepare them for a data-driven future.

In this tutorial, we will learn how to set up Splunk© so that attempted intrusions against your system are automatically blocked. Additionally, we will discuss the check and reports endpoint custom search commands.


Pre-Requisites - Before You Start This Tutorial

Create an AbuseIPDB API v2 key

Before starting this tutorial, we assume that you have an account registered with AbuseIPDB, and have verified your domain and created an API v2 key. The API v2 is free to use, but you do have to create an account.

Download and set up Splunk©

Visit the Splunk© download page and select your Operating System for the proper download link.

Download our custom app from Splunk©.

Please visit our app download page and click the green download button.

Ensure that Splunk© Web is up and running

For information on starting Splunk© for the first time and launching Splunk© Web, visit their Documentation.

Install our app from file

  • Once you are on Splunk© Web, navigate to Apps > Manage Apps.
  • Click install app from file
  • Click choose file, and navigate to our uncompressed app
  • Click Upload
  • You should now see our App after returning to the Splunk© Web home


* The AbuseIPDB App on Splunk© was developed and tested on Splunk© version 9.1.0 *
* The following steps describe setting up the Report to AbuseIPDB for Splunk app on a CentOS 7 server. If you are using a Debian based server please refer to the DEBIAN NOTE at the bottom of this document. *

The AbuseIPDB App integrates with the following AbuseIPDB APIv2 endpoints:

App Setup

As of AbuseIPDB App V2.0.0, you will no longer put your API Key directly into a file. Instead, you now have to input your key on the app's setup page.
To input your key and fully set up the app, you must open the app in your Splunk GUI, at which point you will encounter the setup page and enter your key. Once your key is entered, you are ready to use the app.

If you wish to change the API Key you have saved, please see the AbuseIPDB App Setup tab and go through the process again.

*Note*: The report functionality will require extra setup, explained below. The check and reports functionality will not require setup.


Automatic Reporting Tutorial (report endpoint):

Set up Splunk© data flow

1). Configure a Splunk® forwarder to monitor the /var/log directory.

Please visit the Splunk© forwarder doumentation page for detailed instructions on installing a Splunk© forwarder.

2). Configure a Splunk® receiver to accept data from the port you used in the forwarder setup.

Please visit the Splunk© receiver doumentation page for detailed instructions on installing a Splunk© forwarder.

3). Ensure that there is a data flow

On Splunk© Web, click on the Report to AbuseIPDB for Splunk app icon listed under your apps. Ensure that there is by clicking on the Data Summary button on the Report to AbuseIPDB for Splunk dashboard.

Filtering out your IP:

4). Ensure Splunk© does not report your IP

In the directory $SPLUNK_HOME/etc/apps/abuseipdb_app/default, open the file savedsearches.conf .

Locate the following line under the [Messages Log Report] stanza:

search = host=YOUR_HOST_NAME "SRC=*" SRC!="YOUR_IP_HERE" source="/var/log/messages"

Replace YOUR_HOST_NAME with the host name of your server, and YOUR_IP_HERE with your IP.

*You can exclude other IPs you would not like reported in the same manner by adding SRC!="YOUR_IP_HERE” again to this line. This can be useful if more than one person may need to access the Splunk© Web*

Locate the following line under the [Secure Log Report] stanza:

search = host=YOUR_HOST_NAME  source="/var/log/secure" ip!="YOUR_IP_HERE" ip!="0.0.0.0" | rex [long IP regex...] 

Replace YOUR_HOST_NAME with the host name of your server, and YOUR_IP_HERE with your IP.

*You can exclude other IPs you would not like reported in the same manner by adding NOT “YOUR_IP_HERE” or ip!= “YOUR_IP_HERE” again to this line. This can be useful if more than one person may need to access the Splunk© Web*

5). Restart Splunk©

Navigate to the /opt/splunk/bin directory in your terminal and restart splunk by entering:

./splunk restart

6). Make sure everything is working properly

  • Check the AbuseIPDB dashboard and verify that there is a data flow
  • Check that the alerts are triggering when they are supposed to:
    • The secure log alert triggers anytime there is an invalid/failed login attempt recorded to the secure log file.
    • The messages log alert triggers anytime the kernel blocks an incoming IP with the firewall
      • If your messages log does not currently have any port scanning data, there is not a SRC field and therefore no results for this search. The search and alert will begin working on their own once this data is introduced.
    • Both of these alerts will appear in the Activity>Triggered Alerts tab of Splunk® Web
  • Ensure that the report script is running properly by monitoring your AbuseIPDB report page:
    • If successful, each time a log event occurs on either /var/log/messages or /var/log/secure there should be a report logged to your account on our website.
    • If reports are not showing up, but your alerts are triggering, make sure you added your API Key to the alert_action.conf file

Optional - Steps for creating your own custom alert to report to AbuseIPDB

7). Determine which log file you would like to monitor

In the Report to AbuseIPDB for Splunk search bar type: source="PATH_TO_YOUR_LOG_FILE", replacing PATH_TO_YOUR_LOG_FILE with the actual path to the log file you would like to monitor and run your search.

8). Check your Search Results

Check the list of Fields located on the left of your search results. If you see an ip or SRC field continue to step 9.

If you do not see one of these two fields you will need to extract your own by following these steps:

  • Click extract new fields
  • Select a sample event by clicking a line in your search results and then clicking next on the progress bar at the top of the page.
  • Click I prefer to write the Regular Expression myself
  • Enter the following regex and click preview:
  •  (?\<ip>\d+\.\d+\.\d+\.\d+)
    • This is for IPv4. If you need IPv6 you will need to modify it.
  • Set permissions to app, everyone can read and admin can write
  • Ensure that the extraction name is ip and then click finish

9). Set up the Alert

  • Find and click either the ip or SRC field and then click events with this field
  • In the search bar following the existing search type NOT "YOUR_IP_HERE" making sure to fill in your ip address and then run the search
  • Click Save As then select Alert
  • Enter a title and description of your choosing and enter an expiration that suits your needs
  • Set permissions to Shared in App
  • Set Alert type to Real Time
  • Make sure that the trigger alert when setting is per-result
  • Under trigger actions click add actions and then click Add to triggered alerts with a severity of your choosing
  • Click add actions again and click the report action which has the AbuseIPDB logo with a category and comment of your choosing - please make the comment relatable to the type of attack and click save

10). Modify the Script

  • In the directory $SPLUNK_HOME/etc/apps/abuseipdb_app/bin open the file report.py so that you can edit it
  • Locate the following line in the file:
  •  config = json.load('../default/config.json')
    • Insert the following block of code directly above it:
    •                         elif data["result"]["source"] == PATH_TO_YOUR_FILE:
                                  logfile = A_FILE_OF_YOUR_CHOOSING_TO_RECORD_SERVER_RESPONSE
                                  ip = data["result"]["NAME_OF_YOUR_IP_FIELD_IN_ALERT"]
    • Replace your file names before saving the code
  • Locate the following line:
  • if ip !=
    • Place your server IP in the first set of quotation marks, and your IP in the second

11). Restart Splunk©

Restart Splunk© by going to $SPLUNK_HOME/bin and running

./splunk restart

Debian Note

Differences for Debian based servers

Due to a different file naming system you will need to update your file paths.

Anywhere that we use /var/log/secure your will need to replace it with /var/log/auth.log

Anywhere that we use /var/log/messages you will need to replace it with /var/log/syslog

Files where you will need to make these changes are:

  • /opt/splunk/etc/apps/abuseipdb_app/default/savedsearches.conf
  • /opt/splunk/etc/apps/abuseipdb_app/bin/report.py

Troubleshooting

Data Flow Issues

During testing, we noticed that following the documentation for configuring a forwarder through CLI caused a “TailReader Error”. At this point, Splunk was not receiving data either. The issue was resolved by Going to Settings>Forwarding and Receiving and deleting both the forwarder and receiver.

API Key & Parameter Mistakes

If reports aren't being automatically transmitted to your account, check the custom log file with sudo less /opt/splunk/etc/apps/abuseipdb/bin/secureReport.log. You can verify that IPs are being properly banned by your jails, and check for cURL errors that could be causing your reports to fail.

API Limits & Throttling

By default, API usage limits are capped at 1,000 reports per day. These limits are increased to 3,000 for verified webmasters and 5,000 for contributors, which is highly recommended for Splunk© users (especially if you have Splunk© running on multiple servers reporting to the same API key).

We also throttle the same IP from being reported more than once every 15 minutes in order to avoid duplicate reports. Please keep this in mind when checking if your reports are getting through.


Check Search Command Tutorial (check endpoint):

The check functionality of the AbuseIPDB is a custom search command that allows you to pass in IP addresses from a search event, and get custom AbuseIPDB fields added to your search event after a request to the API.

Once the app is installed and the API key is setup, the check search command will be available in your Splunk GUI.

The syntax of the command is:

| check ip=(fieldname or ip_address) age=number verbose=number

Parameters Explained

  • ip: Either a Splunk field containing IPs to check, or an IP address directly.
  • age: (Optional) An integer between 1 and 365 days for the time range to check. Default is 30 days.
  • verbose: (Optional) An integer value to set verbose flag, which will include reports in response. By default flag is not set.

Sample Usage

The following example will demonstrate pulling IPs from a log file, passing them through the check command, and viewing the results.

Run the search:

source="/var/log/secure" | rex (?<ip>\d+\.\d+\.\d+\.\d+) | check ip=ip | table ip, abuseConfidenceScore

which will output a Splunk table showing the ip address and abuseConfidenceScore of each ip extracted.

In this above example, we are using a regex to pull out an ip field, which is passed to the check command. The fieldname can be any name, once the field contains ip addresses to be checked.

Fields may appear blank after the check command is run. This indicates that there was an issue checking that individual IP, such as a bad or blank IP. In this case, you may click on that event to learn more.

Fields Added After Check Command

After IPs are sent to the API with the check search command, the response adds fields to directly to the event for each IP. The fields added are based on the body of the response from the check request, so you will have access to the same fields you would making a request directly to the API. These fields can be passed to another search command, like in the above example using the table command. This can be useful for taking information from the check results, like the abuseConfidenceScore, and using it to make decisions in your application.

The following fields are added to search event when the check command is used:

abuseConfidenceScore ipAddress isPublic ipVersion isWhitelisted countryCode usageType isp domain hostnames isTor totalReports numDistinctUsers lastReportedAt

An error and message are present with each request indicating any issues with request.

*NOTE* Each individual IP address passed to the check command will result in an individual call to the API. This means if a field of 1000 IPs is passed to the check command, 1000 API calls will be made. When using the check search on sets of IPs, please remember the daily limits on API usage. You can reference our limits here.


Reports Search Command Tutorial (reports endpoint):

The reports functionality of the AbuseIPDB is a custom search command that allows you to pass in IP addresses from a search event, and get custom AbuseIPDB fields added to your search event after a request to the API.

Once the app is installed and the API key is setup, the reports search command will be available in your Splunk GUI.

The syntax of the command is:

| reports ip=(fieldname or ip_address) age=number page=number perPage=number

Parameters Explained

  • ip: Either a Splunk field containing IPs to get reports on, or an IP address directly.
  • age: (Optional) An integer between 1 and 365 days for the time range to check. Default is 30 days.
  • page: (Optional) An integer indicating what from the result set to grab if more than 1 page.
  • perPage: (Optional) An integer between 1 and 100 indicating reports per page. Default is 25.

Sample Usage

The following example will demonstrate pulling IPs from a log file, passing them through the check command, and viewing the results.

Run the search:

 | makeresults | reports ip="127.0.0.1"

which will output a Splunk table showing the ip address and abuseConfidenceScore of each ip extracted.

In this above example, we are using a regex to pull out an ip field, which is passed to the check command. The fieldname can be any name, once the field contains ip addresses to be checked.

Fields may appear blank after the check command is run. This indicates that there was an issue checking that individual IP, such as a bad or blank IP. In this case, you may click on that event to learn more.

Fields Added After Check Command

After IPs are sent to the API with the reports search command, the response adds fields to directly to the event for each IP. The fields added are based on the body of the response from the reports request, so you will have access to the same fields you would making a request directly to the API. These fields can be passed to another search command, like in the above example using the table command. This can be useful for taking information from the check results, like the abuseConfidenceScore, and using it to make decisions in your application.

The following fields are added to search event when the check command is used:

total page count perPage lastPage nextPageUrl previousPageUrl results

Additionally, each result is broken up and given its own field result1,result2, etc. for readability.

An error and message are present with each request indicating any issues with request.

*NOTE* Each individual IP address passed to the reports command will result in an individual call to the API. This means if a field of 1000 IPs is passed to the reports command, 1000 API calls will be made. When using the reports search on sets of IPs, please remember the daily limits on API usage. You can reference our limits here.


Splunk, Splunk>, Turn Data Into Doing, Data-to-Everything, and D2E are trademarks or registered trademarks of Splunk Inc. in the United States and other countries. © 2005-2020 Splunk Inc. All rights reserved.

Thanks for supporting AbuseIPDB! Do you have any feedback or suggestions about this tutorial? Please let us know!