Fail2Ban IDS +
Integrating AbuseIPDB with Fail2Ban - Automatically Report Bad IPs

AbuseIPDB provides a free API for reporting and checking IP addresses. Every day webmasters, system administrators, and other IT professionals use our API to report thousands of IP addresses engaging spamming, hacking, vulnerability scanning, and other malicious activity in real time.

Fail2Ban is open source software that scans log files like /var/log/auth.log and bans IP addresses having too many failed login attempts. It does this by updating system firewall rules to reject new connections from those IP addresses, for a configurable amount of time. It can also detect and ban IPs engaged in attempted web exploits, portscanning, and other abusive activity.

In this tutorial, we will learn how to integrate Fail2Ban with your AbuseIPDB account so that attempted intrusions against your system are automatically reported via the AbuseIPDB API, helping to contribute to AbuseIPDB's distributed IP threat database. If you already have the latest version of Fail2Ban installed, this should only take a few minutes!

Pre-Requisites - Before You Start This Tutorial

1.) Install Fail2Ban on your server

Before starting this tutorial, we assume that you have a Linux server with the Fail2Ban intrusion detection system installed. Fail2Ban is free software available at https://github.com/fail2ban/fail2ban . Please see the Fail2Ban documentation or DigitalOcean's Fail2Ban installation tutorial to install Fail2Ban and set up your basic jail configuration to detect intrusion attempts on SSH, Apache, etc.

2.) Create an AbuseIPDB API key

We also assume that you have an account registered with AbuseIPDB, and have verified your domain and created an API key. The API is free to use, but you do have to create an account.

Verify Fail2Ban AbuseIPDB Reporting Action Is Installed

The ability to report abusive IPs directly to AbuseIPDB was added to the master Fail2Ban repository in v0.10.0 (January 2017). If you have an older version of Fail2Ban installed on your server, you'll either have to update Fail2Ban or install the abuseipdb.conf action file yourself. To check what version of Fail2Ban you have installed, run the following command:

fail2ban-client -V

You can verify that your installation of Fail2Ban supports AbuseIPDB by checking that the action config file /etc/fail2ban/action.d/abuseipdb.conf exists. If it does not exist, you can add it manually by copying the latest config file from the Fail2Ban Github .

Your /etc/fail2ban/jail.local file (the customizable version of jail.conf) should also contain the following definition for "action_abuseipdb":

# Report ban via abuseipdb.com.
#
# See action.d/abuseipdb.conf for usage example and details.
#
action_abuseipdb = abuseipdb

If this action definition code doesn't exist, add it to your jail.local before the line action = %(action_)s. That's all you need to support the AbuseIPDB Fail2Ban action!

What will the AbuseIPDB action do?

The AbuseIPDB action will cause the IP address that triggered a Fail2Ban jail to be automatically reported to the AbuseIPDB API via a cURL command. By default this report will include the the log file snippet that triggered Fail2Ban's ban in the your report's comment field. The comment can be modified or disabled by looking for the snippet 'comment=<matches>' in action.d/abuseipdb.conf as follows:

actionban = curl --tlsv1.0 --fail 'https://api.abuseipdb.com/api/v2/report' \
    -H 'Accept: application/json' \
    -H 'Key: <abuseipdb_apikey>' \
    --data-urlencode 'ip=<ip>' \
    --data-urlencode 'comment=<matches>' \
    --data 'categories=<abuseipdb_category>'

Activate AbuseIPDB Reporting Action

You can invoke the AbuseIPDB action from some or all of the jails configured in jail.local. The action must be called with two parameters - your AbuseIPDB API key, and the abuse category (or categories) you would like to report the IP for. If these parameters are missing or invalid, your reports will fail.

%(action_abuseipdb)s[abuseipdb_apikey="my-api-key", abuseipdb_category="18,22"]

This line of code must be added to each jail for which you want to activate AbuseIPDB reporting. Here's an example of how you would configure the AbuseIPDB report action to run, in addition to your default ban actions, when the sshd brute force jail is triggered:

[sshd]
enabled = true

# To use more aggressive sshd modes set filter parameter "mode" in jail.local:
# normal (default), ddos, extra or aggressive (combines all).
# See "tests/files/logs/sshd" or "filter.d/sshd.conf" for usage example and details.
#mode   = normal
port    = ssh
logpath = %(sshd_log)s
backend = %(sshd_backend)s

# Ban IP and report to AbuseIPDB for SSH Brute-Forcing
action = %(action_)s
         %(action_abuseipdb)s[abuseipdb_apikey="my-api-key", abuseipdb_category="18,22"]

We recommend you add the AbuseIPDB action individually to your jails so you can customize the AbuseIPDB report categories and make your reports more specific. However, the AbuseIPDB action can also be added to the list of Fail2Ban actions in the global [DEFAULT] as shown below, which will cause it to run on all jails without an action specified:

# Choose default action.  To change, just override value of 'action' with the
# interpolation to the chosen action shortcut (e.g.  action_mw, action_mwl, etc) in jail.local
# globally (section [DEFAULT]) or per specific section
action = %(action_)s
         %(action_abuseipdb)s[abuseipdb_apikey="my-api-key", abuseipdb_category="18"]

Here's a table of some of the most popular AbuseIPDB report categories for customizing your reports:

Once you have updated your jail.local configuration, save the file and restart or reload the Fail2Ban service to ensure your configuration is working:

fail2ban-client reload

If your configuration is correct, Fail2Ban should start running the AbuseIPDB action each time a new IP is banned. Log in and check your reported IPs page, and watch as Fail2Ban starts automatically reporting IPs to AbuseIPDB under your account!

Troubleshooting

If reports aren't being automatically transmitted to your account, check the Fail2Ban log file with sudo less /var/log/fail2ban.log. You can verify that IPs are being properly banned by your jails, and check for cURL errors that could be causing your reports to fail.

API Key & Parameter Mistakes:

Issues with your API submission such as an invalid API key or invalid categories may not show up as an error in the Fail2Ban log file. You should verify that your parameters are correct by checking the API documentation, and running a test report via the API. Test reports can be deleted on your reported IPs page.

API Limits & Throttling:

By default, API usage limits are capped at 1,000 reports per day. These limits are increased to 3,000 for verified webmasters and 5,000 for contributors, which is highly recommended for Fail2Ban users (especially if you have Fail2Ban running on multiple servers reporting to the same API key).

We also throttle the same IP from being reported more than once every 15 minutes in order to avoid duplicate reports. We recommend you increase your bantime configuration to keep these bad IPs from repeatedly attacking your system. A value of 900 (15 minutes in seconds) will prevent you from hitting our 15-minute window.

* Note: if you plan to set the findtime configuration option for a jail, please do not go above 60 days (5,184,000 seconds). * This abides by our Reporting Policy.

Duplicate Reports on Restart:

Whenever Fail2Ban restarts, it calls the actionban function for each IP stored in the database file. This causes duplicate reports to AbuseIPDB. If you restart your server often, we have a script that will prevent this from happening.

Follow the steps below to modify your configuration to use the custom script:

1. Create the file abuseipdb-fail2ban-report.sh

Enter touch abuseipdb-fail2ban-report.sh into your terminal.

2. Copy the following code into the created file, updating the path to the REPORTED_IP_LIST_FILE if necessary

#!/bin/bash

REPORTED_IP_LIST_FILE=/home/pi/abuseipdb-reported-ip-list
FAIL2BAN_SQLITE_DB=/var/lib/fail2ban/fail2ban.sqlite3

APIKEY=$1
COMMENT=$2
IP=$3
CATEGORIES=$4
BANTIME=$5

ipMatch=`grep -Fe "IP=$IP L=[0-9\-]+" $REPORTED_IP_LIST_FILE`

shouldBanIP=1
currentTimestamp=`date +%s`

if [ -z $ipMatch ] ; then
	banLength=`echo $ipMatch | sed -E 's/.*L=([0-9\-]+)/\1/'`
	timeOfBan=`sqlite3 $FAIL2BAN_SQLITE_DB "SELECT timeofban FROM bans WHERE ip = '$IP'"`

	if (((banLength == -1 && banLength == BANTIME) || (timeOfBan > 0 && timeOfBan + banLength > currentTimestamp))) ; then
		shouldBanIP=0
	else
		sed -i "/^IP=$IP.*$/d" $REPORTED_IP_LIST_FILE
	fi
fi

if [ $shouldBanIP -eq 1 ] ; then
	echo "IP=$IP L=$BANTIME" >> $REPORTED_IP_LIST_FILE
	curl --fail 'https://api.abuseipdb.com/api/v2/report' \
		-H 'Accept: application/json' \
		-H "Key: $APIKEY" \
		--data-urlencode "comment=$COMMENT" \
		--data-urlencode "ip=$IP" \
		--data "categories=$CATEGORIES"
fi

3. Make the file an executable

Type chmod +x abuseipdb-fail2ban-report.sh into your terminal

4. Modify /etc/fail2ban/action.d/abuseipdb.conf actionban to be the following:

actionban = /path/to/file/abuseipdb-fail2ban-report.sh \

    "<abuseipdb_apikey>" "<matches>" "<ip>" "<abuseipdb_category>" "<bantime>"

5. Change the line %(action_abuseipdb)s[abuseipdb_apikey="my-api-key", abuseipdb_category="18,22"] in your jail.local to:

%(action_abuseipdb)s[abuseipdb_apikey="my-api-key", abuseipdb_category="18,22", bantime="%(bantime)s"] for each jail that you would like this applied to.

6. Restart Fail2Ban using fail2ban-client reload

Thanks for supporting AbuseIPDB! Do you have any feedback or suggestions about this tutorial? Please let us know!