v7.

This allows for some simple form of netmask type patterning
which will work for /8, /16 and /24 subnets to be whitelisted
for ipv4, and for any multiple of /32 subnets for ipv6.

configure.ac

@@ -2,7 +2,7 @@
 # Process this file with autoconf to produce a configure script.
 
 AC_PREREQ([2.64])
-AC_INIT([tallow], [5], [auke-jan.h.kok@intel.com])
+AC_INIT([tallow], [7], [auke-jan.h.kok@intel.com])
 AM_INIT_AUTOMAKE([])
 AC_CONFIG_FILES([Makefile])
 

tallow.1

@@ -1,7 +1,7 @@
 .\" generated with Ronn/v0.7.3
 .\" http://github.com/rtomayko/ronn/tree/0.7.3
 .
-.TH "TALLOW" "1" "May 2017" "" ""
+.TH "TALLOW" "1" "December 2017" "" ""
 .
 .SH "NAME"
 \fBtallow\fR
@@ -13,10 +13,10 @@ Reduce log clutter due to ssh login attempts\.
 \fB/usr/sbin/tallow\fR
 .
 .SH "DESCRIPTION"
-\fBtallow\fR is a daemon that watches the systemd journal for messages from the \fBsshd\fR service\. It parses the messages and looks for attempted random logins such as failed logins to the root account and failed logins to invalid user accounts\.
+\fBtallow\fR is a daemon that watches the systemd journal for messages from the \fBsshd\fR service\. It parses the messages and looks for attempted random logins such as failed logins to the root account and failed logins to invalid user accounts, and various other obviously malicious login attempts that try things as forcing old protocols, or weak key systems\.
 .
 .P
-If such logins were detected, the offending IP address is stored in a list\. Items from this list are regularly purged, but if the amount of times that a specific IP address is seen exceeds a threshold (default 3), an ipset(1) entry is inserted in the \fBtallow\fR or \fBtallow6\fR ipset, and further packets from that ip address will be blocked by an \fBiptables(1)\fR or \fBip6tables(1)\fR rule that tallow creates at startup\.
+If such logins were detected, the offending IP address is stored in a list\. Items from this list are regularly purged, but if the amount of times that a specific IP address is seen exceeds a threshold, an ipset(1) entry is inserted in the \fBtallow\fR or \fBtallow6\fR ipset, and further packets from that ip address will be blocked by an \fBiptables(1)\fR or \fBip6tables(1)\fR rule that tallow creates at startup\. Additionally, certain types of login failure will trigger a short term ban of further packets from the offending IP address immediately\.
 .
 .P
 The system administrator needs to assure that the tallow and tallow6 ipsets are left alone and that the inserted iptables rules are properly matching on packets\.
@@ -28,7 +28,7 @@ Care should be taken to assure that legitimate users are not blocked inadvertent
 The \fBtallow\fR daemon itself has no runtime configuration\. All configuration is done through the tallow\.conf(5) config file\.
 .
 .SH "SEE ALSO"
-systemd\-journald(1), iptables(1), tallow\.conf(5)
+systemd\-journald(1), iptables(1), ipset(1), tallow\.conf(5)
 .
 .SH "BUGS"
 \fBtallow\fR is \fBNOT A SECURITY SOLUTION\fR, nor does it protect against random password logins\. A attacker may still be able to logon to your systems if you allow password logins\.

tallow.1.md

@@ -9,27 +9,31 @@ Reduce log clutter due to ssh login attempts.
 
 ## DESCRIPTION
 
-`tallow` is a daemon that watches the systemd journal for
-messages from the `sshd` service. It parses the messages
-and looks for attempted random logins such as failed logins to the
-root account and failed logins to invalid user accounts.
+`tallow` is a daemon that watches the systemd journal for messages
+from the `sshd` service. It parses the messages and looks for
+attempted random logins such as failed logins to the root account and
+failed logins to invalid user accounts, and various other obviously
+malicious login attempts that try things as forcing old protocols,
+or weak key systems.
 
-If such logins were detected, the offending IP address is stored
-in a list. Items from this list are regularly purged, but if
-the amount of times that a specific IP address is seen exceeds
-a threshold (default 3), an ipset(1) entry is inserted in the
-`tallow` or `tallow6` ipset, and further packets from that ip
-address will be blocked by an `iptables(1)` or `ip6tables(1)`
-rule that tallow creates at startup.
+If such logins were detected, the offending IP address is stored in
+a list. Items from this list are regularly purged, but if the amount
+of times that a specific IP address is seen exceeds a threshold,
+an ipset(1) entry is inserted in the `tallow` or `tallow6`
+ipset, and further packets from that ip address will be blocked
+by an `iptables(1)` or `ip6tables(1)` rule that tallow creates at
+startup. Additionally, certain types of login failure will trigger
+a short term ban of further packets from the offending IP address
+immediately.
 
-The system administrator needs to assure that the tallow
-and tallow6 ipsets are left alone and that the inserted
-iptables rules are properly matching on packets.
+The system administrator needs to assure that the tallow and tallow6
+ipsets are left alone and that the inserted iptables rules are properly
+matching on packets.
 
 Care should be taken to assure that legitimate users are not
 blocked inadvertently. You may wish to list any valid IP address
-with the whitelist option in tallow.conf(5). Multiple addresses
-can be whitelisted.
+with the whitelist option in tallow.conf(5). Multiple addresses can
+be whitelisted.
 
 ## OPTIONS
 
@@ -38,13 +42,13 @@ configuration is done through the tallow.conf(5) config file.
 
 ## SEE ALSO
 
-systemd-journald(1), iptables(1), tallow.conf(5)
+systemd-journald(1), iptables(1), ipset(1), tallow.conf(5)
 
 ## BUGS
 
-`tallow` is `NOT A SECURITY SOLUTION`, nor does it protect
-against random password logins. A attacker may still be able to
-logon to your systems if you allow password logins.
+`tallow` is `NOT A SECURITY SOLUTION`, nor does it protect against
+random password logins. A attacker may still be able to logon to your
+systems if you allow password logins.
 
 ## AUTHOR
 

tallow.c

@@ -24,26 +24,57 @@
 
 #include <systemd/sd-journal.h>
 
+#ifdef DEBUG
+#define dbg(args...) fprintf(stderr, ##args)
+#else
+#define dbg(args...) do {} while (0)
+#endif
+
 struct tallow_struct {
 	char *ip;
-	int count;
+	float score;
 	struct timeval time;
 	struct tallow_struct *next;
+	bool blocked;
 };
 
 static struct tallow_struct *head;
 
-static struct tallow_struct *whitelist;
+struct whitelist_struct {
+	char *ip;
+	size_t len;
+	struct whitelist_struct *next;
+};
+
+static struct whitelist_struct *whitelist;
 
 #define FILTER_STRING "SYSLOG_IDENTIFIER=sshd"
-static char *pattern = "MESSAGE=Failed password for .* from ([0-9a-z:.]+) port \\d+ ssh2";
+struct pattern_struct {
+	int instant_block;
+	float weight;
+	char *pattern;
+	pcre *re;
+};
+
+#define PATTERN_COUNT 9
+static struct pattern_struct patterns[PATTERN_COUNT] = {
+	{ 0, 0.4, "MESSAGE=Failed .* for .* from ([0-9a-z:.]+) port \\d+ ssh2", NULL},
+	{ 0, 0.4, "MESSAGE=error: PAM: Authentication failure for .* from ([0-9a-z:.]+)", NULL},
+	{15, 0.3, "MESSAGE=Invalid user .* from ([0-9a-z:.]+) port \\d+", NULL},
+	{15, 0.3, "MESSAGE=Did not receive identification string from ([0-9a-z:.]+) port \\d+", NULL},
+	{30, 0.3, "MESSAGE=Failed .* for root from ([0-9a-z:.]+) port \\d+ ssh2", NULL},
+	{15, 0.6, "MESSAGE=Bad protocol version identification .* from ([0-9a-z:.]+)", NULL},
+	{15, 0.6, "MESSAGE=Connection closed by authenticating user .* ([0-9a-z:.]+) port \\d+", NULL},
+	{15, 0.6, "MESSAGE=Received disconnect from ([0-9a-z:.]+) port .*\\[preauth\\]", NULL},
+	{60, 0.6, "MESSAGE=Unable to negotiate with ([0-9a-z:.]+) port \\d+: no matching key exchange method found.", NULL}
+};
 
 #define MAX_OFFSETS 30
 
 static char ipt_path[PATH_MAX];
-static int threshold = 3;
 static int expires = 3600;
 static int has_ipv6 = 0;
+static bool nocreate = false;
 static sd_journal *j;
 
 static int ext(char *fmt, ...)
@@ -81,6 +112,9 @@ static void setup(void)
 		return;
 	done = true;
 
+	if (nocreate)
+		return;
+
 	/* init ipset and iptables */
 	/* delete iptables ref to set before the ipset! */
 	ext_ignore("%s/iptables -t filter -D INPUT -m set --match-set tallow src -j DROP 2> /dev/null", ipt_path);
@@ -108,39 +142,58 @@ static void setup(void)
 	}
 }
 
-static void block(struct tallow_struct *s)
+static void block(struct tallow_struct *s, int instant_block)
 {
-	if (s->count != threshold)
-		return;
-
 	setup();
 
+
 	if (strchr(s->ip, ':')) {
-		if (has_ipv6)
-			(void) ext("%s/ipset -A tallow6 %s", ipt_path, s->ip);
+		if (has_ipv6) {
+			if (instant_block > 0) {
+				(void) ext("%s/ipset -! add tallow6 %s timeout %d",
+					   ipt_path, s->ip, instant_block);
+			} else {
+				(void) ext("%s/ipset -! add tallow6 %s", ipt_path, s->ip);
+				s->blocked = true;
+			}
+		}
 	} else {
-		(void) ext("%s/ipset -A tallow %s", ipt_path, s->ip);
+		if (instant_block > 0) {
+			(void) ext("%s/ipset -! add tallow %s timeout %d",
+				   ipt_path, s->ip, instant_block);
+		} else {
+			(void) ext("%s/ipset -! add tallow %s", ipt_path, s->ip);
+			s->blocked = true;
+		}
 	}
 
-	fprintf(stderr, "Blocked %s\n", s->ip);
+	if (s->blocked) {
+		fprintf(stderr, "Blocked %s\n", s->ip);
+	} else {
+		dbg("Throttled %s\n", s->ip);
+	}
 }
 
 static void whitelist_add(char *ip)
 {
-	struct tallow_struct *w = whitelist;
-	struct tallow_struct *n;
+	struct whitelist_struct *w = whitelist;
+	struct whitelist_struct *n;
 
 	while (w && w->next)
 		w = w->next;
 
-	n = malloc(sizeof(struct tallow_struct));
+	n = calloc(1, sizeof(struct whitelist_struct));
 	if (!n) {
 		fprintf(stderr, "Out of memory.\n");
 		exit(EXIT_FAILURE);
 	}
-	memset(n, 0, sizeof(struct tallow_struct));
+
 	n->ip = strdup(ip);
-	n->next = NULL;
+	size_t l = strlen(ip);
+	if ((ip[l-1] == '.') || (ip[l-1] == ':'))
+		n->len = l;
+	else
+		n->len = -1;
 
 	if (!whitelist)
 		whitelist = n;
@@ -148,11 +201,11 @@ static void whitelist_add(char *ip)
 		w->next = n;
 }
 
-static void find(const char *ip)
+static void find(const char *ip, float weight, int instant_block)
 {
 	struct tallow_struct *s = head;
 	struct tallow_struct *n;
-	struct tallow_struct *w = whitelist;
+	struct whitelist_struct *w = whitelist;
 
 	if (!ip)
 		return;
@@ -167,18 +220,33 @@ static void find(const char *ip)
 
 	/* whitelist */
 	while (w) {
-		if (!strcmp(w->ip, ip))
-			return;
+		if (w->len > 0) {
+			if (!strncmp(w->ip, ip, w->len))
+				return;
+		} else {
+			if (!strcmp(w->ip, ip))
+				return;
+		}
 		w = w->next;
 	}
 
 	/* walk and update entry */
 	while (s) {
 		if (!strcmp(s->ip, ip)) {
-			s->count++;
+			s->score += weight;
+			dbg("%s: %1.3f\n", s->ip, s->score);
 			(void) gettimeofday(&s->time, NULL);
 
-			block(s);
+			if (s->blocked) {
+				return;
+			}
+
+			if (s->score >= 1.0) {
+				block(s, 0);
+			} else if (instant_block > 0) {
+				block(s, instant_block);
+			}
+
 			return;
 		}
 
@@ -189,12 +257,11 @@ static void find(const char *ip)
 	}
 
 	/* append */
-	n = malloc(sizeof(struct tallow_struct));
+	n = calloc(1, sizeof(struct tallow_struct));
 	if (!n) {
 		fprintf(stderr, "Out of memory.\n");
 		exit(EXIT_FAILURE);
 	}
-	memset(n, 0, sizeof(struct tallow_struct));
 
 	if (!head)
 		head = n;
@@ -202,11 +269,17 @@ static void find(const char *ip)
 		s->next = n;
 
 	n->ip = strdup(ip);
-	n->count = 1;
+	n->score = weight;
 	n->next = NULL;
+	n->blocked = false;
 	(void) gettimeofday(&n->time, NULL);
+	dbg("%s: %1.3f\n", n->ip, n->score);
 
-	block(n);
+	if (weight >= 1.0) {
+		block(n, 0);
+	} else if (instant_block > 0) {
+		block(n, instant_block);
+	}
 	return;
 }
 
@@ -227,6 +300,16 @@ static void sig(int u __attribute__ ((unused)))
 	exit(EXIT_SUCCESS);
 }
 
+static void sigusr1(int u __attribute__ ((unused)))
+{
+	fprintf(stderr, "Dumping score list on request:\n");
+	struct tallow_struct *s = head;
+	while (s) {
+		fprintf(stderr, "%ld %s %1.3f\n", s->time.tv_sec, s->ip, s->score);
+		s = s->next;
+	}
+}
+
 static void prune(void)
 {
 	struct tallow_struct *s = head;
@@ -237,7 +320,15 @@ static void prune(void)
 	p = NULL;
 
 	while (s) {
-		if ((tv.tv_sec - s->time.tv_sec) > expires) {
+		/*
+		 * Expire all records, but if they are blocked, make sure to
+		 * expire them *before* the ipset rule expires, otherwise
+		 * you might get an IP to bypass checks.
+		 */
+		time_t age = tv.tv_sec - s->time.tv_sec;
+		if ((age > expires) ||
+		    ((s->blocked) && (age > expires / 2))) {
+			dbg("Expired record for %s\n", s->ip);
 			if (p) {
 				p->next = s->next;
 				free(s->ip);
@@ -273,6 +364,10 @@ int main(void)
 	sigaction(SIGTERM, &s, NULL);
 	sigaction(SIGINT, &s, NULL);
 
+	memset(&s, 0, sizeof(struct sigaction));
+	s.sa_handler = sigusr1;
+	sigaction(SIGUSR1, &s, NULL);
+
 	if (access("/proc/sys/net/ipv6", R_OK | X_OK) == 0)
 		has_ipv6 = 1;
 
@@ -301,14 +396,14 @@ int main(void)
 			// todo: filter leading/trailing whitespace
 			if (!strcmp(key, "ipt_path"))
 				strncpy(ipt_path, val, PATH_MAX - 1);
-			if (!strcmp(key, "threshold"))
-				threshold = atoi(val);
 			if (!strcmp(key, "expires"))
 				expires = atoi(val);
 			if (!strcmp(key, "whitelist"))
 				whitelist_add(val);
 			if (!strcmp(key, "ipv6"))
 				has_ipv6 = atoi(val);
+			if (!strcmp(key, "nocreate"))
+				nocreate = (atoi(val) == 1);
 		}
 		fclose(f);
 	}
@@ -330,17 +425,23 @@ int main(void)
 	sd_journal_add_match(j, FILTER_STRING, 0);
 	r = sd_journal_seek_tail(j);
 	sd_journal_wait(j, (uint64_t) 0);
-	fprintf(stderr, "sd_journal_seek_tail() returned %d\n", r);
+	dbg("sd_journal_seek_tail() returned %d\n", r);
 	while (sd_journal_next(j) != 0)
 		r++;
-	fprintf(stderr, "Forwarded through %d items in the journal to reach the end\n", r);
+	dbg("Forwarded through %d items in the journal to reach the end\n", r);
 
 	fprintf(stderr, "Started\n");
 
-	pcre *re = NULL;
-	int err;
-	const char *pcre_err;
-	re = pcre_compile(pattern, 0, &pcre_err, &err, NULL);
+	for (int i = 0; i < PATTERN_COUNT; i++) {
+		int err;
+		const char *pcre_err;
+		patterns[i].re = pcre_compile(patterns[i].pattern, 0, &pcre_err, &err, NULL);
+		if (!patterns[i].re) {
+			fprintf(stderr, "PCRE compilation failed. Pattern %d, offset %d: %s\n",
+				i, err, pcre_err);
+			exit(EXIT_FAILURE);
+		}
+	}
 
 	for (;;) {
 		const void *d;
@@ -363,14 +464,17 @@ int main(void)
 			m = strndup(d, l+1);
 			m[l] = '\0';
 
-			int off[MAX_OFFSETS];
-			int ret = pcre_exec(re, NULL, m, l, 0, 0, off, MAX_OFFSETS);
-			if (ret == 2) {
-				const char *s;
-				ret = pcre_get_substring(m, off, 2, 1, &s);
-				if (ret > 0) {
-					find(s);
-					pcre_free_substring(s);
+			for (int i = 0; i < PATTERN_COUNT; i++) {
+				int off[MAX_OFFSETS];
+				int ret = pcre_exec(patterns[i].re, NULL, m, l, 0, 0, off, MAX_OFFSETS);
+				if (ret == 2) {
+					const char *s;
+					ret = pcre_get_substring(m, off, 2, 1, &s);
+					if (ret > 0) {
+						dbg("%s == %s\n", s, patterns[i].pattern);
+						find(s, patterns[i].weight, patterns[i].instant_block);
+						pcre_free_substring(s);
+					}
 				}
 			}
 

tallow.conf

@@ -1,8 +1,7 @@
 
 # tallow.conf - see `man tallow.conf` for more information
 
-#path_iptables=/usr/sbin
-#threshold=3
+#ipt_path=/usr/sbin
 #expires=3600
 #whitelist=127.0.0.1
 #ipv6=0

tallow.conf.5

@@ -1,7 +1,7 @@
 .\" generated with Ronn/v0.7.3
 .\" http://github.com/rtomayko/ronn/tree/0.7.3
 .
-.TH "TALLOW" "5" "May 2017" "" ""
+.TH "TALLOW" "5" "January 2018" "" ""
 .
 .SH "NAME"
 \fBtallow\fR
@@ -25,16 +25,30 @@ This file is read on startup by the tallow(1) daemon, and can be used to provide
 \fBexpires\fR=\fB<int>\fR The number of seconds that IP addresses are blocked for\. Note that due to the implementation, IP addresses may be blocked for much longer than this period\. If IP addresses are seen, but not blocked within this period, they are also removed from the watch list\. Defaults to 3600s\.
 .
 .P
-\fBthreshold\fR=\fB<int>\fR Specifies the number of times an IP address may appear before it is blocked\. Defaults to 3\.
-.
-.P
 \fBwhitelist\fR=\fB<ipv4 address>\fR Specify an IP address that should never be blocked\. Multiple IP addresses can be included by repeating the \fBwhitelist\fR option several times\. By default, only 127\.0\.0\.1 is whitelisted\.
 .
 .P
 \fBipv6\fR=\fB<0|1>\fR Enable of disable ipv6 (ip6tables) support\. Ipv6 is disabled automatically on systems that do not appear to have ipv6 support and enabled when ipv6 is present\. Use this option to explicitly disable ipv6 support if your system does not have ipv6 or is missing ip6tables\. Even with ipv6 disabled, tallow will track and log ipv6 addresses\.
 .
+.P
+\fBnocreate\fR=\fB<0|1>\fR Disable the creation of iptables rules and ipset sets\. By default, tallow will create new iptables(1) and ip6tables(1) rules when needed automatically\. If set to \fB1\fR, \fBtallow(1)\fR will not create any new iptables rules or ipset sets to work\. You should create them manually before tallow starts up and remove them afterwards\. To create them manually, you can use the following commands:
+.
+.IP "" 4
+.
+.nf
+
+  iptables \-t filter \-I INPUT \-m set \-\-match\-set tallow src \-j DROP
+  ipset create tallow hash:ip family inet timeout 3600
+
+  ip6tables \-t filter \-I INPUT \-m set \-\-match\-set tallow6 src \-j DROP
+  ipset create tallow6 hash:ip family inet6 timeout 3600
+.
+.fi
+.
+.IP "" 0
+.
 .SH "SEE ALSO"
-tallow(1), iptables(1)
+tallow(1)
 .
 .SH "AUTHOR"
 Auke Kok \fIauke\-jan\.h\.kok@intel\.com\fR

tallow.conf.5.md

@@ -30,14 +30,17 @@ longer than this period. If IP addresses are seen, but not
 blocked within this period, they are also removed from the
 watch list. Defaults to 3600s.
 
-`threshold`=`<int>`
-Specifies the number of times an IP address may appear before it
-is blocked. Defaults to 3.
+`whitelist`=`<ip address|pattern>`
+Specify an IP address or `pattern` that should never be
+blocked. Multiple IP addresses can be included by repeating the
+`whitelist` option several times. By default, only 127.0.0.1 is
+whitelisted.
 
-`whitelist`=`<ipv4 address>`
-Specify an IP address that should never be blocked. Multiple IP
-addresses can be included by repeating the `whitelist`
-option several times. By default, only 127.0.0.1 is whitelisted.
+If the last character of the listed ip adress is a `.` or a `:`, then
+the matching is only performed on the leftmost characters of an IP
+address against the whitelist entry. For instance, if you whitelist
+`10.` then all IP addresses in the `10/8` subnet mask will match this
+whitelist entry and never be blocked.
 
 `ipv6`=`<0|1>`
 Enable of disable ipv6 (ip6tables) support. Ipv6 is disabled
@@ -47,9 +50,25 @@ disable ipv6 support if your system does not have ipv6 or is
 missing ip6tables. Even with ipv6 disabled, tallow will track
 and log ipv6 addresses.
 
+`nocreate`=`<0|1>`
+Disable the creation of iptables rules and ipset sets. By default,
+tallow will create new iptables(1) and ip6tables(1) rules when needed
+automatically. If set to `1`, `tallow(1)` will not create any new
+iptables rules or ipset sets to work. You should create them manually
+before tallow starts up and remove them afterwards. To create them
+manually, you can use the following commands:
+
+  ```
+  iptables -t filter -I INPUT -m set --match-set tallow src -j DROP
+  ipset create tallow hash:ip family inet timeout 3600
+
+  ip6tables -t filter -I INPUT -m set --match-set tallow6 src -j DROP
+  ipset create tallow6 hash:ip family inet6 timeout 3600
+  ```
+
 ## SEE ALSO
 
-tallow(1), iptables(1)
+tallow(1)
 
 ## AUTHOR