Release v1.13.0

This release introduces some changes in behavior for the crash probe and
journal probe:

* The crash probe now ensures core files are kept whenever backtraces
  are scrubbed from telemetry records, or when errors occur during runtime
  operation. This improves the developer experience when running ones own
  programs installed at arbitrary locations on the filesystem and also
  avoids data loss; either the crash backtrace is sent via telemetry, or
  when unable to send the backtrace, the core file is kept.

* The journal probe, by default, now only sends telemetry for journal
  log messages indicating systemd service failures. The previous behavior
  of sending any log messages above a certain log level is now wrapped
  behind the "opt-in-no-privacy-filters" config setting.

Bug fixes include:

* Fixing a segfault crash in the configuration code that occurred
  whenever an incomplete configuration file was encountered.

* Fixing a performance issue in 'telem-record-gen' to avoid extraneous
  strlen() calls in the classification input validation code.

Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>

.travis.yml

@@ -16,7 +16,6 @@ addons:
         sources:
             - ubuntu-toolchain-r-test
         packages:
-            - libglib2.0-dev
             - libcurl4-gnutls-dev
             - valgrind
             - autoconf 

README

@@ -1,32 +0,0 @@
-telemetrics-client
-==================
-
-This package provides the front end component of a complete telemetrics
-solution for Linux-based operating systems. Specifically, the front end
-component includes:
-
-- telemetrics probes that collect specific types of data from the operating
-  system.
-
-- a library, libtelemetry, that telemetrics probes use to create telemetrics
-  records and send them to the daemon for further processing.
-
-- a daemon, telemd, that prepares the records to send to a telemetrics server
-  (not included in this source tree), or spools the records on disk in case
-  it's unable to successfully deliver them.
-
-
-Build dependencies
-==================
-
-- libcheck
-
-- libcurl
-
-- glib
-
-- elfutils, which provides libelf and libdwfl libraries..
-
-- (optional) libsystemd, for syslog-style logging to the systemd journal, and
-  socket/path activation of telemd by systemd.
-

README.md

@@ -0,0 +1,192 @@
+Telemetrics-client
+==================
+
+This package provides the front end component of a complete telemetrics
+solution for Linux-based operating systems. Specifically, the front end
+component includes:
+
+- telemetrics probes that collect specific types of data from the operating
+  system.
+
+- a library, libtelemetry, that telemetrics probes use to create telemetrics
+  records and send them to the daemon for further processing.
+
+- a daemon, telemd, that prepares the records to send to a telemetrics server
+  (not included in this source tree), or spools the records on disk in case
+  it's unable to successfully deliver them.
+
+
+Build dependencies
+---------------------
+
+- libcheck
+
+- libcurl
+
+- elfutils, which provides libelf and libdwfl libraries..
+
+- (optional) libsystemd, for syslog-style logging to the systemd journal, and
+  socket/path activation of telemd by systemd.
+
+
+Build and installation
+---------------------
+
+```{r, engine='bash', count_lines}
+$ ./autogen.sh
+$ ./configure
+$ make
+```
+
+Set up
+---------------------
+
+There is a config file installed at
+
+```{r, engine='bash', count_lines}
+/usr/share/defaults/telemetrics/telemetrics.conf
+```
+
+To make modifications, copy that file to
+
+```{r, engine='bash', count_lines}
+/etc/telemetrics/telemetrics.conf
+```
+and modify the /etc version.
+
+Descriptions of config options are listed below in the Usage section.
+
+Starting the daemon
+---------------------
+
+Method 1 (recommended):
+
+```{r, engine='bash', count_lines}
+ systemctl start telemd.socket telemd.path
+```
+
+Note: the above invocation technically readies the service for both socket and
+path activation, so you may not see the daemon start right away.
+
+Method 2:
+
+```{r, engine='bash', count_lines}
+systemctl start telemd.service
+```
+
+Method 3:
+
+```{r, engine='bash', count_lines}
+telemd &
+```
+
+Configure the daemon to autostart at boot
+---------------------
+
+Method 1 (recommended):
+
+Enable the socket-activated service and path unit:
+
+```{r, engine='bash', count_lines}
+systemctl enable telemd.socket telemd.path
+```
+
+Method 2:
+
+Enable the service itself, which automatically enables the socket and path
+units as well:
+
+```{r, engine='bash', count_lines}
+systemctl enable telemd.service
+```
+
+Usage
+---------------------
+
+Once the daemon is running, the telemetrics probes will be ready to use.
+
+Available probes:
+
+* hprobe: A test program that utilizes libtelemetry to ensure
+that telemetrics-client works. It sends a "hello world" record to the server.
+
+* crash probe: A handler for core files that sends the corresponding backtraces
+to the server.
+
+The daemon uses the following configuration options from the configuration file:
+
+* server: This specifies the web server that the daemon sends the telemetry records to
+* socket_path: This specifies the path of the unix domain socket that the
+  daemon listens on for connections from the probes
+* spool_dir: This config option is related to spooling. If the daemon is not
+  able to send the telemetry records to the backend server due to reasons such
+  as the network availability, then it stores the records in a spool directory.
+  This option specifies that path of the spool directory. This directory should
+  be owned by the same user that the daemon is running as.
+
+```{r, engine='bash', count_lines}
+    mkdir -p /var/spool/telemetry
+    chown -R telemetry:telemetry /var/spool/telemetry
+    systemctl restart telemd.service
+```
+
+* record_expiry: This is the time in minutes after which the records in the
+  spool directory are deleted by the daemon.
+* spool_max_size: This specifies the maximum size of the spool directory. When
+  the size of the spool directory reaches this limit, new telemetry records are
+  dropped by the daemon.
+* spool_process_time: This specifies the time interval in seconds that the
+  daemon waits for before checking the spool directory for records. The daemon
+  picks up the records in the order of modification date and tries to send the
+  record to the server. It sends a maximum of 10 records at a time. If it was
+  able to send a record successfully, it deletes the record from the spool.  If
+  the daemon finds a record older than the "record_expiry" time, then it
+  deletes that record. The daemon looks at a maximum of 20 records in a single
+  spool run loop.
+* rate_limit_enabled: This determines whether rate-limiting is enabled or
+  disabled. When enabled, there is a threshold on both records sent within a
+  window of time, and record bytes sent within a window a time.
+* record_burst_limit: This is the maximum amount of records allowed to be
+  passed by the daemon within the record_window_length of time. If set to -1,
+  the rate-limiting for record bursts is disabled.
+* record_window_length: This is the time, in minutes (0-59), that establishes
+  the window length for the record_burst_limit. EX: if record_burst_window=1000
+  and record_window_length=15, then no more than 1000 records can be passed
+  within any given fifteen minute window.
+* byte_burst_limit: This is the maximum amount of bytes that can be passed by
+  the daemon within the byte_window_length of time. If set to -1, the
+  rate-limiting for byte bursts is disabled.
+* byte_window_length: This is the time, in minutes (0-59), that establishes the
+  window length for the byte_burst_limit.
+* rate_limit_strategy: This is the strategy chosen once the rate-limiting
+  threshold has been reached. Currently the options are 'drop' or 'spool', with
+  spool being the default. If spool is chosen, records will be spooled and sent
+  at a later time.
+
+Machine id
+---------------------
+
+The machine id reported by the telemetry client is rotated every 3 days for
+privacy reasons. If you wish to have a static machine id for testing purposes,
+you can opt in by creating a static machine id file named
+"opt-in-static-machine-id" under the directory  "/etc/telemetrics/". Where
+"unique machine ID" is your desired static machine ID:
+
+```{r, engine='bash', count_lines}
+# mkdir -p /etc/telemetrics
+# echo "unique machine ID" > /etc/telemetrics/opt-in-static-machine-id
+```
+
+The telemetry daemon reads, at most, the first 32 characters from this file
+uses it for the machine id. You can put a string like 'my-machine-name' in this
+file to easily identify your machine.  Restart telemd for the machine id
+changes to take effect by running:
+
+```{r, engine='bash', count_lines}
+# systemctl restart telemd.service
+```
+
+You can switch back to the rotating machine id by deleting the override file
+and restarting the daemon.  You can do a quick test to check that your
+machine-id has changed by running "hprobe" and verifying that a record has
+landed on your backend telemetrics server, with the specified machine id.

configure.ac

@@ -2,7 +2,7 @@
 # Process this file with autoconf to produce a configure script.
 
 AC_PREREQ([2.69])
-AC_INIT([telemetrics-client], [1.12.0], [https://clearlinux.org/])
+AC_INIT([telemetrics-client], [1.13.0], [https://clearlinux.org/])
 AC_CONFIG_AUX_DIR([build-aux])
 AM_INIT_AUTOMAKE([1.14 -Wall -Werror -Wno-extra-portability foreign subdir-objects])
 AM_SILENT_RULES([yes])
@@ -25,7 +25,6 @@ AC_PROG_MAKE_SET
 # check >= 0.9.12 is required for TAP output
 PKG_CHECK_MODULES([CHECK], [check >= 0.9.12])
 PKG_CHECK_MODULES([CURL], [libcurl])
-PKG_CHECK_MODULES([GLIB], [glib-2.0])
 AC_CHECK_LIB([elf], [elf_begin], [have_elflib=yes], [AC_MSG_ERROR([Unable to find libelf from elfutils])])
 AC_CHECK_LIB([dw], [dwfl_begin], [have_dwlib=yes], [AC_MSG_ERROR([Unable to find libdw from elfutils])])
 AS_IF([test "x$have_elflib" = "xyes" -a "x$have_dwlib" = "xyes"],

src/configuration.c

@@ -16,7 +16,6 @@
 
 #include <stdio.h>
 #include <stdlib.h>
-#include <glib.h>
 #include <errno.h>
 #include <limits.h>
 #include <dirent.h>
@@ -30,10 +29,12 @@
 #include "util.h"
 #include "common.h"
 
+#include "nica/inifile.h"
+
 static char *config_file = NULL;
 static char *default_config_file = DATADIR "/defaults/telemetrics/telemetrics.conf";
 static char *etc_config_file = "/etc/telemetrics/telemetrics.conf";
-static GKeyFile *keyfile = NULL;
+static NcHashmap *keyfile = NULL;
 static bool cmd_line_cfg = false;
 
 /* Conf strings, integers, and booleans expected in the conf file */
@@ -83,16 +84,12 @@ void set_config_file(char *filename)
 
 bool read_config_from_file(char *config_file, struct configuration *config)
 {
-        GError *error = NULL;
-        GKeyFileFlags flags = G_KEY_FILE_NONE;
-
         if (keyfile != NULL) {
-                g_key_file_free(keyfile);
+                nc_hashmap_free(keyfile);
         }
 
-        keyfile = g_key_file_new();
-
-        if (!g_key_file_load_from_file(keyfile, config_file, flags, &error)) {
+        keyfile = nc_ini_file_parse(config_file);
+        if (!keyfile) {
 #ifdef DEBUG
                 fprintf(stderr, "ERR: Failed to read config file: %s\n",
                         error->message);
@@ -100,24 +97,43 @@ bool read_config_from_file(char *config_file, struct configuration *config)
                 return false;
         } else {
                 for (int i = CONF_STR_MIN + 1; i < CONF_STR_MAX; i++) {
-                        config->strValues[i] = g_key_file_get_string(keyfile,
-                                                                     "settings", config_key_str[i], &error);
-                        if (error != NULL) {
+                        char *ptr;
+                        ptr = nc_hashmap_get(nc_hashmap_get(keyfile, "settings"), config_key_str[i]);
+                        if (ptr) {
+                                config->strValues[i] = strdup(ptr);
+                                if (config->strValues[i] == NULL) {
+                                        return false;
+                                }
+                        } else {
+                                fprintf(stderr, "ERR: missing key with string value: %s\n", config_key_str[i]);
                                 return false;
                         }
                 }
                 for (int i = CONF_INT_MIN + 1; i < CONF_INT_MAX; i++) {
-                        config->intValues[i] = g_key_file_get_int64(keyfile,
-                                                                    "settings", config_key_int[i], &error);
-                        if (error != NULL) {
+                        char *ptr;
+                        ptr = nc_hashmap_get(nc_hashmap_get(keyfile, "settings"), config_key_int[i]);
+                        if (ptr) {
+                                config->intValues[i] = strtoll(ptr, NULL, 10);
+                        } else {
+                                fprintf(stderr, "ERR: missing key with integer value: %s\n", config_key_int[i]);
                                 return false;
                         }
                 }
 
                 for (int i = CONF_BOOL_MIN + 1; i < CONF_BOOL_MAX; i++) {
-                        config->boolValues[i] = g_key_file_get_boolean(keyfile,
-                                                                       "settings", config_key_bool[i], &error);
-                        if (error != NULL) {
+                        char *ptr;
+                        ptr = nc_hashmap_get(nc_hashmap_get(keyfile, "settings"), config_key_bool[i]);
+                        config->boolValues[i] = false;
+
+                        if (ptr) {
+                                if (strcasecmp(ptr, "TRUE") == 0) {
+                                        config->boolValues[i] = true;
+                                }
+                                if (strcasecmp(ptr, "1") == 0) {
+                                        config->boolValues[i] = true;
+                                }
+                        } else {
+                                fprintf(stderr, "ERR: missing key with boolean value: %s\n", config_key_bool[i]);
                                 return false;
                         }
                 }
@@ -181,8 +197,6 @@ void free_configuration(void)
                 free(config.strValues[i]);
         }
 
-        g_key_file_free(keyfile);
-
         if (cmd_line_cfg) {
                 free(config_file);
         }

src/local.mk

@@ -44,20 +44,18 @@ noinst_LTLIBRARIES = %D%/libtelem-shared.la
 	%D%/util.c \
 	%D%/util.h \
 	%D%/configuration.c \
+	%D%/nica/inifile.c \
+	%D%/nica/hashmap.c \
 	%D%/configuration.h \
 	%D%/common.c \
 	%D%/common.h
 
 %C%_libtelem_shared_la_CFLAGS = \
-	$(AM_CFLAGS) \
-	$(GLIB_CFLAGS)
+	$(AM_CFLAGS)
 
 %C%_libtelem_shared_la_LDFLAGS = \
 	$(AM_LDFLAGS)
 
-%C%_libtelem_shared_la_LIBADD = \
-	$(GLIB_LIBS)
-
 lib_LTLIBRARIES = \
 	%D%/libtelemetry.la
 
@@ -69,12 +67,10 @@ include_HEADERS = %D%/telemetry.h
 noinst_HEADERS = %D%/log.h
 
 %C%_libtelemetry_la_CFLAGS = \
-	$(AM_CFLAGS) \
-	$(GLIB_CFLAGS)
+	$(AM_CFLAGS)
 
 %C%_libtelemetry_la_LDFLAGS = \
 	$(AM_LDFLAGS) \
-	$(GLIB_LIBS) \
 	-version-info $(SHAREDLIB_CURRENT):$(SHAREDLIB_REVISION):$(SHAREDLIB_AGE) \
 	-Wl,--version-script=$(top_srcdir)/src/telemetry.sym
 

src/nica/hashmap.c

@@ -0,0 +1,469 @@
+/*
+ * This file is part of libnica.
+ *
+ * Copyright © 2016-2017 Intel Corporation
+ *
+ * libnica is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU Lesser General Public License as
+ * published by the Free Software Foundation; either version 2.1
+ * of the License, or (at your option) any later version.
+ */
+
+#define _GNU_SOURCE
+#include <stdint.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+
+#include "hashmap.h"
+
+#define INITIAL_SIZE 61
+
+/* When we're "full", 70% */
+#define FULL_FACTOR 0.7
+
+/* Multiple to increase bucket size by */
+#define INCREASE_FACTOR 4
+
+/**
+ * An bucket/chain within the hashmap
+ */
+typedef struct NcHashmapEntry {
+        void *hash;                  /**<The key for this item */
+        void *value;                 /**<Value for this item */
+        struct NcHashmapEntry *next; /**<Next item in the chain */
+        bool occ;                    /**<Whether this bucket is occupied */
+} NcHashmapEntry;
+
+/**
+ * A NcHashmap
+ */
+struct NcHashmap {
+        int size;                /**<Current size of the hashmap */
+        int next_size;           /**<Next size at which we need to resize */
+        int n_buckets;           /**<Current number of buckets */
+        NcHashmapEntry *buckets; /**<Stores our bucket chains */
+
+        nc_hash_create_func hash;     /**<Hash generation function */
+        nc_hash_compare_func compare; /**<Key comparison function */
+        nc_hash_free_func key_free;   /**<Cleanup function for keys */
+        nc_hash_free_func value_free; /**<Cleanup function for values */
+};
+
+/**
+ * Iteration object
+ */
+typedef struct _NcHashmapIter {
+        int bucket;     /**<Current bucket position */
+        NcHashmap *map; /**<Associated NcHashmap */
+        void *item;     /**<Current item in this iteration */
+} _NcHashmapIter;
+
+static void nc_hashmap_update_next_size(NcHashmap *self);
+static bool nc_hashmap_resize(NcHashmap *self);
+
+static inline bool nc_hashmap_maybe_resize(NcHashmap *self)
+{
+        if (!self) {
+                return false;
+        }
+        if (self->size >= self->next_size) {
+                return true;
+        }
+        return false;
+}
+
+static NcHashmap *nc_hashmap_new_internal(nc_hash_create_func create, nc_hash_compare_func compare,
+                                          nc_hash_free_func key_free, nc_hash_free_func value_free)
+{
+        NcHashmap *map = NULL;
+        NcHashmapEntry *buckets = NULL;
+
+        map = calloc(1, sizeof(NcHashmap));
+        if (!map) {
+                return NULL;
+        }
+
+        buckets = calloc(INITIAL_SIZE, sizeof(NcHashmapEntry));
+        if (!buckets) {
+                free(map);
+                return NULL;
+        }
+        map->buckets = buckets;
+        map->n_buckets = INITIAL_SIZE;
+        map->hash = create ? create : nc_simple_hash;
+        map->compare = compare ? compare : nc_simple_compare;
+        map->key_free = key_free;
+        map->value_free = value_free;
+        map->size = 0;
+
+        nc_hashmap_update_next_size(map);
+
+        return map;
+}
+
+NcHashmap *nc_hashmap_new(nc_hash_create_func create, nc_hash_compare_func compare)
+{
+        return nc_hashmap_new_internal(create, compare, NULL, NULL);
+}
+
+NcHashmap *nc_hashmap_new_full(nc_hash_create_func create, nc_hash_compare_func compare,
+                               nc_hash_free_func key_free, nc_hash_free_func value_free)
+{
+        return nc_hashmap_new_internal(create, compare, key_free, value_free);
+}
+
+static inline unsigned nc_hashmap_get_hash(NcHashmap *self, const void *key)
+{
+        unsigned hash = self->hash(key);
+        return hash;
+}
+
+static int nc_hashmap_insert_bucket(NcHashmap *self, NcHashmapEntry *buckets, int n_buckets,
+                                    unsigned hash, const void *key, void *value)
+{
+        NcHashmapEntry *row = &(buckets[hash % (unsigned)n_buckets]);
+        NcHashmapEntry *head = NULL;
+        NcHashmapEntry *parent = head = row;
+        NcHashmapEntry *tomb = NULL;
+        int ret = 1;
+
+        while (row) {
+                if (!row->occ) {
+                        tomb = row;
+                }
+                parent = row;
+                if (row->occ && self->compare(row->hash, key)) {
+                        if (self->value_free) {
+                                self->value_free(row->value);
+                        }
+                        if (self->key_free) {
+                                self->key_free(row->hash);
+                        }
+                        row->hash = (void *)key;
+                        row->value = value;
+                        return 0;
+                }
+                row = row->next;
+        }
+
+        if (tomb) {
+                row = tomb;
+        }
+
+        if (!row) {
+                row = calloc(1, sizeof(NcHashmapEntry));
+                if (!row) {
+                        return -1;
+                }
+        }
+
+        row->hash = (void *)key;
+        row->value = value;
+        row->occ = true;
+        if (parent != row && parent) {
+                parent->next = row;
+        }
+
+        return ret;
+}
+
+bool nc_hashmap_put(NcHashmap *self, const void *key, void *value)
+{
+        if (!self) {
+                return false;
+        }
+        int inc;
+
+        if (nc_hashmap_maybe_resize(self)) {
+                if (!nc_hashmap_resize(self)) {
+                        return false;
+                }
+        }
+        unsigned hash = nc_hashmap_get_hash(self, key);
+        inc = nc_hashmap_insert_bucket(self, self->buckets, self->n_buckets, hash, key, value);
+        if (inc >= 0) {
+                self->size += inc;
+                return true;
+        } else {
+                return false;
+        }
+}
+
+static NcHashmapEntry *nc_hashmap_get_entry(NcHashmap *self, const void *key)
+{
+        if (!self) {
+                return NULL;
+        }
+
+        unsigned hash = nc_hashmap_get_hash(self, key);
+        NcHashmapEntry *row = &(self->buckets[hash % (unsigned)self->n_buckets]);
+
+        while (row) {
+                if (self->compare(row->hash, key)) {
+                        return row;
+                }
+                row = row->next;
+        }
+        return NULL;
+}
+
+void *nc_hashmap_get(NcHashmap *self, const void *key)
+{
+        if (!self) {
+                return NULL;
+        }
+
+        NcHashmapEntry *row = nc_hashmap_get_entry(self, key);
+        if (row) {
+                return row->value;
+        }
+        return NULL;
+}
+
+static bool nc_hashmap_remove_internal(NcHashmap *self, const void *key, bool remove)
+{
+        if (!self) {
+                return false;
+        }
+        NcHashmapEntry *row = nc_hashmap_get_entry(self, key);
+
+        if (!row) {
+                return false;
+        }
+
+        if (remove) {
+                if (self->key_free) {
+                        self->key_free(row->hash);
+                }
+                if (self->value_free) {
+                        self->value_free(row->value);
+                }
+        }
+        self->size -= 1;
+        row->hash = NULL;
+        row->value = NULL;
+        row->occ = false;
+
+        return true;
+}
+
+bool nc_hashmap_steal(NcHashmap *self, const void *key)
+{
+        return nc_hashmap_remove_internal(self, key, false);
+}
+
+bool nc_hashmap_remove(NcHashmap *self, const void *key)
+{
+        return nc_hashmap_remove_internal(self, key, true);
+}
+
+bool nc_hashmap_contains(NcHashmap *self, const void *key)
+{
+        return (nc_hashmap_get(self, key)) != NULL;
+}
+
+static inline void nc_hashmap_free_bucket(NcHashmap *self, NcHashmapEntry *bucket, bool nuke)
+{
+        if (!self) {
+                return;
+        }
+        NcHashmapEntry *tmp = bucket;
+        NcHashmapEntry *bk = bucket;
+        NcHashmapEntry *root = bucket;
+
+        while (tmp) {
+                bk = NULL;
+                if (tmp->next) {
+                        bk = tmp->next;
+                }
+
+                if (nuke && tmp->occ) {
+                        if (self->key_free) {
+                                self->key_free(tmp->hash);
+                        }
+                        if (self->value_free) {
+                                self->value_free(tmp->value);
+                        }
+                }
+                if (tmp != root) {
+                        free(tmp);
+                }
+                tmp = bk;
+        }
+}
+
+void nc_hashmap_free(NcHashmap *self)
+{
+        if (!self) {
+                return;
+        }
+        for (int i = 0; i < self->n_buckets; i++) {
+                NcHashmapEntry *row = &(self->buckets[i]);
+                nc_hashmap_free_bucket(self, row, true);
+        }
+        if (self->buckets) {
+                free(self->buckets);
+        }
+
+        free(self);
+}
+
+static void nc_hashmap_update_next_size(NcHashmap *self)
+{
+        if (!self) {
+                return;
+        }
+        self->next_size = (int)(self->n_buckets * FULL_FACTOR);
+}
+
+int nc_hashmap_size(NcHashmap *self)
+{
+        if (!self) {
+                return -1;
+        }
+        return self->size;
+}
+
+static bool nc_hashmap_resize(NcHashmap *self)
+{
+        if (!self || !self->buckets) {
+                return false;
+        }
+
+        NcHashmapEntry *old_buckets = self->buckets;
+        NcHashmapEntry *new_buckets = NULL;
+        NcHashmapEntry *entry = NULL;
+        int incr;
+
+        int old_size, new_size;
+        int items = 0;
+
+        new_size = old_size = self->n_buckets;
+        new_size *= INCREASE_FACTOR;
+
+        new_buckets = calloc((size_t)new_size, sizeof(NcHashmapEntry));
+        if (!new_buckets) {
+                return false;
+        }
+
+        for (int i = 0; i < old_size; i++) {
+                entry = &(old_buckets[i]);
+                while (entry) {
+                        if (entry->occ) {
+                                unsigned hash = nc_hashmap_get_hash(self, entry->hash);
+                                if ((incr = nc_hashmap_insert_bucket(self,
+                                                                     new_buckets,
+                                                                     new_size,
+                                                                     hash,
+                                                                     entry->hash,
+                                                                     entry->value)) > 0) {
+                                        items += incr;
+                                } else {
+                                        /* Likely a memory issue */
+                                        goto failure;
+                                }
+                        }
+                        entry = entry->next;
+                }
+        }
+        /* Successfully resized - do this separately because we need to
+         * guarantee old data is preserved */
+        for (int i = 0; i < old_size; i++) {
+                nc_hashmap_free_bucket(self, &(old_buckets[i]), false);
+        }
+
+        free(old_buckets);
+        self->n_buckets = new_size;
+        self->size = items;
+        self->buckets = new_buckets;
+
+        nc_hashmap_update_next_size(self);
+        return true;
+
+failure:
+        for (int i = 0; i < new_size; i++) {
+                nc_hashmap_free_bucket(self, &(new_buckets[i]), true);
+        }
+        free(new_buckets);
+        return false;
+}
+
+void nc_hashmap_iter_init(NcHashmap *map, NcHashmapIter *citer)
+{
+        _NcHashmapIter *iter = NULL;
+        if (!map || !citer) {
+                return;
+        }
+        iter = (_NcHashmapIter *)citer;
+        _NcHashmapIter it = {
+                .bucket = -1, .map = map, .item = NULL,
+        };
+        *iter = it;
+}
+
+bool nc_hashmap_iter_next(NcHashmapIter *citer, void **key, void **value)
+{
+        _NcHashmapIter *iter = NULL;
+        NcHashmapEntry *item = NULL;
+        NcHashmap *map = NULL;
+        int n_buckets = 0;
+
+        if (!citer) {
+                return false;
+        }
+
+        iter = (_NcHashmapIter *)citer;
+        map = iter->map;
+        if (!map) {
+                return false;
+        }
+        n_buckets = map->n_buckets;
+        item = iter->item;
+
+        for (;;) {
+                if (iter->bucket >= n_buckets) {
+                        if (item && !item->next) {
+                                return false;
+                        }
+                }
+                if (!item) {
+                        iter->bucket++;
+                        if (iter->bucket > n_buckets - 1) {
+                                return false;
+                        }
+                        item = &(map->buckets[iter->bucket]);
+                }
+                if (item) {
+                        if (item->occ) {
+                                goto success;
+                        }
+                        item = item->next;
+                }
+        }
+        return false;
+
+success:
+        iter->item = item->next;
+        if (key) {
+                *key = item->hash;
+        }
+        if (value) {
+                *value = item->value;
+        }
+
+        return true;
+}
+
+/*
+ * Editor modelines  -  https://www.wireshark.org/tools/modelines.html
+ *
+ * Local variables:
+ * c-basic-offset: 8
+ * tab-width: 8
+ * indent-tabs-mode: nil
+ * End:
+ *
+ * vi: set shiftwidth=8 tabstop=8 expandtab:
+ * :indentSize=8:tabSize=8:noTabs=true:
+ */

src/nica/hashmap.h

@@ -0,0 +1,241 @@
+/*
+ * This file is part of libnica.
+ *
+ * Copyright © 2016-2017 Intel Corporation
+ *
+ * libnica is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU Lesser General Public License as
+ * published by the Free Software Foundation; either version 2.1
+ * of the License, or (at your option) any later version.
+ *
+ * This is a chained Hashmap implementation. It focuses on being clean
+ * and efficient, and is comparable (at -O2) to open addressing hashmaps
+ * up until around 105,000 elements, where open addressing will begin
+ * to come out in front. At 1 million elements, open addressing is a clear
+ * winner, however currently we only need a maximum of 70k elements in
+ * our implementation.
+ *
+ * Given the memory required to even begin to deal with 1 million elements,
+ * it becomes very questionable whether a hashmap should have even been
+ * used in the first place (mmap'd db?)
+ */
+
+#pragma once
+
+#define _GNU_SOURCE
+
+#include <stdbool.h>
+#include <stdint.h>
+
+#include "macros.h"
+#include "util.h"
+
+/* Convert between uint and void* */
+#define NC_HASH_KEY(x) ((void *)((uintptr_t)(x)))
+#define NC_HASH_VALUE(x) NC_HASH_KEY(x)
+#define NC_UNHASH_KEY(x) ((unsigned int)((uintptr_t)(x)))
+#define NC_UNHASH_VALUE(x) NC_UNHASH_KEY(x)
+
+typedef struct NcHashmap NcHashmap;
+
+/**
+ * Iteration object
+ */
+typedef struct NcHashmapIter {
+        int n0;
+        void *n1;
+        void *n2;
+} NcHashmapIter;
+
+/**
+ * Hash comparison function definition
+ *
+ * @param l First value to compare
+ * @param r Second value to compare
+ *
+ * @return true if l and r both match, otherwise false
+ */
+typedef bool (*nc_hash_compare_func)(const void *l, const void *r);
+
+/**
+ * Hash creation function definition
+ *
+ * @param key Key to generate a hash for
+ *
+ * @return an unsigned integer hash result
+ */
+typedef unsigned (*nc_hash_create_func)(const void *key);
+
+/**
+ * Callback definition to free keys and values
+ *
+ * @param p Single-depth pointer to either a key or value that should be freed
+ */
+typedef void (*nc_hash_free_func)(void *p);
+
+/**
+ * Default hash/comparison functions
+ *
+ * @note These are only used for comparison of *keys*, not values. Unless
+ * explicitly using string keys, you should most likely stick with the default
+ * nc_simple_hash and nc_simple_compare functions
+ */
+
+/* Default string hash */
+static inline unsigned nc_string_hash(const void *key)
+{
+        unsigned hash = 5381;
+        const signed char *c;
+
+        /* DJB's hash function */
+        for (c = key; *c != '\0'; c++) {
+                hash = (hash << 5) + hash + (unsigned)*c;
+        }
+        return hash;
+}
+
+/**
+ * Trivial pointer->uint hash
+ */
+static inline unsigned nc_simple_hash(const void *source)
+{
+        return NC_UNHASH_KEY(source);
+}
+
+/**
+ * Comparison of string keys
+ */
+static inline bool nc_string_compare(const void *l, const void *r)
+{
+        if (!l || !r) {
+                return false;
+        }
+        return (strcmp(l, r) == 0);
+}
+
+/**
+ * Trivial pointer comparison
+ */
+static inline bool nc_simple_compare(const void *l, const void *r)
+{
+        return (l == r);
+}
+
+/**
+ * Create a new NcHashmap
+ *
+ * @param hash Hash creation function
+ * @param compare Key comparison function
+ *
+ * @return A newly allocated NcHashmap
+ */
+_nica_public_ NcHashmap *nc_hashmap_new(nc_hash_create_func hash, nc_hash_compare_func compare);
+
+/**
+ * Create a new NcHashmap with cleanup functions
+ *
+ * @param hash Hash creation function
+ * @param compare Key comparison function
+ * @param key_free Function to free keys when removed/destroyed
+ * @param value_free Function to free values when removed/destroyed
+ *
+ * @return A newly allocated NcHashmap
+ */
+_nica_public_ NcHashmap *nc_hashmap_new_full(nc_hash_create_func hash, nc_hash_compare_func compare,
+                                             nc_hash_free_func key_free,
+                                             nc_hash_free_func value_free);
+
+/**
+ * Store a key/value pair in the hashmap
+ *
+ * @note This will displace duplicate keys, and may free both the key
+ * and value if key_free and value_free are non null
+ *
+ * @param key Key to store in the hashmap
+ * @param value Value to be associated with the key
+ *
+ * @return true if the operation succeeded.
+ */
+_nica_public_ bool nc_hashmap_put(NcHashmap *map, const void *key, void *value);
+
+/**
+ * Get the value associated with the unique key
+ *
+ * @param key Unique key to obtain a value for
+ * @return The associated value if it exists, otherwise NULL
+ */
+_nica_public_ void *nc_hashmap_get(NcHashmap *map, const void *key);
+
+/**
+ * Determine if the key has an associated value in the NcHashmap
+ *
+ * @param key Unique key to check value for
+ * @return True if the key exists in the hashmap
+ */
+_nica_public_ bool nc_hashmap_contains(NcHashmap *self, const void *key);
+
+/**
+ * Free the given NcHashmap, and all keys/values if appropriate
+ */
+_nica_public_ void nc_hashmap_free(NcHashmap *map);
+
+/**
+ * Remove the value and key identified by key.
+ *
+ * @note If key_free or value_free are non-NULL, they will be invoked
+ * for both the key and value being removed
+ *
+ * @param key The unique key to remove
+ * @return true if the key/value pair were removed
+ */
+_nica_public_ bool nc_hashmap_remove(NcHashmap *map, const void *key);
+
+/**
+ * Remove the value and key identified by key without freeing them
+ *
+ * @return true if the key/value pair were stolen
+ */
+_nica_public_ bool nc_hashmap_steal(NcHashmap *map, const void *key);
+
+/**
+ * Return the current size of the hashmap
+ *
+ * @return size of the current hashmap (element count)
+ */
+_nica_public_ int nc_hashmap_size(NcHashmap *map);
+
+/**
+ * Initialise a NcHashmapIter for iteration purposes
+ *
+ * @note The iter *must* be re-inited to re-use, or if it becomes
+ * exhausted from a previous iteration run
+ *
+ * @param iter pointer to a NcHashmapIter
+ */
+_nica_public_ void nc_hashmap_iter_init(NcHashmap *map, NcHashmapIter *iter);
+
+/**
+ * Iterate every key/value pair in the hashmap
+ *
+ * @param iter A correctly initialised NcHashmapIter
+ * @param key Pointers to store the key in, may be NULL to skip
+ * @param value Pointer to store the value in, may be NULL to skip
+ *
+ * @return true if it's possible to iterate
+ */
+_nica_public_ bool nc_hashmap_iter_next(NcHashmapIter *iter, void **key, void **value);
+
+DEF_AUTOFREE(NcHashmap, nc_hashmap_free)
+
+/*
+ * Editor modelines  -  https://www.wireshark.org/tools/modelines.html
+ *
+ * Local variables:
+ * c-basic-offset: 8
+ * tab-width: 8
+ * indent-tabs-mode: nil
+ * End:
+ *
+ * vi: set shiftwidth=8 tabstop=8 expandtab:
+ * :indentSize=8:tabSize=8:noTabs=true:
+ */

src/nica/inifile.c

@@ -0,0 +1,386 @@
+/*
+ * This file is part of libnica.
+ *
+ * Copyright © 2016-2017 Intel Corporation
+ *
+ * libnica is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU Lesser General Public License as
+ * published by the Free Software Foundation; either version 2.1
+ * of the License, or (at your option) any later version.
+ */
+
+#define _GNU_SOURCE
+
+#include <errno.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+
+#include "hashmap.h"
+#include "inifile.h"
+
+/**
+ * Mapping of @NcIniError to static strings
+ */
+static const char *_errors[] =
+    {[NC_INI_ERROR_FILE] = "",
+     [NC_INI_ERROR_EMPTY_KEY] = "Encountered empty key",
+     [NC_INI_ERROR_NOT_CLOSED] = "Expected closing \']\' for section",
+     [NC_INI_ERROR_NO_SECTION] = "Encountered key=value mapping without a valid section",
+     [NC_INI_ERROR_INVALID_LINE] = "Expected key=value notation",
+     [NC_INI_ERROR_INTERNAL] = "A fatal internal error was encountered" };
+
+const char *nc_ini_error(enum NcIniError error)
+{
+        int j = abs(error);
+        if (j < NC_INI_ERROR_FILE || j >= NC_INI_ERROR_MAX) {
+                return "[Unknown Error]";
+        }
+        return _errors[j];
+}
+
+/**
+ * Strip leading whitespace from string (left)
+ */
+static char *lstrip(char *str, ssize_t len, ssize_t *out_len)
+{
+        ssize_t skip_len = 0;
+
+        for (ssize_t i = 0; i < len; i++) {
+                if (str[i] == ' ' || str[i] == '\t') {
+                        ++skip_len;
+                        continue;
+                }
+                break;
+        }
+        *out_len = len - skip_len;
+        if (skip_len > 0) {
+                char *c = strdup(str + skip_len);
+                free(str);
+                return c;
+        }
+        return str + skip_len;
+}
+
+/**
+ * Strip trailing whitespace from string (right)
+ */
+static char *rstrip(char *str, ssize_t len, ssize_t *out_len)
+{
+        ssize_t skip_len = 0;
+
+        for (ssize_t i = len; i > 0; i--) {
+                if (str[i] == ' ' || str[i] == '\t') {
+                        ++skip_len;
+                        continue;
+                } else if (str[i] == '\0') {
+                        continue;
+                }
+                break;
+        }
+        *out_len = len - skip_len;
+        if (len == skip_len) {
+                return str;
+        }
+        str[(len - skip_len)] = '\0';
+        return str;
+}
+
+/**
+ * Chew both ends of a null terminated string of whitespace
+ * If the left side is whitespace padded, this will always result
+ * in a new allocation due to fast-forwarding the pointer. Keep
+ * this in mind.
+ */
+static char *string_chew_terminated(char *inp)
+{
+        int skip_offset = 0;
+        int len = 0;
+        int end_len = 0;
+        bool count = true;
+
+        if (!inp) {
+                return NULL;
+        }
+
+        for (char *c = inp; *c; c++) {
+                if (count && (*c == ' ' || *c == '\t')) {
+                        ++skip_offset;
+                        continue;
+                } else {
+                        count = false;
+                }
+                ++len;
+        }
+        for (int i = len; i > skip_offset; i--) {
+                if (inp[i] == ' ' || inp[i] == '\t') {
+                        ++end_len;
+                        continue;
+                } else if (inp[i] == '\0') {
+                        continue;
+                }
+                break;
+        }
+
+        if (end_len > 0) {
+                inp[(len - end_len)] = '\0';
+        }
+        if (skip_offset > 0) {
+                char *c = strdup(inp + skip_offset);
+                free(inp);
+                return c;
+        }
+        return inp;
+}
+
+/**
+ * Munch both ends of the string
+ */
+static char *string_strip(char *str, ssize_t len, ssize_t *out_len)
+{
+        ssize_t mylen = len;
+
+        char *c = lstrip(str, len, &mylen);
+        c = rstrip(c, mylen, &mylen);
+        if (out_len) {
+                *out_len = mylen;
+        }
+        return c;
+}
+
+/* eat \s */
+static char *string_unescape(char *str)
+{
+	char *c, *c1, *c2, *c3;;
+
+	if (!str)
+		return NULL;
+
+	c = calloc(strlen(str) + 1, 1);
+	c1 = c;
+
+	c2 = str;
+	while (c2 && *c2 != 0) {
+		c3 = c2 + 1;
+		if (*c2 == '\\') {
+			if (*c3 == 's') {
+				*c = ' ';
+				c++;
+				c2 += 2;
+			} else {
+				c2++;
+			}
+		} else {
+			*c = *c2;
+			c++;
+			c2++;
+		}
+	}
+
+	free(str);	
+        return c1;
+}
+
+NcHashmap *nc_ini_file_parse(const char *path)
+{
+        NcHashmap *ret = NULL;
+        int error_line = 0;
+        int r = 0;
+
+        r = nc_ini_file_parse_full(path, &ret, &error_line);
+        if (r != 0) {
+                if (abs(r) == NC_INI_ERROR_FILE) {
+                        fprintf(stderr, "[inifile] %s: %s\n", strerror(errno), path);
+                } else {
+                        fprintf(stderr,
+                                "[inifile] %s [L%d]: %s\n",
+                                nc_ini_error(r),
+                                error_line,
+                                path);
+                }
+                return NULL;
+        }
+        return ret;
+}
+
+int nc_ini_file_parse_full(const char *path, NcHashmap **out_map, int *error_line_number)
+{
+        autofree(FILE) *file = NULL;
+        char *buf = NULL;
+        ssize_t r = 0;
+        size_t sn = 0;
+        int line_count = 1;
+        char *current_section = NULL;
+        NcHashmap *root_map = NULL;
+        NcHashmap *section_map = NULL;
+        bool failed = false;
+        int err_ret = 0;
+
+        file = fopen(path, "r");
+        if (!file) {
+                return -(NC_INI_ERROR_FILE);
+        }
+
+        if (!out_map) {
+                fprintf(stderr, "nc_ini_file_parse_full(): NcHashmap pointer invalid\n");
+                return -1;
+        }
+
+        root_map = nc_hashmap_new_full(nc_string_hash,
+                                       nc_string_compare,
+                                       free,
+                                       (nc_hash_free_func)nc_hashmap_free);
+
+        while ((r = getline(&buf, &sn, file)) != -1) {
+                char *ch = NULL;
+                ssize_t str_len = r;
+                char *key = NULL;
+                char *value = NULL;
+
+                /* Fix newline */
+                if (buf[r - 1] == '\n') {
+                        buf[r - 1] = '\0';
+                        --r;
+                }
+                str_len = r;
+
+                buf = string_strip(buf, r, &str_len);
+                /* Empty lines are fine */
+                if (streq(buf, "")) {
+                        goto next;
+                }
+
+                if (buf[0] == '[') {
+                        /* Validate section start */
+                        if (buf[str_len - 1] != ']') {
+                                /* Throw error */
+                                err_ret = NC_INI_ERROR_NOT_CLOSED;
+                                goto fail;
+                        }
+                        /* Grab the section name, and "close" last section */
+                        buf[str_len - 1] = '\0';
+                        if (current_section) {
+                                free(current_section);
+                        }
+                        current_section = strdup(buf + 1);
+                        section_map = nc_hashmap_get(root_map, current_section);
+                        if (!section_map) {
+                                /* Create a new section dynamically */
+                                section_map = nc_hashmap_new_full(nc_string_hash,
+                                                                  nc_string_compare,
+                                                                  free,
+                                                                  free);
+                                nc_hashmap_put(root_map, strdup(current_section), section_map);
+                        }
+                        goto next;
+                } else if (buf[0] == '#' || buf[0] == ';') {
+                        /* Skip comment */
+                        goto next;
+                }
+
+                /* Look for key = value */
+                ch = strchr(buf, '=');
+                if (!ch) {
+                        /* Throw error */
+                        err_ret = NC_INI_ERROR_INVALID_LINE;
+                        goto fail;
+                }
+
+                /* Can't have sectionless k->v */
+                if (!current_section) {
+                        err_ret = NC_INI_ERROR_NO_SECTION;
+                        goto fail;
+                }
+
+                long int offset = ch - buf;
+
+                /* Grab the key->value from this assignment line */
+                value = strdup((buf + offset) + 1);
+                buf[offset] = '\0';
+                key = strdup(buf);
+                key = string_chew_terminated(key);
+                value = string_chew_terminated(value);
+		value = string_unescape(value);
+
+                if (streq(key, "")) {
+                        err_ret = NC_INI_ERROR_EMPTY_KEY;
+                        free(key);
+                        key = NULL;
+                        free(value);
+                        value = NULL;
+                        goto fail;
+                }
+
+                /* Ensure a section mapping exists */
+                section_map = nc_hashmap_get(root_map, current_section);
+                if (!section_map) {
+                        err_ret = NC_INI_ERROR_INTERNAL;
+                        fprintf(stderr,
+                                "[inifile] Fatal! No section map for named "
+                                "section: %s\n",
+                                current_section);
+                        goto fail;
+                }
+
+                /* Insert these guys into the map */
+                if (!nc_hashmap_put(section_map, key, value)) {
+                        err_ret = NC_INI_ERROR_INTERNAL;
+                        fprintf(stderr, "[inifile] Fatal! Out of memory\n");
+                        goto fail;
+                }
+        /* Progression + cleanup */
+        next:
+                if (buf) {
+                        free(buf);
+                        buf = NULL;
+                }
+                ++line_count;
+                sn = 0;
+                continue;
+        /* Parsing error, bail */
+        fail:
+                failed = true;
+                if (error_line_number) {
+                        *error_line_number = line_count;
+                }
+                if (key) {
+                        free(key);
+                }
+                if (value) {
+                        free(value);
+                }
+                break;
+        }
+
+        if (buf) {
+                free(buf);
+                buf = NULL;
+        }
+
+        if (current_section) {
+                free(current_section);
+                current_section = NULL;
+        }
+
+        if (failed) {
+                if (root_map) {
+                        nc_hashmap_free(root_map);
+                }
+                return -(err_ret);
+        }
+        *out_map = root_map;
+        return 0;
+}
+
+/*
+ * Editor modelines  -  https://www.wireshark.org/tools/modelines.html
+ *
+ * Local variables:
+ * c-basic-offset: 8
+ * tab-width: 8
+ * indent-tabs-mode: nil
+ * End:
+ *
+ * vi: set shiftwidth=8 tabstop=8 expandtab:
+ * :indentSize=8:tabSize=8:noTabs=true:
+ */

src/nica/inifile.h

@@ -0,0 +1,114 @@
+/*
+ * This file is part of libnica.
+ *
+ * Copyright © 2016-2017 Intel Corporation
+ *
+ * libnica is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU Lesser General Public License as
+ * published by the Free Software Foundation; either version 2.1
+ * of the License, or (at your option) any later version.
+ */
+
+#pragma once
+
+#include "hashmap.h"
+#include "macros.h"
+#include "util.h"
+
+/**
+ * The Nica INI Parser focuses on simplicity and ease of use, therefore it
+ * currently only supports a read-only parse operation.
+ *
+ * The parse functions are used to return a _root_ @NcHashmap which contains
+ * name to section mappings. Each section is in turn an @NcHashmap with string
+ * to string mappings.
+ *
+ * Consider the following example:
+ *
+ *      [Person]
+ *      alive = true
+ *      ; This person is alive
+ *
+ * The returned @NcHashmap will contain the key "Person", which corresponds to
+ * another @NcHashmap. This @NcHashmap will contain the key "alive", which has
+ * the value "true". Therefore we can access this variable using the @NcHashmap
+ * API:
+ *
+ *      char *alive = nc_hashmap_get(nc_hashmap_get(ini, "Person"), "alive");
+ *
+ * Conversely, we can use the API to determine whether a section or key is set:
+ *
+ *      if (nc_hashmap_contains(ini, "Person"))
+ *
+ * As @NcHashmap returns NULL in nc_hashmap_get, you can happily pass NULL
+ * keys, allowing quick key fetches without checking if the section exists.
+ *
+ * In our INI file, lines beginning with ";" or "#" are considered as comments
+ * and are not processed. A line following the "key = value" notation is an
+ * _assignment_. "[Section]" is considered to be a section in the INI file, and
+ * all INI files must have at least one section.
+ *
+ * We do allow duplication section definitions, by following a merge policy. If
+ * a key is redefined then the original key->value mapping is freed and the new
+ * key->value mapping is inserted.
+ *
+ * The parser will return a correctly configured @NcHashmap, the only cleanup
+ * required is to call nc_hashmap_free on it. Alternatively, use the autofree
+ * helper, for a RAII approach:
+ *
+ *      autofree(NcHashmap) *map = nc_ini_file_parse("myfile.ini");
+ */
+
+/**
+ * Errors that are reported when parsing an ini file
+ */
+enum NcIniError {
+        NC_INI_ERROR_MIN = 0,
+        NC_INI_ERROR_FILE,         /**< File based error, check strerror(errno) */
+        NC_INI_ERROR_EMPTY_KEY,    /**< Empty key in an assignment line */
+        NC_INI_ERROR_NOT_CLOSED,   /**< Encountered section start that wasn't closed with a ']' */
+        NC_INI_ERROR_NO_SECTION,   /**< Key assignment with no defined sections */
+        NC_INI_ERROR_INVALID_LINE, /**< Encountered an invalid line (syntax) */
+        NC_INI_ERROR_INTERNAL,     /**< Fatal internal error encountered. Caller should exit */
+        NC_INI_ERROR_MAX
+};
+/**
+
+ * Convenience wrapper for nc_ini_file_parse_full, reports errors to stderrr
+ * and returns an NcHashmap if the parsing succeeded
+ */
+_nica_public_ NcHashmap *nc_ini_file_parse(const char *path);
+
+/**
+ * Parse an INI file into a hash of hashes.
+ *
+ * @param path Path on the filesystem to parse, must be in INI syntax.
+ * @param out_map Pointer to store the resulting root NcHashmap in
+ * @param error_line_number If not null, then the erronous line number is
+ * stored.
+ *
+ * @return 0 if the call was successful, or a negative integer. See nc_ini_error
+ */
+_nica_public_ int nc_ini_file_parse_full(const char *path, NcHashmap **out_map,
+                                         int *error_line_number);
+
+/**
+ * Return a string representation for a given @NcIniError
+ *
+ * @param error Valid NcIniError code
+ * @return a static string owned by the implementation
+ */
+_nica_public_ const char *nc_ini_error(enum NcIniError error);
+
+/*
+ * Editor modelines  -  https://www.wireshark.org/tools/modelines.html
+ *
+ * Local variables:
+ * c-basic-offset: 8
+ * tab-width: 8
+ * indent-tabs-mode: nil
+ * End:
+ *
+ * vi: set shiftwidth=8 tabstop=8 expandtab:
+ * :indentSize=8:tabSize=8:noTabs=true:
+ */

src/nica/local.mk

@@ -1,3 +1,6 @@
 noinst_HEADERS += \
+	%D%/hashmap.h \
+	%D%/inifile.h \
 	%D%/macros.h \
-	%D%/nc-string.h
+	%D%/nc-string.h \
+	%D%/util.h

src/nica/util.h

@@ -0,0 +1,69 @@
+/*
+ * This file is part of libnica.
+ *
+ * Copyright © 2016-2017 Intel Corporation
+ *
+ * libnica is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU Lesser General Public License as
+ * published by the Free Software Foundation; either version 2.1
+ * of the License, or (at your option) any later version.
+ */
+
+#pragma once
+
+#define _GNU_SOURCE
+
+#include <stdbool.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+
+#include <nica/macros.h>
+
+#define DEF_AUTOFREE(N, C)                                                                         \
+        static inline void _autofree_func_##N(void *p)                                             \
+        {                                                                                          \
+                if (p && *(N **)p) {                                                               \
+                        C(*(N **)p);                                                               \
+                        (*(void **)p) = NULL;                                                      \
+                }                                                                                  \
+        }
+
+#define autofree(N) __attribute__((cleanup(_autofree_func_##N))) N
+
+/**
+ * Dump any leaked file descriptors
+ */
+_nica_public_ void nc_dump_file_descriptor_leaks(void);
+
+_nica_public_ void *greedy_realloc(void **p, size_t *allocated, size_t need);
+
+/**
+ * Test string equality of two null terminated strings
+ */
+__attribute__((always_inline)) static inline bool streq(const char *a, const char *b)
+{
+        if (a == b) {
+                return true;
+        }
+        if (!a || !b) {
+                return false;
+        }
+        return strcmp(a, b) == 0;
+}
+
+DEF_AUTOFREE(char, free)
+DEF_AUTOFREE(FILE, fclose)
+
+/*
+ * Editor modelines  -  https://www.wireshark.org/tools/modelines.html
+ *
+ * Local variables:
+ * c-basic-offset: 8
+ * tab-width: 8
+ * indent-tabs-mode: nil
+ * End:
+ *
+ * vi: set shiftwidth=8 tabstop=8 expandtab:
+ * :indentSize=8:tabSize=8:noTabs=true:
+ */

src/probes/crash_probe.c

@@ -68,6 +68,10 @@ static char clr_build_class[40] = "org.clearlinux/crash/clr-build";
 static char error_class[30] = "org.clearlinux/crash/error";
 static char unknown_class[30] = "org.clearlinux/crash/unknown";
 
+static char temp_core[] = "/tmp/corefile-XXXXXX";
+static bool keep_core = false;
+
+
 static const Dwfl_Callbacks cb =
 {
         .find_elf = dwfl_build_id_find_elf,
@@ -147,23 +151,14 @@ static int temp_core_file(void)
         int tmp;
         ssize_t ret;
 
-        char core[PATH_MAX] = "/tmp/corefile-XXXXXX";
-
         /* mkstemp() opens the file with O_EXCL and 0600 permissions, so no need
          * to change umask or manipulate the fd to meet those requirements.
          */
-        if ((tmp = mkstemp(core)) < 0) {
+        if ((tmp = mkstemp(temp_core)) < 0) {
                 telem_perror("Failed to create temp core file");
                 return -1;
         }
 
-#ifndef DEBUG
-        if (unlink(core) < 0) {
-                telem_perror("Failed to unlink temp core file");
-                return -1;
-        }
-#endif
-
         while (true) {
                 // Use Linux-specific splice(2) here;
                 // simplifies copying data from pipe->file
@@ -416,6 +411,18 @@ fail:
         return -1;
 }
 
+static bool startswith(const char *full, const char *prefix)
+{
+        while (*prefix) {
+                if (*prefix != *full) {
+                        return false;
+                }
+                full++;
+                prefix++;
+        }
+        return true;
+}
+
 static bool in_clr_build(char *fullpath)
 {
         // Global override for privacy filters
@@ -427,8 +434,8 @@ static bool in_clr_build(char *fullpath)
          * The build environment for Clear Linux packages is set up by 'mock',
          * and the chroot in which rpmbuild builds the packages has this prefix.
          */
-        if ((strstr(fullpath, "/builddir/build/BUILD/")) ||
-            (strstr(fullpath, "!builddir!build!BUILD!"))) {
+        if (startswith(fullpath, "/builddir/build/BUILD/") ||
+            startswith(fullpath, "!builddir!build!BUILD!")) {
                 return true;
         }
 
@@ -444,13 +451,13 @@ static bool is_banned_path(char *fullpath)
 
         // Anything outside of /usr/, or in /usr/local/, we consider third-party
 
-        if ((strncmp(fullpath, "/usr/", 5) != 0) &&
-            (strncmp(fullpath, "!usr!", 5) != 0)) {
+        if (!startswith(fullpath, "/usr/") &&
+            !startswith(fullpath, "!usr!")) {
                 return true;
         }
 
-        if ((strncmp(fullpath, "/usr/local/", 11) == 0) ||
-            (strncmp(fullpath, "!usr!local!", 11) == 0)) {
+        if (startswith(fullpath, "/usr/local/") ||
+            startswith(fullpath, "!usr!local!")) {
                 return true;
         }
 
@@ -563,28 +570,6 @@ int main(int argc, char **argv)
                 exit(EXIT_FAILURE);
         }
 
-        if (proc_path && in_clr_build(proc_path)) {
-                telem_log(LOG_NOTICE, "Ignoring core (from mock build)\n");
-
-                backtrace = nc_string_dup("Crash from Clear package build\n");
-
-                if (!send_data(&backtrace, unknown_severity, clr_build_class)) {
-                        goto fail;
-                }
-                goto success;
-        }
-
-        if (proc_path && is_banned_path(proc_path)) {
-                telem_log(LOG_NOTICE, "Ignoring core (third-party binary)\n");
-
-                backtrace = nc_string_dup("Crash from third party\n");
-
-                if (!send_data(&backtrace, unknown_severity, unknown_class)) {
-                        goto fail;
-                }
-                goto success;
-        }
-
         if (core_file) {
                 core_fd = open(core_file, O_RDONLY);
                 if (core_fd == -1) {
@@ -617,6 +602,32 @@ int main(int argc, char **argv)
                 }
         }
 
+        if (proc_path && in_clr_build(proc_path)) {
+                telem_log(LOG_NOTICE, "Ignoring core (from mock build)\n");
+
+                backtrace = nc_string_dup("Crash from Clear package build\n");
+
+                keep_core = true;
+
+                if (!send_data(&backtrace, unknown_severity, clr_build_class)) {
+                        goto fail;
+                }
+                goto success;
+        }
+
+        if (proc_path && is_banned_path(proc_path)) {
+                telem_log(LOG_NOTICE, "Ignoring core (third-party binary)\n");
+
+                backtrace = nc_string_dup("Crash from third party\n");
+
+                keep_core = true;
+
+                if (!send_data(&backtrace, unknown_severity, unknown_class)) {
+                        goto fail;
+                }
+                goto success;
+        }
+
         elf_version(EV_CURRENT);
 
         if (prepare_corefile(&e_core, core_fd) < 0) {
@@ -667,6 +678,11 @@ success:
 
         ret = EXIT_SUCCESS;
 fail:
+        // Do not remove the core file if any errors occur
+        if (ret == EXIT_FAILURE) {
+                keep_core = true;
+        }
+
         free(core_file);
         free(proc_name);
         free(proc_path);
@@ -695,6 +711,12 @@ fail:
                 close(core_fd);
         }
 
+        // Remove the core file by default, except when the --core-file option
+        // is specified, or when keep_core is overridden to true.
+        if (!core_file && !keep_core) {
+                unlink(temp_core);
+        }
+
         return ret;
 }
 

src/probes/journal.c

@@ -38,6 +38,7 @@
 #include "config.h"
 #include "log.h"
 #include "telemetry.h"
+#include "probe.h"
 #include "nica/nc-string.h"
 #define BOOT_ID_LEN 33
 
@@ -227,9 +228,14 @@ static bool add_filters(sd_journal *journal)
         /* The semantics of how journal entry matching works is described in
          * detail in sd_journal_add_match(3).
          *
-         * The matches declared here correspond to the logical expression:
+         * When the privacy filter override is enabled, the matches declared
+         * here correspond to this logical expression:
          *
-         * BOOTID && ((P0 || P1 || P2 || P3) || EXITED)
+         *   BOOTID && ((P0 || P1 || P2 || P3) || EXITED)
+         *
+         * Otherwise, the expression is:
+         *
+         *   BOOTID && EXITED
          *
          * BOOTID is short for _BOOT_ID=VAL, where VAL is the boot ID for the
          * current boot. P0, P1, etc stand for PRIORITY=0, etc. And EXITED is
@@ -239,12 +245,20 @@ static bool add_filters(sd_journal *journal)
         JOURNAL_MATCH(data);
         free(data);
         JOURNAL_AND;
-        // The four highest log levels, all indicating errors
-        JOURNAL_MATCH("PRIORITY=0");
-        JOURNAL_MATCH("PRIORITY=1");
-        JOURNAL_MATCH("PRIORITY=2");
-        JOURNAL_MATCH("PRIORITY=3");
-        JOURNAL_OR;
+
+        // Filter messages with the four highest log levels, all indicating
+        // errors, but only when the privacy filters override is in effect;
+        // because the log messages contain arbitrary strings, and this probe
+        // does not yet keep a whitelist of allowed patterns or a blacklist of
+        // banned patterns.
+        if (access(TM_PRIVACY_FILTERS_OVERRIDE, F_OK) == 0) {
+                JOURNAL_MATCH("PRIORITY=0");
+                JOURNAL_MATCH("PRIORITY=1");
+                JOURNAL_MATCH("PRIORITY=2");
+                JOURNAL_MATCH("PRIORITY=3");
+                JOURNAL_OR;
+        }
+
         // Only set for service-level error conditions
         JOURNAL_MATCH("EXIT_CODE=exited");
 

src/probes/klog_main.c

@@ -23,7 +23,6 @@ int main(void)
         char *bufp = NULL;
         size_t buflen = 0;
         int bytes = 0;
-        int loopcount = 0;
 
         oops_parser_init(write_oops_to_file);
 
@@ -45,13 +44,13 @@ int main(void)
                 // Gets the size of the kernel ring buffer
                 log_size = klogctl(SYSLOG_ACTION_SIZE_BUFFER, NULL, 0);
                 if (log_size < 0) {
-                        telem_log(LOG_ERR, "Cannot read size of kernel ring buffer\n");
+                        telem_log(LOG_ERR, "Cannot read size of kernel ring buffer: %s\n", strerror(errno));
                         return 1;
                 }
 
                 buflen = (size_t)log_size;
 
-                if (loopcount++ > 0 && buflen > MAX_BUF) {
+                if (buflen > MAX_BUF) {
                         buflen = MAX_BUF;
                 }
 
@@ -59,7 +58,7 @@ int main(void)
                 bufp = (char *)calloc(buflen, sizeof(char));
 
                 malloc_trim(0);
-                bytes = klogctl(SYSLOG_ACTION_READ, bufp, log_size);
+                bytes = klogctl(SYSLOG_ACTION_READ, bufp, (int)buflen);
                 if (bytes < 0) {
                         telem_log(LOG_ERR, "Cannot read contents of kernel ring buffer: %s\n", strerror(errno));
                         return 1;

src/probes/local.mk

@@ -15,10 +15,8 @@ bin_PROGRAMS += \
 
 %C%_telem_record_gen_SOURCES = %D%/telem_record_gen.c
 %C%_telem_record_gen_CFLAGS = \
-	$(AM_CFLAGS) \
-	$(GLIB_CFLAGS)
-%C%_telem_record_gen_LDADD = $(top_builddir)/src/libtelemetry.la \
-	$(GLIB_LIBS)
+	$(AM_CFLAGS)
+%C%_telem_record_gen_LDADD = $(top_builddir)/src/libtelemetry.la
 %C%_telem_record_gen_LDFLAGS = \
 	$(AM_LDFLAGS) \
 	-pie
@@ -48,13 +46,11 @@ endif
 	src/nica/nc-string.c \
 	%D%/probe.h
 %C%_crashprobe_CFLAGS = \
-	$(AM_CFLAGS) \
-	$(GLIB_CFLAGS)
+	$(AM_CFLAGS)
 %C%_crashprobe_LDADD = \
 	$(top_builddir)/src/libtelemetry.la \
 	$(top_builddir)/src/libtelem-shared.la \
-	@ELFUTILS_LIBS@ \
-	$(GLIB_LIBS)
+	@ELFUTILS_LIBS@
 %C%_crashprobe_LDFLAGS = \
        $(AM_LDFLAGS) \
        -pie
@@ -73,12 +69,10 @@ endif
 	src/nica/nc-string.c \
 	%D%/oops_parser.c
 %C%_pstoreprobe_CFLAGS = \
-	$(AM_CFLAGS) \
-	$(GLIB_CFLAGS)
+	$(AM_CFLAGS)
 %C%_pstoreprobe_LDADD = \
 	$(top_builddir)/src/libtelemetry.la \
-	$(top_builddir)/src/libtelem-shared.la \
-	$(GLIB_LIBS)
+	$(top_builddir)/src/libtelem-shared.la
 
 if LOG_SYSTEMD
 if HAVE_SYSTEMD_JOURNAL
@@ -97,11 +91,9 @@ endif
 	src/nica/nc-string.c \
 	%D%/oops_parser.c
 %C%_klogscanner_CFLAGS = \
-        $(AM_CFLAGS) \
-	$(GLIB_CFLAGS)
+        $(AM_CFLAGS)
 %C%_klogscanner_LDADD = \
-        $(top_builddir)/src/libtelemetry.la \
-	$(GLIB_LIBS)
+        $(top_builddir)/src/libtelemetry.la
 %C%_klogscanner_LDFLAGS = \
         $(AM_LDFLAGS) \
         -pie
@@ -128,12 +120,10 @@ endif
 	src/nica/nc-string.c \
 	%D%/probe.h
 %C%_oopsprobe_CFLAGS = \
-        $(AM_CFLAGS) \
-        $(GLIB_CFLAGS)
+        $(AM_CFLAGS)
 %C%_oopsprobe_LDADD = \
         $(top_builddir)/src/libtelemetry.la \
-        $(top_builddir)/src/libtelem-shared.la \
-        $(GLIB_LIBS)
+        $(top_builddir)/src/libtelem-shared.la
 
 if LOG_SYSTEMD
 if HAVE_SYSTEMD_JOURNAL
@@ -154,13 +144,11 @@ bin_PROGRAMS += \
 	%D%/journal.c
 %C%_journalprobe_CFLAGS = \
 	$(AM_CFLAGS) \
-	$(GLIB_CFLAGS) \
 	$(SYSTEMD_ID128_CFLAGS) \
 	$(SYSTEMD_JOURNAL_CFLAGS)
 %C%_journalprobe_LDADD = \
 	$(top_builddir)/src/libtelemetry.la \
 	$(top_builddir)/src/libtelem-shared.la \
-	$(GLIB_LIBS) \
 	$(SYSTEMD_ID128_LIBS) \
 	$(SYSTEMD_JOURNAL_LIBS)
 %C%_journalprobe_LDFLAGS = \

src/probes/pstore_probe.c

@@ -16,7 +16,6 @@
 #include <stdio.h>
 #include <stdlib.h>
 #include <dirent.h>
-#include <glib.h>
 #include <string.h>
 #include <errno.h>
 #include <assert.h>
@@ -26,6 +25,7 @@
 #include "log.h"
 #include "telemetry.h"
 #include "oops_parser.h"
+#include "nica/hashmap.h"
 
 char *pstore_dump_path = PSTOREDIR;
 
@@ -202,8 +202,8 @@ int main(int argc, char **argv)
         DIR *pstore_dir;
         struct dirent *entry;
         struct chunk_list  *elem, *head;
-        GHashTableIter iter;
-        gpointer key, value;
+        NcHashmapIter iter;
+        void *key, *value;
         int max_part;
         size_t totalsize = 0;
         char *crash_dump = NULL;
@@ -212,7 +212,7 @@ int main(int argc, char **argv)
          * Hash table used to store chunks belonging to a oops counter
          *  key -> count, value->head of linked list of chunks
          */
-        GHashTable *hash = g_hash_table_new(g_int_hash, g_int_equal);
+        NcHashmap *hash = nc_hashmap_new(nc_simple_hash, nc_simple_compare);
 
         pstore_dir = opendir(pstore_dump_path);
         if (pstore_dir == NULL) {
@@ -250,23 +250,20 @@ int main(int argc, char **argv)
                 elem->contents = read_contents(entry->d_name, &(elem->size));
                 elem->next = NULL;
 
-                gpointer head = g_hash_table_lookup(hash, (void *)(&elem->count));
+                void *head = nc_hashmap_get(hash, NC_HASH_KEY(elem->count));
                 if (head != NULL) {
                         elem->next = (struct chunk_list *)head;
                 }
 
-                gint *key = g_new(gint, 1);
-                *key = elem->count;
-                g_hash_table_insert(hash, key, elem);
-
+                nc_hashmap_put(hash, NC_HASH_KEY(elem->count), elem);
         }
 
         closedir(pstore_dir);
 
-        g_hash_table_iter_init (&iter, hash);
-        while (g_hash_table_iter_next (&iter, &key, &value)) {
+        nc_hashmap_iter_init(hash, &iter);
+        while (nc_hashmap_iter_next(&iter, (void **)&key, (void **)&value)) {
                 #ifdef DEBUG
-                printf("Count in the hash: %d\n", *((int *)key));
+                printf("Count in the hash: %d\n", NC_UNHASH_KEY(key));
                 #endif
 
                 head = (struct chunk_list *)value;

src/probes/telem_record_gen.c

@@ -150,7 +150,7 @@ fail:
 
 int validate_opts(void)
 {
-        size_t i;
+        size_t len;
         int ret = 0;
 
         /* classification */
@@ -159,15 +159,15 @@ int validate_opts(void)
                 return ret;
 
         }
-        i = strlen(opt_class);
+        len = strlen(opt_class);
 
-        if ((i == 0) || (i > 120)) {
+        if ((len == 0) || (len > 120)) {
                 fprintf(stderr, "Error: Valid size for classification "
                         "is 1-120 chars\n");
                 return ret;
         }
 
-        for (int c = 0; c < strlen(opt_class); c++) {
+        for (int c = 0; c < len; c++) {
                 if (isascii(opt_class[c]) == 0) {
                         fprintf(stderr, "Error: Non-ascii characters detected "
                                 "in classification - aborting\n");

src/telemdaemon.c

@@ -462,6 +462,11 @@ bool post_record_http(char *headers[], char *body, bool spool)
         const char *cert_file = get_cainfo_config();
         const char *tid_header = get_tidheader_config();
 
+        // Initialize the libcurl global environment once per POST. This lets us
+        // clean up the environment after each POST so that when the daemon is
+        // sitting idle, it will be consuming as little memory as possible.
+        curl_global_init(CURL_GLOBAL_ALL);
+
         curl = curl_easy_init();
         if (!curl) {
                 telem_log(LOG_ERR, "curl_easy_init(): Unable to start libcurl"
@@ -532,6 +537,9 @@ bool post_record_http(char *headers[], char *body, bool spool)
 
         curl_slist_free_all(custom_headers);
         curl_easy_cleanup(curl);
+
+        curl_global_cleanup();
+
         return res ? false : true;
 }
 

tests/local.mk

@@ -24,12 +24,10 @@ dist_check_SCRIPTS = \
 
 %C%_check_config_CFLAGS = \
 	$(AM_CFLAGS) \
-	@CHECK_CFLAGS@ \
-	@GLIB_CFLAGS@ 
+	@CHECK_CFLAGS@
 
 %C%_check_config_LDADD = \
 	@CHECK_LIBS@ \
-	@GLIB_LIBS@ \
 	$(top_builddir)/src/libtelem-shared.la
 
 %C%_check_daemon_SOURCES = \
@@ -40,11 +38,9 @@ dist_check_SCRIPTS = \
 %C%_check_daemon_CFLAGS = \
 	$(AM_CFLAGS) \
 	@CHECK_CFLAGS@ \
-	@GLIB_CFLAGS@ \
 	@CURL_CFLAGS@
 %C%_check_daemon_LDADD = \
 	@CHECK_LIBS@ \
-	@GLIB_LIBS@ \
 	@CURL_LIBS@ \
 	$(top_builddir)/src/libtelem-shared.la
 
@@ -96,11 +92,9 @@ EXTRA_DIST += \
 
 %C%_check_probes_CFLAGS = \
         $(AM_CFLAGS) \
-        @CHECK_CFLAGS@ \
-        @GLIB_CFLAGS@
+        @CHECK_CFLAGS@
 %C%_check_probes_LDADD = \
         @CHECK_LIBS@ \
-        @GLIB_LIBS@ \
         $(top_builddir)/src/libtelem-shared.la
 
 if LOG_SYSTEMD
@@ -118,11 +112,9 @@ endif
 
 %C%_check_libtelemetry_CFLAGS = \
 	$(AM_CFLAGS) \
-	@CHECK_CFLAGS@ \
-	@GLIB_CFLAGS@
+	@CHECK_CFLAGS@
 %C%_check_libtelemetry_LDADD = \
 	@CHECK_LIBS@ \
-        @GLIB_LIBS@ \
 	$(top_builddir)/src/libtelemetry.la \
 	$(top_builddir)/src/libtelem-shared.la