Release v1.11.2

This release includes a couple of fixes for the man pages. Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>
Fix wrong reference to the config file in man pages.
2026-09-01 11:15:51 +00:00 · 2017-04-06 09:59:36 -07:00 · 2017-04-06 09:57:50 -07:00 · 2017-04-06 09:57:50 -07:00 · 2017-03-22 10:49:26 -07:00 · 2017-03-22 10:48:54 -07:00
36 changed files with 1512 additions and 250 deletions
@@ -14,6 +14,7 @@ DISTCHECK_CONFIGURE_FLAGS = \

 include $(top_srcdir)/build-aux/make/cflags.make
 include $(top_srcdir)/build-aux/make/ldflags.make
+include $(top_srcdir)/docs/local.mk
 include $(top_srcdir)/scripts/local.mk
 include $(top_srcdir)/src/local.mk
 include $(top_srcdir)/src/data/local.mk
@@ -30,9 +30,3 @@ Build dependencies
 - (optional) libsystemd, for syslog-style logging to the systemd journal, and
  socket/path activation of telemd by systemd.

-
-Generating HTML documentation
-=============================
-
-At toplevel, run 'doxygen docs/Doxyfile'. The documentation is generated in
-the docs/html/ directory.
@@ -2,7 +2,7 @@
 # Process this file with autoconf to produce a configure script.

 AC_PREREQ([2.69])
-AC_INIT([telemetrics-client], [1.9.0], [https://clearlinux.org/])
+AC_INIT([telemetrics-client], [1.11.2], [https://clearlinux.org/])
 AC_CONFIG_AUX_DIR([build-aux])
 AM_INIT_AUTOMAKE([1.14 -Wall -Werror -Wno-extra-portability foreign subdir-objects])
 AM_SILENT_RULES([yes])
@@ -137,3 +137,27 @@ AC_CONFIG_COMMANDS([config dir],[mkdir -p src/data; touch src/data/.dirstamp])
 AC_CONFIG_FILES([Makefile])
 AC_REQUIRE_AUX_FILE([tap-driver.sh])
 AC_OUTPUT
+
+AC_MSG_NOTICE([
+
+telemetrics-client $VERSION
+===========================
+
+prefix:                 $prefix
+exec_prefix:            $exec_prefix
+libdir:                 $libdir
+bindir:                 $bindir
+datarootdir:            $datarootdir
+
+compiler:               $CC
+cflags:                 $CFLAGS
+ldflags:                $LDFLAGS
+
+unitdir:                $unitpath
+tmpfilesdir:            $tmpfilespath
+sysctldir:              $sysctlpath
+systemconfdir:          $confpath
+socketdir:              $socketpath
+loglevel:               $loglevel
+logtype:                $logtype
+])
@@ -0,0 +1,13 @@
+
+## Notes on manual pages
+
+The manual pages are generated using `ronn(1)`. To recreate them,
+run `make manpages` in the toplevel  folder. If you want to edit
+the documentation for upstream changes, make sure to edit the `*.md`
+files and not the shipped nroff output files.
+
+
+## Generating HTML documentation
+
+At toplevel, run 'doxygen docs/Doxyfile'. The documentation is
+generated in the docs/html/ directory.
@@ -0,0 +1,23 @@
+MANPAGES = \
+	docs/man/telemctl.1 \
+	docs/man/telemd.1 \
+	docs/man/telem-record-gen.1 \
+	docs/man/telemetry.3 \
+	docs/man/telemetrics.conf.5
+
+MANLINKS = \
+	docs/man/tm_create_record.3 \
+	docs/man/tm_send_record.3 \
+	docs/man/tm_set_config_file.3 \
+	docs/man/tm_set_payload.3
+
+manpages:
+	for MANPAGE in $(MANPAGES); do \
+		rst2man.py $${MANPAGE}.rst > $${MANPAGE}; \
+		rst2html.py $${MANPAGE}.rst > $${MANPAGE}.html; \
+	done
+
+dist_man_MANS = \
+	$(MANPAGES) $(MANLINKS)
+
+# vim: filetype=automake tabstop=8 shiftwidth=8 noexpandtab
@@ -0,0 +1 @@
+*.html
@@ -0,0 +1,98 @@
+.\" Man page generated from reStructuredText.
+.
+.TH TELEM-RECORD-GEN 1 "" "" ""
+.SH NAME
+telem-record-gen \- Create custom telemetry record
+.
+.nr rst2man-indent-level 0
+.
+.de1 rstReportMargin
+\\$1 \\n[an-margin]
+level \\n[rst2man-indent-level]
+level margin: \\n[rst2man-indent\\n[rst2man-indent-level]]
+-
+\\n[rst2man-indent0]
+\\n[rst2man-indent1]
+\\n[rst2man-indent2]
+..
+.de1 INDENT
+.\" .rstReportMargin pre:
+. RS \\$1
+. nr rst2man-indent\\n[rst2man-indent-level] \\n[an-margin]
+. nr rst2man-indent-level +1
+.\" .rstReportMargin post:
+..
+.de UNINDENT
+. RE
+.\" indent \\n[an-margin]
+.\" old: \\n[rst2man-indent\\n[rst2man-indent-level]]
+.nr rst2man-indent-level -1
+.\" new: \\n[rst2man-indent\\n[rst2man-indent-level]]
+.in \\n[rst2man-indent\\n[rst2man-indent-level]]u
+..
+.SH SYNOPSIS
+.sp
+\fBtelem\-record\-gen\fP <options>
+.SH DESCRIPTION
+.sp
+Generate and attempt to send a telemetry record to \fBtelemd\fP(1). This
+program can be used to create a telemetry record from a shell script
+or from another program using e.g. \fBsystem()\fP\&. The program connects
+to a local \fBtelemd\fP(1) daemon to spool the record for delivery to
+the actual collection service.
+.sp
+The payload can be provided as standard input, as a file with the
+\fB\-\-payload\-file\fP option, or as a string argument with the \fB\-\-payload\fP
+option. If neither the \fB\-\-payload\fP or \fB\-\-payload\-file\fP is provided,
+the payload is assumed to be on <stdin>.
+.sp
+Each record is expected to have a class identifier, a record version
+number, and a severity level. A class identifier is required to be
+provided.
+.SH OPTIONS
+.INDENT 0.0
+.INDENT 3.5
+.INDENT 0.0
+.IP \(bu 2
+\fB\-h\fP, \fB\-\-help\fP:
+Show help options.
+.IP \(bu 2
+\fB\-V\fP, \fB\-\-version\fP:
+Print the program version.
+.IP \(bu 2
+\fB\-f\fP, \fB\-\-config\-file\fP <file>:
+Path to configuration file (not implemented yet).
+.IP \(bu 2
+\fB\-s\fP, \fB\-\-severity\fP:
+Severity level (1\-4) \- (default 1).
+.IP \(bu 2
+\fB\-c\fP, \fB\-\-class\fP:
+Classification \fBlevel_1\fP/\fBlevel_2\fP/\fBlevel_3\fP\&.
+.IP \(bu 2
+\fB\-p\fP, \fB\-\-payload\fP <payload>:
+Record body (max size = 8k).
+.IP \(bu 2
+\fB\-P\fP, \fB\-\-payload\-file\fP <file>:
+File to read payload from.
+.IP \(bu 2
+\fB\-R\fP, \fB\-\-record\-version\fP <version>:
+Version number for format of payload (default 1).
+.UNINDENT
+.UNINDENT
+.UNINDENT
+.SH RETURN VALUES
+.sp
+0 on success. A non\-zero exit code indicates a failure occurred.
+.SH SEE ALSO
+.INDENT 0.0
+.IP \(bu 2
+\fBtelemd\fP(1)
+.IP \(bu 2
+\fI\%https://github.com/clearlinux/telemetrics\-client\fP
+.IP \(bu 2
+\fI\%https://clearlinux.org/documentation/\fP
+.UNINDENT
+.SH COPYRIGHT
+(C) 2017 Intel Corporation, CC-BY-SA-3.0
+.\" Generated by docutils manpage writer.
+.
@@ -0,0 +1,78 @@
+================
+telem-record-gen
+================
+
+------------------------------
+Create custom telemetry record
+------------------------------
+
+:Copyright: \(C) 2017 Intel Corporation, CC-BY-SA-3.0
+:Manual section: 1
+
+
+SYNOPSIS
+========
+
+``telem-record-gen`` <options>
+
+
+DESCRIPTION
+===========
+
+Generate and attempt to send a telemetry record to ``telemd``\(1). This
+program can be used to create a telemetry record from a shell script
+or from another program using e.g. ``system()``. The program connects
+to a local ``telemd``\(1) daemon to spool the record for delivery to
+the actual collection service.
+
+The payload can be provided as standard input, as a file with the
+``--payload-file`` option, or as a string argument with the ``--payload``
+option. If neither the ``--payload`` or ``--payload-file`` is provided,
+the payload is assumed to be on <stdin>.
+
+Each record is expected to have a class identifier, a record version
+number, and a severity level. A class identifier is required to be
+provided.
+
+
+OPTIONS
+=======
+
+ * ``-h``, ``--help``:
+   Show help options.
+
+ * ``-V``, ``--version``:
+   Print the program version.
+
+ * ``-f``, ``--config-file`` <file>:
+   Path to configuration file (not implemented yet).
+
+ * ``-s``, ``--severity``:
+   Severity level (1-4) - (default 1).
+
+ * ``-c``, ``--class``:
+   Classification ``level_1``/``level_2``/``level_3``.
+
+ * ``-p``, ``--payload`` <payload>:
+   Record body (max size = 8k).
+
+ * ``-P``, ``--payload-file`` <file>:
+   File to read payload from.
+
+ * ``-R``, ``--record-version`` <version>:
+   Version number for format of payload (default 1).
+
+
+RETURN VALUES
+=============
+
+0 on success. A non-zero exit code indicates a failure occurred.
+
+
+SEE ALSO
+========
+
+* ``telemd``\(1)
+* https://github.com/clearlinux/telemetrics-client
+* https://clearlinux.org/documentation/
+
@@ -0,0 +1,76 @@
+.\" Man page generated from reStructuredText.
+.
+.TH TELEMCTL 1 "" "" ""
+.SH NAME
+telemctl \- Telemetry service administration tool
+.
+.nr rst2man-indent-level 0
+.
+.de1 rstReportMargin
+\\$1 \\n[an-margin]
+level \\n[rst2man-indent-level]
+level margin: \\n[rst2man-indent\\n[rst2man-indent-level]]
+-
+\\n[rst2man-indent0]
+\\n[rst2man-indent1]
+\\n[rst2man-indent2]
+..
+.de1 INDENT
+.\" .rstReportMargin pre:
+. RS \\$1
+. nr rst2man-indent\\n[rst2man-indent-level] \\n[an-margin]
+. nr rst2man-indent-level +1
+.\" .rstReportMargin post:
+..
+.de UNINDENT
+. RE
+.\" indent \\n[an-margin]
+.\" old: \\n[rst2man-indent\\n[rst2man-indent-level]]
+.nr rst2man-indent-level -1
+.\" new: \\n[rst2man-indent\\n[rst2man-indent-level]]
+.in \\n[rst2man-indent\\n[rst2man-indent-level]]u
+..
+.SH SYNOPSIS
+.sp
+\fBtelemctl\fP
+.sp
+\fB/etc/telemetrics/opt\-out\fP
+.SH DESCRIPTION
+.sp
+Control actions for telemetry services. The command can be used to start,
+restart, or stop \fBtelemd\fP(1), or to opt\-in or opt\-out of telemetry delivery
+of records to a central telemetry service.
+.SH OPTIONS
+.INDENT 0.0
+.INDENT 3.5
+.INDENT 0.0
+.IP \(bu 2
+\fBstart\(ga\(ga|\(ga\(gastop\(ga\(ga|\(ga\(garestart\fP:
+Starts, stops or restarts all running telemetry services.
+.IP \(bu 2
+\fBopt\-in\fP:
+Opts in to telemetry, and starts telemetry services. The opt\-out file
+\fB/etc/telemetrics/opt\-out\fP is removed.
+.IP \(bu 2
+\fBopt\-out\fP:
+Opts out of telemetry, and stops telemetry services. The opt\-out file
+\fB/etc/telemetrics/opt\-out\fP is created.
+.UNINDENT
+.UNINDENT
+.UNINDENT
+.SH RETURN VALUES
+.sp
+0 on success. A non\-zero exit code indicates a failure occurred.
+.SH SEE ALSO
+.INDENT 0.0
+.IP \(bu 2
+\fBtelemd\fP(1)
+.IP \(bu 2
+\fI\%https://github.com/clearlinux/telemetrics\-client\fP
+.IP \(bu 2
+\fI\%https://clearlinux.org/documentation/\fP
+.UNINDENT
+.SH COPYRIGHT
+(C) 2017 Intel Corporation, CC-BY-SA-3.0
+.\" Generated by docutils manpage writer.
+.
@@ -0,0 +1,56 @@
+========
+telemctl
+========
+
+-------------------------------------
+Telemetry service administration tool
+-------------------------------------
+
+:Copyright: \(C) 2017 Intel Corporation, CC-BY-SA-3.0
+:Manual section: 1
+
+
+SYNOPSIS
+========
+
+``telemctl``
+
+``/etc/telemetrics/opt-out``
+
+
+DESCRIPTION
+===========
+
+Control actions for telemetry services. The command can be used to start,
+restart, or stop ``telemd``\(1), or to opt-in or opt-out of telemetry delivery
+of records to a central telemetry service.
+
+
+OPTIONS
+=======
+
+ * ``start``|``stop``|``restart``:
+   Starts, stops or restarts all running telemetry services.
+
+ * ``opt-in``: 
+   Opts in to telemetry, and starts telemetry services. The opt-out file
+   ``/etc/telemetrics/opt-out`` is removed.
+
+ * ``opt-out``: 
+   Opts out of telemetry, and stops telemetry services. The opt-out file
+   ``/etc/telemetrics/opt-out`` is created.
+
+
+RETURN VALUES
+=============
+
+0 on success. A non-zero exit code indicates a failure occurred.
+
+
+SEE ALSO
+========
+
+* ``telemd``\(1)
+* https://github.com/clearlinux/telemetrics-client
+* https://clearlinux.org/documentation/
+
@@ -0,0 +1,106 @@
+.\" Man page generated from reStructuredText.
+.
+.TH TELEMD 1 "" "" ""
+.SH NAME
+telemd \- Telemetry client service
+.
+.nr rst2man-indent-level 0
+.
+.de1 rstReportMargin
+\\$1 \\n[an-margin]
+level \\n[rst2man-indent-level]
+level margin: \\n[rst2man-indent\\n[rst2man-indent-level]]
+-
+\\n[rst2man-indent0]
+\\n[rst2man-indent1]
+\\n[rst2man-indent2]
+..
+.de1 INDENT
+.\" .rstReportMargin pre:
+. RS \\$1
+. nr rst2man-indent\\n[rst2man-indent-level] \\n[an-margin]
+. nr rst2man-indent-level +1
+.\" .rstReportMargin post:
+..
+.de UNINDENT
+. RE
+.\" indent \\n[an-margin]
+.\" old: \\n[rst2man-indent\\n[rst2man-indent-level]]
+.nr rst2man-indent-level -1
+.\" new: \\n[rst2man-indent\\n[rst2man-indent-level]]
+.in \\n[rst2man-indent\\n[rst2man-indent-level]]u
+..
+.SH SYNOPSIS
+.sp
+\fBtelemd\fP <flags>
+.sp
+\fB/etc/telemetrics/telemetrics.conf\fP
+.sp
+\fB/usr/share/defaults/telemetrics/telemetrics.conf\fP
+.sp
+\fB/etc/telemetrics/opt\-in\-static\-machine\-id\fP
+.SH DESCRIPTION
+.sp
+The \fBtelemd\fP program delivers locally generated telemetry records to a remote
+telemetry service. Telemetry data can be in any format, and is relayed as\-is.
+.SH OPTIONS
+.INDENT 0.0
+.INDENT 3.5
+.INDENT 0.0
+.IP \(bu 2
+\fB\-f\fP, \fB\-\-config_file\fP [<file>]:
+Configuration file. This overides the other parameters.
+.IP \(bu 2
+\fB\-h\fP, \fB\-\-help\fP:
+Display this help message.
+.IP \(bu 2
+\fB\-V\fP, \fB\-\-version\fP:
+Print the program version.
+.UNINDENT
+.UNINDENT
+.UNINDENT
+.SH FILES
+.INDENT 0.0
+.IP \(bu 2
+\fB/usr/share/defaults/telemetrics/telemetrics.conf\fP
+.INDENT 2.0
+.INDENT 3.5
+If no custom configuration file is found, \fBtelemd\fP uses the
+settings in this file.
+.UNINDENT
+.UNINDENT
+.IP \(bu 2
+\fB/etc/telemetrics/telemetrics.conf\fP
+.INDENT 2.0
+.INDENT 3.5
+Custom configuration file that \fBtelemd\fP reads. See \fBtelemetrics.conf\fP(5).
+.UNINDENT
+.UNINDENT
+.IP \(bu 2
+\fB/etc/telemetrics/opt\-in\-static\-machine\-id\fP
+.INDENT 2.0
+.INDENT 3.5
+If this file exists, the first line of this file will be sent to
+the server as the machine ID. Otherwise, the machine ID will be
+random and changed regularly.
+.UNINDENT
+.UNINDENT
+.UNINDENT
+.SH EXIT STATUS
+.sp
+0 when no errors occurred. A non\-zero exit status indicates a failure occurred.
+.SH SEE ALSO
+.INDENT 0.0
+.IP \(bu 2
+\fBtelemetry\fP(3)
+.IP \(bu 2
+\fBtelemetrics.conf\fP(5)
+.IP \(bu 2
+\fI\%https://github.com/clearlinux/telemetrics\-client\fP
+.IP \(bu 2
+\fI\%https://clearlinux.org/documentation/\fP
+.UNINDENT
+.SH COPYRIGHT
+(C) 2017 Intel Corporation, CC-BY-SA-3.0
+.\" Generated by docutils manpage writer.
+.
@@ -0,0 +1,78 @@
+======
+telemd
+======
+
+------------------------
+Telemetry client service
+------------------------
+
+:Copyright: \(C) 2017 Intel Corporation, CC-BY-SA-3.0
+:Manual section: 1
+
+
+SYNOPSIS
+========
+
+``telemd`` \<flags\>
+
+``/etc/telemetrics/telemetrics.conf``
+
+``/usr/share/defaults/telemetrics/telemetrics.conf``
+
+``/etc/telemetrics/opt-in-static-machine-id``
+
+
+DESCRIPTION
+===========
+
+The ``telemd`` program delivers locally generated telemetry records to a remote
+telemetry service. Telemetry data can be in any format, and is relayed as-is.
+
+
+OPTIONS
+=======
+
+  * ``-f``, ``--config_file`` \[\<file\>\]:
+    Configuration file. This overides the other parameters.
+
+  * ``-h``, ``--help``:
+    Display this help message.
+
+  * ``-V``, ``--version``:
+    Print the program version.
+
+
+FILES
+=====
+
+* ``/usr/share/defaults/telemetrics/telemetrics.conf``
+
+    If no custom configuration file is found, ``telemd`` uses the
+    settings in this file.
+
+* ``/etc/telemetrics/telemetrics.conf``
+
+    Custom configuration file that ``telemd`` reads. See ``telemetrics.conf``\(5).
+
+
+* ``/etc/telemetrics/opt-in-static-machine-id``
+
+    If this file exists, the first line of this file will be sent to
+    the server as the machine ID. Otherwise, the machine ID will be
+    random and changed regularly.
+
+
+EXIT STATUS
+===========
+
+0 when no errors occurred. A non-zero exit status indicates a failure occurred.
+
+
+SEE ALSO
+========
+
+* ``telemetry``\(3)
+* ``telemetrics.conf``\(5)
+* https://github.com/clearlinux/telemetrics-client
+* https://clearlinux.org/documentation/
+
@@ -0,0 +1,126 @@
+.\" Man page generated from reStructuredText.
+.
+.TH TELEMETRICS.CONF 5 "" "" ""
+.SH NAME
+telemetrics.conf \- Telemetry service configuration file
+.
+.nr rst2man-indent-level 0
+.
+.de1 rstReportMargin
+\\$1 \\n[an-margin]
+level \\n[rst2man-indent-level]
+level margin: \\n[rst2man-indent\\n[rst2man-indent-level]]
+-
+\\n[rst2man-indent0]
+\\n[rst2man-indent1]
+\\n[rst2man-indent2]
+..
+.de1 INDENT
+.\" .rstReportMargin pre:
+. RS \\$1
+. nr rst2man-indent\\n[rst2man-indent-level] \\n[an-margin]
+. nr rst2man-indent-level +1
+.\" .rstReportMargin post:
+..
+.de UNINDENT
+. RE
+.\" indent \\n[an-margin]
+.\" old: \\n[rst2man-indent\\n[rst2man-indent-level]]
+.nr rst2man-indent-level -1
+.\" new: \\n[rst2man-indent\\n[rst2man-indent-level]]
+.in \\n[rst2man-indent\\n[rst2man-indent-level]]u
+..
+.SH SYNOPSIS
+.sp
+\fB/etc/telemetrics/telemetrics.conf\fP
+.sp
+\fB/usr/share/defaults/telemetrics/telemetrics.conf\fP
+.SH DESCRIPTION
+.sp
+This file contains configuration parameters for the \fBtelemd\fP(1) telemetry
+service daemon. The daemon reads this file at startup if it exists.
+.SH SYNTAX
+.sp
+The configuration file contains \fBkey=value\fP pairs, formatted as plain
+text, one option per line. Comments can be added by preceding them with the
+\fB#\fP character. All configuration options should be in a section marked
+with \fB[settings]\fP\&.
+.SH OPTIONS
+.INDENT 0.0
+.IP \(bu 2
+\fBserver=<url>\fP
+.sp
+Server URL including protocol designator.
+.IP \(bu 2
+\fBsocket_path=<path>\fP
+.sp
+Path to the socket that \fItelemd\fP will listen on.
+.IP \(bu 2
+\fBcainfo=<path>\fP
+.sp
+Certificate file to use for validation of SSL endpoint.
+.IP \(bu 2
+\fBtidheader=<header>\fP
+.sp
+Telemetry ID in header format, usually \fBtidheader=X\-Telemetry\-TID:<uuid>\fP
+post header used to group records in ingestion service, which may be
+ingesting for more than one set of clients. Can be set to any string.
+.IP \(bu 2
+\fBrecord_expiry=<minutes>\fP
+.sp
+Record expiry time in minutes.
+.IP \(bu 2
+\fBspool_dir=<dir>\fP
+.sp
+Local spool directory used to store records being processed.
+.IP \(bu 2
+\fBspool_max_size=<kB>\fP
+.sp
+maximum size of the spool directory in kB. A value of \fB\-1\fP indicates
+no limit. The block size of the files in this directory is considered,
+and not the actual size of the record itself.
+.IP \(bu 2
+\fBspool_process_time=<seconds>\fP
+.sp
+Time in seconds for processing spool. Valid range: 120..3600. Values
+outside this range are clamped.
+.IP \(bu 2
+\fBrate_limit_enabled=<true|false>\fP
+.sp
+Enable rate limiting. If this is set to false then all rate\-limiting
+disabled. It is possible to disable each rate\-limit individually below.
+.IP \(bu 2
+\fBrecord_burst_limit=<limit>\fP
+.sp
+Rate limiting record burst limit. Valid Range:  0..\(ga\(gaINT_MAX\(ga\(ga, \-1 = disabled.
+.IP \(bu 2
+\fBrecord_window_length=<minutes>\fP
+.sp
+Rate limiting record window length in minutes. Valid Range: 0..59.
+.IP \(bu 2
+\fBbyte_burst_limit=<limit>\fP
+.sp
+Rate limiting byte burst limit. Valid Range:  0..\(gaINT_MAX\(ga, \-1 = disabled.
+.IP \(bu 2
+\fBbyte_window_length=<minutes>\fP
+.sp
+Rate limiting byte window length in minutes. Valid Range: 0..59.
+.IP \(bu 2
+\fBrate_limit_strategy=<strategy>\fP
+.sp
+Rate limit strategy \- what to do with record if rate\-limiting prevents
+delivery over network. Valid stategies: \fBspool\fP, \fBdrop\fP\&.
+.UNINDENT
+.SH SEE ALSO
+.INDENT 0.0
+.IP \(bu 2
+\fBtelemd\fP(1)
+.IP \(bu 2
+\fI\%https://github.com/clearlinux/telemetrics\-client\fP
+.IP \(bu 2
+\fI\%https://clearlinux.org/documentation/\fP
+.UNINDENT
+.SH COPYRIGHT
+(C) 2017 Intel Corporation, CC-BY-SA-3.0
+.\" Generated by docutils manpage writer.
+.
@@ -0,0 +1,110 @@
+================
+telemetrics.conf
+================
+
+------------------------------------
+Telemetry service configuration file
+------------------------------------
+
+:Copyright: \(C) 2017 Intel Corporation, CC-BY-SA-3.0
+:Manual section: 5
+
+
+SYNOPSIS
+========
+
+``/etc/telemetrics/telemetrics.conf``
+
+``/usr/share/defaults/telemetrics/telemetrics.conf``
+
+
+DESCRIPTION
+===========
+
+This file contains configuration parameters for the ``telemd``\(1) telemetry
+service daemon. The daemon reads this file at startup if it exists.
+
+
+SYNTAX
+======
+
+The configuration file contains ``key=value`` pairs, formatted as plain
+text, one option per line. Comments can be added by preceding them with the
+``#`` character. All configuration options should be in a section marked
+with ``[settings]``.
+
+
+OPTIONS
+=======
+
+-  ``server=<url>``
+
+   Server URL including protocol designator.
+
+-  ``socket_path=<path>``
+
+   Path to the socket that `telemd` will listen on.
+
+-  ``cainfo=<path>``
+
+   Certificate file to use for validation of SSL endpoint.
+
+-  ``tidheader=<header>``
+
+   Telemetry ID in header format, usually ``tidheader=X-Telemetry-TID:<uuid>``
+   post header used to group records in ingestion service, which may be
+   ingesting for more than one set of clients. Can be set to any string.
+
+-  ``record_expiry=<minutes>``
+
+   Record expiry time in minutes.
+
+-  ``spool_dir=<dir>``
+
+   Local spool directory used to store records being processed.
+
+-  ``spool_max_size=<kB>``
+
+   maximum size of the spool directory in kB. A value of ``-1`` indicates
+   no limit. The block size of the files in this directory is considered,
+   and not the actual size of the record itself.
+
+-  ``spool_process_time=<seconds>``
+
+   Time in seconds for processing spool. Valid range: 120..3600. Values
+   outside this range are clamped.
+
+-  ``rate_limit_enabled=<true|false>``
+
+   Enable rate limiting. If this is set to false then all rate-limiting
+   disabled. It is possible to disable each rate-limit individually below.
+
+-  ``record_burst_limit=<limit>``
+
+   Rate limiting record burst limit. Valid Range:  0..``INT_MAX``, -1 = disabled.
+
+-  ``record_window_length=<minutes>``
+
+   Rate limiting record window length in minutes. Valid Range: 0..59.
+
+-  ``byte_burst_limit=<limit>``
+
+   Rate limiting byte burst limit. Valid Range:  0..`INT_MAX`, -1 = disabled.
+
+-  ``byte_window_length=<minutes>``
+
+   Rate limiting byte window length in minutes. Valid Range: 0..59.
+
+-  ``rate_limit_strategy=<strategy>``
+
+   Rate limit strategy - what to do with record if rate-limiting prevents
+   delivery over network. Valid stategies: ``spool``, ``drop``.
+
+
+SEE ALSO
+========
+
+* ``telemd``\(1)
+* https://github.com/clearlinux/telemetrics-client
+* https://clearlinux.org/documentation/
+
@@ -1,15 +1,84 @@
-.TH TELEMETRY 3 "April 2015" Telemetrics
+.\" Man page generated from reStructuredText.
+.
+.TH TELEMETRY 3 "" "" ""
 .SH NAME
-telemetry \- Telemetry record creation and sending
+telemetry \- C programming interface for telemetrics-client
+.
+.nr rst2man-indent-level 0
+.
+.de1 rstReportMargin
+\\$1 \\n[an-margin]
+level \\n[rst2man-indent-level]
+level margin: \\n[rst2man-indent\\n[rst2man-indent-level]]
+-
+\\n[rst2man-indent0]
+\\n[rst2man-indent1]
+\\n[rst2man-indent2]
+..
+.de1 INDENT
+.\" .rstReportMargin pre:
+. RS \\$1
+. nr rst2man-indent\\n[rst2man-indent-level] \\n[an-margin]
+. nr rst2man-indent-level +1
+.\" .rstReportMargin post:
+..
+.de UNINDENT
+. RE
+.\" indent \\n[an-margin]
+.\" old: \\n[rst2man-indent\\n[rst2man-indent-level]]
+.nr rst2man-indent-level -1
+.\" new: \\n[rst2man-indent\\n[rst2man-indent-level]]
+.in \\n[rst2man-indent\\n[rst2man-indent-level]]u
+..
 .SH SYNOPSIS
-#include <telemetry.h>
 .sp
-tm_create_record()
-.br
-tm_set_payload()
-.br
-tm_send_record()
-.br
-tm_set_config_File()
-
-
+\fB#include "telemetry.h"\fP
+.sp
+\fBstruct telem_ref { struct telem_record *record; };\fP
+.sp
+\fBint tm_create_record(struct telem_ref **t_ref, uint32_t severity, char *classification, uint32_t payload_version)\fP
+.sp
+\fBint tm_set_payload(struct telem_ref *t_ref, char *payload)\fP
+.sp
+\fBint tm_send_record(struct telem_ref *t_ref)\fP
+.sp
+\fBvoid tm_free_record(struct telem_ref *t_ref)\fP
+.sp
+\fBvoid tm_set_config_file(char *c_file)\fP
+.SH DESCRIPTION
+.sp
+The functions in the telemetry library facilitate the delivery of
+telemetry data to the \fBtelemd\fP(1) service.
+.sp
+The function \fBtm_create_record()\fP initializes a telemetry record and
+sets the severity and classification of that record, as well as the
+payload version number. The memory needed to store the telemetry record
+is allocated and should be freed with \fBtm_free_record()\fP when no longer
+needed.
+.sp
+The function \fBtm_set_payload()\fP attaches the provided telemetry record
+data to the telemetry record. The current maximum payload size is 8192b.
+.sp
+The function \fBtm_send_record()\fP delivers the record to the local
+\fBtelemd\fP(1) service.
+.sp
+The function \fBtm_set_config_file()\fP can be used to provide an alternate
+configuration path to the telemetry library.
+.SH RETURN VALUES
+.sp
+All these functions return \fB0\fP on success, or a non\-zero return value
+if an error occurred. The functions \fBtm_free_record()\fP and \fBtm_set_config_file()\fP
+do not return any values.
+.SH SEE ALSO
+.INDENT 0.0
+.IP \(bu 2
+\fBtelemd\fP(1)
+.IP \(bu 2
+\fI\%https://github.com/clearlinux/telemetrics\-client\fP
+.IP \(bu 2
+\fI\%https://clearlinux.org/documentation/\fP
+.UNINDENT
+.SH COPYRIGHT
+(C) 2017 Intel Corporation, CC-BY-SA-3.0
+.\" Generated by docutils manpage writer.
+.
@@ -0,0 +1,67 @@
+=========
+telemetry
+=========
+
+----------------------------------------------
+C programming interface for telemetrics-client
+----------------------------------------------
+
+:Copyright: \(C) 2017 Intel Corporation, CC-BY-SA-3.0
+:Manual section: 3
+
+
+SYNOPSIS
+========
+
+``#include "telemetry.h"``
+  
+``struct telem_ref { struct telem_record *record; };``
+  
+``int tm_create_record(struct telem_ref **t_ref, uint32_t severity, char *classification, uint32_t payload_version)``
+
+``int tm_set_payload(struct telem_ref *t_ref, char *payload)``
+
+``int tm_send_record(struct telem_ref *t_ref)``
+
+``void tm_free_record(struct telem_ref *t_ref)``
+
+``void tm_set_config_file(char *c_file)``
+
+
+DESCRIPTION
+===========
+
+The functions in the telemetry library facilitate the delivery of
+telemetry data to the ``telemd``\(1) service.
+
+The function ``tm_create_record()`` initializes a telemetry record and
+sets the severity and classification of that record, as well as the
+payload version number. The memory needed to store the telemetry record
+is allocated and should be freed with ``tm_free_record()`` when no longer
+needed.
+
+The function ``tm_set_payload()`` attaches the provided telemetry record
+data to the telemetry record. The current maximum payload size is 8192b.
+
+The function ``tm_send_record()`` delivers the record to the local
+``telemd``\(1) service.
+
+The function ``tm_set_config_file()`` can be used to provide an alternate
+configuration path to the telemetry library.
+
+
+RETURN VALUES
+=============
+
+All these functions return ``0`` on success, or a non-zero return value
+if an error occurred. The functions ``tm_free_record()`` and ``tm_set_config_file()``
+do not return any values.
+
+
+SEE ALSO
+========
+
+* ``telemd``\(1)
+* https://github.com/clearlinux/telemetrics-client
+* https://clearlinux.org/documentation/
+
@@ -0,0 +1 @@
+.so man3/telemetry.3
@@ -0,0 +1 @@
+.so man3/telemetry.3
@@ -0,0 +1 @@
+.so man3/telemetry.3
@@ -0,0 +1 @@
+.so man3/telemetry.3
@@ -44,7 +44,8 @@ const char *config_key_int[] = { NULL, "record_expiry", "spool_max_size",
                                 "byte_window_length", "record_burst_limit",
                                 "byte_burst_limit", NULL };

-const char *config_key_bool[] = { NULL, "rate_limit_enabled", NULL };
+const char *config_key_bool[] = { NULL, "rate_limit_enabled",
+				  "daemon_recycling_enabled", NULL };

 static struct configuration config = { { 0 }, { 0 }, { 0 }, false, NULL };

@@ -334,4 +335,10 @@ const char *rate_limit_strategy_config()

        return val;
 }
+
+bool daemon_recycling_enabled_config(void)
+{
+	initialize_config();
+	return config.boolValues[CONF_DAEMON_RECYCLING_ENABLED];
+}
 /* vi: set ts=8 sw=8 sts=4 et tw=80 cino=(0: */
@@ -46,6 +46,7 @@ enum config_int_keys {
 enum config_bool_keys {
        CONF_BOOL_MIN = 0,
        CONF_RATE_LIMIT_ENABLED,
+	CONF_DAEMON_RECYCLING_ENABLED,
        CONF_BOOL_MAX
 };

@@ -113,4 +114,7 @@ const char *get_cainfo_config(void);
 /* Gets tidheader */
 const char *get_tidheader_config(void);

+/* Gets whether recycling is enabled */
+bool daemon_recycling_enabled_config(void);
+
 /* vi: set ts=8 sw=8 sts=4 et tw=80 cino=(0: */
@@ -35,3 +35,6 @@ rate_limit_enabled=true

 #rate limit strategy if record not send
 rate_limit_strategy=spool
+
+#daemon recycling enabled
+daemon_recycling_enabled=true
@@ -51,3 +51,7 @@ byte_window_length=20
 # Valid stategies: spool, drop
 rate_limit_strategy=spool

+# daemon recycling enabled - if daemon has been running for a while (2 hours),
+# has not any client nor spool data, then it exits.
+# this is to ensure that latest code runs.
+daemon_recycling_enabled=true
@@ -20,6 +20,7 @@

 #include "config.h"

+#ifdef TM_LOG_SYSTEMD
 #ifdef HAVE_SYSTEMD_SD_JOURNAL_H

 /* Certain static analysis tools do not understand GCC's __INCLUDE_LEVEL__
@@ -32,6 +33,7 @@
 #endif

 #include <systemd/sd-journal.h>
+#endif
 #else
 #include <errno.h>
 #include <string.h>
@@ -88,10 +90,10 @@
 #define telem_perror(msg) sd_journal_perror(msg)
 #endif
 #ifdef TM_LOG_SYSLOG
-#define telem_perror(msg) syslog(LOG_ERR, msg ": %s", strerror(errno))
+#define telem_perror(msg) syslog(LOG_ERR, msg ": %s\n", strerror(errno))
 #endif
 #ifdef TM_LOG_STDERR
-#define telem_perror(msg) fprintf(stderr, ": %s", msg)
+#define telem_perror(msg) fprintf(stderr, "ERROR: " msg ": %s\n", strerror(errno))
 #endif

 /* vi: set ts=8 sw=8 sts=4 et tw=80 cino=(0: */
@@ -232,6 +232,8 @@ int main(int argc, char **argv)
        time_t last_spool_run_time = time(NULL);
        time_t last_daemon_start_time = time(NULL);

+	bool daemon_recycling_enabled = daemon_recycling_enabled_config();
+
        ret = update_machine_id();
        if (ret == -1) {
                telem_log(LOG_ERR, "Unable to update machine id\n");
@@ -336,7 +338,7 @@ int main(int argc, char **argv)
                } else {
                        time_t now = time(NULL);
                        /* If spool is empty and time to recycle the daemon has elapsed*/
-                        if ((daemon.current_spool_size == 0) &&
+			if (daemon_recycling_enabled && (daemon.current_spool_size == 0) &&
                            difftime(now, last_daemon_start_time) >= TM_DAEMON_EXIT_TIME) {
                                /* Exit */
                                telem_log(LOG_INFO, "Daemon exiting for recycling\n");
@@ -38,6 +38,7 @@

 #include "config.h"
 #include "log.h"
+#include "probe.h"
 #include "telemetry.h"

 static Dwfl *d_core = NULL;
@@ -176,6 +177,55 @@ static int temp_core_file(void)
        return tmp;
 }

+/* There are a number of initialization routines required to initialize the Elf
+ * and Dwfl objects used by libdwfl to later process a core file. The argument
+ * e_core is the address of an Elf pointer declared by the caller, and core_fd
+ * is the open file descriptor for the core file.
+ */
+static int prepare_corefile(Elf **e_core, int core_fd)
+{
+        // Cleanup previous corefile processing if needed
+        if (d_core) {
+                dwfl_end(d_core);
+        }
+        if (*e_core) {
+                elf_end(*e_core);
+        }
+
+        // Boilerplate initialization for elfutils (libdwfl)
+        if (!(*e_core = elf_begin(core_fd, ELF_C_READ, NULL))) {
+                tm_elf_err("Failed to get file descriptor for ELF core file");
+                goto fail;
+        }
+
+        if (!(d_core = dwfl_begin(&cb))) {
+                tm_dwfl_err("Failed to start new libdwfl session");
+                goto fail;
+        }
+
+        int core_modules = 0;
+        core_modules = dwfl_core_file_report(d_core, *e_core, NULL);
+        if (core_modules == -1) {
+                tm_dwfl_err("Failed to report modules for ELF core file");
+                goto fail;
+        }
+
+        if (dwfl_report_end(d_core, NULL, NULL) != 0) {
+                tm_dwfl_err("Failed to finish reporting modules");
+                goto fail;
+        }
+
+        if ((core_for_pid = dwfl_core_file_attach(d_core, *e_core)) < 0) {
+                tm_dwfl_err("Failed to prepare libdwfl session for thread"
+                            " iteration");
+                goto fail;
+        }
+
+        return 0;
+fail:
+        return -1;
+}
+
 /* This callback is invoked for every frame in a thread. From a Dwfl_Frame, we
 * are able to extract the program counter (PC), and from that, the procedure
 * name via a Dwfl_Module.
@@ -305,8 +355,48 @@ fail:
        return false;
 }

+/* This is the entry point for libdwfl to unwind the backtrace from the core
+ * file. All data necessary for processing is referenced by d_core (a *Dwfl),
+ * which must be initialized prior to calling this function.  The callback
+ * function for processing each core thread is passed as the second argument to
+ * dwfl_getthreads(). The backtrace argument is the address of a pointer to a
+ * GString object declared by the caller which stores the backtrace data as a
+ * string.
+ */
+static int process_corefile(GString **backtrace)
+{
+        if (*backtrace) {
+                g_string_free(*backtrace, TRUE);
+        }
+        *backtrace = g_string_new(NULL);
+
+        if (dwfl_getthreads(d_core, thread_cb, backtrace) != DWARF_CB_OK) {
+                /* When errors occur during the unwinding, we reach this point.
+                 * If an error string is set for the particular error, send an
+                 * "error" record to capture at least a partial backtrace if
+                 * some frames were processed.
+                 */
+                if (errorstr != NULL) {
+                        telem_log(LOG_ERR, "%s", errorstr);
+                        g_string_append_printf(header, "Error: %s", errorstr);
+                        g_string_prepend(*backtrace, header->str);
+                        send_data(backtrace, error_severity, error_class);
+                }
+                goto fail;
+        }
+
+        return 0;
+fail:
+        return -1;
+}
+
 static bool in_clr_build(gchar *fullpath)
 {
+        // Global override for privacy filters
+        if (access(TM_PRIVACY_FILTERS_OVERRIDE, F_OK) == 0) {
+                return false;
+        }
+
        /*
         * The build environment for Clear Linux packages is set up by 'mock',
         * and the chroot in which rpmbuild builds the packages has this prefix.
@@ -319,17 +409,22 @@ static bool in_clr_build(gchar *fullpath)
        return false;
 }

-static bool filter_binaries(gchar *fullpath)
+static bool is_banned_path(gchar *fullpath)
 {
-        // Anything outside of /usr/, or in /usr/local/, we consider third-party
-        if (g_regex_match_simple("^[!/]usr[!/]", fullpath, 0, 0) == FALSE) {
-                return false;
-        } else if (g_regex_match_simple("^[!/]usr[!/]local[!/]", fullpath,
-                                        0, 0) == TRUE) {
+        // Global override for privacy filters
+        if (access(TM_PRIVACY_FILTERS_OVERRIDE, F_OK) == 0) {
                return false;
        }

-        return true;
+        // Anything outside of /usr/, or in /usr/local/, we consider third-party
+        if (g_regex_match_simple("^[!/]usr[!/]", fullpath, 0, 0) == FALSE) {
+                return true;
+        } else if (g_regex_match_simple("^[!/]usr[!/]local[!/]", fullpath,
+                                        0, 0) == TRUE) {
+                return true;
+        }
+
+        return false;
 }

 static gchar *config_file = NULL;
@@ -373,13 +468,20 @@ static void free_glib_strings(void)
 int main(int argc, char **argv)
 {
        Elf *e_core = NULL;
-        int core_modules = 0;
        int ret = EXIT_FAILURE;
        int core_fd = STDIN_FILENO;
        GString *backtrace = NULL;
        GError *error = NULL;
        GOptionContext *context;

+        if (fcntl(STDERR_FILENO, F_GETFL) < 0) {
+                // redirect stderr to avoid bad things to happen with
+                // fd 2 when launched from kernel core pattern
+                if (freopen("/dev/null", "w", stderr) == NULL) {
+                        exit(EXIT_FAILURE);
+                }
+        }
+
        // Since this program handles core files, make sure it does not produce
        // core files itself, or else it will be invoked endlessly...
        prctl(PR_SET_DUMPABLE, 0);
@@ -415,7 +517,7 @@ int main(int argc, char **argv)
                goto success;
        }

-        if (proc_path && !filter_binaries(proc_path)) {
+        if (proc_path && is_banned_path(proc_path)) {
                telem_log(LOG_NOTICE, "Ignoring core (third-party binary)\n");

                backtrace = g_string_new("Crash from third party\n");
@@ -460,30 +562,7 @@ int main(int argc, char **argv)

        elf_version(EV_CURRENT);

-        if (!(e_core = elf_begin(core_fd, ELF_C_READ, NULL))) {
-                tm_elf_err("Failed to get file descriptor for ELF core file");
-                goto fail;
-        }
-
-        if (!(d_core = dwfl_begin(&cb))) {
-                tm_dwfl_err("Failed to start new libdwfl session");
-                goto fail;
-        }
-
-        core_modules = dwfl_core_file_report(d_core, e_core, NULL);
-        if (core_modules == -1) {
-                tm_dwfl_err("Failed to report modules for ELF core file");
-                goto fail;
-        }
-
-        if (dwfl_report_end(d_core, NULL, NULL) != 0) {
-                tm_dwfl_err("Failed to finish reporting modules");
-                goto fail;
-        }
-
-        if ((core_for_pid = dwfl_core_file_attach(d_core, e_core)) < 0) {
-                tm_dwfl_err("Failed to prepare libdwfl session for thread"
-                            " iteration");
+        if (prepare_corefile(&e_core, core_fd) < 0) {
                goto fail;
        }

@@ -496,17 +575,27 @@ int main(int argc, char **argv)
                g_string_append_printf(header, "Signal: %d\n", signal_num);
        }

-        backtrace = g_string_new(NULL);
-        if (dwfl_getthreads(d_core, thread_cb, &backtrace) != DWARF_CB_OK) {
-                if (errorstr != NULL) {
-                        telem_log(LOG_ERR, "%s", errorstr);
-                        g_string_append_printf(header, "Error: %s", errorstr);
-                        g_string_prepend(backtrace, header->str);
-                        send_data(&backtrace, error_severity, error_class);
-                }
+        if (process_corefile(&backtrace) < 0) {
                goto fail;
        }

+        /* On Clear Linux OS, missing symbols may appear if automatic debuginfo
+         * downloads are still in flight. So if any missing symbols appear on
+         * the first run (indicated by the presence of "??? - ["), wait 10
+         * seconds and try again.
+         */
+        if (strstr(backtrace->str, "??? - [")) {
+                sleep(10);
+
+                if (prepare_corefile(&e_core, core_fd) < 0) {
+                        goto fail;
+                }
+
+                if (process_corefile(&backtrace) < 0) {
+                        goto fail;
+                }
+        }
+
        g_string_prepend(backtrace, header->str);

        if (!send_data(&backtrace, default_severity, clr_class)) {
@@ -1,7 +1,7 @@
 /*
 * This program is part of the Clear Linux Project
 *
- * Copyright 2015 Intel Corporation
+ * Copyright 2015-2017 Intel Corporation
 *
 * This program is free software; you can redistribute it and/or modify it under
 * the terms and conditions of the GNU Lesser General Public License, as
@@ -61,11 +61,11 @@ static inline void tm_journal_match_err(int ret)
 static void add_to_payload(const void *data, size_t length)
 {
        if (payload != NULL) {
-                g_string_append_printf(payload, "%.*s", (int)length,
+                g_string_append_printf(payload, "%.*s\n", (int)length,
                                       (char *)data);
        } else {
                payload = g_string_new(NULL);
-                g_string_printf(payload, "%.*s", (int)length,
+                g_string_printf(payload, "%.*s\n", (int)length,
                                (char *)data);
        }
 }
@@ -77,7 +77,7 @@ static bool send_data(char *class)

        if ((ret = tm_create_record(&handle, severity, class,
                                    payload_version)) < 0) {
-                telem_log(LOG_ERR, "Failed to create record: %s",
+                telem_log(LOG_ERR, "Failed to create record: %s\n",
                          strerror(-ret));
                goto fail;
        }
@@ -86,7 +86,7 @@ static bool send_data(char *class)
        payload = NULL;

        if ((ret = tm_set_payload(handle, (char *)payload_str)) < 0) {
-                telem_log(LOG_ERR, "Failed to set payload: %s", strerror(-ret));
+                telem_log(LOG_ERR, "Failed to set payload: %s\n", strerror(-ret));
                free(payload_str);
                tm_free_record(handle);
                goto fail;
@@ -95,7 +95,7 @@ static bool send_data(char *class)
        free(payload_str);

        if ((ret = tm_send_record(handle)) < 0) {
-                telem_log(LOG_ERR, "Failed to send record: %s", strerror(-ret));
+                telem_log(LOG_ERR, "Failed to send record: %s\n", strerror(-ret));
                tm_free_record(handle);
                goto fail;
        }
@@ -122,6 +122,16 @@ static int read_new_entries(sd_journal *journal)

                add_to_payload(data, length);

+                // For now, we send one record per log message, in case the
+                // there is a large backlog of messages and we exceed the
+                // payload size limit (8KB). And ignore errors, hoping that it's
+                // a transient problem.
+
+                if (!send_data(error_class)) {
+                        telem_log(LOG_ERR, "Failed to send data. Ignoring.\n");
+                        return num_entries;
+                }
+
                num_entries++;
        }

@@ -148,19 +158,13 @@ static bool process_existing_entries(sd_journal *journal)
                return false;
        }

-        if (!read_new_entries(journal)) {
+        ret = read_new_entries(journal);
+        if (ret < 0) {
                return false;
-        }
-
-        if (!payload) {
-                telem_log(LOG_DEBUG, "No existing entries found\n");
+        } else if (ret == 0) {
                return true;
        }

-        if (!send_data(error_class)) {
-                return false;
-        }
-
        return true;
 }

@@ -185,6 +189,33 @@ static bool get_boot_id(char **data)
        return true;
 }

+#define JOURNAL_MATCH(data) \
+        do { \
+                r = sd_journal_add_match(journal, data, 0); \
+                if (r < 0) { \
+                        tm_journal_match_err(r); \
+                        return false; \
+                } \
+        } while (0);
+
+#define JOURNAL_AND \
+        do { \
+                r = sd_journal_add_conjunction(journal); \
+                if (r < 0) { \
+                        tm_journal_match_err(r); \
+                        return false; \
+                } \
+        } while (0);
+
+#define JOURNAL_OR \
+        do { \
+                r = sd_journal_add_disjunction(journal); \
+                if (r < 0) { \
+                        tm_journal_match_err(r); \
+                        return false; \
+                } \
+        } while (0);
+
 static bool add_filters(sd_journal *journal)
 {
        char *data = NULL;
@@ -194,24 +225,29 @@ static bool add_filters(sd_journal *journal)
                return false;
        }

-        r = sd_journal_add_match(journal, data, 0);
-        if (r < 0) {
-                tm_journal_match_err(r);
-                return false;
-        }
+        /* The semantics of how journal entry matching works is described in
+         * detail in sd_journal_add_match(3).
+         *
+         * The matches declared here correspond to the logical expression:
+         *
+         * BOOTID && ((P0 || P1 || P2 || P3) || EXITED)
+         *
+         * BOOTID is short for _BOOT_ID=VAL, where VAL is the boot ID for the
+         * current boot. P0, P1, etc stand for PRIORITY=0, etc. And EXITED is
+         * short for EXIT_CODE=exited.
+         */
+
+        JOURNAL_MATCH(data);
        free(data);
-
-        r = sd_journal_add_match(journal, "SYSLOG_IDENTIFIER=crashprobe", 0);
-        if (r < 0) {
-                tm_journal_match_err(r);
-                return false;
-        }
-
-        r = sd_journal_add_match(journal, "PRIORITY=3", 0);
-        if (r < 0) {
-                tm_journal_match_err(r);
-                return false;
-        }
+        JOURNAL_AND;
+        // The four highest log levels, all indicating errors
+        JOURNAL_MATCH("PRIORITY=0");
+        JOURNAL_MATCH("PRIORITY=1");
+        JOURNAL_MATCH("PRIORITY=2");
+        JOURNAL_MATCH("PRIORITY=3");
+        JOURNAL_OR;
+        // Only set for service-level error conditions
+        JOURNAL_MATCH("EXIT_CODE=exited");

        return true;
 }
@@ -258,10 +294,6 @@ static bool process_journal(void)
                                } else if (r < 0) {
                                        return false;
                                }
-
-                                if (!send_data(error_class)) {
-                                        return false;
-                                }
                        }
                }
        }
@@ -44,7 +44,8 @@ endif


 %C%_crashprobe_SOURCES = \
-	%D%/crash_probe.c
+	%D%/crash_probe.c \
+	%D%/probe.h
 %C%_crashprobe_CFLAGS = \
 	$(AM_CFLAGS) \
 	$(GLIB_CFLAGS)
@@ -120,7 +121,8 @@ endif

 %C%_oopsprobe_SOURCES = \
        %D%/oops_probe.c \
-	%D%/oops_parser.c
+	%D%/oops_parser.c \
+	%D%/probe.h
 %C%_oopsprobe_CFLAGS = \
        $(AM_CFLAGS) \
        $(GLIB_CFLAGS)
@@ -21,9 +21,11 @@
 #include <string.h>
 #include <ctype.h>
 #include <inttypes.h>
+#include <unistd.h>

 #include "oops_parser.h"
 #include "log.h"
+#include "probe.h"

 #define NUM_TAINTED_FLAGS 16

@@ -182,7 +184,7 @@ char *skip_timestamp(char *line)
        return start;
 }

-char *skip_spaces(char *line)
+char *skip_space(char *line)
 {
        char *start = line;

@@ -192,6 +194,16 @@ char *skip_spaces(char *line)
        return start;
 }

+char *skip_spaces(char *line)
+{
+        char *start = line;
+
+        while (*start && isspace(*start)) {
+                start++;
+        }
+        return start;
+}
+
 bool starts_with(const char *line, const char *line_end, const char *substr)
 {
        size_t substrlen = strlen(substr);
@@ -344,7 +356,7 @@ void parse_single_line(char *line, size_t size)
        line_end = line + size;
        start = skip_log_level(line);
        start = skip_timestamp(start);
-        start = skip_spaces(start);
+        start = skip_space(start);

        struct oops_pattern *pattern;
        if (oops_msg.length == 0) {
@@ -376,13 +388,16 @@ void parse_single_line(char *line, size_t size)
                } else if (strstr(start, "[ end trace")) {
                        end_found = true;
                } else if (!in_stack_dump) {
-                        if (starts_with(start, line_end, " [<") ||
-                            starts_with(start, line_end, " <") ||      //case : <EOI> [<fff73..]
-                            starts_with(start, line_end, "Call Trace:")) {
+                        // This line indicates the beginning of a stack trace;
+                        // the next line is most likely the topmost frame.
+                        if (starts_with(start, line_end, "Call Trace:")) {
                                in_stack_dump = true;
                        }
                } else if (in_stack_dump) {
-                        if (!(starts_with(start, line_end, " [") || starts_with(start, line_end, " <")) ||
+                        // In Linux 4.10+, the oops format changed, so we look
+                        // for a single leading space character to indicate a
+                        // stack frame line.
+                        if (!(starts_with(start, line_end, " ")) ||
                            (strlen(start) < 8) ||
                            (strstr(start, "Code:") != NULL) ||
                            (strstr(start, "Instruction Dump::") != NULL)) {
@@ -425,9 +440,20 @@ struct stack_frame {
 void stack_frame_append(struct stack_frame **head, struct stack_frame **tail, char *start)
 {
        /*
-         * Format is as :
+         * Format is:
+         *
         *  [<ffffffffa1002128>] do_one_initcall+0xb8/0x1e0
         *  [<ffffffffa118784a>] ? __vunmap+0x9a/0x100
+         *
+         * OR
+         *
+         *  <IRQ>  [<ffffffff816606a8>] dump_stack+0x19/0x1b
+         *
+         * OR (in 4.10+)
+         *
+         *  do_one_initcall+0xb8/0x1e0
+         *  ? __vunmap+0x9a/0x100
+         *
         */

        struct stack_frame *frame = NULL;
@@ -436,18 +462,22 @@ void stack_frame_append(struct stack_frame **head, struct stack_frame **tail, ch
        if (!start) {
                return;
        }
-        start = start + 2;
+
+        // Skip leading spaces
+        start = skip_spaces(start);

        //case: <IRQ>  [<ffffffff816606a8>] dump_stack+0x19/0x1b
-        if (!strncmp(start, "IRQ>", 4) || !strncmp(start, "NMI>", 4) ||
-            !strncmp(start, "EOI>", 4) || !strncmp(start, "<EOE>>", 6)) {
-                while (*start && *start != '[') {
+        if (!strncmp(start, "<IRQ>", 5) || !strncmp(start, "<NMI>", 5) ||
+            !strncmp(start, "<EOI>", 5) || !strncmp(start, "<<EOE>>", 7)) {
+                while (*start && !isspace(*start)) {
                        start++;
                }
-                if (*start == '\0') {
+                if (start && *start == '\0') {
                        return;
                }
-                start++;
+
+                // Skip intervening space
+                start = skip_spaces(start);
        }

        frame = malloc(sizeof(struct stack_frame));
@@ -462,26 +492,29 @@ void stack_frame_append(struct stack_frame **head, struct stack_frame **tail, ch
                *tail = frame;
        }

-        if (*start == '<') {
+        // Skip memory address parsing if the address is absent (as in Linux 4.10+).
+        if (!strncmp(start, "[", 1)) {
                start++;
-                frame->addr = (uint64_t)strtoll(start, (char **)&start, 16);
-        } else {
-                frame->addr = 0;
-        }

-        /* Skip up to the space after the closing bracket */
-        while (*start && !isspace(*start)) {
-                start++;
-        }
-        if (*start) {
-                start++;
-        }
+                if (*start == '<') {
+                        start++;
+                        frame->addr = (uint64_t)strtoll(start, (char **)&start, 16);
+                } else {
+                        frame->addr = 0;
+                }

-        if (*start && *start == '?') {
-                start++;
-                if (*start && isspace(*start)) {
+                // Skip up to the space after the closing bracket
+                while (*start && !isspace(*start)) {
                        start++;
                }
+                if (start && *start == '\0') {
+                        return;
+                }
+
+                // Skip intervening space
+                start = skip_spaces(start);
+        } else {
+                frame->addr = 0;
        }

        offset_ptr = strchr(start, '+');
@@ -711,13 +744,14 @@ void append_registers_to_bt(GString **backtrace)
        struct reg_s *reg_entry = reg_head;

        while (reg_entry != NULL) {
-                #ifdef DEBUG
-                g_string_append_printf(*backtrace, "Register %s: %" PRIx64 "\n", reg_entry->reg_name,
-                                       reg_entry->reg_value);
-                #else
-                g_string_append_printf(*backtrace, "Register %s: %s\n", reg_entry->reg_name,
-                                       reg_entry->reg_value ? "Non-zero" : "Zero");
-                #endif
+                // Global override for privacy filters
+                if (access(TM_PRIVACY_FILTERS_OVERRIDE, F_OK) == 0) {
+                        g_string_append_printf(*backtrace, "Register %s: %" PRIx64 "\n", reg_entry->reg_name,
+                                               reg_entry->reg_value);
+                } else {
+                        g_string_append_printf(*backtrace, "Register %s: %s\n", reg_entry->reg_name,
+                                               reg_entry->reg_value ? "Non-zero" : "Zero");
+                }
                reg_entry = reg_entry->next;
        }

@@ -736,17 +770,28 @@ GString *parse_backtrace(struct oops_log_msg *msg)
        GString *backtrace = NULL;
        int frame_counter = 1;
        char *modules = NULL, *kernel_version = NULL, *tainted = NULL;
+        // Since lines are processed from last to first, the stack trace lines
+        // will appear first.
+        bool in_trace = true;

        for (int i = msg->length - 1; i > 0; i--) {
                /* Check if this line is part of a stack trace */
                line = msg->lines[i];

-                if (starts_with(line, line + strlen(line), " [") ||
-                    starts_with(line, line + strlen(line), " <")) {
+                if (in_trace && starts_with(line, line + strlen(line), " ")) {
                        stack_frame_append(&head, &tail, line);
                        continue;
                }

+                // We've reached the end of the stack trace
+                if (starts_with(line, line + strlen(line), "Call Trace:")) {
+                        in_trace = false;
+                }
+
+                // Stack trace lines must be consecutive, so if this point is
+                // reached, we're outside the trace boundary.
+                in_trace = false;
+
                if (str_starts_with_casei(line, "Modules linked in: ")) {
                        modules = line + strlen("Modules linked in: ");
                        continue;
@@ -0,0 +1,26 @@
+/*
+ * This program is part of the Clear Linux Project
+ *
+ * Copyright 2017 Intel Corporation
+ *
+ * This program is free software; you can redistribute it and/or modify it under
+ * the terms and conditions of the GNU Lesser General Public License, as
+ * published by the Free Software Foundation; either version 2.1 of the License,
+ * or (at your option) any later version.
+ *
+ * This program is distributed in the hope it will be useful, but WITHOUT ANY
+ * WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR
+ * A PARTICULAR PURPOSE. See the GNU Lesser General Public License for more
+ * details.
+ */
+
+#pragma once
+
+/**
+ * Existence of the opt-in-no-privacy-filters file disables certain privacy
+ * filters enforced in telemetry probes.
+ */
+#define TM_PRIVACY_FILTERS_OVERRIDE "/etc/telemetrics/opt-in-no-privacy-filters"
+
+
+/* vi: set ts=8 sw=8 sts=4 et tw=80 cino=(0: */
@@ -56,6 +56,20 @@ static GOptionEntry options[] = {
        { NULL }
 };

+const unsigned int count_chars(const char *check, const char character)
+{
+        unsigned int count = 0U;
+        if (check == NULL) {
+                return count;
+        }
+        for (; *check != '\0'; ++check) {
+                if (*check == character) {
+                        ++count;
+                }
+        }
+        return count;
+}
+
 static void free_glib_strings(void)
 {
        if (config_file) {
@@ -122,6 +136,12 @@ int validate_opts(void)
                return ret;
        }

+        if (count_chars(opt_class, '/') != 2) {
+                fprintf(stderr, "Error: Classification needs to be in "
+                        "most/to/least specific format, 2 \'/\' required.\n");
+                return ret;
+        }
+
        /* Severity */
        if ((severity) < 1 || (severity > 4)) {
                fprintf(stderr, "Error: Valid range for severity is 1-4\n");
@@ -50,6 +50,7 @@ START_TEST(check_read_valid_config)
        ck_assert_str_eq(config.strValues[CONF_CAINFO], "/tmp/cacert.crt");
        ck_assert_str_eq(config.strValues[CONF_TIDHEADER],
                         "X-Telemetry-TID: 6907c830-eed9-4ce9-81ae-76daf8d88f0f");
+	ck_assert(config.boolValues[CONF_DAEMON_RECYCLING_ENABLED] == true);

 }
 END_TEST
@@ -75,6 +76,7 @@ START_TEST(check_config_initialised)
        ck_assert_str_eq(get_cainfo_config(), "/tmp/cacert.crt");
        ck_assert_str_eq(get_tidheader_config(),
                         "X-Telemetry-TID: 6907c830-eed9-4ce9-81ae-76daf8d88f0f");
+	ck_assert(daemon_recycling_enabled_config() == true);
 }
 END_TEST

@@ -100,23 +100,23 @@ START_TEST(watchdog_payload)
        ck_assert(strstr(pl->str, "#1 dump_stack"));
        ck_assert(strstr(pl->str, "#2 warn_slowpath_common"));
        ck_assert(strstr(pl->str, "#3 warn_slowpath_fmt"));
-        ck_assert(strstr(pl->str, "#4 __queue_work"));
+        ck_assert(strstr(pl->str, "#4 ? __queue_work"));
        ck_assert(strstr(pl->str, "#5 dev_watchdog"));
-        ck_assert(strstr(pl->str, "#6 dev_deactivate_queue.constprop.30"));
+        ck_assert(strstr(pl->str, "#6 ? dev_deactivate_queue.constprop.30"));
        ck_assert(strstr(pl->str, "#7 call_timer_fn"));
-        ck_assert(strstr(pl->str, "#8 dev_deactivate_queue.constprop.30"));
+        ck_assert(strstr(pl->str, "#8 ? dev_deactivate_queue.constprop.30"));
        ck_assert(strstr(pl->str, "#9 run_timer_softirq"));
        ck_assert(strstr(pl->str, "#10 __do_softirq"));
        ck_assert(strstr(pl->str, "#11 irq_exit"));
        ck_assert(strstr(pl->str, "#12 smp_apic_timer_interrupt"));
        ck_assert(strstr(pl->str, "#13 apic_timer_interrupt"));
-        ck_assert(strstr(pl->str, "#14 __hrtimer_start_range_ns"));
-        ck_assert(strstr(pl->str, "#15 cpuidle_enter_state"));
-        ck_assert(strstr(pl->str, "#16 cpuidle_enter_state"));
+        ck_assert(strstr(pl->str, "#14 ? __hrtimer_start_range_ns"));
+        ck_assert(strstr(pl->str, "#15 ? cpuidle_enter_state"));
+        ck_assert(strstr(pl->str, "#16 ? cpuidle_enter_state"));
        ck_assert(strstr(pl->str, "#17 cpuidle_idle_call"));
        ck_assert(strstr(pl->str, "#18 arch_cpu_idle"));
        ck_assert(strstr(pl->str, "#19 cpu_startup_entry"));
-        ck_assert(strstr(pl->str, "#20 clockevents_register_device"));
+        ck_assert(strstr(pl->str, "#20 ? clockevents_register_device"));
        ck_assert(strstr(pl->str, "#21 start_secondary"));

        g_string_free(pl, true);
@@ -141,8 +141,8 @@ START_TEST(alsa_bug_payload)
        ck_assert(strstr(pl->str, "Tainted : Not tainted"));

        ck_assert(strstr(pl->str, "#1 warn_on_slowpath"));
-        ck_assert(strstr(pl->str, "#2 vsnprintf"));
-        ck_assert(strstr(pl->str, "#3 _spin_unlock_irqrestore"));
+        ck_assert(strstr(pl->str, "#2 ? vsnprintf"));
+        ck_assert(strstr(pl->str, "#3 ? _spin_unlock_irqrestore"));
        ck_assert(strstr(pl->str, "#10 start_kernel"));
        g_string_free(pl, true);
 }
@@ -173,14 +173,14 @@ START_TEST(warning_payload)
        ck_assert(strstr(pl->str, "#3 warn_slowpath_fmt"));
        ck_assert(strstr(pl->str, "#4 preempt_notifier_register"));
        ck_assert(strstr(pl->str, "#5 VBoxHost_RTThreadCtxHooksRegister"));
-        ck_assert(strstr(pl->str, "#6 update_curr"));
-        ck_assert(strstr(pl->str, "#7 supdrvIOCtlFast"));
-        ck_assert(strstr(pl->str, "#8 VBoxDrvLinuxIOCtl_4_3_28"));
-        ck_assert(strstr(pl->str, "#9 put_prev_entity"));
-        ck_assert(strstr(pl->str, "#10 do_vfs_ioctl"));
-        ck_assert(strstr(pl->str, "#11 pick_next_task_rt"));
-        ck_assert(strstr(pl->str, "#12 SyS_ioctl"));
-        ck_assert(strstr(pl->str, "#13 entry_SYSCALL_64_fastpath"));
+        ck_assert(strstr(pl->str, "#6 ? update_curr"));
+        ck_assert(strstr(pl->str, "#7 ? supdrvIOCtlFast"));
+        ck_assert(strstr(pl->str, "#8 ? VBoxDrvLinuxIOCtl_4_3_28"));
+        ck_assert(strstr(pl->str, "#9 ? put_prev_entity"));
+        ck_assert(strstr(pl->str, "#10 ? do_vfs_ioctl"));
+        ck_assert(strstr(pl->str, "#11 ? pick_next_task_rt"));
+        ck_assert(strstr(pl->str, "#12 ? SyS_ioctl"));
+        ck_assert(strstr(pl->str, "#13 ? entry_SYSCALL_64_fastpath"));

        g_string_free(pl, true);
 }
@@ -208,16 +208,16 @@ START_TEST(warn_on_payload)
        ck_assert(strstr(pl->str, "#2 warn_slowpath_common"));
        ck_assert(strstr(pl->str, "#3 warn_slowpath_fmt"));
        ck_assert(strstr(pl->str, "#4 dev_watchdog"));
-        ck_assert(strstr(pl->str, "#5 qdisc_rcu_free"));
+        ck_assert(strstr(pl->str, "#5 ? qdisc_rcu_free"));
        ck_assert(strstr(pl->str, "#6 call_timer_fn"));
-        ck_assert(strstr(pl->str, "#7 qdisc_rcu_free"));
+        ck_assert(strstr(pl->str, "#7 ? qdisc_rcu_free"));
        ck_assert(strstr(pl->str, "#8 run_timer_softirq"));
        ck_assert(strstr(pl->str, "#9 __do_softirq"));
        ck_assert(strstr(pl->str, "#10 irq_exit"));
        ck_assert(strstr(pl->str, "#11 smp_apic_timer_interrupt"));
        ck_assert(strstr(pl->str, "#12 apic_timer_interrupt"));
-        ck_assert(strstr(pl->str, "#13 cpuidle_enter_state"));
-        ck_assert(strstr(pl->str, "#14 cpuidle_enter_state"));
+        ck_assert(strstr(pl->str, "#13 ? cpuidle_enter_state"));
+        ck_assert(strstr(pl->str, "#14 ? cpuidle_enter_state"));
        ck_assert(strstr(pl->str, "#15 cpuidle_enter"));
        ck_assert(strstr(pl->str, "#16 cpu_startup_entry"));
        ck_assert(strstr(pl->str, "#17 start_secondary"));
@@ -285,20 +285,20 @@ START_TEST(sysctl2_payload)
        ck_assert(strstr(pl->str, "Kernel Version : 2.6.32.53-bigmem-5-openvz #28"));
        ck_assert(strstr(pl->str, "Tainted : P"));

-        ck_assert(strstr(pl->str, "#1 set_fail"));
-        ck_assert(strstr(pl->str, "#2 sysctl_check_table"));
-        ck_assert(strstr(pl->str, "#3 sysctl_check_lookup"));
-        ck_assert(strstr(pl->str, "#4 sysctl_check_table"));
-        ck_assert(strstr(pl->str, "#5 sysctl_check_lookup"));
-        ck_assert(strstr(pl->str, "#6 sysctl_check_table"));
-        ck_assert(strstr(pl->str, "#7 __kmalloc"));
-        ck_assert(strstr(pl->str, "#8 __register_sysctl_paths"));
-        ck_assert(strstr(pl->str, "#9 proc_register"));
-        ck_assert(strstr(pl->str, "#10 register_net_sysctl_table"));
-        ck_assert(strstr(pl->str, "#11 nf_conntrack_net_init"));
-        ck_assert(strstr(pl->str, "#12 setup_net"));
-        ck_assert(strstr(pl->str, "#13 copy_net_ns"));
-        ck_assert(strstr(pl->str, "#14 create_new_namespaces"));
+        ck_assert(strstr(pl->str, "#1 ? set_fail"));
+        ck_assert(strstr(pl->str, "#2 ? sysctl_check_table"));
+        ck_assert(strstr(pl->str, "#3 ? sysctl_check_lookup"));
+        ck_assert(strstr(pl->str, "#4 ? sysctl_check_table"));
+        ck_assert(strstr(pl->str, "#5 ? sysctl_check_lookup"));
+        ck_assert(strstr(pl->str, "#6 ? sysctl_check_table"));
+        ck_assert(strstr(pl->str, "#7 ? __kmalloc"));
+        ck_assert(strstr(pl->str, "#8 ? __register_sysctl_paths"));
+        ck_assert(strstr(pl->str, "#9 ? proc_register"));
+        ck_assert(strstr(pl->str, "#10 ? register_net_sysctl_table"));
+        ck_assert(strstr(pl->str, "#11 ? nf_conntrack_net_init"));
+        ck_assert(strstr(pl->str, "#12 ? setup_net"));
+        ck_assert(strstr(pl->str, "#13 ? copy_net_ns"));
+        ck_assert(strstr(pl->str, "#14 ? create_new_namespaces"));

        g_string_free(pl, true);
 }
@@ -327,7 +327,7 @@ START_TEST(softlockup_payload)
        ck_assert(strstr(pl->str, "#2 btrfs_tree_read_lock"));
        ck_assert(strstr(pl->str, "#3 btrfs_read_lock_root_node"));
        ck_assert(strstr(pl->str, "#4 btrfs_search_slot"));
-        ck_assert(strstr(pl->str, "#5 crypto_shash_update"));
+        ck_assert(strstr(pl->str, "#5 ? crypto_shash_update"));
        ck_assert(strstr(pl->str, "#6 btrfs_lookup_xattr"));
        ck_assert(strstr(pl->str, "#7 __btrfs_getxattr"));
        ck_assert(strstr(pl->str, "#8 btrfs_getxattr"));
@@ -356,19 +356,19 @@ START_TEST(rtnl_payload)

        ck_assert(strstr(pl->str, "Tainted : Not tainted"));

-        ck_assert(strstr(pl->str, "#1 dump_stack"));
-        ck_assert(strstr(pl->str, "#2 bond_set_rx_mode"));
-        ck_assert(strstr(pl->str, "#3 __dev_mc_add"));
-        ck_assert(strstr(pl->str, "#4 igmp6_group_added"));
-        ck_assert(strstr(pl->str, "#5 sock_def_readable"));
-        ck_assert(strstr(pl->str, "#6 ipv6_dev_mc_inc"));
-        ck_assert(strstr(pl->str, "#7 ipv6_sock_mc_join"));
-        ck_assert(strstr(pl->str, "#8 do_ipv6_setsockopt.isra.6"));
-        ck_assert(strstr(pl->str, "#9 pipe_write"));
-        ck_assert(strstr(pl->str, "#10 SYSC_sendto"));
-        ck_assert(strstr(pl->str, "#11 ipv6_setsockopt"));
-        ck_assert(strstr(pl->str, "#12 SyS_setsockopt"));
-        ck_assert(strstr(pl->str, "#13 system_call_fastpath"));
+        ck_assert(strstr(pl->str, "#1 ? dump_stack"));
+        ck_assert(strstr(pl->str, "#2 ? bond_set_rx_mode"));
+        ck_assert(strstr(pl->str, "#3 ? __dev_mc_add"));
+        ck_assert(strstr(pl->str, "#4 ? igmp6_group_added"));
+        ck_assert(strstr(pl->str, "#5 ? sock_def_readable"));
+        ck_assert(strstr(pl->str, "#6 ? ipv6_dev_mc_inc"));
+        ck_assert(strstr(pl->str, "#7 ? ipv6_sock_mc_join"));
+        ck_assert(strstr(pl->str, "#8 ? do_ipv6_setsockopt.isra.6"));
+        ck_assert(strstr(pl->str, "#9 ? pipe_write"));
+        ck_assert(strstr(pl->str, "#10 ? SYSC_sendto"));
+        ck_assert(strstr(pl->str, "#11 ? ipv6_setsockopt"));
+        ck_assert(strstr(pl->str, "#12 ? SyS_setsockopt"));
+        ck_assert(strstr(pl->str, "#13 ? system_call_fastpath"));

        g_string_free(pl, true);
 }
@@ -454,49 +454,49 @@ START_TEST(irq_payload)
        ck_assert(strstr(pl->str, "#1 dump_stack"));
        ck_assert(strstr(pl->str, "#2 __report_bad_irq"));
        ck_assert(strstr(pl->str, "#3 note_interrupt"));
-        ck_assert(strstr(pl->str, "#4 __do_softirq"));
+        ck_assert(strstr(pl->str, "#4 ? __do_softirq"));
        ck_assert(strstr(pl->str, "#5 handle_irq_event_percpu"));
        ck_assert(strstr(pl->str, "#6 handle_irq_event"));
-        ck_assert(strstr(pl->str, "#7 handle_nested_irq"));
+        ck_assert(strstr(pl->str, "#7 ? handle_nested_irq"));
        ck_assert(strstr(pl->str, "#8 handle_level_irq"));
-        ck_assert(strstr(pl->str, "#9 do_IRQ"));
-        ck_assert(strstr(pl->str, "#10 common_interrupt"));
-        ck_assert(strstr(pl->str, "#11 __do_softirq"));
-        ck_assert(strstr(pl->str, "#12 handle_irq"));
-        ck_assert(strstr(pl->str, "#13 irq_exit"));
-        ck_assert(strstr(pl->str, "#14 do_IRQ"));
-        ck_assert(strstr(pl->str, "#15 pci_write"));
-        ck_assert(strstr(pl->str, "#16 common_interrupt"));
-        ck_assert(strstr(pl->str, "#17 audit_make_tree"));
-        ck_assert(strstr(pl->str, "#18 usbdev_ioctl"));
-        ck_assert(strstr(pl->str, "#19 _raw_spin_unlock_irqrestore"));
-        ck_assert(strstr(pl->str, "#20 __setup_irq"));
-        ck_assert(strstr(pl->str, "#21 kmem_cache_alloc_trace"));
-        ck_assert(strstr(pl->str, "#22 vsnprintf"));
-        ck_assert(strstr(pl->str, "#23 request_threaded_irq"));
-        ck_assert(strstr(pl->str, "#24 usb_hcd_platform_shutdown"));
-        ck_assert(strstr(pl->str, "#25 request_threaded_irq"));
-        ck_assert(strstr(pl->str, "#26 usb_add_hcd"));
-        ck_assert(strstr(pl->str, "#27 usb_hcd_pci_probe"));
-        ck_assert(strstr(pl->str, "#28 rpm_resume"));
-        ck_assert(strstr(pl->str, "#29 pci_device_probe"));
-        ck_assert(strstr(pl->str, "#30 driver_probe_device"));
-        ck_assert(strstr(pl->str, "#31 pci_match_device"));
-        ck_assert(strstr(pl->str, "#32 __driver_attach"));
-        ck_assert(strstr(pl->str, "#33 driver_probe_device"));
-        ck_assert(strstr(pl->str, "#34 bus_for_each_dev"));
-        ck_assert(strstr(pl->str, "#35 driver_attach"));
-        ck_assert(strstr(pl->str, "#36 driver_probe_device"));
-        ck_assert(strstr(pl->str, "#37 bus_add_driver"));
-        ck_assert(strstr(pl->str, "#38 pci_match_id"));
-        ck_assert(strstr(pl->str, "#39 driver_register"));
-        ck_assert(strstr(pl->str, "#40 ohci_hcd_mod_init"));
-        ck_assert(strstr(pl->str, "#41 ohci_hcd_mod_init"));
-        ck_assert(strstr(pl->str, "#42 __pci_register_driver"));
-        ck_assert(strstr(pl->str, "#43 uhci_hcd_init"));
-        ck_assert(strstr(pl->str, "#44 do_one_initcall"));
-        ck_assert(strstr(pl->str, "#45 kernel_init_freeable"));
-        ck_assert(strstr(pl->str, "#46 do_early_param"));
+        ck_assert(strstr(pl->str, "#9 ? do_IRQ"));
+        ck_assert(strstr(pl->str, "#10 ? common_interrupt"));
+        ck_assert(strstr(pl->str, "#11 ? __do_softirq"));
+        ck_assert(strstr(pl->str, "#12 ? handle_irq"));
+        ck_assert(strstr(pl->str, "#13 ? irq_exit"));
+        ck_assert(strstr(pl->str, "#14 ? do_IRQ"));
+        ck_assert(strstr(pl->str, "#15 ? pci_write"));
+        ck_assert(strstr(pl->str, "#16 ? common_interrupt"));
+        ck_assert(strstr(pl->str, "#17 ? audit_make_tree"));
+        ck_assert(strstr(pl->str, "#18 ? usbdev_ioctl"));
+        ck_assert(strstr(pl->str, "#19 ? _raw_spin_unlock_irqrestore"));
+        ck_assert(strstr(pl->str, "#20 ? __setup_irq"));
+        ck_assert(strstr(pl->str, "#21 ? kmem_cache_alloc_trace"));
+        ck_assert(strstr(pl->str, "#22 ? vsnprintf"));
+        ck_assert(strstr(pl->str, "#23 ? request_threaded_irq"));
+        ck_assert(strstr(pl->str, "#24 ? usb_hcd_platform_shutdown"));
+        ck_assert(strstr(pl->str, "#25 ? request_threaded_irq"));
+        ck_assert(strstr(pl->str, "#26 ? usb_add_hcd"));
+        ck_assert(strstr(pl->str, "#27 ? usb_hcd_pci_probe"));
+        ck_assert(strstr(pl->str, "#28 ? rpm_resume"));
+        ck_assert(strstr(pl->str, "#29 ? pci_device_probe"));
+        ck_assert(strstr(pl->str, "#30 ? driver_probe_device"));
+        ck_assert(strstr(pl->str, "#31 ? pci_match_device"));
+        ck_assert(strstr(pl->str, "#32 ? __driver_attach"));
+        ck_assert(strstr(pl->str, "#33 ? driver_probe_device"));
+        ck_assert(strstr(pl->str, "#34 ? bus_for_each_dev"));
+        ck_assert(strstr(pl->str, "#35 ? driver_attach"));
+        ck_assert(strstr(pl->str, "#36 ? driver_probe_device"));
+        ck_assert(strstr(pl->str, "#37 ? bus_add_driver"));
+        ck_assert(strstr(pl->str, "#38 ? pci_match_id"));
+        ck_assert(strstr(pl->str, "#39 ? driver_register"));
+        ck_assert(strstr(pl->str, "#40 ? ohci_hcd_mod_init"));
+        ck_assert(strstr(pl->str, "#41 ? ohci_hcd_mod_init"));
+        ck_assert(strstr(pl->str, "#42 ? __pci_register_driver"));
+        ck_assert(strstr(pl->str, "#43 ? uhci_hcd_init"));
+        ck_assert(strstr(pl->str, "#44 ? do_one_initcall"));
+        ck_assert(strstr(pl->str, "#45 ? kernel_init_freeable"));
+        ck_assert(strstr(pl->str, "#46 ? do_early_param"));

        g_string_free(pl, true);

@@ -521,12 +521,12 @@ START_TEST(general_protection_fault_payload)
        ck_assert(strstr(pl->str, "Modules : tun 8021q garp mrp bridge stp llc bonding ipt_REJECT xt_comment xt_multiport"));
        ck_assert(strstr(pl->str, "lpc_ich iw_cm mfd_core nfsd auth_rpcgss wmi nfs_acl lockd ipmi_si ipmi_devintf"));

-        ck_assert(strstr(pl->str, "#1 alloc_pipe_info"));
+        ck_assert(strstr(pl->str, "#1 ? alloc_pipe_info"));
        ck_assert(strstr(pl->str, "#2 alloc_pipe_info"));
        ck_assert(strstr(pl->str, "#3 create_pipe_files"));
        ck_assert(strstr(pl->str, "#4 __do_pipe_flags"));
        ck_assert(strstr(pl->str, "#5 SyS_pipe2"));
-        ck_assert(strstr(pl->str, "#6 __audit_syscall_exit"));
+        ck_assert(strstr(pl->str, "#6 ? __audit_syscall_exit"));
        ck_assert(strstr(pl->str, "#7 SyS_pipe"));
        ck_assert(strstr(pl->str, "#8 system_call_fastpath"));

@@ -594,29 +594,76 @@ START_TEST(bug_kernel_handle_payload)
        ck_assert(strstr(pl->str, "Modules : nf_conntrack_netbios_ns nf_conntrack_broadcast ipt_MASQUERADE be2iscsi"));
        ck_assert(strstr(pl->str, "drm_kms_helper firewire_ohci drm firewire_core tg3 crc_itu_t i2c_core video usb_storage sunrpc"));

-        ck_assert(strstr(pl->str, "#1 _nv007312rm"));
-        ck_assert(strstr(pl->str, "#2 _nv007847rm"));
-        ck_assert(strstr(pl->str, "#3 _nv004049rm"));
-        ck_assert(strstr(pl->str, "#4 _nv004049rm"));
-        ck_assert(strstr(pl->str, "#5 _nv010019rm"));
-        ck_assert(strstr(pl->str, "#6 _nv014983rm"));
-        ck_assert(strstr(pl->str, "#7 _nv001097rm"));
-        ck_assert(strstr(pl->str, "#8 rm_init_adapter"));
+        ck_assert(strstr(pl->str, "#1 ? _nv007312rm"));
+        ck_assert(strstr(pl->str, "#2 ? _nv007847rm"));
+        ck_assert(strstr(pl->str, "#3 ? _nv004049rm"));
+        ck_assert(strstr(pl->str, "#4 ? _nv004049rm"));
+        ck_assert(strstr(pl->str, "#5 ? _nv010019rm"));
+        ck_assert(strstr(pl->str, "#6 ? _nv014983rm"));
+        ck_assert(strstr(pl->str, "#7 ? _nv001097rm"));
+        ck_assert(strstr(pl->str, "#8 ? rm_init_adapter"));
+        ck_assert(strstr(pl->str, "#9 ? nv_kern_open"));
+        ck_assert(strstr(pl->str, "#10 ? chrdev_open"));
+        ck_assert(strstr(pl->str, "#11 ? do_dentry_open"));
+        ck_assert(strstr(pl->str, "#12 ? cdev_put"));
+        ck_assert(strstr(pl->str, "#13 ? finish_open"));
+        ck_assert(strstr(pl->str, "#14 ? do_last"));
+        ck_assert(strstr(pl->str, "#15 ? inode_permission"));
+        ck_assert(strstr(pl->str, "#16 ? link_path_walk"));
+        ck_assert(strstr(pl->str, "#17 ? path_openat"));
+        ck_assert(strstr(pl->str, "#18 ? do_filp_open"));
+        ck_assert(strstr(pl->str, "#19 ? __alloc_fd"));
+        ck_assert(strstr(pl->str, "#20 ? do_sys_open"));
+        ck_assert(strstr(pl->str, "#21 ? __audit_syscall_entry"));
+        ck_assert(strstr(pl->str, "#22 ? sys_open"));
+        ck_assert(strstr(pl->str, "#23 ? system_call_fastpath"));
+
+        g_string_free(pl, true);
+
+}
+END_TEST
+
+START_TEST(bug_kernel_handle_payload_new_format)
+{
+        char *oopsfile = NULL;
+
+        oopsfile = TESTOOPSDIR "/bug_kernel_handle_new.txt";
+        setup_payload(oopsfile);
+
+        telem_log(LOG_ERR, "Bug kernel handle backtrace: %s\n", pl->str);
+
+        ck_assert(pl->len > 0);
+        ck_assert_str_eq(reason, "BUG: unable to handle kernel paging request at 0000000000002658");
+
+        ck_assert(strstr(pl->str, "Kernel Version : 3.7.9-201.fc18.x86_64 #1 Apple Inc. MacBookPro6,2/Mac-F22586C8"));
+        ck_assert(strstr(pl->str, "Tainted : PF"));
+
+        ck_assert(strstr(pl->str, "Modules : nf_conntrack_netbios_ns nf_conntrack_broadcast ipt_MASQUERADE be2iscsi"));
+        ck_assert(strstr(pl->str, "drm_kms_helper firewire_ohci drm firewire_core tg3 crc_itu_t i2c_core video usb_storage sunrpc"));
+
+        ck_assert(strstr(pl->str, "#1 ? _nv007312rm"));
+        ck_assert(strstr(pl->str, "#2 ? _nv007847rm"));
+        ck_assert(strstr(pl->str, "#3 ? _nv004049rm"));
+        ck_assert(strstr(pl->str, "#4 ? _nv004049rm"));
+        ck_assert(strstr(pl->str, "#5 ? _nv010019rm"));
+        ck_assert(strstr(pl->str, "#6 ? _nv014983rm"));
+        ck_assert(strstr(pl->str, "#7 ? _nv001097rm"));
+        ck_assert(strstr(pl->str, "#8 ? rm_init_adapter"));
        ck_assert(strstr(pl->str, "#9 nv_kern_open"));
-        ck_assert(strstr(pl->str, "#10 chrdev_open"));
-        ck_assert(strstr(pl->str, "#11 do_dentry_open"));
-        ck_assert(strstr(pl->str, "#12 cdev_put"));
-        ck_assert(strstr(pl->str, "#13 finish_open"));
-        ck_assert(strstr(pl->str, "#14 do_last"));
-        ck_assert(strstr(pl->str, "#15 inode_permission"));
+        ck_assert(strstr(pl->str, "#10 ? chrdev_open"));
+        ck_assert(strstr(pl->str, "#11 ? do_dentry_open"));
+        ck_assert(strstr(pl->str, "#12 ? cdev_put"));
+        ck_assert(strstr(pl->str, "#13 ? finish_open"));
+        ck_assert(strstr(pl->str, "#14 ? do_last"));
+        ck_assert(strstr(pl->str, "#15 ? inode_permission"));
        ck_assert(strstr(pl->str, "#16 link_path_walk"));
        ck_assert(strstr(pl->str, "#17 path_openat"));
-        ck_assert(strstr(pl->str, "#18 do_filp_open"));
-        ck_assert(strstr(pl->str, "#19 __alloc_fd"));
-        ck_assert(strstr(pl->str, "#20 do_sys_open"));
-        ck_assert(strstr(pl->str, "#21 __audit_syscall_entry"));
-        ck_assert(strstr(pl->str, "#22 sys_open"));
-        ck_assert(strstr(pl->str, "#23 system_call_fastpath"));
+        ck_assert(strstr(pl->str, "#18 ? do_filp_open"));
+        ck_assert(strstr(pl->str, "#19 ? __alloc_fd"));
+        ck_assert(strstr(pl->str, "#20 ? do_sys_open"));
+        ck_assert(strstr(pl->str, "#21 ? __audit_syscall_entry"));
+        ck_assert(strstr(pl->str, "#22 ? sys_open"));
+        ck_assert(strstr(pl->str, "#23 ? system_call_fastpath"));

        g_string_free(pl, true);

@@ -648,6 +695,7 @@ Suite *config_suite(void)
        tcase_add_test(t, double_fault_payload);
        tcase_add_test(t, bad_page_map_payload);
        tcase_add_test(t, bug_kernel_handle_payload);
+        tcase_add_test(t, bug_kernel_handle_payload_new_format);

        //TODO fix
        //tcase_add_test(t, badness_payload);
@@ -72,6 +72,7 @@ EXTRA_DIST += \
 	%D%/oops_test_files/badness.txt \
 	%D%/oops_test_files/bad_page_map.txt \
 	%D%/oops_test_files/bug_kernel_handle.txt \
+	%D%/oops_test_files/bug_kernel_handle_new.txt \
 	%D%/oops_test_files/double_fault_DONE.txt \
 	%D%/oops_test_files/double_fault.txt \
 	%D%/oops_test_files/general_protection_fault.txt \
@@ -0,0 +1,51 @@
+<4>[    68764.975947] BUG: unable to handle kernel paging request at 0000000000002658
+<4>[    68764.975947] IP: [<ffffffffa07f92e5>] _nv007245rm+0x54/0xc4 [nvidia]
+<4>[    68764.975947] PGD 22d005067 PUD 22d006067 PMD 0 
+<4>[    68764.975947] Oops: 0000 [#1] SMP 
+<4>[    68764.975947] Modules linked in: nf_conntrack_netbios_ns nf_conntrack_broadcast ipt_MASQUERADE be2iscsi iscsi_boot_sysfs bnx2i cnic uio cxgb4i cxgb4 cxgb3i cxgb3 mdio ip6table_mangle libcxgbi ib_iser rdma_cm ib_addr iw_cm ib_cm ib_sa ib_mad ib_core iscsi_tcp libiscsi_tcp ip6t_REJECT libiscsi scsi_transport_iscsi nf_conntrack_ipv6 nf_defrag_ipv6 iptable_nat nf_nat_ipv4 nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ebtable_filter ebtables rfcomm ip6table_filter ip6_tables bnep nls_utf8 hfsplus binfmt_misc snd_hda_codec_hdmi arc4 brcmsmac cordic brcmutil mac80211 cfg80211 joydev iTCO_wdt iTCO_vendor_support nvidia(POF) coretemp applesmc input_polldev microcode uvcvideo i2c_i801 intel_ips snd_hda_codec_cirrus videobuf2_vmalloc videobuf2_memops videobuf2_core videodev btusb media bluetooth snd_hda_intel snd_hda_codec rfkill snd_hwdep bcm5974 snd_seq snd_seq_device snd_pcm lpc_ich mfd_core bcma snd_page_alloc snd_timer vhost_net tun snd macvtap macvlan soundcore kvm_intel kvm apple_gmux apple_bl uinput crc32c_intel i915 ghash_clmulni_intel i2c_algo_bit drm_kms_helper firewire_ohci drm firewire_core tg3 crc_itu_t i2c_core video usb_storage sunrpc
+<4>[    68764.975947] CPU 2 
+<4>[    68764.975947] Pid: 1059, comm: Xorg Tainted: PF          O 3.7.9-201.fc18.x86_64 #1 Apple Inc. MacBookPro6,2/Mac-F22586C8
+<4>[    68764.975947] RIP: 0010:[<ffffffffa07f92e5>]  [<ffffffffa07f92e5>] _nv007245rm+0x54/0xc4 [nvidia]
+<4>[    68764.975947] RSP: 0018:ffff88022d0639b8  EFLAGS: 00010282
+<4>[    68764.975947] RAX: 0000000000000000 RBX: ffff88022d190008 RCX: 0000000000000000
+<4>[    68764.975947] RDX: 0000000000000000 RSI: 0000000000000015 RDI: 0000000000000000
+<4>[    68764.975947] RBP: ffff88022d30df08 R08: 0000000000000002 R09: ffff88023f342508
+<4>[    68764.975947] R10: ffff88022d951030 R11: 0000000000000000 R12: 0000000000000000
+<4>[    68764.975947] R13: ffff88023f3b0008 R14: 0000000000000000 R15: ffff88022da96008
+<4>[    68764.975947] FS:  00007f771f4f2940(0000) GS:ffff88024fc80000(0000) knlGS:0000000000000000
+<4>[    68764.975947] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
+<4>[    68764.975947] CR2: 0000000000002658 CR3: 000000022d004000 CR4: 00000000000007e0
+<4>[    68764.975947] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
+<4>[    68764.975947] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
+<4>[    68764.975947] Process Xorg
+<4>[    68764.975947] Stack:
+<4>[    68764.975947]  ffff88022d33a008 ffff88022d33a008 0000000000000002 ffff88023f3b0008
+<4>[    68764.975947]  ffff88022d190008 ffffffffa07deec8 ffff88023d67b008 ffff88022d190008
+<4>[    68764.975947]  ffff88023f3b0008 ffff88023f3b0008 ffff88022da94808 ffffffffa083e77f
+<4>[    68764.975947] Call Trace:
+<4>[    68764.975947]  ? _nv007312rm+0x7f8/0x938 [nvidia]
+<4>[    68764.975947]  ? _nv007847rm+0x80/0x276 [nvidia]
+<4>[    68764.975947]  ? _nv004049rm+0x85fb/0xaef0 [nvidia]
+<4>[    68764.975947]  ? _nv004049rm+0x705f/0xaef0 [nvidia]
+<4>[    68764.975947]  ? _nv010019rm+0x25/0x40 [nvidia]
+<4>[    68764.975947]  ? _nv014983rm+0x7c9/0x943 [nvidia]
+<4>[    68764.975947]  ? _nv001097rm+0x42b/0x661 [nvidia]
+<4>[    68764.975947]  ? rm_init_adapter+0xac/0x146 [nvidia]
+<4>[    68764.975947]  nv_kern_open+0x4b4/0x800 [nvidia]
+<4>[    68764.975947]  ? chrdev_open+0x9b/0x180
+<4>[    68764.975947]  ? do_dentry_open+0x203/0x290
+<4>[    68764.975947]  ? cdev_put+0x30/0x30
+<4>[    68764.975947]  ? finish_open+0x35/0x50
+<4>[    68764.975947]  ? do_last+0x6de/0xe00
+<4>[    68764.975947]  ? inode_permission+0x18/0x50
+<4>[    68764.975947]  link_path_walk+0x77/0x870
+<4>[    68764.975947]  path_openat+0xbc/0x4d0
+<4>[    68764.975947]  ? do_filp_open+0x41/0xa0
+<4>[    68764.975947]  ? __alloc_fd+0x42/0x110
+<4>[    68764.975947]  ? do_sys_open+0xf4/0x1e0
+<4>[    68764.975947]  ? __audit_syscall_entry+0xcc/0x300
+<4>[    68764.975947]  ? sys_open+0x21/0x30
+<4>[    68764.975947]  ? system_call_fastpath+0x16/0x1b
+<4>[    68764.975947] Code: e0 05 00 00 00 00 00 00 41 bc 00 00 00 00 eb 18 44 89 e2 48 89 de 4c 89 ef ff 93 48 1a 00 00 01 83 e0 05 00 00 41 ff c4 4c 89 f7 <41> ff 96 58 26 00 00 44 39 e0 77 d9 ba 74 06 10 00 be 00 00 00 
+<4>[    68764.975947] RIP  [<ffffffffa07f92e5>] _nv007245rm+0x54/0xc4 [nvidia]
+<4>[    68764.975947]  RSP <ffff88022d0639b8>
Author	SHA1	Message	Date
Patrick McCarty	356be6738f	Release v1.11.2 This release includes a couple of fixes for the man pages. Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>	2017-04-06 09:59:36 -07:00
Auke Kok	e73870b916	Fix wrong reference to the config file in man pages.	2017-04-06 09:57:50 -07:00
Auke Kok	c7e783f2b1	Add FILES section, explain static machine id file.	2017-04-06 09:57:50 -07:00
Patrick McCarty	3bb04aff3f	Release v1.11.1 This release includes updates for man pages and improves support for the new oops format in Linux 4.10 and newer. Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>	2017-03-22 10:49:26 -07:00
Patrick McCarty	26fc7b69bb	Fix `make distcheck` I forgot to add the new test file to EXTRA_DIST, so add it here. Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>	2017-03-22 10:48:54 -07:00
Auke Kok	e3ede3cd88	Refresh the existing man pages' output. Made with `make manpages` of course.	2017-03-21 22:57:07 -07:00
Auke Kok	dd44e91d4a	Convert man pages to RST.	2017-03-21 22:57:07 -07:00
Patrick McCarty	a403d884bc	tests: add test case for new oops format Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>	2017-03-21 22:55:03 -07:00
Patrick McCarty	e699fd34e2	oops parser: support oops stack traces for Linux 4.10+ Linux 4.10 introduced some format changes for oopses; one of the changes was removing memory addresses from each stack frame. This commit adds support for the new format and retains compatibility for the previous format (4.9 and earlier). Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>	2017-03-21 22:55:03 -07:00
Patrick McCarty	266039a4e5	oops parser: add/use skip_space; fix skip_spaces The former skip_spaces() function only skipped a single space, so rename it to skip_space(), and make skip_spaces() greedily consume spaces. Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>	2017-03-21 22:55:03 -07:00
Patrick McCarty	3d24527096	Release v1.11.0 This release reworks the journal probe to send records for log messages with higher log levels (LOG_EMERG, LOG_ALERT, LOG_CRIT, LOG_ERR), and also for messages that match EXIT_CODE="exited", which is set for systemd services that have exited with an error code. Also, a new opt-in feature has been added to disable certain privacy filters in the probes. With the filters disabled, the crash probe will collect backtraces for any crashing binary on the system, regardless of where the binary is installed. Also, the oops probe will collect real register values from kernel oopses instead of using "zero" or "non-zero". The opt in feature described above can be enabled by creating a file at: /etc/telemetrics/opt-in-no-privacy-filters Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>	2017-02-17 11:39:33 -08:00
Patrick McCarty	9b435f39e5	journal probe: remove obsolete LOG_DEBUG message A return value of 0 can also indicate send_data() failure, so remove this log message. Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>	2017-02-17 11:35:25 -08:00
Patrick McCarty	20c051fe59	Document the logical expression for journal entry matching Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>	2017-02-17 11:35:25 -08:00
Patrick McCarty	96c1cd5547	journal probe: filter LOG_EMERG logs as well I omitted log level 0 from the filter, so add it here. Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>	2017-02-17 11:35:25 -08:00
Patrick McCarty	e375fe6825	journal probe: send records for every log message The previous logic concatenates log messages on initial startup, when the entire journal is read to process existing messages. But doing so might run into the payload size limit (8KB), and thus fail to create a record. Sending one record per log message will ensure that the payload size remains relatively small, almost always below the 8KB size limit. Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>	2017-02-17 11:35:25 -08:00
Patrick McCarty	160a4d01bf	journal probe: filter on EXIT_CODE as well This field appears to be set when services fail, so filter on it as well. Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>	2017-02-17 11:35:25 -08:00
Patrick McCarty	3f7b839cd8	journal probe: use the new macros Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>	2017-02-17 11:35:25 -08:00
Patrick McCarty	07595885c4	journal probe: add macros to help with using the journal API To avoid having to read the repetitive error handling when adding journal filters, add some helper macros. Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>	2017-02-17 11:35:25 -08:00
Patrick McCarty	625eea615e	journal probe: fix formatting of some log messages Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>	2017-02-17 11:35:25 -08:00
Patrick McCarty	8a69b22895	journal probe: add newlines for to each message in the payload The make payloads more legible, make sure separate messages are newline separated. Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>	2017-02-17 11:35:25 -08:00
Patrick McCarty	b4a0989d0e	journal probe: match all messages ERR or higher Previous behavior was to only filter LOG_ERR messages from the telemetrics crashprobe, but it will be helpful to make this probe more generic to capture log messages with the highest log levels. Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>	2017-02-17 11:35:25 -08:00
Patrick McCarty	41f53507d1	journal probe: fix error handling of read_new_entries() The error case is negative, so fix the return code conditional check appropriately. Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>	2017-02-17 11:35:25 -08:00
Patrick McCarty	3aeb31ee3d	Add opt-in feature to disable privacy filters There are some filters in place that prevent probes from collecting specific types of data. In particular, the oops probe will report "zero" or "non-zero" for register values in oopses instead of the actual values. And the crash probe does not send backtraces if binaries live outside of /usr or under /usr/local. This commit keeps these filters in place but offers the capability to disable the filters by creating a file named /etc/telemetrics/opt-in-no-privacy-filters Any future telemetry privacy filters can check for this file to alter the reporting level. Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>	2017-02-17 10:56:55 -08:00
Patrick McCarty	4ce8ff9990	Clarify path filtering logic in the crash probe Since only one path is being checked for a program, and it returns a boolean, rename the function and call site appropriately. Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>	2017-02-17 10:56:55 -08:00
Patrick McCarty	b671e770c9	Release v1.10.2 This release improves reliability of both the crash and oops probes. * The crash probe will now reprocess a core file if it encounters missing symbols when creating backtraces the first time. This improves telemetry when run on Clear Linux OS, which downloads debuginfo on-the-fly as needed. * The oops parsing will now capture "unreliable" stack frames for kernel oopses using the same notation as the kernel ("? "). Previously, the parsing code ignored the unreliable flag, so those frames appeared to be trustworthy frames. Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>	2017-02-06 14:50:43 -08:00
Patrick McCarty	2409452740	crash probe: retry processing when missing symbols Missing symbols in backtraces might indicate missing debuginfo from the crashed program, but another possibility is that on-the-fly debuginfo has not finished downloading. The Clear Linux OS uses an on-the-fly debuginfo setup, so this condition is likely for the first instance of a program crashing on the system. To decrease likelihood of missing symbols appearing, scan the backtrace for a frame with incomplete data after first processing, and if found, reprocess the core file after a 10 second pause. Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>	2017-02-06 14:28:17 -08:00
Patrick McCarty	ecc60b6a3c	crash probe: split core process routine out of main For future reuse of the code for processing a core file, move it to a new function, and call it from main(). Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>	2017-02-06 14:28:17 -08:00
Patrick McCarty	579e7a8674	crash probe: split elfutils boilerplate out of main For future reuse of the code to initialize elfutils for processing a core file, move it to a new function, and call it from main(). Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>	2017-02-06 14:28:17 -08:00
Patrick McCarty	003928f52e	Preserve unreliable frames for kernel oopses When frame addresses detected during the stack scan were not previously found by unwinding, the string "? " is added as a prefix to the function name. However, the current oops parsing code strips "? " if encountered, so the backtraces from kernel oopses are missing vital information; the presence of "? " provides a hint for debugging a stack trace and indicates that the frame info is "unreliable". This commit removes the "? " strip code, ensuring the prefix is retained by the function name, and updates unit tests that check for oops lines that should contain the prefix. Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>	2017-02-02 12:22:22 -08:00
Patrick McCarty	c54bdab485	build: echo build configuration for configure It's often useful to print a summary of configuration options for the build, so add a AC_MSG_NOTICE() to the tail end of the configure script. Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>	2017-01-20 17:35:42 -08:00
Robert Nesius	2af1d7e742	v1.10.1 Since I mistagged v1.10.0 in github.	2016-12-09 02:15:39 +00:00
Robert Nesius	278c27f9a3	v1.10.0	2016-12-09 02:09:49 +00:00
Auke Kok	3e4360e2cb	Docs: Add manual pages using `ronn`. This adds several new manual pages to the project, ships them using `dist` and has a simple target to recreate the new roff files using the `ronn` utility: `make manpages`. These man pages are still very crude and contain the absolute minimum amount of information. I've attempted to cross link them as much as is reasonable. The telemetry API is documented in `telemetry(3)`. The important functions themselves are `linked` with a ref link that should make `man tm_create_record` (etc.) link to telemetry(3). I attempted to make this pass `make distcheck` and `make install` properly, but this should still be checked.	2016-12-08 17:47:25 -08:00
John Andersen	cb622a0eb0	Validation for classification to ensure 2 slashes (#4 ) * Validation for classification to ensure 2 slashes Signed-off-by: John Andersen <john.s.andersen@intel.com> * Update telem_record_gen.c Fixed code style issues	2016-09-02 00:43:48 -07:00
Jeremy Rocher	44e0d4d3c5	crashprobe: fix stderr when launched from kernel stderr could be used for logging or in libtelemetry in DEBUG mode but when crashprobe is launched through kernel core_pattern, stderr is not open, only stdin (fd 0), see this dump from gdb: (gdb) shell ls -l /proc/15478/fd total 0 lr-x------ 1 root root 64 24 juin 15:02 0 -> pipe:[82555] If stderr is not open, when socket to telemd is open it could take fd 2 and in this case all fprint to stderr are written to socket which is bad ... this is the case when doing static linking, which ends to telemd crashed. Fix it by ensuring stderr is open. Signed-off-by: Jeremy Rocher <jeremy.rocher@intel.com>	2016-08-17 21:09:33 +00:00
Loic Poulain	36f4854967	Make daemon recycling configurable The user may want to disable recycling (daemon auto exit). This can be the case if telemd daemon is not configured to respawn or if service is automatically restarted by the system after package update. This patch introduces the daemon_recycling_enabled config entry.	2016-08-17 21:08:24 +00:00
Jeremy Rocher	524d5d2345	fix telem_perror when logging to stderr When logging to stderr, telem_perror output is incomplete and badly formatted. Print errno correctly, fix newline and fix includes. Signed-off-by: Jeremy Rocher <jeremy.rocher@intel.com>	2016-08-17 13:42:39 -07:00