Compare commits

...
20 Commits
43 ... 50
Author SHA1 Message Date
Patrick McCarty bca3bdefda Release v50
- Convert clr_debug_prepare to a bash script, also incorporating a few
  improvements and bug fixes into the rewrite.

Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>
2019-06-05 15:34:59 -07:00
Patrick McCarty 179392c601 Convert clr_debug_prepare to a bash script
For easier parallelization, I opted to replace the C program with a bash
script that uses GNU Parallel, find, and GAWK for the core control flow.

For compatibility, running the shell script without arguments has the
same general behavior as the C program. Some bugs have been fixed as
well, so the end result should be an improvement over the original.

Bugs fixed:

- A race condition in the C program with the "unsymlink" operation has
  been addressed: instead of replacing the valid symlinks within
  debuginfo.raw, the symlink targets are added directly to the automatic
  tarballs, and tar's "transform" feature is used to rewrite the filename.

- Running system("tar ... &") to create the tarballs often had the side
  effect of bogging down the system, since the number of active tar
  processes had no upper bound. The shell script rewrite uses GNU
  Parallel, which limits (by default) the number of parallel jobs.

- Directory tarballs were unintentionally re-created whenever content
  within the directories were modified, due to the stat() mtime changing.
  With the new script, directory tarball creation only occurs for those
  directories that do not already exist at the destination.

Once the script begins to create tarballs, performance is largely the
same as the C program. The script has a "precompute" phase where it
determines the list of needed tarball creations up-front, which is
different (and probably a bit slower) than the on-the-fly stat()s used
by the C program, but the runtime cost of this new precompute phase is
negligible compared to the tarball creation phase.

Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>
2019-06-05 15:28:20 -07:00
Patrick McCarty b802948209 Release v49
- List tarball contents prior to extraction to improve robustness
- Plug some memory leaks
- Print errors to stderr instead of stdout

Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>
2019-05-24 13:04:07 -07:00
Auke Kok 4495b923a9 Redo tar extraction - try and make this safe(r).
Instead of extracting to a tmp folder, just tar `tf` it first. This
keeps extracting to the proper locations trivial, at the cost of
decompression the content twice.
2019-05-10 11:08:06 -07:00
Auke Kok 8f7289f028 Test result of tar extraction before trusting the result.
We extract the tarball to a temporary file, before actually allowing
gdb to use this file. This allows us to make sure that gdb doesn't
see the file if it is corrupt. Only if tar exceeds, we rename() the
temp result into the actual needed file. If tar fails, we throw away
the file.
2019-05-07 16:19:36 -07:00
Auke Kok dc376d5e8b Update to v48 2019-01-15 12:47:32 -08:00
Auke Kok c831ecb902 Use select() to time out, and exit.
With socket activation, we can now select() and time out before accept()
and exit, releasing all resource we used back to the system.

The default timeout is set for 10 minutes. If no more connections
arrive, the daemon exits.

Socket activation is done through the clr_debug_daemon.socket unit, which
the clr_debug_fuse service requires, so this should be extra robust when
starting up.

The daemon ignores stat() requests for trash that gnome sends when it
sees the fuse system mounted, and returns the underlying stat() for
the root node instead of a remote value, because there is no value
in fetching this info over curl.
2019-01-09 12:58:42 -08:00
Auke Kok 0978d429b2 Clean up some of the socket activation stuff. 2019-01-08 12:37:56 -08:00
Patrick McCarty 6653992aba Release v47
Fixes a bug with clr_debug_prepare's handling of broken symlinks.

Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>
2018-12-10 17:10:44 -08:00
Patrick McCarty 0af0308dd2 Revert "have tar follow symlinks"
After closer inspection, I've determined that prior to this commit,
clr_debug_prepare was behaving properly with respect to the handling of
symlinks in debuginfo packages. Reverting this commit fixes the issue.

Details:

 - Symlinks under /usr/lib/debug/.build-id that are not broken should be
 unsymlinked so that the actual debuginfo is added to the automatic
 debuginfo tarballs. In the debuginfo packages, these files have a
 .debug extension and point to files under /usr/lib/debug/usr.

 - Symlinks under /usr/lib/debug/.build-id that *are* broken cannot be
 unsymlinked, and therefore, tar should not follow these symlinks,
 because fatal errors (and empty tarballs) are the result. These
 symlinks lack the .debug extension and point to the corresponding
 libraries or binaries under /usr when exposed via clr_debug_fuse. So it
 is correct for the symlinks to be broken when packaged by
 clr_debug_prepare, since they point to non-debuginfo files, and the
 debuginfo.raw directory tree only contains debuginfo content.

This reverts commit 7dcddfd0ca8f5539ef34028a1c680f6e82dcf246.
2018-12-10 17:04:05 -08:00
Patrick McCarty 8ace503438 Release v46
This release fixes a build issue when not passing -Wl,--copy-dt-needed-entries
to the compiler.

Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>
2018-05-25 21:56:43 -07:00
Patrick McCarty 2c87f83adc build: add LIBSYSTEMD_LIBS to link line
For the regular Clear Linux build, the -lsystemd option was not needed at link
time due to -Wl,--copy-dt-needed-entries being used. Without that option
though, the linking of clr_debug_daemon will fail.

Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>
2018-05-25 21:55:28 -07:00
Patrick McCarty 67b9b3acc8 Update gitignore
Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>
2018-05-25 21:38:06 -07:00
Arjan van de Ven d108c0af64 add cdn-alt 2018-05-17 13:54:16 +00:00
Auke Kok 98e802b4cc Also install socket unit. 2018-04-24 13:04:33 -07:00
Auke Kok 612801b0c8 Make -daemon socket activated. 2018-04-23 15:15:33 -07:00
Arjan van de Ven 3c2ad26baa have tar follow symlinks 2017-12-23 00:49:24 +00:00
Arjan van de Ven d20d585e5c much shorter timeouts 2017-12-10 14:25:07 +00:00
Icarus Sparry 1f3a0f0be4 Extract tar file as dbginfo
Make /var/cache/debuginfo and its contents owned by dgbinfo. Create
the directories as this user, and switch to run as this user.

Signed-off-by: Icarus Sparry <icarus.w.sparry@intel.com>
2017-10-31 13:35:29 -07:00
Icarus Sparry d2d67bdb1d Change tmpfiles.d configuration
Use the dbginfo user for the directories.

Signed-off-by: Icarus Sparry <icarus.w.sparry@intel.com>
2017-10-31 09:47:39 -07:00
11 changed files with 331 additions and 305 deletions
+1 -1
View File
@@ -2,12 +2,12 @@
*.o *.o
clr_debug_fuse clr_debug_fuse
clr_debug_daemon clr_debug_daemon
clr_debug_prepare
.deps/ .deps/
.dirstamp .dirstamp
Makefile Makefile
Makefile.in Makefile.in
aclocal.m4 aclocal.m4
ar-lib
autom4te.cache/ autom4te.cache/
compile compile
config.h config.h
+8 -8
View File
@@ -1,4 +1,4 @@
EXTRA_DIST = COPYING clr_debug_fuse.service clr_debug_daemon.service debuginfo.conf EXTRA_DIST = COPYING clr_debug_fuse.service clr_debug_daemon.service clr_debug_daemon.socket debuginfo.conf
DISTCHECK_CONFIGURE_FLAGS = \ DISTCHECK_CONFIGURE_FLAGS = \
--with-systemdsystemunitdir=$$dc_install_base/$(systemdsystemunitdir) \ --with-systemdsystemunitdir=$$dc_install_base/$(systemdsystemunitdir) \
@@ -14,7 +14,9 @@ AM_CFLAGS = \
AM_CPPFLAGS = $(AM_CFLAGS) ${curl_CFLAGS} ${fuse_CFLAGS} AM_CPPFLAGS = $(AM_CFLAGS) ${curl_CFLAGS} ${fuse_CFLAGS}
bin_PROGRAMS = clr_debug_fuse clr_debug_daemon clr_debug_prepare bin_PROGRAMS = clr_debug_fuse clr_debug_daemon
dist_bin_SCRIPTS = scripts/clr_debug_prepare
noinst_LTLIBRARIES = \ noinst_LTLIBRARIES = \
libnica.la libnica.la
@@ -33,15 +35,13 @@ clr_debug_fuse_SOURCES = src/fuse.c src/client.c
clr_debug_daemon_SOURCES = src/server.c clr_debug_daemon_SOURCES = src/server.c
clr_debug_daemon_CFLAGS = \ clr_debug_daemon_CFLAGS = \
-pthread \ -pthread \
$(AM_CFLAGS) $(AM_CFLAGS) \
$(LIBSYSTEMD_CFLAGS)
clr_debug_prepare_SOURCES = src/prepare.c
clr_debug_fuse_LDADD = ${fuse_LIBS} libnica.la clr_debug_fuse_LDADD = ${fuse_LIBS} libnica.la
clr_debug_daemon_LDADD = ${curl_LIBS} libnica.la clr_debug_daemon_LDADD = ${curl_LIBS} libnica.la ${LIBSYSTEMD_LIBS}
clr_debug_prepare_LDADD = libnica.la
systemdsystemunit_DATA = clr_debug_fuse.service clr_debug_daemon.service systemdsystemunit_DATA = clr_debug_fuse.service clr_debug_daemon.service clr_debug_daemon.socket
tmpfiles_DATA = debuginfo.conf tmpfiles_DATA = debuginfo.conf
+9
View File
@@ -0,0 +1,9 @@
[Unit]
Description=Clear Linux OS debuginfo daemon
[Socket]
ListenStream=/run/clr-debug-info
SocketMode=0600
[Install]
WantedBy=sockets.target
+2 -1
View File
@@ -1,6 +1,7 @@
[Unit] [Unit]
Description=Clear Linux debuginfo fuse monitor Description=Clear Linux debuginfo fuse monitor
After=clr_debug_daemon.service After=clr_debug_daemon.socket
Requires=clr_debug_daemon.socket
DefaultDependencies=no DefaultDependencies=no
[Service] [Service]
+5 -1
View File
@@ -2,17 +2,21 @@
# Process this file with autoconf to produce a configure script. # Process this file with autoconf to produce a configure script.
AC_PREREQ([2.66]) AC_PREREQ([2.66])
AC_INIT(clr-debug-info, 43, arjan@linux.intel.com) AC_INIT(clr-debug-info, 50, arjan@linux.intel.com)
AM_INIT_AUTOMAKE([foreign -Wall -W subdir-objects]) AM_INIT_AUTOMAKE([foreign -Wall -W subdir-objects])
AM_SILENT_RULES([yes]) AM_SILENT_RULES([yes])
AC_PROG_CC AC_PROG_CC
AM_PROG_AR
AC_LANG(C) AC_LANG(C)
AC_CONFIG_HEADERS([config.h]) AC_CONFIG_HEADERS([config.h])
PKG_CHECK_MODULES([curl], [libcurl]) PKG_CHECK_MODULES([curl], [libcurl])
PKG_CHECK_MODULES([fuse], [fuse]) PKG_CHECK_MODULES([fuse], [fuse])
PKG_CHECK_MODULES([SYSTEMD], [systemd]) PKG_CHECK_MODULES([SYSTEMD], [systemd])
PKG_CHECK_MODULES([LIBSYSTEMD], [libsystemd])
LT_INIT LT_INIT
AC_DEFINE([SOCKET_PATH], ["/run/clr-debug-info"], [path to communication socket])
dir="" dir=""
AC_ARG_WITH([systemdsystemunitdir], AS_HELP_STRING([--with-systemdsystemunitdir=DIR], AC_ARG_WITH([systemdsystemunitdir], AS_HELP_STRING([--with-systemdsystemunitdir=DIR],
[path to systemd system service dir @<:@default=/usr/lib/systemd/system@:>@]), [dir=${withval}], [path to systemd system service dir @<:@default=/usr/lib/systemd/system@:>@]), [dir=${withval}],
+4 -2
View File
@@ -8,5 +8,7 @@
# See tmpfiles.d(5) for details # See tmpfiles.d(5) for details
# Clear tmp directories separately, to make them easier to override # Clear tmp directories separately, to make them easier to override
d /var/cache/debuginfo/lib 755 root root 10d # Unfortunatly tmpfiles doesn't change the ownership for things if they
d /var/cache/debuginfo/src 755 root root 1d # are not listed.
d /var/cache/debuginfo/lib 755 dbginfo dbginfo 10d
d /var/cache/debuginfo/src 755 dbginfo dbginfo 1d
+146
View File
@@ -0,0 +1,146 @@
#!/bin/bash
#
# Clear Linux -- automatic debuginfo preparation
#
# Copyright 2019 Intel Corporation
#
# This program is free software: you can redistribute it and/or modify it under
# the terms of the GNU General Public License as published by the Free Software
# Foundation, version 3 or later of the License.
#
# This program is distributed in the hope that it will be useful, but WITHOUT
# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
# FOR A PARTICULAR PURPOSE. See the GNU General Public License for more
# details.
#
# You should have received a copy of the GNU General Public License along with
# this program. If not, see <http://www.gnu.org/licenses/>.
if [ $# -eq 1 ] || [ $# -gt 2 ]; then
echo "Usage: $0 [SOURCEDIR] [DESTDIR]"
echo "Generates automatic debuginfo tarballs in DESTDIR using content from SOURCEDIR."
echo "The default SOURCEDIR and DESTDIR are /var/www/html/debuginfo.raw and"
echo "/var/www/html/debuginfo, respectively."
exit 0
fi >&2
if [ $# -eq 2 ]; then
if [ ! -d "$1" ]; then
echo "ERROR: SOURCEDIR argument must be an existing directory"
exit 1
fi
fi >&2
export SRC="${1:-/var/www/html/debuginfo.raw}"
export DEST="${2:-/var/www/html/debuginfo}"
srclist=$(mktemp -p .)
destlist=$(mktemp -p .)
trap "rm $srclist $destlist" EXIT
set -o pipefail
# First, scan the state of the SRC tree, containing all content for debuginfo RPMs.
if ! find "$SRC" -mindepth 4 -printf "%p %T@ %y\n" | sed "s|$SRC||" | sort -k 1 > "$srclist"; then
echo "ERROR: expected content in $SRC"
exit 1
fi
# Next, scan the state of the DEST tree, containing previously generated
# automatic debuginfo tarballs. Will be empty on first run.
mkdir -p "$DEST"
find "$DEST" -mindepth 2 -name '*.tar' -printf "%p %T@ %y\n" | sed "s|$DEST||;s|\.tar | |" | sort -k 1 > "$destlist"
process_one() {
srcname="$1"
destname="$2"
filetype="$3"
srcloc="$SRC/$srcname"
srcdir="$SRC/$(echo "$srcname" | sed -r 's@^(/usr/(lib|share|src)/debug(/src)?).*@\1@')"
tarcontent="$(echo "$srcname" | sed -r 's@^/usr/(lib|share|src)/debug(/src)?@.@')"
destdir="$DEST/$(dirname "$destname")"
dest="$DEST/$destname.tar"
echo "Creating $destname.tar ..."
mkdir -p "$destdir"
if [ "$filetype" = "f" ]; then
tar --no-recursion -C "$srcdir" -Jcf "$dest" "$tarcontent"
elif [ "$filetype" = "l" ]; then
# We "unsymlink" non-broken symlinks as an optimization by adding the
# symlink target to the dest tarball, transforming the name as appropriate.
if target=$(realpath -e "$srcloc" --relative-to="$srcdir" 2> /dev/null); then
tar --no-recursion -C "$srcdir" -Jcf "$dest" --transform='s|.*|'"$tarcontent"'|' "$target"
else
tar --no-recursion -C "$srcdir" -Jcf "$dest" "$tarcontent"
fi
elif [ "$filetype" = "d" ]; then
tar --no-recursion -C "$srcdir" -Jcf "$dest" "$tarcontent"
fi
}
export -f process_one
gawk '
BEGIN { OFS = "\t" }
LIST == "src" {
name = $1
srcname = name
ret = sub(/^\/usr\/lib\/debug/, "/lib", name)
if (ret == 0) {
ret = sub(/^\/usr\/src\/debug/, "/src", name)
if (ret == 0) {
ret = sub(/^\/usr\/share\/debug\/src/, "/src", name)
if (ret == 0) {
ret = sub(/^\/usr\/share\/debug/, "/lib", name)
if (ret == 0) {
print "Error: unexpected debuginfo file:", name
exit 1
}
}
}
}
src[name]["srcname"] = srcname
src[name]["time"] = $2
src[name]["type"] = $3
src[name]["done"] = 0
}
LIST == "dest" {
dest_content = 1
name = $1
time = $2
type = $3
if (name in src) {
# No need to recreate directory tars if they exist
if (src[name]["type"] == "d") {
src[name]["done"] = 1
next
} else {
# Only regenerate content if timestamp is newer
if (time < src[name]["time"]) {
print src[name]["srcname"], name, src[name]["type"]
}
src[name]["done"] = 1
}
}
}
END {
PROCINFO["sorted_in"] = "@ind_str_asc"
if (dest_content == 0) {
for (s in src) {
print src[s]["srcname"], s, src[s]["type"]
}
} else {
for (s in src) {
if (src[s]["done"] == 0) {
print src[s]["srcname"], s, src[s]["type"]
}
}
}
}
' LIST="src" "$srclist" LIST="dest" "$destlist" \
| parallel --colsep '\t' process_one
# vi: ft=sh et sw=2 sts=2
+8 -7
View File
@@ -36,9 +36,11 @@
#include <time.h> #include <time.h>
#include <unistd.h> #include <unistd.h>
#include "config.h"
/* 0.75 seconds timeout */ /* 0.75 seconds timeout */
#define TIMEOUT 750000 #define TIMEOUT 75000
#define TIMEOUT2 15000 #define TIMEOUT2 1500
#define TIMEOUT3 500 #define TIMEOUT3 500
char *prefix = "src"; char *prefix = "src";
@@ -58,7 +60,7 @@ void try_to_get(const char *path, int pid, time_t timestamp)
int shorttime = 0; int shorttime = 0;
__nc_unused__ size_t wr = -1; __nc_unused__ size_t wr = -1;
// printf("Trying to aquire %s\n", path); // printf("Trying to aquire %s\n", path);
sockfd = socket(AF_UNIX, SOCK_STREAM, 0); sockfd = socket(AF_UNIX, SOCK_STREAM, 0);
if (sockfd < 0) { if (sockfd < 0) {
@@ -66,14 +68,13 @@ void try_to_get(const char *path, int pid, time_t timestamp)
} }
sun.sun_family = AF_UNIX; sun.sun_family = AF_UNIX;
strcpy(sun.sun_path, ":clr-debug-info"); strcpy(sun.sun_path, SOCKET_PATH);
sun.sun_path[0] = 0; /* anonymous unix socket */
ret = connect(sockfd, ret = connect(sockfd,
(struct sockaddr *)&sun, (struct sockaddr *)&sun,
offsetof(struct sockaddr_un, sun_path) + strlen(":clr-debug-info") + 1); offsetof(struct sockaddr_un, sun_path) + strlen(SOCKET_PATH) + 1);
if (ret < 0) { if (ret < 0) {
printf("Cannot connect %s\n", strerror(errno)); printf("Cannot connect to %s: %s\n", SOCKET_PATH, strerror(errno));
close(sockfd); close(sockfd);
return; return;
} }
+10
View File
@@ -75,6 +75,16 @@ static int xmp_getattr(const char *path, struct stat *stbuf)
memset(stbuf, 0, sizeof(struct stat)); memset(stbuf, 0, sizeof(struct stat));
res = lstat(newpath, stbuf); res = lstat(newpath, stbuf);
/*
* filter out things that never should get fetched
* this prevents us from asking curl to fetch us useless things
*/
if (strncmp(path, "/.Trash", strlen("/.Trash")) == 0) {
return -ENOENT;
} else if (strcmp(path, "/") == 0) {
return 0;
}
/* /*
* get the file. if the st_mtime is set, this is just an async refresh, otherwise it's * get the file. if the st_mtime is set, this is just an async refresh, otherwise it's
* a synchronous request. * a synchronous request.
-243
View File
@@ -1,243 +0,0 @@
/*
* Fenrus Linux -- automatic debug information installation
* Clear Linux -- automatic debug information installation
* Server side preparation logic
*
* Copyright (C) 2013 Arjan van de Ven
* Copyright (C) 2014 Intel Corporation
*
* This program is free software: you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
* the Free Software Foundation, version 2 or later of the License.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with this program. If not, see <http://www.gnu.org/licenses/>.
*
* Authors:
* Arjan van de Ven <arjanvandeven@gmail.com>
* Ikey Doherty <michael.i.doherty@intel.com>
*
*/
#define _GNU_SOURCE
#include <dirent.h>
#include <errno.h>
#include <libgen.h>
#include <stddef.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <sys/socket.h>
#include <sys/stat.h>
#include <sys/types.h>
#include <sys/types.h>
#include <sys/un.h>
#include <unistd.h>
#include "nica/files.h"
/*
this program takes a directory /var/www/html/debuginfo.raw,
in which all debuginfo rpms have previously been rpm2cpio'd, and
creates the content for /var/www/html/debuginfo
which contains tar'd up/compressed versions of all the files.
As part of this symlinks will be resolved server side
as much as possible.
*/
static void unsymlink(char *filename)
{
char *target;
__nc_unused__ int r;
target = canonicalize_file_name(filename);
// sleep(1);
// printf("Symlink %s points to %s\n", filename, target);
unlink(filename);
r = link(target, filename);
}
static void wait_for_loadavg(void)
{
FILE *file;
char line[4096];
__nc_unused__ char *ret = NULL;
while (1) {
double d;
file = fopen("/proc/loadavg", "r");
if (!file) {
return;
}
line[0] = 0;
ret = fgets(line, 4096, file);
fclose(file);
d = strtod(line, NULL);
if (d < 50) {
return;
}
sleep(5);
}
}
int tarcount;
static void do_one_file(char *base1, char *base2, char *path, int isdir)
{
char *dir2 = NULL;
autofree(char) * fullpath1, *fullpath2, *dir = NULL;
struct stat buf1, buf2;
int ret;
int sret;
if (asprintf(&fullpath1, "%s%s", base1, path) < 0) {
return;
}
if (asprintf(&fullpath2, "%s%s.tar", base2, path) < 0) {
return;
}
ret = lstat(fullpath1, &buf1);
if (ret) {
return;
}
if (S_ISLNK(buf1.st_mode)) {
// printf("%s is a symlink\n", fullpath1);
ret = stat(fullpath1, &buf2);
if (!ret) {
unsymlink(fullpath1);
}
ret = lstat(fullpath1, &buf1);
}
ret = lstat(fullpath2, &buf2);
if (ret || buf1.st_mtime > buf2.st_mtime) {
char *command = NULL;
unlink(fullpath2);
dir = strdup(fullpath2);
dir2 = dirname(dir);
if (dir2) {
(void)nc_mkdir_p(dir2, 00755);
}
if (chdir(base1) != 0) {
fprintf(stderr, "Failed to chdir: %s\n", strerror(errno));
return;
}
if (!isdir &&
asprintf(&command,
"tar --no-recursion -C %s -Jcf %s %s &",
base1,
fullpath2,
path) >= 0) {
// printf("Running -%s-\n", command);
printf("Processing %s\n", fullpath2);
tarcount++;
if (tarcount > 50) {
wait_for_loadavg();
tarcount = 0;
}
if ((sret = system(command)) != 0) {
fprintf(stderr, "Failure in command: [%d] %s\n", sret, command);
}
free(command);
}
if (isdir &&
asprintf(&command,
"tar --no-recursion -C %s -Jcf %s `find %s -type d` `find %s "
"-maxdepth 1 -type l` &",
base1,
fullpath2,
path,
path) >= 0) {
// printf("Running -%s-\n", command);
printf("Processing %s\n", fullpath2);
tarcount++;
if (tarcount > 50) {
wait_for_loadavg();
tarcount = 0;
}
if ((sret = system(command)) != 0) {
fprintf(stderr, "Failure in command: [%d] %s\n", sret, command);
}
free(command);
}
}
}
static void recurse_dir(char *base1, char *base2, char *path)
{
DIR *dir;
char *fullpath1 = NULL;
struct dirent *entry;
if (asprintf(&fullpath1, "%s%s", base1, path) < 0) {
return;
}
dir = opendir(fullpath1);
if (!dir) {
return;
}
while (1) {
autofree(char) *fullpath2 = NULL;
autofree(char) *newpath = NULL;
entry = readdir(dir);
if (!entry) {
break;
}
if (strcmp(entry->d_name, ".") == 0) {
continue;
}
if (strcmp(entry->d_name, "..") == 0) {
continue;
}
newpath = NULL;
fullpath2 = NULL;
if (asprintf(&fullpath2, "%s/%s", fullpath1, entry->d_name) < 0) {
return;
}
if (asprintf(&newpath, "%s/%s", path, entry->d_name) >= 0) {
struct stat sb;
stat(fullpath2, &sb);
if (entry->d_type == DT_DIR || S_ISDIR(sb.st_mode)) {
do_one_file(base1, base2, newpath, 1);
recurse_dir(base1, base2, newpath);
} else {
do_one_file(base1, base2, newpath, 0);
}
}
}
closedir(dir);
}
int main(int argc, char **argv)
{
if (argc < 0)
(void)argv;
recurse_dir("/var/www/html/debuginfo.raw/usr/lib/debug/",
"/var/www/html/debuginfo/lib/",
".");
recurse_dir("/var/www/html/debuginfo.raw/usr/src/debug/",
"/var/www/html/debuginfo/src/",
".");
return EXIT_SUCCESS;
}
+137 -41
View File
@@ -2,7 +2,7 @@
* Clear Linux -- automatic debug information installation * Clear Linux -- automatic debug information installation
* *
* Copyright (C) 2013 Arjan van de Ven * Copyright (C) 2013 Arjan van de Ven
* Curl portions borrowed from the Fenrus Update code * Curl portions borrowed from the Fenrus Update code
* which in part is (C) 2012 Intel Corporation * which in part is (C) 2012 Intel Corporation
* Copyright (C) 2014 Intel Corporation * Copyright (C) 2014 Intel Corporation
* *
@@ -27,9 +27,12 @@
#define _GNU_SOURCE #define _GNU_SOURCE
#include <errno.h> #include <errno.h>
#include <grp.h>
#include <linux/capability.h> #include <linux/capability.h>
#include <malloc.h> #include <malloc.h>
#include <pthread.h> #include <pthread.h>
#include <pwd.h>
#include <libgen.h>
#include <signal.h> #include <signal.h>
#include <stddef.h> #include <stddef.h>
#include <stdio.h> #include <stdio.h>
@@ -47,16 +50,20 @@
#include <curl/curl.h> #include <curl/curl.h>
#include "systemd/sd-daemon.h"
#include "config.h" #include "config.h"
#ifdef HAVE_ATOMIC_SUPPORT #ifdef HAVE_ATOMIC_SUPPORT
#include <stdatomic.h> #include <stdatomic.h>
#endif #endif
#define TIMEOUT 600 /* 10 minutes */
static pthread_mutex_t dupes_mutex = PTHREAD_MUTEX_INITIALIZER; static pthread_mutex_t dupes_mutex = PTHREAD_MUTEX_INITIALIZER;
char *urls[2] = { "https://cdn.download.clearlinux.org/debuginfo/", char *urls[2] = { "https://cdn.download.clearlinux.org/debuginfo/",
"https://cdn.download.clearlinux.org/debuginfo/" }; "https://cdn-alt.download.clearlinux.org/debuginfo/" };
int urlcounter = 1; int urlcounter = 1;
static NcHashmap *hash = NULL; static NcHashmap *hash = NULL;
@@ -164,10 +171,9 @@ static int curl_get_file(const char *url, const char *prefix, time_t timestamp)
long ret; long ret;
long changed; long changed;
int fd; int fd;
char filename[PATH_MAX]; autofree(char) *filename = NULL;
CURL *curl = NULL; CURL *curl = NULL;
FILE *file; FILE *file;
struct stat statbuf;
if (avoid_dupes(url)) { if (avoid_dupes(url)) {
return 300; return 300;
@@ -178,12 +184,15 @@ static int curl_get_file(const char *url, const char *prefix, time_t timestamp)
return 301; return 301;
} }
strcpy(filename, "/tmp/clr-debug-info-XXXXXX"); // fprintf(stderr, "Fetching %s, prefix %s, path %s\n", url, prefix, path);
if (asprintf(&filename, "/tmp/clr-debug-info-XXXXXX") < 0) {
curl_easy_cleanup(curl);
return 418;
}
fd = mkstemp(filename); fd = mkstemp(filename);
if (fd < 0) { if (fd < 0) {
curl_easy_cleanup(curl); curl_easy_cleanup(curl);
return 500; return 418;
} }
file = fdopen(fd, "w"); file = fdopen(fd, "w");
@@ -221,7 +230,7 @@ static int curl_get_file(const char *url, const char *prefix, time_t timestamp)
curl_easy_getinfo(curl, CURLINFO_RESPONSE_CODE, &ret); curl_easy_getinfo(curl, CURLINFO_RESPONSE_CODE, &ret);
fflush(file); fflush(file);
// printf("HTTP return code is %i\n", ret); // printf("HTTP return code is %i\n", ret);
/* HTTP 304 is returned if (a) the cached debuginfo has the same /* HTTP 304 is returned if (a) the cached debuginfo has the same
* timestamp or is newer than that on the server and (b) we haven't * timestamp or is newer than that on the server and (b) we haven't
@@ -234,6 +243,9 @@ static int curl_get_file(const char *url, const char *prefix, time_t timestamp)
} }
if (ret == 200) { if (ret == 200) {
autofree(char) *command = NULL;
struct stat statbuf;
/* get timestamp, if any */ /* get timestamp, if any */
curl_easy_getinfo(curl, CURLINFO_FILETIME, &changed); curl_easy_getinfo(curl, CURLINFO_FILETIME, &changed);
if (changed >= 0) { if (changed >= 0) {
@@ -245,21 +257,45 @@ static int curl_get_file(const char *url, const char *prefix, time_t timestamp)
futimens(fd, times); futimens(fd, times);
} }
autofree(char) *command = NULL;
// printf("Filename is %s\n", filename);
memset(&statbuf, 0, sizeof(statbuf)); memset(&statbuf, 0, sizeof(statbuf));
stat(filename, &statbuf); stat(filename, &statbuf);
if (statbuf.st_size > 0 && if (statbuf.st_size <= 0) {
asprintf(&command, ret = 418;
"tar -C /var/cache/debuginfo/%s --no-same-owner --no-same-permissions -xf %s", goto out;
}
/* test extraction first */
if (asprintf(&command, "tar -C /var/cache/debuginfo/%s --no-same-owner "
"--no-same-permissions -tf %s",
prefix, prefix,
filename) >= 0) { filename) < 0) {
if (system(command) != 0) { ret = 418;
fputs("Warning: tar extraction failed\n", stderr); goto out;
} }
if (system(command) != 0) {
ret = 418;
fprintf(stderr, "Error: tar validation failed\n");
goto out;
}
free(command); /* reuse */
if (asprintf(&command, "tar -C /var/cache/debuginfo/%s --no-same-owner "
"--no-same-permissions -xf %s",
prefix,
filename) < 0) {
ret = 418;
goto out;
}
if (system(command) != 0) {
ret = 418;
fprintf(stderr, "Error: tar extraction failed\n");
goto out;
} }
} }
out:
unlink(filename); unlink(filename);
curl_easy_cleanup(curl); curl_easy_cleanup(curl);
fclose(file); fclose(file);
@@ -333,7 +369,7 @@ static void *server_thread(void *arg)
goto thread_end; goto thread_end;
} }
// printf("Getting url %s %i:%06i\n", url, before.tv_sec, before.tv_usec); // printf("Getting url %s %i:%06i\n", url, before.tv_sec, before.tv_usec);
ret = curl_get_file(url, prefix, timestamp); ret = curl_get_file(url, prefix, timestamp);
switch (ret) { switch (ret) {
@@ -344,7 +380,7 @@ static void *server_thread(void *arg)
// ignore these error codes // ignore these error codes
break; break;
default: default:
printf("Request for %s resulted in error %i\n", url, ret); fprintf(stderr, "Request for %s resulted in error %i\n", url, ret);
break; break;
} }
@@ -375,8 +411,19 @@ int main(__nc_unused__ int argc, __nc_unused__ char **argv)
struct sockaddr_un sun; struct sockaddr_un sun;
int ret; int ret;
int curl_done = 0; int curl_done = 0;
uid_t dbg_user = 0;
gid_t dbg_group = 0;
struct passwd *passwdentry;
const char *required_paths[] = { "/var/cache/debuginfo/lib", "/var/cache/debuginfo/src" }; const char *required_paths[] = { "/var/cache/debuginfo/lib", "/var/cache/debuginfo/src" };
umask(0);
passwdentry = getpwnam("dbginfo");
if (passwdentry) {
dbg_user = passwdentry->pw_uid;
dbg_group = passwdentry->pw_gid;
}
endpwent();
if (prctl(PR_SET_DUMPABLE, 0) != 0) { if (prctl(PR_SET_DUMPABLE, 0) != 0) {
fprintf(stderr, fprintf(stderr,
"Failed to disable PR_SET_DUMPABLE. Do NOT gdb attach this process: %s\n", "Failed to disable PR_SET_DUMPABLE. Do NOT gdb attach this process: %s\n",
@@ -391,44 +438,95 @@ int main(__nc_unused__ int argc, __nc_unused__ char **argv)
for (size_t i = 0; i < ARRAY_SIZE(required_paths); i++) { for (size_t i = 0; i < ARRAY_SIZE(required_paths); i++) {
const char *req_path = required_paths[i]; const char *req_path = required_paths[i];
if (nc_file_exists(req_path)) { struct stat st = { .st_ino = 0 };
continue; if (lstat(req_path, &st) == 0) {
/* If the file already exists, check ownership
* and delete tree if incorrect. Essentially a
* one-off operation to transition from root owned to
* dbginfo owned */
if (st.st_uid == dbg_user) {
continue;
}
fprintf(stderr, "Removing old debug information %s\n", req_path);
nc_rm_rf(req_path);
} }
if (!nc_mkdir_p(req_path, 00755)) { if (!nc_mkdir_p(req_path, 00755)) {
fprintf(stderr, "Failed to mkdir: %s %s\n", strerror(errno), req_path); fprintf(stderr, "Failed to mkdir: %s %s\n", strerror(errno), req_path);
return EXIT_FAILURE; return EXIT_FAILURE;
} }
if (chown(req_path, dbg_user, dbg_group) != 0) {
fprintf(stderr, "Failed to chown: %s %s\n", strerror(errno), req_path);
return EXIT_FAILURE;
}
} }
signal(SIGPIPE, SIG_IGN); signal(SIGPIPE, SIG_IGN);
sockfd = socket(AF_UNIX, SOCK_STREAM, 0); if (sd_listen_fds(0) == 1) {
if (sockfd < 0) { /* systemd socket activation */
return EXIT_FAILURE; sockfd = SD_LISTEN_FDS_START + 0;
} else if (sd_listen_fds(0) > 1) {
fprintf(stderr, "Too many file descriptors received.\n");
exit(EXIT_FAILURE);
} else {
sockfd = socket(AF_UNIX, SOCK_STREAM, 0);
if (sockfd < 0) {
exit(EXIT_FAILURE);
}
sun.sun_family = AF_UNIX;
strcpy(sun.sun_path, SOCKET_PATH);
ret = bind(sockfd,
(struct sockaddr *)&sun,
offsetof(struct sockaddr_un, sun_path) + strlen(SOCKET_PATH) + 1);
if (ret < 0) {
fprintf(stderr, "Failed to bind:%s \n", strerror(errno));
exit(EXIT_FAILURE);
}
if (listen(sockfd, 16) < 0) {
fprintf(stderr, "Failed to listen:%s \n", strerror(errno));
exit(EXIT_FAILURE);
}
} }
sun.sun_family = AF_UNIX; if (setgid(dbg_group)) {
strcpy(sun.sun_path, ":clr-debug-info"); fprintf(stderr, "Unable to drop privileges setgid %s\n", strerror(errno));
sun.sun_path[0] = 0; /* anonymous unix socket */ exit(EXIT_FAILURE);
ret = bind(sockfd,
(struct sockaddr *)&sun,
offsetof(struct sockaddr_un, sun_path) + strlen(":clr-debug-info") + 1);
if (ret < 0) {
printf("Failed to bind:%s \n", strerror(errno));
return EXIT_FAILURE;
} }
if (setgroups(1, &dbg_group)) {
if (listen(sockfd, 16) < 0) { fprintf(stderr, "Unable to drop privileges setgroups %s\n", strerror(errno));
printf("Failed to listen:%s \n", strerror(errno)); exit(EXIT_FAILURE);
return EXIT_FAILURE; }
if (setuid(dbg_user)) {
fprintf(stderr, "Unable to drop privileges setuid %s\n", strerror(errno));
exit(EXIT_FAILURE);
} }
while (1) { while (1) {
fd_set rfds;
struct timeval tv;
int ret;
int clientsock; int clientsock;
pthread_t thread; pthread_t thread;
malloc_trim(0); malloc_trim(0);
/* use select() to timeout and exit gracefully */
FD_ZERO(&rfds);
FD_SET(sockfd, &rfds);
tv.tv_sec = TIMEOUT;
tv.tv_usec = 0;
ret = select(sockfd + 1, &rfds, NULL, NULL, &tv);
if (ret == -1) {
perror("select()");
exit(EXIT_FAILURE);
} else if (ret == 0) {
break;
}
clientsock = accept(sockfd, NULL, NULL); clientsock = accept(sockfd, NULL, NULL);
/* Too many connections, wait for the next loop/retry */ /* Too many connections, wait for the next loop/retry */
@@ -453,6 +551,4 @@ int main(__nc_unused__ int argc, __nc_unused__ char **argv)
if (hash) { if (hash) {
nc_hashmap_free(hash); nc_hashmap_free(hash);
} }
close(sockfd);
} }