Merge branch 'for-mingo' of git://git.kernel.org/pub/scm/linux/kernel/git/paulmck/linux-rcu into core/rcu

Pull v5.10 RCU changes from Paul E. McKenney:

- Debugging for smp_call_function().

- Strict grace periods for KASAN.  The point of this series is to find
  RCU-usage bugs, so the corresponding new RCU_STRICT_GRACE_PERIOD
  Kconfig option depends on both DEBUG_KERNEL and RCU_EXPERT, and is
  further disabled by dfefault.  Finally, the help text includes
  a goodly list of scary caveats.

- New smp_call_function() torture test.

- Torture-test updates.

- Documentation updates.

- Miscellaneous fixes.

Signed-off-by: Ingo Molnar <mingo@kernel.org>
This commit is contained in:
Ingo Molnar
2020-10-09 08:21:56 +02:00
57 changed files with 1583 additions and 422 deletions
@@ -1,12 +1,12 @@
#!/bin/bash
# SPDX-License-Identifier: GPL-2.0+
#
# Analyze a given results directory for rcuperf performance measurements,
# Analyze a given results directory for rcuscale performance measurements,
# looking for ftrace data. Exits with 0 if data was found, analyzed, and
# printed. Intended to be invoked from kvm-recheck-rcuperf.sh after
# printed. Intended to be invoked from kvm-recheck-rcuscale.sh after
# argument checking.
#
# Usage: kvm-recheck-rcuperf-ftrace.sh resdir
# Usage: kvm-recheck-rcuscale-ftrace.sh resdir
#
# Copyright (C) IBM Corporation, 2016
#
@@ -1,9 +1,9 @@
#!/bin/bash
# SPDX-License-Identifier: GPL-2.0+
#
# Analyze a given results directory for rcuperf performance measurements.
# Analyze a given results directory for rcuscale scalability measurements.
#
# Usage: kvm-recheck-rcuperf.sh resdir
# Usage: kvm-recheck-rcuscale.sh resdir
#
# Copyright (C) IBM Corporation, 2016
#
@@ -20,7 +20,7 @@ fi
PATH=`pwd`/tools/testing/selftests/rcutorture/bin:$PATH; export PATH
. functions.sh
if kvm-recheck-rcuperf-ftrace.sh $i
if kvm-recheck-rcuscale-ftrace.sh $i
then
# ftrace data was successfully analyzed, call it good!
exit 0
@@ -30,12 +30,12 @@ configfile=`echo $i | sed -e 's/^.*\///'`
sed -e 's/^\[[^]]*]//' < $i/console.log |
awk '
/-perf: .* gps: .* batches:/ {
/-scale: .* gps: .* batches:/ {
ngps = $9;
nbatches = $11;
}
/-perf: .*writer-duration/ {
/-scale: .*writer-duration/ {
gptimes[++n] = $5 / 1000.;
sum += $5 / 1000.;
}
@@ -43,7 +43,7 @@ awk '
END {
newNR = asort(gptimes);
if (newNR <= 0) {
print "No rcuperf records found???"
print "No rcuscale records found???"
exit;
}
pct50 = int(newNR * 50 / 100);
@@ -79,5 +79,5 @@ END {
print "99th percentile grace-period duration: " gptimes[pct99];
print "Maximum grace-period duration: " gptimes[newNR];
print "Grace periods: " ngps + 0 " Batches: " nbatches + 0 " Ratio: " ngps / nbatches;
print "Computed from rcuperf printk output.";
print "Computed from rcuscale printk output.";
}'
+38
View File
@@ -0,0 +1,38 @@
#!/bin/bash
# SPDX-License-Identifier: GPL-2.0+
#
# Analyze a given results directory for rcutorture progress.
#
# Usage: kvm-recheck-rcu.sh resdir
#
# Copyright (C) Facebook, 2020
#
# Authors: Paul E. McKenney <paulmck@kernel.org>
i="$1"
if test -d "$i" -a -r "$i"
then
:
else
echo Unreadable results directory: $i
exit 1
fi
. functions.sh
configfile=`echo $i | sed -e 's/^.*\///'`
nscfs="`grep 'scf_invoked_count ver:' $i/console.log 2> /dev/null | tail -1 | sed -e 's/^.* scf_invoked_count ver: //' -e 's/ .*$//' | tr -d '\015'`"
if test -z "$nscfs"
then
echo "$configfile ------- "
else
dur="`sed -e 's/^.* scftorture.shutdown_secs=//' -e 's/ .*$//' < $i/qemu-cmd 2> /dev/null`"
if test -z "$dur"
then
rate=""
else
nscfss=`awk -v nscfs=$nscfs -v dur=$dur '
BEGIN { print nscfs / dur }' < /dev/null`
rate=" ($nscfss/s)"
fi
echo "${configfile} ------- ${nscfs} SCF handler invocations$rate"
fi
@@ -66,6 +66,7 @@ config_override_param () {
echo > $T/KcList
config_override_param "$config_dir/CFcommon" KcList "`cat $config_dir/CFcommon 2> /dev/null`"
config_override_param "$config_template" KcList "`cat $config_template 2> /dev/null`"
config_override_param "--gdb options" KcList "$TORTURE_KCONFIG_GDB_ARG"
config_override_param "--kasan options" KcList "$TORTURE_KCONFIG_KASAN_ARG"
config_override_param "--kcsan options" KcList "$TORTURE_KCONFIG_KCSAN_ARG"
config_override_param "--kconfig argument" KcList "$TORTURE_KCONFIG_ARG"
@@ -152,7 +153,11 @@ qemu_append="`identify_qemu_append "$QEMU"`"
boot_args="`configfrag_boot_params "$boot_args" "$config_template"`"
# Generate kernel-version-specific boot parameters
boot_args="`per_version_boot_params "$boot_args" $resdir/.config $seconds`"
echo $QEMU $qemu_args -m $TORTURE_QEMU_MEM -kernel $KERNEL -append \"$qemu_append $boot_args\" > $resdir/qemu-cmd
if test -n "$TORTURE_BOOT_GDB_ARG"
then
boot_args="$boot_args $TORTURE_BOOT_GDB_ARG"
fi
echo $QEMU $qemu_args -m $TORTURE_QEMU_MEM -kernel $KERNEL -append \"$qemu_append $boot_args\" $TORTURE_QEMU_GDB_ARG > $resdir/qemu-cmd
if test -n "$TORTURE_BUILDONLY"
then
@@ -171,14 +176,26 @@ echo "NOTE: $QEMU either did not run or was interactive" > $resdir/console.log
# Attempt to run qemu
( . $T/qemu-cmd; wait `cat $resdir/qemu_pid`; echo $? > $resdir/qemu-retval ) &
commandcompleted=0
sleep 10 # Give qemu's pid a chance to reach the file
if test -s "$resdir/qemu_pid"
if test -z "$TORTURE_KCONFIG_GDB_ARG"
then
qemu_pid=`cat "$resdir/qemu_pid"`
echo Monitoring qemu job at pid $qemu_pid
else
qemu_pid=""
echo Monitoring qemu job at yet-as-unknown pid
sleep 10 # Give qemu's pid a chance to reach the file
if test -s "$resdir/qemu_pid"
then
qemu_pid=`cat "$resdir/qemu_pid"`
echo Monitoring qemu job at pid $qemu_pid
else
qemu_pid=""
echo Monitoring qemu job at yet-as-unknown pid
fi
fi
if test -n "$TORTURE_KCONFIG_GDB_ARG"
then
echo Waiting for you to attach a debug session, for example: > /dev/tty
echo " gdb $base_resdir/vmlinux" > /dev/tty
echo 'After symbols load and the "(gdb)" prompt appears:' > /dev/tty
echo " target remote :1234" > /dev/tty
echo " continue" > /dev/tty
kstarttime=`gawk 'BEGIN { print systime() }' < /dev/null`
fi
while :
do
+31 -5
View File
@@ -31,6 +31,9 @@ TORTURE_DEFCONFIG=defconfig
TORTURE_BOOT_IMAGE=""
TORTURE_INITRD="$KVM/initrd"; export TORTURE_INITRD
TORTURE_KCONFIG_ARG=""
TORTURE_KCONFIG_GDB_ARG=""
TORTURE_BOOT_GDB_ARG=""
TORTURE_QEMU_GDB_ARG=""
TORTURE_KCONFIG_KASAN_ARG=""
TORTURE_KCONFIG_KCSAN_ARG=""
TORTURE_KMAKE_ARG=""
@@ -46,6 +49,7 @@ jitter="-1"
usage () {
echo "Usage: $scriptname optional arguments:"
echo " --allcpus"
echo " --bootargs kernel-boot-arguments"
echo " --bootimage relative-path-to-kernel-boot-image"
echo " --buildonly"
@@ -55,17 +59,19 @@ usage () {
echo " --defconfig string"
echo " --dryrun sched|script"
echo " --duration minutes"
echo " --gdb"
echo " --help"
echo " --interactive"
echo " --jitter N [ maxsleep (us) [ maxspin (us) ] ]"
echo " --kconfig Kconfig-options"
echo " --kmake-arg kernel-make-arguments"
echo " --mac nn:nn:nn:nn:nn:nn"
echo " --memory megabytes | nnnG"
echo " --memory megabytes|nnnG"
echo " --no-initrd"
echo " --qemu-args qemu-arguments"
echo " --qemu-cmd qemu-system-..."
echo " --results absolute-pathname"
echo " --torture rcu"
echo " --torture lock|rcu|rcuscale|refscale|scf"
echo " --trust-make"
exit 1
}
@@ -126,6 +132,14 @@ do
dur=$(($2*60))
shift
;;
--gdb)
TORTURE_KCONFIG_GDB_ARG="CONFIG_DEBUG_INFO=y"; export TORTURE_KCONFIG_GDB_ARG
TORTURE_BOOT_GDB_ARG="nokaslr"; export TORTURE_BOOT_GDB_ARG
TORTURE_QEMU_GDB_ARG="-s -S"; export TORTURE_QEMU_GDB_ARG
;;
--help|-h)
usage
;;
--interactive)
TORTURE_QEMU_INTERACTIVE=1; export TORTURE_QEMU_INTERACTIVE
;;
@@ -184,13 +198,13 @@ do
shift
;;
--torture)
checkarg --torture "(suite name)" "$#" "$2" '^\(lock\|rcu\|rcuperf\|refscale\)$' '^--'
checkarg --torture "(suite name)" "$#" "$2" '^\(lock\|rcu\|rcuscale\|refscale\|scf\)$' '^--'
TORTURE_SUITE=$2
shift
if test "$TORTURE_SUITE" = rcuperf || test "$TORTURE_SUITE" = refscale
if test "$TORTURE_SUITE" = rcuscale || test "$TORTURE_SUITE" = refscale
then
# If you really want jitter for refscale or
# rcuperf, specify it after specifying the rcuperf
# rcuscale, specify it after specifying the rcuscale
# or the refscale. (But why jitter in these cases?)
jitter=0
fi
@@ -248,6 +262,15 @@ do
done
touch $T/cfgcpu
configs_derep="`echo $configs_derep | sed -e "s/\<CFLIST\>/$defaultconfigs/g"`"
if test -n "$TORTURE_KCONFIG_GDB_ARG"
then
if test "`echo $configs_derep | wc -w`" -gt 1
then
echo "The --config list is: $configs_derep."
echo "Only one --config permitted with --gdb, terminating."
exit 1
fi
fi
for CF1 in $configs_derep
do
if test -f "$CONFIGFRAG/$CF1"
@@ -323,6 +346,9 @@ TORTURE_BUILDONLY="$TORTURE_BUILDONLY"; export TORTURE_BUILDONLY
TORTURE_DEFCONFIG="$TORTURE_DEFCONFIG"; export TORTURE_DEFCONFIG
TORTURE_INITRD="$TORTURE_INITRD"; export TORTURE_INITRD
TORTURE_KCONFIG_ARG="$TORTURE_KCONFIG_ARG"; export TORTURE_KCONFIG_ARG
TORTURE_KCONFIG_GDB_ARG="$TORTURE_KCONFIG_GDB_ARG"; export TORTURE_KCONFIG_GDB_ARG
TORTURE_BOOT_GDB_ARG="$TORTURE_BOOT_GDB_ARG"; export TORTURE_BOOT_GDB_ARG
TORTURE_QEMU_GDB_ARG="$TORTURE_QEMU_GDB_ARG"; export TORTURE_QEMU_GDB_ARG
TORTURE_KCONFIG_KASAN_ARG="$TORTURE_KCONFIG_KASAN_ARG"; export TORTURE_KCONFIG_KASAN_ARG
TORTURE_KCONFIG_KCSAN_ARG="$TORTURE_KCONFIG_KCSAN_ARG"; export TORTURE_KCONFIG_KCSAN_ARG
TORTURE_KMAKE_ARG="$TORTURE_KMAKE_ARG"; export TORTURE_KMAKE_ARG
@@ -33,8 +33,8 @@ then
fi
cat /dev/null > $file.diags
# Check for proper termination, except for rcuperf and refscale.
if test "$TORTURE_SUITE" != rcuperf && test "$TORTURE_SUITE" != refscale
# Check for proper termination, except for rcuscale and refscale.
if test "$TORTURE_SUITE" != rcuscale && test "$TORTURE_SUITE" != refscale
then
# check for abject failure
@@ -67,6 +67,7 @@ then
grep --binary-files=text 'torture:.*ver:' $file |
egrep --binary-files=text -v '\(null\)|rtc: 000000000* ' |
sed -e 's/^(initramfs)[^]]*] //' -e 's/^\[[^]]*] //' |
sed -e 's/^.*ver: //' |
awk '
BEGIN {
ver = 0;
@@ -74,13 +75,13 @@ then
}
{
if (!badseq && ($5 + 0 != $5 || $5 <= ver)) {
if (!badseq && ($1 + 0 != $1 || $1 <= ver)) {
badseqno1 = ver;
badseqno2 = $5;
badseqno2 = $1;
badseqnr = NR;
badseq = 1;
}
ver = $5
ver = $1
}
END {
@@ -16,5 +16,6 @@ CONFIG_RCU_NOCB_CPU=y
CONFIG_DEBUG_LOCK_ALLOC=y
CONFIG_PROVE_LOCKING=y
#CHECK#CONFIG_PROVE_RCU=y
CONFIG_PROVE_RCU_LIST=y
CONFIG_DEBUG_OBJECTS_RCU_HEAD=n
CONFIG_RCU_EXPERT=y
@@ -1,2 +0,0 @@
CONFIG_RCU_PERF_TEST=y
CONFIG_PRINTK_TIME=y
@@ -0,0 +1,2 @@
CONFIG_RCU_SCALE_TEST=y
CONFIG_PRINTK_TIME=y
@@ -11,6 +11,6 @@
#
# Adds per-version torture-module parameters to kernels supporting them.
per_version_boot_params () {
echo $1 rcuperf.shutdown=1 \
rcuperf.verbose=1
echo $1 rcuscale.shutdown=1 \
rcuscale.verbose=1
}
@@ -0,0 +1,2 @@
NOPREEMPT
PREEMPT
@@ -0,0 +1,2 @@
CONFIG_SCF_TORTURE_TEST=y
CONFIG_PRINTK_TIME=y
@@ -0,0 +1,9 @@
CONFIG_SMP=y
CONFIG_PREEMPT_NONE=y
CONFIG_PREEMPT_VOLUNTARY=n
CONFIG_PREEMPT=n
CONFIG_HZ_PERIODIC=n
CONFIG_NO_HZ_IDLE=n
CONFIG_NO_HZ_FULL=y
CONFIG_DEBUG_LOCK_ALLOC=n
CONFIG_PROVE_LOCKING=n
@@ -0,0 +1 @@
nohz_full=1
@@ -0,0 +1,9 @@
CONFIG_SMP=y
CONFIG_PREEMPT_NONE=n
CONFIG_PREEMPT_VOLUNTARY=n
CONFIG_PREEMPT=y
CONFIG_HZ_PERIODIC=n
CONFIG_NO_HZ_IDLE=y
CONFIG_NO_HZ_FULL=n
CONFIG_DEBUG_LOCK_ALLOC=y
CONFIG_PROVE_LOCKING=y
@@ -0,0 +1,30 @@
#!/bin/bash
# SPDX-License-Identifier: GPL-2.0+
#
# Torture-suite-dependent shell functions for the rest of the scripts.
#
# Copyright (C) Facebook, 2020
#
# Authors: Paul E. McKenney <paulmck@kernel.org>
# scftorture_param_onoff bootparam-string config-file
#
# Adds onoff scftorture module parameters to kernels having it.
scftorture_param_onoff () {
if ! bootparam_hotplug_cpu "$1" && configfrag_hotplug_cpu "$2"
then
echo CPU-hotplug kernel, adding scftorture onoff. 1>&2
echo scftorture.onoff_interval=1000 scftorture.onoff_holdoff=30
fi
}
# per_version_boot_params bootparam-string config-file seconds
#
# Adds per-version torture-module parameters to kernels supporting them.
per_version_boot_params () {
echo $1 `scftorture_param_onoff "$1" "$2"` \
scftorture.stat_interval=15 \
scftorture.shutdown_secs=$3 \
scftorture.verbose=1 \
scf
}
@@ -1,12 +1,11 @@
The rcutorture scripting tools automatically create the needed initrd
directory using dracut. Failing that, this tool will create an initrd
containing a single statically linked binary named "init" that loops
over a very long sleep() call. In both cases, this creation is done
by tools/testing/selftests/rcutorture/bin/mkinitrd.sh.
The rcutorture scripting tools automatically create an initrd containing
a single statically linked binary named "init" that loops over a
very long sleep() call. In both cases, this creation is done by
tools/testing/selftests/rcutorture/bin/mkinitrd.sh.
However, if you are attempting to run rcutorture on a system that does
not have dracut installed, and if you don't like the notion of static
linking, you might wish to press an existing initrd into service:
However, if you don't like the notion of statically linked bare-bones
userspace environments, you might wish to press an existing initrd
into service:
------------------------------------------------------------------------
cd tools/testing/selftests/rcutorture
@@ -15,24 +14,3 @@ mkdir initrd
cd initrd
cpio -id < /tmp/initrd.img.zcat
# Manually verify that initrd contains needed binaries and libraries.
------------------------------------------------------------------------
Interestingly enough, if you are running rcutorture, you don't really
need userspace in many cases. Running without userspace has the
advantage of allowing you to test your kernel independently of the
distro in place, the root-filesystem layout, and so on. To make this
happen, put the following script in the initrd's tree's "/init" file,
with 0755 mode.
------------------------------------------------------------------------
#!/bin/sh
while :
do
sleep 10
done
------------------------------------------------------------------------
This approach also allows most of the binaries and libraries in the
initrd filesystem to be dispensed with, which can save significant
space in rcutorture's "res" directory.
@@ -1,8 +1,33 @@
This document describes one way to create the rcu-test-image file
that contains the filesystem used by the guest-OS kernel. There are
probably much better ways of doing this, and this filesystem could no
doubt be smaller. It is probably also possible to simply download
an appropriate image from any number of places.
Normally, a minimal initrd is created automatically by the rcutorture
scripting. But minimal really does mean "minimal", namely just a single
root directory with a single statically linked executable named "init":
$ size tools/testing/selftests/rcutorture/initrd/init
text data bss dec hex filename
328 0 8 336 150 tools/testing/selftests/rcutorture/initrd/init
Suppose you need to run some scripts, perhaps to monitor or control
some aspect of the rcutorture testing. This will require a more fully
filled-out userspace, perhaps containing libraries, executables for
the shell and other utilities, and soforth. In that case, place your
desired filesystem here:
tools/testing/selftests/rcutorture/initrd
For example, your tools/testing/selftests/rcutorture/initrd/init might
be a script that does any needed mount operations and starts whatever
scripts need starting to properly monitor or control your testing.
The next rcutorture build will then incorporate this filesystem into
the kernel image that is passed to qemu.
Or maybe you need a real root filesystem for some reason, in which case
please read on!
The remainder of this document describes one way to create the
rcu-test-image file that contains the filesystem used by the guest-OS
kernel. There are probably much better ways of doing this, and this
filesystem could no doubt be smaller. It is probably also possible to
simply download an appropriate image from any number of places.
That said, here are the commands:
@@ -36,7 +61,7 @@ References:
https://help.ubuntu.com/community/JeOSVMBuilder
http://wiki.libvirt.org/page/UbuntuKVMWalkthrough
http://www.moe.co.uk/2011/01/07/pci_add_option_rom-failed-to-find-romfile-pxe-rtl8139-bin/ -- "apt-get install kvm-pxe"
http://www.landley.net/writing/rootfs-howto.html
http://en.wikipedia.org/wiki/Initrd
http://en.wikipedia.org/wiki/Cpio
https://www.landley.net/writing/rootfs-howto.html
https://en.wikipedia.org/wiki/Initrd
https://en.wikipedia.org/wiki/Cpio
http://wiki.libvirt.org/page/UbuntuKVMWalkthrough