On systems with v8.2 we switch the 'vaxorcism' of guest SError with an
alternative sequence that uses the ESB-instruction, then reads DISR_EL1.
This saves the unmasking and remasking of asynchronous exceptions.
We do this after we've saved the guest registers and restored the
host's. Any SError that becomes pending due to this will be accounted
to the guest, when it actually occurred during host-execution.
Move the ESB-instruction as early as possible. Any guest SError
will become pending due to this ESB-instruction and then consumed to
DISR_EL1 before the host touches anything.
This lets us account for host/guest SError precisely on the guest
exit exception boundary.
Because the ESB-instruction now lands in the preamble section of
the vectors, we need to add it to the unpatched indirect vectors
too, and to any sequence that may be patched in over the top.
The ESB-instruction always lives in the head of the vectors,
to be before any memory write. Whereas the register-store always
lives in the tail.
Signed-off-by: James Morse <james.morse@arm.com>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
The KVM indirect vectors support is a little complicated. Different CPUs
may use different exception vectors for KVM that are generated at boot.
Adding new instructions involves checking all the possible combinations
do the right thing.
To make changes here easier to review lets state what we expect of the
preamble:
1. The first vector run, must always run the preamble.
2. Patching the head or tail of the vector shouldn't remove
preamble instructions.
Today, this is easy as we only have one instruction in the preamble.
Change the unpatched tail of the indirect vector so that it always
runs this, regardless of patching.
Signed-off-by: James Morse <james.morse@arm.com>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
The EL2 vector hardening feature causes KVM to generate vectors for
each type of CPU present in the system. The generated sequences already
do some of the early guest-exit work (i.e. saving registers). To avoid
duplication the generated vectors branch to the original vector just
after the preamble. This size is hard coded.
Adding new instructions to the HYP vector causes strange side effects,
which are difficult to debug as the affected code is patched in at
runtime.
Add KVM_VECTOR_PREAMBLE to tell kvm_patch_vector_branch() how big
the preamble is. The valid_vect macro can then validate this at
build time.
Reviewed-by: Julien Thierry <julien.thierry@arm.com>
Signed-off-by: James Morse <james.morse@arm.com>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
The ESB-instruction is a nop on CPUs that don't implement the RAS
extensions. This lets us use it in places like the vectors without
having to use alternatives.
If someone disables CONFIG_ARM64_RAS_EXTN, this instruction still has
its RAS extensions behaviour, but we no longer read DISR_EL1 as this
register does depend on alternatives.
This could go wrong if we want to synchronize an SError from a KVM
guest. On a CPU that has the RAS extensions, but the KConfig option
was disabled, we consume the pending SError with no chance of ever
reading it.
Hide the ESB-instruction behind the CONFIG_ARM64_RAS_EXTN option,
outputting a regular nop if the feature has been disabled.
Reported-by: Julien Thierry <julien.thierry@arm.com>
Signed-off-by: James Morse <james.morse@arm.com>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
The original implementation of vq_present() relied on aggressive
inlining in order for the compiler to know that the code is
correct, due to some const-casting issues. This was causing sparse
and clang to complain, while GCC compiled cleanly.
Commit 0c529ff789 addressed this problem, but since vq_present()
is no longer a function, there is now no implicit casting of the
returned value to the return type (bool).
In set_sve_vls(), this uncast bit value is compared against a bool,
and so may spuriously compare as unequal when both are nonzero. As
a result, KVM may reject valid SVE vector length configurations as
invalid, and vice versa.
Fix it by forcing the returned value to a bool.
Signed-off-by: Zhang Lei <zhang.lei@jp.fujitsu.com>
Fixes: 0c529ff789 ("KVM: arm64: Implement vq_present() as a macro")
Signed-off-by: Dave Martin <Dave.Martin@arm.com> [commit message rewrite]
Cc: Viresh Kumar <viresh.kumar@linaro.org>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
ARCH_WANT_HUGE_PMD_SHARE config was declared in both architectures:
move this declaration in arch/Kconfig and make those architectures
select it.
Signed-off-by: Alexandre Ghiti <alex@ghiti.fr>
Reviewed-by: Palmer Dabbelt <palmer@sifive.com>
Acked-by: Ingo Molnar <mingo@kernel.org>
Acked-by: Catalin Marinas <catalin.marinas@arm.com> # for arm64
Reviewed-by: Hanjun Guo <guohanjun@huawei.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Paul Walmsley <paul.walmsley@sifive.com>
This implements 5-way interleaving for ECB, CBC decryption and CTR,
resulting in a speedup of ~11% on Marvell ThunderX2, which has a
very deep pipeline and therefore a high issue latency for NEON
instructions operating on the same registers.
Note that XTS is left alone: implementing 5-way interleave there
would either involve spilling of the calculated tweaks to the
stack, or recalculating them after the encryption operation, and
doing either of those would most likely penalize low end cores.
For ECB, this is not a concern at all, given that we have plenty
of spare registers. For CTR and CBC decryption, we take advantage
of the fact that v16 is not used by the CE version of the code
(which is the only one targeted by the optimization), and so we
can reshuffle the code a bit and avoid having to spill to memory
(with the exception of one extra reload in the CBC routine)
Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
In preparation of tweaking the accelerated AES chaining mode routines
to be able to use a 5-way stride, implement the core routines to
support processing 5 blocks of input at a time. While at it, drop
the 2 way versions, which have been unused for a while now.
Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Pull arm64 fixes from Will Deacon:
"Fix a build failure with the LLVM linker and a module allocation
failure when KASLR is active:
- Fix module allocation when running with KASLR enabled
- Fix broken build due to bug in LLVM linker (ld.lld)"
* tag 'arm64-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux:
arm64/efi: Mark __efistub_stext_offset as an absolute symbol explicitly
arm64: kaslr: keep modules inside module region when KASAN is enabled
New boards the Khadas Edge family of sbcs and the Hugsun X99 TV box,
both based on rk3399. Small improvements for RockPi, Sapphire and
rk3328-roc-cc boards. Improvements for the thermal handling on rk3399
as well as the rock960 board. rk3399 dwc3 clock updates and a small
start of the dtsi for the new rk3399pro (the one with the connected
npu).
* tag 'v5.3-rockchip-dts64-2' of git://git.kernel.org/pub/scm/linux/kernel/git/mmind/linux-rockchip:
arm64: dts: rockchip: Fix USB3 Type-C on rk3399-sapphire
arm64: dts: rockchip: Update DWC3 modules on RK3399 SoCs
arm64: dts: rockchip: enable rk3328 watchdog clock
arm64: dts: rockchip: Add support for Hugsun X99 TV Box
arm64: dts: rockchip: Define values for the IPA governor for rock960
arm64: dts: rockchip: Fix multiple thermal zones conflict in rk3399.dtsi
arm64: dts: rockchip: add core dtsi file for RK3399Pro SoCs
arm64: dts: rockchip: improve rk3328-roc-cc rgmii performance.
arm64: dts: rockchip: Add support for Khadas Edge/Edge-V/Captain boards
arm64: dts: rockchip: Enable HDMI audio on Rock Pi
Signed-off-by: Olof Johansson <olof@lixom.net>
Spreadtrum's devicetree for v5.3-rc1
This tag contains only two patches for updating coresight compatible string.
* tag 'sprd-dt-v5.3-rc1' of https://github.com/lyrazhang/linux:
arm64: dts: sc9860: Update coresight DT bindings
arm64: dts: sc9836: Update coresight DT bindings
Signed-off-by: Olof Johansson <olof@lixom.net>
Doing the indirection through macros for the regs accessors just
makes them harder to read, so implement the helpers directly.
Note that only the helpers actually used are implemented now.
Signed-off-by: Christoph Hellwig <hch@lst.de>
Acked-by: Catalin Marinas <catalin.marinas@arm.com>
Signed-off-by: Arnd Bergmann <arnd@arndb.de>
Qualcomm ARM64 Updates for v5.3 Part 2
* Add SDM845 Cheza support
* Add TSENS controller and thermal zones for QCS404
* tag 'qcom-arm64-for-5.3-2' of git://git.kernel.org/pub/scm/linux/kernel/git/qcom/linux:
arm64: dts: qcom: qcs404: Add missing space for cooling-cells property
arm64: dts: qcom: sdm845-cheza: add initial cheza dt
arm64: dts: qcom: qcs404: Add thermal zones for each sensor
arm64: dts: qcom: qcs404: Add tsens controller
Signed-off-by: Olof Johansson <olof@lixom.net>
There should be a space both before and after the equal sign.
Add a missing space for the cooling cells property.
Fixes: f48cee3239 ("arm64: dts: qcom: qcs404: Add thermal zones for each sensor")
Signed-off-by: Niklas Cassel <niklas.cassel@linaro.org>
Acked-by: Amit Kucheria <amit.kucheria@linaro.org>
Signed-off-by: Andy Gross <agross@kernel.org>
The new route handling in ip_mc_finish_output() from 'net' overlapped
with the new support for returning congestion notifications from BPF
programs.
In order to handle this I had to take the dev_loopback_xmit() calls
out of the switch statement.
The aquantia driver conflicts were simple overlapping changes.
Signed-off-by: David S. Miller <davem@davemloft.net>
Pull ARM SoC fixes from Olof Johansson:
"A smaller batch of fixes, nothing that stands out as risky or scary.
Mostly DTS tweaks for a few issues:
- GPU fixlets for Meson
- CPU idle fix for LS1028A
- PWM interrupt fixes for i.MX6UL
Also, enable a driver (FSL_EDMA) on arm64 defconfig, and a warning and
two MAINTAINER tweaks"
* tag 'armsoc-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/soc/soc:
ARM: dts: imx6ul: fix PWM[1-4] interrupts
ARM: omap2: remove incorrect __init annotation
ARM: dts: gemini Fix up DNS-313 compatible string
ARM: dts: Blank D-Link DIR-685 console
arm64: defconfig: Enable FSL_EDMA driver
arm64: dts: ls1028a: Fix CPU idle fail.
MAINTAINERS: BCM53573: Add internal Broadcom mailing list
MAINTAINERS: BCM2835: Add internal Broadcom mailing list
ARM: dts: meson8b: fix the operating voltage of the Mali GPU
ARM: dts: meson8b: drop undocumented property from the Mali GPU node
ARM: dts: meson8: fix GPU interrupts and drop an undocumented property
ACPI 6.3 adds additional fields to the MADT GICC
structure to describe SPE PPI's. We pick these out
of the cached reference to the madt_gicc structure
similarly to the core PMU code. We then create a platform
device referring to the IRQ and let the user/module loader
decide whether to load the SPE driver.
Tested-by: Hanjun Guo <hanjun.guo@linaro.org>
Reviewed-by: Sudeep Holla <sudeep.holla@arm.com>
Reviewed-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Signed-off-by: Jeremy Linton <jeremy.linton@arm.com>
Signed-off-by: Will Deacon <will@kernel.org>
Before this patch, the Type-C port on the Sapphire board is dead.
If setting the 'regulator-always-on' property to 'vcc5v0_typec0'
then the port works for about 4 seconds at start-up. This is a
sample trace with a memory stick plugged in:
1.- The memory stick LED lights on and kernel reports:
[ 4.782999] scsi 0:0:0:0: Direct-Access USB DISK PMAP PQ: 0 ANSI: 4
[ 5.904580] sd 0:0:0:0: [sdb] 3913344 512-byte logical blocks: (2.00 GB/1.87 GiB)
[ 5.906860] sd 0:0:0:0: [sdb] Write Protect is off
[ 5.908973] sd 0:0:0:0: [sdb] Mode Sense: 23 00 00 00
[ 5.909122] sd 0:0:0:0: [sdb] No Caching mode page found
[ 5.911214] sd 0:0:0:0: [sdb] Assuming drive cache: write through
[ 5.951585] sdb: sdb1
[ 5.954816] sd 0:0:0:0: [sdb] Attached SCSI removable disk
2.- 4 seconds later the memory stick LED lights off and kernel reports:
[ 9.082822] phy phy-ff770000.syscon:usb2-phy@e450.2: charger = USB_DCP_CHARGER
3.- After a minute the kernel reports:
[ 71.666761] usb 5-1: USB disconnect, device number 2
It has been checked that, although the LED is off, VBUS is present.
If, instead, the dr_mode is changed to host and the phy-supply changed
accordingly, then it works. It has only been tested in host mode.
Signed-off-by: Vicente Bergas <vicencb@gmail.com>
Signed-off-by: Heiko Stuebner <heiko@sntech.de>
As per binding documentation [1], the DWC3 core should have the "ref",
"bus_early" and "suspend" clocks. As explained in the binding, those
clocks are required for new platforms but not for existing platforms
before commit fe8abf332b ("usb: dwc3: support clocks and resets for
DWC3 core").
However, as those clocks are really treated as required, this ends with
having some annoying messages when the "rockchip,rk3399-dwc3" is used:
[ 1.724107] dwc3 fe800000.dwc3: Failed to get clk 'ref': -2
[ 1.731893] dwc3 fe900000.dwc3: Failed to get clk 'ref': -2
[ 2.495937] dwc3 fe800000.dwc3: Failed to get clk 'ref': -2
[ 2.647239] dwc3 fe900000.dwc3: Failed to get clk 'ref': -2
In order to remove those annoying messages, update the DWC3 hardware
module node and add all the required clocks. With this change, both, the
glue node and the DWC3 core node, have the clocks defined, but that's
not really a problem and there isn't a side effect on do this. So, we
can get rid of the annoying get clk error messages.
[1] Documentation/devicetree/bindings/usb/dwc3.txt
Signed-off-by: Enric Balletbo i Serra <enric.balletbo@collabora.com>
Signed-off-by: Heiko Stuebner <heiko@sntech.de>
Add the missing clock property for the watchdog on rk3328.
Signed-off-by: Leonidas P. Papadakos <papadakospan@gmail.com>
[set wdt node to always enabled, as it is not board-specific]
Signed-off-by: Heiko Stuebner <heiko@sntech.de>
Samsung DTS ARM64 changes for v5.3
Add Mali nodes to Exynos5433 and Exynos7.
* tag 'samsung-dt64-5.3' of https://git.kernel.org/pub/scm/linux/kernel/git/krzk/linux:
arm64: dts: exynos: Add GPU/Mali T760 node to Exynos7
arm64: dts: exynos: Add GPU/Mali T760 node to Exynos5433
Signed-off-by: Olof Johansson <olof@lixom.net>
UniPhier ARM64 SoC DT updates for v5.3
- Migrate to the new binding for the Denali NAND controller
- Use reserved-memory node instead of /memreserve/ for the
secure memory area
* tag 'uniphier-dt64-v5.3' of git://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-uniphier:
arm64: dts: uniphier: add reserved-memory for secure memory
arm64: dts: uniphier: update to new Denali NAND binding
Signed-off-by: Olof Johansson <olof@lixom.net>
Our usual bunch of arm64 defconfig changes, this time mostly to enable
some missing drivers for the Allwinner A64.
* tag 'sunxi-config64-for-5.3-201906210813' of https://git.kernel.org/pub/scm/linux/kernel/git/sunxi/linux:
arm64: defconfig: enable Allwinner DMA drivers
arm64: defconfig: enable sunxi watchdog
arm64: defconfig: add allwinner sid support
Signed-off-by: Olof Johansson <olof@lixom.net>
Add devicetree support for Hugsun X99 TV Box based on RK3399 SoC
Tested with LibreElec running kernel v5.1.2.
Following peripherals tested and work:
Peripheral works:
- UART2 debug
- eMMC
- USB 3.0 port
- USB 2.0 port
- sdio, sd-card
- HDMI
- Ethernet
- WiFi/BT
Not tested:
- Type-C port
- OPTICAL
- IR
Signed-off-by: Vivek Unune <npcomplete13@gmail.com>
Signed-off-by: Heiko Stuebner <heiko@sntech.de>
Currently the default thermal values for the rk3399-rock960 board is
inherited from the generic definition in rk3399.dtsi.
In order to ensure the rock960 has more room for througput before
being capped by the thermal framework and is correctly supported by
the IPA governor, let's define the power values and the right trip
points for better performances:
- sustainable power is tested to be 1550mW
- increase the first mitigation point to 75°C in order to get better
performances
- the first trip point is 65°C in order to let the IPA to collect
enough data for the PID regulation when it reaches 75°C
- restrict the cooling device to the big CPUs as the little CPUs
contribution to the heating effect can be considered negligible
The intelligent power allocator PID coefficient to be set in sysfs
are:
k_d: 0
k_po: 79
k_i: 10
k_pu: 50
Signed-off-by: Daniel Lezcano <daniel.lezcano@linaro.org>
Signed-off-by: Heiko Stuebner <heiko@sntech.de>
Currently the common thermal zones definitions for the rk3399 assumes
multiple thermal zones are supported by the governors. This is not the
case and each thermal zone has its own governor instance acting
individually without collaboration with other governors.
As the cooling device for the CPU and the GPU thermal zones is the
same, each governors take different decisions for the same cooling
device leading to conflicting instructions and an erratic behavior.
As the cooling-maps is about to become an optional property, let's
remove the cpu cooling device map from the GPU thermal zone.
Signed-off-by: Daniel Lezcano <daniel.lezcano@linaro.org>
Signed-off-by: Heiko Stuebner <heiko@sntech.de>
This patch adds core dtsi file for Rockchip RK3399Pro SoCs,
include rk3399.dtsi. Also enable pciei0/pcie_phy for AP to
talk to NPU part inside SoC.
Signed-off-by: Jianqun Xu <jay.xu@rock-chips.com>
Acked-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
Signed-off-by: Heiko Stuebner <heiko@sntech.de>
Currently the rk3328-roc-cc ethernet is enabled using "snps,force_thresh_dma_mode".
While this works, the performance leaves a lot to be desired.
A previous attempt to improve performance used "snps,txpbl = <0x4>".
This also allowed networking to function, but performance varied between boards.
This patch takes that one step further.
Set txpbl and rxpbl to 0x4.
This can also be accomplished with "snps,pbl =<0x4>" which affects both.
Also set "snps,aal" which forces address aligned DMA mode.
Fixes: 4bc4d6013b (arm64: dts: rockchip: fix rk3328-roc-cc gmac2io stability issues)
Signed-off-by: Peter Geis <pgwipeout@gmail.com>
Tested-by: Leonidas P. Papadakos <papadakospan@gmail.com>
Signed-off-by: Heiko Stuebner <heiko@sntech.de>
In traps.c, only __die calls dump_instr.
However, this function has sub-function as __dump_instr.
dump_kernel_instr can replace those functions.
By using aarch64_insn_read, it does not have to change fs to KERNEL_DS.
Acked-by: Will Deacon <will.deacon@arm.com>
Signed-off-by: jinho lim <jordan.lim@samsung.com>
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
After r363059 and r363928 in LLVM, a build using ld.lld as the linker
with CONFIG_RANDOMIZE_BASE enabled fails like so:
ld.lld: error: relocation R_AARCH64_ABS32 cannot be used against symbol
__efistub_stext_offset; recompile with -fPIC
Fangrui and Peter figured out that ld.lld is incorrectly considering
__efistub_stext_offset as a relative symbol because of the order in
which symbols are evaluated. _text is treated as an absolute symbol
and stext is a relative symbol, making __efistub_stext_offset a
relative symbol.
Adding ABSOLUTE will force ld.lld to evalute this expression in the
right context and does not change ld.bfd's behavior. ld.lld will
need to be fixed but the developers do not see a quick or simple fix
without some research (see the linked issue for further explanation).
Add this simple workaround so that ld.lld can continue to link kernels.
Link: https://github.com/ClangBuiltLinux/linux/issues/561
Link: https://github.com/llvm/llvm-project/commit/025a815d75d2356f2944136269aa5874721ec236
Link: https://github.com/llvm/llvm-project/commit/249fde85832c33f8b06c6b4ac65d1c4b96d23b83
Acked-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Debugged-by: Fangrui Song <maskray@google.com>
Debugged-by: Peter Smith <peter.smith@linaro.org>
Suggested-by: Fangrui Song <maskray@google.com>
Signed-off-by: Nathan Chancellor <natechancellor@gmail.com>
[will: add comment]
Signed-off-by: Will Deacon <will@kernel.org>
When KASLR and KASAN are both enabled, we keep the modules where they
are, and randomize the placement of the kernel so it is within 2 GB
of the module region. The reason for this is that putting modules in
the vmalloc region (like we normally do when KASLR is enabled) is not
possible in this case, given that the entire vmalloc region is already
backed by KASAN zero shadow pages, and so allocating dedicated KASAN
shadow space as required by loaded modules is not possible.
The default module allocation window is set to [_etext - 128MB, _etext]
in kaslr.c, which is appropriate for KASLR kernels booted without a
seed or with 'nokaslr' on the command line. However, as it turns out,
it is not quite correct for the KASAN case, since it still intersects
the vmalloc region at the top, where attempts to allocate shadow pages
will collide with the KASAN zero shadow pages, causing a WARN() and all
kinds of other trouble. So cap the top end to MODULES_END explicitly
when running with KASAN.
Cc: <stable@vger.kernel.org> # 4.9+
Acked-by: Catalin Marinas <catalin.marinas@arm.com>
Tested-by: Catalin Marinas <catalin.marinas@arm.com>
Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Signed-off-by: Will Deacon <will@kernel.org>
This was added part of the original commit which added MMU definitions.
commit 4f04d8f005 ("arm64: MMU definitions").
These symbols never got used as confirmed from a git log search.
git log -p arch/arm64/ | grep PTE_TYPE_FAULT
git log -p arch/arm64/ | grep PMD_TYPE_FAULT
These probably meant to identify non present entries which can now be
achieved with PMD_SECT_VALID or PTE_VALID bits. Hence just drop these
unused symbols which are not required anymore.
Cc: Will Deacon <will.deacon@arm.com>
Cc: Steve Capper <steve.capper@arm.com>
Signed-off-by: Anshuman Khandual <anshuman.khandual@arm.com>
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
Enable CONFIG_KEYBOARD_SNVS_PWRKEY as module to support i.MX8M
series SoCs' power key.
Signed-off-by: Anson Huang <Anson.Huang@nxp.com>
Signed-off-by: Shawn Guo <shawnguo@kernel.org>
For imx8m we need a separate small driver to read "speed grading"
information from fuses and determine which OPPs are supported.
Signed-off-by: Leonard Crestez <leonard.crestez@nxp.com>
Signed-off-by: Shawn Guo <shawnguo@kernel.org>
James Morse