vhost kernel driver does not support mutiple queue yet,
Tweak queue number will fail with "--net mode=tap,vhost=1,mq=2"
as below when lkvm trying to set ring kick fd for queue 2:
VHOST_SET_VRING_KICK failed: No buffer space available
Error on this scenario, and overide with the default one queue
configuration.
Signed-off-by: Fan Du <fan.du@intel.com>
To detach tap device automatically from bridge when exiting,
just like what the reverse of "script" does.
Signed-off-by: Fan Du <fan.du@intel.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
Due to our kernel heritage we have code in kvmtool that relies on
the (still) implicit -std=gnu89 compiler switch.
It turns out that this just affects some structure initialization,
where we currently provide a cast to the type, which upsets GCC for
anything beyond -std=gnu89 (for instance gnu99 or gnu11).
We do need the casts when initializing structures that are not
assigned to the same type, so we put it there explicitly.
This allows us to compile with all the three GNU standards GCC
currently supports: gnu89/90, gnu99 and gnu11.
GCC threatens people with moving to gnu11 as the new default standard,
so lets fix this better sooner than later.
(Compiling without GNU extensions still breaks and I don't bother to
fix that without very good reasons.)
Signed-off-by: Andre Przywara <andre.przywara@arm.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
Otherwise vhost does not work if a virtio descriptor is used that was
allocated from a guest memory bank not registered as
vhost_memory_region.
Signed-off-by: Andreas Herrmann <andreas.herrmann@caviumnetworks.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
Since commit d2a7ddff4 (Add minimal support for macvtap) opening
of tap device might fail. lkvm shows
Warning: Config tap device error. Are you root?
virtio_net_request_tap passed wrong pointer for struct ifreq to
TUNSETIFF ioctl.
Signed-off-by: Andreas Herrmann <andreas.herrmann@caviumnetworks.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
The terminal handling thread and the virtio-net-ctrl don't
set their name, which ends up as follows up:
terminal => lkvm
virtio-net-ctrl => kvm-cpu-X !!
Set the thread name explicitly to term-poll and virtio-net-ctrl
respectively
Signed-off-by: Suzuki K. Poulose <suzuki.poulose@arm.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
include/kvm/types.h seemed to be in use once, but it does no longer
contain any useful definition. Remove it.
Signed-off-by: Andre Przywara <andre.przywara@arm.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
In order to be useable by kvmtool, a macvtap interface requires
some minimal configuration (basically setting up the offload bits).
This requires skipping some of the low level TUN/TAP setup.
To avoid adding yet another option, we extend the 'tapif' option
to detect the use of a file (such as /dev/tap23).
Assuming you've run the following as root:
# ip link add link eth0 name kvmtap0 type macvtap mode bridge
# chgrp kvm /dev/tap$(< /sys/class/net/kvmtap0/ifindex)
# chmod g+rw /dev/tap$(< /sys/class/net/kvmtap0/ifindex)
it is fairly easy to have a script that does the following:
#!/bin/sh
addr=$(< /sys/class/net/kvmtap0/address)
tap=/dev/tap$(< /sys/class/net/kvmtap0/ifindex)
kvmtool/lkvm run --console virtio \
-k /boot/zImage \
-p "console=hvc0 earlyprintk" \
-n trans=mmio,mode=tap,tapif=$tap,guest_mac=$addr
and you now have your VM running, directly attached to the network.
This patch also removes the TUNSETNOCSUM ioctl that has declared
obsolete for quite some time now...
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
lkvm by default sets up a virtio-pci transport for network, if none is
specified. This can be a problem on archs (e.g ARM64), where virtio-pci is
not supported yet and cause the following warning at exit.
# KVM compatibility warning.
virtio-net device was not detected.
This patch changes it to make use of the default transport method for the
architecture when none is specified. This will ensure that on every arch
we get the network up by default in the VM.
Signed-off-by: Suzuki K. Poulose <suzuki.poulose@arm.com>
Acked-by: Will Deacon <will.deacon@arm.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
Configure the queues to follow the guest endianness, and make sure
the configuration space is doing the same.
Extra care is taken for the handling of the virtio_net_hdr structures
on both the TX and RX ends.
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
Signed-off-by: Pekka Enberg <penberg@kernel.org>
In order to overcome the fact that a TAP interface can only be created
by root, allow the use of an interface that has already been created,
configured, made persistent and owned by a specific user/group (such
as done with tunctl).
In this case, any kind of configuration can be skipped (IP, up and
running mode), and the TAP is assumed to be ready for use.
This is done by introducing the "tapif" option, as used here:
--network trans=mmio,mode=tap,tapif=blah
where "blah" is a TAP interface.
This allow the creation/configuration of the interface to be controlled
by root, and lkvm to be run as a normal user.
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
Signed-off-by: Pekka Enberg <penberg@kernel.org>
When attempting to initialise a mixture of pci and mmio virtio devices,
we cannot share an ops structure, otherwise the transport-specific
fields (init/exit and signal handling) will be globally set to the
transport of the last registered device.
This patch dynamically allocates a new ops structure for each instance
of a virtio net device.
Signed-off-by: Will Deacon <will.deacon@arm.com>
Signed-off-by: Pekka Enberg <penberg@kernel.org>
Currently, if a ->tx or ->rx callback into the net_dev_operations
encounters an error, it returns -1 to the virtio-net code, which in turn
treats this as an unsigned (size_t) size describing the data available.
The resulting memcpy operation then quickly explodes with a SEGV.
This patch detects the error code from the low-level callbacks and
exits the thread dealing with the erroneous queue.
Signed-off-by: Will Deacon <will.deacon@arm.com>
Signed-off-by: Pekka Enberg <penberg@kernel.org>
After features negotiation, kvmtool should tell vhost-net that he's
using mergeable rx buffers.
Signed-off-by: Ying-Shiuan Pan <yspan@itri.org.tw>
Signed-off-by: Pekka Enberg <penberg@kernel.org>
Enabling vhost-net encounted an error:
Fatal: VHOST_NET_SET_BACKEND failed 88
The reason is that vhost-net requires tap_fd for VHOST_NET_SET_BACKEND,
however tap_fd is opened after VIRTIO_CONFIG_S_DRIVER_OK. Because the
initialization needs to know the guest features, I suppose the initialization
could be moved to set_guest_features(). Therefore, initialization can be
finished before status VIRTIO_CONFIG_S_DRIVER_OK, and tap_fd can be set
before vhost-net sets backend.
Signed-off-by: Ying-Shiuan Pan <yspan@itri.org.tw>
Signed-off-by: Pekka Enberg <penberg@kernel.org>
Milan Kocian writes:
I found the crash in virtio-net-rx thread (I can reproduce it every
time by 'aptitude update' in VM):
traps: virtio-net-rx[28933] general protection ip:7f00dda3d107 sp:7f00c58f4de8 error:0 in libc-2.17.so[7f00dd90f000+1a2000]
gdb backtrace:
(gdb) bt
#0 0x00007fb6a548e107 in ?? () from /lib/x86_64-linux-gnu/libc.so.6
#1 0x000000000041259c in memcpy_toiovecend (iov=0x7fb68d346ea0, iov@entry=0x7fb68d345e90,
kdata=<optimized out>, kdata@entry=0x7fb68d346e90 "", offset=<optimized out>, len=<optimized out>)
at util/iovec.c:70
#2 0x000000000040c66d in virtio_net_rx_thread (p=0x23688a0) at virtio/net.c:117
#3 0x00007fb6a5b2ee0e in start_thread () from /lib/x86_64-linux-gnu/libpthread.so.0
#4 0x00007fb6a54489ed in clone () from /lib/x86_64-linux-gnu/libc.so.6
I tried to add some printf to diagnose it but it isn't clear to me:
virtio_net_rx_thread: before memcpy_toiovecend; copied: 0, len: 18890, iovsize: 4096, realiovsize: 4096
memcpy_toiovecend: offset: 0, len: 4096
memcpy_toiovecend: iov_len: 4096, len: 4096
virtio_net_rx_thread: before memcpy_toiovecend; copied: 4096, len: 18890, iovsize: 4096, realiovsize: 4096
memcpy_toiovecend: offset: 4096, len: 4096
memcpy_toiovecend: iov_len: 4096, len: 4096
memcpy_toiovecend: iov_len: 0, len: 4096
memcpy_toiovecend: iov_len: 0, len: 4096
.
N x memcpy_toiovecend: iov_len: 0, len: 4096
.
memcpy_toiovecend: iov_len: 0, len: 4096
memcpy_toiovecend: iov_len: 0, len: 4096
memcpy_toiovecend: iov_len: 1519143547641528320, len: 4096
memcpy_toiovecend: iov_len: 193827583623176, len: 4096
./runlkvm.sh: line 2: 16090 Segmentation fault
IMHO problem come when received len size is bigger than maximum of the
dst iovec (realiovsize). Only iovec size is copied and in the next run
isn't place to copy the rest of len size.
Asias He writes:
We should skip copied bytes from the buffer not from the iov itself
which memcpy_toiovecend does.
Reported-and-tested-by: Milan Kocian <milon@wq.cz>
Signed-off-by: Asias He <asias.hejun@gmail.com>
Signed-off-by: Pekka Enberg <penberg@iki.fi>
When fa7226f (kvm tools: init network devices only when the virtio
driver is ready to go) was introduced, a tiny detail was overlooked:
- Initialization of the uip layer is now coming in very late (only
when the guest driver says it is ready).
- In parallel, the rx thread is created quite early (as soon as the
queues are allocated).
This cause the rx thread to call uip_rx, which calls uip_buf_get_used,
which starts to use buf_lock mutex/the buf_used_cond, which haven't
been initialized yet. Tears and devastation follow, not to mention a
certain lack of network connectivity for the unsuspecting guest.
The (not so pretty) fix is to split uip_init:
- uip_static_init: initialize the lists, mutexes and conditions,
called from virtio_net__init_one.
- uip_init: perform the dynamic memory allocations, called from
notify_status.
This allows the network to be safely initialized.
Cc: Sasha Levin <sasha.levin@oracle.com>
Cc: Pekka Enberg <penberg@kernel.org>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
Signed-off-by: Pekka Enberg <penberg@kernel.org>
Commit "kvm tools: virtio: remove hardcoded assumptions
about guest page size" has introduced a bug that prevented
guests with more than 4gb of ram from booting.
The issue is that 'pfn' is a 32bit integer, so when multiplying
it by page size to get the actual page will cause an overflow if
the pfn referred to a memory area above 4gb.
Acked-by: Will Deacon <will.deacon@arm.com>
Signed-off-by: Sasha Levin <sasha.levin@oracle.com>
Signed-off-by: Pekka Enberg <penberg@kernel.org>
Support mergable rx buffers for virtio-net. This helps reduce the amount
of memory the guest kernel has to allocate per rx vq.
Signed-off-by: Sasha Levin <sasha.levin@oracle.com>
Signed-off-by: Pekka Enberg <penberg@kernel.org>
vnet header size depends on whether we use mergable rx buffers.
Signed-off-by: Sasha Levin <sasha.levin@oracle.com>
Signed-off-by: Pekka Enberg <penberg@kernel.org>
We may need to know what features are supported before we can init the
network on the host side.
Signed-off-by: Sasha Levin <sasha.levin@oracle.com>
Signed-off-by: Pekka Enberg <penberg@kernel.org>
We broke networking using vhost with the introduction of a ctrl vq,
make sure that that queue get treated like a special case.
Signed-off-by: Sasha Levin <sasha.levin@oracle.com>
Signed-off-by: Pekka Enberg <penberg@kernel.org>
We want to make the size of the virtio net header opaque to
uip.
Signed-off-by: Sasha Levin <sasha.levin@oracle.com>
Signed-off-by: Pekka Enberg <penberg@kernel.org>
This patch makes the maximum amount of vqs configurable. To use it pass a 'mq'
option to network device configuration. For example:
vm run -n mode=tap,mq=4
Will allow up to 4 queue pairs for that network device.
Note that not specifiying mq, or setting mq=0 will disable virtio-net
multiqueuing.
Acked-by: Will Deacon <will.deacon@arm.com>
Tested-by: Will Deacon <will.deacon@arm.com>
Signed-off-by: Sasha Levin <sasha.levin@oracle.com>
Signed-off-by: Pekka Enberg <penberg@kernel.org>
This patch adds support for multiple virtio-net queues. Each queue gets
assigned to it's own corresponding rx/tx thread.
The max amount of queue pairs is 8 right now, it's up to the guest
to decide the actual amount of queues it wants using ethtool. For
example, if I want 2 queue pairs on eth0:
ethtool -L eth0 combined 2
Signed-off-by: Sasha Levin <sasha.levin@oracle.com>
Signed-off-by: Pekka Enberg <penberg@kernel.org>
We don't need much out of it at this point, but we will need it for virtio-net
mq patch.
Signed-off-by: Sasha Levin <sasha.levin@oracle.com>
Signed-off-by: Pekka Enberg <penberg@kernel.org>
virtio-based PCI devices deal only with 4k memory granules, making
direct use of the VIRTIO_PCI_VRING_ALIGN and VIRTIO_PCI_QUEUE_ADDR_SHIFT
constants when initialising the virtqueues for a device.
For MMIO-based devices, the guest page size is arbitrary and may differ
from that of the host (this is the case on AArch64, where both 4k and
64k pages are supported).
This patch fixes the virtio drivers to honour the guest page size passed
when configuring the virtio device and align the virtqueues accordingly.
Signed-off-by: Will Deacon <will.deacon@arm.com>
Signed-off-by: Pekka Enberg <penberg@kernel.org>
We already have something to wrap pthread with mutex_[init,lock,unlock]
calls. This patch creates a new struct mutex abstraction and moves
everything to work with it.
Signed-off-by: Sasha Levin <sasha.levin@oracle.com>
Signed-off-by: Pekka Enberg <penberg@kernel.org>
Give threads a meaningful name. This makes debugging much easier, and
everything else much prettier.
Suggested-by: Ingo Molnar <mingo@kernel.org>
Signed-off-by: Sasha Levin <sasha.levin@oracle.com>
[ penberg@kernel.org: specify vcpu names ]
Signed-off-by: Pekka Enberg <penberg@kernel.org>
Switch to using init/exit calls instead of the repeating call blocks in builtin-run.
Signed-off-by: Sasha Levin <levinsasha928@gmail.com>
Signed-off-by: Pekka Enberg <penberg@kernel.org>
Make the init/exit of virtio-net self-contained, so the global init code
won't need to check if it was selected or not.
This also moves the bulk of the net-specific initialization code, including
the parser, into virtio-net itself.
Signed-off-by: Sasha Levin <levinsasha928@gmail.com>
Signed-off-by: Pekka Enberg <penberg@kernel.org>
Instead of a get/set for config values, just request the address of the
config region, and handle that by simply reading directly from that region.
Signed-off-by: Sasha Levin <levinsasha928@gmail.com>
Signed-off-by: Pekka Enberg <penberg@kernel.org>
This error message is missing a space, and has a redundant ":" at the end,
currently it produces:
You have requested a TAP device, but creation of one hasfailed because:: No such file or directory
Add a space to "hasfailed" and remove the extra ":".
Don't split the line to improve grepability.
Acked-by: Asias He <asias.hejun@gmail.com>
Signed-off-by: Michael Ellerman <michael@ellerman.id.au>
Signed-off-by: Pekka Enberg <penberg@kernel.org>
If vhost is enabled for a virtio device, vhost will poll the ioeventfd
in kernel side and there is no need to poll it in userspace. Otherwise,
both vhost kernel and userspace will race to poll.
Signed-off-by: Asias He <asias.hejun@gmail.com>
Signed-off-by: Pekka Enberg <penberg@kernel.org>
VHOST_SET_MEM_TABLE failed: Operation not supported
In vhost_set_memory(), We have
if (mem.padding)
return -EOPNOTSUPP;
So, we need to zero struct vhost_memory.
Signed-off-by: Asias He <asias.hejun@gmail.com>
Signed-off-by: Pekka Enberg <penberg@kernel.org>
compat_id is initialized to -1 for each type of device. We should add
compat message if the compat_id == -1 which means we haven't added
compat message for this type of device.
Signed-off-by: Asias He <asias.hejun@gmail.com>
Signed-off-by: Pekka Enberg <penberg@kernel.org>
This patch introduces a helper virtio_compat_add_message() to simplify
adding compat message for virtio device.
Signed-off-by: Asias He <asias.hejun@gmail.com>
Signed-off-by: Pekka Enberg <penberg@kernel.org>
set_size_vq does not do the actual work, we can implement this later.
Signed-off-by: Asias He <asias.hejun@gmail.com>
Signed-off-by: Pekka Enberg <penberg@kernel.org>
Increase virtio-blk and virtio-net queue size to 256.
This makes virtio-mmio work.
Signed-off-by: Asias He <asias.hejun@gmail.com>
Signed-off-by: Pekka Enberg <penberg@kernel.org>
To select virtio-mmio instead of virito-pci transport:
lkvm run --network trans=mmio
Signed-off-by: Asias He <asias.hejun@gmail.com>
Signed-off-by: Pekka Enberg <penberg@kernel.org>
This patch reworks on virtio transport abstraction.
* Move virtio transport operations to virtio operations and drop
virtio/trans.c This makes the abstraction much cleaner.
* Rename struct virtio_trans to struct virtio_device
struct virtio_trans {
void *virtio;
enum virtio_trans_type type;
struct virtio_trans_ops *trans_ops;
struct virtio_ops *virtio_ops;
};
struct virtio_device {
void *virtio;
struct virtio_ops *ops;
};
The virtio_trans struct is bit confusing since it also includes virtio
operations.
* Introduce virtio_init()
To init device, e.g.
Before:
virtio_trans_init()
ndev->vtrans.trans_ops->init()
ndev->vtrans.virtio_ops = &net_dev_virtio_ops
After:
virtio_init()
Signed-off-by: Asias He <asias.hejun@gmail.com>
Signed-off-by: Pekka Enberg <penberg@kernel.org>
This allows users to pass a pre-configured fd to use for the network
interface.
For example:
kvm run -n mode=tap,fd=3 3<>/dev/net/tap3
Acked-by: Daniel P. Berrange <berrange@redhat.com>
Cc: Osier Yang <jyang@redhat.com>
Signed-off-by: Sasha Levin <levinsasha928@gmail.com>
Signed-off-by: Pekka Enberg <penberg@kernel.org>
Indirect buffers are ring descriptors which point to more (even more)
descriptors.
This can be used to increase the effective ring capacity, which helps the
guest to batch large requests - very useful for blk devices.
This patch also enables indirect buffers for virtio-net and virtio-blk.
The patch is based on the lguest's code which does the same.
Signed-off-by: Sasha Levin <levinsasha928@gmail.com>
Signed-off-by: Pekka Enberg <penberg@kernel.org>
virtio/net.c: In function ???virtio_net__vhost_init???:
virtio/net.c:476:22: error: cast from pointer to integer of different
size [-Werror=pointer-to-int-cast]
cc1: all warnings being treated as errors
Signed-off-by: Asias He <asias.hejun@gmail.com>
Signed-off-by: Pekka Enberg <penberg@kernel.org>
James Hunt