Tell the USB core that we can do DMA directly (instead of needing it to
memory-map the buffers for PIO). If the xHCI host supports 64-bit addresses,
set the DMA mask accordingly. Otherwise indicate the host can handle 32-bit DMA
addresses.
This improves performance because the USB core doesn't have to spend time
remapping buffers in high memory into the 32-bit address range.
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
To tell the host controller that there are transfers on the endpoint
rings, we need to ring the endpoint doorbell. This is a PCI MMIO write,
which can be delayed until another register read is queued.
The previous code would flush the doorbell write by reading the doorbell
register after the write. This may take time, and it's not necessary to
force the host controller to know about the transfers right away. Don't
flush the doorbell register writes.
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
The interrupter register set includes a register that says whether interrupts
are pending for each event ring (the IP bit). Each MSI-X vector will get its
own interrupter set with separate IP bits. The status register includes an
"Event Interrupt (EINT)" bit that is set when an IP bit is set in any of the
interrupters.
When PCI interrupts are used, the EINT bit exactly mirrors the IP bit in the
single interrupter set, and it is a waste of time to check both registers when
trying to figure out if the xHC interrupted or another device on the shared IRQ
line interrupted. Only check the IP bit to reduce register reads.
The IP bit is automatically cleared by the xHC when MSI or MSI-X is enabled. It
doesn't make sense to read that register to check for shared interrupts (since
MSI and MSI-X aren't shared). It also doesn't make sense to write to that
register to clear the IP bit, since it is cleared by the hardware.
We can tell whether MSI or MSI-X is enabled by looking at the irq number in
hcd->irq. If it's -1, we know MSI or MSI-X is enabled.
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
Now that the event handler functions no longer use xhci_set_hc_event_deq()
to update the event ring dequeue pointer, that function is not used by
anything in xhci-ring.c. Move that function into xhci-mem.c and make it
static.
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
The xHCI specification suggests that writing the hardware event ring dequeue
pointer register too often can be an expensive operation for the xHCI hardware
to manage. It suggests minimizing the number of writes to that register.
Originally, the driver wrote the event ring dequeue pointer after each
event was processed. Depending on how the event ring moderation register
is set up and how fast the transfers are completing, there may be several
events processed for each interrupt. This patch makes the hardware event
ring dequeue pointer be written only once per interrupt.
Make the transfer event handler and port status event handler only write
the software event ring dequeue pointer. Move the updating of the
hardware event ring dequeue pointer into the interrupt function. Move the
contents of xhci_set_hc_event_deq() into the interrupt handler. The
interrupt handler must clear the event handler busy flag, so it might as
well also write the dequeue pointer to the same register. This eliminates
two 32-bit PCI reads and two 32-bit PCI writes.
Reported-by: Andiry Xu <andiry.xu@amd.com>
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
xhci_handle_event() is now only called from within xhci-ring.c, so make it
static.
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
Remove a duplicate register read of the interrupt pending register from
xhci_irq(). Also, remove waiting on the posted write of that register.
The host will see it eventually. It will probably read the register
itself before deciding whether to interrupt the system again, forcing the
posted write to complete.
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
When we move xhci_work() into xhci_irq(), we don't need to read the operational
register status field twice.
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
Most of the work for interrupt handling is done in xhci-ring.c, so it makes
sense to move the functions that are first called when an interrupt happens
(xhci_irq() or xhci_msi_irq()) into xhci-ring.c, so that the compiler can better
optimize them.
Shorten some lines to make it pass checkpatch.
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
I've been using perf to measure the top symbols while transferring 1GB of data
on a USB 3.0 drive with dd. This is using the raw disk with /dev/sdb, with a
block size of 1K.
During performance testing, the top symbol was xhci_triad_to_transfer_ring(), a
function that should return immediately if streams are not enabled for an
endpoint. It turned out that the functions to find the endpoint ring was
defined in xhci-mem.c and used in xhci-ring.c and xhci-hcd.c. I moved a copy of
xhci_triad_to_transfer_ring() and xhci_urb_to_transfer_ring() into xhci-ring.c
and declared them static. I also made a static version of
xhci_urb_to_transfer_ring() in xhci.c.
This improved throughput on a 1GB read of the raw disk with dd from
186MB/s to 195MB/s, and perf reported sampling the xhci_triad_to_transfer_ring()
0.06% of the time, rather than 9.26% of the time.
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
When code to manipulate the command register was refactored from
xhci_run() to xhci_start(), a debugging statement was left behind that no
longer applies. Remove that statement.
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Reported-by: Sergei Shtylyov <sshtylyov@mvista.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
The bmAttributes field of the SuperSpeed Endpoint Companion Descriptor has
different meanings, depending on the endpoint type. If the endpoint is
isochronous, the bmAttributes field is the maximum number of packets
within a service interval that this endpoint supports. If the endpoint is
bulk, it's the number of stream IDs this endpoint supports.
Only set the Mult field of the xHCI endpoint context using the
bmAttributes field if the endpoint is isochronous, and the device is a
SuperSpeed device.
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Cc: stable <stable@kernel.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
When a configured device is reset, the control endpoint's ring is reused.
If control transfers to the device were issued before the device is reset,
the dequeue pointer will be somewhere in the middle of the ring. If the
device is then issued an address with the set address command, the xHCI
driver must provide a valid input context for control endpoint zero.
The original code would give the hardware the original input context,
which had a dequeue pointer set to the top of the ring. This would cause
the host to re-execute any control transfers until it reached the ring's
enqueue pointer. When issuing a set address command for a device that has
just been configured and then reset, use the control endpoint's enqueue
pointer as the hardware's dequeue pointer.
Assumption: All control transfers will be completed or cancelled before
the set address command is issued to the device. If there are any
outstanding control transfers, this code will not work.
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
Without this fix, a USB 3.0 port is downgraded to full speed after a port
reset of a configured device. The USB 3.0 terminations will be disabled
permanently, and USB 3.0 devices will always enumerate as full speed
devices, until the host controller is unplugged (if it is an ExpressCard)
or the computer is rebooted.
Fajun Chen traced this traced the speed downgrade issue to the port reset
and the interpretation of port status in USB hub driver code. The hub
code was not testing for the port being a SuperSpeed port, and it fell
through to the else case of Full Speed.
The following patch adds SuperSpeed mapping from the port status, and
fixes the speed downgrade issue.
Reported-by: Fajun Chen <fajun.chen@seagate.com>
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Cc: stable <stable@kernel.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
Commit 6c12db90f1 also seems to have
introduced a bug that is triggered when the command ring is about to wrap.
The inc_enq() function will not have moved the enqueue pointer past the
link TRB. It is supposed to be moved past the link TRB in prepare_ring(),
which should be called before a TD is enqueued. However, the
queue_command() function never calls the prepare_ring() function because
prepare_ring() is only supposed to be used for endpoint rings. That means
the enqueue pointer will not be moved past the link TRB, and will get
overwritten.
The fix is to make queue_command() call prepare_ring() with a fake
endpoint status (set to running). Then the enqueue pointer will get moved
past the link TRB.
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
Commit 6c12db90f1 introduced a bug for
control transfers. The patch was supposed to change when the link TRBs at
the end of each ring segment were given to the hardware. If a transfer
descriptor (TD) ended just before the link TRB, the code wouldn't give
back the link TRB to the hardware; instead it would be given back in
prepare_ring() just before the next TD was enqueued at the top of the
ring.
Unfortunately, the code relied on checking the chain bit of the TRB to
determine whether the TD ended just before the link TRB. It assumed that
the ring enqueuing code would call prepare_ring() before enqueuing the
next TD. However, control transfers are made of multiple TDs, and
prepare_ring() is only called once before enqueuing two or three TDs.
If the first or second TD of the control transfer ended just before the
link TRB, then the code in inc_enq() would not move the enqueue pointer
past the link TRB, and the link TRB would get overwritten. This would
cause the xHCI driver to start writing to memory past the ring segment,
and eventually the system would crash or hang.
The fix is to add a flag to inc_enq() that says whether the caller will
enqueue more TDs before calling prepare_ring(). If the chain bit is
cleared (meaning this is the last TRB in a TD), and the caller will not
enqueue more TDs, then we defer giving back the link TRB.
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Cc: stable <stable@kernel.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
The NEC xHCI host controller firmware version can be found by putting a
vendor-specific command on the command ring and extracting the BCD
encoded-version out of the vendor-specific event TRB.
The firmware version debug line in dmesg will look like:
xhci_hcd 0000:05:00.0: NEC firmware version 30.21
(NEC merged with Renesas Technologies and became Renesas Electronics on
April 1, 2010. I have their OK to merge this vendor-specific code.)
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Cc: Satoshi Otani <satoshi.otani.xm@renesas.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
When the run bit is set in the xHCI command register, it may take a few
microseconds for the host to start running. We cannot ring any doorbells
until the host is actually running, so wait until the status register says
the host is running.
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Reported-by: Shinya Saito <shinya.saito.sx@renesas.com>
Cc: stable <stable@kernel.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
After software resets an xHCI host controller, it must wait for the
"Controller Not Ready" (CNR) bit in the status register to be cleared.
Software is not supposed to ring any doorbells or write to any registers
except the status register until this bit is cleared.
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Cc: stable <stable@kernel.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
After using state stored in xhci_virt_ep to clean up a stalled endpoint,
be sure to set the stalled stream ID back to 0.
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
When a device is disconnected, xhci_free_virt_device() is called. Ramya
found that if the device had streams enabled, and then the driver freed
the streams with a call to usb_free_streams(), then about a minute after
he had called this, his machine crashed with a Bad DMA error. It turns
out that xhci_free_virt_device() would attempt to free the endpoint's
stream_info data structure if it wasn't NULL, and the free streams
function was not setting it to NULL after freeing it.
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Tested-by: Ramya Desai <ramya.desai@gmail.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
Bulk endpoint streams were added in the USB 3.0 specification. Streams
allow a device driver to overload a bulk endpoint so that multiple
transfers can be queued at once.
The device then decides which transfer it wants to work on first, and can
queue part of a transfer before it switches to a new stream. All this
switching is invisible to the device driver, which just gets a completion
for the URB. Drivers that use streams must be able to handle URBs
completing in a different order than they were submitted to the endpoint.
This requires adding new API to set up xHCI data structures to support
multiple queues ("stream rings") per endpoint. Drivers will allocate a
number of stream IDs before enqueueing URBs to the bulk endpoints of the
device, and free the stream IDs in their disconnect function. See
Documentation/usb/bulk-streams.txt for details.
The new mass storage device class, USB Attached SCSI Protocol (UASP), uses
these streams API.
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
Much of the xHCI driver code assumes that endpoints only have one ring.
Now an endpoint can have one ring per enabled stream ID, so correct that
assumption. Use functions that translate the stream_id field in the URB
or the DMA address of a TRB into the correct stream ring.
Correct the polling loop to print out all enabled stream rings. Make the
URB cancellation routine find the correct stream ring if the URB has
stream_id set. Make sure the URB enqueueing routine does the same. Also
correct the code that handles stalled/halted endpoints.
Check that commands and registers that can take stream IDs handle them
properly. That includes ringing an endpoint doorbell, resetting a
stalled/halted endpoint, and setting a transfer ring dequeue pointer
(since that command can set the dequeue pointer in a stream context or an
endpoint context).
Correct the transfer event handler to translate a TRB DMA address into the
stream ring it was enqueued to. Make the code to allocate and prepare TD
structures adds the TD to the right td_list for the stream ring. Make
sure the code to give the first TRB in a TD to the hardware manipulates
the correct stream ring.
When an endpoint stalls, store the stream ID of the stream ring that
stalled in the xhci_virt_ep structure. Use that instead of the stream ID
in the URB, since an URB may be re-used after it is given back after a
non-control endpoint stall.
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
Add support for allocating streams for USB 3.0 bulk endpoints. See
Documentation/usb/bulk-streams.txt for more information about how and why
you would use streams.
When an endpoint has streams enabled, instead of having one ring where all
transfers are enqueued to the hardware, it has several rings. The ring
dequeue pointer in the endpoint context is changed to point to a "Stream
Context Array". This is basically an array of pointers to transfer rings,
one for each stream ID that the driver wants to use.
The Stream Context Array size must be a power of two, and host controllers
can place a limit on the size of the array (4 to 2^16 entries). These
two facts make calculating the size of the Stream Context Array and the
number of entries actually used by the driver a bit tricky.
Besides the Stream Context Array and rings for all the stream IDs, we need
one more data structure. The xHCI hardware will not tell us which stream
ID a transfer event was for, but it will give us the slot ID, endpoint
index, and physical address for the TRB that caused the event. For every
endpoint on a device, add a radix tree to map physical TRB addresses to
virtual segments within a stream ring.
Keep track of whether an endpoint is transitioning to using streams, and
don't enqueue any URBs while that's taking place. Refuse to transition an
endpoint to streams if there are already URBs enqueued for that endpoint.
We need to make sure that freeing streams does not fail, since a driver's
disconnect() function may attempt to do this, and it cannot fail.
Pre-allocate the command structure used to issue the Configure Endpoint
command, and reserve space on the command ring for each stream endpoint.
This may be a bit overkill, but it is permissible for the driver to
allocate all streams in one call and free them in multiple calls. (It is
not advised, however, since it is a waste of resources and time.)
Even with the memory and ring room pre-allocated, freeing streams can
still fail because the xHC rejects the configure endpoint command. It is
valid (by the xHCI 0.96 spec) to return a "Bandwidth Error" or a "Resource
Error" for a configure endpoint command. We should never see a Bandwidth
Error, since bulk endpoints do not effect the reserved bandwidth. The
host controller can still return a Resource Error, but it's improbable
since the xHC would be going from a more resource-intensive configuration
(streams) to a less resource-intensive configuration (no streams).
If the xHC returns a Resource Error, the endpoint will be stuck with
streams and will be unusable for drivers. It's an unavoidable consequence
of broken host controller hardware.
Includes bug fixes from the original patch, contributed by
John Youn <John.Youn@synopsys.com> and Andy Green <AGreen@PLXTech.com>
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
Bulk endpoint streams were added in the USB 3.0 specification. Streams
allow a device driver to overload a bulk endpoint so that multiple
transfers can be queued at once.
Add a new field, stream_id, to struct urb so that USB 3.0 drivers can
specify which stream they want the URB to be queued to.
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
Allow the xHCI drivers (and any new USB 3.0 drivers) to parse the
SuperSpeed endpoint companion descriptor to find the maximum number of
bulk endpoint streams the endpoint supports. This is used to calculate
the maximum total number of streams the driver can allocate.
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
The length of the scatter gather list a driver can enqueue is limited by
the bus' sg_tablesize to 62 entries. Each entry will be described by at
least one transfer request block (TRB). If the entry's buffer crosses a
64KB boundary, then that entry will have to be described by two or more
TRBs. So even if the USB device driver respects sg_tablesize, the whole
scatter list may take more than 62 TRBs to describe, and won't fit on
the ring.
Don't assume that an empty ring means there is enough room on the
transfer ring. The old code would unconditionally queue this too-large
transfer, and over write the beginning of the transfer. This would mean
the cycle bit was unchanged in those overwritten transfers, causing the
hardware to think it didn't own the TRBs, and the host would seem to
hang.
Now drivers may see submit_urb() fail with -ENOMEM if the transfers are
too big to fit on the ring.
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Cc: stable <stable@kernel.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
When a scatter-gather list is enqueued to the xHCI driver, it translates
each entry into a transfer request block (TRB). Only 63 TRBs can be
used per ring segment, and there must be one additional TRB reserved to
make sure the hardware does not think the ring is empty (so the enqueue
pointer doesn't equal the dequeue pointer). Limit the bus sg_tablesize
to 62 TRBs.
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Cc: stable <stable@kernel.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
When the USB core installs a new interface, it unconditionally clears the
halts on all the endpoints on the new interface. Usually the xHCI host
needs to know when an endpoint is reset, so it can change its internal
endpoint state. In this case, it doesn't care, because the endpoints were
never halted in the first place.
To avoid issuing a redundant Reset Endpoint command, the xHCI driver looks
at xhci_virt_ep->stopped_td to determine if the endpoint was actually
halted. However, the functions that handle the stall never set that
variable to NULL after it dealt with the stall. So if an endpoint stalled
and a Reset Endpoint command completed, and then the class driver tried to
install a new alternate setting, the xHCI driver would access the old
xhci_virt_ep->stopped_td pointer. A similar problem occurs if the
endpoint has been stopped to cancel a transfer.
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Cc: stable <stable@kernel.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
For periodic endpoints, we must let the xHCI hardware know the maximum
payload an endpoint can transfer in one service interval. The xHCI
specification refers to this as the Maximum Endpoint Service Interval Time
Payload (Max ESIT Payload). This is used by the hardware for bandwidth
management and scheduling of packets.
For SuperSpeed endpoints, the maximum is calculated by multiplying the max
packet size by the number of bursts and the number of opportunities to
transfer within a service interval (the Mult field of the SuperSpeed
Endpoint companion descriptor). Devices advertise this in the
wBytesPerInterval field of their SuperSpeed Endpoint Companion Descriptor.
For high speed devices, this is taken by multiplying the max packet size by the
"number of additional transaction opportunities per microframe" (the high
bits of the wMaxPacketSize field in the endpoint descriptor).
For FS/LS devices, this is just the max packet size.
The other thing we must set in the endpoint context is the Average TRB
Length. This is supposed to be the average of the total bytes in the
transfer descriptor (TD), divided by the number of transfer request blocks
(TRBs) it takes to describe the TD. This gives the host controller an
indication of whether the driver will be enqueuing a scatter gather list
with many entries comprised of small buffers, or one contiguous buffer.
It also takes into account the number of extra TRBs you need for every TD.
This includes No-op TRBs and Link TRBs used to link ring segments
together. Some drivers may choose to chain an Event Data TRB on the end
of every TD, thus increasing the average number of TRBs per TD. The Linux
xHCI driver does not use Event Data TRBs.
In theory, if there was an API to allow drivers to state what their
bandwidth requirements are, we could set this field accurately. For now,
we set it to the same number as the Max ESIT payload.
The Average TRB Length should also be set for bulk and control endpoints,
but I have no idea how to guess what it should be.
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Cc: stable <stable@kernel.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
A SuperSpeed interrupt or isochronous endpoint can define the number of
"burst transactions" it can handle in a service interval. This is
indicated by the "Mult" bits in the bmAttributes of the SuperSpeed
Endpoint Companion Descriptor. For example, if it has a max packet size
of 1024, a max burst of 11, and a mult of 3, the host may send 33
1024-byte packets in one service interval.
We must tell the xHCI host controller the number of multiple service
opportunities (mults) the device can handle when the endpoint is
installed. We do that by setting the Mult field of the Endpoint Context
before a configure endpoint command is sent down. The Mult field is
invalid for control or bulk SuperSpeed endpoints.
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Cc: stable <stable@kernel.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
The xHCI hardware can only handle polling intervals that are a power of
two. When we add a new endpoint during a bandwidth allocation, and the
polling interval is rounded down to a power of two, print the original
polling interval in the endpoint descriptor.
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
USB devices do not have to sort interfaces in their descriptors based on
the interface number, and they may choose to skip interface numbers. The
USB bandwidth allocation code for installing a new configuration assumes
the for loop variable will match the interface number. Make it use the
interface number (bInterfaceNumber) in the descriptor instead.
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
Randy Dunlap reported this error when compiling the xHCI driver:
linux-next-20100104/drivers/usb/host/xhci.h:1214:
sorry, unimplemented: inlining failed in call to 'xhci_get_slot_state': function body not available
The xhci_get_slot_state() function belongs in xhci-dbg.c, since it
involves debugging internal xHCI structures. However, it is only used in
xhci-hcd.c. Some toolchains may have issues since the inlined function
body is not in the xhci.h header file. Remove the inline keyword to avoid
this.
Reported-by: Randy Dunlap <randy.dunlap@oracle.com>
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Acked-by: Randy Dunlap <randy.dunlap@oracle.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
Add a new host controller driver method, reset_device(), that the USB core
will use to notify the host of a successful device reset. The call may
fail due to out-of-memory errors; attempt the port reset sequence again if
that happens. Update hub_port_init() to allow resetting a configured
device.
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
When a USB device is reset, the xHCI hardware must know, in order to match
the device state and disable all endpoints except control endpoint 0.
Issue a Reset Device command after a USB device is successfully reset.
Wait on the command to finish, and then cache or free the disabled
endpoint rings.
There are four different USB device states that the xHCI hardware tracks:
- disabled/enabled - device connection has just been detected,
- default - the device has been reset and has an address of 0,
- addressed - the device has a non-zero address but no configuration has
been set,
- configured - a set configuration succeeded.
The USB core may issue a port reset when a device is in any state, but the
Reset Device command will fail for a 0.96 xHC if the device is not in the
addressed or configured state. Don't consider this failure as an error,
but don't free any endpoint rings if this command fails.
A storage driver may request that the USB device be reset during error
handling, so use GPF_NOIO instead of GPF_KERNEL while allocating memory
for the Reset Device command.
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
Add the hub emulation code to allow ports on an xHCI root hub to be
disabled. Add the code to clear the port enabled/disabled bit, and clear
the port enabled/disabled change bit. Like EHCI, the port cannot be
enabled by setting the port enabled/disabled bit. Instead, a port is
enabled by the host controller after a reset.
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
Refactor the code to clear the port change bits in the port status
register. All port status change bits are write one to clear.
Remove a redundant port status read that was supposed to unblock any
posted writes. We read the port after the write to get the updated status
for debugging, so the port read after that is unnecessary.
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
All commands that can be issued to the xHCI hardware can come back with
vendor-specific "informational" completion codes. These are to be treated
like a successful completion code. Refactor out the code to test for the
range of these codes and print debugging messages.
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
The xhci_command structure is the basic structure for issuing commands to
the xHCI hardware. It contains a struct completion (so that the issuing
function can wait on the command), command status, and a input context
that is used to pass information to the hardware. Not all commands need
the input context, so make it optional to allocate. Allow
xhci_free_container_ctx() to be passed a NULL input context, to make
freeing the xhci_command structure simple.
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
Refactor out the code to cache or free endpoint rings from recently
dropped or disabled endpoints. This code will be used by a new function
to reset a device and disable all endpoints except control endpoint 0.
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
If we fail to queue an evaluate context command or a configure endpoint
command to the command ring in xhci_configure_endpoint(), we need to
remove the xhci_command structure from the device's command list before
returning. If the command is left on the command list, it will sit there
indefinitely, blocking commands submitted after this fails.
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
Borislav Petkov reports issues with duplicate sysfs endpoint files after a
resume from a hibernate. It turns out that the code to support alternate
settings under xHCI has issues when a device with a non-default alternate
setting is reset during the hibernate:
[ 427.681810] Restarting tasks ...
[ 427.681995] hub 1-0:1.0: state 7 ports 6 chg 0004 evt 0000
[ 427.682019] usb usb3: usb resume
[ 427.682030] ohci_hcd 0000:00:12.0: wakeup root hub
[ 427.682191] hub 1-0:1.0: port 2, status 0501, change 0000, 480 Mb/s
[ 427.682205] usb 1-2: usb wakeup-resume
[ 427.682226] usb 1-2: finish reset-resume
[ 427.682886] done.
[ 427.734658] ehci_hcd 0000:00:12.2: port 2 high speed
[ 427.734663] ehci_hcd 0000:00:12.2: GetStatus port 2 status 001005 POWER sig=se0 PE CONNECT
[ 427.746682] hub 3-0:1.0: hub_reset_resume
[ 427.746693] hub 3-0:1.0: trying to enable port power on non-switchable hub
[ 427.786715] usb 1-2: reset high speed USB device using ehci_hcd and address 2
[ 427.839653] ehci_hcd 0000:00:12.2: port 2 high speed
[ 427.839666] ehci_hcd 0000:00:12.2: GetStatus port 2 status 001005 POWER sig=se0 PE CONNECT
[ 427.847717] ohci_hcd 0000:00:12.0: GetStatus roothub.portstatus [1] = 0x00010100 CSC PPS
[ 427.915497] hub 1-2:1.0: remove_intf_ep_devs: if: ffff88022f9e8800 ->ep_devs_created: 1
[ 427.915774] hub 1-2:1.0: remove_intf_ep_devs: bNumEndpoints: 1
[ 427.915934] hub 1-2:1.0: if: ffff88022f9e8800: endpoint devs removed.
[ 427.916158] hub 1-2:1.0: create_intf_ep_devs: if: ffff88022f9e8800 ->ep_devs_created: 0, ->unregistering: 0
[ 427.916434] hub 1-2:1.0: create_intf_ep_devs: bNumEndpoints: 1
[ 427.916609] ep_81: create, parent hub
[ 427.916632] ------------[ cut here ]------------
[ 427.916644] WARNING: at fs/sysfs/dir.c:477 sysfs_add_one+0x82/0x96()
[ 427.916649] Hardware name: System Product Name
[ 427.916653] sysfs: cannot create duplicate filename '/devices/pci0000:00/0000:00:12.2/usb1/1-2/1-2:1.0/ep_81'
[ 427.916658] Modules linked in: binfmt_misc kvm_amd kvm powernow_k8 cpufreq_ondemand cpufreq_powersave cpufreq_userspace freq_table cpufreq_conservative ipv6 vfat fat
+8250_pnp 8250 pcspkr ohci_hcd serial_core k10temp edac_core
[ 427.916694] Pid: 278, comm: khubd Not tainted 2.6.33-rc2-00187-g08d869a-dirty #13
[ 427.916699] Call Trace:
The problem is caused by a mismatch between the USB core's view of the
device state and the USB device and xHCI host's view of the device state.
After the device reset and re-configuration, the device and the xHCI host
think they are using alternate setting 0 of all interfaces. However, the
USB core keeps track of the old state, which may include non-zero
alternate settings. It uses intf->cur_altsetting to keep the endpoint
sysfs files for the old state across the reset.
The bandwidth allocation functions need to know what the xHCI host thinks
the current alternate settings are, so original patch set
intf->cur_altsetting to the alternate setting 0. This caused duplicate
endpoint files to be created.
The solution is to not set intf->cur_altsetting before calling
usb_set_interface() in usb_reset_and_verify_device(). Instead, we add a
new flag to struct usb_interface to tell usb_hcd_alloc_bandwidth() to use
alternate setting 0 as the currently installed alternate setting.
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Tested-by: Borislav Petkov <petkovbb@googlemail.com>
Cc: Alan Stern <stern@rowland.harvard.edu>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
Make the USB core check the bandwidth when switching from one
interface alternate setting to another. Also check the bandwidth
when resetting a configuration (so that alt setting 0 is used). If
this check fails, the device's state is unchanged. If the device
refuses the new alt setting, re-instate the old alt setting in the
host controller hardware.
If a USB device doesn't have an alternate interface setting 0, install
the first alt setting in its descriptors when a new configuration is
requested, or the device is reset.
Add a mutex per root hub to protect bandwidth operations:
adding/reseting/changing configurations, and changing alternate interface
settings. We want to ensure that the xHCI host controller and the USB
device are set up for the same configurations and alternate settings.
There are two (possibly three) steps to do this:
1. The host controller needs to check that bandwidth is available for a
different setting, by issuing and waiting for a configure endpoint
command.
2. Once that returns successfully, a control message is sent to the
device.
3. If that fails, the host controller must be notified through another
configure endpoint command.
The mutex is used to make these three operations seem atomic, to prevent
another driver from using more bandwidth for a different device while
we're in the middle of these operations.
While we're touching the bandwidth code, rename usb_hcd_check_bandwidth()
to usb_hcd_alloc_bandwidth(). This function does more than just check
that the bandwidth change won't exceed the bus bandwidth; it actually
changes the bandwidth configuration in the xHCI host controller.
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
Refactor out the code to find alternate interface settings into
usb_find_alt_setting(). Print a debugging message and return null if the
alt setting is not found.
While we're at it, correct a bug in the refactored code. The interfaces
in the configuration's interface cache are not necessarily in numerical
order, so we can't just use the interface number as an array index. Loop
through the interface caches, looking for the correct interface.
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
The xHCI driver issues a Configure Endpoint command for two reasons:
- a new configuration or alternate interface setting is selected
- a quirky Fresco Logic prototype requires the command after a Reset
Endpoint command.
The xHCI driver only waits on the command in the first case.
When a configure endpoint command completes, the driver needs to know why
the command was generated. When the driver only supported selecting an
initial configuration, the check was simple. Unfortunately that check
doesn't work now that the driver supports alternate interfaces. If an
endpoint must be dropped (because it's not in the new alternate setting)
and no new endpoints are added, the math involving
xhci_last_valid_endpoint() will assign -1 to an unsigned integer and cause
an out-of-bounds array access.
Move the check for the quirky hardware sooner and avoid the bad array
access.
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
When a driver wants to switch to a different alternate setting for an
interface, the USB core will (soon) check whether there is enough
bandwidth. Once the new alternate setting is installed in the xHCI
hardware, the USB core will send a USB_REQ_SET_INTERFACE control
message. That can fail in various ways, and the USB core needs to be
able to reinstate the old alternate setting.
With the old code, reinstating the old alt setting could fail if the
there's not enough memory to allocate new endpoint rings. Keep
around a cache of (at most 31) endpoint rings for this case. When we
successfully switch the xHCI hardware to the new alt setting, the old
alt setting's rings will be stored in the cache. Therefore we'll
always have enough rings to satisfy a conversion back to a previous
device setting.
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
Make the USB Test & Measurement driver use usb_clear_halt() instead of
usb_control_msg() to clear a stalled endpoint. This will allow devices to
be tested under an xHCI host controller. The endpoint stall will not be
cleared in the internal xHCI hardware state unless usb_clear_halt() is
used.
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Cc: Steve Holland <sdh4@iastate.edu>
Cc: Oliver Neukum <oliver@neukum.org>
Cc: Jouni Ryno <Jouni.Ryno@fmi.fi>
Cc: Gergely Imreh <imrehg@gmail.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
Add the xHCI driver files to its MAINTAINERS entry so that I'm Cc'd on
cleanup patches. Update the email address to one I actually use for
sending patches and responding to Linux mailing list emails.
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Cc: stable <stable@kernel.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
The xHCI 0.95 and 0.96 specification defines several transfer buffer
request completion codes that indicate a USB transaction error occurred.
When a stall, babble, transaction, or split transaction error completion code
is set, the xHCI has halted that endpoint ring. Software must issue a
Reset Endpoint command and a Set Transfer Ring Dequeue Pointer command
to clean up the halted ring.
The USB device driver is supposed to call into usb_reset_endpoint() when
an endpoint stalls. That calls into the xHCI driver to issue the proper
commands. However, drivers don't call that function for the other
errors that cause the xHC to halt the endpoint ring. If a babble,
transaction, or split transaction error occurs, check if the endpoint
context reports a halted condition, and clean up the endpoint ring if it
does.
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
An xHCI host controller manufacturer can choose to implement several
vendor-specific informational completion codes. These are all to be
treated like a successful transfer completion.
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
When the xHCI hardware says a transfer completed with a split
transaction error, set the URB status to -EPROTO.
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
The transfer descriptor (TD) is a series of transfer request buffers
(TRBs) that describe the buffer pointer, length, and other
characteristics. The xHCI controllers want to know an estimate of how
long the TD is, for caching reasons. In each TRB, there is a "TD size"
field that provides a rough estimate of the remaining buffers to be
transmitted, including the buffer pointed to by that TRB.
The TD size is 5 bits long, and contains the remaining size in bytes,
right shifted by 10 bits. So a remaining TD size less than 1024 would get
a zero in the TD size field, and a remaining size greater than 32767 would
get 31 in the field.
This patches fixes a bug in the TD_REMAINDER macro that is triggered when
the URB has a scatter gather list with a size bigger than 32767 bytes.
Not all host controllers pay attention to the TD size field, so the bug
will not appear on all USB 3.0 hosts.
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
It's not surprising that the transfer request buffer (TRB) physical to
virtual address translation function has bugs in it, since I wrote most of
it at 4am last October. Add a test suite to check the TRB math. This
runs at memory initialization time, and causes the driver to fail to load
if the TRB math fails.
Please excuse the excessively long lines in the test vectors; they can't
really be made shorter and still be readable.
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
The EHCI specification says that an EHCI host controller may cache part of
the isochronous schedule. The EHCI controller must advertise how much it
caches in the schedule through the HCCPARAMS register isochronous
scheduling threshold (IST) bits.
In theory, adding new iTDs within the IST should be harmless. The HW will
follow the old cached linked list and miss the new iTD. SW will notice HW
missed the iTD and return 0 for the transfer length.
However, Intel ICH9 chipsets (and some later chipsets) have issues when SW
attempts to schedule a split transaction within the IST. All transfers
will cease being sent out that port, and the drivers will see isochronous
packets complete with a length of zero. Start of frames may or may not
also disappear, causing the device to go into auto-suspend. This "bus
stall" will continue until a control or bulk transfer is queued to a
device under that roothub.
Most drivers will never cause this behavior, because they use multiple
URBs with multiple packets to keep the bus busy. If you limit the number
of URBs to one, you may be able to hit this bug.
Make sure the EHCI driver does not schedule full-speed transfers within
the IST under an Intel chipset. Make sure that when we fall behind the
current microframe plus IST, we schedule the new transfer at the next
periodic interval after the IST.
Don't change the scheduling for new transfers, since the schedule slop will
always be greater than the IST. Allow high speed isochronous transfers to
be scheduled within the IST, since this doesn't trigger the Intel chipset
bug.
Make sure that if the host caches the full frame, the EHCI driver's
internal isochronous threshold (ehci->i_thresh) is set to
8 microframes + 2 microframes wiggle room. This is similar to what is done in
the case where the host caches less than the full frame.
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Cc: Alan Stern <stern@rowland.harvard.edu>
Cc: David Brownell <dbrownell@users.sourceforge.net>
Cc: Clemens Ladisch <clemens@ladisch.de>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
Change the constant SCHEDULE_SLOP to be 80 microframes, instead of 10
frames. It was always multiplied by 8 to convert frames to microframes.
SCHEDULE_SLOP is only used in ehci-sched.c.
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Cc: David Brownell <dbrownell@users.sourceforge.net>
Acked-by: Alan Stern <stern@rowland.harvard.edu>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
CONFIG_USB_HCD_STAT was used in an abandoned patch to track host
controller throughput statistics. Since CONFIG_USB_HCD_STAT will never be
defined, remove code that can never run.
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
In order to giveback a canceled URB, we must ensure that the xHCI
hardware will not access the buffer in an URB. We can't modify the
buffer pointers on endpoint rings without issuing and waiting for a stop
endpoint command. Since URBs can be canceled in interrupt context, we
can't wait on that command. The old code trusted that the host
controller would respond to the command, and would giveback the URBs in
the event handler. If the hardware never responds to the stop endpoint
command, the URBs will never be completed, and we might hang the USB
subsystem.
Implement a watchdog timer that is spawned whenever a stop endpoint
command is queued. If a stop endpoint command event is found on the
event ring during an interrupt, we need to stop the watchdog timer with
del_timer(). Since del_timer() can fail if the timer is running and
waiting on the xHCI lock, we need a way to signal to the timer that
everything is fine and it should exit. If we simply clear
EP_HALT_PENDING, a new stop endpoint command could sneak in and set it
before the watchdog timer can grab the lock.
Instead we use a combination of the EP_HALT_PENDING flag and a counter
for the number of pending stop endpoint commands
(xhci_virt_ep->stop_cmds_pending). If we need to cancel the watchdog
timer and del_timer() succeeds, we decrement the number of pending stop
endpoint commands. If del_timer() fails, we leave the number of pending
stop endpoint commands alone. In either case, we clear the
EP_HALT_PENDING flag.
The timer will decrement the number of pending stop endpoint commands
once it obtains the lock. If the timer is the tail end of the last stop
endpoint command (xhci_virt_ep->stop_cmds_pending == 0), and the
endpoint's command is still pending (EP_HALT_PENDING is set), we assume
the host is dying. The watchdog timer will set XHCI_STATE_DYING, try to
halt the xHCI host, and give back all pending URBs.
Various other places in the driver need to check whether the xHCI host
is dying. If the interrupt handler ever notices, it should immediately
stop processing events. The URB enqueue function should also return
-ESHUTDOWN. The URB dequeue function should simply return the value
of usb_hcd_check_unlink_urb() and the watchdog timer will take care of
giving the URB back. When a device is disconnected, the xHCI hardware
structures should be freed without issuing a disable slot command (since
the hardware probably won't respond to it anyway). The debugging
polling loop should stop polling if the host is dying.
When a device is disconnected, any pending watchdog timers are killed
with del_timer_sync(). It must be synchronous so that the watchdog
timer doesn't attempt to access the freed endpoint structures.
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
xhci_quiesce() is basically a no-op right now. It's only called if
HC_IS_RUNNING() is true, and the body of the function consists of a
BUG_ON if HC_IS_RUNNING() is false. For the new xHCI watchdog timer, we
need a new function that clears the xHCI running bit in the command
register, but doesn't wait for the halt status to show up in the status
register. Re-purpose xhci_quiesce() to do that.
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
In the old code, there was a race condition between the stop endpoint
command and the URB submission process. When the stop endpoint command is
handled by the event handler, the endpoint ring is assumed to be stopped.
When a stop endpoint command is queued, URB submissions are to not ring
the doorbell. The old code would check the number of pending URBs to be
canceled, and would not ring the doorbell if it was non-zero.
However, the following race condition could occur with the old code:
1. Cancel an URB, add it to the list of URBs to be canceled, queue the stop
endpoint command, and increment ep->cancels_pending to 1.
2. The URB finishes on the HW, and an event is enqueued to the event ring
(at the same time as 1).
3. The stop endpoint command finishes, and the endpoint is halted. An
event is queued to the event ring.
4. The event handler sees the finished URB, notices it was to be
canceled, decrements ep->cancels_pending to 0, and removes it from the to
be canceled list.
5. The event handler drops the lock and gives back the URB. The
completion handler requeues the URB (or a different driver enqueues a new
URB). This causes the endpoint's doorbell to be rung, since
ep->cancels_pending == 0. The endpoint is now running.
6. A second URB is canceled, and it's added to the canceled list.
Since ep->cancels_pending == 0, a new stop endpoint command is queued, and
ep->cancels_pending is incremented to 1.
7. The event handler then sees the completed stop endpoint command. The
handler assumes the endpoint is stopped, but it isn't. It attempts to
move the dequeue pointer or change TDs to cancel the second URB, while the
hardware is actively accessing the endpoint ring.
To eliminate this race condition, a new endpoint state bit is introduced,
EP_HALT_PENDING. When this bit is set, a stop endpoint command has been
queued, and the command handler has not begun to process the URB
cancellation list yet. The endpoint doorbell should not be rung when this
is set. Set this when a stop endpoint command is queued, clear it when
the handler for that command runs, and check if it's set before ringing a
doorbell. ep->cancels_pending is eliminated, because it is no longer
used.
Make sure to ring the doorbell for an endpoint when the stop endpoint
command handler runs, even if the canceled URB list is empty. All
canceled URBs could have completed and new URBs could have been enqueued
without the doorbell being rung before the command was handled.
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
The scratchpad_free() function uses xhci->page_size to free some memory
with pci_free_consistent(). However, the page_size is set to zero before
the call, causing kernel oopses on driver unload. Call scratchpad_free()
before setting xhci->page_size to zero.
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Acked-by: John Youn <John.Youn@synopsys.com>
Cc: stable <stable@kernel.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
The trb_in_td() function in the xHCI driver is supposed to translate a
physical transfer buffer request (TRB) into a virtual pointer to the ring
segment that TRB is in.
Unfortunately, a mistake in this function may cause endless loops as the
driver searches through the linked list of ring segments over and over
again. Fix a couple bugs that may lead to loops or bad output:
1. Bail out if we get a NULL pointer when translating the segment's
private structure and the starting DMA address of the segment chunk. If
this happens, we've been handed a starting TRB pointer from a different
ring.
2. Make sure the function works when there's multiple segments in the
ring. In the while loop to search through the ring segments, use the
current segment variable (cur_seg), rather than the starting segment
variable (start_seg) that is passed in.
3. Stop searching the ring if we've run through all the segments in the
ring.
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Cc: stable <stable@kernel.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
If the xHCI driver fails during the memory initialization, xhci->ir_set
may not be a valid pointer. Check that it points to valid DMA'able memory
before writing to that address during the memory freeing process.
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Cc: stable <stable@kernel.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
When the EHCI driver falls behind in its scheduling, the active stream's
first empty microframe may be in the past with respect to the current
microframe. The code attempts to move the starting microframe ("start") N
number of microframes forward, where N is the interval of endpoint.
However, stream->interval is a copy of the endpoint's bInterval, which is
designated in frames for FS devices, and microframes for HS devices.
Convert stream->interval to microframes before using it to move the
starting microframe forward.
Acked-by: Alan Stern <stern@rowland.harvard.edu>
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
The EHCI driver does some bounds checking when it's scheduling an iTD for
an active endpoint. It sets the local variable start to
stream->next_uframe and moves that variable further in the schedule if
necessary. However, the driver fails to do anything with start before
jumping to the ready label and setting the URB's starting frame to
stream->next_uframe. Alan Stern confirms the EHCI driver should set
stream->next_uframe to start before jumping.
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Acked-by: Alan Stern <stern@rowland.harvard.edu>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
When an endpoint is to be dropped from the hardware bandwidth schedule, we
want to clear its add flag.
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
When the host controller dies or is removed while a device is plugged in,
the USB core will attempt to deallocate the struct usb_device. That will
call into xhci_free_dev(). This function used to attempt to submit a
disable slot command to the host controller and clean up the device
structures when that command returned. Change xhci_free_dev() to skip the
command submission and just free the memory if the host controller died.
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
When the host controller dies (e.g. it is removed from a PCI card slot),
the xHCI driver cannot expect commands to complete. The buggy code this
patch fixes would mark an URB as canceled and then expect the URB to be
completed when the stop endpoint command completed. That would never
happen if the host controller was dead, so the USB core would just hang in
the disconnect code.
If the host controller died, and the driver asks to cancel an URB, free
any structures associated with that URB and immediately give it back.
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
If the host controller card is removed from the system, stop the timer
function to debug the xHCI rings.
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
Add a hook for updating xHCI internal structures after khubd fetches the
hub descriptor and sets up the hub's TT information. The xHCI driver must
update the internal structures before devices under the hub can be
enumerated.
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
For a USB hub to work under an xHCI host controller, the xHC's internal
scheduler must be made aware of the hub's characteristics. Add an xHCI
hook that the USB core will call after it fetches the hub descriptor.
This hook will add hub information to the slot context for that device,
including whether it has multiple TTs or a single TT, the number of ports
on the hub, and TT think time.
Setting up the slot context for the device is different for 0.95 and 0.96
xHCI host controllers.
Some of the slot context reserved fields in the 0.95 specification were
changed into hub fields in the 0.96 specification. Don't set the TT think
time or number of ports for a hub if we're dealing with a 0.95-compliant
xHCI host controller.
The 0.95 xHCI specification says that to modify the hub flag, we need to
issue an evaluate context command. The 0.96 specification says that flag
can be set with a configure endpoint command. Issue the correct command
based on the version reported by the hardware.
This patch does not add support for multi-TT hubs. Multi-TT hubs expose
a single TT on alt setting 0, and multi-TT on alt setting 1. The xHCI
driver can't handle setting alternate interfaces yet.
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
When setting up a slot context for an address device command, set the
multi-TT field if this is a low or full speed device under a HS hub with
multiple transaction translators.
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
The xHCI driver needs to set the route string in the slot context of all
devices, not just SuperSpeed devices. The route string concept was added
in the USB 3.0 specification, section 10.1.3.2. Each hub in the topology
is expected to have no more than 15 ports in order for the route string of
a device to be unique. SuperSpeed hubs are restricted to only having 15
ports, but FS/LS/HS hubs are not. The xHCI specification says that if the
port number the device is under is greater than 15, that portion of the
route string shall be set to 15.
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
In the xHCI driver, configure endpoint commands that are submitted to the
hardware may involve one of two data structures. If the configure
endpoint command is setting up a new configuration or modifying max packet
sizes, the data structures and completions are statically allocated in the
xhci_virt_device structure. If the command is being used to set up
streams or add hub information, then the data structures are dynamically
allocated, and placed on a device command waiting list.
Break out the code to check whether a completed command is in the device
command waiting list. Fix a subtle bug in the old code: continue
processing the command if the command isn't in the wait list. In the old
code, if there was a command in the wait list, but it didn't match the
completed command, the completed command event would be dropped.
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
Some commands to the xHCI hardware cannot be allowed to fail due to out of
memory issues or the command ring being full.
Add a way to reserve a TRB on the command ring, and make all command
queueing functions indicate whether they are using a reserved TRB.
Add a way to pre-allocate all the memory a command might need. A command
needs an input context, a variable to store the status, and (optionally) a
completion for the caller to wait on. Change all code that assumes the
input device context, status, and completion for a command is stored in
the xhci virtual USB device structure (xhci_virt_device).
Store pending completions in a FIFO in xhci_virt_device. Make the event
handler for a configure endpoint command check to see whether a pending
command in the list has completed. We need to use separate input device
contexts for some configure endpoint commands, since multiple drivers can
submit requests at the same time that require a configure endpoint
command.
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
Refactor common code to set up the add and drop flags for the input device
context setup. This setup is used before a configure endpoint command for
the reset endpoint quirk, and will be used for the command to alloc or
free streams rings.
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
The xhci_ring structure contained information that is really related to an
endpoint, not a ring. This will cause problems later when endpoint
streams are supported and there are multiple rings per endpoint.
Move the endpoint state and cancellation information into a new virtual
endpoint structure, xhci_virt_ep. The list of TRBs to be cancelled should
be per endpoint, not per ring, for easy access. There can be only one TRB
that the endpoint stopped on after a stop endpoint command (even with
streams enabled); move the stopped TRB information into the new virtual
endpoint structure. Also move the 31 endpoint rings and temporary ring
storage from the virtual device structure (xhci_virt_device) into the
virtual endpoint structure (xhci_virt_ep).
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
When there's a descriptor after the SuperSpeed endpoint companion
descriptor, the previous code would have skipped over twice the length it
was supposed to. This code fixes crashes seen with UASP devices (which
have a UASP descriptor after the SS endpoint companion descriptor).
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Cc: stable <stable@kernel.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
Interrupt transfers are submitted to the xHCI hardware using the same TRB
type as bulk transfers. Re-use the bulk transfer enqueueing code to
enqueue interrupt transfers.
Interrupt transfers are a bit different than bulk transfers. When the
interrupt endpoint is to be serviced, the xHC will consume (at most) one
TD. A TD (comprised of sg list entries) can take several service
intervals to transmit. The important thing for device drivers to note is
that if they use the scatter gather interface to submit interrupt
requests, they will not get data sent from two different scatter gather
lists in the same service interval.
For now, the xHCI driver will use the service interval from the endpoint's
descriptor (bInterval). Drivers will need a hook to poll at a more
frequent interval. Set urb->interval to the interval that the xHCI
hardware will use.
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Cc: stable <stable@kernel.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
The xHCI hardware reports the number of bytes untransferred for a given
transfer buffer. If the hardware reports a bytes untransferred value
greater than the submitted buffer size, we want to play it safe and say no
data was transferred. If the driver considers a short packet to be an
error, remember to set -EREMOTEIO.
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Cc: stable <stable@kernel.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
Make sure that the driver that submitted the URB considers a short packet
an error before setting -EREMOTEIO during a short control transfer.
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Cc: stable <stable@kernel.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
Make sure that the amount of data the xHC says was transmitted is less
than or equal to the size of the requested transfer buffer. Before, if
the host controller erroneously reported that the number of bytes
untransferred was bigger than the buffer in the URB, urb->actual_length
could be set to a very large size.
Make sure urb->actual_length <= urb->transfer_buffer_length.
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Cc: stable <stable@kernel.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
On a successful transfer, urb->td is freed before the URB is ready to be
given back to the driver. Don't touch urb->td after it's freed. This bug
would have only shown up when xHCI debugging was turned on, and the freed
memory was quickly reused for something else.
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Cc: stable <stable@kernel.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
The 0.95 xHCI spec says that non-control endpoints will be halted if a
babble is detected on a transfer. The 0.96 xHCI spec says all types of
endpoints will be halted when a babble is detected. Some hardware that
claims to be 0.95 compliant halts the control endpoint anyway.
When a babble is detected on a control endpoint, check the hardware's
output endpoint context to see if the endpoint is marked as halted. If
the control endpoint is halted, a reset endpoint command must be issued
and the transfer ring dequeue pointer needs to be moved past the stopped
transfer. Basically, we treat it as if the control endpoint had stalled.
Handle bulk babbling endpoints as if we got a completion event with a
stall completion code.
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Cc: stable <stable@kernel.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
This Fresco Logic xHCI host controller chip revision puts bad data into
the output endpoint context after a Reset Endpoint command. It needs a
Configure Endpoint command (instead of a Set TR Dequeue Pointer command)
after the reset endpoint command.
Set up the input context before issuing the Reset Endpoint command so we
don't copy bad data from the output endpoint context. The HW also can't
handle two commands queued at once, so submit the TRB for the Configure
Endpoint command in the event handler for the Reset Endpoint command.
Devices that stall on control endpoints before a configuration is selected
will not work under this Fresco Logic xHCI host controller revision.
This patch is for prototype hardware that will be given to other companies
for evaluation purposes only, and should not reach consumer hands. Fresco
Logic's next chip rev should have this bug fixed.
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Cc: stable <stable@kernel.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
When a control endpoint stalls, the next control transfer will clear the
stall. The USB core doesn't call down to the host controller driver's
endpoint_reset() method when control endpoints stall, so the xHCI driver
has to do all its stall handling for internal state in its interrupt handler.
When the host stalls on a control endpoint, it may stop on the data phase
or status phase of the control transfer. Like other stalled endpoints,
the xHCI driver needs to queue a Reset Endpoint command and move the
hardware's control endpoint ring dequeue pointer past the failed control
transfer (with a Set TR Dequeue Pointer or a Configure Endpoint command).
Since the USB core doesn't call usb_hcd_reset_endpoint() for control
endpoints, we need to do this in interrupt context when we get notified of
the stalled transfer. URBs may be queued to the hardware before these two
commands complete. The endpoint queue will be restarted once both
commands complete.
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Cc: stable <stable@kernel.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
Full speed devices have varying max packet sizes (8, 16, 32, or 64) for
endpoint 0. The xHCI hardware needs to know the real max packet size
that the USB core discovers after it fetches the first 8 bytes of the
device descriptor.
In order to fix this without adding a new hook to host controller drivers,
the xHCI driver looks for an updated max packet size for control
endpoints. If it finds an updated size, it issues an evaluate context
command and waits for that command to finish. This should only happen in
the initialization and device descriptor fetching steps in the khubd
thread, so blocking should be fine.
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Cc: stable <stable@kernel.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
Set the max packet size for the default control endpoint on high speed
devices to be 64 bytes. High speed devices always have a max packet size
of 64 bytes. There's no use setting it to eight for the initial 8 byte
descriptor fetch and then issuing (and waiting for) an evaluate context
command to update it to 64 bytes for the subsequent control transfers.
The USB core guesses that the max packet size on a full speed control
endpoint is 64 bytes, and then updates it after the first 8-byte
descriptor fetch. Change the initial setup for the xHCI internal
representation of the full speed device to have a 64 byte max packet size.
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Cc: stable <stable@kernel.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
Refactor out the code issue, wait for, and parse the event completion code
for a configure endpoint command. Modify it to support the evaluate
context command, which has a very similar submission process. Add
functions to copy parts of the output context into the input context
(which will be used in the evaluate context command).
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Cc: stable <stable@kernel.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
Use the virtual address of the memory hardware uses, not the address for
the container of that memory.
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Cc: stable <stable@kernel.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
Different sections of the xHCI 0.95 specification had opposing
requirements for the chain bit in a link transaction request buffer (TRB).
The chain bit is used to designate that adjacent TRBs are all part of the
same scatter gather list that should be sent to the device. Link TRBs can
be in the middle, or at the beginning or end of these chained TRBs.
Sections 4.11.5.1 and 6.4.4.1 both stated the link TRB "shall have the
chain bit set to 1", meaning it is always chained to the next TRB.
However, section 4.6.9 on the stop endpoint command has specific cases for
what the hardware must do for a link TRB with the chain bit set to 0. The
0.96 specification errata later cleared up this issue by fixing the
4.11.5.1 and 6.4.4.1 sections to state that a link TRB can have the chain
bit set to 1 or 0.
The problem is that the xHCI cancellation code depends on the chain bit of
the link TRB being cleared when it's at the end of a TD, and some 0.95
xHCI hardware simply stops processing the ring when it encounters a link
TRB with the chain bit cleared.
Allow users who are testing 0.95 xHCI prototypes to set a module parameter
(link_quirk) to turn on this link TRB work around. Cancellation may not
work if the ring is stopped exactly on a link TRB with chain bit set, but
cancellation should be a relatively uncommon case.
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Cc: stable <stable@kernel.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
Correct the xHCI code to handle stalls on USB endpoints. We need to move
the endpoint ring's dequeue pointer past the stalled transfer, or the HW
will try to restart the transfer the next time the doorbell is rung.
Don't attempt to clear a halt on an endpoint if we haven't seen a stalled
transfer for it. The USB core will attempt to clear a halt on all
endpoints when it selects a new configuration.
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
Make sure the xHCI output device context is 64-byte aligned. Previous
code was using the same structure for both the output device context and
the input control context. Since the structure had 32 bytes of flags
before the device context, the output device context wouldn't be 64-byte
aligned. Define a new structure to use for the output device context and
clean up the debugging for these two structures.
The copy of the device context in the input control context does *not*
need to be 64-byte aligned.
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
usb_parse_ss_endpoint_companion() was supposed to allocate a structure to
hold the SuperSpeed Endpoint Companion descriptor, and either copy the
values the device returned, or fill in default values if the device
descriptor did not include the companion descriptor.
However, the previous code would miss the last endpoint in a configuration
with no descriptors after it. Make usb_parse_endpoint() allocate the SS
endpoint companion descriptor and fill it with default values, even if
we've run out of buffer space in this configuration descriptor.
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
This is a work around for a bug in the SuperSpeed Endpoint Companion Descriptor
parsing code. It fails in some corner cases, which means ep->ss_ep_comp may be
NULL.
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
Pass back a babble error when this error code is seen in the transfer event TRB.
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
The xHCI host controller can be programmed to retry a transfer a certain number
of times per endpoint before it passes back an error condition to the host
controller driver. The xHC will return an error code when the error count
transitions from 1 to 0. Programming an error count of 3 means the xHC tries
the transfer 3 times, programming it with a 1 means it tries to transfer once,
and programming it with 0 means the HW tries the transfer infinitely.
We want isochronous transfers to only be tried once, so set the error count to
one.
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
Sarah Sharp