Instead of single isp_intr() function doing all possible magic, introduce
four different functions to handle mailbox operation completions, async
events, response and ATIO queues. The goal is to isolate different code
paths to make code more readable, and to make easier support for multiple
interrupt vectors. Even oldest hardware in many cases can identify what
code path it should run on interrupt. Contemporary hardware can assign
them to different interrupt vectors.
MFC after: 2 weeks
It was implemented to reduce context switches when uploading firmware to
card's RAM. But this mechanism is not used last 10 years since all mbox
operations are now polled, and it was never used for cards produced in
last 15 years. Newer cards can use DMA to upload firmware.
MFC after: 2 weeks
This change fixes DMA resource leak on driver unload. Also it removes
DMA resources allocation for hardcoded number of requests before fetching
the real number from firmware. Also it prepares ground for more flexible
IRQs allocation according to firmware capabilities.
MFC after: 2 weeks
Its more important for SPI HBAs, as they don't support CDBs above 12 bytes.
The new error code makes CAM to fall back to alternative commands.
MFC after: 2 weeks
The isp(4) driver was changing the tag type for REQUEST SENSE
commands to Head of Queue, when the CAM CCB flag
CAM_TAG_ACTION_VALID was NOT set. CAM_TAG_ACTION_VALID is set
when the tag action in the XPT_SCSI_IO is not CAM_TAG_ACTION_NONE
and when the target has tagged queueing turned on.
In most cases when CAM_TAG_ACTION_VALID is not set, it is because
the target is not doing tagged queueing. In those cases, trying to
send a Head of Queue tag may cause problems. Instead, default to
sending a simple tag.
IBM tape drives claim to support tagged queueing in their standard
Inquiry data, but have the DQue bit set in the control mode page
(mode page 10). CAM correctly detects that these drives do not
support tagged queueing, and clears the CAM_TAG_ACTION_VALID flag
on CCBs sent down to the drives.
This caused the isp(4) driver to go down the path of setting the
tag action to a default value, and for Request Sense commands only,
set the tag action to Head of Queue.
If an IBM tape drive does get a Head of Queue tag, it rejects it with
Invalid Message Error (0x49,0x00). (The Qlogic firmware translates that
to a Transport Error, which the driver translates to an Unrecoverable
HBA Error, or CAM_UNREC_HBA_ERROR.) So, by default, it wasn't possible
to get a good response from a REQUEST SENSE to an FC-attached IBM
tape drive with the isp(4) driver.
IBM tape drives (tested on an LTO-5 with G9N1 firmware and a TS1150
with 4470 firmware) also have a bug in that sending a command with a
non-simple tag attribute breaks the tape drive's Command Reference
Number (CRN) accounting and causes it to ignore all subsequent
commands because it and the initiator disagree about the next
expected CRN. The drives do reject the initial command with a head
of queue tag with an Invalid Message Error (0x49,0x00), but after that
they ignore any subsequent commands. IBM confirmed that it is a bug,
and sent me test firmware that fixes the bug. However tape drives in
the field will still exhibit the bug until they are upgraded.
Request Sense is not often sent to targets because most errors are
reported automatically through autosense in Fibre Channel and other
modern transports. ("Modern" meaning post SCSI-2.) So this is not
an error that would crop up frequently. But Request Sense is useful on
tape devices to report status information, aside from error reporting.
This problem is less serious without FC-Tape features turned on,
specifically precise delivery of commands (which enables Command
Reference Numbers), enabled on the target and initiator. Without
FC-Tape features turned on, the target would return an error and
things would continue on.
And it also does not cause problems for targets that do tagged
queueing, because in those cases the isp(4) driver just uses the
tag type that is specified in the CCB, assuming the
CAM_TAG_ACTION_VALID flag is set, and defaults to sending a Simple
tag action if it isn't an ordered or head of queue tag.
sys/dev/isp/isp.c:
In isp_start(), don't try to send Request Sense commands
with the Head of Queue tag attribute if the CCB doesn't
have a valid tag action. The tag action likely isn't valid
because the target doesn't support tagged queueing.
Sponsored by: Spectra Logic
MFC after: 3 days
Replace archaic "busses" with modern form "buses."
Intentionally excluded:
* Old/random drivers I didn't recognize
* Old hardware in general
* Use of "busses" in code as identifiers
No functional change.
http://grammarist.com/spelling/buses-busses/
PR: 216099
Reported by: bltsrc at mail.ru
Sponsored by: Dell EMC Isilon
This code was originally implemented 7 years ago, but never really worked
due to trivial error. I think this functionality may be not required.
Initiators supporting optional periodic command status checks detected
those terminated commands and retried them 3 seconds later. But thinking
about less featured initiators and the fact that it is our race makes
virtual ports "unknown" it may be good to have this feature.
It is normal for ZOMBIE ports to be logged out. This status is not really
an error until Gone Device Timeout expires, so make CAM retry after delay.
MFC after: 1 week
Firmware automatically logs in only to local loop ports, and those ports
can be easily identified without extra flag by zero domain and area IDs.
MFC after: 1 week
Since we no longer need additional buffers for request and response IOCBs,
we can increase receive space by 192 bytes, that is enough for fetching 48
more ports. The new limit is 1020 fabric ports per virtual port.
MFC after: 1 month
This should close the race between request arriving on new target mode
virtual port and its scanner thread finally fetch its address for request
routing.
For some reason firmware sends Port Database Changed notifications in case
of explicit login requests from the driver when target port is unavailabe.
Those notifications don't give driver any new information, but only cause
infinite scan loop.
Previously we had to do it synchronously because we could not drop the lock
due to potential scratch memory use conflicts. Previous commits fixed that
collision, so here it goes -- slower and less reliable external requests
are executed asynchronously without spinning in tight loop and with more
safe timeout handling.
Usually IOCBs should be put on queue for asynchronous processing and should
not require additional DMA memory. But there are some cases like aborts and
resets that for external reasons has to be synchronous. Give those cases
separate 2*64 byte DMA area to decouple them from other DMA scratch area
users, using it for asynchronous requests.
This is cosmetics that simplifies identification of new ports on FC switch.
It would be good to use target name from CTL here instead of hostname, but
it is not passed here through CAM now.
MFC after: 2 weeks
This space does not require DMA syncing. It reduces lock scope of the DMA
scratch space. It allows whole DMA scratch space to be used to I/O, so now
we can fetch up to ~1000 ports from SNS.
Due to the last fact, increase maximal number of ports from 256 to 1024.
Before this change virtual ports control IOCBs were executed synchronously
via Execute IOCB mailbox command. It required exclusive use of scratch
space of driver and mailbox registers of the hardware. Because of that
shared resources use this code could not really sleep, having to spin for
completion, blocking any other operation.
This change introduces new asynchronous design, sending the IOCBs directly
on request queue and gracefully waiting for their return on response queue.
Returned IOCBs are identified with unified handle space from r292725.
I am not sure why this was split long ago, but I see no reason for it.
At this point this unification just slightly reduces memory usage, but
as next step I plan to reuse shared handle space for other IOCB types.
- Make scan aborted by event restart immediately and infinitely.
- Improve handling of some loop events from firmware.
- Remove loop down timer, adding its functionality to scanner thread.
- Some more unification and simplification.
Hacks to enable target mode there complicated code, while didn't really
work. And for outdated hardware fixing it is not really interesting.
Initiator mode tested with Qlogic 1080 adapter is still working fine.
While later firmware always registers for RSCN requests, older one does
it only in initiator mode. But in target mode there RSCN can be the only
way to detect gone intiator.
For those chips we are not receiving login events, adding initiators
based on ATIO requests. But there is no port ID in that structure, so
in fabric mode we have to explicitly fetch it from firmware to be able
to do normal scan after that.
This change simplifies and unifies port adding/updating for loop and
fabric scanners. It also fixes problems with scanning restarts due to
concurrent port databases changes. It also fixes many cosmetic issues.
Modern cards in most cases operate abstract port handles, that have no
any relation to real loop IDs. Leave loopid used only where it really
goes about local loop IDs.
While there, fix few more cases where LUNs were still printed in decimal.
For the most of chips (except anscient ones) port handlers have no relation
to port IDs. In such situation old code scanning first 125 handlers was
quite naive. Instead of doing that, send to chip single request to get full
list of port handlers available on specific virtual port and scan only them.
Old code had problems with case of several virtual ports enabled, when port
handlers allocated from global address space could easily go above 125.
This change was successfully tested on 23xx, 24xx and 25xx chips in loop
mode with 4 virtual initiator ports, each seing 50 virtual target ports.
Now on 24xx and above chips it is really possible to simulate several
virtual FC ports with single physical one. For example, it allows to
configure several targets in ctl.conf, assign each of them to separate
virtual port, and let user to control access to them with switch zoning.
I still doubt that all problems are solved there, but at now it passes
at least basic tests.
Now 24xx and above chips support full 8-byte LUN address space.
Older FC chips may support up to 16K LUNs when firmware allows.
Tested in both initiator and target modes for 23xx, 24xx and 25xx.
Aside from cleaner and more consistent code, this allows ports to be both
target and initiator same time, and easily switch from any role to any.
Sponsored by: iXsystems, Inc.
FreeBSD never had limitation on number of target IDs, and there is no
any other requirement to allocate them densely. Since slots of port
database already populated just sequentially, there is no much need
for another indirection to allocate sequentially too.
It is odd that driver first tries to generate synthetic WWNN based on
WWPN starting from 2, but then refuses to use it. If we don't trust
generated WWNN, we should probably not generate it. Same time this
limitation prevents potentially valid WWNN setting by user.
The newer boards don't have the response field that indicates
whether the SCSI status byte is present. You have to just look to
see whether it is non-zero.
The code was looking to see whether the sense length was valid
before propagating the SCSI status byte (and sense information) up
the stack. With a status like Reservation Conflict, there is no
sense information, only the SCSI status byte. So it wasn't getting
correctly returned.
isp.c:
In isp_intr(), if we are on a 2400 or 2500 type board and
get a response, look at the actual contents of the
SCSI status value and set the RQSF_GOT_STATUS flag
accordingly so that return any SCSI status value we get. The
RQSF_GOT_SENSE flag will get set later on if there is
actual sense information returned.
Submitted by: ken
MFC after: 1 week
Sponsored by: Spectra Logic
MFSpectraBSD: 1112791 on 2015/01/15
- Process ATIO queue only if interrupt status tells so;
- Do not update queue out pointers after each processed command, do it
only once at the end of the loop.
- Remove two excessive and slow register reads from isp_intr(). Instead
of rereading value every time, assume that registers contain what we have
written there.
- Avoid sequential search through 4096 array elements when looking for
command tag. Use hash of lists to store active tags separately from free
ones and so greatly speedup the searches.
Reviewed by: mjacob
a mailbox command and which registers to copy back in when
the command completes, the bits being set need to not only
specify what bits you want to add from the default from the
table but also what bits you want *subtract* (mask) from the
default from the table.
A failing ISP2200 command pointed this out.
Much appreciation to: marius, who persisted and narrowed down what
the failure delta was, and shamed me into actually fixing it.
MFC after: 1 week
might have been enabled for them- now that we use all 32 bits of handle.
Fast Posting doesn't pass the full 32 bits.
Noticed by: Bugs in NetBSD. Only a NetBSD user might actually still use such old hardware.
MFC after: 1 week
MISC CHANGES
Add a new async event- ISP_TARGET_NOTIFY_ACK, that will guarantee
eventual delivery of a NOTIFY ACK. This is tons better than just
ignoring the return from isp_notify_ack and hoping for the best.
Clean up the lower level lun enable code to be a bit more sensible.
Fix a botch in isp_endcmd which was messing up the sense data.
Fix notify ack for SRR to use a sensible error code in the case
of a reject.
Clean up and make clear what kind of firmware we've loaded and
what capabilities it has.
-----------
FULL (252 byte) SENSE DATA
In CTIOs for the ISP, there's only a limimted amount of space
to load SENSE DATA for associated CHECK CONDITIONS (24 or 26
bytes). This makes it difficult to send full SENSE DATA that can
be up to 252 bytes.
Implement MODE 2 responses which have us build the FCP Response
in system memory which the ISP will put onto the wire directly.
On the initiator side, the same problem occurs in that a command
status response only has a limited amount of space for SENSE DATA.
This data is supplemented by status continuation responses that
the ISP pushes onto the response queue after the status response.
We now pull them all together so that full sense data can be
returned to the periph driver.
This is supported on 23XX, 24XX and 25XX cards.
This is also preparation for doing >16 byte CDBs.
-----------
FC TAPE
Implement full FC-TAPE on both initiator and target mode side. This
capability is driven by firmware loaded, board type, board NVRAM
settings, or hint configuration options to enable or disable. This
is supported for 23XX, 24XX and 25XX cards.
On the initiator side, we pretty much just have to generate a command
reference number for each command we send out. This is FCP-4 compliant
in that we do this per ITL nexus to generate the allowed 1 thru 255
CRN.
In order to support the target side of FC-TAPE, we now pay attention
to more of the PRLI word 3 parameters which will tell us whether
an initiator wants confirmed responses. While we're at it, we'll
pay attention to the initiator view too and report it.
On sending back CTIOs, we will notice whether the initiator wants
confirmed responses and we'll set up flags to do so.
If a response or data frame is lost the initiator sends us an SRR
(Sequence Retransmit Request) ELS which shows up as an SRR notify
and all outstanding CTIOs are nuked with SRR Received status. The
SRR notify contains the offset that the initiator wants us to restart
the data transfer from or to retransmit the response frame.
If the ISP driver still has the CCB around for which the data segment
or response applies, it will retransmit.
However, we typically don't know about a lost data frame until we
send the FCP Response and the initiator totes up counters for data
moved and notices missing segments. In this case we've already
completed the data CCBs already and sent themn back up to the periph
driver. Because there's no really clean mechanism yet in CAM to
handle this, a hack has been put into place to complete the CTIO
CCB with the CAM_MESSAGE_RECV status which will have a MODIFY DATA
POINTER extended message in it. The internal ISP target groks this
and ctl(8) will be modified to deal with this as well.
At any rate, the data is retransmitted and an an FCP response is
sent. The whole point here is to successfully complete a command
so that you don't have to depend on ULP (SCSI) to have to recover,
which in the case of tape is not really possible (hence the name
FC-TAPE).
Sponsored by: Spectralogic
MFC after: 1 month
not by some hint setting. Do more preparations for FC-Tape.
Clean up resource counting for 24XX or later chipsets so
we find out after EXEC_FIRMWARE what is actually supported.
Set target mode exchange count based upon whether or not
we are supporting simultaneous target/initiator mode. Clean
up some old (pre-24XX) xfwoption and zfwoption issues.
Sponsored by: Spectralogic
MFC after: 3 days
and crosschecks against firmware documentation. We now check and report
FC firmware attributes and at least are now prepared for the upper 48 bits
of f/w attributes (which are probably for the 8100 or later cards). This
involed changing how inbits and outbits are calculated for varios commands,
hopefully clearer and cleaner. This also caused me to clean up the actual
mailbox register usage. Finally, we are now unconditionally using a CRN
for initiator mode.
A longstanding issue with the 2400/2500 is that they do *not* support
a "Prefer PTP followed by loop", which explains why enabling that
caused the f/w to crash.
A slightly more invasive change is to let the firmware load entirely
drive whether multi_id support is enabled or not.
Sponsored by: Spectralogic
MFC after: 1 week
