Since we no longer need additional buffers for request and response IOCBs,
we can increase receive space by 192 bytes, that is enough for fetching 48
more ports. The new limit is 1020 fabric ports per virtual port.
MFC after: 1 month
This should close the race between request arriving on new target mode
virtual port and its scanner thread finally fetch its address for request
routing.
For some reason firmware sends Port Database Changed notifications in case
of explicit login requests from the driver when target port is unavailabe.
Those notifications don't give driver any new information, but only cause
infinite scan loop.
Previously we had to do it synchronously because we could not drop the lock
due to potential scratch memory use conflicts. Previous commits fixed that
collision, so here it goes -- slower and less reliable external requests
are executed asynchronously without spinning in tight loop and with more
safe timeout handling.
Usually IOCBs should be put on queue for asynchronous processing and should
not require additional DMA memory. But there are some cases like aborts and
resets that for external reasons has to be synchronous. Give those cases
separate 2*64 byte DMA area to decouple them from other DMA scratch area
users, using it for asynchronous requests.
This is cosmetics that simplifies identification of new ports on FC switch.
It would be good to use target name from CTL here instead of hostname, but
it is not passed here through CAM now.
MFC after: 2 weeks
This space does not require DMA syncing. It reduces lock scope of the DMA
scratch space. It allows whole DMA scratch space to be used to I/O, so now
we can fetch up to ~1000 ports from SNS.
Due to the last fact, increase maximal number of ports from 256 to 1024.
Before this change virtual ports control IOCBs were executed synchronously
via Execute IOCB mailbox command. It required exclusive use of scratch
space of driver and mailbox registers of the hardware. Because of that
shared resources use this code could not really sleep, having to spin for
completion, blocking any other operation.
This change introduces new asynchronous design, sending the IOCBs directly
on request queue and gracefully waiting for their return on response queue.
Returned IOCBs are identified with unified handle space from r292725.
I am not sure why this was split long ago, but I see no reason for it.
At this point this unification just slightly reduces memory usage, but
as next step I plan to reuse shared handle space for other IOCB types.
- Make scan aborted by event restart immediately and infinitely.
- Improve handling of some loop events from firmware.
- Remove loop down timer, adding its functionality to scanner thread.
- Some more unification and simplification.
Hacks to enable target mode there complicated code, while didn't really
work. And for outdated hardware fixing it is not really interesting.
Initiator mode tested with Qlogic 1080 adapter is still working fine.
While later firmware always registers for RSCN requests, older one does
it only in initiator mode. But in target mode there RSCN can be the only
way to detect gone intiator.
For those chips we are not receiving login events, adding initiators
based on ATIO requests. But there is no port ID in that structure, so
in fabric mode we have to explicitly fetch it from firmware to be able
to do normal scan after that.
This change simplifies and unifies port adding/updating for loop and
fabric scanners. It also fixes problems with scanning restarts due to
concurrent port databases changes. It also fixes many cosmetic issues.
Modern cards in most cases operate abstract port handles, that have no
any relation to real loop IDs. Leave loopid used only where it really
goes about local loop IDs.
While there, fix few more cases where LUNs were still printed in decimal.
For the most of chips (except anscient ones) port handlers have no relation
to port IDs. In such situation old code scanning first 125 handlers was
quite naive. Instead of doing that, send to chip single request to get full
list of port handlers available on specific virtual port and scan only them.
Old code had problems with case of several virtual ports enabled, when port
handlers allocated from global address space could easily go above 125.
This change was successfully tested on 23xx, 24xx and 25xx chips in loop
mode with 4 virtual initiator ports, each seing 50 virtual target ports.
Now on 24xx and above chips it is really possible to simulate several
virtual FC ports with single physical one. For example, it allows to
configure several targets in ctl.conf, assign each of them to separate
virtual port, and let user to control access to them with switch zoning.
I still doubt that all problems are solved there, but at now it passes
at least basic tests.
Now 24xx and above chips support full 8-byte LUN address space.
Older FC chips may support up to 16K LUNs when firmware allows.
Tested in both initiator and target modes for 23xx, 24xx and 25xx.
Aside from cleaner and more consistent code, this allows ports to be both
target and initiator same time, and easily switch from any role to any.
Sponsored by: iXsystems, Inc.
FreeBSD never had limitation on number of target IDs, and there is no
any other requirement to allocate them densely. Since slots of port
database already populated just sequentially, there is no much need
for another indirection to allocate sequentially too.
It is odd that driver first tries to generate synthetic WWNN based on
WWPN starting from 2, but then refuses to use it. If we don't trust
generated WWNN, we should probably not generate it. Same time this
limitation prevents potentially valid WWNN setting by user.
The newer boards don't have the response field that indicates
whether the SCSI status byte is present. You have to just look to
see whether it is non-zero.
The code was looking to see whether the sense length was valid
before propagating the SCSI status byte (and sense information) up
the stack. With a status like Reservation Conflict, there is no
sense information, only the SCSI status byte. So it wasn't getting
correctly returned.
isp.c:
In isp_intr(), if we are on a 2400 or 2500 type board and
get a response, look at the actual contents of the
SCSI status value and set the RQSF_GOT_STATUS flag
accordingly so that return any SCSI status value we get. The
RQSF_GOT_SENSE flag will get set later on if there is
actual sense information returned.
Submitted by: ken
MFC after: 1 week
Sponsored by: Spectra Logic
MFSpectraBSD: 1112791 on 2015/01/15
- Process ATIO queue only if interrupt status tells so;
- Do not update queue out pointers after each processed command, do it
only once at the end of the loop.
- Remove two excessive and slow register reads from isp_intr(). Instead
of rereading value every time, assume that registers contain what we have
written there.
- Avoid sequential search through 4096 array elements when looking for
command tag. Use hash of lists to store active tags separately from free
ones and so greatly speedup the searches.
Reviewed by: mjacob
a mailbox command and which registers to copy back in when
the command completes, the bits being set need to not only
specify what bits you want to add from the default from the
table but also what bits you want *subtract* (mask) from the
default from the table.
A failing ISP2200 command pointed this out.
Much appreciation to: marius, who persisted and narrowed down what
the failure delta was, and shamed me into actually fixing it.
MFC after: 1 week
might have been enabled for them- now that we use all 32 bits of handle.
Fast Posting doesn't pass the full 32 bits.
Noticed by: Bugs in NetBSD. Only a NetBSD user might actually still use such old hardware.
MFC after: 1 week
MISC CHANGES
Add a new async event- ISP_TARGET_NOTIFY_ACK, that will guarantee
eventual delivery of a NOTIFY ACK. This is tons better than just
ignoring the return from isp_notify_ack and hoping for the best.
Clean up the lower level lun enable code to be a bit more sensible.
Fix a botch in isp_endcmd which was messing up the sense data.
Fix notify ack for SRR to use a sensible error code in the case
of a reject.
Clean up and make clear what kind of firmware we've loaded and
what capabilities it has.
-----------
FULL (252 byte) SENSE DATA
In CTIOs for the ISP, there's only a limimted amount of space
to load SENSE DATA for associated CHECK CONDITIONS (24 or 26
bytes). This makes it difficult to send full SENSE DATA that can
be up to 252 bytes.
Implement MODE 2 responses which have us build the FCP Response
in system memory which the ISP will put onto the wire directly.
On the initiator side, the same problem occurs in that a command
status response only has a limited amount of space for SENSE DATA.
This data is supplemented by status continuation responses that
the ISP pushes onto the response queue after the status response.
We now pull them all together so that full sense data can be
returned to the periph driver.
This is supported on 23XX, 24XX and 25XX cards.
This is also preparation for doing >16 byte CDBs.
-----------
FC TAPE
Implement full FC-TAPE on both initiator and target mode side. This
capability is driven by firmware loaded, board type, board NVRAM
settings, or hint configuration options to enable or disable. This
is supported for 23XX, 24XX and 25XX cards.
On the initiator side, we pretty much just have to generate a command
reference number for each command we send out. This is FCP-4 compliant
in that we do this per ITL nexus to generate the allowed 1 thru 255
CRN.
In order to support the target side of FC-TAPE, we now pay attention
to more of the PRLI word 3 parameters which will tell us whether
an initiator wants confirmed responses. While we're at it, we'll
pay attention to the initiator view too and report it.
On sending back CTIOs, we will notice whether the initiator wants
confirmed responses and we'll set up flags to do so.
If a response or data frame is lost the initiator sends us an SRR
(Sequence Retransmit Request) ELS which shows up as an SRR notify
and all outstanding CTIOs are nuked with SRR Received status. The
SRR notify contains the offset that the initiator wants us to restart
the data transfer from or to retransmit the response frame.
If the ISP driver still has the CCB around for which the data segment
or response applies, it will retransmit.
However, we typically don't know about a lost data frame until we
send the FCP Response and the initiator totes up counters for data
moved and notices missing segments. In this case we've already
completed the data CCBs already and sent themn back up to the periph
driver. Because there's no really clean mechanism yet in CAM to
handle this, a hack has been put into place to complete the CTIO
CCB with the CAM_MESSAGE_RECV status which will have a MODIFY DATA
POINTER extended message in it. The internal ISP target groks this
and ctl(8) will be modified to deal with this as well.
At any rate, the data is retransmitted and an an FCP response is
sent. The whole point here is to successfully complete a command
so that you don't have to depend on ULP (SCSI) to have to recover,
which in the case of tape is not really possible (hence the name
FC-TAPE).
Sponsored by: Spectralogic
MFC after: 1 month
not by some hint setting. Do more preparations for FC-Tape.
Clean up resource counting for 24XX or later chipsets so
we find out after EXEC_FIRMWARE what is actually supported.
Set target mode exchange count based upon whether or not
we are supporting simultaneous target/initiator mode. Clean
up some old (pre-24XX) xfwoption and zfwoption issues.
Sponsored by: Spectralogic
MFC after: 3 days
and crosschecks against firmware documentation. We now check and report
FC firmware attributes and at least are now prepared for the upper 48 bits
of f/w attributes (which are probably for the 8100 or later cards). This
involed changing how inbits and outbits are calculated for varios commands,
hopefully clearer and cleaner. This also caused me to clean up the actual
mailbox register usage. Finally, we are now unconditionally using a CRN
for initiator mode.
A longstanding issue with the 2400/2500 is that they do *not* support
a "Prefer PTP followed by loop", which explains why enabling that
caused the f/w to crash.
A slightly more invasive change is to let the firmware load entirely
drive whether multi_id support is enabled or not.
Sponsored by: Spectralogic
MFC after: 1 week
Make the default role NONE if target mode is selected. This
allows ctl(8) to switch to/from target mode via knob settings.
If we default to role 'none', this causes a reset of the
24XX f/w which then causes initiators to wake up and notice
when we come online.
Reviewed by: kdm
MFC after: 2 weeks
Sponsored by: Spectralogic
is actually broken, or needs a BIOS upgrade for 64 bit loads, but this uncovered
a couple of misplaced opcode definitions and some missing continual mbox command
cases, so might as well update them here.
We also revive loop down freezes. We also externaliz within isp
isp_prt_endcmd so something outside the core module can print
something about a command completing. Also some work in progress to
assist in handling timed out commands better.
Partially Sponsored by: Panasas
Approved by: re (kib)
MFC after: 1 month
- Allocate coherent DMA memory for the request/response queue area and
and the FC scratch area.
These changes allow isp(4) to work properly on sparc64 with usage of the
IOMMU streaming buffers enabled.
Approved by: mjacob
MFC after: 2 weeks
on debug output. Add a new platform function requirement to allow
for printing based upon the ITL nexus instead of the isp unit plus
channel, target and lun. This allows some printouts and error messages
from the core code to appear in the same format as the platform's
subsystem (in FreeBSD's case, CAM path).
MFC after: 1 week
32 bit handles. The RIO (reduced interrupt operation) and fast posting
for the parallel SCSI cards were all 16 bit handles. Furthermore,
target mode parallel SCSI only can have 16 bit handles.
Use part of a supplied patch to switch over to using 32 bit handles.
Be a bit more conservative here and only do this for parallel SCSI
for the 12160 (Ultra3) cards. There were a lot of marginal Ultra2
cards, and, frankly, few are findable now for testing.
Fix the target handle routine to only do 16 bit handles for parallel
SCSI cards. This is okay because the upper sixteen bits of the new
32 bit handles is a sequence number to help protect against duplicate
completions. This would be very unlikely to happen with parallel
SCSI target mode, and wasn't present before, so we're no worse off
than we used to be.
While we're at it, finally split the async mailbox completion handlers
into FC and parallel SCSI functions. This makes it much cleaner and
easier to figure out what is or isn't a legal async mailbox completion
code for different card classes.
PR: kern/144250
Submitted partially by: Charles D
MFC after: 1 week
numbers and handle types in rational way. This will better protect from
(unwittingly) dealing with stale handles/commands.
Fix the watchdog timeout code to better protect itself from mistakes.
If we run an abort on a putatively timed out command, the command
may in fact get completed, so check to make sure the command we're
timing it out is still around. If the abort succeeds, btw, the command
should get returned via a different path.
If the NAA is type 2, the Node WWN is the Port WWN with the 12 bits
of port (48..60) cleared. This iff a wwn fetched from NVRAM is zero.
MFC after: 1 week
Add a maximum response length for FCP RSPNS IUs.
Clarify some of the FC option words for setting parameters
and try and disable automatic PRLI when in target mode- this
should correct some cases of N-port topologies with 23XX cards
where we put out an illegal PRLI (in target mode only we're
not supposed to put out a PRLI).
