is actually broken, or needs a BIOS upgrade for 64 bit loads, but this uncovered
a couple of misplaced opcode definitions and some missing continual mbox command
cases, so might as well update them here.
We also revive loop down freezes. We also externaliz within isp
isp_prt_endcmd so something outside the core module can print
something about a command completing. Also some work in progress to
assist in handling timed out commands better.
Partially Sponsored by: Panasas
Approved by: re (kib)
MFC after: 1 month
- Allocate coherent DMA memory for the request/response queue area and
and the FC scratch area.
These changes allow isp(4) to work properly on sparc64 with usage of the
IOMMU streaming buffers enabled.
Approved by: mjacob
MFC after: 2 weeks
Untangle some of the confusion about what role means when it's in the FCPARAM/SDPARAM
or isp_fc/isp_spi structures. This fixed a problem about seeing targets appear if you've
turned off autologin and find them, or rather don't, via camcontrol rescan.
MFC after: 1 month
32 bit handles. The RIO (reduced interrupt operation) and fast posting
for the parallel SCSI cards were all 16 bit handles. Furthermore,
target mode parallel SCSI only can have 16 bit handles.
Use part of a supplied patch to switch over to using 32 bit handles.
Be a bit more conservative here and only do this for parallel SCSI
for the 12160 (Ultra3) cards. There were a lot of marginal Ultra2
cards, and, frankly, few are findable now for testing.
Fix the target handle routine to only do 16 bit handles for parallel
SCSI cards. This is okay because the upper sixteen bits of the new
32 bit handles is a sequence number to help protect against duplicate
completions. This would be very unlikely to happen with parallel
SCSI target mode, and wasn't present before, so we're no worse off
than we used to be.
While we're at it, finally split the async mailbox completion handlers
into FC and parallel SCSI functions. This makes it much cleaner and
easier to figure out what is or isn't a legal async mailbox completion
code for different card classes.
PR: kern/144250
Submitted partially by: Charles D
MFC after: 1 week
numbers and handle types in rational way. This will better protect from
(unwittingly) dealing with stale handles/commands.
Fix the watchdog timeout code to better protect itself from mistakes.
If we run an abort on a putatively timed out command, the command
may in fact get completed, so check to make sure the command we're
timing it out is still around. If the abort succeeds, btw, the command
should get returned via a different path.
firmware loading bugs.
Target mode support has received some serious attention to make it
more usable and stable.
Some backward compatible additions to CAM have been made that make
target mode async events easier to deal with have also been put
into place.
Further refinement and better support for NP-IV (N-port Virtualization)
is now in place.
Code for release prior to RELENG_7 has been stripped away for code clarity.
Sponsored by: Copan Systems
Reviewed by: scottl, ken, jung-uk kim
Approved by: re
put out a ispreqt2e_t structure onto the request queue- not a ispreqt2_t
structure. I forgot that the 23XX can use a t2 structure.
Approved by: re (ken, implicitly)
MFC after: 3 days
to put out a ispreqt3e_t structure onto the request queue-
not a ispreqt3_t structure. We weren't. This turns out only
to really matter for big endian machines.
Approved by: re (ken)
MFC after: 3 days
Seems to work on RELENG_4 through -current and also on sparc64
now. There may still be some issues with the auto attach/detach
code to sort out.
MFC after: 3 days
When the linux port changes were imported which split the
target command list to be separate from the initiator command
list and the handle format changed to encode a type in the handle
the implications to the function isp_handle_index (which only
the NetBSD/OpenBSD/FreeBSD ports use) were overlooked.
The fault is twofold: first, the index into the DMA maps
in isp_pci is wrong because a target command handle with
the type bit left in place caused a bad index (and panic)
into dma map. Secondly, the assumption of the array
of DMA maps in either PCS or SBUS attachment structures is
that there is a linear mapping between handle index and
DMA map index. This can no longer be true if there are
overlapping index spaces for initiator mode and target
mode commands.
These changes bandaid around the problem by forcing us
to not have simultaneous dual roles and doing the appropriate
masking to make sure things are indexed correctly. A longer
term fix is being devloped.
tokens into the common isp_osinfo structure instead of being
in bus specific structures. This allows us to implement
a SYNC_REG MEMORYBARRIER call (using bus_space_barrier)
and also reduce the amount of bus specific wrapper structure
usages in isp_pci && isp_sbus.
MFC after: 3 days
early, we haven't set board type, so we can't correctly check for
some options. Fix this by splitting option setting/getting into
generic, pci and then later board specific, option setting/getting.
This was noticed when setting 'iid' (or 'hard loop id') didn't work
all of a sudden.
Noticed by: Mike Drangula (thanks!) via Jung-uk Kim (thanks!)
Either they're there early and the ispfw sets have
registered themselves, or they're not.
The module dependency stuff isn't quite what we want
anyway. If the user doesn't want the load placed on
system memory by loading the firmware, they don't
specify it to be loaded (either by being linked in
or via being a module to be loaded and then hooked
in with firmware(9)). It doesn't then make sense to
then override what they want by pulling it in anyway.
This might be able to work if we were able to pull in
just exactly what we needed for the card we have- but
that's an optimization left for the future.
and provied an isp_control entry point so that the outer layers can
do PLOGI/LOGO explicitly. Add MS IOCB support. This completes the cycle
for base support for SMI-S.
gone device timers and zombie state entries. There are tunables
that can be used to select a number of parameters.
loop_down_limit - how long to wait for loop to come back up before
declaring
all devices dead (default 300 seconds)
gone_device_time- how long to wait for a device that has appeared
to leave the loop or fabric to reappear (default 30 seconds)
Internal tunables include (which should be externalized):
quick_boot_time- how long to wait when booting for loop to come up
change_is_bad- whether or not to accept devices with the same
WWNN/WWPN that reappear at a different PortID as being the 'same'
device.
Keen students of some of the subtle issues here will ask how
one can keep devices from being re-accepted at all (the answer
is to set a gone_device_time to zero- that effectively would
be the same thing).
required by arches like sparc64 (not yet implemented) and sun4v where there
are seperate IOMMU's for each PCI bus... For all other arches, it will
end up returning NULL, which makes it a no-op...
Convert a few drivers (the ones we've been working w/ on sun4v) to the
new convection... Eventually all drivers will need to replace the parent
tag of NULL, w/ bus_get_dma_tag(dev), though dev is usually different for
each driver, and will require hand inspection...
Reviewed by: scottl (earlier version)
it ended up defaulting to ISP_ROLE_NONE. My testing hadn't caught it
because I was deliberatly setting role via ioctl.
Thanks to user Toni for lending me an alpha to test this on.
MFC after: 0 days
FreeBSD repository and to clean up the license header so as to
not pollute the license with file function.
Zero all mailbox structures prior to use (just in case). Change
the outgoing mailbox count for INIT_FIRMWARE to be correct.
loading for the QLogic cards.
Because isp(4) exists before the root is mounted, it's not really
possible for us to use the kernel's linker to load modules directly
from disk- that's really too bad.
However, the this is still a net win in in that the firmware has
been split up on a per chip (and in some cases, functionality)
basis, so the amount of stuff loaded *can* be substantially less
than the 1.5MB of firmware images that ispfw now manages. That is,
each specific f/w set is now also built as a module. For example,
QLogic 2322 f/w is built as isp_2322.ko and Initiator/Target 1080
firmware is built as isp_1080_it.ko.
For compatibility purposes (i.e., to perturb folks the least), we
also still build all of the firmware as one ispfw.ko module.
This allows us to let 'ispfw_LOAD' keep on working in existing
loader.conf files. If you now want to strip this down to just
the firmware for your h/w, you can then change loader.conf to
load the f/w you specifically want.
We also still allow for ispfw to be statically built (e.g., for
PAE and sparc64).
Future changes will look at f/w unloading and also role switching
that then uses the kernel linker to load different ips f/w sets.
MFC after: 2 months
(and by extension, the 2422).
One peculiar thing I've found with the 2322 is that if you
don't force it to do Hard LoopID acquisition, the firmware
crashes. This took a while to figure out.
While we're at it, fix various bugs having to do with NVRAM
reading and option setting with respect to pieces of NVRAM.
Be cognizant as to whether we're running 2KLogin f/w in target mode and
do the appropriate loopid load based upon that.
Do a first cut (seems to work, at least for amd64) at 64 bit target
mode for fibre channel cards. We could probably also do it for SPI
cards, but that's not supported right now.
Pull in some target mode changes from a private branch.
Pull in some more RELENG_4 compilation changes.
A lot of lines changed, but not much content change yet.
to getting rid u_int for uint and so on).
b) Turn back on 64 bit DAC support. Cheeze it a bit in that we have two
DMA callback functions- one when we have bus_addr_t > 4 bits in width and
the other which should be normal. Even Cheezier in that we turn off setting
up DMA maps to be BUS_SPACE_MAXADDR if we're in ISP_TARGET_MODE. More work
on this in a week or so.
c) Tested under amd64 and 1MB DFLTPHYS, sparc64, i386 (PAE, but insufficient
memory to really test > 4GB). LINT check under amd64.
MFC after: 1 month
are bus_addr_t, not bus_size_t.
In any case, turn off DAC support entirely until it is revamped to actually
work *correctly* for 64 bit platforms (not using a PAE definition and for
both initiator and target mode).
up to date. Principle changes for this reelase is to support 2K Port Login
firmware. This allows us to support the 2322 (and 2422 4Gb) cards which only
come with the 2K Port Login firmware. The 2322 should now work- but we don't
have firmware sets for it in ispfw (as the change to load 2K Port Login f/w
hasn't been made- that f/w is so big it has to be loaded in more than one
chunk).
Other changes are the beginnings of cleaning up some long standing target
mode issues. The next changes here will incorporate a lot of bug fixes
from others.
Finally, some copyright cleanup and attempts to make the parts of the
driver that are FreeBSD specific start conforming more to FreeBSD style.
MFC after: 1 month
I was unable to test this as the PAE kernel crashed with a "cannot copy
LDT" before coming up. When this gets a bit more testing, I'll fix the PAE
conf file to allow isp devices.
PR: 59728
Add two new arguments to bus_dma_tag_create(): lockfunc and lockfuncarg.
Lockfunc allows a driver to provide a function for managing its locking
semantics while using busdma. At the moment, this is used for the
asynchronous busdma_swi and callback mechanism. Two lockfunc implementations
are provided: busdma_lock_mutex() performs standard mutex operations on the
mutex that is specified from lockfuncarg. dftl_lock() is a panic
implementation and is defaulted to when NULL, NULL are passed to
bus_dma_tag_create(). The only time that NULL, NULL should ever be used is
when the driver ensures that bus_dmamap_load() will not be deferred.
Drivers that do not provide their own locking can pass
busdma_lock_mutex,&Giant args in order to preserve the former behaviour.
sparc64 and powerpc do not provide real busdma_swi functions, so this is
largely a noop on those platforms. The busdma_swi on is64 is not properly
locked yet, so warnings will be emitted on this platform when busdma
callback deferrals happen.
If anyone gets panics or warnings from dflt_lock() being called, please
let me know right away.
Reviewed by: tmm, gibbs
defined, we set the address space limitation to BUS_SPACE_UNRESTRICTED,
otherwise to BUS_SPACE_MAXADDR_32BIT.
If we have a 1240, ULTRA2 or better, or an FC card, the boundary limit
is BUS_SPACE_UNRESTRICTED and segment limit is BUS_SPACE_MAXADDR_32BIT.
The older 1020/1040 cards have boundary and segment limits of
BUS_SPACE_MAXADDR_24BIT.
fast posting command completion, and fast post CTIO completion,
the upper half of Risc2Host is a copy of mailbox #1- *not*
mailbox #0.
MFC after: 1 day
flags include INTR_MPSAFE. Put the flags in a common place so that
both isp_sbus && isp_pci DTRT.
In isp_mbxdma setup, drop any locks prior to calling things like
bus_dmatag_create. This gets rid of these obnoxious WITNESS messages
about 'sleeping with locks held' blah blah blah blah blah.
This code does not imply that SBus cards work yet. They hang for me.
But I can't netboot the latest snapshot on my ultra1e, and things
hang at bus_setup_intr time.
Since I'm offline for a while, I thought I'd toss this in in case somebody
else who has a bit better luck wants to fart around with it. Please try
and wait until I get back to check things in.
running ABOUT FIRMWARE with some that were started by BIOS downloads).
Redo CTIO2 dma mapping- use continuation segments instead of multiple
CTIO2s. Thanks to Veritas for sponsoring this work (in a different
context).
MFC after: 1 week
Grumble. I've seen better documented architectures out of Redmond.
Redo fabric evaluation to not use GET ALL NEXT (GA_NXT). Switches seem
to be trying to wriggle out of supporting this well. Instead, use
GID_FT to get a list of Port IDs and then use GPN_ID/GNN_ID to find the
port and node wwn. This should make working on fabrics a bit cleaner and
more stable.
This also caused some cleanup of SNS subcommand canonicalization so that
we can actually check for FS_ACC and FS_RJT, and if we get an FS_RJT,
print out the reason and explanation codes.
We'll keep the old GA_NXT method around if people want to uncomment a
controlling definition in ispvar.h.
This also had us clean up ISPASYNC_FABRICDEV to use a local lportdb argument
and to have the caller explicitly say that a device is at the end of the
fabric list.
MFC after: 1 week
most cases NULL is passed, but in some cases such as network driver locks
(which use the MTX_NETWORK_LOCK macro) and UMA zone locks, a name is used.
Tested on: i386, alpha, sparc64
stuff was right, but the busdma stuff was massively not right.
Didn't really test on ia64 or i386- don't have the former h/w and my
FreeBSD-current disk is unwell right now. Hope that this is okay.
MFC after: 1 week
disable MWI on 2300
based on function code, set an 'isp_port' for the 2312- it's a
separate instance, but the NVRAM is shared, and the second port's
NVRAM is at offset 256.
+ Enable RIO operation for LVD SCSI cards. This makes a *big* difference
as even under reasonable load we get batched completions of about 30
commands at a time on, say, an ISP1080.
+ Do 'continuation' mailbox commands- this allows us to specify a work
area within the softc and 'continue' repeated mailbox commands. This is
more or less on an ad hoc basis and is currently only used for firmware
loading (which f/w now loads substantially faster becuase the calling
thread is only woken when all the f/w words are loaded- not for each
one of the 40000 f/w words that gets loaded).
+ If we're about to return from isp_intr with a 'bogus interrupt' indication,
and we're not a 23XX card, check to see whether the semaphore register is
currently *2* (not *1* as it should be) and whether there's an async completion
sitting in outgoing mailbox0. This seems to capture cases of lost fast posting
and RIO interrupts that the 12160 && 1080 have been known to pump out under
extreme load (extreme, as in > 250 active commands).
+ FC_SCRATCH_ACQUIRE/FC_SCRATCH_RELEASE macros.
+ Endian correct swizzle/unswizzle of an ATIO2 that has a WWPN in it.
MFC after: 1 week
the response queue. Instead of the ad hoc ISP_SWIZZLE_REQUEST, we now have
a complete set of inline functions in isp_inline.h. Each platform is
responsible for providing just one of a set of ISP_IOX_{GET,PUT}{8,16,32}
macros.
The reason this needs to be done is that we need to have a single set of
functions that will work correctly on multiple architectures for both little
and big endian machines. It also needs to work correctly in the case that
we have the request or response queues in memory that has to be treated
specially (e.g., have ddi_dma_sync called on it for Solaris after we update
it or before we read from it). It also has to handle the SBus cards (for
platforms that have them) which, while on a Big Endian machine, do *not*
require *most* of the request/response queue entry fields to be swizzled
or unswizzled.
One thing that falls out of this is that we no longer build requests in the
request queue itself. Instead, we build the request locally (e.g., on the
stack) and then as part of the swizzling operation, copy it to the request
queue entry we've allocated. I thought long and hard about whether this was
too expensive a change to make as it in a lot of cases requires an extra
copy. On balance, the flexbility is worth it. With any luck, the entry that
we build locally stays in a processor writeback cache (after all, it's only
64 bytes) so that the cost of actually flushing it to the memory area that is
the shared queue with the PCI device is not all that expensive. We may examine
this again and try to get clever in the future to try and avoid copies.
Another change that falls out of this is that MEMORYBARRIER should be taken
a lot more seriously. The macro ISP_ADD_REQUEST does a MEMORYBARRIER on the
entry being added. But there had been many other places this had been missing.
It's now very important that it be done.
Additional changes:
Fix a longstanding buglet of sorts. When we get an entry via isp_getrqentry,
the iptr value that gets returned is the value we intend to eventually plug
into the ISP registers as the entry *one past* the last one we've written-
*not* the current entry we're updating. All along we've been calling sync
functions on the wrong index value. Argh. The 'fix' here is to rename all
'iptr' variables as 'nxti' to remember that this is the 'next' pointer-
not the current pointer.
Devote a single bit to mboxbsy- and set aside bits for output mbox registers
that we need to pick up- we can have at least one command which does not
have any defined output registers (MBOX_EXECUTE_FIRMWARE).
MFC after: 2 weeks
to see if there's an interrupt (avoids PCI parity errors
which can occur on the 2312 if you access some registers
from the host at the same time the RISC on the 2312 is
C accessing them).
MFC after: 1 day
and cv_wait for mailbox commands to complete if we start them from
here.
Fix residuals for target mode such that we only check the residual and
set it in the CTIO if this is the last CTIO (when we're sending status).
MFC after: 4 weeks
some reworking (and consequent cleanup) of the interrupt service code.
Also begin to start a cleanup of target mode support that will (eventually)
not require more inforamtion routed with the ATIO to come back with the
CTIO other than tag.
MFC after: 4 weeks
554: passing arg 4 of `resource_string_value' from incompatible pointer type
576: passing arg 4 of `resource_string_value' from incompatible pointer type
593: passing arg 4 of `resource_string_value' from incompatible pointer type
----
Make a device for each ISP- really usable only with devfs and add an ioctl
entry point (this can be used to (re)set debug levels, reset the HBA,
rescan the fabric, issue lips, etc).
----
Add in a kernel thread for Fibre Channel cards. The purpose of this
thread is to be woken up to clean up after Fibre Channel events
block things. Basically, any FC event that casts doubt on the
location or identify of FC devices blocks the queues. When, and
if, we get the PORT DATABASE CHANGED or NAME SERVER DATABASE CHANGED
async event, we activate the kthread which will then, in full thread
context, re-evaluate the local loop and/or the fabric. When it's
satisfied that things are stable, it can then release the blocked
queues and let commands flow again.
The prior mechanism was a lazy evaluation. That is, the next command
to come down the pipe after change events would pay the full price
for re-evaluation. And if this was done off of a softcall, it really
could hang up the system.
These changes brings the FreeBSD port more in line with the Solaris,
Linux and NetBSD ports. It also, more importantly, gets us being
more proactive about topology changes which could then be reflected
upwards to CAM so that the periph driver can be informed sooner
rather than later when things arrive or depart.
---
Add in the (correct) usage of locking macros- we now have lock transition
macros which allow us to transition from holding the CAM lock (Giant)
and grabbing the softc lock and vice versa. Switch over to having this
HBA do real locking. Some folks claim this won't be a win. They're right.
But you have to start somewhere, and this will begin to teach us how
to DTRT for HBAs, etc.
--
Start putting in prototype 2300 support. Add back in LIP
and Loop Reset as async events that each platform will handle.
Add in another int_bogus instrumentation point.
Do some more substantial target mode cleanups.
MFC after: 8 weeks
overflow the request queue. The reason we want to do this is that we
now push out completed CTIOs as we complete them- this gets the QLogic
working on them quicker. So we need to know whether we can put the entire
burrito out before we start.
We now support conjoint status with data for the last CTIO for both Fibre
Channel and SCSI. Leave the old code in place in case we need to go back
(minor 3 line ifdef).
Ultra-ultra important- *don't* set rq->req_seg_count for non-data
target mode requests in isp_pci_dmasetup. D'oh- this is actually
the tag value area for a CTIO. What *was* I thinking? Boy howdy
does both aic7xxx and sym get awfully unhappy when on reconnect
you give them a constant '1' for a tag value.
I was hanging after sending a xfer CTIO and a status CTIO for a non-discon
INQUIRY- the xfer CTIO was returned as completed OK, but the status CTIO
was dropped on the floor. All the fields looked good. I don't know why
it got dropped. But allowing status to go back with data xfer seemed to
work. I also noticed that with a non-disconnecting command that the
firmware handle in the ATIO is zero- this leads me to believe that the
f/w really can only handle one CTIO at a time in the discon case, and
it had no idea what to do with the second (status) CTIO.
This is a pretty invasive change, but there are three good
reasons to do this:
1. We'll never have > 16 bits of handle.
2. We can (eventually) enable the RIO (Reduced Interrupt Operation)
bits which return multiple completing 16 bit handles in mailbox
registers.
3. The !)$*)$*~)@$*~)$* Qlogic target mode for parallel SCSI spec
changed such that at_reserved (which was 32 bits) was split into
two pieces- and one of which was a 16 bit handle id that functions
like the at_rxid for Fibre Channel (a tag for the f/w to correlate
CTIOs with a particular command). Since we had to muck with that
and this changed the whole handler architecture, we might as well...
Propagate new at_handle on through int ct_fwhandle. Follow
implications of changing to 16 bit handles.
These above changes at least get Qlogic 1040 cards working in target
mode again. 1080/12160 cards don't work yet.
In isp.c:
Prepare for doing all loop management in outer layers.
for selecting unit). Instead, use the resource hints mechanism.
One unfortunate situation here is that there is no resource_quad_value
function- which is what I needed for WWN boot time replacement. Worse-
you can't store the hint as just plain
hint.isp.0.nodewwn="0x50000000aaaa0001"
because this gets interpreted as an int- incorrectly because it can't
be converted to an int. I can't even get this as a string. To work
around this particular case for nodewwn && portwwn setting, this
rather grotesque form will be used:
hint.isp.0.nodewwn="w50000000aaaa0001"
hint.isp.0.portwwn="w50000000aaaa0002"
At the same time, if we have no hinted WWN, set the default WWN (which, btw,
gets overridden if the card has valid NVRAM, which is usual) to
0x400000007F000009ull (which translates to NAA == IPv4, 127.0.0.9).
Eliminate more printf's and replace them either with device_printf or
isp_prt calls.
the unit number doesn't get reused.
Make sure that if we've compiled for ISP_TARGET_MODE we set the
default role to be ISP_ROLE_INITIATOR|ISP_ROLE_TARGET.
Do some misc other cleanups.
