upstream/opnsense-src
1
0
Fork 0
mirror of https://github.com/opnsense/src.git synced 2026-05-28 04:12:45 -04:00
Code Issues Projects Releases Packages Wiki Activity Actions

o Add support for eMMC HS200 and HS400 bus speed modes at 200 MHz to

Browse source 
sdhci(4), mmc(4) and mmcsd(4). For the most part, this consists of:
  - Correcting and extending the infrastructure for negotiating and
    enabling post-DDR52 modes already added as part of r315598. In
    fact, HS400ES now should work as well but hasn't been activated
    due to lack of corresponding hardware.
  - Adding support executing standard SDHCI initial tuning as well
    as re-tuning as required for eMMC HS200/HS400 and the fast UHS-I
    SD card modes. Currently, corresponding methods are only hooked
    up to the ACPI and PCI front-ends of sdhci(4), though. Moreover,
    sdhci(4) won't offer any modes requiring (re-)tuning to the MMC/SD
    layer in order to not break operations with other sdhci(4) front-
    ends. Likewise, sdhci(4) now no longer offers modes requiring the
    set_uhs_timing method introduced in r315598 to be implemented/
    hooked up (previously, this method was used with DDR52 only, which
    in turn is only available with Intel controllers so far, i. e. no
    such limitation was necessary before). Similarly for 1.2/1.8 V VCCQ
    support and the switch_vccq method.
  - Addition of locking to the IOCTL half of mmcsd(4) to prevent races
    with detachment and suspension, especially since it's required to
    immediately switch away from RPMB partitions again after an access
    to these (so re-tuning can take place anew, given that the current
    eMMC specification v5.1 doesn't allow tuning commands to be issued
    with a RPMB partition selected). Therefore, the existing part_mtx
    lock in the mmcsd(4) softc is additionally renamed to disk_mtx in
    order to denote that it only refers to the disk(9) half, likewise
    for corresponding macros.

  On the system where the addition of DDR52 support increased the read
  throughput to ~80 MB/s (from ~45 MB/s at high speed), HS200 yields
  ~154 MB/s and HS400 ~187 MB/s, i. e. performance now has more than
  quadrupled compared to pre-r315598.

  Also, with the advent of (re-)tuning support, most infrastructure
  necessary for SD card UHS-I modes up to SDR104 now is also in place.
  Note, though, that the standard SDHCI way of (re-)tuning is special
  in several ways, which also is why sending the actual tuning requests
  to the device is part of sdhci(4). SDHCI implementations not following
  the specification, MMC and non-SDHCI SD card controllers likely will
  use a generic implementation in the MMC/SD layer for executing tuning,
  which hasn't been written so far, though.

  However, in fact this isn't a feature-only change; there are boards
  based on Intel Bay Trail where DDR52 is problematic and the suggested
  workaround is to use HS200 mode instead. So far exact details are
  unknown, however, i. e. whether that's due to a defect in these SoCs
  or on the boards.

  Moreover, due to the above changes requiring to be aware of possible
  MMC siblings in the fast path of mmc(4), corresponding information
  now is cached in mmc_softc. As a side-effect, mmc_calculate_clock(),
  mmc_delete_cards(), mmc_discover_cards() and mmc_rescan_cards() now
  all are guaranteed to operate on the same set of devices as there no
  longer is any use of device_get_children(9), which can fail in low
  memory situations. Likewise, mmc_calculate_clock() now longer will
  trigger a panic due to the latter.

o Fix a bug in the failure reporting of mmcsd_delete(); in case of an
  error when the starting block of a previously stored erase request
  is used (in order to be able to erase a full erase sector worth of
  data), the starting block of the newly supplied bio_pblkno has to be
  returned for indicating no progress. Otherwise, upper layers might
  be told that a negative number of BIOs have been completed, leading
  to a panic.

o Fix 2 bugs on resume:
  - Things done in fork1(9) like the acquisition of an SX lock or the
    sleepable memory allocation are incompatible with a MTX_DEF taken.
    Thus, mmcsd_resume() must not call kproc_create(9), which in turn
    uses fork1(9), with the disk_mtx (formerly part_mtx) held.
  - In mmc_suspend(), the bus is powered down, which in the typical
    case of a device being selected at the time of suspension, causes
    the device deselection as part of the bus acquisition by mmc(4) in
    mmc_scan() to fail as the bus isn't powered up again before later
    in mmc_go_discovery(). Thus, power down with the bus acquired in
    mmc_suspend(), which will trigger the deselection up-front.

o Fix a memory leak in mmcsd_ioctl() in case copyin(9) fails. [1]

o Fix missing variable initialization in mmc_switch_status(). [2]

o Fix R1_SWITCH_ERROR detection in mmc_switch_status(). [3]

o Handle the case of device_add_child(9) failing, for example due to
  a memory shortage, gracefully in mmc(4) and sdhci(4), including not
  leaking memory for the instance variables in case of mmc(4) (which
  might or might not fix [4] as the latter problem has been discovered
  independently).

o Handle the case of an unknown SD CSD version in mmc_decode_csd_sd()
  gracefully instead of calling panic(9).

o Again, check and handle the return values of some additional function
  calls in mmc(4) instead of assuming that everything went right or mark
  non-fatal errors by casting the return value to void.

o Correct a typo in the Linux IOCTL compatibility; it should have been
  MMC_IOC_MULTI_CMD rather than MMC_IOC_CMD_MULTI.

o Now that we are reaching ever faster speeds (more improvement in this
  regard is to be expected when adding ADMA support to sdhci(4)), apply
  a few micro-optimizations like predicting mmc(4) and sdhci(4) debugging
  to be off or caching erase sector and maximum data sizes as well support
  of block addressing in mmsd(4) (instead of doing 2 indirections on every
  read/write request for determining the maximum data size for example).

Reported by:	Coverity
CID:		1372612 [1], 1372624 [2], 1372594 [3], 1007069 [4]
This commit is contained in:
Marius Strobl 2017-07-23 16:11:47 +00:00
parent d0d51a3322
commit aca38eab8a
15 changed files with 1471 additions and 338 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

6
sys/dev/mmc/bridge.h
View file

@ -137,6 +137,10 @@ enum mmc_card_mode {
mode_mmc, mode_sd
};
enum mmc_retune_req {
retune_req_none = 0, retune_req_normal, retune_req_reset
};
struct mmc_host {
int f_min;
int f_max;
@ -178,7 +182,7 @@ struct mmc_host {
extern driver_t mmc_driver;
extern devclass_t mmc_devclass;
#define MMC_VERSION 3
#define MMC_VERSION 4
#define MMC_DECLARE_BRIDGE(name) \
DRIVER_MODULE(mmc, name, mmc_driver, mmc_devclass, NULL, NULL); \

854
sys/dev/mmc/mmc.c
View file

File diff suppressed because it is too large Load diff

2
sys/dev/mmc/mmc_ioctl.h
View file

@ -54,7 +54,7 @@ struct mmc_ioc_multi_cmd {
#define MMC_IOC_BASE 'M'
#define MMC_IOC_CMD _IOWR(MMC_IOC_BASE, 0, struct mmc_ioc_cmd)
#define MMC_IOC_CMD_MULTI _IOWR(MMC_IOC_BASE, 1, struct mmc_ioc_multi_cmd)
#define MMC_IOC_MULTI_CMD _IOWR(MMC_IOC_BASE, 1, struct mmc_ioc_multi_cmd)
/* Maximum accepted data transfer size */
#define MMC_IOC_MAX_BYTES (512 * 256)

11
sys/dev/mmc/mmc_private.h
View file

@ -60,9 +60,14 @@ struct mmc_softc {
struct mtx sc_mtx;
struct intr_config_hook config_intrhook;
device_t owner;
uint32_t last_rca;
int squelched; /* suppress reporting of (expected) errors */
int log_count;
device_t *child_list;
int child_count;
uint16_t last_rca;
uint16_t retune_paused;
uint8_t retune_needed;
uint8_t retune_ongoing;
uint16_t squelched; /* suppress reporting of (expected) errors */
int log_count;
struct timeval log_time;
};

20
sys/dev/mmc/mmc_subr.c
View file

@ -138,7 +138,6 @@ mmc_wait_for_app_cmd(device_t brdev, device_t reqdev, uint16_t rca,
sc->squelched--;
if (err != MMC_ERR_NONE && brdev == reqdev) {
sc = device_get_softc(brdev);
if (sc->squelched == 0 && ppsratecheck(&sc->log_time,
&sc->log_count, LOG_PPS)) {
device_printf(sc->dev, "ACMD%d failed, RESULT: %d\n",
@ -154,10 +153,13 @@ mmc_switch(device_t brdev, device_t reqdev, uint16_t rca, uint8_t set,
uint8_t index, uint8_t value, u_int timeout, bool status)
{
struct mmc_command cmd;
struct mmc_softc *sc;
int err;
KASSERT(timeout != 0, ("%s: no timeout", __func__));
sc = device_get_softc(brdev);
memset(&cmd, 0, sizeof(cmd));
cmd.opcode = MMC_SWITCH_FUNC;
cmd.arg = (MMC_SWITCH_FUNC_WR << 24) | (index << 16) | (value << 8) |
@ -172,10 +174,19 @@ mmc_switch(device_t brdev, device_t reqdev, uint16_t rca, uint8_t set,
cmd.flags = MMC_RSP_R1 | MMC_CMD_AC;
else
cmd.flags = MMC_RSP_R1B | MMC_CMD_AC;
/*
* Pause re-tuning so it won't interfere with the busy state and also
* so that the result of CMD13 will always refer to switching rather
* than to a tuning command that may have snuck in between.
*/
sc->retune_paused++;
err = mmc_wait_for_cmd(brdev, reqdev, &cmd, CMD_RETRIES);
if (err != MMC_ERR_NONE || status == false)
return (err);
return (mmc_switch_status(brdev, reqdev, rca, timeout));
goto out;
err = mmc_switch_status(brdev, reqdev, rca, timeout);
out:
sc->retune_paused--;
return (err);
}
int
@ -192,6 +203,7 @@ mmc_switch_status(device_t brdev, device_t reqdev, uint16_t rca, u_int timeout)
* type MMC_CAP_WAIT_WHILE_BUSY will issue mmc_send_status() only
* once and then exit the loop.
*/
end.tv_sec = end.tv_usec = 0;
for (;;) {
err = mmc_send_status(brdev, reqdev, rca, &status);
if (err != MMC_ERR_NONE)
@ -208,7 +220,7 @@ mmc_switch_status(device_t brdev, device_t reqdev, uint16_t rca, u_int timeout)
break;
}
}
if (err == MMC_ERR_NONE && R1_CURRENT_STATE(status) == R1_SWITCH_ERROR)
if (err == MMC_ERR_NONE && (status & R1_SWITCH_ERROR) != 0)
return (MMC_ERR_FAILED);
return (err);
}

40
sys/dev/mmc/mmcbr_if.m
View file

@ -54,7 +54,6 @@
#
#include <sys/types.h>
#include <dev/mmc/bridge.h>
#include <dev/mmc/mmcreg.h>
#
@ -74,6 +73,22 @@ CODE {
return (0);
}
static int
null_retune(device_t brdev __unused, device_t reqdev __unused,
bool reset __unused)
{
return (0);
}
static int
null_tune(device_t brdev __unused, device_t reqdev __unused,
bool hs400 __unused)
{
return (0);
}
};
#
@ -94,11 +109,30 @@ METHOD int switch_vccq {
device_t reqdev;
} DEFAULT null_switch_vccq;
#
# Called by the mmcbus with the bridge claimed to execute initial tuning.
#
METHOD int tune {
device_t brdev;
device_t reqdev;
bool hs400;
} DEFAULT null_tune;
#
# Called by the mmcbus with the bridge claimed to execute re-tuning.
#
METHOD int retune {
device_t brdev;
device_t reqdev;
bool reset;
} DEFAULT null_retune;
#
# Called by the mmcbus or its children to schedule a mmc request. These
# requests are queued. Time passes. The bridge then gets notification
# of the status of the request, who then notifies the requesting device
# by calling the completion function supplied as part of the request.
# Requires the bridge to be claimed.
#
METHOD int request {
device_t brdev;
@ -121,7 +155,7 @@ METHOD int get_ro {
METHOD int acquire_host {
device_t brdev;
device_t reqdev;
}
};
#
# Release the current bridge.
@ -129,4 +163,4 @@ METHOD int acquire_host {
METHOD int release_host {
device_t brdev;
device_t reqdev;
}
};

17
sys/dev/mmc/mmcbrvar.h
View file

@ -56,6 +56,7 @@
#define DEV_MMC_MMCBRVAR_H
#include <dev/mmc/mmcreg.h>
#include "mmcbr_if.h"
enum mmcbr_device_ivars {
@ -69,6 +70,7 @@ enum mmcbr_device_ivars {
MMCBR_IVAR_MODE,
MMCBR_IVAR_OCR,
MMCBR_IVAR_POWER_MODE,
MMCBR_IVAR_RETUNE_REQ,
MMCBR_IVAR_VDD,
MMCBR_IVAR_VCCQ,
MMCBR_IVAR_CAPS,
@ -93,6 +95,7 @@ MMCBR_ACCESSOR(host_ocr, HOST_OCR, int)
MMCBR_ACCESSOR(mode, MODE, int)
MMCBR_ACCESSOR(ocr, OCR, int)
MMCBR_ACCESSOR(power_mode, POWER_MODE, int)
MMCBR_ACCESSOR(retune_req, RETUNE_REQ, int)
MMCBR_ACCESSOR(vdd, VDD, int)
MMCBR_ACCESSOR(vccq, VCCQ, int)
MMCBR_ACCESSOR(caps, CAPS, int)
@ -107,6 +110,20 @@ mmcbr_update_ios(device_t dev)
return (MMCBR_UPDATE_IOS(device_get_parent(dev), dev));
}
static int __inline
mmcbr_tune(device_t dev, bool hs400)
{
return (MMCBR_TUNE(device_get_parent(dev), dev, hs400));
}
static int __inline
mmcbr_retune(device_t dev, bool reset)
{
return (MMCBR_RETUNE(device_get_parent(dev), dev, reset));
}
static int __inline
mmcbr_switch_vccq(device_t dev)
{

36
sys/dev/mmc/mmcbus_if.m
View file

@ -53,18 +53,36 @@
# $FreeBSD$
#
#include <sys/types.h>
#include <dev/mmc/mmcreg.h>
#include <dev/mmc/bridge.h>
#
# This is the set of callbacks that mmc bridges call into the bus, or
# that mmc/sd card drivers call to make requests.
# This is the set of callbacks that the MMC subroutines and the mmc/sd card
# driver call into the bus to make requests.
#
INTERFACE mmcbus;
#
# Queue and wait for a request.
# Pause re-tuning, optionally with triggering re-tuning up-front. Requires
# the bus to be claimed.
#
METHOD void retune_pause {
device_t brdev;
device_t reqdev;
bool retune;
};
#
# Unpause re-tuning. Requires the bus to be claimed.
#
METHOD void retune_unpause {
device_t brdev;
device_t reqdev;
};
#
# Queue and wait for a request. Requires the bus to be claimed.
#
METHOD int wait_for_request {
device_t brdev;
@ -73,18 +91,18 @@ METHOD int wait_for_request {
};
#
# Claim the current bridge, blocking the current thread until the host
# is no longer busy.
# Claim the current bus, blocking the current thread until the host is no
# longer busy.
#
METHOD int acquire_bus {
device_t brdev;
device_t reqdev;
}
};
#
# Release the current bridge.
# Release the current bus.
#
METHOD int release_bus {
device_t brdev;
device_t reqdev;
}
};

12
sys/dev/mmc/mmcreg.h
View file

@ -204,6 +204,7 @@ struct mmc_request {
void *done_data; /* requestor set data */
uint32_t flags;
#define MMC_REQ_DONE 1
#define MMC_TUNE_DONE 2
};
/* Command definitions */
@ -431,8 +432,8 @@ struct mmc_request {
#define EXT_CSD_HS_TIMING_BC 0
#define EXT_CSD_HS_TIMING_HS 1
#define EXT_CSD_HS_TIMING_DDR200 2
#define EXT_CSD_HS_TIMING_DDR400 3
#define EXT_CSD_HS_TIMING_HS200 2
#define EXT_CSD_HS_TIMING_HS400 3
#define EXT_CSD_HS_TIMING_DRV_STR_SHIFT 4
#define EXT_CSD_POWER_CLASS_8BIT_MASK 0xf0
@ -448,7 +449,6 @@ struct mmc_request {
#define EXT_CSD_CARD_TYPE_HS200_1_2V 0x0020
#define EXT_CSD_CARD_TYPE_HS400_1_8V 0x0040
#define EXT_CSD_CARD_TYPE_HS400_1_2V 0x0080
#define EXT_CSD_CARD_TYPE_HS400ES 0x0100
#define EXT_CSD_BUS_WIDTH_1 0
#define EXT_CSD_BUS_WIDTH_4 1
@ -457,6 +457,8 @@ struct mmc_request {
#define EXT_CSD_BUS_WIDTH_8_DDR 6
#define EXT_CSD_BUS_WIDTH_ES 0x80
#define EXT_CSD_STROBE_SUPPORT_EN 0x01
#define MMC_TYPE_HS_26_MAX 26000000
#define MMC_TYPE_HS_52_MAX 52000000
#define MMC_TYPE_DDR52_MAX 52000000
@ -657,6 +659,10 @@ struct mmc_sd_status
#define MMC_PART_GP_MAX 4
#define MMC_PART_MAX 8
#define MMC_TUNING_MAX 64 /* Maximum tuning iterations */
#define MMC_TUNING_LEN 64 /* Size of tuning data */
#define MMC_TUNING_LEN_HS200 128 /* Size of tuning data in HS200 mode */
/*
* Older versions of the MMC standard had a variable sector size. However,
* I've been able to find no old MMC or SD cards that have a non 512

294
sys/dev/mmc/mmcsd.c
View file

@ -99,7 +99,8 @@ __FBSDID("$FreeBSD$");
struct mmcsd_softc;
struct mmcsd_part {
struct mtx part_mtx;
struct mtx disk_mtx;
struct mtx ioctl_mtx;
struct mmcsd_softc *sc;
struct disk *disk;
struct proc *p;
@ -109,6 +110,7 @@ struct mmcsd_part {
u_int type;
int running;
int suspend;
int ioctl;
bool ro;
char name[MMCSD_PART_NAMELEN];
};
@ -118,6 +120,9 @@ struct mmcsd_softc {
device_t mmcbr;
struct mmcsd_part *part[MMC_PART_MAX];
enum mmc_card_mode mode;
u_int max_data; /* Maximum data size [blocks] */
u_int erase_sector; /* Device native erase sector size [blocks] */
uint8_t high_cap; /* High Capacity device (block addressed) */
uint8_t part_curr; /* Partition currently switched to */
uint8_t ext_csd[MMC_EXTCSD_SIZE];
uint16_t rca;
@ -178,15 +183,25 @@ static int mmcsd_slicer(device_t dev, const char *provider,
static int mmcsd_switch_part(device_t bus, device_t dev, uint16_t rca,
u_int part);
#define MMCSD_PART_LOCK(_part) mtx_lock(&(_part)->part_mtx)
#define MMCSD_PART_UNLOCK(_part) mtx_unlock(&(_part)->part_mtx)
#define MMCSD_PART_LOCK_INIT(_part) \
mtx_init(&(_part)->part_mtx, (_part)->name, "mmcsd part", MTX_DEF)
#define MMCSD_PART_LOCK_DESTROY(_part) mtx_destroy(&(_part)->part_mtx);
#define MMCSD_PART_ASSERT_LOCKED(_part) \
mtx_assert(&(_part)->part_mtx, MA_OWNED);
#define MMCSD_PART_ASSERT_UNLOCKED(_part) \
mtx_assert(&(_part)->part_mtx, MA_NOTOWNED);
#define MMCSD_DISK_LOCK(_part) mtx_lock(&(_part)->disk_mtx)
#define MMCSD_DISK_UNLOCK(_part) mtx_unlock(&(_part)->disk_mtx)
#define MMCSD_DISK_LOCK_INIT(_part) \
mtx_init(&(_part)->disk_mtx, (_part)->name, "mmcsd disk", MTX_DEF)
#define MMCSD_DISK_LOCK_DESTROY(_part) mtx_destroy(&(_part)->disk_mtx);
#define MMCSD_DISK_ASSERT_LOCKED(_part) \
mtx_assert(&(_part)->disk_mtx, MA_OWNED);
#define MMCSD_DISK_ASSERT_UNLOCKED(_part) \
mtx_assert(&(_part)->disk_mtx, MA_NOTOWNED);
#define MMCSD_IOCTL_LOCK(_part) mtx_lock(&(_part)->ioctl_mtx)
#define MMCSD_IOCTL_UNLOCK(_part) mtx_unlock(&(_part)->ioctl_mtx)
#define MMCSD_IOCTL_LOCK_INIT(_part) \
mtx_init(&(_part)->ioctl_mtx, (_part)->name, "mmcsd IOCTL", MTX_DEF)
#define MMCSD_IOCTL_LOCK_DESTROY(_part) mtx_destroy(&(_part)->ioctl_mtx);
#define MMCSD_IOCTL_ASSERT_LOCKED(_part) \
mtx_assert(&(_part)->ioctl_mtx, MA_OWNED);
#define MMCSD_IOCLT_ASSERT_UNLOCKED(_part) \
mtx_assert(&(_part)->ioctl_mtx, MA_NOTOWNED);
static int
mmcsd_probe(device_t dev)
@ -214,6 +229,18 @@ mmcsd_attach(device_t dev)
sc->dev = dev;
sc->mmcbr = mmcbr = device_get_parent(dev);
sc->mode = mmcbr_get_mode(mmcbr);
/*
* Note that in principle with an SDHCI-like re-tuning implementation,
* the maximum data size can change at runtime due to a device removal/
* insertion that results in switches to/from a transfer mode involving
* re-tuning, iff there are multiple devices on a given bus. Until now
* mmc(4) lacks support for rescanning already attached buses, however,
* and sdhci(4) has no support for embedded/shared buses in the first
* place either.
*/
sc->max_data = mmc_get_max_data(dev);
sc->erase_sector = mmc_get_erase_sector(dev);
sc->high_cap = mmc_get_high_cap(dev);
sc->rca = mmc_get_rca(dev);
/* Only MMC >= 4.x devices support EXT_CSD. */
@ -267,7 +294,7 @@ mmcsd_attach(device_t dev)
(ext_csd[EXT_CSD_ENH_START_ADDR + 1] << 8) +
(ext_csd[EXT_CSD_ENH_START_ADDR + 2] << 16) +
(ext_csd[EXT_CSD_ENH_START_ADDR + 3] << 24)) *
(mmc_get_high_cap(dev) ? MMC_SECTOR_SIZE : 1);
(sc->high_cap != 0 ? MMC_SECTOR_SIZE : 1);
} else if (bootverbose)
device_printf(dev,
"enhanced user data area spans entire device\n");
@ -280,7 +307,7 @@ mmcsd_attach(device_t dev)
ro = mmc_get_read_only(dev);
mmcsd_add_part(sc, EXT_CSD_PART_CONFIG_ACC_DEFAULT, "mmcsd",
device_get_unit(dev), mmc_get_media_size(dev) * sector_size,
mmc_get_erase_sector(dev) * sector_size, ro);
sc->erase_sector * sector_size, ro);
if (mmc_get_spec_vers(dev) < 3)
return (0);
@ -417,7 +444,16 @@ mmcsd_add_part(struct mmcsd_softc *sc, u_int type, const char *name, u_int cnt,
part->ro = ro;
snprintf(part->name, sizeof(part->name), name, device_get_unit(dev));
/* For the RPMB partition, allow IOCTL access only. */
MMCSD_IOCTL_LOCK_INIT(part);
/*
* For the RPMB partition, allow IOCTL access only.
* NB: If ever attaching RPMB partitions to disk(9), the re-tuning
* implementation and especially its pausing need to be revisited,
* because then re-tuning requests may be issued by the IOCTL half
* of this driver while re-tuning is already paused by the disk(9)
* one and vice versa.
*/
if (type == EXT_CSD_PART_CONFIG_ACC_RPMB) {
make_dev_args_init(&args);
args.mda_flags = MAKEDEV_CHECKNAME | MAKEDEV_WAITOK;
@ -432,7 +468,7 @@ mmcsd_add_part(struct mmcsd_softc *sc, u_int type, const char *name, u_int cnt,
return;
}
} else {
MMCSD_PART_LOCK_INIT(part);
MMCSD_DISK_LOCK_INIT(part);
d = part->disk = disk_alloc();
d->d_open = mmcsd_open;
@ -444,7 +480,7 @@ mmcsd_add_part(struct mmcsd_softc *sc, u_int type, const char *name, u_int cnt,
d->d_name = part->name;
d->d_drv1 = part;
d->d_sectorsize = mmc_get_sector_size(dev);
d->d_maxsize = mmc_get_max_data(dev) * d->d_sectorsize;
d->d_maxsize = sc->max_data * d->d_sectorsize;
d->d_mediasize = media_size;
d->d_stripesize = erase_size;
d->d_unit = cnt;
@ -471,7 +507,7 @@ mmcsd_add_part(struct mmcsd_softc *sc, u_int type, const char *name, u_int cnt,
part->name, cnt, bytes, unit, mmc_get_card_id_string(dev),
ro ? " (read-only)" : "", device_get_nameunit(mmcbr),
speed / 1000000, (speed / 100000) % 10,
mmcsd_bus_bit_width(dev), mmc_get_max_data(dev));
mmcsd_bus_bit_width(dev), sc->max_data);
} else if (type == EXT_CSD_PART_CONFIG_ACC_RPMB) {
printf("%s: %ju%sB partion %d%s at %s\n", part->name, bytes,
unit, type, ro ? " (read-only)" : "",
@ -559,19 +595,27 @@ mmcsd_detach(device_t dev)
for (i = 0; i < MMC_PART_MAX; i++) {
part = sc->part[i];
if (part != NULL && part->disk != NULL) {
MMCSD_PART_LOCK(part);
part->suspend = 0;
if (part->running > 0) {
/* kill thread */
part->running = 0;
wakeup(part);
/* wait for thread to finish. */
while (part->running != -1)
msleep(part, &part->part_mtx, 0,
"detach", 0);
if (part != NULL) {
if (part->disk != NULL) {
MMCSD_DISK_LOCK(part);
part->suspend = 0;
if (part->running > 0) {
/* kill thread */
part->running = 0;
wakeup(part);
/* wait for thread to finish. */
while (part->running != -1)
msleep(part, &part->disk_mtx, 0,
"mmcsd disk detach", 0);
}
MMCSD_DISK_UNLOCK(part);
}
MMCSD_PART_UNLOCK(part);
MMCSD_IOCTL_LOCK(part);
while (part->ioctl > 0)
msleep(part, &part->ioctl_mtx, 0,
"mmcsd IOCTL detach", 0);
part->ioctl = -1;
MMCSD_IOCTL_UNLOCK(part);
}
}
@ -587,8 +631,9 @@ mmcsd_detach(device_t dev)
/* kill disk */
disk_destroy(part->disk);
MMCSD_PART_LOCK_DESTROY(part);
MMCSD_DISK_LOCK_DESTROY(part);
}
MMCSD_IOCTL_LOCK_DESTROY(part);
free(part, M_DEVBUF);
}
}
@ -604,19 +649,27 @@ mmcsd_suspend(device_t dev)
for (i = 0; i < MMC_PART_MAX; i++) {
part = sc->part[i];
if (part != NULL && part->disk != NULL) {
MMCSD_PART_LOCK(part);
part->suspend = 1;
if (part->running > 0) {
/* kill thread */
part->running = 0;
wakeup(part);
/* wait for thread to finish. */
while (part->running != -1)
msleep(part, &part->part_mtx, 0,
"detach", 0);
if (part != NULL) {
if (part->disk != NULL) {
MMCSD_DISK_LOCK(part);
part->suspend = 1;
if (part->running > 0) {
/* kill thread */
part->running = 0;
wakeup(part);
/* wait for thread to finish. */
while (part->running != -1)
msleep(part, &part->disk_mtx, 0,
"mmcsd disk suspension", 0);
}
MMCSD_DISK_UNLOCK(part);
}
MMCSD_PART_UNLOCK(part);
MMCSD_IOCTL_LOCK(part);
while (part->ioctl > 0)
msleep(part, &part->ioctl_mtx, 0,
"mmcsd IOCTL suspension", 0);
part->ioctl = -1;
MMCSD_IOCTL_UNLOCK(part);
}
}
return (0);
@ -631,16 +684,22 @@ mmcsd_resume(device_t dev)
for (i = 0; i < MMC_PART_MAX; i++) {
part = sc->part[i];
if (part != NULL && part->disk != NULL) {
MMCSD_PART_LOCK(part);
part->suspend = 0;
if (part->running <= 0) {
part->running = 1;
kproc_create(&mmcsd_task, part, &part->p, 0, 0,
"%s%d: mmc/sd card", part->name, part->cnt);
MMCSD_PART_UNLOCK(part);
} else
MMCSD_PART_UNLOCK(part);
if (part != NULL) {
if (part->disk != NULL) {
MMCSD_DISK_LOCK(part);
part->suspend = 0;
if (part->running <= 0) {
part->running = 1;
MMCSD_DISK_UNLOCK(part);
kproc_create(&mmcsd_task, part,
&part->p, 0, 0, "%s%d: mmc/sd card",
part->name, part->cnt);
} else
MMCSD_DISK_UNLOCK(part);
}
MMCSD_IOCTL_LOCK(part);
part->ioctl = 0;
MMCSD_IOCTL_UNLOCK(part);
}
}
return (0);
@ -668,13 +727,13 @@ mmcsd_strategy(struct bio *bp)
part = bp->bio_disk->d_drv1;
sc = part->sc;
MMCSD_PART_LOCK(part);
MMCSD_DISK_LOCK(part);
if (part->running > 0 || part->suspend > 0) {
bioq_disksort(&part->bio_queue, bp);
MMCSD_PART_UNLOCK(part);
MMCSD_DISK_UNLOCK(part);
wakeup(part);
} else {
MMCSD_PART_UNLOCK(part);
MMCSD_DISK_UNLOCK(part);
biofinish(bp, NULL, ENXIO);
}
}
@ -710,9 +769,9 @@ mmcsd_ioctl(struct mmcsd_part *part, u_long cmd, void *data, int fflag)
switch (cmd) {
case MMC_IOC_CMD:
mic = data;
err = mmcsd_ioctl_cmd(part, data, fflag);
err = mmcsd_ioctl_cmd(part, mic, fflag);
break;
case MMC_IOC_CMD_MULTI:
case MMC_IOC_MULTI_CMD:
mimc = data;
if (mimc->num_of_cmds == 0)
break;
@ -722,12 +781,12 @@ mmcsd_ioctl(struct mmcsd_part *part, u_long cmd, void *data, int fflag)
size = sizeof(*mic) * cnt;
mic = malloc(size, M_TEMP, M_WAITOK);
err = copyin((const void *)mimc->cmds, mic, size);
if (err != 0)
break;
for (i = 0; i < cnt; i++) {
err = mmcsd_ioctl_cmd(part, &mic[i], fflag);
if (err != 0)
break;
if (err == 0) {
for (i = 0; i < cnt; i++) {
err = mmcsd_ioctl_cmd(part, &mic[i], fflag);
if (err != 0)
break;
}
}
free(mic, M_TEMP);
break;
@ -756,11 +815,31 @@ mmcsd_ioctl_cmd(struct mmcsd_part *part, struct mmc_ioc_cmd *mic, int fflag)
if (part->ro == TRUE && mic->write_flag != 0)
return (EROFS);
/*
* We don't need to explicitly lock against the disk(9) half of this
* driver as MMCBUS_ACQUIRE_BUS() will serialize us. However, it's
* necessary to protect against races with detachment and suspension,
* especially since it's required to switch away from RPMB partitions
* again after an access (see mmcsd_switch_part()).
*/
MMCSD_IOCTL_LOCK(part);
while (part->ioctl != 0) {
if (part->ioctl < 0) {
MMCSD_IOCTL_UNLOCK(part);
return (ENXIO);
}
msleep(part, &part->ioctl_mtx, 0, "mmcsd IOCTL", 0);
}
part->ioctl = 1;
MMCSD_IOCTL_UNLOCK(part);
err = 0;
dp = NULL;
len = mic->blksz * mic->blocks;
if (len > MMC_IOC_MAX_BYTES)
return (EOVERFLOW);
if (len > MMC_IOC_MAX_BYTES) {
err = EOVERFLOW;
goto out;
}
if (len != 0) {
dp = malloc(len, M_TEMP, M_WAITOK);
err = copyin((void *)(uintptr_t)mic->data_ptr, dp, len);
@ -815,7 +894,7 @@ mmcsd_ioctl_cmd(struct mmcsd_part *part, struct mmc_ioc_cmd *mic, int fflag)
err = mmcsd_set_blockcount(sc, mic->blocks,
mic->write_flag & (1 << 31));
if (err != MMC_ERR_NONE)
goto release;
goto switch_back;
}
if (mic->is_acmd != 0)
(void)mmc_wait_for_app_cmd(mmcbr, dev, rca, &cmd, 0);
@ -837,6 +916,7 @@ mmcsd_ioctl_cmd(struct mmcsd_part *part, struct mmc_ioc_cmd *mic, int fflag)
DELAY(1000);
} while (retries-- > 0);
switch_back:
/* ... and always switch back to the default partition. */
err = mmcsd_switch_part(mmcbr, dev, rca,
EXT_CSD_PART_CONFIG_ACC_DEFAULT);
@ -887,6 +967,10 @@ release:
err = EIO;
out:
MMCSD_IOCTL_LOCK(part);
part->ioctl = 0;
MMCSD_IOCTL_UNLOCK(part);
wakeup(part);
if (dp != NULL)
free(dp, M_TEMP);
return (err);
@ -938,10 +1022,23 @@ mmcsd_switch_part(device_t bus, device_t dev, uint16_t rca, u_int part)
sc = device_get_softc(dev);
if (sc->part_curr == part)
if (sc->mode == mode_sd)
return (MMC_ERR_NONE);
if (sc->mode == mode_sd)
/*
* According to section "6.2.2 Command restrictions" of the eMMC
* specification v5.1, CMD19/CMD21 aren't allowed to be used with
* RPMB partitions. So we pause re-tuning along with triggering
* it up-front to decrease the likelihood of re-tuning becoming
* necessary while accessing an RPMB partition. Consequently, an
* RPMB partition should immediately be switched away from again
* after an access in order to allow for re-tuning to take place
* anew.
*/
if (part == EXT_CSD_PART_CONFIG_ACC_RPMB)
MMCBUS_RETUNE_PAUSE(sc->mmcbr, sc->dev, true);
if (sc->part_curr == part)
return (MMC_ERR_NONE);
value = (sc->ext_csd[EXT_CSD_PART_CONFIG] &
@ -949,10 +1046,15 @@ mmcsd_switch_part(device_t bus, device_t dev, uint16_t rca, u_int part)
/* Jump! */
err = mmc_switch(bus, dev, rca, EXT_CSD_CMD_SET_NORMAL,
EXT_CSD_PART_CONFIG, value, sc->part_time, true);
if (err != MMC_ERR_NONE)
if (err != MMC_ERR_NONE) {
if (part == EXT_CSD_PART_CONFIG_ACC_RPMB)
MMCBUS_RETUNE_UNPAUSE(sc->mmcbr, sc->dev);
return (err);
}
sc->ext_csd[EXT_CSD_PART_CONFIG] = value;
if (sc->part_curr == EXT_CSD_PART_CONFIG_ACC_RPMB)
MMCBUS_RETUNE_UNPAUSE(sc->mmcbr, sc->dev);
sc->part_curr = part;
return (MMC_ERR_NONE);
}
@ -963,7 +1065,7 @@ mmcsd_errmsg(int e)
if (e < 0 || e > MMC_ERR_MAX)
return "Bad error code";
return errmsg[e];
return (errmsg[e]);
}
static daddr_t
@ -976,7 +1078,7 @@ mmcsd_rw(struct mmcsd_part *part, struct bio *bp)
struct mmc_data data;
struct mmcsd_softc *sc;
device_t dev, mmcbr;
int numblocks, sz;
u_int numblocks, sz;
char *vaddr;
sc = part->sc;
@ -988,7 +1090,7 @@ mmcsd_rw(struct mmcsd_part *part, struct bio *bp)
end = bp->bio_pblkno + (bp->bio_bcount / sz);
while (block < end) {
vaddr = bp->bio_data + (block - bp->bio_pblkno) * sz;
numblocks = min(end - block, mmc_get_max_data(dev));
numblocks = min(end - block, sc->max_data);
memset(&req, 0, sizeof(req));
memset(&cmd, 0, sizeof(cmd));
memset(&stop, 0, sizeof(stop));
@ -1008,7 +1110,7 @@ mmcsd_rw(struct mmcsd_part *part, struct bio *bp)
cmd.opcode = MMC_WRITE_BLOCK;
}
cmd.arg = block;
if (!mmc_get_high_cap(dev))
if (sc->high_cap == 0)
cmd.arg <<= 9;
cmd.flags = MMC_RSP_R1 | MMC_CMD_ADTC;
data.data = vaddr;
@ -1048,7 +1150,7 @@ mmcsd_delete(struct mmcsd_part *part, struct bio *bp)
struct mmc_request req;
struct mmcsd_softc *sc;
device_t dev, mmcbr;
int erase_sector, sz;
u_int erase_sector, sz;
sc = part->sc;
dev = sc->dev;
@ -1063,7 +1165,7 @@ mmcsd_delete(struct mmcsd_part *part, struct bio *bp)
if (end >= part->eblock && end < part->eend)
end = part->eend;
/* Safe round to the erase sector boundaries. */
erase_sector = mmc_get_erase_sector(dev);
erase_sector = sc->erase_sector;
start = block + erase_sector - 1; /* Round up. */
start -= start % erase_sector;
stop = end; /* Round down. */
@ -1075,6 +1177,12 @@ mmcsd_delete(struct mmcsd_part *part, struct bio *bp)
return (end);
}
/*
* Pause re-tuning so it won't interfere with the order of erase
* commands. Note that these latter don't use the data lines, so
* re-tuning shouldn't actually become necessary during erase.
*/
MMCBUS_RETUNE_PAUSE(mmcbr, dev, false);
/* Set erase start position. */
memset(&req, 0, sizeof(req));
memset(&cmd, 0, sizeof(cmd));
@ -1085,13 +1193,15 @@ mmcsd_delete(struct mmcsd_part *part, struct bio *bp)
else
cmd.opcode = MMC_ERASE_GROUP_START;
cmd.arg = start;
if (!mmc_get_high_cap(dev))
if (sc->high_cap == 0)
cmd.arg <<= 9;
cmd.flags = MMC_RSP_R1 | MMC_CMD_AC;
MMCBUS_WAIT_FOR_REQUEST(mmcbr, dev, &req);
if (req.cmd->error != MMC_ERR_NONE) {
printf("erase err1: %d\n", req.cmd->error);
return (block);
device_printf(dev, "Setting erase start position failed %d\n",
req.cmd->error);
block = bp->bio_pblkno;
goto unpause;
}
/* Set erase stop position. */
memset(&req, 0, sizeof(req));
@ -1102,14 +1212,16 @@ mmcsd_delete(struct mmcsd_part *part, struct bio *bp)
else
cmd.opcode = MMC_ERASE_GROUP_END;
cmd.arg = stop;
if (!mmc_get_high_cap(dev))
if (sc->high_cap == 0)
cmd.arg <<= 9;
cmd.arg--;
cmd.flags = MMC_RSP_R1 | MMC_CMD_AC;
MMCBUS_WAIT_FOR_REQUEST(mmcbr, dev, &req);
if (req.cmd->error != MMC_ERR_NONE) {
printf("erase err2: %d\n", req.cmd->error);
return (block);
device_printf(dev, "Setting erase stop position failed %d\n",
req.cmd->error);
block = bp->bio_pblkno;
goto unpause;
}
/* Erase range. */
memset(&req, 0, sizeof(req));
@ -1120,8 +1232,11 @@ mmcsd_delete(struct mmcsd_part *part, struct bio *bp)
cmd.flags = MMC_RSP_R1B | MMC_CMD_AC;
MMCBUS_WAIT_FOR_REQUEST(mmcbr, dev, &req);
if (req.cmd->error != MMC_ERR_NONE) {
printf("erase err3 %d\n", req.cmd->error);
return (block);
device_printf(dev, "erase err3: %d\n", req.cmd->error);
device_printf(dev, "Issuing erase command failed %d\n",
req.cmd->error);
block = bp->bio_pblkno;
goto unpause;
}
/* Store one of remaining parts for the next call. */
if (bp->bio_pblkno >= part->eblock || block == start) {
@ -1131,7 +1246,10 @@ mmcsd_delete(struct mmcsd_part *part, struct bio *bp)
part->eblock = block; /* Predict next backward. */
part->eend = start;
}
return (end);
block = end;
unpause:
MMCBUS_RETUNE_UNPAUSE(mmcbr, dev);
return (block);
}
static int
@ -1192,16 +1310,16 @@ mmcsd_task(void *arg)
mmcbr = sc->mmcbr;
while (1) {
MMCSD_PART_LOCK(part);
MMCSD_DISK_LOCK(part);
do {
if (part->running == 0)
goto out;
bp = bioq_takefirst(&part->bio_queue);
if (bp == NULL)
msleep(part, &part->part_mtx, PRIBIO,
"jobqueue", 0);
msleep(part, &part->disk_mtx, PRIBIO,
"mmcsd disk jobqueue", 0);
} while (bp == NULL);
MMCSD_PART_UNLOCK(part);
MMCSD_DISK_UNLOCK(part);
if (bp->bio_cmd != BIO_READ && part->ro) {
bp->bio_error = EROFS;
bp->bio_resid = bp->bio_bcount;
@ -1242,7 +1360,7 @@ release:
out:
/* tell parent we're done */
part->running = -1;
MMCSD_PART_UNLOCK(part);
MMCSD_DISK_UNLOCK(part);
wakeup(part);
kproc_exit(0);

488
sys/dev/sdhci/sdhci.c
View file

@ -1,5 +1,6 @@
/*-
* Copyright (c) 2008 Alexander Motin <mav@FreeBSD.org>
* Copyright (c) 2017 Marius Strobl <marius@FreeBSD.org>
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
@ -32,7 +33,9 @@ __FBSDID("$FreeBSD$");
#include <sys/callout.h>
#include <sys/conf.h>
#include <sys/kernel.h>
#include <sys/kobj.h>
#include <sys/lock.h>
#include <sys/malloc.h>
#include <sys/module.h>
#include <sys/mutex.h>
#include <sys/resource.h>
@ -48,6 +51,8 @@ __FBSDID("$FreeBSD$");
#include <dev/mmc/mmcreg.h>
#include <dev/mmc/mmcbrvar.h>
#include <dev/sdhci/sdhci.h>
#include <cam/cam.h>
#include <cam/cam_ccb.h>
#include <cam/cam_debug.h>
@ -55,7 +60,6 @@ __FBSDID("$FreeBSD$");
#include <cam/cam_xpt_sim.h>
#include "mmcbr_if.h"
#include "sdhci.h"
#include "sdhci_if.h"
#include "opt_mmccam.h"
@ -84,13 +88,15 @@ SYSCTL_INT(_hw_sdhci, OID_AUTO, quirk_set, CTLFLAG_RWTUN, &sdhci_quirk_set, 0,
#define WR_MULTI_4(slot, off, ptr, count) \
SDHCI_WRITE_MULTI_4((slot)->bus, (slot), (off), (ptr), (count))
static void sdhci_card_poll(void *arg);
static void sdhci_card_task(void *arg, int pending);
static int sdhci_exec_tuning(struct sdhci_slot *slot, bool reset);
static void sdhci_req_wakeup(struct mmc_request *req);
static void sdhci_retune(void *arg);
static void sdhci_set_clock(struct sdhci_slot *slot, uint32_t clock);
static void sdhci_start(struct sdhci_slot *slot);
static void sdhci_start_data(struct sdhci_slot *slot, struct mmc_data *data);
static void sdhci_card_poll(void *);
static void sdhci_card_task(void *, int);
#ifdef MMCCAM
/* CAM-related */
int sdhci_cam_get_possible_host_clock(struct sdhci_slot *slot, int proposed_clock);
@ -105,6 +111,7 @@ static int sdhci_cam_settran_settings(struct sdhci_slot *slot, union ccb *ccb);
static void sdhci_dumpregs(struct sdhci_slot *slot);
static int slot_printf(struct sdhci_slot *slot, const char * fmt, ...)
__printflike(2, 3);
static uint32_t sdhci_tuning_intmask(struct sdhci_slot *slot);
#define SDHCI_LOCK(_slot) mtx_lock(&(_slot)->mtx)
#define SDHCI_UNLOCK(_slot) mtx_unlock(&(_slot)->mtx)
@ -185,13 +192,13 @@ sdhci_dumpregs(struct sdhci_slot *slot)
RD1(slot, SDHCI_TIMEOUT_CONTROL), RD4(slot, SDHCI_INT_STATUS));
slot_printf(slot, "Int enab: 0x%08x | Sig enab: 0x%08x\n",
RD4(slot, SDHCI_INT_ENABLE), RD4(slot, SDHCI_SIGNAL_ENABLE));
slot_printf(slot, "AC12 err: 0x%08x | Host ctl2: 0x%08x\n",
slot_printf(slot, "AC12 err: 0x%08x | Host ctl2:0x%08x\n",
RD2(slot, SDHCI_ACMD12_ERR), RD2(slot, SDHCI_HOST_CONTROL2));
slot_printf(slot, "Caps: 0x%08x | Caps2: 0x%08x\n",
RD4(slot, SDHCI_CAPABILITIES), RD4(slot, SDHCI_CAPABILITIES2));
slot_printf(slot, "Max curr: 0x%08x | ADMA err: 0x%08x\n",
RD4(slot, SDHCI_MAX_CURRENT), RD1(slot, SDHCI_ADMA_ERR));
slot_printf(slot, "ADMA addr: 0x%08x | Slot int: 0x%08x\n",
slot_printf(slot, "ADMA addr:0x%08x | Slot int: 0x%08x\n",
RD4(slot, SDHCI_ADMA_ADDRESS_LO), RD2(slot, SDHCI_SLOT_INT_STATUS));
slot_printf(slot,
@ -258,6 +265,21 @@ sdhci_reset(struct sdhci_slot *slot, uint8_t mask)
}
}
static uint32_t
sdhci_tuning_intmask(struct sdhci_slot *slot)
{
uint32_t intmask;
intmask = 0;
if (slot->opt & SDHCI_TUNING_SUPPORTED) {
intmask |= SDHCI_INT_TUNEERR;
if (slot->retune_mode == SDHCI_RETUNE_MODE_2 ||
slot->retune_mode == SDHCI_RETUNE_MODE_3)
intmask |= SDHCI_INT_RETUNE;
}
return (intmask);
}
static void
sdhci_init(struct sdhci_slot *slot)
{
@ -277,7 +299,7 @@ sdhci_init(struct sdhci_slot *slot)
slot->intmask |= SDHCI_INT_CARD_REMOVE | SDHCI_INT_CARD_INSERT;
}
WR4(slot, SDHCI_INT_ENABLE, slot->intmask);
WR4(slot, SDHCI_INT_ENABLE, slot->intmask | sdhci_tuning_intmask(slot));
WR4(slot, SDHCI_SIGNAL_ENABLE, slot->intmask);
}
@ -583,10 +605,12 @@ sdhci_card_task(void *arg, int pending __unused)
SDHCI_UNLOCK(slot);
xpt_rescan(ccb);
#else
slot->dev = device_add_child(slot->bus, "mmc", -1);
device_set_ivars(slot->dev, slot);
d = slot->dev = device_add_child(slot->bus, "mmc", -1);
SDHCI_UNLOCK(slot);
device_probe_and_attach(slot->dev);
if (d) {
device_set_ivars(d, slot);
(void)device_probe_and_attach(d);
}
#endif
} else
SDHCI_UNLOCK(slot);
@ -627,7 +651,11 @@ sdhci_card_task(void *arg, int pending __unused)
SDHCI_UNLOCK(slot);
xpt_rescan(ccb);
#else
slot->intmask &= ~sdhci_tuning_intmask(slot);
WR4(slot, SDHCI_SIGNAL_ENABLE, slot->intmask);
slot->opt &= ~SDHCI_TUNING_ENABLED;
SDHCI_UNLOCK(slot);
callout_drain(&slot->retune_callout);
device_delete_child(slot->bus, d);
#endif
} else
@ -687,6 +715,8 @@ sdhci_card_poll(void *arg)
int
sdhci_init_slot(device_t dev, struct sdhci_slot *slot, int num)
{
kobjop_desc_t kobj_desc;
kobj_method_t *kobj_method;
uint32_t caps, caps2, freq, host_caps;
int err;
@ -728,8 +758,6 @@ sdhci_init_slot(device_t dev, struct sdhci_slot *slot, int num)
return (EFAULT);
}
/* Initialize slot. */
sdhci_init(slot);
slot->version = (RD2(slot, SDHCI_HOST_VERSION)
>> SDHCI_SPEC_VER_SHIFT) & SDHCI_SPEC_VER_MASK;
if (slot->quirks & SDHCI_QUIRK_MISSING_CAPS) {
@ -797,6 +825,7 @@ sdhci_init_slot(device_t dev, struct sdhci_slot *slot, int num)
device_printf(dev, "Hardware doesn't report any "
"support voltages.\n");
}
host_caps = MMC_CAP_4_BIT_DATA;
if (caps & SDHCI_CAN_DO_8BITBUS)
host_caps |= MMC_CAP_8_BIT_DATA;
@ -806,6 +835,8 @@ sdhci_init_slot(device_t dev, struct sdhci_slot *slot, int num)
host_caps |= MMC_CAP_BOOT_NOACC;
if (slot->quirks & SDHCI_QUIRK_WAIT_WHILE_BUSY)
host_caps |= MMC_CAP_WAIT_WHILE_BUSY;
/* Determine supported UHS-I and eMMC modes. */
if (caps2 & (SDHCI_CAN_SDR50 | SDHCI_CAN_SDR104 | SDHCI_CAN_DDR50))
host_caps |= MMC_CAP_UHS_SDR12 | MMC_CAP_UHS_SDR25;
if (caps2 & SDHCI_CAN_SDR104) {
@ -822,12 +853,91 @@ sdhci_init_slot(device_t dev, struct sdhci_slot *slot, int num)
if (slot->quirks & SDHCI_QUIRK_CAPS_BIT63_FOR_MMC_HS400 &&
caps2 & SDHCI_CAN_MMC_HS400)
host_caps |= MMC_CAP_MMC_HS400;
/*
* Disable UHS-I and eMMC modes if the set_uhs_timing method is the
* default NULL implementation.
*/
kobj_desc = &sdhci_set_uhs_timing_desc;
kobj_method = kobj_lookup_method(((kobj_t)dev)->ops->cls, NULL,
kobj_desc);
if (kobj_method == &kobj_desc->deflt)
host_caps &= ~(MMC_CAP_UHS_SDR12 | MMC_CAP_UHS_SDR25 |
MMC_CAP_UHS_SDR50 | MMC_CAP_UHS_DDR50 | MMC_CAP_UHS_SDR104 |
MMC_CAP_MMC_DDR52 | MMC_CAP_MMC_HS200 | MMC_CAP_MMC_HS400);
#define SDHCI_CAP_MODES_TUNING(caps2) \
(((caps2) & SDHCI_TUNE_SDR50 ? MMC_CAP_UHS_SDR50 : 0) | \
MMC_CAP_UHS_DDR50 | MMC_CAP_UHS_SDR104 | MMC_CAP_MMC_HS200 | \
MMC_CAP_MMC_HS400)
/*
* Disable UHS-I and eMMC modes that require (re-)tuning if either
* the tune or re-tune method is the default NULL implementation.
*/
kobj_desc = &mmcbr_tune_desc;
kobj_method = kobj_lookup_method(((kobj_t)dev)->ops->cls, NULL,
kobj_desc);
if (kobj_method == &kobj_desc->deflt)
goto no_tuning;
kobj_desc = &mmcbr_retune_desc;
kobj_method = kobj_lookup_method(((kobj_t)dev)->ops->cls, NULL,
kobj_desc);
if (kobj_method == &kobj_desc->deflt) {
no_tuning:
host_caps &= ~(SDHCI_CAP_MODES_TUNING(caps2));
}
/* Allocate tuning structures and determine tuning parameters. */
if (host_caps & SDHCI_CAP_MODES_TUNING(caps2)) {
slot->opt |= SDHCI_TUNING_SUPPORTED;
slot->tune_req = malloc(sizeof(*slot->tune_req), M_DEVBUF,
M_WAITOK);
slot->tune_cmd = malloc(sizeof(*slot->tune_cmd), M_DEVBUF,
M_WAITOK);
slot->tune_data = malloc(sizeof(*slot->tune_data), M_DEVBUF,
M_WAITOK);
if (caps2 & SDHCI_TUNE_SDR50)
slot->opt |= SDHCI_SDR50_NEEDS_TUNING;
slot->retune_mode = (caps2 & SDHCI_RETUNE_MODES_MASK) >>
SDHCI_RETUNE_MODES_SHIFT;
if (slot->retune_mode == SDHCI_RETUNE_MODE_1) {
slot->retune_count = (caps2 & SDHCI_RETUNE_CNT_MASK) >>
SDHCI_RETUNE_CNT_SHIFT;
if (slot->retune_count > 0xb) {
device_printf(dev, "Unknown re-tuning count "
"%x, using 1 sec\n", slot->retune_count);
slot->retune_count = 1;
} else if (slot->retune_count != 0)
slot->retune_count =
1 << (slot->retune_count - 1);
}
}
#undef SDHCI_CAP_MODES_TUNING
/* Determine supported VCCQ signaling levels. */
host_caps |= MMC_CAP_SIGNALING_330;
if (host_caps & (MMC_CAP_UHS_SDR12 | MMC_CAP_UHS_SDR25 |
MMC_CAP_UHS_SDR50 | MMC_CAP_UHS_SDR104 | MMC_CAP_UHS_DDR50 |
MMC_CAP_UHS_SDR50 | MMC_CAP_UHS_DDR50 | MMC_CAP_UHS_SDR104 |
MMC_CAP_MMC_DDR52_180 | MMC_CAP_MMC_HS200_180 |
MMC_CAP_MMC_HS400_180))
host_caps |= MMC_CAP_SIGNALING_180;
host_caps |= MMC_CAP_SIGNALING_120 | MMC_CAP_SIGNALING_180;
/*
* Disable 1.2 V and 1.8 V signaling if the switch_vccq method is the
* default NULL implementation. Disable 1.2 V support if it's the
* generic SDHCI implementation.
*/
kobj_desc = &mmcbr_switch_vccq_desc;
kobj_method = kobj_lookup_method(((kobj_t)dev)->ops->cls, NULL,
kobj_desc);
if (kobj_method == &kobj_desc->deflt)
host_caps &= ~(MMC_CAP_SIGNALING_120 | MMC_CAP_SIGNALING_180);
else if (kobj_method->func == (kobjop_t)sdhci_generic_switch_vccq)
host_caps &= ~MMC_CAP_SIGNALING_120;
/* Determine supported driver types (type B is always mandatory). */
if (caps2 & SDHCI_CAN_DRIVE_TYPE_A)
host_caps |= MMC_CAP_DRIVER_TYPE_A;
if (caps2 & SDHCI_CAN_DRIVE_TYPE_C)
@ -866,9 +976,9 @@ sdhci_init_slot(device_t dev, struct sdhci_slot *slot, int num)
(caps & SDHCI_CAN_VDD_180) ? " 1.8V" : "",
(host_caps & MMC_CAP_SIGNALING_180) ? " 1.8V" : "",
(host_caps & MMC_CAP_SIGNALING_120) ? " 1.2V" : "",
(caps2 & SDHCI_CAN_DRIVE_TYPE_A) ? "A" : "",
(caps2 & SDHCI_CAN_DRIVE_TYPE_C) ? "C" : "",
(caps2 & SDHCI_CAN_DRIVE_TYPE_D) ? "D" : "",
(host_caps & MMC_CAP_DRIVER_TYPE_A) ? "A" : "",
(host_caps & MMC_CAP_DRIVER_TYPE_C) ? "C" : "",
(host_caps & MMC_CAP_DRIVER_TYPE_D) ? "D" : "",
(slot->opt & SDHCI_HAVE_DMA) ? "DMA" : "PIO");
if (host_caps & (MMC_CAP_MMC_DDR52 | MMC_CAP_MMC_HS200 |
MMC_CAP_MMC_HS400 | MMC_CAP_MMC_ENH_STROBE))
@ -888,6 +998,9 @@ sdhci_init_slot(device_t dev, struct sdhci_slot *slot, int num)
(host_caps & MMC_CAP_UHS_SDR50) ? " SDR50" : "",
(host_caps & MMC_CAP_UHS_SDR104) ? " SDR104" : "",
(host_caps & MMC_CAP_UHS_DDR50) ? " DDR50" : "");
if (slot->opt & SDHCI_TUNING_SUPPORTED)
slot_printf(slot, "Re-tuning count %d secs, mode %d\n",
slot->retune_count, slot->retune_mode + 1);
sdhci_dumpregs(slot);
}
@ -901,6 +1014,7 @@ sdhci_init_slot(device_t dev, struct sdhci_slot *slot, int num)
sdhci_card_task, slot);
callout_init(&slot->card_poll_callout, 1);
callout_init_mtx(&slot->timeout_callout, &slot->mtx, 0);
callout_init_mtx(&slot->retune_callout, &slot->mtx, 0);
if ((slot->quirks & SDHCI_QUIRK_POLL_CARD_PRESENT) &&
!(slot->opt & SDHCI_NON_REMOVABLE)) {
@ -908,6 +1022,8 @@ sdhci_init_slot(device_t dev, struct sdhci_slot *slot, int num)
SDHCI_CARD_PRESENT_TICKS, sdhci_card_poll, slot);
}
sdhci_init(slot);
return (0);
}
@ -925,6 +1041,7 @@ sdhci_cleanup_slot(struct sdhci_slot *slot)
callout_drain(&slot->timeout_callout);
callout_drain(&slot->card_poll_callout);
callout_drain(&slot->retune_callout);
taskqueue_drain(taskqueue_swi_giant, &slot->card_task);
taskqueue_drain_timeout(taskqueue_swi_giant, &slot->card_delayed_task);
@ -941,6 +1058,11 @@ sdhci_cleanup_slot(struct sdhci_slot *slot)
bus_dmamap_unload(slot->dmatag, slot->dmamap);
bus_dmamem_free(slot->dmatag, slot->dmamem, slot->dmamap);
bus_dma_tag_destroy(slot->dmatag);
if (slot->opt & SDHCI_TUNING_SUPPORTED) {
free(slot->tune_req, M_DEVBUF);
free(slot->tune_cmd, M_DEVBUF);
free(slot->tune_data, M_DEVBUF);
}
SDHCI_LOCK_DESTROY(slot);
@ -951,7 +1073,16 @@ int
sdhci_generic_suspend(struct sdhci_slot *slot)
{
/*
* We expect the MMC layer to issue initial tuning after resume.
* Otherwise, we'd need to indicate re-tuning including circuit reset
* being required at least for re-tuning modes 1 and 2 ourselves.
*/
callout_drain(&slot->retune_callout);
SDHCI_LOCK(slot);
slot->opt &= ~SDHCI_TUNING_ENABLED;
sdhci_reset(slot, SDHCI_RESET_ALL);
SDHCI_UNLOCK(slot);
return (0);
}
@ -960,7 +1091,9 @@ int
sdhci_generic_resume(struct sdhci_slot *slot)
{
SDHCI_LOCK(slot);
sdhci_init(slot);
SDHCI_UNLOCK(slot);
return (0);
}
@ -994,15 +1127,18 @@ sdhci_generic_set_uhs_timing(device_t brdev __unused, struct sdhci_slot *slot)
if (slot->version < SDHCI_SPEC_300)
return;
SDHCI_ASSERT_LOCKED(slot);
ios = &slot->host.ios;
sdhci_set_clock(slot, 0);
hostctrl2 = RD2(slot, SDHCI_HOST_CONTROL2);
hostctrl2 &= ~SDHCI_CTRL2_UHS_MASK;
if (ios->timing == bus_timing_mmc_hs400 ||
ios->timing == bus_timing_mmc_hs400es)
hostctrl2 |= SDHCI_CTRL2_MMC_HS400;
else if (ios->clock > SD_SDR50_MAX)
hostctrl2 |= SDHCI_CTRL2_UHS_SDR104;
if (ios->clock > SD_SDR50_MAX) {
if (ios->timing == bus_timing_mmc_hs400 ||
ios->timing == bus_timing_mmc_hs400es)
hostctrl2 |= SDHCI_CTRL2_MMC_HS400;
else
hostctrl2 |= SDHCI_CTRL2_UHS_SDR104;
}
else if (ios->clock > SD_SDR25_MAX)
hostctrl2 |= SDHCI_CTRL2_UHS_SDR50;
else if (ios->clock > SD_SDR12_MAX) {
@ -1114,13 +1250,196 @@ done:
return (err);
}
int
sdhci_generic_tune(device_t brdev __unused, device_t reqdev, bool hs400)
{
struct sdhci_slot *slot = device_get_ivars(reqdev);
struct mmc_ios *ios = &slot->host.ios;
struct mmc_command *tune_cmd;
struct mmc_data *tune_data;
uint32_t opcode;
int err;
if (!(slot->opt & SDHCI_TUNING_SUPPORTED))
return (0);
slot->retune_ticks = slot->retune_count * hz;
opcode = MMC_SEND_TUNING_BLOCK;
SDHCI_LOCK(slot);
switch (ios->timing) {
case bus_timing_mmc_hs400:
slot_printf(slot, "HS400 must be tuned in HS200 mode\n");
SDHCI_UNLOCK(slot);
return (EINVAL);
case bus_timing_mmc_hs200:
/*
* In HS400 mode, controllers use the data strobe line to
* latch data from the devices so periodic re-tuning isn't
* expected to be required.
*/
if (hs400)
slot->retune_ticks = 0;
opcode = MMC_SEND_TUNING_BLOCK_HS200;
break;
case bus_timing_uhs_ddr50:
case bus_timing_uhs_sdr104:
break;
case bus_timing_uhs_sdr50:
if (slot->opt & SDHCI_SDR50_NEEDS_TUNING)
break;
/* FALLTHROUGH */
default:
SDHCI_UNLOCK(slot);
return (0);
}
tune_cmd = slot->tune_cmd;
memset(tune_cmd, 0, sizeof(*tune_cmd));
tune_cmd->opcode = opcode;
tune_cmd->flags = MMC_RSP_R1 | MMC_CMD_ADTC;
tune_data = tune_cmd->data = slot->tune_data;
memset(tune_data, 0, sizeof(*tune_data));
tune_data->len = (opcode == MMC_SEND_TUNING_BLOCK_HS200 &&
ios->bus_width == bus_width_8) ? MMC_TUNING_LEN_HS200 :
MMC_TUNING_LEN;
tune_data->flags = MMC_DATA_READ;
tune_data->mrq = tune_cmd->mrq = slot->tune_req;
slot->opt &= ~SDHCI_TUNING_ENABLED;
err = sdhci_exec_tuning(slot, true);
if (err == 0) {
slot->opt |= SDHCI_TUNING_ENABLED;
slot->intmask |= sdhci_tuning_intmask(slot);
WR4(slot, SDHCI_SIGNAL_ENABLE, slot->intmask);
if (slot->retune_ticks) {
callout_reset(&slot->retune_callout, slot->retune_ticks,
sdhci_retune, slot);
}
}
SDHCI_UNLOCK(slot);
return (err);
}
int
sdhci_generic_retune(device_t brdev __unused, device_t reqdev, bool reset)
{
struct sdhci_slot *slot = device_get_ivars(reqdev);
int err;
if (!(slot->opt & SDHCI_TUNING_ENABLED))
return (0);
/* HS400 must be tuned in HS200 mode. */
if (slot->host.ios.timing == bus_timing_mmc_hs400)
return (EINVAL);
SDHCI_LOCK(slot);
err = sdhci_exec_tuning(slot, reset);
/*
* There are two ways sdhci_exec_tuning() can fail:
* EBUSY should not actually happen when requests are only issued
* with the host properly acquired, and
* EIO re-tuning failed (but it did work initially).
*
* In both cases, we should retry at later point if periodic re-tuning
* is enabled. Note that due to slot->retune_req not being cleared in
* these failure cases, the MMC layer should trigger another attempt at
* re-tuning with the next request anyway, though.
*/
if (slot->retune_ticks) {
callout_reset(&slot->retune_callout, slot->retune_ticks,
sdhci_retune, slot);
}
SDHCI_UNLOCK(slot);
return (err);
}
static int
sdhci_exec_tuning(struct sdhci_slot *slot, bool reset)
{
struct mmc_request *tune_req;
struct mmc_command *tune_cmd;
int i;
uint32_t intmask;
uint16_t hostctrl2;
u_char opt;
SDHCI_ASSERT_LOCKED(slot);
if (slot->req != NULL)
return (EBUSY);
/* Tuning doesn't work with DMA enabled. */
opt = slot->opt;
slot->opt = opt & ~SDHCI_HAVE_DMA;
/*
* Ensure that as documented, SDHCI_INT_DATA_AVAIL is the only
* kind of interrupt we receive in response to a tuning request.
*/
intmask = slot->intmask;
slot->intmask = SDHCI_INT_DATA_AVAIL;
WR4(slot, SDHCI_SIGNAL_ENABLE, SDHCI_INT_DATA_AVAIL);
hostctrl2 = RD2(slot, SDHCI_HOST_CONTROL2);
if (reset)
hostctrl2 &= ~SDHCI_CTRL2_SAMPLING_CLOCK;
else
hostctrl2 |= SDHCI_CTRL2_SAMPLING_CLOCK;
WR2(slot, SDHCI_HOST_CONTROL2, hostctrl2 | SDHCI_CTRL2_EXEC_TUNING);
tune_req = slot->tune_req;
tune_cmd = slot->tune_cmd;
for (i = 0; i < MMC_TUNING_MAX; i++) {
memset(tune_req, 0, sizeof(*tune_req));
tune_req->cmd = tune_cmd;
tune_req->done = sdhci_req_wakeup;
tune_req->done_data = slot;
slot->req = tune_req;
slot->flags = 0;
sdhci_start(slot);
while (!(tune_req->flags & MMC_REQ_DONE))
msleep(tune_req, &slot->mtx, 0, "sdhciet", 0);
if (!(tune_req->flags & MMC_TUNE_DONE))
break;
hostctrl2 = RD2(slot, SDHCI_HOST_CONTROL2);
if (!(hostctrl2 & SDHCI_CTRL2_EXEC_TUNING))
break;
if (tune_cmd->opcode == MMC_SEND_TUNING_BLOCK)
DELAY(1000);
}
slot->opt = opt;
slot->intmask = intmask;
WR4(slot, SDHCI_SIGNAL_ENABLE, intmask);
if ((hostctrl2 & (SDHCI_CTRL2_EXEC_TUNING |
SDHCI_CTRL2_SAMPLING_CLOCK)) == SDHCI_CTRL2_SAMPLING_CLOCK) {
slot->retune_req = 0;
return (0);
}
slot_printf(slot, "Tuning failed, using fixed sampling clock\n");
WR2(slot, SDHCI_HOST_CONTROL2, hostctrl2 & ~(SDHCI_CTRL2_EXEC_TUNING |
SDHCI_CTRL2_SAMPLING_CLOCK));
sdhci_reset(slot, SDHCI_RESET_CMD | SDHCI_RESET_DATA);
return (EIO);
}
static void
sdhci_retune(void *arg)
{
struct sdhci_slot *slot = arg;
slot->retune_req |= SDHCI_RETUNE_REQ_NEEDED;
}
#ifdef MMCCAM
static void
sdhci_req_done(struct sdhci_slot *slot)
{
union ccb *ccb;
if (sdhci_debug > 1)
if (__predict_false(sdhci_debug > 1))
slot_printf(slot, "%s\n", __func__);
if (slot->ccb != NULL && slot->curcmd != NULL) {
callout_stop(&slot->timeout_callout);
@ -1153,19 +1472,29 @@ sdhci_req_done(struct sdhci_slot *slot)
}
#endif
static void
sdhci_req_wakeup(struct mmc_request *req)
{
struct sdhci_slot *slot;
slot = req->done_data;
req->flags |= MMC_REQ_DONE;
wakeup(req);
}
static void
sdhci_timeout(void *arg)
{
struct sdhci_slot *slot = arg;
if (slot->curcmd != NULL) {
slot_printf(slot, " Controller timeout\n");
slot_printf(slot, "Controller timeout\n");
sdhci_dumpregs(slot);
sdhci_reset(slot, SDHCI_RESET_CMD | SDHCI_RESET_DATA);
slot->curcmd->error = MMC_ERR_TIMEOUT;
sdhci_req_done(slot);
} else {
slot_printf(slot, " Spurious timeout - no active command\n");
slot_printf(slot, "Spurious timeout - no active command\n");
}
}
@ -1236,15 +1565,19 @@ sdhci_start_command(struct sdhci_slot *slot, struct mmc_command *cmd)
/* Wait for free DAT if we have data or busy signal. */
if (cmd->data != NULL || (cmd->flags & MMC_RSP_BUSY))
mask |= SDHCI_DAT_INHIBIT;
/* We shouldn't wait for DAT for stop commands. */
/*
* We shouldn't wait for DAT for stop commands or CMD19/CMD21. Note
* that these latter are also special in that SDHCI_CMD_DATA should
* be set below but no actual data is ever read from the controller.
*/
#ifdef MMCCAM
struct ccb_mmcio *mmcio = &slot->ccb->mmcio;
if (cmd == &mmcio->stop)
mask &= ~SDHCI_DAT_INHIBIT;
if (cmd == &slot->ccb->mmcio.stop ||
#else
if (cmd == slot->req->stop)
mask &= ~SDHCI_DAT_INHIBIT;
if (cmd == slot->req->stop ||
#endif
__predict_false(cmd->opcode == MMC_SEND_TUNING_BLOCK ||
cmd->opcode == MMC_SEND_TUNING_BLOCK_HS200))
mask &= ~SDHCI_DAT_INHIBIT;
/*
* Wait for bus no more then 250 ms. Typically there will be no wait
* here at all, but when writing a crash dump we may be bypassing the
@ -1301,7 +1634,7 @@ sdhci_start_command(struct sdhci_slot *slot, struct mmc_command *cmd)
WR4(slot, SDHCI_ARGUMENT, cmd->arg);
/* Set data transfer mode. */
sdhci_set_transfer_mode(slot, cmd->data);
if (sdhci_debug > 1)
if (__predict_false(sdhci_debug > 1))
slot_printf(slot, "Starting command!\n");
/* Start command. */
WR2(slot, SDHCI_COMMAND_FLAGS, (cmd->opcode << 8) | (flags & 0xff));
@ -1317,7 +1650,7 @@ sdhci_finish_command(struct sdhci_slot *slot)
uint32_t val;
uint8_t extra;
if (sdhci_debug > 1)
if (__predict_false(sdhci_debug > 1))
slot_printf(slot, "%s: called, err %d flags %d\n",
__func__, slot->curcmd->error, slot->curcmd->flags);
slot->cmd_done = 1;
@ -1326,9 +1659,14 @@ sdhci_finish_command(struct sdhci_slot *slot)
* Main restore point for the case when command interrupt
* happened first.
*/
WR4(slot, SDHCI_SIGNAL_ENABLE, slot->intmask |= SDHCI_INT_RESPONSE);
if (__predict_true(slot->curcmd->opcode != MMC_SEND_TUNING_BLOCK &&
slot->curcmd->opcode != MMC_SEND_TUNING_BLOCK_HS200))
WR4(slot, SDHCI_SIGNAL_ENABLE, slot->intmask |=
SDHCI_INT_RESPONSE);
/* In case of error - reset host and return. */
if (slot->curcmd->error) {
if (slot->curcmd->error == MMC_ERR_BADCRC)
slot->retune_req |= SDHCI_RETUNE_REQ_RESET;
sdhci_reset(slot, SDHCI_RESET_CMD);
sdhci_reset(slot, SDHCI_RESET_DATA);
sdhci_start(slot);
@ -1353,7 +1691,7 @@ sdhci_finish_command(struct sdhci_slot *slot)
} else
slot->curcmd->resp[0] = RD4(slot, SDHCI_RESPONSE);
}
if (sdhci_debug > 1)
if (__predict_false(sdhci_debug > 1))
printf("Resp: %02x %02x %02x %02x\n",
slot->curcmd->resp[0], slot->curcmd->resp[1],
slot->curcmd->resp[2], slot->curcmd->resp[3]);
@ -1439,7 +1777,7 @@ sdhci_start_data(struct sdhci_slot *slot, struct mmc_data *data)
/* Set block count. */
WR2(slot, SDHCI_BLOCK_COUNT, (data->len + 511) / 512);
if (sdhci_debug > 1)
if (__predict_false(sdhci_debug > 1))
slot_printf(slot, "Block size: %02x, count %lu\n",
(unsigned int)SDHCI_MAKE_BLKSZ(DMA_BOUNDARY, (data->len < 512) ? data->len : 512),
(unsigned long)(data->len + 511) / 512);
@ -1474,6 +1812,8 @@ sdhci_finish_data(struct sdhci_slot *slot)
slot->data_done = 1;
/* If there was error - reset the host. */
if (slot->curcmd->error) {
if (slot->curcmd->error == MMC_ERR_BADCRC)
slot->retune_req |= SDHCI_RETUNE_REQ_RESET;
sdhci_reset(slot, SDHCI_RESET_CMD);
sdhci_reset(slot, SDHCI_RESET_DATA);
sdhci_start(slot);
@ -1514,7 +1854,7 @@ sdhci_start(struct sdhci_slot *slot)
return;
}
*/
if (sdhci_debug > 1)
if (__predict_false(sdhci_debug > 1))
slot_printf(slot, "result: %d\n", mmcio->cmd.error);
if (mmcio->cmd.error == 0 &&
(slot->quirks & SDHCI_QUIRK_RESET_AFTER_REQUEST)) {
@ -1545,7 +1885,7 @@ sdhci_start(struct sdhci_slot *slot)
sdhci_start_command(slot, req->stop);
return;
}
if (sdhci_debug > 1)
if (__predict_false(sdhci_debug > 1))
slot_printf(slot, "result: %d\n", req->cmd->error);
if (!req->cmd->error &&
((slot->curcmd == req->stop &&
@ -1570,7 +1910,7 @@ sdhci_generic_request(device_t brdev __unused, device_t reqdev,
SDHCI_UNLOCK(slot);
return (EBUSY);
}
if (sdhci_debug > 1) {
if (__predict_false(sdhci_debug > 1)) {
slot_printf(slot,
"CMD%u arg %#x flags %#x dlen %u dflags %#x\n",
req->cmd->opcode, req->cmd->arg, req->cmd->flags,
@ -1689,6 +2029,15 @@ sdhci_data_irq(struct sdhci_slot *slot, uint32_t intmask)
goto done;
}
/* Handle tuning completion interrupt. */
if (__predict_false((intmask & SDHCI_INT_DATA_AVAIL) &&
(slot->curcmd->opcode == MMC_SEND_TUNING_BLOCK ||
slot->curcmd->opcode == MMC_SEND_TUNING_BLOCK_HS200))) {
slot->req->flags |= MMC_TUNE_DONE;
sdhci_finish_command(slot);
sdhci_finish_data(slot);
return;
}
/* Handle PIO interrupt. */
if (intmask & (SDHCI_INT_DATA_AVAIL | SDHCI_INT_SPACE_AVAIL)) {
if ((slot->opt & SDHCI_PLATFORM_TRANSFER) &&
@ -1781,9 +2130,21 @@ sdhci_generic_intr(struct sdhci_slot *slot)
SDHCI_UNLOCK(slot);
return;
}
if (sdhci_debug > 2)
if (__predict_false(sdhci_debug > 2))
slot_printf(slot, "Interrupt %#x\n", intmask);
/* Handle tuning error interrupt. */
if (__predict_false(intmask & SDHCI_INT_TUNEERR)) {
slot_printf(slot, "Tuning error indicated\n");
slot->retune_req |= SDHCI_RETUNE_REQ_RESET;
if (slot->curcmd) {
slot->curcmd->error = MMC_ERR_BADCRC;
sdhci_finish_command(slot);
}
}
/* Handle re-tuning interrupt. */
if (__predict_false(intmask & SDHCI_INT_RETUNE))
slot->retune_req |= SDHCI_RETUNE_REQ_NEEDED;
/* Handle card presence interrupts. */
if (intmask & (SDHCI_INT_CARD_INSERT | SDHCI_INT_CARD_REMOVE)) {
present = (intmask & SDHCI_INT_CARD_INSERT) != 0;
@ -1796,7 +2157,6 @@ sdhci_generic_intr(struct sdhci_slot *slot)
WR4(slot, SDHCI_INT_STATUS, intmask &
(SDHCI_INT_CARD_INSERT | SDHCI_INT_CARD_REMOVE));
sdhci_handle_card_present_locked(slot, present);
intmask &= ~(SDHCI_INT_CARD_INSERT | SDHCI_INT_CARD_REMOVE);
}
/* Handle command interrupts. */
if (intmask & SDHCI_INT_CMD_MASK) {
@ -1815,17 +2175,14 @@ sdhci_generic_intr(struct sdhci_slot *slot)
WR4(slot, SDHCI_INT_STATUS, SDHCI_INT_ACMD12ERR);
sdhci_acmd_irq(slot);
}
intmask &= ~(SDHCI_INT_CMD_MASK | SDHCI_INT_DATA_MASK);
intmask &= ~SDHCI_INT_ACMD12ERR;
intmask &= ~SDHCI_INT_ERROR;
/* Handle bus power interrupt. */
if (intmask & SDHCI_INT_BUS_POWER) {
WR4(slot, SDHCI_INT_STATUS, SDHCI_INT_BUS_POWER);
slot_printf(slot,
"Card is consuming too much power!\n");
intmask &= ~SDHCI_INT_BUS_POWER;
slot_printf(slot, "Card is consuming too much power!\n");
}
intmask &= ~(SDHCI_INT_ERROR | SDHCI_INT_TUNEERR | SDHCI_INT_RETUNE |
SDHCI_INT_CARD_INSERT | SDHCI_INT_CARD_REMOVE | SDHCI_INT_CMD_MASK |
SDHCI_INT_DATA_MASK | SDHCI_INT_ACMD12ERR | SDHCI_INT_BUS_POWER);
/* The rest is unknown. */
if (intmask) {
WR4(slot, SDHCI_INT_STATUS, intmask);
@ -1879,6 +2236,19 @@ sdhci_generic_read_ivar(device_t bus, device_t child, int which,
case MMCBR_IVAR_VDD:
*result = slot->host.ios.vdd;
break;
case MMCBR_IVAR_RETUNE_REQ:
if (slot->opt & SDHCI_TUNING_ENABLED) {
if (slot->retune_req & SDHCI_RETUNE_REQ_RESET) {
*result = retune_req_reset;
break;
}
if (slot->retune_req & SDHCI_RETUNE_REQ_NEEDED) {
*result = retune_req_normal;
break;
}
}
*result = retune_req_none;
break;
case MMCBR_IVAR_VCCQ:
*result = slot->host.ios.vccq;
break;
@ -1889,6 +2259,16 @@ sdhci_generic_read_ivar(device_t bus, device_t child, int which,
*result = slot->host.ios.timing;
break;
case MMCBR_IVAR_MAX_DATA:
/*
* Re-tuning modes 1 and 2 restrict the maximum data length
* per read/write command to 4 MiB.
*/
if (slot->opt & SDHCI_TUNING_ENABLED &&
(slot->retune_mode == SDHCI_RETUNE_MODE_1 ||
slot->retune_mode == SDHCI_RETUNE_MODE_2)) {
*result = 4 * 1024 * 1024 / MMC_SECTOR_SIZE;
break;
}
*result = 65535;
break;
case MMCBR_IVAR_MAX_BUSY_TIMEOUT:
@ -1971,19 +2351,13 @@ sdhci_generic_write_ivar(device_t bus, device_t child, int which,
case MMCBR_IVAR_F_MIN:
case MMCBR_IVAR_F_MAX:
case MMCBR_IVAR_MAX_DATA:
case MMCBR_IVAR_RETUNE_REQ:
return (EINVAL);
}
return (0);
}
#ifdef MMCCAM
/* CAM-related functions */
#include <cam/cam.h>
#include <cam/cam_ccb.h>
#include <cam/cam_debug.h>
#include <cam/cam_sim.h>
#include <cam/cam_xpt_sim.h>
void
sdhci_cam_start_slot(struct sdhci_slot *slot)
{
@ -2123,7 +2497,7 @@ sdhci_cam_action(struct cam_sim *sim, union ccb *ccb)
* At some point in the future an interrupt comes.
* Then the request will be marked as completed.
*/
if (sdhci_debug > 1)
if (__predict_false(sdhci_debug > 1))
slot_printf(slot, "Got XPT_MMC_IO\n");
ccb->ccb_h.status = CAM_REQ_INPROG;
@ -2272,7 +2646,7 @@ sdhci_cam_request(struct sdhci_slot *slot, union ccb *ccb)
return (EBUSY);
}
*/
if (sdhci_debug > 1) {
if (__predict_false(sdhci_debug > 1)) {
slot_printf(slot, "CMD%u arg %#x flags %#x dlen %u dflags %#x\n",
mmcio->cmd.opcode, mmcio->cmd.arg, mmcio->cmd.flags,
mmcio->cmd.data != NULL ? (unsigned int) mmcio->cmd.data->len : 0,

20
sys/dev/sdhci/sdhci.h
View file

@ -328,6 +328,7 @@ extern u_int sdhci_quirk_clear;
extern u_int sdhci_quirk_set;
struct sdhci_slot {
struct mtx mtx; /* Slot mutex */
u_int quirks; /* Chip specific quirks */
u_int caps; /* Override SDHCI_CAPABILITIES */
u_int caps2; /* Override SDHCI_CAPABILITIES2 */
@ -338,6 +339,9 @@ struct sdhci_slot {
#define SDHCI_HAVE_DMA 0x01
#define SDHCI_PLATFORM_TRANSFER 0x02
#define SDHCI_NON_REMOVABLE 0x04
#define SDHCI_TUNING_SUPPORTED 0x08
#define SDHCI_TUNING_ENABLED 0x10
#define SDHCI_SDR50_NEEDS_TUNING 0x20
u_char version;
int timeout; /* Transfer timeout */
uint32_t max_clk; /* Max possible freq */
@ -351,14 +355,27 @@ struct sdhci_slot {
card_delayed_task;/* Card insert delayed task */
struct callout card_poll_callout;/* Card present polling callout */
struct callout timeout_callout;/* Card command/data response timeout */
struct callout retune_callout; /* Re-tuning mode 1 callout */
struct mmc_host host; /* Host parameters */
struct mmc_request *req; /* Current request */
struct mmc_command *curcmd; /* Current command of current request */
struct mmc_request *tune_req; /* Tuning request */
struct mmc_command *tune_cmd; /* Tuning command of tuning request */
struct mmc_data *tune_data; /* Tuning data of tuning command */
uint32_t retune_ticks; /* Re-tuning callout ticks [hz] */
uint32_t intmask; /* Current interrupt mask */
uint32_t clock; /* Current clock freq. */
size_t offset; /* Data buffer offset */
uint8_t hostctrl; /* Current host control register */
uint8_t retune_count; /* Controller re-tuning count [s] */
uint8_t retune_mode; /* Controller re-tuning mode */
#define SDHCI_RETUNE_MODE_1 0x00
#define SDHCI_RETUNE_MODE_2 0x01
#define SDHCI_RETUNE_MODE_3 0x02
uint8_t retune_req; /* Re-tuning request status */
#define SDHCI_RETUNE_REQ_NEEDED 0x01 /* Re-tuning w/o circuit reset needed */
#define SDHCI_RETUNE_REQ_RESET 0x02 /* Re-tuning w/ circuit reset needed */
u_char power; /* Current power */
u_char bus_busy; /* Bus busy status */
u_char cmd_done; /* CMD command part done flag */
@ -368,7 +385,6 @@ struct sdhci_slot {
#define STOP_STARTED 2
#define SDHCI_USE_DMA 4 /* Use DMA for this req. */
#define PLATFORM_DATA_STARTED 8 /* Data xfer is handled by platform */
struct mtx mtx; /* Slot mutex */
#ifdef MMCCAM
/* CAM stuff */
@ -392,7 +408,9 @@ int sdhci_cleanup_slot(struct sdhci_slot *slot);
int sdhci_generic_suspend(struct sdhci_slot *slot);
int sdhci_generic_resume(struct sdhci_slot *slot);
int sdhci_generic_update_ios(device_t brdev, device_t reqdev);
int sdhci_generic_tune(device_t brdev, device_t reqdev, bool hs400);
int sdhci_generic_switch_vccq(device_t brdev, device_t reqdev);
int sdhci_generic_retune(device_t brdev, device_t reqdev, bool reset);
int sdhci_generic_request(device_t brdev, device_t reqdev,
struct mmc_request *req);
int sdhci_generic_get_ro(device_t brdev, device_t reqdev);

2
sys/dev/sdhci/sdhci_acpi.c
View file

@ -371,6 +371,8 @@ static device_method_t sdhci_methods[] = {
/* mmcbr_if */
DEVMETHOD(mmcbr_update_ios, sdhci_generic_update_ios),
DEVMETHOD(mmcbr_switch_vccq, sdhci_generic_switch_vccq),
DEVMETHOD(mmcbr_tune, sdhci_generic_tune),
DEVMETHOD(mmcbr_retune, sdhci_generic_retune),
DEVMETHOD(mmcbr_request, sdhci_generic_request),
DEVMETHOD(mmcbr_get_ro, sdhci_generic_get_ro),
DEVMETHOD(mmcbr_acquire_host, sdhci_generic_acquire_host),

5
sys/dev/sdhci/sdhci_if.m
View file

@ -58,11 +58,14 @@
# that mmc/sd card drivers call to make requests.
#
#include <sys/lock.h>
#include <sys/mutex.h>
#include <sys/types.h>
#include <sys/bus.h>
#include <sys/sysctl.h>
#include <sys/taskqueue.h>
#include <machine/bus.h>
#include <dev/mmc/bridge.h>
#include <dev/sdhci/sdhci.h>

2
sys/dev/sdhci/sdhci_pci.c
View file

@ -496,6 +496,8 @@ static device_method_t sdhci_methods[] = {
/* mmcbr_if */
DEVMETHOD(mmcbr_update_ios, sdhci_generic_update_ios),
DEVMETHOD(mmcbr_switch_vccq, sdhci_generic_switch_vccq),
DEVMETHOD(mmcbr_tune, sdhci_generic_tune),
DEVMETHOD(mmcbr_retune, sdhci_generic_retune),
DEVMETHOD(mmcbr_request, sdhci_generic_request),
DEVMETHOD(mmcbr_get_ro, sdhci_generic_get_ro),
DEVMETHOD(mmcbr_acquire_host, sdhci_generic_acquire_host),