OSDN Git Service
Robin Murphy [Wed, 4 Mar 2015 16:41:05 +0000 (16:41 +0000)]
iommu/arm-smmu: set a more appropriate DMA mask
Since we use dma_map_page() as an architecture-independent means of
making page table updates visible to non-coherent SMMUs, we need to
have a suitable DMA mask set to discourage the DMA mapping layer from
creating bounce buffers and flushing those instead, if said page tables
happen to lie outside the default 32-bit mask.
Change-Id: Ia97122cef853cb48c9ad45a2a35717dd85cd9764
Tested-by: Mitchel Humpherys <mitchelh@codeaurora.org>
Signed-off-by: Robin Murphy <robin.murphy@arm.com>
[will: added error checking]
Signed-off-by: Will Deacon <will.deacon@arm.com>
[pdaly@codeaurora.org Take upstream version]
Will Deacon [Wed, 4 Mar 2015 12:21:03 +0000 (12:21 +0000)]
iommu/arm-smmu: ensure CBA2R is initialised before CBAR on SMMUv2
The VMID16 (8.1) extension to SMMUv2 added a 16-bit VMID16 field to the
CBA2R registers. Unfortunately, if software writes this field as zero
after setting an 8-bit VMID in a stage-2 CBAR, then the VMID may also be
overwritten with zero on some early implementations (the architecture
was later updated to fix this issue).
This patch ensures that we initialise CBA2R before CBAR, therefore
ensuring that the VMID is set correctly.
Tested-by: Manish Jaggi <mjaggi@caviumnetworks.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
Baptiste Reynal [Wed, 4 Mar 2015 15:51:06 +0000 (16:51 +0100)]
iommu/arm-smmu: fix ARM_SMMU_FEAT_TRANS_OPS condition
This patch is a fix to "iommu/arm-smmu: add support for iova_to_phys
through ATS1PR".
According to ARM documentation, translation registers are optional even
in SMMUv1, so ID0_S1TS needs to be checked to verify their presence.
Also, we check that the domain is a stage-1 domain.
Change-Id: I2164ddb3806f941d21463731d0991ce1a83a5221
Signed-off-by: Baptiste Reynal <b.reynal@virtualopensystems.com>
Acked-by: Will Deacon <will.deacon@arm.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
Will Deacon [Mon, 16 Feb 2015 18:38:20 +0000 (18:38 +0000)]
iommu/io-pgtable-arm: Fix self-test WARNs on i386
Various build/boot bots have reported WARNs being triggered by the ARM
iopgtable LPAE self-tests on i386 machines.
This boils down to two instances of right-shifting a 32-bit unsigned
long (i.e. an iova) by more than the size of the type. On 32-bit ARM,
this happens to give us zero, hence my testing didn't catch this
earlier.
This patch fixes the issue by using DIV_ROUND_UP and explicit case to
to avoid the erroneous shifts.
Reported-by: Fengguang Wu <fengguang.wu@intel.com>
Reported-by: Huang Ying <ying.huang@intel.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
Joerg Roedel [Wed, 4 Feb 2015 15:12:55 +0000 (16:12 +0100)]
iommu: Update my email address
The AMD address is dead for a long time already, replace it
with a working one.
Signed-off-by: Joerg Roedel <jroedel@suse.de>
Shuah Khan [Fri, 16 Jan 2015 02:29:43 +0000 (19:29 -0700)]
iommu: Fix trace_map() to report original iova and original size
iommu_map() calls trace_map() with iova and size. trace_map()
should report original iova and original size as opposed to
iova and size after they get changed during mapping. size is
always zero at the end of mapping which is useless to report
and iova as it gets incremented, it is not as useful as the
original iova. Change iommu_map() to call trace_map() to
report original iova and original size.
Signed-off-by: Shuah Khan <shuahkh@osg.samsung.com>
Reported-by: Alex Williamson <alex.williamson@redhat.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
Shuah Khan [Sat, 17 Jan 2015 03:53:17 +0000 (20:53 -0700)]
iommu: Change trace unmap api to report unmapped size
Currently map and unmap are implemented as events under a
common trace class declaration. The common class forces
trace_unmap() to require a bogus physical address argument
that it doesn't use. Changing unmap to report unmapped size
will provide useful information for debugging. Remove common
map_unmap trace class and change map and unmap into separate
events as opposed to events under the same class to allow for
differences in the reporting information. In addition, map and
unmap are changed to handle size value as size_t instead of int
to match the passed size value and avoid overflow.
Change-Id: Ic231e28c2784fbfe41a6749175cc390920165e81
Signed-off-by: Shuah Khan <shuahkh@osg.samsung.com>
Suggested-by: Alex Williamson <alex.williamson@redhat.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
[pdaly@codeaurora.org Keep map_unmap class]
Shuah Khan [Fri, 16 Jan 2015 23:47:19 +0000 (16:47 -0700)]
iommu: Fix trace_unmap() to report original iova
iommu_unmap() calls trace_unmap() with changed iova and original
size. trace_unmap() should report original iova instead. Change
iommu_unmap() to call trace_unmap() with original iova.
Change-Id: Iedbaf6ddef566626f5c3dd656a8c1b8cf8c15663
Signed-off-by: Shuah Khan <shuahkh@osg.samsung.com>
Reported-by: Alex Williamson <alex.williamson@redhat.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
[pdaly@codeaurora.org Resolve minor conflicts]
Robin Murphy [Tue, 25 Nov 2014 17:50:55 +0000 (17:50 +0000)]
iommu: Decouple iommu_map_sg from CPU page size
If the IOMMU supports pages smaller than the CPU page size, segments
which lie at offsets within the CPU page may be mapped based on the
finer-grained IOMMU page boundaries. This minimises the amount of
non-buffer memory between the CPU page boundary and the start of the
segment which must be mapped and therefore exposed to the device, and
brings the default iommu_map_sg implementation in line with
iommu_map/unmap with respect to alignment.
Signed-off-by: Robin Murphy <robin.murphy@arm.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
Marek Szyprowski [Wed, 19 Nov 2014 11:15:31 +0000 (11:15 +0000)]
iommu: fix initialization without 'add_device' callback
IOMMU drivers can be initialized from of_iommu helpers. Such drivers don't
need to provide device_add callbacks to operate properly, so there is no
need to fail initialization if the callback is missing.
Acked-by: Arnd Bergmann <arnd@arndb.de>
Signed-off-by: Marek Szyprowski <m.szyprowski@samsung.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
Will Deacon [Wed, 27 Aug 2014 15:15:59 +0000 (16:15 +0100)]
iommu: add new iommu_ops callback for adding an OF device
This patch adds a new function to the iommu_ops structure to allow an
OF device to be added to a specific IOMMU instance using the recently
merged generic devicetree binding for IOMMUs. The callback (of_xlate)
takes a struct device representing the master and an of_phandle_args
representing the IOMMU and the correspondong IDs for the new master.
Acked-by: Arnd Bergmann <arnd@arndb.de>
Acked-by: Joerg Roedel <jroedel@suse.de>
Acked-by: Marek Szyprowski <m.szyprowski@samsung.com>
Tested-by: Robin Murphy <robin.murphy@arm.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
Will Deacon [Wed, 27 Aug 2014 13:40:58 +0000 (14:40 +0100)]
iommu: provide early initialisation hook for IOMMU drivers
IOMMU drivers must be initialised before any of their upstream devices,
otherwise the relevant iommu_ops won't be configured for the bus in
question. To solve this, a number of IOMMU drivers use initcalls to
initialise the driver before anything has a chance to be probed.
Whilst this solves the immediate problem, it leaves the job of probing
the IOMMU completely separate from the iommu_ops to configure the IOMMU,
which are called on a per-bus basis and require the driver to figure out
exactly which instance of the IOMMU is being requested. In particular,
the add_device callback simply passes a struct device to the driver,
which then has to parse firmware tables or probe buses to identify the
relevant IOMMU instance.
This patch takes the first step in addressing this problem by adding an
early initialisation pass for IOMMU drivers, giving them the ability to
store some per-instance data in their iommu_ops structure and store that
in their of_node. This can later be used when parsing OF masters to
identify the IOMMU instance in question.
Change-Id: Ie7051c90c0f2e790d016eb465f10ddda48b99cd1
Acked-by: Arnd Bergmann <arnd@arndb.de>
Acked-by: Joerg Roedel <jroedel@suse.de>
Acked-by: Marek Szyprowski <m.szyprowski@samsung.com>
Tested-by: Robin Murphy <robin.murphy@arm.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
[pdaly@codeaurora.org Resolve minor conflicts]
Antonios Motakis [Mon, 13 Oct 2014 13:06:18 +0000 (14:06 +0100)]
iommu/arm-smmu: add IOMMU_CAP_NOEXEC to the ARM SMMU driver
The ARM SMMU supports the IOMMU_NOEXEC protection flag. Add the
corresponding IOMMU capability.
Signed-off-by: Antonios Motakis <a.motakis@virtualopensystems.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
Antonios Motakis [Mon, 13 Oct 2014 13:06:17 +0000 (14:06 +0100)]
iommu: add capability IOMMU_CAP_NOEXEC
Some IOMMUs accept an IOMMU_NOEXEC protection flag in addition to
IOMMU_READ and IOMMU_WRITE. Expose this as an IOMMU capability.
Signed-off-by: Antonios Motakis <a.motakis@virtualopensystems.com>
Acked-by: Joerg Roedel <jroedel@suse.de>
Signed-off-by: Will Deacon <will.deacon@arm.com>
Heiko Stübner [Wed, 29 Oct 2014 00:22:56 +0000 (01:22 +0100)]
iommu: Improve error handling when setting bus iommu
When some part of bus_set_iommu fails it should undo any made changes
and not simply leave everything as is.
This includes unregistering the bus notifier in iommu_bus_init when
add_iommu_group fails and also setting the bus->iommu_ops back to NULL.
Signed-off-by: Heiko Stuebner <heiko@sntech.de>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
Joerg Roedel [Tue, 4 Nov 2014 13:53:51 +0000 (14:53 +0100)]
iommu: Do more input validation in iommu_map_sg()
The IOMMU-API works on page boundarys, unlike the DMA-API
which can work with sub-page buffers. The sg->offset
field does not make sense on the IOMMU level, so force it to
be 0. Do some error-path consolidation while at it.
Signed-off-by: Joerg Roedel <jroedel@suse.de>
Wolfram Sang [Mon, 20 Oct 2014 14:20:42 +0000 (16:20 +0200)]
iommu: drop owner assignment from platform_drivers
A platform_driver does not need to set an owner, it will be populated by the
driver core.
Change-Id: I4382a4b82bba56cf5cd2c2bfe56cdbafeb647cb8
Signed-off-by: Wolfram Sang <wsa@the-dreams.de>
[pdaly@codeaurora.org Resolve minor conflicts]
Patrick Daly [Sat, 21 Nov 2015 04:33:04 +0000 (20:33 -0800)]
iommu/iommu-debug: Add 3 more signifigant digits
Print time with ns precision.
Ex:
size iommu_map iommu_unmap
4K 43.021 us 99.479 us
Change-Id: Ib7f3db2be533c77adf0a20de557615a0e2433dd9
Signed-off-by: Patrick Daly <pdaly@codeaurora.org>
Patrick Daly [Sat, 21 Nov 2015 04:00:50 +0000 (20:00 -0800)]
iommu/iommu-debug: Improve statistics collection
Add a nr_iters file which specifies the number of runs to average
performance over. Alternatively, setting this to 1 allows the user
to collect raw data.
Change-Id: Ifab7215027b597fb2650ab2f0c5065bebe0b8c5d
Signed-off-by: Patrick Daly <pdaly@codeaurora.org>
Shiraz Hashim [Wed, 18 Nov 2015 07:16:05 +0000 (12:46 +0530)]
dma-mapping: fix build when !CONFIG_ARM_DMA_USE_IOMMU
Allow compilation when CONFIG_ARM_DMA_USE_IOMMU is not
selected by providing necessary stub functions and fix few
mismatch in function declarations.
Following this, remove the changes introduced by
f3d8d1061fb0b146b3f5 ("msm: ipa: add empty implementation
for iommu functions") as they are no longer needed.
Change-Id: I04e3aa63407064e8d9c9550a5cb0a82ede899f00
Signed-off-by: Shiraz Hashim <shashim@codeaurora.org>
Mitchel Humpherys [Thu, 5 Nov 2015 19:50:29 +0000 (11:50 -0800)]
iommu/arm-smmu: Make unmapped accesses more obvious
The fault handler currently spits out a ton of information that can make
clients' eyes glaze over, and understandably so. One of the most
important pieces of information we obtain from the fault handler is
whether the faulting address is actually mapped (which we check with a
software table walk), but that information is currently not obvious (you
have to know that "soft iova-to-phys=0" means that the software table
walk failed). Fix this by adding a more obvious error message when we
detect an unmapped access.
Change-Id: Ib95a916baf3e73934642676de4852a550ae0f54b
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
Mitchel Humpherys [Mon, 9 Nov 2015 19:50:58 +0000 (11:50 -0800)]
iommu/arm-smmu: Reduce TLB sync timeout
We're currently waiting for up to 1 second for the TLB sync command to
finish. Unfortunately, this is long enough to trigger a watchdog bark
since the time between the watchdog pet timer interrupt and the bark
interrupt is only 1 second. Fix this by reducing the timeout to 500
milliseconds, which is still more than enough time for the TLB sync
command to complete.
CRs-Fixed: 929880
Change-Id: I467cd5fd9caa24b4f120a7fe8c8f57c73687006b
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
Jaydeep Sen [Sun, 8 Nov 2015 11:23:13 +0000 (16:53 +0530)]
iommu: arm-smmu: update regulator notification for SMMU
The SMMU halt needs to be accessed before regulator is disabled. So move
notification from REGULATOR_EVENT_DISABLE to REGULATOR_EVENT_PRE_DISABLE.
Regulator framework will notify client on other events too, return
NOTIFY_OK.
Change-Id: Ieed950418d5afc6f5b84b13131ef39964f3168e2
Signed-off-by: Jaydeep Sen <jsen@codeaurora.org>
Mitchel Humpherys [Mon, 2 Nov 2015 22:27:44 +0000 (14:27 -0800)]
iommu/arm-smmu: Use correct mask when printing SID during fault
As per the spec, FRSYNRA contains the SID in the lower 16 bits.
Currently we're masking off all but the lower 8 bits, so we could be
losing bits when we print SIDs. Fix this by using a 16 bit mask.
Change-Id: I4a16c169a52d31b43b2f850a5558eecb64b93902
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
Susheel Khiani [Tue, 6 Oct 2015 13:59:21 +0000 (19:29 +0530)]
iommu/arm-smmu: add option to enable halt/resume of SMMU
Before SMMU is powered down, SMMU needs to be in
idle state prior to power collapse. When 'halt' is
received by SMMU, it ensures that no new requests
enters and all outstanding requests are completed
and generates an acknowledgment for halt request.
Add an option to register a notifier on regulator
so that SMMU can be halted/resumed when regulator
is powered down/up. Some of the targets have built
in provision in hardware to halt SMMU when there
are no on going transactions. We should keep this
option disable for those targets.
Change-Id: Ia2f8a934a9d64daefdacd517eb22f09de5eeb273
Signed-off-by: Susheel Khiani <skhiani@codeaurora.org>
Mitchel Humpherys [Wed, 21 Oct 2015 20:59:50 +0000 (13:59 -0700)]
iommu/arm-smmu: Make fault triggering more reliable
We're currently disabling clocks/power immediately after writing to
FSRRESTORE, but if the fault is not handled before we disable the
clocks/power then the fault could be lost. Add a 1-second sleep after
writing to FSRRESTORE to give the interrupt time to fire before turning
clocks/power back off.
Change-Id: Ieeba49686da5da92e43f03c3c593166592184433
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
Mitchel Humpherys [Thu, 22 Oct 2015 20:48:29 +0000 (13:48 -0700)]
iommu: io-pgtable-arm: Correctly update *size in map_sg
Currently, in the arm_lpae_map_sg early return path (when neither READ
nor WRITE permissions were specified), we don't store the "partially
mapped" size in the size output parameter, as is expected. This could
result in someone up the stack using an uninitialized variable. Fix
this by storing 0 in *size when there's nothing to do.
Change-Id: I3b0952aa5bcc2810843b7c513c32c3d665169e40
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
Mitchel Humpherys [Wed, 30 Sep 2015 21:22:39 +0000 (14:22 -0700)]
iommu/io-pgtable: Improve documentation of .map_sg callback
The return value of the .map_sg callback is currently undocumented.
Document it.
Change-Id: I195cbfb78eaf173d916dd03190f89b2fe04e144e
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
Mitchel Humpherys [Sat, 26 Sep 2015 00:29:27 +0000 (17:29 -0700)]
iommu/arm-smmu: Implement DOMAIN_ATTR_NON_FATAL_FAULTS
Unhandled context faults should be fatal unless explicitly requested
otherwise by the client. Implement the DOMAIN_ATTR_NON_FATAL_FAULTS
attribute for this purpose.
Change-Id: I414fcc6000a6c47b3cbf77b1098c8b7895cbb20d
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
Mitchel Humpherys [Sat, 26 Sep 2015 00:26:31 +0000 (17:26 -0700)]
iommu: Add domain attribute to make faults non-fatal
During development, any IOMMU faults most likely need to be debugged
immediately. Allowing the system to continue to run has a tendency to
mask the original problem, which makes debugging more difficult. Some
drivers might want to make all faults fatal by default, unless
explicitly requested otherwise by a client. Add a domain attribute for
this.
Change-Id: I454b143ea3cae1f3de36e0888afd15936aa63ce1
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
Mitchel Humpherys [Tue, 20 Oct 2015 00:13:47 +0000 (17:13 -0700)]
iommu/arm-smmu: Don't ignore clock/regulator API errors
The clock and regulator APIs can fail, but we're ignoring their return
values. This results in strange code paths where we misuse the
API (e.g. we enable clocks after a failed prepare, which makes the clock
driver yell). Fix this by checking all return values and propagating
the errors up to clients as needed.
CRs-Fixed: 924600
Change-Id: Ib6cb3afac1f5472f3a5abb5c251765344fe1a7a9
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
Mitchel Humpherys [Tue, 20 Oct 2015 18:37:41 +0000 (11:37 -0700)]
iommu/arm-smmu: Move device config probe messages to debug level
The device config probe messages are for informational purposes only.
Move them to debug level to avoid spamming the boot log unnecessarily.
CRs-Fixed: 918584
Change-Id: If9085e2165f26b66ff94b258894f01471be82a5b
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
Mitchel Humpherys [Fri, 2 Oct 2015 23:17:57 +0000 (16:17 -0700)]
iommu/iommu-debug: Use more iterations while profiling
Currently we only take one sample per buffer size per API when doing
profiling. This results in high run-to-run variance in the results.
Use more iterations to help smooth this out.
Change-Id: I3779007a2f69ef79b573285b2422554f42dda99f
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
Mitchel Humpherys [Fri, 2 Oct 2015 23:02:31 +0000 (16:02 -0700)]
iommu/iommu-debug: Add support for secure domain profiling
Secure domains do different amounts of work than non-secure domains, so
we need to be able to profile them separately. Add another debugfs
file, "secure_profiling", that does the same thing as the "profiling"
file.
Change-Id: I6968036bfdcc63449f27336771b3956a2b7754f6
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
Mitchel Humpherys [Tue, 13 Oct 2015 01:13:06 +0000 (18:13 -0700)]
msm: secure_buffer: Add missing include
secure_buffer.h makes use of the scatterlist structure, but doesn't
include the header file that defines it. Until now we've been getting
lucky that all users of secure_buffer.h have been including the
scatterlist header file themselves, but this might not always be the
case. Add the missing include.
Change-Id: If7598c26a1ef3551132c6eb62ac9bd6d447654d0
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
Mitchel Humpherys [Fri, 2 Oct 2015 00:06:14 +0000 (17:06 -0700)]
iommu/arm-smmu: Pool secure page table memory
The hypervisor assign call has substantial overhead, so calling it every
time we need to allocate some page table memory is currently killing
performance. Fix this by hanging on to secure page table memory when
it's freed, keeping it in a pool of pre-assigned memory for future
allocations.
Change-Id: Iefa45e6bc23e841d168be4f17c1f669f2e932a3b
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
Mitchel Humpherys [Thu, 1 Oct 2015 22:56:03 +0000 (15:56 -0700)]
iommu/arm-smmu: Collect forward declarations at the top
We've been slowly littering arm-smmu.c with forward declarations all
over the place, moving them around when they're not high enough in the
file, etc. Just move them all to the top of the file once and for all.
Change-Id: I8f2f075546f287db17f1b6e3b08390296cb9b9ff
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
Neeti Desai [Mon, 14 Sep 2015 23:30:04 +0000 (16:30 -0700)]
iommu/arm-smmu: Add new lock to protect assign and unassign list
Simultaneous maps and/or unmaps will lead to a race condition,
where if the assign list has multiple entries and one call is
in the middle of an assign operation for one entry, and the other
call removes the next element from the list and queues itself for
the assign operation. When the first call returns from assign, and
the list is empty, it will return back to the caller without
completing the assign on all the ptes allocated for that mapping.
This results in a page fault when the client is actually accessing
the buffer.
Protect the assign and unassign lists with a mutex to ensure that
the operation is serialized and the control does not return back
to the client before all the ptes have been correctly assigned.
Change-Id: Idaf894671fee9fc9c8d339bfa17344b68dd0ad77
Signed-off-by: Neeti Desai <neetid@codeaurora.org>
Mitchel Humpherys [Wed, 23 Sep 2015 20:56:27 +0000 (13:56 -0700)]
iommu/io-pgtable-arm: Stricter double-map checking
Currently there's a check to make sure that a mapping isn't being
installed on top of an existing leaf mapping. However, mapping on top
of an existing table mapping is silently ignored. Make the check more
strict by testing for any valid page table entry (table, block, or page
descriptor) and by making any overlap fatal.
Change-Id: Ibdffbfdb2b44cf9da5044c2c1828a32979a39760
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
Mitchel Humpherys [Tue, 22 Sep 2015 19:10:39 +0000 (12:10 -0700)]
iommu/arm-smmu: Remove some duplicate macro definitions
When we brought back hardware ATOS support [
94bc89042be5cb:
"iommu/arm-smmu: Resurrect hardware iova-to-phys"] we added some macros
back that were never deleted in the first place. Clean up the redundant
macros.
Change-Id: I0129313fa4132f45fe5805e9ed3477ef7819e70e
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
Mitchel Humpherys [Tue, 22 Sep 2015 18:52:53 +0000 (11:52 -0700)]
iommu/arm-smmu: Don't wipe out MICRO_MMU_CTRL while halting
The MICRO_MMU_CTRL register is used for halt control, but it also
contains some important configuration bits. Currently, when we request
a halt, we're wiping out all of the other bits in the register, losing
any previous configuration. Fix this by reading the value and OR'ing in
the bits we need.
Change-Id: Iafdc5b2a344916dce9bac7f958a153dbee042ca8
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
Rohit Vaswani [Fri, 28 Aug 2015 22:33:14 +0000 (15:33 -0700)]
iommu/arm-smmu: Reorganize code to allow no gdsc
Some SMMUs may not have a dedicated gdsc for it.
Allow for the enable code to prepare clocks and vote
for bus requests even if there is no gdsc provided.
Change-Id: Iaedfb73d8ae7f12895472893468e5c32d2bf2462
Signed-off-by: Rohit Vaswani <rvaswani@codeaurora.org>
Shrenuj Bansal [Fri, 18 Sep 2015 21:59:09 +0000 (14:59 -0700)]
iommu/arm-smmu: Always set resume to terminate the transaction
When using the client fault handler, set resume to terminate the
faulting transaction similar to the way of the default fault
handler.
Its possible that if the client fault handler doesn't "fully"
handle the fault, there would be a stream of page faults on the
same address since we would keep on retrying the same
transaction.
Change-Id: Id7aa439fd90aa5b63364c8ac270593330c55d672
Signed-off-by: Shrenuj Bansal <shrenujb@codeaurora.org>
Jeremy Gebben [Wed, 16 Sep 2015 20:04:42 +0000 (14:04 -0600)]
iommu/io-pgtable-arm: add support for the IOMMU_PRIV flag
Allow the creation of privileged mode mappings, for stage 1 only.
Change-Id: Ie91168012b61c0774d840a35b3ed2f18b834873f
Signed-off-by: Jeremy Gebben <jgebben@codeaurora.org>
Mitchel Humpherys [Mon, 14 Sep 2015 19:08:09 +0000 (12:08 -0700)]
iommu/arm-smmu: Rate-limit the context fault handler
It's not uncommon for us to take a flood of context faults when
something terrible happens. This usually brings down the system due to
excessive logging, which makes it difficult to debug the actual root
cause of the issue. Fix this by rate-limiting the output from the
context fault handler.
Change-Id: I1fe83bbbc7cc7aa2cf6303c6c7d2bf5733f63557
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
Shiraz Hashim [Thu, 25 Jun 2015 11:15:24 +0000 (16:45 +0530)]
arm: dma-mapping: flush highmem mappings
DMA allocations with no kernel mapping are likely
protected as well. Stale highmem mappings in such cases
with cache-able attributes may lead to speculative fetch
,as highmem mappings are not cleared out.
Hence flush out unused highmem mappings explicitly when
allocation request with no kernel mapping is requested.
Change-Id: Ic1de633c6364eaa1b6d5b0932f2cfe17d64d920e
Signed-off-by: Shiraz Hashim <shashim@codeaurora.org>
Laura Abbott [Thu, 17 Jan 2013 02:23:19 +0000 (18:23 -0800)]
ARM: dma-mapping: Allow highmem pages to not have a mapping
The DMA_ATTR_NO_KERNEL_MAPPING is used to make sure that CMA
pages have no kernel mapping. Add support to make sure that
highmem pages have no mapping.
Change-Id: Ife76df126ecfedf0dba81a35e0de8a1787355b3d
Signed-off-by: Laura Abbott <lauraa@codeaurora.org>
Signed-off-by: Rohit Vaswani <rvaswani@codeaurora.org>
Patrick Daly [Wed, 17 Feb 2016 23:58:22 +0000 (15:58 -0800)]
arm: dma: Allow CMA pages to not have a kernel mapping.
Currently, there are use cases where not having any kernel
mapping is required; if the CMA memory needs to be used as
a pool which can have both cached and uncached mappings we
need to remove the mapping to avoid the multiple mapping
problem. Extend the dma APIs to use the DMA_ATTR_NO_KERNEL_MAPPING
with CMA. This doesn't end up saving any virtual address space
but the mapping will still not be present.
Change-Id: I64d21250abbe615c43e2b5b1272ee2b6d106705a
Signed-off-by: Laura Abbott <lauraa@codeaurora.org>
Signed-off-by: Patrick Daly <pdaly@codeaurora.org>
Taniya Das [Thu, 12 Jun 2014 14:12:06 +0000 (19:42 +0530)]
arm: Skip DMA zeroing completely only if specified
With the dma attribute 'SKIP_ZEROING', dma allocation skips memset and
also cache flush operations. This leaves dirty lines in the caches which
when evicted later would cause issue to the allocated buffer. For some
clients this may be okay but other clients need to do the actual sync.
Switch the code to only skip __dma_clear_buffer if both
DMA_ATTR_SKIP_ZEROING and DMA_ATTR_SKIP_CPU_SYNC are specified. If only
DMA_ATTR_SKIP_ZEROING is specified, just skip the zeroing but still do the
cache operations.
Change-Id: I81dd6bbeed7ec7f1dce28cd8d0aba7c8900ec550
Signed-off-by: Taniya Das <tdas@codeaurora.org>
Signed-off-by: Laura Abbott <lauraa@codeaurora.org>
Laura Abbott [Thu, 22 May 2014 21:38:48 +0000 (14:38 -0700)]
arm: Add option to skip buffer zeroing
The DMA framework currently zeros all buffers because it (righfully so)
assumes that drivers will soon need to pass the memory to a device.
Some devices/use case may not require zeroed memory and there can
be an increase in performance if we skip the zeroing. Add a DMA_ATTR
to allow skipping of DMA zeroing.
Change-Id: Ib78a19cb341c3c441f91d5b004c6375c80c10413
Signed-off-by: Laura Abbott <lauraa@codeaurora.org>
[pdaly@codeaurora.org Resolve minor conflicts]
Signed-off-by: Patrick Daly <pdaly@codeaurora.org>
Mitchel Humpherys [Mon, 31 Aug 2015 18:26:27 +0000 (11:26 -0700)]
iommu/arm-smmu: Add support for dynamic bus scale voting
Some SMMU operations (like register access and ATOS) require downstream
buses to be enabled. We've been getting lucky for a long time since
other clients have been voting for the buses in question, but recently
we've seen some unclocked accesses due to our missing vote. Add support
for bus scale voting to the driver. The voting parameters will be
provided later in the per-target device tree SMMU configuration.
Change-Id: I6b31c887ad817a6628f4234622c73dee046d6442
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
Mitchel Humpherys [Wed, 26 Aug 2015 18:52:57 +0000 (11:52 -0700)]
iommu/iommu-debug: Only uniquify directory names for dynamic domains
The subdirectories under "attach" currently get a UUID appended to their
name to ensure that there won't be any collisions when multiple domains
are attached to the same device. This is only needed for dynamic
domains since those are the only types of domains that can be attached
to the same device concurrently. Only do the uniquification for dynamic
domains since the resulting directory name is a bit messy with the UUID.
Change-Id: Ia86b192e5b9d6a63132228fa86549a2977ec74a4
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
Mitchel Humpherys [Fri, 28 Aug 2015 17:51:24 +0000 (10:51 -0700)]
iommu/iommu-debug: Set DOMAIN_ATTR_ATOMIC during profiling
There's some non-deterministic latency incurred by enabling clocks and
regulators. Clocks and regulators are left on for atomic domains, which
makes them a good choice for profiling the page table management code.
Use an atomic domain for map/unmap profiling to make things more
deterministic.
Change-Id: Ib9fa48699c9d6009dd3f3b322002e2fb0f2ff99c
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
Jeremy Gebben [Mon, 24 Aug 2015 19:33:51 +0000 (13:33 -0600)]
iommu/io-pgtable: don't flush the tlb in free_io_pgtable_ops()
The iommu domain must be detached before the pagetable is freed,
if any TLB maintenance is needed it should be done by
iommu_detach_device().
Change-Id: I119d0e127e122de1263c80c73d40ff6edf16c875
Signed-off-by: Jeremy Gebben <jgebben@codeaurora.org>
Jeremy Gebben [Mon, 24 Aug 2015 22:16:31 +0000 (16:16 -0600)]
iommu/arm-smmu: fix a DOMAIN_ATTR_DYNAMIC memory leak
The pagetable memory for a domain is freed by free_io_pgtable_ops(), which
is only called from arm_smmu_destroy_domain_context(). This function also
cleans up the hardware's context bank state and it is called during
iommu_device_detach(). Because dynamic domains don't have context bank
state, they don't call arm_smmu_destroy_domain_context() and thus their
pagetable memory was leaked.
Fix this by instead calling free_io_pgtable_ops() during
iommu_domain_destroy(). This ensures that the pagetable is freed for all
domains.
Change-Id: Id349fdc7df673cd4513468f23dcc95a9281d2657
Signed-off-by: Jeremy Gebben <jgebben@codeaurora.org>
Rohit Vaswani [Wed, 19 Aug 2015 00:57:44 +0000 (17:57 -0700)]
iommu/arm-smmu: Check the return type of map_sg and take appropriate action
We cannot call unmap directly if the map_sg fails partially as the tlb
invalidate functions need to enable/prepare clocks, which require
non-atomic context. Let map_sg return the failure and handle this when
we are out of the atomic context.
Change-Id: I6401c1e281850aeda27e32524cae34324045f762
Signed-off-by: Rohit Vaswani <rvaswani@codeaurora.org>
Mitchel Humpherys [Fri, 21 Aug 2015 21:08:40 +0000 (14:08 -0700)]
iommu/iommu-debug: Provide debugfs for iommu_reg_{read,write}
It can be useful to read and modify SMMU settings interactively. Add a
debugfs interface for the recently added iommu_reg_read and
iommu_reg_write functions for this purpose.
Change-Id: If3c9bf1833dc950c7ca1e2ce1a373bcaa31ebcf0
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
Mitchel Humpherys [Fri, 21 Aug 2015 21:07:59 +0000 (14:07 -0700)]
iommu/arm-smmu: Implement .reg_read and .reg_write
Implement the register reader and writer functions so that clients can
peek and poke their SMMU's registers.
Change-Id: I9dac59a16b42fec6b2a7639603289911f88b067d
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
Mitchel Humpherys [Fri, 21 Aug 2015 21:06:14 +0000 (14:06 -0700)]
iommu: Add iommu_reg_read and iommu_reg_write
It might be useful for IOMMU clients to peek and poke at their IOMMU's
registers, but knowing how to access those registers is really the job
of the IOMMU driver (it might need to enable specific clocks and
regulators, for example). Provide an API to read and write IOMMU
registers that can be implemented by the driver.
Change-Id: I5b2f19225f8bd258278780ff24b4ea96460857aa
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
Neeti Desai [Wed, 19 Aug 2015 22:19:25 +0000 (15:19 -0700)]
iommu/arm-smmu: Pass the correct size to be unmapped
For all but the last level of the page tables, we
need to call unmap with supported IOMMU pages sizes.
However currently to get the maximum page size that can be
unmapped, we are passing the original size parameter,
instead of the remaining size to be unmapped, and that can
end up unmapping more than what is intended.
Fix this by passing the remaining size parameter to
find the correct IOMMU page size.
Change-Id: Id4bd188ff6a3b4b7d34ba43ae6a61efb3c65b281
Signed-off-by: Neeti Desai <neetid@codeaurora.org>
Jeremy Gebben [Thu, 20 Aug 2015 18:49:02 +0000 (12:49 -0600)]
iommu/arm-smmu: fix arm_smmu_attach_dynamic error path
Call the correct function to free the io_pgtable_ops when
there is an error.
Change-Id: Ib01041f874a203b18265b46b56b375c76c0ab211
Signed-off-by: Jeremy Gebben <jgebben@codeaurora.org>
Mitchel Humpherys [Wed, 19 Aug 2015 18:41:20 +0000 (11:41 -0700)]
iommu/arm-smmu: Use the flag value when triggering faults
iommu_trigger_fault now passes a flag value when triggering faults. Use
that value instead of hard-coding a translation fault.
Change-Id: Iac9b97cebef99e260641bde70c1ddfaedd0be502
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
Mitchel Humpherys [Wed, 19 Aug 2015 19:13:28 +0000 (12:13 -0700)]
iommu/arm-smmu: Do a local halt around impl def programming
As per the hardware spec, the SMMU must be halted during programming of
the implementation defined registers. Add the missing halt.
Change-Id: Id28b546e5b5b646c263d7431a32f3c95ed3f2042
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
Mitchel Humpherys [Wed, 19 Aug 2015 18:39:23 +0000 (11:39 -0700)]
iommu/iommu-debug: Pass a flags value to iommu_trigger_fault
iommu_trigger_fault now accepts a flags argument which can be handled by
the individual IOMMU drivers. Take the value written to the debugfs
file by the user and pass it as the flags value.
Change-Id: I803e6585aea35fbb0f6e03e4279587b36a7bad92
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
Mitchel Humpherys [Wed, 19 Aug 2015 18:33:42 +0000 (11:33 -0700)]
iommu: Add flags parameter to .trigger_fault
IOMMU drivers might want more control over the types of faults being
triggered with iommu_trigger_fault. Add a flags parameter that can be
used to provide more control over the types of faults that can be
triggered.
Change-Id: I2f21b383437430e957ab52070d3575e8cb3dee90
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
Mitchel Humpherys [Wed, 19 Aug 2015 18:02:33 +0000 (11:02 -0700)]
iommu/arm-smmu: Pass external fault information to client handlers
A client fault handler flag was recently introduced to indicate external
faults. Use it.
Change-Id: Ib9ea59befc3c8324de4fc94a0dbe4acafaa6f9b8
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
Mitchel Humpherys [Wed, 19 Aug 2015 17:57:55 +0000 (10:57 -0700)]
iommu: Add client fault handler flag for external faults
Some IOMMUs (like the ARM SMMU) can raise "external faults" in addition
to translation and permission faults. Add an IOMMU fault flag so that
this fault type can be propagated to client fault handlers.
Change-Id: Ia476ed6c727f5f01011747f30b3a74ebf0fc07f7
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
Neeti Desai [Thu, 13 Aug 2015 00:54:44 +0000 (17:54 -0700)]
iommu/arm-smmu: Protect the lists used for assign/unassign with a spinlock
Simultaneous assign or unassign on the same domain during map and unmap
operations manipulate the same pte_info lists and thus can ask hypervisor
to assign the same pte address twice which results in double mapping error.
The lists(pte_info_list and unassign_list) needs to be locked before
traversing it for making the assign/unassign calls.
Change-Id: Iebe3d703d80918af4ab18208f4b5a010c4c14791
Signed-off-by: Neeti Desai <neetid@codeaurora.org>
Liam Mark [Sun, 16 Aug 2015 04:58:11 +0000 (21:58 -0700)]
iommu/io-pgtable: fix __arm_lpae_free_pgtable leak
When unmapping 2MB mappings, which are 2MB aligned, the smmu driver
is leaking the 3rd level page tables.
Fix this leak by updating __arm_lpae_free_pgtable so that it no
longer leaks leaf table entries.
To reproduce this leak simply map and unmap a non-block 2MB mapping
which is 2MB aligned.
Change-Id: Ibdbdb084ceb8d03ebe0a04e8777e3eb9419e9b87
Signed-off-by: Liam Mark <lmark@codeaurora.org>
Stepan Moskovchenko [Wed, 10 Jun 2015 03:23:04 +0000 (20:23 -0700)]
iommu/io-pgtable-arm: Optimize map by batching flushes
Currently, the page table is flushed after the installation of each
individual page table entry. This is not terribly efficient since
virtual address ranges are often mapped with physically contiguous
chunks of page table memory. Optimize the map operation by factoring
out the page table flushing so that contiguous ranges of page table
memory can be flushed in one go.
Change-Id: Ie80eb57ef50d253db6489a6f75824d4c746314c7
Signed-off-by: Stepan Moskovchenko <stepanm@codeaurora.org>
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
Signed-off-by: Neeti Desai <neetid@codeaurora.org>
Mitchel Humpherys [Mon, 17 Aug 2015 21:26:26 +0000 (14:26 -0700)]
iommu/io-pgtable-arm: Generate a fault on TTBR1 usage
TTBR1 shouldn't currently be used at all. Any such usage is the result
of a bug. Configure TCR such that any usage of TTBR1 will generate a
fault, which will help catch such bugs earlier.
Change-Id: I74f2dc9580e5aed5d391debe23c7f5cc9fc1f672
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
Chintan Pandya [Tue, 11 Aug 2015 09:48:15 +0000 (15:18 +0530)]
iommu: debug: add support for MSM iommu devices
MSM iommu devices' DT bindings defer from upstream
way. And hence, the debug driver doesn't support
to profile MSM iommu devices. Add that support.
Change-Id: I507d500785408b1d96f017648475933ee408b664
Signed-off-by: Chintan Pandya <cpandya@codeaurora.org>
Mitchel Humpherys [Fri, 31 Jul 2015 03:17:19 +0000 (20:17 -0700)]
iommu/arm-smmu: Assign page table memory on domain finalization
When a domain is attached for the first time, memory for its top level
of page tables is allocated. If it's a secure domain, this memory needs
to be assigned to a secure VMID. Currently, it's not being assigned to
a secure VMID, so when the domain is torn down and unassigned we get a
warning. Fix this by adding the missing assign call after initial
domain allocation.
Change-Id: I9ff2418749e768e17b4598f5f89e587ad7828d52
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
Mitchel Humpherys [Thu, 30 Jul 2015 20:04:22 +0000 (13:04 -0700)]
iommu/arm-smmu: Make arm_smmu_unprepare_pgtable atomic
Currently, arm_smmu_unprepare_pgtable makes a hypervisor call to
unassign the memory being freed up. However, this is a sleeping call
and arm_smmu_unprepare_pgtable can be called from atomic context. Fix
this by making arm_smmu_unprepare_pgtable atomic in a similar manner to
arm_smmu_prepare_pgtable (put the addresses to be unprepared on a list
and unprepare them at the end of unmap).
Change-Id: Ifba40ca3ceb4aaf328a1a570e0ee6c39088075ba
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
Mitchel Humpherys [Thu, 30 Jul 2015 20:41:12 +0000 (13:41 -0700)]
iommu/io-pgtable: Delegate page table memory management to drivers
Some IOMMU drivers (like arm-smmu) need to perform fixups to page table
memory when it's allocated and freed. Some callbacks were added for
this purpose ({prepare,unprepare}_pgtable) that are called from the
io-pgtable code after each allocation and before each free. However,
this approach is prone to bugs where new calls to allocate/free are
added without corresponding calls to the fixup callbacks. Furthermore,
allocation and free is often done from atomic context, so if the driver
needs to do non-atomic fixups during free they are out of luck since the
memory will be freed back to the system by the time control is turned
back over to the driver.
Adding yet another callback for non-atomic fixups would start to get
tedious, and would only increase the chance of missing callbacks as new
allocations/frees are added. Instead, fix this by allowing all page
table memory allocation and free to be delegated entirely to the driver.
Fall back to {alloc,free}_pages_exact by default if the driver doesn't
need any special handling of page table memory.
Change-Id: I0361bb81e25ff5ad4ef93a45330a35af47bc6013
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
Mitchel Humpherys [Fri, 31 Jul 2015 21:58:13 +0000 (14:58 -0700)]
iommu/io-pgtable-arm: Correct size in arm_lpae_split_blk_unmap
Currently, arm_lpae_split_blk_unmap is assuming that the size to be
remapped is the block size of the next level. However, optimizations
have been made to the unmap code that result in the entire remaining
size being passed in, rather than just doing it one block at a time,
which breaks arm_lpae_split_blk_unmap.
Fix this by overriding the size passed in to be the block size of the
next level.
Change-Id: Ifce5b2e07dc15aba3cd37b7ac249e00decd2923f
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
Mitchel Humpherys [Fri, 31 Jul 2015 02:40:21 +0000 (19:40 -0700)]
iommu/iommu-debug: Add secure_attach debugfs file
It can be useful to perform tests on secure domains. Add support for
this with a "secure_attach" file that will also set the
DOMAIN_ATTR_SECURE_VMID to the value written to the file.
Example usage:
# cd /sys/kernel/debug/iommu/tests/<device>
# echo 0xA > secure_attach
# echo 0x1000,0x1000,0x1000,1 > map
# echo 0x1000,0x1000 > unmap
# echo 0 > secure_attach
Change-Id: I0181e360f3e98e0e47a3af8c6adb71c9c4a3c9b5
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
Mitchel Humpherys [Fri, 31 Jul 2015 02:25:54 +0000 (19:25 -0700)]
iommu/iommu-debug: Show secure VMID in attach info
It can be useful to know a domain's secure VMID value. Add this
information to the attach info debugfs file.
Change-Id: Ic8a73000b8337639f95376b13ef58a0e2a089dde
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
Mitchel Humpherys [Fri, 31 Jul 2015 02:39:15 +0000 (19:39 -0700)]
iommu/arm-smmu: Add getter for DOMAIN_ATTR_SECURE_VMID
It can be useful to query a previously set secure VMID. Add a domain
attribute getter for this.
Change-Id: Iaf14201d321161ce4ccc69b674a3b1247d6a7e94
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
Neeti Desai [Tue, 28 Jul 2015 18:02:02 +0000 (11:02 -0700)]
msm: Update the permissions while making assign call
When memory that is assigned a particular VMID is re-assigned
back to HLOS, it needs to get RWX permissions so that HLOS can
use it for data as well as instructions.
Change-Id: Ib5413861de877d9081bdf5bd397f528293deb6e8
Signed-off-by: Neeti Desai <neetid@codeaurora.org>
Mitchel Humpherys [Wed, 29 Jul 2015 19:45:29 +0000 (12:45 -0700)]
iommu/iommu-debug: Fix parsing of unmap size
The unmap debugfs file write handler is currently incorrectly parsing
the size to be unmapped (storing the IOVA as the size). Fix this by
parsing from the correct offset (just after the first comma).
Change-Id: I438851882052acc511c713706bc4f0cbf373353e
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
Mitchel Humpherys [Tue, 14 Jul 2015 23:41:29 +0000 (16:41 -0700)]
iommu/io-pgtable-arm: Don't leave dangling table entries
Currently, when all of the 4K PTEs beneath a 2M table entry are
unmapped, that 2M table entry is left intact, even though it doesn't
point to any valid 4K mappings anymore. This results in a warning if a
subsequent block mapping lands on top of the dangling table entry, since
we require empty page table entries when we map. It also causes the
page at which that the stomped-on table was pointing to be leaked. Fix
this by keeping track of how many entries are currently mapped beneath a
table. When the map count goes to zero (in unmap), free up the page the
table is pointing at and zero out the table entry.
Change-Id: I470e6ffb2206a09fe7c24253e3fd64a744337a7f
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
Mitchel Humpherys [Thu, 16 Jul 2015 01:25:07 +0000 (18:25 -0700)]
iommu/io-pgtable: Add memory stats debugfs file
It can be useful for debugging to know how much memory is being used for
IOMMU page tables. Add some dedicated allocation functions and a
debugfs file for this.
Change-Id: Id69b4b1b5df5dcc6c604eec3a12a894b8eab0eb6
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
Mitchel Humpherys [Thu, 16 Jul 2015 01:27:36 +0000 (18:27 -0700)]
iommu: Create iommu debugfs directory from IOMMU code
Currently we're creating an "iommu" debugfs directory from the
iommu-debug code. Other IOMMU modules might want to make use of this
same directory, so create it from the IOMMU framework code itself.
Change-Id: I679fdfc34ba5fcbd927dc5981438c6fabcfa3639
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
Jeremy Gebben [Fri, 10 Jul 2015 22:43:23 +0000 (16:43 -0600)]
iommu/arm-smmu: add support for DOMAIN_ATTR_DYNAMIC
Implement support for dynamic domain switching. This feature is
only enabled when the qcom,dynamic device tree attribute for an smmu
instance.
In order to use dynamic domains, a non-dynamic domain must first
be created and attached. This non-dynamic domain must remain
attached while the device is in use.
All domains must be attached before calling any mapping functions, such as
iommu_map(). This allows the pagetable setup to be set up during attach
based on the hardware configuration for the smmu.
Before attaching a dynamic domain, the DOMAIN_ATTR_CONTEXT_BANK must be
set to indicate which context bank registers should be used for
any register programming. Attaching dynamic domain doesn't cause
hardware register programming, but mapping operations may cause
TLBI operations. Additionally, the upstream driver or hardware may
do other register programming.
Because the arm-smmu driver assigns context banks dynamically, the
upstream driver should query DOMAIN_ATTR_CONTEXT_BANK on its non-dynamic
domain, to ensure the correct value is used for all dynamic domains
created.
To switch domains dynamically, the upstream driver or hardware
must program the TTBR0 and CONTEXTIDR registers with the values
from the DOMAIN_ATTR_TTBR0 and DOMAIN_ATTR_CONTEXTIDR attributes
for the desired domain. The upstream driver may also need to do
other hardware specific register programming to properly
synchronize the domain switch. It must ensure that all register
state, except for CONTEXTIDR and TTBR0 registers, is restored
at the end of the domain switch operation.
DOMAIN_ATTR_PROCID may be set to any value for each domain
before it is attached. This value is part of the CONTEXTIDR
register, but it is not used by the SMMU hardware. Setting a unique
value for this attribute in every domain can be useful for debugging.
Change-Id: Ib92d06db06832700bdf56265b169ccddfb192698
Signed-off-by: Jeremy Gebben <jgebben@codeaurora.org>
Jeremy Gebben [Fri, 10 Jul 2015 22:43:23 +0000 (16:43 -0600)]
iommu: introduce DOMAIN_ATTR_DYNAMIC
Some devices can support per-process iommu domains, which are used
asynchronously by the hardware which directly updates to the TTBR0
and CONTEXTIDR registers.
Add DOMAIN_ATTR_DYNAMIC to do indicate a domain may be used dynamically.
This attribute must be set before attaching, as it changes what
domain and hardware configuration is done by the iommu driver.
Before attaching any dynamic domains, the driver must first attach
a non-dynamic domain to do initial hardware configuration.
A simplified example:
void use_domain(struct iommu_domain *domain, struct job *job)
{
u64 ttbr0;
u32 contextidr;
iommu_domain_get_attr(domain, DOMAIN_ATTR_TTBR0, &ttbr0);
iommu_domain_get_attr(domain, DOMAIN_ATTR_CONTEXTIDR, &contextidr);
/*
* Schedule work on the hardware, which is time sliced between
* clients. Each client has a separate iommu domain and when
* a job is run, the hardware first programs TTBR0 and
* CONTEXTIDR to use the appropriate domain.
*/
submit_job(ttbr0, contextidr, job);
}
void init(void)
{
struct iommu_domain *master_domain, *dyn_domain1, *dyn_domain2;
int dynamic = 1;
master_domain = iommu_domain_alloc(bus);
dyn_domain1 = iommu_domain_alloc(bus);
dyn_domain2 = iommu_domain_alloc(bus);
iommu_attach_device(base_domain, dev);
iommu_domain_set_attr(dyn_domain1, DOMAIN_ATTR_DYNAMIC, &dynamic);
iommu_domain_set_attr(dyn_domain2, DOMAIN_ATTR_DYNAMIC, &dynamic);
iommu_attach_device(dyn_domain1, dev);
iommu_attach_device(dyn_domain2, dev);
while (keep_going) {
iommu_map(dyn_domain1, ...);
iommu_map(dyn_domain2, ...);
use_domain(dyn_domain1, job1);
use_domain(dyn_domain2, job2);
iommu_unmap(dyn_domain1, ...);
iommu_unmap(dyn_domain2, ...);
}
iommu_detach_device(dyn_domain2, dev);
iommu_detach_device(dyn_domain1, dev);
iommu_detach_device(master_domain, dev);
}
Change-Id: Ic4fa0a831751eb4b10fff5d9aec28a411856fbd1
Signed-off-by: Jeremy Gebben <jgebben@codeaurora.org>
Jeremy Gebben [Fri, 10 Jul 2015 22:43:22 +0000 (16:43 -0600)]
iommu/arm-smmu: add support for TTBR0, CONTEXTIDR and PROCID attributes
Add fields to arm_smmu_cfg to store asid, vmid, and procid so that they
can queried easily.
Program the CONTEXTIDR register during attach so that the PROCID
attribute functions as expected.
Change-Id: I2e2b3926fbf021754e89edda9a6857d2e3a7138b
Signed-off-by: Jeremy Gebben <jgebben@codeaurora.org>
Jeremy Gebben [Fri, 10 Jul 2015 22:43:22 +0000 (16:43 -0600)]
iommu: introduce TTBR0, CONTEXTIDR, and PROCID domain attributes
In the ARM SMMU architecture, pagetable programming is controlled
by the TTBR0 and CONTEXTIDR registers. The layout of these
registers varies depending on the pagetable format in use.
In particular, the ASID (address space ID) field is found in
CONTEXTIDR when using V7S format and in the top bits of TTBR0
for V7L and V8L.
Some drivers need to program hardware to switch domains on the
fly. These attributes allow the correct settings to be determined
by querying the domain rather than directly reading registers and
making assumptions about the pagetable format. The domain must be
attached before TTBR0 and CONTEXTIDR may be queried.
The PROCID attribute allows driver set a debug field in the
CONTEXTIDR register. This attribute may only be set before
attaching, but may be queried at any time. The SMMU hardware
doesn't use the contents of this field, but debug can be simpler
if each domain stores a unique value in it.
Change-Id: I175aa78fee02c3e4e0071496d9cc2b8841ff9e3c
Signed-off-by: Jeremy Gebben <jgebben@codeaurora.org>
Mitchel Humpherys [Tue, 14 Jul 2015 21:30:33 +0000 (14:30 -0700)]
iommu/iommu-debug: Don't add debugfs entries until we init
The attachment tracking code adds a node to debugfs every time a client
attaches a domain to an IOMMU device. The problem is that clients can
start making those attachments during early boot, before iommu-debug
initializes (including setting up the root debugfs directory for the
attachment tracking), which means they get stuck in the root debugfs
directory. Trying to initialize iommu-debug earlier than all possible
IOMMU clients is tricky, so fix this by only installing debugfs entries
onces we've initialized, installing debugfs entries for any early
attachments at initialization time.
Change-Id: I8364015346105187e0c8f787fc2b4155d72b3584
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
Mitchel Humpherys [Thu, 9 Jul 2015 23:55:08 +0000 (16:55 -0700)]
iommu/iommu-debug: Add debugfs file to trigger context faults
It can be useful during development to trigger faults. Add a debugfs
file to do so.
Change-Id: Ic7b304ef0d908ebd506979f0c87189e34d7dfc67
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
Mitchel Humpherys [Fri, 10 Jul 2015 00:26:15 +0000 (17:26 -0700)]
iommu/arm-smmu: Implement .trigger_fault
In order to facilitate debugging and development, implement the
.trigger_fault method from iommu_ops. This can be done on ARM SMMUv2 by
writing to the FSRRESTORE register. Do it.
Change-Id: Ia8339b54fbb9263d8cf23ff61c4615122316729a
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
Mitchel Humpherys [Thu, 9 Jul 2015 23:59:02 +0000 (16:59 -0700)]
iommu: Add iommu_trigger_fault
It can be useful to trigger an IOMMU fault during development and
debugging. Add support to the IOMMU framework to do so.
Change-Id: I908c9f5b52c6abe937f031de546d290027ba64b5
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
Mitchel Humpherys [Thu, 9 Jul 2015 22:55:02 +0000 (15:55 -0700)]
iommu/io-pgtable-arm: Use non-cacheable page table memory
Requests from the walker to page table memory are currently inner- and
outer-cacheable. This configuration hasn't been fully validated for
functionality or characterized for performance. Configure these
requests as non-cacheable.
Change-Id: I7efb0a697faff68a67ee0afdb933b6dd6926f30a
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
Mitchel Humpherys [Thu, 9 Jul 2015 22:29:35 +0000 (15:29 -0700)]
iommu/arm-smmu: Disable coherent table walk by default
Coherent hardware table walking hasn't been properly validated or
characterized, so all clients are expected to disable it. This is a
recipe for disaster since it could be easy for a new client to come
along without knowing that they need to disable it. Just disable it by
default. Clients can always explicitly enable it in the future if it's
found to be beneficial.
Change-Id: I4badfe33e815a6ba7b25507f5dd5a42f68d4bfa6
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
Mitchel Humpherys [Thu, 9 Jul 2015 22:24:57 +0000 (15:24 -0700)]
iommu/iommu-debug: Add COHERENT_HTW_DISABLE to attach info
It can be useful to check whether or not coherent hardware table walking
has been explicitly disabled on attached domains. Add this to the
attach info debugfs file.
Change-Id: I432303ecb734d32eaa02038694daad0d8c4d8aba
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
Mitchel Humpherys [Thu, 9 Jul 2015 22:02:03 +0000 (15:02 -0700)]
iommu/iommu-debug: Move attachment info file to subdirectory
Currently IOMMU attachment info is available in debugfs files located at
<debugfs_root>/iommu/attachments/<attached_device>. However, there are
more actions that can be taken on attached devices besides just printing
their info. Make room for more debugfs files for attached devices by
creating a directory for each one, and move the existing info file to:
<debugfs_root>/iommu/attachments/<attached_device>/info.
Change-Id: Ia56efc3aeb5e82afc34314fe48aaa0cd6e5579be
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
Mitchel Humpherys [Thu, 9 Jul 2015 21:50:22 +0000 (14:50 -0700)]
iommu/iommu-debug: Initialize debug device to 0
Currently the debug device structure is allocated with kmalloc, without
initializing all of the fields in the structure. Later, those fields
might be uses before they've every been assigned. For example, if a user
executes the following code on a fresh boot:
# cd /sys/kernel/debug/iommu/tests/some_device
# echo 0 > attach
The kernel crashes with something like this (assuming page poisoning is
enabled):
Unable to handle kernel paging request at virtual address
aaaaaaaaaaaaaaaa
pgd =
ffffffc0a92a1000
[
aaaaaaaaaaaaaaaa] *pgd=
0000000000000000, *pud=
0000000000000000
Fix this by initializing all the fields in the structure to 0 by using
kzalloc instead of kmalloc.
Change-Id: I3514bf7bf174e176ff7a310c7134d0f53e22d771
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
Mitchel Humpherys [Mon, 6 Jul 2015 22:21:24 +0000 (15:21 -0700)]
iommu/iommu-debug: Add translation and mapping test files
Running ATOS commands on custom mappings is a useful tool for debugging.
Add a new debugfs file for interactively attaching, detaching, mapping,
unmapping, and issuing ATOS commands.
Example usage:
# cd /sys/kernel/debug/iommu/tests/soc:qcom,msm-audio-ion
# echo 1 > attach
# echo 0x1000,0x5000,0x1000,1 > map
# echo 0x1008 > atos
# cat atos
0x5008
# echo 0x2000 > atos
# cat atos
FAIL
# echo 0 > attach
Change-Id: I596cd3f05fcb59653e2acddc17d175855a1eb9a1
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
Mitchel Humpherys [Mon, 6 Jul 2015 21:58:23 +0000 (14:58 -0700)]
iommu/iommu-debug: Rename profiling device functions
The functions for iommu/devices/<device>/profiling don't actually have
the word `profiling' in the name, which will be confusing as we add more
files to that directory. Rename them for clarity.
Change-Id: Ic57d9400d8784d2cbd667185c5b2b0e1275461dd
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
Mitchel Humpherys [Mon, 6 Jul 2015 20:58:05 +0000 (13:58 -0700)]
iommu/arm-smmu: Do an ATOS in the fault handler
In order to facilitate debugging of context faults, the result of a
software table walk is currently printed as part of the fault handling.
However, there are certain bugs and error conditions that can't be
caught by doing a software table walk (like missing TLB invalidation or
misbehaving hardware). Add a hardware table walk (via ATOS) to improve
debugging capabilities.
Change-Id: Ie89019df62f115627359e29b1f6cc5de3a36d1b5
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>