OSDN Git Service
Mitchel Humpherys [Fri, 10 Jul 2015 00:26:15 +0000 (17:26 -0700)]
iommu/arm-smmu: Implement .trigger_fault
In order to facilitate debugging and development, implement the
.trigger_fault method from iommu_ops. This can be done on ARM SMMUv2 by
writing to the FSRRESTORE register. Do it.
Change-Id: Ia8339b54fbb9263d8cf23ff61c4615122316729a
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
Mitchel Humpherys [Thu, 9 Jul 2015 23:59:02 +0000 (16:59 -0700)]
iommu: Add iommu_trigger_fault
It can be useful to trigger an IOMMU fault during development and
debugging. Add support to the IOMMU framework to do so.
Change-Id: I908c9f5b52c6abe937f031de546d290027ba64b5
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
Mitchel Humpherys [Thu, 9 Jul 2015 22:55:02 +0000 (15:55 -0700)]
iommu/io-pgtable-arm: Use non-cacheable page table memory
Requests from the walker to page table memory are currently inner- and
outer-cacheable. This configuration hasn't been fully validated for
functionality or characterized for performance. Configure these
requests as non-cacheable.
Change-Id: I7efb0a697faff68a67ee0afdb933b6dd6926f30a
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
Mitchel Humpherys [Thu, 9 Jul 2015 22:29:35 +0000 (15:29 -0700)]
iommu/arm-smmu: Disable coherent table walk by default
Coherent hardware table walking hasn't been properly validated or
characterized, so all clients are expected to disable it. This is a
recipe for disaster since it could be easy for a new client to come
along without knowing that they need to disable it. Just disable it by
default. Clients can always explicitly enable it in the future if it's
found to be beneficial.
Change-Id: I4badfe33e815a6ba7b25507f5dd5a42f68d4bfa6
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
Mitchel Humpherys [Thu, 9 Jul 2015 22:24:57 +0000 (15:24 -0700)]
iommu/iommu-debug: Add COHERENT_HTW_DISABLE to attach info
It can be useful to check whether or not coherent hardware table walking
has been explicitly disabled on attached domains. Add this to the
attach info debugfs file.
Change-Id: I432303ecb734d32eaa02038694daad0d8c4d8aba
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
Mitchel Humpherys [Thu, 9 Jul 2015 22:02:03 +0000 (15:02 -0700)]
iommu/iommu-debug: Move attachment info file to subdirectory
Currently IOMMU attachment info is available in debugfs files located at
<debugfs_root>/iommu/attachments/<attached_device>. However, there are
more actions that can be taken on attached devices besides just printing
their info. Make room for more debugfs files for attached devices by
creating a directory for each one, and move the existing info file to:
<debugfs_root>/iommu/attachments/<attached_device>/info.
Change-Id: Ia56efc3aeb5e82afc34314fe48aaa0cd6e5579be
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
Mitchel Humpherys [Thu, 9 Jul 2015 21:50:22 +0000 (14:50 -0700)]
iommu/iommu-debug: Initialize debug device to 0
Currently the debug device structure is allocated with kmalloc, without
initializing all of the fields in the structure. Later, those fields
might be uses before they've every been assigned. For example, if a user
executes the following code on a fresh boot:
# cd /sys/kernel/debug/iommu/tests/some_device
# echo 0 > attach
The kernel crashes with something like this (assuming page poisoning is
enabled):
Unable to handle kernel paging request at virtual address
aaaaaaaaaaaaaaaa
pgd =
ffffffc0a92a1000
[
aaaaaaaaaaaaaaaa] *pgd=
0000000000000000, *pud=
0000000000000000
Fix this by initializing all the fields in the structure to 0 by using
kzalloc instead of kmalloc.
Change-Id: I3514bf7bf174e176ff7a310c7134d0f53e22d771
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
Mitchel Humpherys [Mon, 6 Jul 2015 22:21:24 +0000 (15:21 -0700)]
iommu/iommu-debug: Add translation and mapping test files
Running ATOS commands on custom mappings is a useful tool for debugging.
Add a new debugfs file for interactively attaching, detaching, mapping,
unmapping, and issuing ATOS commands.
Example usage:
# cd /sys/kernel/debug/iommu/tests/soc:qcom,msm-audio-ion
# echo 1 > attach
# echo 0x1000,0x5000,0x1000,1 > map
# echo 0x1008 > atos
# cat atos
0x5008
# echo 0x2000 > atos
# cat atos
FAIL
# echo 0 > attach
Change-Id: I596cd3f05fcb59653e2acddc17d175855a1eb9a1
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
Mitchel Humpherys [Mon, 6 Jul 2015 21:58:23 +0000 (14:58 -0700)]
iommu/iommu-debug: Rename profiling device functions
The functions for iommu/devices/<device>/profiling don't actually have
the word `profiling' in the name, which will be confusing as we add more
files to that directory. Rename them for clarity.
Change-Id: Ic57d9400d8784d2cbd667185c5b2b0e1275461dd
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
Mitchel Humpherys [Mon, 6 Jul 2015 20:58:05 +0000 (13:58 -0700)]
iommu/arm-smmu: Do an ATOS in the fault handler
In order to facilitate debugging of context faults, the result of a
software table walk is currently printed as part of the fault handling.
However, there are certain bugs and error conditions that can't be
caught by doing a software table walk (like missing TLB invalidation or
misbehaving hardware). Add a hardware table walk (via ATOS) to improve
debugging capabilities.
Change-Id: Ie89019df62f115627359e29b1f6cc5de3a36d1b5
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
Mitchel Humpherys [Mon, 6 Jul 2015 20:53:51 +0000 (13:53 -0700)]
iommu/arm-smmu: Resurrect hardware iova-to-phys
The hardware address translation operations (ATOS) can be useful for
debugging. arm-smmu used to have support for ATOS, but it was ripped
out while moving to the io-pgtable framework. Resurrect the old ATOS
code with the following modifications:
- Remove errata workarounds for deprecated hardware.
- Move the atos lock to a spinlock (since the only reason a mutex was
being used previously was due to the fact that some of the old
errata workarounds required sleeping operations).
Change-Id: I1a021026b9ee41ba2c1761bd5d5b7a13399c6417
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
Mitchel Humpherys [Mon, 6 Jul 2015 22:24:22 +0000 (15:24 -0700)]
iommu: Add iommu_iova_to_phys_hard
Some IOMMU hardware implementations provide hardware translation
operations that can be useful during debugging and development. Add a
function for this purpose along with an associated op in the iommu_ops
structure so that drivers can implement it.
Change-Id: I54ad5df526cdce05f8e04206a4f01253b3976b48
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
Mitchel Humpherys [Fri, 10 Jul 2015 18:56:56 +0000 (11:56 -0700)]
iommu/iommu-debug: Make attachment directories unique
Currently, the device name for the SMMU context bank device is used as
the filename for the IOMMU debug info file. This doesn't work in cases
where multiple domains can be attached to a single SMMU context bank
device (like dynamic domains). Make these filenames unique by appending
a 16-byte uuid to the name.
Change-Id: Ie26ece773bfa2e8c75a329a8cb8461bcd598218e
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
Jordan Crouse [Tue, 7 Jul 2015 18:55:48 +0000 (12:55 -0600)]
iommu: io-pgtable-arm: Fix unmap optimization for pagetable boundaries
The last level optimization for __arm_lpae_unmap assumes that
consecutive blocks of 2MB addresses are located in the same
1GB mapping but this isn't always true - if the address spans
a 1GB boundary the next iova is in a different pagetable.
Only perform the optimization for the current pagetable entry and
then kick it back for the loop in arm_lpae_unmap to try again
with the updated iova. All this means that __arm_lpae_unmap may
not unmap all of the size that it was given. This is okay assuming
at least something was unmapped so don't jump out of the loop
in arm_lpae_unmap until the child function returns 0 or the entire
block is freed, whichever comes first.
CRs-Fixed: 867143
Change-Id: Ic0dedbad407d60365a95afdaf03ec3b91f53960d
Signed-off-by: Jordan Crouse <jcrouse@codeaurora.org>
Rohit Vaswani [Tue, 7 Jul 2015 00:46:35 +0000 (17:46 -0700)]
arm64: mm: dma: Use dma attribute to allow executeable mappings
The default mapping type is non-executeable. Check for the
DMA_ATTR_EXEC_MAPPING attribute which allows clients to
request an executeable mapping.
Change-Id: I24a170990cc04a848b6779871ae2025721177d46
Signed-off-by: Rohit Vaswani <rvaswani@codeaurora.org>
Rohit Vaswani [Tue, 7 Jul 2015 00:45:20 +0000 (17:45 -0700)]
common: DMA-mapping: Add EXEC_MAPPING attribute
DMA_ATTR_EXEC_MAPPING specifies that an executable mapping
should be created for the requested buffer. By default, the
DMA mappings are non-executable.
Change-Id: I135077e14996e92fa9d199bdee043c443db48924
Signed-off-by: Rohit Vaswani <rvaswani@codeaurora.org>
Rohit Vaswani [Thu, 7 May 2015 00:02:07 +0000 (17:02 -0700)]
iommu: msm: Provide the IOMMU_NOEXEC flag explicitly during mapping
The logic for the iommu executable flag is inverted now and
all the iommu mappings are executable by default.
Provide the IOMMU_NOEXEC flag where the mapping needs to be non-executable.
Change-Id: Ifa0aa3d17ae79c16abdf66d2177a09b868a9f45f
Signed-off-by: Rohit Vaswani <rvaswani@codeaurora.org>
[pdaly@codeaurora.org Remove gpu/mdss]
Signed-off-by: Patrick Daly <pdaly@codeaurora.org>
Rohit Vaswani [Mon, 6 Jul 2015 23:27:43 +0000 (16:27 -0700)]
iommu: msm: Allow passing dma_attrs for lazy mapping
The msm lazy mapping APIs did not allow to pass in
dma attributes that could be passed to the dma-mapping
driver. This patch allows users to specify dma attributes for the
msm lazy mappings.
Change-Id: I3e4cd2bb99d205dce78083a256f4d444d865f3cc
Signed-off-by: Rohit Vaswani <rvaswani@codeaurora.org>
Rohit Vaswani [Mon, 6 Jul 2015 23:22:29 +0000 (16:22 -0700)]
common: DMA-mapping: add NO_DELAYED_UNMAP attribute
DMA_ATTR_NO_DELAYED_UNMAP specifies to the msm lazy mapping
driver that this buffer should be immediately unmapped once
it is freed.
Change-Id: I43e6a6058705502cf91bf5f0c530c3099cba06ae
Signed-off-by: Rohit Vaswani <rvaswani@codeaurora.org>
Mitchel Humpherys [Tue, 30 Jun 2015 23:12:38 +0000 (16:12 -0700)]
arm64: dma-mapping: remove order parameter from arm_iommu_create_mapping()
arm32 recently removed the `order' parameter from
arm_iommu_create_mapping: (
68efd7d2fb32c: arm: dma-mapping: remove order
parameter from arm_iommu_create_mapping()) in order to make the API
easier to understand. The arm32 DMA IOMMU mapper has dynamic resizing
of the iova bitmap, so there was no reason to keep the `order' parameter
around (which was introduced to reduce the size of the bitmap).
Although we don't have dynamic iova bitmap reallocation on arm64, we'd
still like to get rid of the `order' parameter since it's confusing and
doesn't really help much (especially since all known clients on our
system are passing order=0). Remove it.
Change-Id: I35e32fdfbe05ec434f64a3a316d13c8f43304bc6
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
[pdaly@codeaurora.org Remove gpu/ipa etc modifications]
Signed-off-by: Patrick Daly <pdaly@codeaurora.org>
Matt Wagantall [Tue, 30 Jun 2015 01:53:20 +0000 (18:53 -0700)]
arm64: dma-mapping: fix attribs for non-coherent strongly-ordered mappings
An incorrect conflict resolution when picking up
07c17f4e0b71d ("arm64:
add support for NO_KERNEL_MAPPING and STRONGLY_ORDERED") caused
MT_NORMAL_NC attributes being set for strongly-ordered mappings, rather
than MT_DEVICE_nGnRnE attributes.
As a result of this, speculative data fetches of the incorrectly-
mapped memory have been observed. In cases where this memory is
XPU protected or reads result in side-effects, this may result in
device crashes.
Fix this by setting the attributes returned by pgprot_noncached()
regardless of the value of the coherent flag passed to
__get_dma_pgprot()
Change-Id: Iec56027e280ae0920016df3066045b71299a915b
Signed-off-by: Matt Wagantall <mattw@codeaurora.org>
Mitchel Humpherys [Tue, 23 Jun 2015 23:29:16 +0000 (16:29 -0700)]
iommu: Add debugging infrastructure
Currently, debugging IOMMU issues is done via manual instrumentation of
the code or via low-level techniques like using a JTAG debugger.
Introduce a set of library functions and debugfs files to facilitate
interactive debugging and testing.
This patch introduces the basic infrastructure as well an initial
debugfs for:
- viewing IOMMU attachments (domain->dev mappings created by
iommu_attach_device) and resulting attributes (like the base address
of the page tables)
- basic performance profiling
Example usage:
# cd /sys/kernel/debug/iommu/attachments
# cat b40000.qcom,kgsl-iommu:iommu_kgsl_cb2
Domain: 0xffffffc0cb983f00
PT_BASE_ADDR: virt=0xffffffc057eca000 phys=0x00000000d7eca000
# cd /sys/kernel/debug/iommu/tests
# cat soc:qcom,cam_smmu:msm_cam_smmu_cb1/profiling
size iommu_map iommu_unmap
4K 47 us 909 us
64K 97 us 594 us
2M 1536 us 605 us
12M 8737 us 1193 us
20M 26517 us 1121 us
size iommu_map_sg iommu_unmap
64K 31 us 656 us
2M 885 us 600 us
12M 2674 us 687 us
20M 4352 us 1096 us
Change-Id: I1c301eec6e64688831cad80ffd0380743f7f0df6
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
Mitchel Humpherys [Sat, 27 Jun 2015 02:11:53 +0000 (19:11 -0700)]
iommu/arm-smmu: Protect against concurrent attach from different domains
Currently we're relying on the smmu_domain->lock for synchronizing
attach and detach. This is a problem because each domain has its own
smmu_domain->lock, so if multiple different domains try to attach to the
same device at the same time, they'll be racing.
Fix the race by holding a lock that's part of the smmu
structure (attach_lock should do just fine).
The test case that uncovered this was:
# cd /sys/kernel/debug/iommu/tests/soc:qcom,msm-audio-ion/
# while :; do cat profiling; done &
# while :; do cat profiling; done &
Change-Id: I8a60cdc214c91967aff63882e3a7280865ffda9e
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
Mitchel Humpherys [Sat, 27 Jun 2015 01:24:16 +0000 (18:24 -0700)]
iommu/arm-smmu: Add missing unlock in attach error path
We currently have an error path in arm_smmu_attach_dev where we're
returning with a mutex locked. Fix this.
Change-Id: I197edea7cefe361027cf46e22313ebe844684ec8
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
Mitchel Humpherys [Fri, 26 Jun 2015 01:17:15 +0000 (18:17 -0700)]
iommu/arm-smmu: Poll TLBSTATUS with a tight loop
Currently we're using a 10us delay while polling the TLB status register
after doing a TLB operation. These operations almost always finish on
the first iteration or two, so the delay is unnecessary. Just do a
tight poll.
Change-Id: I7d5787ea92e227ded5a0578c1c647e8317c8ceca
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
Mitchel Humpherys [Fri, 19 Jun 2015 22:00:14 +0000 (15:00 -0700)]
iommu/arm-smmu: Use context bank TLBSTATUS registers
There are TLBSTATUS registers in SMMU global register space as well as
context bank register space. Currently we're polling the global
TLBSTATUS registers after TLB invalidation, even when using the TLB
invalidation registers from context bank address space. This violates
the usage model described in the ARM SMMU spec. Fix this by polling
context bank TLBSTATUS registers for context bank TLB operations, and
global TLBSTATUS registers for global TLB operations.
Change-Id: I8aa916f7bc71793cad4c9224aa85d5310eacec75
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
Neeti Desai [Thu, 25 Jun 2015 01:20:51 +0000 (18:20 -0700)]
iommu/arm-smmu: Fix size parameter in .unprepare_pgtable()
The size parameter in .unprepare_pgtable() and arm_smmu_assign_table()
needs to be the same since the functions are complimentary.
Use PAGE_SIZE in both functions instead of relying on the calling
function for .unprepare_pgtable().
Change-Id: Ic6fade307360254329968e1b4548732d045b8205
Signed-off-by: Neeti Desai <neetid@codeaurora.org>
Abhimanyu Kapur [Tue, 23 Jun 2015 22:33:01 +0000 (15:33 -0700)]
iommu: msm: surround the msm_dma_iommu apis with iommu ifdef
Make all msm_dma_iommu apis depend on CONFIG_IOMMU_API as it is
only used when we have the linux iommu layer available.
Change-Id: I879dc1a9174d498b9b4bc68b2418165f3b2675a3
Signed-off-by: Abhimanyu Kapur <abhimany@codeaurora.org>
Liam Mark [Fri, 19 Jun 2015 17:14:31 +0000 (10:14 -0700)]
iommu/arm-smmu: lock clock enabling and reference counting
Ensure that clock enabling and reference counting is done atomically
to avoid any potential race conditions.
An example of a potential race condition is that while thread one is
enabling the clocks thread two could enter and then exit the clock
enabling function early because of reference counting. This could
lead to thread two attempting to access registers before the clocks
are enabled.
Have removed the regulator reference because enabling the regulators
involves the use of a mutex so spin locks cannot be used to protect
the reference count. Also the use of a regulator reference count is
of limited benefit since there is only one regulator to enable.
Change-Id: I7215bbf9157907fde24c94841e347370769423c8
Signed-off-by: Liam Mark <lmark@codeaurora.org>
Jeremy Gebben [Tue, 16 Jun 2015 16:59:29 +0000 (10:59 -0600)]
iommu/arm-smmu: add support for DOMAIN_ATTR_CONTEXT_BANK
Because the ARM SMMU driver assigns context banks dynamically,
some drivers need a way to know which one they are using.
Change-Id: Ic0dedbad4327ef86c5a893a48b57f0f9417800e9
Signed-off-by: Jordan Crouse <jcrouse@codeaurora.org>
Signed-off-by: Jeremy Gebben <jgebben@codeaurora.org>
Jeremy Gebben [Tue, 16 Jun 2015 16:54:01 +0000 (10:54 -0600)]
iommu: introduce DOMAIN_ATTR_CONTEXT_BANK
After attaching a domain, this attribute may be
queried to determine which hardware context bank
was assigned.
Change-Id: I31e674672041103007fcaff3f83a0cc2c33a4a6d
Signed-off-by: Jeremy Gebben <jgebben@codeaurora.org>
Jeremy Gebben [Tue, 16 Jun 2015 18:56:45 +0000 (12:56 -0600)]
iommu/arm-smmu: hold init_mutex in iommu_domain_get_attr
Some attributes are changed during domain attach and detach,
so hold init_mutex to ensure consistency.
Change-Id: I450a9a2da4bfe3616ef6dd0a6426271d25c292ce
Signed-off-by: Jeremy Gebben <jgebben@codeaurora.org>
Jeremy Gebben [Tue, 16 Jun 2015 18:45:31 +0000 (12:45 -0600)]
iommu/arm-smmu: Tear down domain context if attach fails
Currently we're leaving domains half-initialized after a
partially-successful attach. Fix this by destroying the
domain in the error path.
Change-Id: I36c529ed4974c01fba96088b6f57a8e82b350252
Signed-off-by: Jeremy Gebben <jgebben@codeaurora.org>
Mitchel Humpherys [Mon, 15 Jun 2015 22:29:07 +0000 (15:29 -0700)]
iommu/arm-smmu: Remove domain lock variable
The `lock' field of struct arm_smmu_domain was replaced by `init_mutex'
in
9725ec12d27e215 (iommu/arm-smmu: re-use the init_mutex for protecting
smmu_domain.smmu), but the `lock' field itself was not deleted. It's
not meant to be used anymore, so delete it. Some usages of the crufty
lock have also crept up, so fix those as well.
Change-Id: I33c2f83e7b15f0ec2cb08c784a84991a7c57950f
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
Shiraz Hashim [Fri, 5 Jun 2015 05:26:14 +0000 (10:56 +0530)]
of: update reserved-memory document to include removed-dma-pool
removed-dma-pool compatibility creates a carve out region,
include the documentation for the same.
Change-Id: I6c2cd9a9dac4afab106b0d4d49db4cd51c63fbf7
Signed-off-by: Shiraz Hashim <shashim@codeaurora.org>
Neeti Desai [Sat, 6 Jun 2015 01:44:53 +0000 (18:44 -0700)]
iommu/arm-smmu: Add support for page table donation
For secure domains, the page tables need to be assigned
to the correct VMIDs. Add support for doing the assignment.
Change-Id: I60009ef96ae1c8965d57096b6a1b0658ae6acc9a
Signed-off-by: Neeti Desai <neetid@codeaurora.org>
Neeti Desai [Sat, 6 Jun 2015 01:44:24 +0000 (18:44 -0700)]
msm: secure_buffer: Update the hyp_assign_phys() api
The hyp_assign_phys() api can be called by different
usecases where it is not guaranteed that the source vm is
always VMID_HLOS.
Pass the responsibility of setting the source_vm to
caller of the function.
Change-Id: I3851a6681f49d4bb6fa5b7a889a16a158497e9e6
Signed-off-by: Neeti Desai <neetid@codeaurora.org>
Mitchel Humpherys [Wed, 17 Jun 2015 20:56:13 +0000 (13:56 -0700)]
Revert "iommu: io-pgtable-arm: set page tables as outer shareable"
This reverts commit
0c78cf6e138f ("iommu: io-pgtable-arm: set page
tables as outer shareable"). We actually don't want outer-shareable
since we'd like to disable coherent table walking.
Change-Id: Id38e931864c4c1a0d77bb06d0da231b546bedf6d
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
Sushmita Susheelendra [Tue, 2 Jun 2015 21:46:24 +0000 (15:46 -0600)]
iommu/arm-smmu: Pass pagefault information to client handlers
Fill out the pagefault flags more fully when calling
client fault handlers so that clients do not have to
read hardware state themselves. Also, respect the -EBUSY
return code from the client by not clearing the FSR or
resuming/terminating the stalled transaction.
Change-Id: I03a2546e8f90a1fa937ccd31bdd062fa05d76adb
Signed-off-by: Sushmita Susheelendra <ssusheel@codeaurora.org>
Sushmita Susheelendra [Tue, 2 Jun 2015 21:46:24 +0000 (15:46 -0600)]
iommu: Improve client fault handling
On some platforms, certain IOMMU hardware state
might be cleared before reaching client fault handlers,
making it difficult for client fault handlers to do much
useful processing. Add some flags so that this information
can be passed to client fault handlers directly, rather
than expecting clients to read the hardware themselves.
Also provide a mechanism for client fault handlers to
request that the IOMMU driver not clear the fault or
resume/retry the faulting transaction. The client fault
handler can return -EBUSY to request this behavior.
Change-Id: I9780beb52b4257fff99d708a493173c9fe0a9d8a
Signed-off-by: Sushmita Susheelendra <ssusheel@codeaurora.org>
Shiraz Hashim [Fri, 11 Jul 2014 18:03:53 +0000 (13:03 -0500)]
iommu: msm: drop upstream iommu implementation
Drop upstream iommu implementation as it is very old and has
conflicting file names that needs to be replaced by internal
one.
Change-Id: I7b61e6b5d2ce2a47b2b13c71c321dea62be940a0
Signed-off-by: Shiraz Hashim <shashim@codeaurora.org>
Shalaj Jain [Tue, 3 Mar 2015 21:34:59 +0000 (13:34 -0800)]
iommu/arm-smmu: Print SID info on context faults
When a context fault occurs, it can be useful for debugging to know the
stream ID of the faulting transaction. This information is available in
the CBFRSYNRAn register. Read and print the SID value when a context
fault occurs.
Change-Id: If8b47b801bc72a053b1198767de58799606ca626
Signed-off-by: Shalaj Jain <shalajj@codeaurora.org>
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
Liam Mark [Wed, 10 Jun 2015 16:15:56 +0000 (09:15 -0700)]
iommu/arm-smmu: ref count clocks and regulators
Use reference counting when enabling and disabling clocks and
regulators in order to improve performance.
Change-Id: I89a3eec17fd551f0625da8a504634f0df311d64f
Signed-off-by: Liam Mark <lmark@codeaurora.org>
Mitchel Humpherys [Tue, 2 Jun 2015 22:22:00 +0000 (15:22 -0700)]
Revert "iommu: io-pgtable-arm: flush tlb for stale mappings"
This reverts commit
713d52a0acca ("iommu: io-pgtable-arm: flush tlb for
stale mappings"), which was a workaround for some other bugs in the page
table mapping code. Those bugs have since been fixed, so the workaround
is no longer needed.
Change-Id: Ic699328dd60baffd1c6080e6b0d9b2fb0feea831
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
Mitchel Humpherys [Tue, 2 Jun 2015 17:52:28 +0000 (10:52 -0700)]
iommu/arm-smmu: Don't enable clocks for map
We don't need to enable clocks during map because we don't need to do
anything through hardware (unlike unmap, which needs to do TLB
invalidation). We had to enable clocks at one point in order to enable
a workaround for some software bugs in the page table code. These
workarounds are no longer present, so we don't need to enable clocks.
Rip out the clock/regulator enablement.
This seems to improve the performance of iommu_map by several orders of
magnitude. The performance impact on iommu_map_sg is smaller, maybe a
percent or two.
Change-Id: Iddf530bc35f96840413a5c46ad9ead5334b9abd1
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
Mitchel Humpherys [Mon, 1 Jun 2015 23:12:26 +0000 (16:12 -0700)]
iommu: io-pgtable-arm: Improve coverage of arm_lpae_range_has_mapping
The arm_lpae_range_has_mapping currently checks if there are any
non-mapped slots in a given iova range, but it's really meant to check
if there is *any* mapping whatsoever in a given iova range. Fix this.
Change-Id: I90e426ab157cc194328b754ac5021051ac883603
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
Mitchel Humpherys [Mon, 1 Jun 2015 23:10:20 +0000 (16:10 -0700)]
iommu: io-pgtable-arm: Only suppress map failures during map tests
We're currently suppressing all map failures during the entirety of the
selftests. We really only want to suppress those failures during
individual negative map test cases to avoid logspam, but we *do* want to
see other map failures during the selftests. Fix this by only
suppressing map failures during negative map test cases.
Change-Id: If51a95dd4d8c5b756cfa4597a5bdd7c75afe2637
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
Mitchel Humpherys [Mon, 1 Jun 2015 22:44:49 +0000 (15:44 -0700)]
iommu: io-pgtable-arm: Add selftest for mixed block + page mappings
There can be page table bugs when both block and page mappings are used
to make up the mapping for a single VA range. Add a test case for this
to the selftests.
Change-Id: Ic2b943dd74f1ed2ed1e5e03832742f0e6deff58e
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
Mitchel Humpherys [Mon, 1 Jun 2015 22:37:23 +0000 (15:37 -0700)]
iommu: io-pgtable-arm: Correctly unmap the last level
Currently we have an optimization in place for unmapping the last level
of the page tables. We do this by memset()'ing the entire last level at
once rather than calling unmap on each individual page mapping at the
last level. For this to work we have to pass in sizes that aren't equal
to any of the supported IOMMU page sizes. However, our optimization
only applies at the last level. Unmapping at the other levels still
relies on the fact that unmap is only called with supported IOMMU page
sizes, which means it's currently broken.
Fix this by always calling unmap with an IOMMU page size, unless we're
at the last level of the page tables (i.e. the size to be unmapped is
less than the block size at the second-to-last level), in which case we
can pass in the entire remaining size.
Change-Id: Ie3716002c793af3dca51e0e3363d261f345e9e25
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
Mitchel Humpherys [Mon, 1 Jun 2015 19:50:38 +0000 (12:50 -0700)]
iommu/arm-smmu: Only prepare clocks when necessary
arm_smmu_enable_regulators also prepares all of our clocks (similarly
for arm_smmu_disable_regulators), and is always called from
arm_smmu_enable_clocks. arm_smmu_enable_clocks, also prepares our
clocks, so clocks are being prepared twice, which is once more than we
need. Fix this by enabling (not preparing) clocks in
arm_smmu_enable_clocks, relying on arm_smmu_enable_regulators to prepare
the clocks beforehand.
Change-Id: Id07848f64a81522e27198d6e708159941b07d444
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
Mitchel Humpherys [Tue, 26 May 2015 21:55:22 +0000 (14:55 -0700)]
iommu/arm-smmu: Restore usage of the NOSIGN SEP value
When we swapped out the page table code we also lost our change to use
the NOSIGN SEP value for all SMMUs. As noted in the original change,
this is needed for correct functionality of the SMMU. Restore this
change.
Change-Id: I0154003de92f59172a7c1e49aa68c387e87e2aa1
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
Neeti Desai [Wed, 6 May 2015 22:38:34 +0000 (15:38 -0700)]
arm64: dma-mapping: Split large pages when allocating in atomic context
In atomic context, gen_pool_alloc allocates a single page large
enough to accomodate the requested size. However __iommu_create_mapping
always maps pages assuming they are of size 4K. Thus only the first
4K of the buffer is mapped and a translation fault is generated
during an unmap.
Fix this by splitting the larger pages into 4K pages.
Change-Id: Ifcbe29477ad210204028486bd011470fe8b50852
Signed-off-by: Neeti Desai <neetid@codeaurora.org>
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
[pdaly@codeaurora.org Keep upstream version of alloc_from_pool]
Signed-off-by: Patrick Daly <pdaly@codeaurora.org>
Neeti Desai [Wed, 3 Jun 2015 18:07:48 +0000 (11:07 -0700)]
msm: Return -ENOSYS when secure buffer apis are not implemented
The return value -EINVAL is returned in the case of invalid
arguments, and is not the correct value when the function is not
implemented.
Return -ENOSYS instead.
Change-Id: I196537f121d5a290fec74e2b7bcb1cfd490468c7
Signed-off-by: Neeti Desai <neetid@codeaurora.org>
[pdaly@codeaurora.org Resolve minor conflicts]
Signed-off-by: Patrick Daly <pdaly@codeaurora.org>
Liam Mark [Sat, 6 Jun 2015 00:10:00 +0000 (17:10 -0700)]
iommu: add ftrace profiling for map and unmap
Add ftrace start and end logging for map, iommu_map_sg and unmap
in order to facilitate performance testing.
Change-Id: I9ddf241ffa6cf519f6abece7b0820640f5ce1975
Signed-off-by: Liam Mark <lmark@codeaurora.org>
Mitchel Humpherys [Wed, 27 May 2015 02:26:38 +0000 (19:26 -0700)]
iommu: io-pgtable-arm: unmap correct addresses in error path
If something bad happens in arm_lpae_map_sg we try to clean up after
ourselves by unmapping any partial mappings that succeeded. However,
we're currently passing in the wrong iova to the unmap function during
the cleanup. Fix this.
Change-Id: Ieb30616141f3fb709d02abd147f9f598e2db07cc
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
Mitchel Humpherys [Fri, 29 May 2015 18:04:26 +0000 (11:04 -0700)]
iommu/arm-smmu: restrict virtual address range to 36 bits
There appear to be bugs in the 4-level support of io-pgtable-arm. Work
around this by forcing all VAs to be 36 bits maximum so that we only
ever use 3 levels.
Change-Id: I5354afad05f74e12c51b86c97cdf1b2e86b68949
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
Mitchel Humpherys [Thu, 21 May 2015 21:11:22 +0000 (14:11 -0700)]
iommu: io-pgtable: use size_t for unmap return value
Unmap returns a size_t all throughout the IOMMU framework. Make
io-pgtable match this convention.
Change-Id: Ice4c75a428f0f95a665e2fbe4210349d6f78c220
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
Mitchel Humpherys [Tue, 19 May 2015 00:56:56 +0000 (17:56 -0700)]
iommu: io-pgtable-arm: set page tables as outer shareable
Coherent table walk is broken on our system. Set page tables as outer
shareable so that the SMMU doesn't try to walk them in the cache.
Change-Id: Id9dd3d10139750b0dbb77842c12efd49e2672645
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
Mitchel Humpherys [Mon, 18 May 2015 18:56:42 +0000 (11:56 -0700)]
Revert "arm64: dma-mapping: avoid calling iommu_iova_to_phys"
This reverts commit
0d02975d9ffd55f1c0fe5db08f45a9ee1d22f354 since it's
causing problems for some reason. This should really be debugged but
for now just revert it.
Change-Id: I31f382c1945cd1cd84dbbd3dfb715009b8442fe9
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
Neeti Desai [Sun, 3 May 2015 18:15:20 +0000 (11:15 -0700)]
arm64: dma_mapping: Add support for .dma_mapping_error() ops
If dma_map_single() or dma_map_page() fail to create a mapping, clients
can check these errors by testing the returned address with
.dma_mapping_error(). The implementation was missing for DMA IOMMU
mapper. Add it.
Change-Id: I7e62bdbcd013cab915134fce9e55b1e7b580915a
Signed-off-by: Neeti Desai <neetid@codeaurora.org>
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
Neeti Desai [Thu, 14 May 2015 00:57:25 +0000 (17:57 -0700)]
arm64: dma_mapping: Implement .dma_supported()
The dma_supported() API checks to see if the device can support DMA to
the memory described by the mask. Implement this for the DMA IOMMU
mapper by calling into the IOMMU's iommu_dma_supported API.
Change-Id: I5e31e386cd40c3f3312e1be1ca516c6e72213547
Signed-off-by: Neeti Desai <neetid@codeaurora.org>
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
Neeti Desai [Sun, 3 May 2015 18:08:56 +0000 (11:08 -0700)]
iommu/arm-smmu: Implement .dma_supported()
The .dma_supported() checks to see if the device can support DMA to the
memory described by the mask. Implement this for the arm-smmu driver.
Change-Id: Ie494d574b613c1c76e6f878048c74444dc25902c
Signed-off-by: Neeti Desai <neetid@codeaurora.org>
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
Neeti Desai [Thu, 14 May 2015 00:55:53 +0000 (17:55 -0700)]
iommu: Add support for .dma_supported() ops
The .dma_supported() checks to see if the device can support DMA to the
memory described by the mask. Add support for this operation in the
iommu layer
Change-Id: Icf37b9540aa68c2be3fd603a48402d6fcccd8208
Signed-off-by: Neeti Desai <neetid@codeaurora.org>
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
Neeti Desai [Tue, 12 May 2015 23:39:58 +0000 (16:39 -0700)]
iommu/arm-smmu: Add support for map/unmap to be atomic
Certain clients need map and unmap operations to be in the atomic
context. This currently doesn't work since we need to prepare clocks
and enable regulators (for tlb invalidation), and take a mutex (to
protect the smmu_domain->smmu instance), which are all sleeping
operations.
Fix this by introducing a domain attribute to indicate that map and
unmap should be atomic. When this domain attribute is set we assume:
(1) clocks are already prepared
(2) regulators are already on
(3) the domain is attached to an SMMU
(1) and (2) allow us to skip preparing clocks and enabling regulators
during map and unmap and are taken care of internally in the
driver. (3) is a contract with clients that allows us to skip taking
the mutex that protects smmu_domain->smmu since we assume that the SMMU
instance is valid and will not be going away. If clients break this
contract then there are race conditions all over the place that will
eventually result in a NULL dereference (specifically, of
smmu_domain->smmu).
Change-Id: I3e21df02f7b0cd116fb558715eea16f2119f1d03
Signed-off-by: Neeti Desai <neetid@codeaurora.org>
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
Mitchel Humpherys [Fri, 8 May 2015 23:05:20 +0000 (16:05 -0700)]
iommu: io-pgtable-arm: flush tlb for stale mappings
There seems to be a bug in the unmap code that results in us leaving
stale mappings in the page table. We can actually live with this as
long as we invalidate the tlb when a new mapping comes in on the same
virtual address (to prevent the walker from using the old, bogus
iova->phys mappings).
Change-Id: If5923e853e7ec542b12ca954d5f1c22dec5e5bb2
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
Mitchel Humpherys [Fri, 8 May 2015 23:05:37 +0000 (16:05 -0700)]
iommu/arm-smmu: silently succeed the secure VMID domain attr
With the recent page table refactor we've lost our code to assign SMMU
page tables to secure VMIDs. This functionality will be added back
later but for now just silently succeed when clients set this attribute.
This works for now because secure use cases haven't been brought up yet.
Change-Id: Ic2970e01f2899a9649c883f85aaaec828f2e7597
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
Mitchel Humpherys [Tue, 12 May 2015 16:44:32 +0000 (09:44 -0700)]
iommu: add more debug for iommu_pgsize failure
We're currently BUG()'ing when we can't find a valid IOMMU page size
without printing any other information. Add some more information about
the parameters passed to the function to aide in debugging.
Change-Id: I1797bdfa2ef9d899ef4ffcb36fea769b67a1f991
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
Mitchel Humpherys [Mon, 11 May 2015 23:22:17 +0000 (16:22 -0700)]
iommu: io-pgtable-arm: use correct mask during iova_to_phys
In ARMv8, the output address from a page table walk is obtained by
combining some bits from the physical address in the leaf page table
entry with some bits from the input virtual address. The number of bits
that should be taken from the virtual address varies based on the lookup
level and descriptor type. However, we're currently always using
data->pg_shift bits, which is a constant.
Conveniently there's already a macro to compute the number of bits we
want (ARM_LPAE_LVL_SHIFT). Use this macro instead of data->pg_shift to
build the virtual address mask.
Change-Id: Id7f8aa2c553cc004e5d895d05c9226a896d22ce6
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
Mitchel Humpherys [Mon, 11 May 2015 20:40:38 +0000 (13:40 -0700)]
iommu: io-pgtable-arm: make selftests more thorough
The io-pgtable-arm unit tests currently only check a few mappings within
a range to determine whether or not that range "looks good". There can
be subtle bugs that don't show up until you get to a certain offset
within the range, etc. Check the entire range before assuming that it's
good.
Change-Id: I244a2150d38f57d95a5c81854cdeaf59ab4ace06
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
Mitchel Humpherys [Fri, 8 May 2015 23:06:08 +0000 (16:06 -0700)]
dma-mapping: check for failed IOVA allocation
Currently if we fail to allocate an IOVA from our VA pool we proceed to
map DMA_ERROR_CODE into the IOMMU, which is not what the client
requested or expected. Fix this by checking for an error return value
while allocating an IOVA and bailing out as appropriate.
Change-Id: I13df7b9ab52536b757d8ae0ca74a5b9e1e1438b5
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
Mitchel Humpherys [Fri, 8 May 2015 17:56:54 +0000 (10:56 -0700)]
iommu: io-pgtable-arm: add support for IOMMU_DEVICE
Clients might want to map device memory into their SMMU. Add support
for these device mappings through the IOMMU_DEVICE flag.
Change-Id: I756720181aa0d531f4c56453ef832f81b36ffccd
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
Mitchel Humpherys [Fri, 8 May 2015 00:41:17 +0000 (17:41 -0700)]
iommu/arm-smmu: respect DOMAIN_ATTR_COHERENT_HTW_DISABLE
If coherent hardware table walks are disabled via the
DOMAIN_ATTR_COHERENT_HTW_DISABLE attribute we need to flush the page
tables to memory every time we touch them. Do it.
Change-Id: I34189b1312eb294102eeaa59ad46d164777a6789
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
Mitchel Humpherys [Thu, 30 Apr 2015 16:49:29 +0000 (09:49 -0700)]
iommu: io-pgtable-arm: unmap last level all at once
Currently we walk each last-level leaf pte during unmap and zero them
out individually. Since these last-level ptes are all contiguous (up to
512 entries), optimize the unmapping process by simply zeroing them all
out at once rather than operating on them individually.
Change-Id: I21d490e8a94355df4d4caecab33774b5f8ecf3ca
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
Mitchel Humpherys [Thu, 30 Apr 2015 16:54:07 +0000 (09:54 -0700)]
iommu/arm-smmu: add missing cleanup to attach error path
We need to disable clocks and unlock a mutex if
arm_smmu_init_domain_context fails during attach. Add the proper
cleanup.
Change-Id: Id097ee6815d63b61dd3db1843f054ed991d34224
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
Mitchel Humpherys [Sat, 25 Apr 2015 00:10:59 +0000 (17:10 -0700)]
iommu: io-pgtable-arm: Flush all tlbs at end of unmap
Rather than calling the tlb maintenance routines throughout the course
of the unmap operation, just flush the entire tlb for the context in
question all at once, at the very end of the unmap. This greatly
improves performance for large page tables (which is common for large
buffers in a heavily fragmented system).
In my testing, this optimization gave a ~10% speedup when unmapping 64K.
Change-Id: Iaa2b211e730dad6bd9235ef98dd2a89cf541e663
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
Mitchel Humpherys [Thu, 23 Apr 2015 23:29:23 +0000 (16:29 -0700)]
iommu/arm-smmu: wire up .map_sg from the ARM LPAE io-pgtable
The ARM LPAE io-pgtable provider now has support for .map_sg. Wire it
up.
Change-Id: I2eaea6f245e1059582f3bf04829e9e3f24675782
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
Mitchel Humpherys [Thu, 23 Apr 2015 20:41:31 +0000 (13:41 -0700)]
iommu: io-pgtable-arm: add self-test for .mag_sg
io-pgtable-arm has just gotten support for .map_sg. Add a test to the
suite of self-tests for this.
Change-Id: Iba56bb801c1f9ef151827598022411c95d389faa
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
Mitchel Humpherys [Thu, 23 Apr 2015 23:19:05 +0000 (16:19 -0700)]
iommu: io-pgtable-arm: implement .map_sg
Mapping an entire scatterlist at once is faster than calling iommu_map
on each link individually. Implement .map_sg in the ARM LPAE page table
allocator so that drivers using the allocator can leverage this
performance boost.
Change-Id: I77f62a2566058693c3f58fc0b05d715a780ae5d8
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
Mitchel Humpherys [Thu, 23 Apr 2015 23:16:56 +0000 (16:16 -0700)]
iommu: io-pgtable: add .map_sg op to io_pgtable_ops
The default implementation of map_sg (provided by default_iommu_map_sg
from the IOMMU API layer) actually just calls iommu_map repeatedly.
Often times there are locks or other initial setup in the implementation
of iommu_map, so there's opportunity to drastically improve performance
by factoring that out of the loop. For this reason, io-pgtable
providers might want to implement .map_sg. Add the declaration to the
io_pgtable_ops structure.
Change-Id: I5aa5e4e24a68e303ce2c005dc1dd0b33d5635ab3
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
Mitchel Humpherys [Sat, 2 May 2015 00:13:21 +0000 (17:13 -0700)]
iommu: export iommu_pgsize
It can be useful in IOMMU drivers. Export it.
Change-Id: I4c423d256312250f1e33ca8d64dfe1626f008b5e
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
Mitchel Humpherys [Thu, 16 Apr 2015 21:03:20 +0000 (14:03 -0700)]
iommu/arm-smmu: enable clocks in tlb invalidation routines
Previously we were relying on the callers of the tlb invalidation
functions to enable clocks. However, the new io-pgtable framework
doesn't know anything about clocks and calls directly into the tlb
invalidation functions, so we need to put the clocks enable/disable
inside the tlb functions themselves.
Change-Id: I73a55e3f78deb3501df5615e22cd298fba6e4551
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
Mitchel Humpherys [Wed, 15 Apr 2015 22:17:56 +0000 (15:17 -0700)]
iommu/arm-smmu: re-use the init_mutex for protecting smmu_domain.smmu
The newly-introduced smmu_domain.init_mutex can be used everywhere that
we're currently using smmu_domain.lock to protect smmu_domain.smmu.
Move to `init_mutex' and delete `lock'.
Change-Id: I92aa237e72cafd0c8fa39fe6279c44bd194c11b4
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
Mitchel Humpherys [Wed, 12 Nov 2014 23:11:33 +0000 (15:11 -0800)]
iommu/arm-smmu: implement DOMAIN_ATTR_PT_BASE_ADDR using ttbr[0]
Some clients need to get the base address of the page table for a given
IOMMU domain. This functionality is provided by the
DOMAIN_ATTR_PT_BASE_ADDR domain attribute. Implement it in the ARM SMMU
driver by returning ttbr[0] from the page table config.
Change-Id: Ie9b0241c7c7df18b25761bae79c1be1a283ff3a4
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
Mitchel Humpherys [Wed, 15 Apr 2015 22:14:15 +0000 (15:14 -0700)]
iommu/arm-smmu: save the pgtbl_cfg in the domain
The pgtbl_cfg object has a few handy properties that we'd like to make
use of later (returning the pgd in a domain attribute, for example).
Keep track of the domain pgtbl_cfg in the domain structure.
Change-Id: Icce9638a270ca98d6ed6d61b3ef1c35d42a869dc
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
Laurent Pinchart [Sun, 14 Dec 2014 21:34:50 +0000 (23:34 +0200)]
iommu: io-pgtable-arm: add non-secure quirk
The quirk causes the Non-Secure bit to be set in all page table entries.
Change-Id: I937fb7dec4214eca33f8014c664cfc5c99cb0027
Signed-off-by: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
Git-repo: git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
Git-commit:
c896c132b01895fd1445d178e36155b671c6f9ee
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
Will Deacon [Fri, 14 Nov 2014 17:17:54 +0000 (17:17 +0000)]
iommu/arm-smmu: make use of generic LPAE allocator
The ARM SMMU can walk LPAE page tables, so make use of the generic
allocator.
Change-Id: I7acd0e6c6e8046a4f0497a8fbdba46986192f83f
Signed-off-by: Will Deacon <will.deacon@arm.com>
Git-repo: git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
Git-commit:
518f7136244c167538f732691be589959310b295
[mitchelh: merge with our changes to domain locking, non-shareable
page tables, and pte zero'ing, some of which will need to be added
back now in the generic allocator code]
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
Will Deacon [Mon, 17 Nov 2014 23:31:12 +0000 (23:31 +0000)]
iommu: add self-consistency tests to ARM LPAE IO page table allocator
This patch adds a series of basic self-consistency tests to the ARM LPAE
IO page table allocator that exercise corner cases in map/unmap, as well
as testing all valid configurations of pagesize, ias and stage.
Change-Id: I703df977b7e5914e0ccf9aaca2174cf5956dd604
Signed-off-by: Will Deacon <will.deacon@arm.com>
Git-commit:
fe4b991dcd84e0104cf2e29223a819335ed048a7
Git-repo: git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
Will Deacon [Fri, 14 Nov 2014 17:18:23 +0000 (17:18 +0000)]
iommu: add ARM LPAE page table allocator
A number of IOMMUs found in ARM SoCs can walk architecture-compatible
page tables.
This patch adds a generic allocator for Stage-1 and Stage-2 v7/v8
long-descriptor page tables. 4k, 16k and 64k pages are supported, with
up to 4-levels of walk to cover a 48-bit address space.
Change-Id: I32740cfa795c55e0d3683b42105b4f49c9dcf984
Tested-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
Git-commit:
e1d3c0fd701df831169b116cd5c5d6203ac07f70
Git-repo: git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
Will Deacon [Fri, 14 Nov 2014 17:16:49 +0000 (17:16 +0000)]
iommu: introduce generic page table allocation framework
This patch introduces a generic framework for allocating page tables for
an IOMMU. There are a number of reasons we want to do this:
- It avoids duplication of complex table management code in IOMMU
drivers that use the same page table format
- It removes any coupling with the CPU table format (and even the
architecture!)
- It defines an API for IOMMU TLB maintenance
Change-Id: Ia7c22397f0afe94898cd91d8d9c30be985993d1d
Tested-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
Git-commit:
fdb1d7be7c4d452e9735aeb2b60ae8a2fcf0a514
Git-repo: git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
Mitchel Humpherys [Wed, 15 Apr 2015 22:27:43 +0000 (15:27 -0700)]
Revert "iommu/arm-smmu: implement DOMAIN_ATTR_PT_BASE_ADDR"
This reverts commit
c8690d83e1e1b09d55ed75d6776efe42915b9c95 since it
won't compile when arm-smmu.c moves to the generic page table
allocator (since cfg.pgd is being deleted there). We'll bring this back
in with the appropriate fix once the generic page table allocator patch
is in.
Change-Id: I956d902999a05653e70f7cb7a90ac5010f9926de
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
Will Deacon [Wed, 25 Jun 2014 21:46:31 +0000 (22:46 +0100)]
iommu/arm-smmu: add support for DOMAIN_ATTR_NESTING attribute
When domains are set with the DOMAIN_ATTR_NESTING flag, we must ensure
that we allocate them to stage-2 context banks if the hardware permits
it.
This patch adds support for the attribute to the ARM SMMU driver, with
the actual stage being determined depending on the features supported
by the hardware.
Change-Id: I17e027388efbeaae65886160f35a5f8068fd7734
Signed-off-by: Will Deacon <will.deacon@arm.com>
Git-commit:
c752ce45b213de8532baaf987ba930638f77c439
Git-repo: git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
[mitchelh: resolved conflicts with our existing domain attr
setter/getter, and locking differences]
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
Thierry Reding [Fri, 7 Nov 2014 15:26:18 +0000 (15:26 +0000)]
iommu/arm-smmu: Play nice on non-ARM/SMMU systems
Currently the driver registers IOMMU bus operations for all busses even
if no ARM SMMU is present on a system. Depending on the driver probing
order this prevents the driver for the real IOMMU to register itself as
the bus-wide IOMMU.
Change-Id: I345b0e922ae3d0d71622217fd869747d25ae2bb4
Signed-off-by: Thierry Reding <treding@nvidia.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
Git-commit:
0e7d37adbe45404a76d05d4ef11544f23cf639dd
Git-repo: git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
Antonios Motakis [Mon, 13 Oct 2014 13:06:16 +0000 (14:06 +0100)]
iommu/arm-smmu: change IOMMU_EXEC to IOMMU_NOEXEC
Exposing the XN flag of the SMMU driver as IOMMU_NOEXEC instead of
IOMMU_EXEC makes it enforceable, since for IOMMUs that don't support
the XN flag pages will always be executable.
Change-Id: Ic6b1f0ae8873035dcb798e9c996396fb2bc4921e
Signed-off-by: Antonios Motakis <a.motakis@virtualopensystems.com>
Acked-by: Joerg Roedel <jroedel@suse.de>
Signed-off-by: Will Deacon <will.deacon@arm.com>
Git-commit:
a720b41c41f5a7e4c51558cf087882c57331581f
Git-repo: git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
[mitchelh: context fixups]
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
Mitchel Humpherys [Wed, 27 May 2015 23:36:56 +0000 (16:36 -0700)]
iommu/arm-smmu: Check of_match_node for NULL
In ["iommu/arm-smmu: fix architecture version detection"] we added the
ability to determine the SMMU architecture version from DT match data by
using of_match_node, but neglected to check the return value for NULL.
Fix this.
Change-Id: Ib9049f2b2f1204b52bc07ba8caf075449af4cd35
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
Mitchel Humpherys [Mon, 27 Apr 2015 22:35:21 +0000 (15:35 -0700)]
Revert "iommu/arm-smmu: Add support for page table donation"
This reverts commit
e6e701ddb09b2b6b43ffc7ff788701eda03703b8. This code
is about to be ripped out in favor of the generic io-pgtable code, so
leaving it in will only cause merged conflicts.
Don't revert the introduction of the DOMAIN_ATTR_SECURE_VMID attribute
to avoid unnecessary churn in client code.
Change-Id: I58af07686f11b14a82d44bd7999ba50127530f1f
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
Mitchel Humpherys [Mon, 27 Apr 2015 22:35:20 +0000 (15:35 -0700)]
Revert "Add support of the IOMMU_DEVICE flag."
This reverts commit
fd3161483bd7af420b503a8e63d3c1f24a7cf936. This code
is about to be ripped out in favor of the generic io-pgtable code, so
leaving it in will only cause merged conflicts.
Change-Id: I1404f79f08340d3de4a52155c3c82fae6df6515c
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
Mitchel Humpherys [Mon, 27 Apr 2015 22:35:19 +0000 (15:35 -0700)]
Revert "iommu/arm-smmu: Make the arm_smmu_map operation atomic"
This reverts commit
0385f613fc41c394873d64785c073dffdfaf0fc4. This code
is about to be ripped out in favor of the generic io-pgtable code, so
leaving it in will only cause merged conflicts.
Change-Id: Id8474f2a400736c288d9db5f9608eddbd24b0793
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
Mitchel Humpherys [Tue, 19 May 2015 04:16:10 +0000 (21:16 -0700)]
Revert "iommu/arm-smmu: change IOMMU_EXEC to IOMMU_NOEXEC"
This reverts commit
433a4b6a37c8f96295a7e80cb603c2318872c1f2. An
equivalent patch is being brought in with the upcoming page table
refactoring, so rip this guy out in preparation.
This also reverts commit
af3e3c36d08077856640b28d68c9f99d2188a000, which
introduced users of the IOMMU_NOEXEC flag.
Change-Id: Ibb36f60f2148170d8a01632043bf36b5f8c60777
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
[pdaly@codeaurora.org Resolve minor conflicts]
Signed-off-by: Patrick Daly <pdaly@codeaurora.org>
Chintan Pandya [Tue, 26 May 2015 12:29:11 +0000 (17:59 +0530)]
arm: dma-mapping: map sg lists into the SMMU as virtually contiguous
Following commit implements mapping of sg virtually contiguously.
"arm64: dma-mapping: map sg lists into the SMMU as virtually
contiguous"
a03f74ef16cc73531795176d3ea8b82b66ed0146
This has been left for ARM (32-bit). Implement the same for ARM.
Change-Id: Ibf67f29a60b8d19e526c4719590f2f473ea9dca5
Signed-off-by: Chintan Pandya <cpandya@codeaurora.org>
Pratik Patel [Tue, 30 Sep 2014 16:30:25 +0000 (09:30 -0700)]
soc: qcom: hvc: add missing x7 argument to the 32bit __hvc stub
__hvc should take 11 arguments instead of 10. Add the missing x7
argument to avoid compilation error if the driver gets enabled on
32bit targets.
Change-Id: I09985235fdbfb64e81743a71ceb2764c32d3a3b1
Signed-off-by: Pratik Patel <pratikp@codeaurora.org>