OSDN Git Service
Markus Armbruster [Thu, 24 Oct 2019 11:02:33 +0000 (13:02 +0200)]
qapi: Fold normalize_features() into check_features()
check_features() is always called together with normalize_features():
the former in check_struct() and check_command(), the latter in their
caller check_exprs(). Fold the latter into the former.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <
20191024110237.30963-16-armbru@redhat.com>
Markus Armbruster [Thu, 24 Oct 2019 11:02:32 +0000 (13:02 +0200)]
qapi: Fold normalize_if() into check_if()
check_if() is always called together with normalize_if(). Fold the
latter into the former.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <
20191024110237.30963-15-armbru@redhat.com>
Markus Armbruster [Thu, 24 Oct 2019 11:02:31 +0000 (13:02 +0200)]
qapi: Eliminate .check_doc() overrides
All sub-classes of QAPISchemaEntity now override .check_doc() the same
way, except for QAPISchemaType and and QAPISchemaArrayType.
Put the overrides' code in QAPISchemaEntity.check_doc(), and drop the
overrides. QAPISchemaType doesn't care because it's abstract.
QAPISchemaArrayType doesn't care because its .doc is always None.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <
20191024110237.30963-14-armbru@redhat.com>
Markus Armbruster [Thu, 24 Oct 2019 11:02:30 +0000 (13:02 +0200)]
qapi: Simplify ._make_implicit_object_type()
All callers now pass doc=None. Drop the argument.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <
20191024110237.30963-13-armbru@redhat.com>
Markus Armbruster [Thu, 24 Oct 2019 11:02:29 +0000 (13:02 +0200)]
qapi: Fix doc comment checking for commands and events
When a command's 'data' is an object, its doc comment describes the
arguments defined there. When 'data' names a type, the doc comment
does not describe arguments. Instead, the doc generator inserts a
pointer to the named type.
An event's doc comment works the same.
We don't actually check doc comments for commands and events.
Instead, QAPISchema._def_command() forwards the doc comment to the
implicit argument type, where it gets checked. Works because the
check only cares for the implicit argument type's members.
Not only is this needlessly hard to understand, it actually falls
apart in two cases:
* When 'data' is empty, there is nothing to forward to, and the doc
comment remains unchecked. Demonstrated by test doc-bad-event-arg.
* When 'data' names a type, we can't forward, as the type has its own
doc comment. The command or event's doc comment remains unchecked.
Demonstrated by test doc-bad-boxed-command-arg.
The forwarding goes back to commit
069fb5b250 "qapi: Prepare for
requiring more complete documentation", put to use in commit
816a57cd6e "qapi: Fix detection of bogus member documentation". That
fix was incomplete.
To fix this, make QAPISchemaCommand and QAPISchemaEvent check doc
comments, and drop the forwarding of doc comments to implicit argument
types.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <
20191024110237.30963-12-armbru@redhat.com>
Markus Armbruster [Thu, 24 Oct 2019 11:02:28 +0000 (13:02 +0200)]
qapi: Clean up doc comment checking for implicit union base
An object type's doc comment describes the type's members, less the
ones defined in a named base type. Cases:
* Struct: the members are defined in 'data' and inherited from 'base'.
Since the base type cannot be implicit, the doc comment describes
just 'data'.
* Simple union: the only member is the implicit tag member @type, and
the doc comment describes it.
* Flat union with implicit base type: the members are defined in
'base', and the doc comment describes it.
* Flat union with named base type: the members are inherited from
'base'. The doc comment describes no members.
Before we can check a doc comment with .check_doc(), we need
.connect_doc() connect each of its "argument sections" to the member
it documents.
For structs and simple unions, this is straightforward: the members in
question are in .local_members, and .connect_doc() connects them.
For flat unions with a named base type, it's trivial: .local_members
is empty, and .connect_doc() does nothing.
For flat unions with an implicit base type, it's tricky. We have
QAPISchema._make_implicit_object_type() forward the union's doc
comment to the implicit base type, so that the base type's
.connect_doc() connects the members. The union's .connect_doc() does
nothing, as .local_members is empty.
Dirt effect: we check the doc comment twice, once for the union type,
and once for the implicit base type.
This is needlessly brittle and hard to understand. Clean up as
follows. Make the union's .connect_doc() connect an implicit base's
members itself. Do not forward the union's doc comment to its
implicit base type.
Requires extending .connect_doc() so it can work with a doc comment
other than self.doc. Add an optional argument for that.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <
20191024110237.30963-11-armbru@redhat.com>
Markus Armbruster [Thu, 24 Oct 2019 11:02:27 +0000 (13:02 +0200)]
qapi: Fix enum doc comment checking
Enumeration type documentation comments are not checked, as
demonstrated by test doc-bad-enum-member. This is because we neglect
to call self.doc.check() for enumeration types. Messed up in
816a57cd6e "qapi: Fix detection of bogus member documentation". Fix
it.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <
20191024110237.30963-10-armbru@redhat.com>
Markus Armbruster [Thu, 24 Oct 2019 11:02:26 +0000 (13:02 +0200)]
qapi: Split .connect_doc(), .check_doc() off .check()
Splitting documentation checking off the .check() methods makes them a
bit more focused, which is welcome, as some of them are pretty big.
It also prepares the ground for the following commits.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <
20191024110237.30963-9-armbru@redhat.com>
Markus Armbruster [Thu, 24 Oct 2019 11:02:25 +0000 (13:02 +0200)]
qapi: De-duplicate entity documentation generation code
QAPISchemaGenDocVisitor.visit_command() duplicates texi_entity() for
its boxed arguments case. The previous commit added another copy in
.visit_event().
Replace texi_entity() by texi_type() and texi_msg(). Use texi_msg()
for the boxed arguments case as well.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <
20191024110237.30963-8-armbru@redhat.com>
Markus Armbruster [Thu, 24 Oct 2019 11:02:24 +0000 (13:02 +0200)]
qapi: Implement boxed event argument documentation
Generate a reference "Arguments: the members of ...", just like we do
for commands since commit
c2dd311cb7 "qapi2texi: Implement boxed
argument documentation".
No change to generated QMP documentation; we don't yet use boxed
events outside tests/.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <
20191024110237.30963-7-armbru@redhat.com>
Markus Armbruster [Thu, 24 Oct 2019 11:02:23 +0000 (13:02 +0200)]
qemu-doc: Belatedly document QMP command deprecation
Commit
24fb413300 "qmp: Introduce blockdev-change-medium" (v2.5.0)
deprecated change.
Commit
2ff3025797 "migrate: move max-bandwidth and downtime-limit to
migrate_set_parameter" (v2.8.0) deprecated migrate_set_downtime and
migrate_set_speed.
These wre missed in commit
eb22aeca65 "docs: document deprecation
policy & deprecated features in appendix" (v2.10.0).
Commit
73af8dd8d7 "migration: Make xbzrle_cache_size a migration
parameter" (v2.11.0) deprecated migrate-set-cache-size and
query-migrate-cache-size, but neglected to update qemu-doc.
Make up for that.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <
20191024110237.30963-6-armbru@redhat.com>
Markus Armbruster [Thu, 24 Oct 2019 11:02:22 +0000 (13:02 +0200)]
tests/qapi-schema: Fix feature documentation testing
Commit
8aa3a33e44 "tests/qapi-schema: Test for good feature lists in
structs" made test-qapi.py show features, but neglected to show their
documentation. Fix that.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <
20191024110237.30963-5-armbru@redhat.com>
Markus Armbruster [Thu, 24 Oct 2019 11:02:21 +0000 (13:02 +0200)]
tests/qapi-schema: Cover alternate documentation comments
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <
20191024110237.30963-4-armbru@redhat.com>
Markus Armbruster [Thu, 24 Oct 2019 11:02:20 +0000 (13:02 +0200)]
tests/qapi-schema: Demonstrate command and event doc comment bugs
Add negative tests doc-bad-boxed-command-arg and doc-bad-event-arg to
cover boxed and no arguments. They demonstrate insufficient doc
comment checking.
Update positive test doc-good to cover boxed event arguments. It
demonstrates the generated doc comment misses arguments.
These bugs will be fixed later in this series.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <
20191024110237.30963-3-armbru@redhat.com>
Markus Armbruster [Thu, 24 Oct 2019 11:02:19 +0000 (13:02 +0200)]
tests/qapi-schema: Demonstrate feature and enum doc comment bugs
Add negative tests doc-bad-enum-member and doc-bad-feature to cover
documentation for nonexistent enum members and features, and test
doc-undoc-feature to cover features lacking documentation. None of
them works. To be fixed later in this series.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <
20191024110237.30963-2-armbru@redhat.com>
Peter Maydell [Mon, 28 Oct 2019 14:40:00 +0000 (14:40 +0000)]
Merge remote-tracking branch 'remotes/maxreitz/tags/pull-block-2019-10-28' into staging
Block patches for softfreeze:
- iotest patches
- Improve performance of the mirror block job in write-blocking mode
- Limit memory usage for the backup block job
- Add discard and write-zeroes support to the NVMe host block driver
- Fix a bug in the mirror job
- Prevent the qcow2 driver from creating technically non-compliant qcow2
v3 images (where there is not enough extra data for snapshot table
entries)
- Allow callers of bdrv_truncate() (etc.) to determine whether the file
must be resized to the exact given size or whether it is OK for block
devices not to shrink
# gpg: Signature made Mon 28 Oct 2019 12:13:53 GMT
# gpg: using RSA key
91BEB60A30DB3E8857D11829F407DB0061D5CF40
# gpg: issuer "mreitz@redhat.com"
# gpg: Good signature from "Max Reitz <mreitz@redhat.com>" [full]
# Primary key fingerprint: 91BE B60A 30DB 3E88 57D1 1829 F407 DB00 61D5 CF40
* remotes/maxreitz/tags/pull-block-2019-10-28: (69 commits)
qemu-iotests: restrict 264 to qcow2 only
Revert "qemu-img: Check post-truncation size"
block: Pass truncate exact=true where reasonable
block: Let format drivers pass @exact
block: Evaluate @exact in protocol drivers
block: Add @exact parameter to bdrv_co_truncate()
block: Do not truncate file node when formatting
block/cor: Drop cor_co_truncate()
block: Handle filter truncation like native impl.
iotests: Test qcow2's snapshot table handling
iotests: Add peek_file* functions
qcow2: Fix v3 snapshot table entry compliancy
qcow2: Repair snapshot table with too many entries
qcow2: Fix overly long snapshot tables
qcow2: Keep track of the snapshot table length
qcow2: Fix broken snapshot table entries
qcow2: Add qcow2_check_fix_snapshot_table()
qcow2: Separate qcow2_check_read_snapshot_table()
qcow2: Write v3-compliant snapshot list on upgrade
qcow2: Put qcow2_upgrade() into its own function
...
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Peter Maydell [Mon, 28 Oct 2019 13:32:40 +0000 (13:32 +0000)]
Merge remote-tracking branch 'remotes/mst/tags/for_upstream' into staging
virtio: features, tests
libqos update with support for virtio 1.
Packed ring support for virtio.
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
# gpg: Signature made Fri 25 Oct 2019 12:47:59 BST
# gpg: using RSA key
281F0DB8D28D5469
# gpg: Good signature from "Michael S. Tsirkin <mst@kernel.org>" [full]
# gpg: aka "Michael S. Tsirkin <mst@redhat.com>" [full]
# Primary key fingerprint: 0270 606B 6F3C DF3D 0B17 0970 C350 3912 AFBE 8E67
# Subkey fingerprint: 5D09 FD08 71C8 F85B 94CA 8A0D 281F 0DB8 D28D 5469
* remotes/mst/tags/for_upstream: (25 commits)
virtio: drop unused virtio_device_stop_ioeventfd() function
libqos: add VIRTIO PCI 1.0 support
libqos: extract Legacy virtio-pci.c code
libqos: make the virtio-pci BAR index configurable
libqos: expose common virtqueue setup/cleanup functions
libqos: add MSI-X callbacks to QVirtioPCIDevice
libqos: pass full QVirtQueue to set_queue_address()
libqos: add iteration support to qpci_find_capability()
libqos: access VIRTIO 1.0 vring in little-endian
libqos: implement VIRTIO 1.0 FEATURES_OK step
libqos: enforce Device Initialization order
libqos: add missing virtio-9p feature negotiation
tests/virtio-blk-test: set up virtqueue after feature negotiation
virtio-scsi-test: add missing feature negotiation
libqos: extend feature bits to 64-bit
libqos: read QVIRTIO_MMIO_VERSION register
tests/virtio-blk-test: read config space after feature negotiation
virtio: add property to enable packed virtqueue
vhost_net: enable packed ring support
virtio: event suppression support for packed ring
...
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Vladimir Sementsov-Ogievskiy [Fri, 25 Oct 2019 14:50:22 +0000 (17:50 +0300)]
qemu-iotests: restrict 264 to qcow2 only
264 is unprepared to run with different formats, for example luks needs
handling keys, cloop doesn't support image creation, vpc creates image
larger than requested (which breaks "Backup completed:
5242880" in test
output).
The test is here to check nbd-reconnect feature and we actually don't
need it for all formats. Let's restrict it to qcow2 only.
Reported-by: Max Reitz <mreitz@redhat.com>
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-id:
20191025145023.6182-1-vsementsov@virtuozzo.com
Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
Max Reitz [Wed, 18 Sep 2019 09:51:44 +0000 (11:51 +0200)]
Revert "qemu-img: Check post-truncation size"
This reverts commit
5279b30392da7a3248b320c75f20c61e3a95863c.
We no longer need this check because exact=true forces the block driver
to give the image the exact size requested by the user.
Signed-off-by: Max Reitz <mreitz@redhat.com>
Message-id:
20190918095144.955-9-mreitz@redhat.com
Reviewed-by: Maxim Levitsky <mlevitsk@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
Max Reitz [Wed, 18 Sep 2019 09:51:43 +0000 (11:51 +0200)]
block: Pass truncate exact=true where reasonable
This is a change in behavior, so all instances need a good
justification. The comments added here should explain my reasoning.
qed already had a comment that suggests it always expected
bdrv_truncate()/blk_truncate() to behave as if exact=true were passed
(
c743849bee7 came eight months before
55b949c8476), so it was simply
broken until now.
Signed-off-by: Max Reitz <mreitz@redhat.com>
Message-id:
20190918095144.955-8-mreitz@redhat.com
Reviewed-by: Maxim Levitsky <mlevitsk@redhat.com>
[mreitz: Changed comment in qed.c to explain why a new QED file must be
empty, as requested and suggested by Maxim]
Signed-off-by: Max Reitz <mreitz@redhat.com>
Max Reitz [Wed, 18 Sep 2019 09:51:42 +0000 (11:51 +0200)]
block: Let format drivers pass @exact
When truncating a format node, the @exact parameter is generally handled
simply by virtue of the format storing the new size in the image
metadata. Such formats do not need to pass on the parameter to their
file nodes.
There are exceptions, though:
- raw and crypto cannot store the image size, and thus must pass on
@exact.
- When using qcow2 with an external data file, it just makes sense to
keep its size in sync with the qcow2 virtual disk (because the
external data file is the virtual disk). Therefore, we should pass
@exact when truncating it.
Signed-off-by: Max Reitz <mreitz@redhat.com>
Message-id:
20190918095144.955-7-mreitz@redhat.com
Reviewed-by: Maxim Levitsky <mlevitsk@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
Max Reitz [Wed, 18 Sep 2019 09:51:41 +0000 (11:51 +0200)]
block: Evaluate @exact in protocol drivers
We have two protocol drivers that return success when trying to shrink a
block device even though they cannot shrink it. This behavior is now
only allowed with exact=false, so they should return an error with
exact=true.
Signed-off-by: Max Reitz <mreitz@redhat.com>
Message-id:
20190918095144.955-6-mreitz@redhat.com
Reviewed-by: Maxim Levitsky <mlevitsk@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
Max Reitz [Wed, 18 Sep 2019 09:51:40 +0000 (11:51 +0200)]
block: Add @exact parameter to bdrv_co_truncate()
We have two drivers (iscsi and file-posix) that (in some cases) return
success from their .bdrv_co_truncate() implementation if the block
device is larger than the requested offset, but cannot be shrunk. Some
callers do not want that behavior, so this patch adds a new parameter
that they can use to turn off that behavior.
This patch just adds the parameter and lets the block/io.c and
block/block-backend.c functions pass it around. All other callers
always pass false and none of the implementations evaluate it, so that
this patch does not change existing behavior. Future patches take care
of that.
Suggested-by: Maxim Levitsky <mlevitsk@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
Message-id:
20190918095144.955-5-mreitz@redhat.com
Reviewed-by: Maxim Levitsky <mlevitsk@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
Max Reitz [Wed, 18 Sep 2019 09:51:39 +0000 (11:51 +0200)]
block: Do not truncate file node when formatting
There is no reason why the format drivers need to truncate the protocol
node when formatting it. When using the old .bdrv_co_create_ops()
interface, the file will be created with no size option anyway, which
generally gives it a size of 0. (Exceptions are block devices, which
cannot be truncated anyway.)
When using blockdev-create, the user must have given the file node some
size anyway, so there is no reason why we should override that.
qed is an exception, it needs the file to start completely empty (as
explained by
c743849bee7333c7ef256b7e12e34ed6f907064f).
Signed-off-by: Max Reitz <mreitz@redhat.com>
Message-id:
20190918095144.955-4-mreitz@redhat.com
Reviewed-by: Maxim Levitsky <mlevitsk@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
Max Reitz [Wed, 18 Sep 2019 09:51:38 +0000 (11:51 +0200)]
block/cor: Drop cor_co_truncate()
No other filter driver has a .bdrv_co_truncate() implementation, and
there is no need to because the general block layer code can handle it
just as well.
Signed-off-by: Max Reitz <mreitz@redhat.com>
Message-id:
20190918095144.955-3-mreitz@redhat.com
Reviewed-by: Maxim Levitsky <mlevitsk@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
Max Reitz [Wed, 18 Sep 2019 09:51:37 +0000 (11:51 +0200)]
block: Handle filter truncation like native impl.
Make the filter truncation (passing it through to bs->file) a
first-class citizen and handle it exactly as if it was the filter
driver's native implementation of .bdrv_co_truncate().
I do not see a reason not to, it makes the code a bit shorter, and may
be even more correct because this gets us to finish the write_req that
we prepared before (may be important to e.g. bring dirty bitmaps to the
correct size).
Signed-off-by: Max Reitz <mreitz@redhat.com>
Message-id:
20190918095144.955-2-mreitz@redhat.com
Reviewed-by: Maxim Levitsky <mlevitsk@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
Max Reitz [Fri, 11 Oct 2019 15:28:14 +0000 (17:28 +0200)]
iotests: Test qcow2's snapshot table handling
Add a test how our qcow2 driver handles extra data in snapshot table
entries, and how it repairs overly long snapshot tables.
Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-id:
20191011152814.14791-17-mreitz@redhat.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
Max Reitz [Fri, 11 Oct 2019 15:28:13 +0000 (17:28 +0200)]
iotests: Add peek_file* functions
Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-id:
20191011152814.14791-16-mreitz@redhat.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
Max Reitz [Fri, 11 Oct 2019 15:28:12 +0000 (17:28 +0200)]
qcow2: Fix v3 snapshot table entry compliancy
qcow2 v3 images require every snapshot table entry to have at least 16
bytes of extra data. If they do not, let qemu-img check -r all fix it.
Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-id:
20191011152814.14791-15-mreitz@redhat.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
Max Reitz [Fri, 11 Oct 2019 15:28:11 +0000 (17:28 +0200)]
qcow2: Repair snapshot table with too many entries
The user cannot choose which snapshots are removed. This is fine
because we have chosen the maximum snapshot table size to be so large
(65536 entries) that it cannot be reasonably reached. If the snapshot
table exceeds this size, the image has probably been corrupted in some
way; in this case, it is most important to just make the image usable
such that the user can copy off at least the active layer.
(Also note that the snapshots will be removed only with "-r all", so a
plain "check" or "check -r leaks" will not delete any data.)
Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-id:
20191011152814.14791-14-mreitz@redhat.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
Max Reitz [Fri, 11 Oct 2019 15:28:10 +0000 (17:28 +0200)]
qcow2: Fix overly long snapshot tables
We currently refuse to open qcow2 images with overly long snapshot
tables. This patch makes qemu-img check -r all drop all offending
entries past what we deem acceptable.
The user cannot choose which snapshots are removed. This is fine
because we have chosen the maximum snapshot table size to be so large
(64 MB) that it cannot be reasonably reached. If the snapshot table
exceeds this size, the image has probably been corrupted in some way; in
this case, it is most important to just make the image usable such that
the user can copy off at least the active layer.
(Also note that the snapshots will be removed only with "-r all", so a
plain "check" or "check -r leaks" will not delete any data.)
Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-id:
20191011152814.14791-13-mreitz@redhat.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
Max Reitz [Fri, 11 Oct 2019 15:28:09 +0000 (17:28 +0200)]
qcow2: Keep track of the snapshot table length
When repairing the snapshot table, we truncate entries that have too
much extra data. This frees up space that we do not have to count
towards the snapshot table size.
Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-id:
20191011152814.14791-12-mreitz@redhat.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
Max Reitz [Fri, 11 Oct 2019 15:28:08 +0000 (17:28 +0200)]
qcow2: Fix broken snapshot table entries
The only case where we currently reject snapshot table entries is when
they have too much extra data. Fix them with qemu-img check -r all by
counting it as a corruption, reducing their extra_data_size, and then
letting qcow2_check_fix_snapshot_table() do the rest.
Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-id:
20191011152814.14791-11-mreitz@redhat.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
Max Reitz [Fri, 11 Oct 2019 15:28:07 +0000 (17:28 +0200)]
qcow2: Add qcow2_check_fix_snapshot_table()
qcow2_check_read_snapshot_table() can perform consistency checks, but it
cannot fix everything. Specifically, it cannot allocate new clusters,
because that should wait until the refcount structures are known to be
consistent (i.e., after qcow2_check_refcounts()). Thus, it cannot call
qcow2_write_snapshots().
Do that in qcow2_check_fix_snapshot_table(), which is called after
qcow2_check_refcounts().
Currently, there is nothing that would set result->corruptions, so this
is a no-op. A follow-up patch will change that.
Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-id:
20191011152814.14791-10-mreitz@redhat.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
Max Reitz [Fri, 11 Oct 2019 15:28:06 +0000 (17:28 +0200)]
qcow2: Separate qcow2_check_read_snapshot_table()
Reading the snapshot table can fail. That is a problem when we want to
repair the image.
Therefore, stop reading the snapshot table in qcow2_do_open() in check
mode. Instead, add a new function qcow2_check_read_snapshot_table()
that reads the snapshot table at a later point. In the future, we want
to handle errors here and fix them.
Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-id:
20191011152814.14791-9-mreitz@redhat.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
Max Reitz [Fri, 11 Oct 2019 15:28:05 +0000 (17:28 +0200)]
qcow2: Write v3-compliant snapshot list on upgrade
qcow2 v3 requires every snapshot table entry to have two extra data
fields: The 64-bit VM state size, and the virtual disk size. Both are
optional for v2 images, so they may not be present.
qcow2_upgrade() therefore should update the snapshot table to ensure all
entries have these extra data fields.
Buglink: https://bugzilla.redhat.com/show_bug.cgi?id=1727347
Reported-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-id:
20191011152814.14791-8-mreitz@redhat.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
Max Reitz [Fri, 11 Oct 2019 15:28:04 +0000 (17:28 +0200)]
qcow2: Put qcow2_upgrade() into its own function
This does not make sense right now, but it will make sense once we need
to do more than to just update s->qcow_version.
Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-id:
20191011152814.14791-7-mreitz@redhat.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
Max Reitz [Fri, 11 Oct 2019 15:28:03 +0000 (17:28 +0200)]
qcow2: Make qcow2_write_snapshots() public
Updating the snapshot list will be useful when upgrading a v2 image to
v3, so we will need to call this function in qcow2.c.
Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-id:
20191011152814.14791-6-mreitz@redhat.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
Max Reitz [Fri, 11 Oct 2019 15:28:02 +0000 (17:28 +0200)]
qcow2: Keep unknown extra snapshot data
The qcow2 specification says to ignore unknown extra data fields in
snapshot table entries. Currently, we discard it whenever we update the
image, which is a bit different from "ignore".
This patch makes the qcow2 driver keep all unknown extra data fields
when updating an image's snapshot table.
Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-id:
20191011152814.14791-5-mreitz@redhat.com
[mreitz: Adjusted comments as proposed by Eric]
Signed-off-by: Max Reitz <mreitz@redhat.com>
Max Reitz [Fri, 11 Oct 2019 15:28:01 +0000 (17:28 +0200)]
qcow2: Add Error ** to qcow2_read_snapshots()
Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-id:
20191011152814.14791-4-mreitz@redhat.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
Max Reitz [Fri, 11 Oct 2019 15:28:00 +0000 (17:28 +0200)]
qcow2: Use endof()
Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-id:
20191011152814.14791-3-mreitz@redhat.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
Max Reitz [Fri, 11 Oct 2019 15:27:59 +0000 (17:27 +0200)]
include: Move endof() up from hw/virtio/virtio.h
endof() is a useful macro, we can make use of it outside of virtio.
Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-id:
20191011152814.14791-2-mreitz@redhat.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
Max Reitz [Mon, 14 Oct 2019 15:39:28 +0000 (17:39 +0200)]
mirror: Do not dereference invalid pointers
mirror_exit_common() may be called twice (if it is called from
mirror_prepare() and fails, it will be called from mirror_abort()
again).
In such a case, many of the pointers in the MirrorBlockJob object will
already be freed. This can be seen most reliably for s->target, which
is set to NULL (and then dereferenced by blk_bs()).
Cc: qemu-stable@nongnu.org
Fixes:
737efc1eda23b904fbe0e66b37715fb0e5c3e58b
Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: John Snow <jsnow@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-id:
20191014153931.20699-2-mreitz@redhat.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
Maxim Levitsky [Fri, 13 Sep 2019 13:36:27 +0000 (16:36 +0300)]
block/nvme: add support for discard
Signed-off-by: Maxim Levitsky <mlevitsk@redhat.com>
Message-id:
20190913133627.28450-3-mlevitsk@redhat.com
Reviewed-by: John Snow <jsnow@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
Maxim Levitsky [Fri, 13 Sep 2019 13:36:26 +0000 (16:36 +0300)]
block/nvme: add support for write zeros
Signed-off-by: Maxim Levitsky <mlevitsk@redhat.com>
Message-id:
20190913133627.28450-2-mlevitsk@redhat.com
Reviewed-by: John Snow <jsnow@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
Vladimir Sementsov-Ogievskiy [Tue, 22 Oct 2019 11:18:05 +0000 (14:18 +0300)]
block/block-copy: increase buffered copy request
No reason to limit buffered copy to one cluster. Let's allow up to 1
MiB.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-id:
20191022111805.3432-7-vsementsov@virtuozzo.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
Vladimir Sementsov-Ogievskiy [Tue, 22 Oct 2019 11:18:04 +0000 (14:18 +0300)]
block/block-copy: add memory limit
Currently total allocation for parallel requests to block-copy instance
is unlimited. Let's limit it to 128 MiB.
For now block-copy is used only in backup, so actually we limit total
allocation for backup job.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-id:
20191022111805.3432-6-vsementsov@virtuozzo.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
Vladimir Sementsov-Ogievskiy [Tue, 22 Oct 2019 11:18:03 +0000 (14:18 +0300)]
util: introduce SharedResource
Introduce an API for some shared splittable resource, like memory.
It's going to be used by backup. Backup uses both read/write io and
copy_range. copy_range may consume memory implictly, so the new API is
abstract: it doesn't allocate any real memory but only hands out
tickets.
The idea is that we have some total amount of something and callers
should wait in coroutine queue if there is not enough of the resource
at the moment.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-id:
20191022111805.3432-5-vsementsov@virtuozzo.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
Vladimir Sementsov-Ogievskiy [Tue, 22 Oct 2019 11:18:02 +0000 (14:18 +0300)]
block/block-copy: refactor copying
Merge copying code into one function block_copy_do_copy, which only
calls bdrv_ io functions and don't do any synchronization (like dirty
bitmap set/reset).
Refactor block_copy() function so that it takes full decision about
size of chunk to be copied and does all the synchronization (checking
intersecting requests, set/reset dirty bitmaps).
It will help:
- introduce parallel processing of block_copy iterations: we need to
calculate chunk size, start async chunk copying and go to the next
iteration
- simplify synchronization improvement (like memory limiting in
further commit and reducing critical section (now we lock the whole
requested range, when actually we need to lock only dirty region
which we handle at the moment))
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-id:
20191022111805.3432-4-vsementsov@virtuozzo.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
Vladimir Sementsov-Ogievskiy [Tue, 22 Oct 2019 11:18:01 +0000 (14:18 +0300)]
block/block-copy: limit copy_range_size to 16 MiB
Large copy range may imply memory allocation and large io effort, so
using 2G copy range request may be bad idea. Let's limit it to 16 MiB.
It also helps the following patch to refactor copy-with-offload
fallback to copy-with-bounce-buffer.
Note, that total memory usage of backup is still not limited, it will
be fixed in further commit.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-id:
20191022111805.3432-3-vsementsov@virtuozzo.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
Vladimir Sementsov-Ogievskiy [Tue, 22 Oct 2019 11:18:00 +0000 (14:18 +0300)]
block/block-copy: allocate buffer in block_copy_with_bounce_buffer
Move bounce_buffer allocation block_copy_with_bounce_buffer. This
commit simplifies further work on implementing copying by larger chunks
(of different size) and further asynchronous handling of block_copy
iterations (with help of block/aio_task API).
Allocation works fast, a lot faster than disk io, so it's not a problem
that we now allocate/free bounce_buffer more times. And we anyway will
have to allocate several bounce_buffers for parallel execution of loop
iterations in future.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-id:
20191022111805.3432-2-vsementsov@virtuozzo.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
Max Reitz [Thu, 17 Oct 2019 13:31:55 +0000 (15:31 +0200)]
iotests: Drop TEST_DIR filter from _filter_nbd
Sockets should be placed into $SOCK_DIR instead of $TEST_DIR, so remove
the $TEST_DIR filter from _filter_nbd.
Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-id:
20191017133155.5327-24-mreitz@redhat.com
Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
Max Reitz [Thu, 17 Oct 2019 13:31:54 +0000 (15:31 +0200)]
iotests/267: Create socket in $SOCK_DIR
Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-id:
20191017133155.5327-23-mreitz@redhat.com
Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
Max Reitz [Thu, 17 Oct 2019 13:31:53 +0000 (15:31 +0200)]
iotests/240: Create socket in $SOCK_DIR
Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-id:
20191017133155.5327-22-mreitz@redhat.com
Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
Max Reitz [Thu, 17 Oct 2019 13:31:52 +0000 (15:31 +0200)]
iotests/223: Create socket in $SOCK_DIR
Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-id:
20191017133155.5327-21-mreitz@redhat.com
Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
Max Reitz [Thu, 17 Oct 2019 13:31:51 +0000 (15:31 +0200)]
iotests/222: Create socket in $SOCK_DIR
Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-id:
20191017133155.5327-20-mreitz@redhat.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
Max Reitz [Thu, 17 Oct 2019 13:31:50 +0000 (15:31 +0200)]
iotests/209: Create socket in $SOCK_DIR
Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-id:
20191017133155.5327-19-mreitz@redhat.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
Max Reitz [Thu, 17 Oct 2019 13:31:49 +0000 (15:31 +0200)]
iotests/208: Create socket in $SOCK_DIR
Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-id:
20191017133155.5327-18-mreitz@redhat.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
Max Reitz [Thu, 17 Oct 2019 13:31:48 +0000 (15:31 +0200)]
iotests/205: Create socket in $SOCK_DIR
Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-id:
20191017133155.5327-17-mreitz@redhat.com
Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
Max Reitz [Thu, 17 Oct 2019 13:31:47 +0000 (15:31 +0200)]
iotests/201: Create socket in $SOCK_DIR
Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-id:
20191017133155.5327-16-mreitz@redhat.com
Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
Max Reitz [Thu, 17 Oct 2019 13:31:46 +0000 (15:31 +0200)]
iotests/194: Create sockets in $SOCK_DIR
Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-id:
20191017133155.5327-15-mreitz@redhat.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
Max Reitz [Thu, 17 Oct 2019 13:31:45 +0000 (15:31 +0200)]
iotests/192: Create socket in $SOCK_DIR
Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-id:
20191017133155.5327-14-mreitz@redhat.com
Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
Max Reitz [Thu, 17 Oct 2019 13:31:44 +0000 (15:31 +0200)]
iotests/183: Create socket in $SOCK_DIR
Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-id:
20191017133155.5327-13-mreitz@redhat.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
Max Reitz [Thu, 17 Oct 2019 13:31:43 +0000 (15:31 +0200)]
iotests/182: Create socket in $SOCK_DIR
Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-id:
20191017133155.5327-12-mreitz@redhat.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
Max Reitz [Thu, 17 Oct 2019 13:31:42 +0000 (15:31 +0200)]
iotests/181: Create socket in $SOCK_DIR
Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-id:
20191017133155.5327-11-mreitz@redhat.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
Max Reitz [Thu, 17 Oct 2019 13:31:41 +0000 (15:31 +0200)]
iotests/147: Create socket in $SOCK_DIR
Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-id:
20191017133155.5327-10-mreitz@redhat.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
Max Reitz [Thu, 17 Oct 2019 13:31:40 +0000 (15:31 +0200)]
iotests/143: Create socket in $SOCK_DIR
Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-id:
20191017133155.5327-9-mreitz@redhat.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
Max Reitz [Thu, 17 Oct 2019 13:31:39 +0000 (15:31 +0200)]
iotests/140: Create socket in $SOCK_DIR
Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-id:
20191017133155.5327-8-mreitz@redhat.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
Max Reitz [Thu, 17 Oct 2019 13:31:38 +0000 (15:31 +0200)]
iotests/083: Create socket in $SOCK_DIR
Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-id:
20191017133155.5327-7-mreitz@redhat.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
Max Reitz [Thu, 17 Oct 2019 13:31:37 +0000 (15:31 +0200)]
iotests: Let common.nbd create socket in $SOCK_DIR
In addition, drop the nbd_unix_socket assignment in 241 because it does
not really do anything.
Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-id:
20191017133155.5327-6-mreitz@redhat.com
Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
Max Reitz [Thu, 17 Oct 2019 13:31:36 +0000 (15:31 +0200)]
iotests: Filter $SOCK_DIR
Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-id:
20191017133155.5327-5-mreitz@redhat.com
Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
Max Reitz [Thu, 17 Oct 2019 13:31:35 +0000 (15:31 +0200)]
iotests.py: Add @base_dir to FilePaths etc.
Specifying this optional parameter allows creating temporary files in
other directories than the test_dir; for example in sock_dir.
Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-id:
20191017133155.5327-4-mreitz@redhat.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
Max Reitz [Thu, 17 Oct 2019 13:31:34 +0000 (15:31 +0200)]
iotests.py: Store socket files in $SOCK_DIR
iotests.py itself does not store socket files, but machine.py and
qtest.py do. iotests.py needs to pass the respective path to them, and
they need to adhere to it.
Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-id:
20191017133155.5327-3-mreitz@redhat.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
Max Reitz [Thu, 17 Oct 2019 13:31:33 +0000 (15:31 +0200)]
iotests: Introduce $SOCK_DIR
Unix sockets generally have a maximum path length. Depending on your
$TEST_DIR, it may be exceeded and then all tests that create and use
Unix sockets there may fail.
Circumvent this by adding a new scratch directory specifically for
Unix socket files. It defaults to a temporary directory (mktemp -d)
that is completely removed after the iotests are done.
(By default, mktemp -d creates a /tmp/tmp.XXXXXXXXXX directory, which
should be short enough for our use cases.)
Use mkdir -p to create the directory (because it seems right), and do
the same for $TEST_DIR (because there is no reason for that to be
created in any different way).
Signed-off-by: Max Reitz <mreitz@redhat.com>
Message-id:
20191017133155.5327-2-mreitz@redhat.com
Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
Vladimir Sementsov-Ogievskiy [Fri, 11 Oct 2019 09:07:11 +0000 (12:07 +0300)]
Revert "mirror: Only mirror granularity-aligned chunks"
This reverts commit
9adc1cb49af8d4e54f57980b1eed5c0a4b2dafa6.
"mirror: Only mirror granularity-aligned chunks"
Since previous commit unaligned chunks are supported by
do_sync_target_write.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-id:
20191011090711.19940-6-vsementsov@virtuozzo.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
Vladimir Sementsov-Ogievskiy [Fri, 11 Oct 2019 09:07:10 +0000 (12:07 +0300)]
block/mirror: support unaligned write in active mirror
Prior
9adc1cb49af8d do_sync_target_write had a bug: it reset aligned-up
region in the dirty bitmap, which means that we may not copy some bytes
and assume them copied, which actually leads to producing corrupted
target.
So
9adc1cb49af8d forced dirty bitmap granularity to be
request_alignment for mirror-top filter, so we are not working with
unaligned requests. However forcing large alignment obviously decreases
performance of unaligned requests.
This commit provides another solution for the problem: if unaligned
padding is already dirty, we can safely ignore it, as
1. It's dirty, it will be copied by mirror_iteration anyway
2. It's dirty, so skipping it now we don't increase dirtiness of the
bitmap and therefore don't damage "synchronicity" of the
write-blocking mirror.
If unaligned padding is not dirty, we just write it, no reason to touch
dirty bitmap if we succeed (on failure we'll set the whole region
ofcourse, but we loss "synchronicity" on failure anyway).
Note: we need to disable dirty_bitmap, otherwise we will not be able to
see in do_sync_target_write bitmap state before current operation. We
may of course check dirty bitmap before the operation in
bdrv_mirror_top_do_write and remember it, but we don't need active
dirty bitmap for write-blocking mirror anyway.
New code-path is unused until the following commit reverts
9adc1cb49af8d.
Suggested-by: Denis V. Lunev <den@openvz.org>
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-id:
20191011090711.19940-5-vsementsov@virtuozzo.com
Reviewed-by: Max Reitz <mreitz@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
Vladimir Sementsov-Ogievskiy [Fri, 11 Oct 2019 09:07:09 +0000 (12:07 +0300)]
block/block-backend: add blk_co_pwritev_part
Add blk write function with qiov_offset parameter. It's needed for the
following commit.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-id:
20191011090711.19940-4-vsementsov@virtuozzo.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
Vladimir Sementsov-Ogievskiy [Fri, 11 Oct 2019 09:07:08 +0000 (12:07 +0300)]
block/mirror: simplify do_sync_target_write
do_sync_target_write is called from bdrv_mirror_top_do_write after
write/discard operation, all inside active_write/active_write_settle
protecting us from mirror iteration. So the whole area is dirty for
sure, no reason to examine dirty bitmap.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-id:
20191011090711.19940-3-vsementsov@virtuozzo.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
Vladimir Sementsov-Ogievskiy [Fri, 11 Oct 2019 09:07:07 +0000 (12:07 +0300)]
hbitmap: handle set/reset with zero length
Passing zero length to these functions leads to unpredicted results.
Zero-length set/reset may occur in active-mirror, on zero-length write
(which is unlikely, but not guaranteed to never happen).
Let's just do nothing on zero-length request.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-id:
20191011090711.19940-2-vsementsov@virtuozzo.com
Reviewed-by: Max Reitz <mreitz@redhat.com>
Cc: qemu-stable@nongnu.org
Signed-off-by: Max Reitz <mreitz@redhat.com>
Max Reitz [Tue, 17 Sep 2019 09:20:04 +0000 (11:20 +0200)]
iotests: Cache supported_formats()
Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: John Snow <jsnow@redhat.com>
Message-id:
20190917092004.999-8-mreitz@redhat.com
Reviewed-by: Andrey Shinkevich <andrey.shinkevich@virtuozzo.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
Max Reitz [Tue, 17 Sep 2019 09:20:03 +0000 (11:20 +0200)]
iotests: Test driver whitelisting in 136
null-aio may not be whitelisted. Skip all test cases that require it.
Signed-off-by: Max Reitz <mreitz@redhat.com>
Message-id:
20190917092004.999-7-mreitz@redhat.com
Reviewed-by: Andrey Shinkevich <andrey.shinkevich@virtuozzo.com>
Reviewed-by: John Snow <jsnow@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
Max Reitz [Tue, 17 Sep 2019 09:20:02 +0000 (11:20 +0200)]
iotests: Test driver whitelisting in 093
null-aio may not be whitelisted. Skip all test cases that require it.
(And skip the whole test if null-co is not whitelisted.)
Signed-off-by: Max Reitz <mreitz@redhat.com>
Message-id:
20190917092004.999-6-mreitz@redhat.com
Reviewed-by: Andrey Shinkevich <andrey.shinkevich@virtuozzo.com>
Reviewed-by: John Snow <jsnow@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
Max Reitz [Tue, 17 Sep 2019 09:20:01 +0000 (11:20 +0200)]
iotests: Let skip_if_unsupported accept a function
This lets tests use skip_if_unsupported() with a potentially variable
list of required formats.
Suggested-by: Kevin Wolf <kwolf@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
Message-id:
20190917092004.999-5-mreitz@redhat.com
Reviewed-by: Andrey Shinkevich <andrey.shinkevich@virtuozzo.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
Max Reitz [Tue, 17 Sep 2019 09:20:00 +0000 (11:20 +0200)]
iotests: Use case_skip() in skip_if_unsupported()
skip_if_unsupported() should use the stronger variant case_skip(),
because this allows it to be used even with setUp() (in a meaningful
way).
In the process, make it explicit what we expect the first argument of
the func_wrapper to be (namely something derived of QMPTestCase).
Signed-off-by: Max Reitz <mreitz@redhat.com>
Message-id:
20190917092004.999-4-mreitz@redhat.com
Reviewed-by: Andrey Shinkevich <andrey.shinkevich@virtuozzo.com>
Reviewed-by: John Snow <jsnow@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
Max Reitz [Tue, 17 Sep 2019 09:19:59 +0000 (11:19 +0200)]
iotests: Allow skipping test cases
case_notrun() does not actually skip the current test case. It just
adds a "notrun" note and then returns to the caller, who manually has to
skip the test. Generally, skipping a test case is as simple as
returning from the current function, but not always: For example, this
model does not allow skipping tests already in the setUp() function.
Thus, add a QMPTestCase.case_skip() function that invokes case_notrun()
and then self.skipTest(). To make this work, we need to filter the
information on how many test cases were skipped from the unittest
output.
Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: John Snow <jsnow@redhat.com>
Reviewed-by: Andrey Shinkevich <andrey.shinkevich@virtuozzo.com>
Message-id:
20190917092004.999-3-mreitz@redhat.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
Max Reitz [Tue, 17 Sep 2019 09:19:58 +0000 (11:19 +0200)]
iotests: Prefer null-co over null-aio
We use null-co basically everywhere in the iotests. Unless we want to
test null-aio specifically, we should use it instead (for consistency).
Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: John Snow <jsnow@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-id:
20190917092004.999-2-mreitz@redhat.com
Reviewed-by: Andrey Shinkevich <andrey.shinkevich@virtuozzo.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
Peter Maydell [Sun, 27 Oct 2019 19:44:59 +0000 (19:44 +0000)]
Merge remote-tracking branch 'remotes/bonzini/tags/for-upstream' into staging
* Bulgarian translation update (Alexander)
* RTC and PC refactorings (Hervé, Philippe, Sergio)
* RTC fix (Marcelo)
* More comprehensive MCE logging (Mario)
* x86 IGNNE implementation (Paolo)
* Microvm machine type (Sergio)
* Support for UMONITOR/UMWAIT/TPAUSE (Tao)
* Do not use %m in common code (Thomas)
* NoNonArchitecturalCoreSharing Hyper-V enlightenment (Vitaly)
* getpagesize cleanups (Wei)
# gpg: Signature made Sat 26 Oct 2019 14:39:56 BST
# gpg: using RSA key
BFFBD25F78C7AE83
# gpg: Good signature from "Paolo Bonzini <bonzini@gnu.org>" [full]
# gpg: aka "Paolo Bonzini <pbonzini@redhat.com>" [full]
# Primary key fingerprint: 46F5 9FBD 57D6 12E7 BFD4 E2F7 7E15 100C CD36 69B1
# Subkey fingerprint: F133 3857 4B66 2389 866C 7682 BFFB D25F 78C7 AE83
* remotes/bonzini/tags/for-upstream: (39 commits)
i386: implement IGNNE
target/i386: introduce cpu_set_fpus
target/i386: move FERR handling to target/i386
core: replace getpagesize() with qemu_real_host_page_size
audio: fix missing break
mc146818rtc: always register rtc to rtc list
mc146818rtc: Include mc146818rtc_regs.h directly in mc146818rtc.c
mc146818rtc: Move RTC_ISA_IRQ definition
mc146818rtc: move structure to header file
hw/i386/pc: Remove kvm_i386.h include
hw/i386/pc: Extract pc_i8259_create()
hw/i386/pc: Move gsi_state creation code
hw/i386/pc: Extract pc_gsi_create()
target/i386: Add support for save/load IA32_UMWAIT_CONTROL MSR
x86/cpu: Add support for UMONITOR/UMWAIT/TPAUSE
hw/timer/mc146818rtc: Only include qapi-commands-misc on I386
runstate: ignore exit request in finish migrate state
checkpatch: suggest qemu_real_host_page_size instead of getpagesize() or sysconf(_SC_PAGESIZE)
MAINTAINERS: add microvm related files
hw/i386: Introduce the microvm machine type
...
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Paolo Bonzini [Wed, 16 Oct 2019 08:34:39 +0000 (10:34 +0200)]
i386: implement IGNNE
Change the handling of port F0h writes and FPU exceptions to implement IGNNE.
The implementation mixes a bit what the chipset and processor do in real
hardware, but the effect is the same as what happens with actual FERR#
and IGNNE# pins: writing to port F0h asserts IGNNE# in addition to lowering
FP_IRQ; while clearing the SE bit in the FPU status word deasserts IGNNE#.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Paolo Bonzini [Wed, 16 Oct 2019 08:02:57 +0000 (10:02 +0200)]
target/i386: introduce cpu_set_fpus
In the next patch, this will provide a hook to detect clearing of
FSW.ES.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Paolo Bonzini [Wed, 16 Oct 2019 08:18:10 +0000 (10:18 +0200)]
target/i386: move FERR handling to target/i386
Move it out of pc.c since it is strictly tied to TCG. This is
almost exclusively code movement, the next patch will implement
IGNNE.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Wei Yang [Sun, 13 Oct 2019 02:11:45 +0000 (10:11 +0800)]
core: replace getpagesize() with qemu_real_host_page_size
There are three page size in qemu:
real host page size
host page size
target page size
All of them have dedicate variable to represent. For the last two, we
use the same form in the whole qemu project, while for the first one we
use two forms: qemu_real_host_page_size and getpagesize().
qemu_real_host_page_size is defined to be a replacement of
getpagesize(), so let it serve the role.
[Note] Not fully tested for some arch or device.
Signed-off-by: Wei Yang <richardw.yang@linux.intel.com>
Message-Id: <
20191013021145.16011-3-richardw.yang@linux.intel.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Paolo Bonzini [Wed, 23 Oct 2019 08:24:20 +0000 (10:24 +0200)]
audio: fix missing break
Reported by Coverity (CID
1406449).
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Paolo Bonzini [Sat, 26 Oct 2019 13:36:22 +0000 (15:36 +0200)]
Merge commit '
df84f17' into HEAD
This merge fixes a semantic conflict with the trivial tree.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Peter Maydell [Sat, 26 Oct 2019 09:13:48 +0000 (10:13 +0100)]
Merge remote-tracking branch 'remotes/stsquad/tags/pull-testing-next-251019-3' into staging
Testing updates (split from mega PR)
- various Travis dependency updates
- enable tcg debug for check-tcg
- additional Xcode build for Cirrus
- dependency tweak for gitlab
# gpg: Signature made Fri 25 Oct 2019 20:35:56 BST
# gpg: using RSA key
6685AE99E75167BCAFC8DF35FBD0DB095A9E2A44
# gpg: Good signature from "Alex Bennée (Master Work Key) <alex.bennee@linaro.org>" [full]
# Primary key fingerprint: 6685 AE99 E751 67BC AFC8 DF35 FBD0 DB09 5A9E 2A44
* remotes/stsquad/tags/pull-testing-next-251019-3:
tests/docker: update Travis image to a more current version
tests/docker: set HOST_ARCH if we don't have ARCH
travis.yml: --enable-debug-tcg to check-tcg
gitlab-ci.yml: Use libvdeplug-dev to compile-test the VDE network backend
travis.yml: cache the clang sanitizer build
tests/vm/netbsd: Disable IPv6
tests/vm: Let subclasses disable IPv6
cirrus.yml: add latest Xcode build target
travis.yml: bump Xcode 10 to latest dot release
travis.yml: Test the release tarball
travis.yml: Fix the ccache lines
travis.yml: Use newer version of libgnutls and libpng
travis.yml: Use libsdl2 instead of libsdl1.2, and install libsdl2-image
travis.yml: Add libvdeplug-dev to compile-test net/vde.c
travis.yml: reduce scope of the --enable-debug build
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Peter Maydell [Fri, 25 Oct 2019 20:57:41 +0000 (21:57 +0100)]
Merge remote-tracking branch 'remotes/stefanha/tags/block-pull-request' into staging
Pull request
# gpg: Signature made Fri 25 Oct 2019 20:18:23 BST
# gpg: using RSA key
8695A8BFD3F97CDAAC35775A9CA4ABB381AB73C8
# gpg: Good signature from "Stefan Hajnoczi <stefanha@redhat.com>" [full]
# gpg: aka "Stefan Hajnoczi <stefanha@gmail.com>" [full]
# Primary key fingerprint: 8695 A8BF D3F9 7CDA AC35 775A 9CA4 ABB3 81AB 73C8
* remotes/stefanha/tags/block-pull-request:
yield_until_fd_readable: make it work with any AioContect
virtio-blk: Add blk_drain() to virtio_blk_device_unrealize()
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Alex Bennée [Thu, 24 Oct 2019 10:14:32 +0000 (11:14 +0100)]
tests/docker: update Travis image to a more current version
This isn't the latest one available on hub.docker.com but it does
match the ID reported by the Xenial builds running on Travis:
instance: ... travis-ci-sardonyx-xenial-
1553530528-
f909ac5
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Alex Bennée [Mon, 7 Oct 2019 19:03:44 +0000 (19:03 +0000)]
tests/docker: set HOST_ARCH if we don't have ARCH
As the docker rules want to be able to be run on a virgin unconfigured
checkout add a fallback and use it if we need to.
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Alex Bennée [Wed, 23 Oct 2019 16:47:51 +0000 (17:47 +0100)]
travis.yml: --enable-debug-tcg to check-tcg
This adds a whole bunch of asserts which will catch bugs you might
introduce into the TCG code.
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Thomas Huth [Wed, 16 Oct 2019 13:10:02 +0000 (15:10 +0200)]
gitlab-ci.yml: Use libvdeplug-dev to compile-test the VDE network backend
The libvdeplug-dev package is required to compile-test net/vde.c.
Signed-off-by: Thomas Huth <thuth@redhat.com>
Message-Id: <
20191016131002.29663-1-thuth@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Alex Bennée [Tue, 22 Oct 2019 13:07:40 +0000 (14:07 +0100)]
travis.yml: cache the clang sanitizer build
Hopefully we'll see the same benefits as the other builds.
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>