OSDN Git Service
Anton Nefedov [Thu, 6 Jul 2017 12:08:58 +0000 (15:08 +0300)]
virtio-console: chardev hotswap support
In case of a backend change, the handler functions and the watch have
to be reset.
Signed-off-by: Anton Nefedov <anton.nefedov@virtuozzo.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <
1499342940-56739-12-git-send-email-anton.nefedov@virtuozzo.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Anton Nefedov [Thu, 6 Jul 2017 12:08:57 +0000 (15:08 +0300)]
hmp: add hmp analogue for qmp-chardev-change
Signed-off-by: Anton Nefedov <anton.nefedov@virtuozzo.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Acked-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Message-Id: <
1499342940-56739-11-git-send-email-anton.nefedov@virtuozzo.com>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Anton Nefedov [Thu, 6 Jul 2017 12:08:56 +0000 (15:08 +0300)]
test-char: add hotswap test
Signed-off-by: Anton Nefedov <anton.nefedov@virtuozzo.com>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Message-Id: <
1499342940-56739-10-git-send-email-anton.nefedov@virtuozzo.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Anton Nefedov [Thu, 6 Jul 2017 12:08:55 +0000 (15:08 +0300)]
test-char: split char_file_test
makes it possible to test the existing chardev-file
Signed-off-by: Anton Nefedov <anton.nefedov@virtuozzo.com>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Message-Id: <
1499342940-56739-9-git-send-email-anton.nefedov@virtuozzo.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Anton Nefedov [Thu, 6 Jul 2017 12:08:54 +0000 (15:08 +0300)]
test-char: split char_udp_test
makes it possible to test the existing chardev-udp
Signed-off-by: Anton Nefedov <anton.nefedov@virtuozzo.com>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Message-Id: <
1499342940-56739-8-git-send-email-anton.nefedov@virtuozzo.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Anton Nefedov [Thu, 6 Jul 2017 12:08:53 +0000 (15:08 +0300)]
test-char: destroy chardev-udp after test
this is only not a problem if the test is last in a suite,
otherwise it makes the following main_loop() calls to fail
Signed-off-by: Anton Nefedov <anton.nefedov@virtuozzo.com>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Message-Id: <
1499342940-56739-7-git-send-email-anton.nefedov@virtuozzo.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Anton Nefedov [Thu, 6 Jul 2017 12:08:52 +0000 (15:08 +0300)]
char: avoid chardevice direct access
frontends should avoid accessing CharDriver struct where possible
Signed-off-by: Anton Nefedov <anton.nefedov@virtuozzo.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Message-Id: <
1499342940-56739-6-git-send-email-anton.nefedov@virtuozzo.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Anton Nefedov [Thu, 6 Jul 2017 12:08:51 +0000 (15:08 +0300)]
char: forbid direct chardevice access for hotswap devices
qemu_chr_fe_get_driver() is unsafe, frontends with hotswap support
should not access CharDriver ptr directly as CharDriver might change.
Signed-off-by: Anton Nefedov <anton.nefedov@virtuozzo.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Message-Id: <
1499342940-56739-5-git-send-email-anton.nefedov@virtuozzo.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Anton Nefedov [Thu, 6 Jul 2017 12:08:50 +0000 (15:08 +0300)]
char: chardevice hotswap
This patch adds a possibility to change a char device without a frontend
removal.
Ideally, it would have to happen transparently to a frontend, i.e.
frontend would continue its regular operation.
However, backends are not stateless and are set up by the frontends
via qemu_chr_fe_<> functions, and it's not (generally) possible to replay
that setup entirely in a backend code, as different chardevs respond
to the setup calls differently, so do frontends work differently basing
on those setup responses.
Moreover, some frontend can generally get and save the backend pointer
(qemu_chr_fe_get_driver()), and it will become invalid after backend change.
So, a frontend which would like to support chardev hotswap has to register
a "backend change" handler, and redo its backend setup there.
Signed-off-by: Anton Nefedov <anton.nefedov@virtuozzo.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <
1499342940-56739-4-git-send-email-anton.nefedov@virtuozzo.com>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Anton Nefedov [Thu, 6 Jul 2017 12:08:49 +0000 (15:08 +0300)]
char: add backend hotswap handler
Frontends should have an interface to setup the handler of a backend change.
The interface will be used in the next commits
Signed-off-by: Anton Nefedov <anton.nefedov@virtuozzo.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Message-Id: <
1499342940-56739-3-git-send-email-anton.nefedov@virtuozzo.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Anton Nefedov [Thu, 6 Jul 2017 12:08:48 +0000 (15:08 +0300)]
char: move QemuOpts->ChardevBackend translation to a separate func
parse function will be used by the following patch
Signed-off-by: Anton Nefedov <anton.nefedov@virtuozzo.com>
Message-Id: <
1499342940-56739-2-git-send-email-anton.nefedov@virtuozzo.com>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Marc-André Lureau [Thu, 6 Jul 2017 17:03:53 +0000 (19:03 +0200)]
chardev: block during sync read
A sync read should block until all requested data is
available (instead of retrying in qemu_chr_fe_read_all). Change the
channel to blocking during sync_read.
Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Message-Id: <
20170706170353.32601-1-marcandre.lureau@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Philippe Mathieu-Daudé [Thu, 29 Jun 2017 15:03:08 +0000 (12:03 -0300)]
MAINTAINERS: add entry for "Unimplemented" device
Also voluntary myself as reviewer
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-Id: <
20170629150308.22766-6-f4bug@amsat.org>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Philippe Mathieu-Daudé [Thu, 29 Jun 2017 15:03:07 +0000 (12:03 -0300)]
MAINTAINERS: update TCI entry
moved in
244f1441 to tcg/
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-Id: <
20170629150308.22766-5-f4bug@amsat.org>
Reviewed-by: Stefan Weil <sw@weilnetz.de>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Philippe Mathieu-Daudé [Thu, 29 Jun 2017 15:03:06 +0000 (12:03 -0300)]
MAINTAINERS: update Xen entries
moved in
56e2cd24..
28b99f47 to hw/xen/ and hw/i386/xen/
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Acked-by: Anthony PERARD <anthony.perard@citrix.com>
Message-Id: <
20170629150308.22766-4-f4bug@amsat.org>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Philippe Mathieu-Daudé [Thu, 29 Jun 2017 15:03:05 +0000 (12:03 -0300)]
MAINTAINERS: update KVM entries
moved in
92229a57 to accel/
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-Id: <
20170629150308.22766-3-f4bug@amsat.org>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Philippe Mathieu-Daudé [Thu, 29 Jun 2017 15:03:04 +0000 (12:03 -0300)]
MAINTAINERS: update TCG entries
moved in
a9ded601..
244f1441 to accel/
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-Id: <
20170629150308.22766-2-f4bug@amsat.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Paolo Bonzini [Tue, 11 Jul 2017 08:08:55 +0000 (10:08 +0200)]
build: add -Wexpansion-to-defined
This warning is included in -Wall by clang, but not by GCC (which only
enables it for -Wextra). Include it in the list of warnings we enable
to minimize the differences between the compilers:
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Peter Maydell [Thu, 13 Jul 2017 15:56:06 +0000 (16:56 +0100)]
Merge remote-tracking branch 'remotes/armbru/tags/pull-qapi-2017-07-12' into staging
QAPI patches for 2017-07-12
# gpg: Signature made Wed 12 Jul 2017 17:07:20 BST
# gpg: using RSA key 0x3870B400EB918653
# gpg: Good signature from "Markus Armbruster <armbru@redhat.com>"
# gpg: aka "Markus Armbruster <armbru@pond.sub.org>"
# Primary key fingerprint: 354B C8B3 D7EB 2A6B 6867 4E5F 3870 B400 EB91 8653
* remotes/armbru/tags/pull-qapi-2017-07-12:
scripts: use build_ prefix for string not piped through cgen()
qobject: Update coccinelle script to catch Q{INC, DEC}REF
qobject: Catch another straggler for use of qdict_put_str()
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Peter Maydell [Thu, 13 Jul 2017 15:15:52 +0000 (16:15 +0100)]
Merge remote-tracking branch 'remotes/mjt/tags/trivial-patches-fetch' into staging
trivial patches for 2017-07-12
# gpg: Signature made Wed 12 Jul 2017 14:58:43 BST
# gpg: using RSA key 0x701B4F6B1A693E59
# gpg: Good signature from "Michael Tokarev <mjt@tls.msk.ru>"
# gpg: aka "Michael Tokarev <mjt@corpit.ru>"
# gpg: aka "Michael Tokarev <mjt@debian.org>"
# Primary key fingerprint: 6EE1 95D1 886E 8FFB 810D 4324 457C E0A0 8044 65C5
# Subkey fingerprint: 7B73 BAD6 8BE7 A2C2 8931 4B22 701B 4F6B 1A69 3E59
* remotes/mjt/tags/trivial-patches-fetch:
include/hw/ptimer.h: Add documentation comments
hxtool: remove dead -q option
qga-win32: Fix memory leak of device information set
hw/core: fix missing return value in load_image_targphys_as()
elf-loader: warn about invalid endianness
configure: Handle having no c++ compiler in FORTIFY_SOURCE check
hw/pci: define msi_nonbroken in pci-stub
hw/misc: add missing includes
configure: Fix build with pkg-config and --static --enable-sdl
util/qemu-sockets: Drop unused helper socket_address_to_string()
target/xtensa: gdbstub: drop dead return statement
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Peter Maydell [Thu, 13 Jul 2017 12:38:57 +0000 (13:38 +0100)]
Merge remote-tracking branch 'remotes/maxreitz/tags/pull-block-2017-07-11' into staging
Block layer patches
# gpg: Signature made Tue 11 Jul 2017 17:05:56 BST
# gpg: using RSA key 0xF407DB0061D5CF40
# gpg: Good signature from "Max Reitz <mreitz@redhat.com>"
# Primary key fingerprint: 91BE B60A 30DB 3E88 57D1 1829 F407 DB00 61D5 CF40
* remotes/maxreitz/tags/pull-block-2017-07-11: (85 commits)
iotests: Add preallocated growth test for qcow2
iotests: Add preallocated resize test for raw
block/qcow2: falloc/full preallocating growth
block/qcow2: Rename "fail_block" to just "fail"
block/qcow2: Add qcow2_refcount_area()
block/qcow2: Metadata preallocation for truncate
block/qcow2: Lock s->lock in preallocate()
block/qcow2: Generalize preallocate()
block/file-posix: Preallocation for truncate
block/file-posix: Generalize raw_regular_truncate
block/file-posix: Extract raw_regular_truncate()
block/file-posix: Small fixes in raw_create()
qemu-img: Expose PreallocMode for resizing
block: Add PreallocMode to blk_truncate()
block: Add PreallocMode to bdrv_truncate()
block: Add PreallocMode to BD.bdrv_truncate()
iotests: add test 178 for qemu-img measure
qemu-iotests: support per-format golden output files
qemu-img: add measure subcommand
qcow2: add bdrv_measure() support
...
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Peter Maydell [Thu, 13 Jul 2017 11:48:37 +0000 (12:48 +0100)]
Merge remote-tracking branch 'remotes/yongbok/tags/mips-
20170711' into staging
MIPS patches 2017-07-11
Changes:
* Fix MSA copy_[s|u]_df corner case of rd = 0
* Update malta to load the initrd at the end of the low memory
# gpg: Signature made Tue 11 Jul 2017 15:42:20 BST
# gpg: using RSA key 0x2238EB86D5F797C2
# gpg: Good signature from "Yongbok Kim <yongbok.kim@imgtec.com>"
# gpg: WARNING: This key is not certified with a trusted signature!
# gpg: There is no indication that the signature belongs to the owner.
# Primary key fingerprint: 8600 4CF5 3415 A5D9 4CFA 2B5C 2238 EB86 D5F7 97C2
* remotes/yongbok/tags/mips-
20170711:
mips/malta: load the initrd at the end of the low memory
target/mips: fix msa copy_[s|u]_df rd = 0 corner case
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Peter Maydell [Thu, 13 Jul 2017 09:47:10 +0000 (10:47 +0100)]
Merge remote-tracking branch 'remotes/pmaydell/tags/pull-target-arm-
20170711' into staging
target-arm queue:
* v7M: ignore writes to CONTROL.SPSEL from Thread mode
* KVM: Enable in-kernel timers with user space gic
* aspeed: Register all watchdogs
* hw/misc: Add Exynos4210 Pseudo Random Number Generator
# gpg: Signature made Tue 11 Jul 2017 11:28:15 BST
# gpg: using RSA key 0x3C2525ED14360CDE
# gpg: Good signature from "Peter Maydell <peter.maydell@linaro.org>"
# gpg: aka "Peter Maydell <pmaydell@gmail.com>"
# gpg: aka "Peter Maydell <pmaydell@chiark.greenend.org.uk>"
# Primary key fingerprint: E1A5 C593 CD41 9DE2 8E83 15CF 3C25 25ED 1436 0CDE
* remotes/pmaydell/tags/pull-target-arm-
20170711:
target-arm: v7M: ignore writes to CONTROL.SPSEL from Thread mode
ARM: KVM: Enable in-kernel timers with user space gic
aspeed: Register all watchdogs
hw/misc: Add Exynos4210 Pseudo Random Number Generator
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Marc-André Lureau [Thu, 1 Jun 2017 12:41:41 +0000 (16:41 +0400)]
scripts: use build_ prefix for string not piped through cgen()
The gen_ prefix is awkward. Generated C should go through cgen()
exactly once (see commit
1f9a7a1). The common way to get this wrong is
passing a foo=gen_foo() keyword argument to mcgen(). I'd like us to
adopt a naming convention where gen_ means "something that's been piped
through cgen(), and thus must not be passed to cgen() or mcgen()".
Requires renaming gen_params(), gen_marshal_proto() and
gen_event_send_proto().
Suggested-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Message-Id: <
20170601124143.10915-1-marcandre.lureau@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Eric Blake [Sat, 24 Jun 2017 18:10:08 +0000 (12:10 -0600)]
qobject: Update coccinelle script to catch Q{INC, DEC}REF
The recent commit
b097efc0 used qobject_decref(QOBJECT(E)), even
though we already have QDECREF(E) for that purpose. We can update
our coccinelle script to catch any future relapses; with that in
place, the rest of the patch is generated with:
spatch --sp-file scripts/coccinelle/qobject.cocci \
--macro-file scripts/cocci-macro-file.h --dir . --in-place
Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <
20170624181008.25497-3-eblake@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Eric Blake [Sat, 24 Jun 2017 18:10:07 +0000 (12:10 -0600)]
qobject: Catch another straggler for use of qdict_put_str()
Dan's addition of key-secret improvements in commit
29cf9336 was
developed prior to the addition of QDict scalar insertion macros,
but merged after the general cleanup in commit
46f5ac20.
Patch created mechanically by rerunning:
spatch --sp-file scripts/coccinelle/qobject.cocci \
--macro-file scripts/cocci-macro-file.h --dir . --in-place
Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Alberto Garcia <berto@igalia.com>
Message-Id: <
20170624181008.25497-2-eblake@redhat.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Peter Maydell [Tue, 11 Jul 2017 16:13:49 +0000 (17:13 +0100)]
Merge remote-tracking branch 'remotes/stefanha/tags/tracing-pull-request' into staging
# gpg: Signature made Tue 11 Jul 2017 09:35:26 BST
# gpg: using RSA key 0x9CA4ABB381AB73C8
# gpg: Good signature from "Stefan Hajnoczi <stefanha@redhat.com>"
# gpg: aka "Stefan Hajnoczi <stefanha@gmail.com>"
# Primary key fingerprint: 8695 A8BF D3F9 7CDA AC35 775A 9CA4 ABB3 81AB 73C8
* remotes/stefanha/tags/tracing-pull-request:
backends: remove empty trace-events file
trace: Fix early setting of events with the "vcpu" property
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Max Reitz [Tue, 13 Jun 2017 20:21:07 +0000 (22:21 +0200)]
iotests: Add preallocated growth test for qcow2
Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-id:
20170613202107.10125-17-mreitz@redhat.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
Max Reitz [Tue, 13 Jun 2017 20:21:06 +0000 (22:21 +0200)]
iotests: Add preallocated resize test for raw
Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-id:
20170613202107.10125-16-mreitz@redhat.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
Max Reitz [Tue, 13 Jun 2017 20:21:05 +0000 (22:21 +0200)]
block/qcow2: falloc/full preallocating growth
Implement the preallocation modes falloc and full for growing qcow2
images.
Signed-off-by: Max Reitz <mreitz@redhat.com>
Message-id:
20170613202107.10125-15-mreitz@redhat.com
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
Max Reitz [Tue, 13 Jun 2017 20:21:04 +0000 (22:21 +0200)]
block/qcow2: Rename "fail_block" to just "fail"
Now alloc_refcount_block() only contains a single fail label, so it
makes more sense to just name it "fail" instead of "fail_block".
Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-id:
20170613202107.10125-14-mreitz@redhat.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
Max Reitz [Tue, 13 Jun 2017 20:21:03 +0000 (22:21 +0200)]
block/qcow2: Add qcow2_refcount_area()
This function creates a collection of self-describing refcount
structures (including a new refcount table) at the end of a qcow2 image
file. Optionally, these structures can also describe a number of
additional clusters beyond themselves; this will be important for
preallocated truncation, which will place the data clusters and L2
tables there.
For now, we can use this function to replace the part of
alloc_refcount_block() that grows the refcount table (from which it is
actually derived).
Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-id:
20170613202107.10125-13-mreitz@redhat.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
Max Reitz [Tue, 13 Jun 2017 20:21:02 +0000 (22:21 +0200)]
block/qcow2: Metadata preallocation for truncate
We can support PREALLOC_MODE_METADATA by invoking preallocate() in
qcow2_truncate().
Signed-off-by: Max Reitz <mreitz@redhat.com>
Message-id:
20170613202107.10125-12-mreitz@redhat.com
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
Max Reitz [Tue, 13 Jun 2017 20:21:01 +0000 (22:21 +0200)]
block/qcow2: Lock s->lock in preallocate()
preallocate() is and will be called only from places that do not
otherwise need to lock s->lock: Currently that is qcow2_create2(), as of
a future patch it will be called from qcow2_truncate(), too.
It therefore makes sense to move locking that mutex into preallocate()
itself.
Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-id:
20170613202107.10125-11-mreitz@redhat.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
Max Reitz [Tue, 13 Jun 2017 20:21:00 +0000 (22:21 +0200)]
block/qcow2: Generalize preallocate()
This patch adds two new parameters to the preallocate() function so we
will be able to use it not just for preallocating a new image but also
for preallocated image growth.
The offset parameter allows the caller to specify a virtual offset from
which to start preallocating. For newly created images this is always 0,
but for preallocating growth this will be the old image length.
The new_length parameter specifies the supposed new length of the image
(basically the "end offset" for preallocation). During image truncation,
bdrv_getlength() will return the old image length so we cannot rely on
its return value then.
Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-id:
20170613202107.10125-10-mreitz@redhat.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
Max Reitz [Tue, 13 Jun 2017 20:20:59 +0000 (22:20 +0200)]
block/file-posix: Preallocation for truncate
By using raw_regular_truncate() in raw_truncate(), we can now easily
support preallocation.
Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-id:
20170613202107.10125-9-mreitz@redhat.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
Max Reitz [Tue, 13 Jun 2017 20:20:58 +0000 (22:20 +0200)]
block/file-posix: Generalize raw_regular_truncate
Currently, raw_regular_truncate() is intended for setting the size of a
newly created file. However, we also want to use it for truncating an
existing file in which case only the newly added space (when growing)
should be preallocated.
This also means that if resizing failed, we should try to restore the
original file size. This is important when using preallocation.
Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-id:
20170613202107.10125-8-mreitz@redhat.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
Max Reitz [Tue, 13 Jun 2017 20:20:57 +0000 (22:20 +0200)]
block/file-posix: Extract raw_regular_truncate()
This functionality is part of raw_create() which we will be able to
reuse nicely in raw_truncate().
Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-id:
20170613202107.10125-7-mreitz@redhat.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
Max Reitz [Tue, 13 Jun 2017 20:20:56 +0000 (22:20 +0200)]
block/file-posix: Small fixes in raw_create()
Variables should be declared at the start of a block, and if a certain
parameter value is not supported it may be better to return -ENOTSUP
instead of -EINVAL.
Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-id:
20170613202107.10125-6-mreitz@redhat.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
Max Reitz [Tue, 13 Jun 2017 20:20:55 +0000 (22:20 +0200)]
qemu-img: Expose PreallocMode for resizing
Add a --preallocation command line option to qemu-img resize which can
be used to set the PreallocMode parameter of blk_truncate().
While touching this code, fix the fact that we did not handle errors
returned by blk_getlength().
Signed-off-by: Max Reitz <mreitz@redhat.com>
Message-id:
20170613202107.10125-5-mreitz@redhat.com
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
Max Reitz [Tue, 13 Jun 2017 20:20:54 +0000 (22:20 +0200)]
block: Add PreallocMode to blk_truncate()
blk_truncate() itself will pass that value to bdrv_truncate(), and all
callers of blk_truncate() just set the parameter to PREALLOC_MODE_OFF
for now.
Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-id:
20170613202107.10125-4-mreitz@redhat.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
Max Reitz [Tue, 13 Jun 2017 20:20:53 +0000 (22:20 +0200)]
block: Add PreallocMode to bdrv_truncate()
For block drivers that just pass a truncate request to the underlying
protocol, we can now pass the preallocation mode instead of aborting if
it is not PREALLOC_MODE_OFF.
Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-id:
20170613202107.10125-3-mreitz@redhat.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
Max Reitz [Tue, 13 Jun 2017 20:20:52 +0000 (22:20 +0200)]
block: Add PreallocMode to BD.bdrv_truncate()
Add a PreallocMode parameter to the bdrv_truncate() function implemented
by each block driver. Currently, we always pass PREALLOC_MODE_OFF and no
driver accepts anything else.
Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-id:
20170613202107.10125-2-mreitz@redhat.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
Stefan Hajnoczi [Wed, 5 Jul 2017 12:57:38 +0000 (13:57 +0100)]
iotests: add test 178 for qemu-img measure
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: Alberto Garcia <berto@igalia.com>
Message-id:
20170705125738.8777-10-stefanha@redhat.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
Stefan Hajnoczi [Wed, 5 Jul 2017 12:57:37 +0000 (13:57 +0100)]
qemu-iotests: support per-format golden output files
Some tests produce format-dependent output. Either the difference is
filtered out and ignored, or the test case is format-specific so we
don't need to worry about per-format output differences.
There is a third case: the test script is the same for all image formats
and the format-dependent output is relevant. An ugly workaround is to
copy-paste the test into multiple per-format test cases. This
duplicates code and is not maintainable.
This patch allows test cases to add per-format golden output files so a
single test case can work correctly when format-dependent output must be
checked:
123.out.qcow2
123.out.raw
123.out.vmdk
...
This naming scheme is not composable with 123.out.nocache or 123.pc.out,
two other scenarios where output files are split. I don't think it
matters since few test cases need these features.
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: Alberto Garcia <berto@igalia.com>
Message-id:
20170705125738.8777-9-stefanha@redhat.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
Stefan Hajnoczi [Wed, 5 Jul 2017 12:57:36 +0000 (13:57 +0100)]
qemu-img: add measure subcommand
The measure subcommand calculates the size required by a new image file.
This can be used by users or management tools that need to allocate
space on an LVM volume, SAN LUN, etc before creating or converting an
image file.
Suggested-by: Maor Lipchuk <mlipchuk@redhat.com>
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: Alberto Garcia <berto@igalia.com>
Message-id:
20170705125738.8777-8-stefanha@redhat.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
Stefan Hajnoczi [Wed, 5 Jul 2017 12:57:35 +0000 (13:57 +0100)]
qcow2: add bdrv_measure() support
Use qcow2_calc_prealloc_size() to get the required file size.
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: Alberto Garcia <berto@igalia.com>
Message-id:
20170705125738.8777-7-stefanha@redhat.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
Stefan Hajnoczi [Wed, 5 Jul 2017 12:57:34 +0000 (13:57 +0100)]
qcow2: extract image creation option parsing
The image creation options parsed by qcow2_create() are also needed to
implement .bdrv_measure(). Extract the parsing code, including input
validation.
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: Alberto Garcia <berto@igalia.com>
Message-id:
20170705125738.8777-6-stefanha@redhat.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
Stefan Hajnoczi [Wed, 5 Jul 2017 12:57:33 +0000 (13:57 +0100)]
qcow2: make refcount size calculation conservative
The refcount metadata size calculation is inaccurate and can produce
numbers that are too small. This is bad because we should calculate a
conservative number - one that is guaranteed to be large enough.
This patch switches the approach to a fixed point calculation because
the existing equation is hard to solve when inaccuracies are taken care
of.
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: Alberto Garcia <berto@igalia.com>
Message-id:
20170705125738.8777-5-stefanha@redhat.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
Stefan Hajnoczi [Wed, 5 Jul 2017 12:57:32 +0000 (13:57 +0100)]
qcow2: extract preallocation calculation function
Calculating the preallocated image size will be needed to implement
.bdrv_measure(). Extract the code out into a separate function.
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: Alberto Garcia <berto@igalia.com>
Message-id:
20170705125738.8777-4-stefanha@redhat.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
Stefan Hajnoczi [Wed, 5 Jul 2017 12:57:31 +0000 (13:57 +0100)]
raw-format: add bdrv_measure() support
Maximum size calculation is trivial for the raw format: it's just the
requested image size (because there is no metadata).
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: Alberto Garcia <berto@igalia.com>
Message-id:
20170705125738.8777-3-stefanha@redhat.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
Stefan Hajnoczi [Wed, 5 Jul 2017 12:57:30 +0000 (13:57 +0100)]
block: add bdrv_measure() API
bdrv_measure() provides a conservative maximum for the size of a new
image. This information is handy if storage needs to be allocated (e.g.
a SAN or an LVM volume) ahead of time.
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: Alberto Garcia <berto@igalia.com>
Message-id:
20170705125738.8777-2-stefanha@redhat.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
Eric Blake [Mon, 3 Jul 2017 18:09:50 +0000 (13:09 -0500)]
tests: Avoid non-portable 'echo -ARG'
POSIX says that backslashes in the arguments to 'echo', as well as
any use of 'echo -n' and 'echo -e', are non-portable; it recommends
people should favor 'printf' instead. This is definitely true where
we do not control which shell is running (such as in makefile snippets
or in documentation examples). But even for scripts where we
require bash (and therefore, where echo does what we want by default),
it is still possible to use 'shopt -s xpg_echo' to change bash's
behavior of echo. And setting a good example never hurts when we are
not sure if a snippet will be copied from a bash-only script to a
general shell script (although I don't change the use of non-portable
\e for ESC when we know the running shell is bash).
Replace 'echo -n "..."' with 'printf %s "..."', and 'echo -e "..."'
with 'printf %b "...\n"', with the optimization that the %s/%b
argument can be omitted if the string being printed is a strict
literal with no '%', '$', or '`' (we could technically also make
this optimization when there are $ or `` substitutions but where
we can prove their results will not be problematic, but proving
that such substitutions are safe makes the patch less trivial
compared to just being consistent).
In the qemu-iotests check script, fix unusual shell quoting
that would result in word-splitting if 'date' outputs a space.
In test 051, take an opportunity to shorten the line.
In test 068, get rid of a pointless second invocation of bash.
CC: qemu-trivial@nongnu.org
Signed-off-by: Eric Blake <eblake@redhat.com>
Message-id:
20170703180950.9895-1-eblake@redhat.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
Max Reitz [Sun, 2 Jul 2017 15:05:10 +0000 (17:05 +0200)]
iotests: Add test for colon handling
Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
Message-id:
20170702150510.23276-3-mreitz@redhat.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
Max Reitz [Sun, 2 Jul 2017 15:05:09 +0000 (17:05 +0200)]
iotests: Use absolute paths for executables
A user may specify a relative path for accessing qemu, qemu-img, etc.
through environment variables ($QEMU_PROG and friends) or a symlink.
If a test decides to change its working directory, relative paths will
cease to work, however. Work around this by making all of the paths to
programs that should undergo testing absolute. Besides "realpath", we
also have to use "type -p" to support programs in $PATH.
As a side effect, this fixes specifying these programs as symlinks for
out-of-tree builds: Before, you would have to create two symlinks, one
in the build and one in the source tree (the first one for common.config
to find, the second one for the iotest to use). Now it is sufficient to
create one in the build tree because common.config will resolve it.
Reported-by: Kevin Wolf <kwolf@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
Message-id:
20170702150510.23276-2-mreitz@redhat.com
Reviewed-by: Eric Blake <eblake@redhat.com>
Tested-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
Daniel P. Berrange [Mon, 26 Jun 2017 12:35:10 +0000 (13:35 +0100)]
iotests: chown LUKS device before qemu-io launches
On some distros, whenever you close a block device file
descriptor there is a udev rule that resets the file
permissions. This can race with the test script when
we run qemu-io multiple times against the same block
device. Occasionally the second qemu-io invocation
will find udev has reset the permissions causing failure.
Reviewed-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
Message-id:
20170626123510.20134-6-berrange@redhat.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
Daniel P. Berrange [Mon, 26 Jun 2017 12:35:09 +0000 (13:35 +0100)]
iotests: add more LUKS hash combination tests
Add tests for sha224, sha512, sha384 and ripemd160 hash
algorithms.
Reviewed-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
Message-id:
20170626123510.20134-5-berrange@redhat.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
Daniel P. Berrange [Mon, 26 Jun 2017 12:35:08 +0000 (13:35 +0100)]
iotests: reduce PBKDF iterations when testing LUKS
By default the PBKDF algorithm used with LUKS is tuned
based on the number of iterations to produce 1 second
of running time. This makes running the I/O test with
the LUKS format orders of magnitude slower than with
qcow2/raw formats.
When creating LUKS images, set the iteration time to
a 10ms to reduce the time overhead for LUKS, since
security does not matter in I/O tests.
Previously a full 'check -luks' would take
$ time ./check -luks
Passed all 22 tests
real 23m9.988s
user 21m46.223s
sys 0m22.841s
Now it takes
$ time ./check -luks
Passed all 22 tests
real 4m39.235s
user 3m29.590s
sys 0m24.234s
Still slow compared to qcow2/raw, but much improved
none the less.
Reviewed-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
Message-id:
20170626123510.20134-4-berrange@redhat.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
Daniel P. Berrange [Mon, 26 Jun 2017 12:35:07 +0000 (13:35 +0100)]
iotests: fix remainining tests to work with LUKS
The tests 033, 140, 145 and 157 were all broken
when run with LUKS, since they did not correctly use
the required image opts args syntax to specify the
decryption secret. Further, the 120 test simply does
not make sense to run with luks, as the scenario
exercised is not relevant.
The test 181 was broken when run with LUKS because
it didn't take account of fact that $TEST_IMG was
already in image opts syntax. The launch_qemu
helper also didn't register the secret object
providing the LUKS password.
Reviewed-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
Message-id:
20170626123510.20134-3-berrange@redhat.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
Daniel P. Berrange [Mon, 26 Jun 2017 12:35:06 +0000 (13:35 +0100)]
iotests: skip 159 & 170 with luks format
While the qemu-img dd command does accept --image-opts
this is not sufficient to make it work with the LUKS
image yet. This is because bdrv_create() still always
requires the non-image-opts syntax.
Thus we must skip 159/170 with luks for now
Reviewed-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
Message-id:
20170626123510.20134-2-berrange@redhat.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
Vladimir Sementsov-Ogievskiy [Wed, 28 Jun 2017 12:05:30 +0000 (15:05 +0300)]
block: release persistent bitmaps on inactivate
We should release them here to reload on invalidate cache.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-id:
20170628120530.31251-31-vsementsov@virtuozzo.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
Vladimir Sementsov-Ogievskiy [Wed, 28 Jun 2017 12:05:29 +0000 (15:05 +0300)]
qmp: block-dirty-bitmap-remove: remove persistent
Remove persistent bitmap from the storage on block-dirty-bitmap-remove.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: John Snow <jsnow@redhat.com>
Message-id:
20170628120530.31251-30-vsementsov@virtuozzo.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
Vladimir Sementsov-Ogievskiy [Wed, 28 Jun 2017 12:05:28 +0000 (15:05 +0300)]
qcow2: add .bdrv_remove_persistent_dirty_bitmap
Realize .bdrv_remove_persistent_dirty_bitmap interface.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: John Snow <jsnow@redhat.com>
Message-id:
20170628120530.31251-29-vsementsov@virtuozzo.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
Vladimir Sementsov-Ogievskiy [Wed, 28 Jun 2017 12:05:27 +0000 (15:05 +0300)]
block/dirty-bitmap: add bdrv_remove_persistent_dirty_bitmap
Interface for removing persistent bitmap from its storage.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: John Snow <jsnow@redhat.com>
Message-id:
20170628120530.31251-28-vsementsov@virtuozzo.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
Vladimir Sementsov-Ogievskiy [Wed, 28 Jun 2017 12:05:26 +0000 (15:05 +0300)]
iotests: test qcow2 persistent dirty bitmap
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-id:
20170628120530.31251-27-vsementsov@virtuozzo.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
Vladimir Sementsov-Ogievskiy [Wed, 28 Jun 2017 12:05:25 +0000 (15:05 +0300)]
qmp: add x-debug-block-dirty-bitmap-sha256
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-id:
20170628120530.31251-26-vsementsov@virtuozzo.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
Vladimir Sementsov-Ogievskiy [Wed, 28 Jun 2017 12:05:24 +0000 (15:05 +0300)]
qmp: add autoload parameter to block-dirty-bitmap-add
Optional. Default is false.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Signed-off-by: Denis V. Lunev <den@openvz.org>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: John Snow <jsnow@redhat.com>
Message-id:
20170628120530.31251-25-vsementsov@virtuozzo.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
Vladimir Sementsov-Ogievskiy [Wed, 28 Jun 2017 12:05:23 +0000 (15:05 +0300)]
qmp: add persistent flag to block-dirty-bitmap-add
Add optional 'persistent' flag to qmp command block-dirty-bitmap-add.
Default is false.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Signed-off-by: Denis V. Lunev <den@openvz.org>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: John Snow <jsnow@redhat.com>
Message-id:
20170628120530.31251-24-vsementsov@virtuozzo.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
Vladimir Sementsov-Ogievskiy [Wed, 28 Jun 2017 12:05:22 +0000 (15:05 +0300)]
qcow2: add .bdrv_can_store_new_dirty_bitmap
Realize .bdrv_can_store_new_dirty_bitmap interface.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: John Snow <jsnow@redhat.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-id:
20170628120530.31251-23-vsementsov@virtuozzo.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
Vladimir Sementsov-Ogievskiy [Wed, 28 Jun 2017 12:05:21 +0000 (15:05 +0300)]
block: add bdrv_can_store_new_dirty_bitmap
This will be needed to check some restrictions before making bitmap
persistent in qmp-block-dirty-bitmap-add (this functionality will be
added by future patch)
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: John Snow <jsnow@redhat.com>
Message-id:
20170628120530.31251-22-vsementsov@virtuozzo.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
Vladimir Sementsov-Ogievskiy [Wed, 28 Jun 2017 12:05:20 +0000 (15:05 +0300)]
qcow2: store bitmaps on reopening image as read-only
Store bitmaps and mark them read-only on reopening image as read-only.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-id:
20170628120530.31251-21-vsementsov@virtuozzo.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
Vladimir Sementsov-Ogievskiy [Wed, 28 Jun 2017 12:05:19 +0000 (15:05 +0300)]
qcow2: add persistent dirty bitmaps support
Store persistent dirty bitmaps in qcow2 image.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-id:
20170628120530.31251-20-vsementsov@virtuozzo.com
[mreitz: Always assign ret in store_bitmap() in case of an error]
Signed-off-by: Max Reitz <mreitz@redhat.com>
Vladimir Sementsov-Ogievskiy [Wed, 28 Jun 2017 12:05:18 +0000 (15:05 +0300)]
block/dirty-bitmap: add bdrv_dirty_bitmap_next()
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: John Snow <jsnow@redhat.com>
Message-id:
20170628120530.31251-19-vsementsov@virtuozzo.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
Vladimir Sementsov-Ogievskiy [Wed, 28 Jun 2017 12:05:17 +0000 (15:05 +0300)]
block: introduce persistent dirty bitmaps
New field BdrvDirtyBitmap.persistent means, that bitmap should be saved
by format driver in .bdrv_close and .bdrv_inactivate. No format driver
supports it for now.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-id:
20170628120530.31251-18-vsementsov@virtuozzo.com
[mreitz: Fixed indentation]
Signed-off-by: Max Reitz <mreitz@redhat.com>
Vladimir Sementsov-Ogievskiy [Wed, 28 Jun 2017 12:05:16 +0000 (15:05 +0300)]
block: bdrv_close: release bitmaps after drv->bdrv_close
Release bitmaps after 'if (bs->drv) { ... }' block. This will allow
format driver to save persistent bitmaps, which will appear in following
commits.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-id:
20170628120530.31251-17-vsementsov@virtuozzo.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
Vladimir Sementsov-Ogievskiy [Wed, 28 Jun 2017 12:05:15 +0000 (15:05 +0300)]
block/dirty-bitmap: add autoload field to BdrvDirtyBitmap
Mirror AUTO flag from Qcow2 bitmap in BdrvDirtyBitmap. This will be
needed in future, to save this flag back to Qcow2 for persistent
bitmaps.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-id:
20170628120530.31251-16-vsementsov@virtuozzo.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
Vladimir Sementsov-Ogievskiy [Wed, 28 Jun 2017 12:05:14 +0000 (15:05 +0300)]
qcow2: support .bdrv_reopen_bitmaps_rw
Realize bdrv_reopen_bitmaps_rw interface.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: John Snow <jsnow@redhat.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-id:
20170628120530.31251-15-vsementsov@virtuozzo.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
Vladimir Sementsov-Ogievskiy [Wed, 28 Jun 2017 12:05:13 +0000 (15:05 +0300)]
block: new bdrv_reopen_bitmaps_rw interface
Add format driver handler, which should mark loaded read-only
bitmaps as 'IN_USE' in the image and unset read_only field in
corresponding BdrvDirtyBitmap's.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: John Snow <jsnow@redhat.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-id:
20170628120530.31251-14-vsementsov@virtuozzo.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
Vladimir Sementsov-Ogievskiy [Wed, 28 Jun 2017 12:05:12 +0000 (15:05 +0300)]
block: refactor bdrv_reopen_commit
Add bs local variable to simplify code.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: John Snow <jsnow@redhat.com>
Message-id:
20170628120530.31251-13-vsementsov@virtuozzo.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
Vladimir Sementsov-Ogievskiy [Wed, 28 Jun 2017 12:05:11 +0000 (15:05 +0300)]
qcow2: autoloading dirty bitmaps
Auto loading bitmaps are bitmaps in Qcow2, with the AUTO flag set. They
are loaded when the image is opened and become BdrvDirtyBitmaps for the
corresponding drive.
Extra data in bitmaps is not supported for now.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: John Snow <jsnow@redhat.com>
Message-id:
20170628120530.31251-12-vsementsov@virtuozzo.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
Vladimir Sementsov-Ogievskiy [Wed, 28 Jun 2017 12:05:10 +0000 (15:05 +0300)]
block/dirty-bitmap: add readonly field to BdrvDirtyBitmap
It will be needed in following commits for persistent bitmaps.
If bitmap is loaded from read-only storage (and we can't mark it
"in use" in this storage) corresponding BdrvDirtyBitmap should be
read-only.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-id:
20170628120530.31251-11-vsementsov@virtuozzo.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
Vladimir Sementsov-Ogievskiy [Wed, 28 Jun 2017 12:05:09 +0000 (15:05 +0300)]
block/dirty-bitmap: fix comment for BlockDirtyBitmap.disabled field
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: John Snow <jsnow@redhat.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-id:
20170628120530.31251-10-vsementsov@virtuozzo.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
Vladimir Sementsov-Ogievskiy [Wed, 28 Jun 2017 12:05:08 +0000 (15:05 +0300)]
qcow2: add bitmaps extension
Add bitmap extension as specified in docs/specs/qcow2.txt.
For now, just mirror extension header into Qcow2 state and check
constraints. Also, calculate refcounts for qcow2 bitmaps, to not break
qemu-img check.
For now, disable image resize if it has bitmaps. It will be fixed later.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: John Snow <jsnow@redhat.com>
Message-id:
20170628120530.31251-9-vsementsov@virtuozzo.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
Vladimir Sementsov-Ogievskiy [Wed, 28 Jun 2017 12:05:07 +0000 (15:05 +0300)]
qcow2-refcount: rename inc_refcounts() and make it public
This is needed for the following patch, which will introduce refcounts
checking for qcow2 bitmaps.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: John Snow <jsnow@redhat.com>
Message-id:
20170628120530.31251-8-vsementsov@virtuozzo.com
[mreitz: s/inc_refcounts/qcow2_inc_refcounts_imrt/ in one more (new)
place]
Signed-off-by: Max Reitz <mreitz@redhat.com>
Vladimir Sementsov-Ogievskiy [Wed, 28 Jun 2017 12:05:06 +0000 (15:05 +0300)]
block/dirty-bitmap: add deserialize_ones func
Add bdrv_dirty_bitmap_deserialize_ones() function, which is needed for
qcow2 bitmap loading, to handle unallocated bitmap parts, marked as
all-ones.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Reviewed-by: John Snow <jsnow@redhat.com>
Message-id:
20170628120530.31251-7-vsementsov@virtuozzo.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
Vladimir Sementsov-Ogievskiy [Wed, 28 Jun 2017 12:05:05 +0000 (15:05 +0300)]
block: fix bdrv_dirty_bitmap_granularity signature
Make getter signature const-correct. This allows other functions with
const dirty bitmap parameter use bdrv_dirty_bitmap_granularity().
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: John Snow <jsnow@redhat.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Message-id:
20170628120530.31251-6-vsementsov@virtuozzo.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
Vladimir Sementsov-Ogievskiy [Wed, 28 Jun 2017 12:05:04 +0000 (15:05 +0300)]
tests: add hbitmap iter test
Test that hbitmap iter is resistant to bitmap resetting.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Signed-off-by: Denis V. Lunev <den@openvz.org>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: John Snow <jsnow@redhat.com>
Message-id:
20170628120530.31251-5-vsementsov@virtuozzo.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
Vladimir Sementsov-Ogievskiy [Wed, 28 Jun 2017 12:05:03 +0000 (15:05 +0300)]
hbitmap: improve dirty iter
Make dirty iter resistant to resetting bits in corresponding HBitmap.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: John Snow <jsnow@redhat.com>
Message-id:
20170628120530.31251-4-vsementsov@virtuozzo.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
Vladimir Sementsov-Ogievskiy [Wed, 28 Jun 2017 12:05:02 +0000 (15:05 +0300)]
specs/qcow2: do not use wording 'bitmap header'
A bitmap directory entry is sometimes called a 'bitmap header'. This
patch leaves only one name - 'bitmap directory entry'. The name 'bitmap
header' creates misunderstandings with 'qcow2 header' and 'qcow2 bitmap
header extension' (which is extension of qcow2 header)
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: John Snow <jsnow@redhat.com>
Message-id:
20170628120530.31251-3-vsementsov@virtuozzo.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
Vladimir Sementsov-Ogievskiy [Wed, 28 Jun 2017 12:05:01 +0000 (15:05 +0300)]
specs/qcow2: fix bitmap granularity qemu-specific note
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: John Snow <jsnow@redhat.com>
Message-id:
20170628120530.31251-2-vsementsov@virtuozzo.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
sochin.jiang [Mon, 26 Jun 2017 11:04:24 +0000 (19:04 +0800)]
mirror: Fix inconsistent backing AioContext for after mirroring
mirror_complete opens the backing chain, which should have the same
AioContext as the top when using iothreads. Make the code guarantee
this, which fixes a failed assertion in bdrv_attach_child.
Signed-off-by: sochin.jiang <sochin.jiang@huawei.com>
Message-id:
1498475064-39816-1-git-send-email-sochin.jiang@huawei.com
[mreitz: Reworded commit message]
Signed-off-by: Max Reitz <mreitz@redhat.com>
Max Reitz [Wed, 21 Jun 2017 13:11:57 +0000 (15:11 +0200)]
iotests: 181 does not work for all formats
Test 181 only works for formats which support live migration (naturally,
as it is a live migration test). Disable it for all formats which do
not.
Signed-off-by: Max Reitz <mreitz@redhat.com>
Message-id:
20170621131157.16584-1-mreitz@redhat.com
Reviewed-by: John Snow <jsnow@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
Daniel P. Berrange [Fri, 23 Jun 2017 16:24:19 +0000 (17:24 +0100)]
docs: document encryption options for qcow, qcow2 and luks
Expand the image format docs to cover the new options for
the qcow, qcow2 and luks disk image formats
Reviewed-by: Alberto Garcia <berto@igalia.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
Message-id:
20170623162419.26068-21-berrange@redhat.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
Daniel P. Berrange [Fri, 23 Jun 2017 16:24:18 +0000 (17:24 +0100)]
qcow2: report encryption specific image information
Currently 'qemu-img info' reports a simple "encrypted: yes"
field. This is not very useful now that qcow2 can support
multiple encryption formats. Users want to know which format
is in use and some data related to it.
Wire up usage of the qcrypto_block_get_info() method so that
'qemu-img info' can report about the encryption format
and parameters in use
$ qemu-img create \
--object secret,id=sec0,data=123456 \
-o encrypt.format=luks,encrypt.key-secret=sec0 \
-f qcow2 demo.qcow2 1G
Formatting 'demo.qcow2', fmt=qcow2 size=
1073741824 \
encryption=off encrypt.format=luks encrypt.key-secret=sec0 \
cluster_size=65536 lazy_refcounts=off refcount_bits=16
$ qemu-img info demo.qcow2
image: demo.qcow2
file format: qcow2
virtual size: 1.0G (
1073741824 bytes)
disk size: 480K
encrypted: yes
cluster_size: 65536
Format specific information:
compat: 1.1
lazy refcounts: false
refcount bits: 16
encrypt:
ivgen alg: plain64
hash alg: sha256
cipher alg: aes-256
uuid:
3fa930c4-58c8-4ef7-b3c5-
314bb5af21f3
format: luks
cipher mode: xts
slots:
[0]:
active: true
iters:
1839058
key offset: 4096
stripes: 4000
[1]:
active: false
key offset: 262144
[2]:
active: false
key offset: 520192
[3]:
active: false
key offset: 778240
[4]:
active: false
key offset:
1036288
[5]:
active: false
key offset:
1294336
[6]:
active: false
key offset:
1552384
[7]:
active: false
key offset:
1810432
payload offset:
2068480
master key iters: 438487
corrupt: false
With the legacy "AES" encryption we just report the format
name
$ qemu-img create \
--object secret,id=sec0,data=123456 \
-o encrypt.format=aes,encrypt.key-secret=sec0 \
-f qcow2 demo.qcow2 1G
Formatting 'demo.qcow2', fmt=qcow2 size=
1073741824 \
encryption=off encrypt.format=aes encrypt.key-secret=sec0 \
cluster_size=65536 lazy_refcounts=off refcount_bits=16
$ ./qemu-img info demo.qcow2
image: demo.qcow2
file format: qcow2
virtual size: 1.0G (
1073741824 bytes)
disk size: 196K
encrypted: yes
cluster_size: 65536
Format specific information:
compat: 1.1
lazy refcounts: false
refcount bits: 16
encrypt:
format: aes
corrupt: false
Reviewed-by: Alberto Garcia <berto@igalia.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
Message-id:
20170623162419.26068-20-berrange@redhat.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
Daniel P. Berrange [Fri, 23 Jun 2017 16:24:17 +0000 (17:24 +0100)]
block: pass option prefix down to crypto layer
While the crypto layer uses a fixed option name "key-secret",
the upper block layer may have a prefix on the options. e.g.
"encrypt.key-secret", in order to avoid clashes between crypto
option names & other block option names. To ensure the crypto
layer can report accurate error messages, we must tell it what
option name prefix was used.
Reviewed-by: Alberto Garcia <berto@igalia.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
Message-id:
20170623162419.26068-19-berrange@redhat.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
Daniel P. Berrange [Fri, 23 Jun 2017 16:24:16 +0000 (17:24 +0100)]
block: remove all encryption handling APIs
Now that all encryption keys must be provided upfront via
the QCryptoSecret API and associated block driver properties
there is no need for any explicit encryption handling APIs
in the block layer. Encryption can be handled transparently
within the block driver. We only retain an API for querying
whether an image is encrypted or not, since that is a
potentially useful piece of metadata to report to the user.
Reviewed-by: Alberto Garcia <berto@igalia.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
Message-id:
20170623162419.26068-18-berrange@redhat.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
Daniel P. Berrange [Fri, 23 Jun 2017 16:24:15 +0000 (17:24 +0100)]
block: rip out all traces of password prompting
Now that qcow & qcow2 are wired up to get encryption keys
via the QCryptoSecret object, nothing is relying on the
interactive prompting for passwords. All the code related
to password prompting can thus be ripped out.
Reviewed-by: Alberto Garcia <berto@igalia.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
Message-id:
20170623162419.26068-17-berrange@redhat.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
Daniel P. Berrange [Fri, 23 Jun 2017 16:24:14 +0000 (17:24 +0100)]
iotests: enable tests 134 and 158 to work with qcow (v1)
The 138 and 158 iotests exercise the legacy qcow2 aes encryption
code path and they work fine with qcow v1 too.
Reviewed-by: Alberto Garcia <berto@igalia.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
Message-id:
20170623162419.26068-16-berrange@redhat.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
Daniel P. Berrange [Fri, 23 Jun 2017 16:24:13 +0000 (17:24 +0100)]
qcow2: add iotests to cover LUKS encryption support
This extends the 087 iotest to cover LUKS encryption when doing
blockdev-add.
Two further tests are added to validate read/write of LUKS
encrypted images with a single file and with a backing file.
Reviewed-by: Alberto Garcia <berto@igalia.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
Message-id:
20170623162419.26068-15-berrange@redhat.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
Daniel P. Berrange [Fri, 23 Jun 2017 16:24:12 +0000 (17:24 +0100)]
qcow2: add support for LUKS encryption format
This adds support for using LUKS as an encryption format
with the qcow2 file, using the new encrypt.format parameter
to request "luks" format. e.g.
# qemu-img create --object secret,data=123456,id=sec0 \
-f qcow2 -o encrypt.format=luks,encrypt.key-secret=sec0 \
test.qcow2 10G
The legacy "encryption=on" parameter still results in
creation of the old qcow2 AES format (and is equivalent
to the new 'encryption-format=aes'). e.g. the following are
equivalent:
# qemu-img create --object secret,data=123456,id=sec0 \
-f qcow2 -o encryption=on,encrypt.key-secret=sec0 \
test.qcow2 10G
# qemu-img create --object secret,data=123456,id=sec0 \
-f qcow2 -o encryption-format=aes,encrypt.key-secret=sec0 \
test.qcow2 10G
With the LUKS format it is necessary to store the LUKS
partition header and key material in the QCow2 file. This
data can be many MB in size, so cannot go into the QCow2
header region directly. Thus the spec defines a FDE
(Full Disk Encryption) header extension that specifies
the offset of a set of clusters to hold the FDE headers,
as well as the length of that region. The LUKS header is
thus stored in these extra allocated clusters before the
main image payload.
Aside from all the cryptographic differences implied by
use of the LUKS format, there is one further key difference
between the use of legacy AES and LUKS encryption in qcow2.
For LUKS, the initialiazation vectors are generated using
the host physical sector as the input, rather than the
guest virtual sector. This guarantees unique initialization
vectors for all sectors when qcow2 internal snapshots are
used, thus giving stronger protection against watermarking
attacks.
Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
Message-id:
20170623162419.26068-14-berrange@redhat.com
Reviewed-by: Alberto Garcia <berto@igalia.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>