OSDN Git Service
Daniel P. Berrangé [Tue, 5 May 2020 10:24:25 +0000 (11:24 +0100)]
ui: update keycodemapdb submodule commit
Primarily this is to pull in a fix for Win32 keycodes. The other useful
change is the removal of build timestamp from generated files which is
desirable for reproducable builds.
The make rules need updating due to slightly changed CLI syntax - more
args must now come after the command name.
6119e6e19a050df847418de7babe5166779955e4 Fix scan codes for Korean keys
685684a8404301780714e8a89a871981e7cae988 Fix argument order in output headers
b3774853042c951b200d767697285781cc59a83c Add HTML entries for Korean layout keys
8e54850d800e4697a2798fb82ac740e760f8530b Add macOS entries for Japanese keyboards
27acf0ef828bf719b2053ba398b195829413dbdd Fix win32 keycode for VK_OEM_102
317d3eeb963a515e15a63fa356d8ebcda7041a51 Add support for generating RST formatted docs pages
7381b9bfadd31c4c9e9a10b5bb5032f9189d4352 Introduce separate args for title & subtitle with docs generator
6280c94f306df6a20bbc100ba15a5a81af0366e6 keymap-gen: Name sections in pod output
df4e56f8fab65ba714ec18f4e7338a966a1620ad Add an empty meson project
16e5b0787687d8904dad2c026107409eb9bfcb95 remove buildtime from generated files
044f21dd0d4f62519aae9f1d53a026407a0b664f add header file generators
7779876a6b06755e3bb2c94ee3ded50635bcb0fa c++: add extern declaration to the generated file
0e0a317889464397d6f1ae03aad0d2ca593aab04 move CLanguageGenerator closer to CLanguageGenerator itself
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
shiliyang [Tue, 3 Nov 2020 03:10:32 +0000 (11:10 +0800)]
crypto: Add spaces around operator
I am reading crypto related code, find some code style problems while
using checkpatch.pl to check crypto folder. Fix the error style
problems.
Signed-off-by: Liyang Shi <shiliyang@huawei.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Dan Streetman [Tue, 19 Jan 2021 17:20:46 +0000 (12:20 -0500)]
configure: replace --enable/disable-git-update with --with-git-submodules
Replace the --enable-git-update and --disable-git-update configure params
with the param --with-git-submodules=(update|validate|ignore) to
allow 3 options for building from a git repo.
This is needed because downstream packagers, e.g. Debian, Ubuntu, etc,
also keep the source code in git, but do not want to enable the
'git_update' mode; with the current code, that's not possible even
if the downstream package specifies --disable-git-update.
The previous parameters are deprecated but still available; the
--enable-git-update parameter maps to --with-git-submodules=update and
--disable-git-update parameter maps to --with-git-submodules=validate.
The configure script behavior is slightly modified, where previously
the dtc, capstone, and slirp submodules were not validated when
--disable-git-update was specified (but were updated with git-update
enabled), now they are validated when using --with-git-submodules=validate
and are only ignored when using --with-git-submodules=ignore.
Signed-off-by: Dan Streetman <ddstreet@canonical.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Daniel P. Berrangé [Fri, 27 Nov 2020 10:32:45 +0000 (10:32 +0000)]
docs: fix missing backslash in certtool shell example
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Daniel P. Berrangé [Thu, 17 Sep 2020 15:50:47 +0000 (16:50 +0100)]
docs: simplify and clarify the platform support rules
The distinction between short life and long life Linux distributions
turned out to be redundant. They can both be covered in a simple way
by noting support will target the current release, and the previous
release for a period of two years or until its EOL. This rule can also
apply to the other UNIX based distros, leaving only Windows needing a
different set of rules.
This also clarifies that Debian LTS is out of scope, because the LTS
support is provided by a separate group from the main Debian maintainer
team.
Reviewed-by: Eduardo Habkost <ehabkost@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Daniel P. Berrangé [Wed, 4 Nov 2020 13:57:21 +0000 (13:57 +0000)]
Prefer 'on' | 'off' over 'yes' | 'no' for bool options
Update some docs and test cases to use 'on' | 'off' as the preferred
value for bool options.
Reviewed-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Daniel P. Berrangé [Tue, 20 Oct 2020 16:08:27 +0000 (17:08 +0100)]
os: deprecate the -enable-fips option and QEMU's FIPS enforcement
The -enable-fips option was added a long time ago to prevent the use of
single DES when VNC when FIPS mode is enabled. It should never have been
added, because apps are supposed to unconditionally honour FIPS mode
based on the '/proc/sys/crypto/fips_enabled' file contents.
In addition there is more to achieving FIPS compliance than merely
blocking use of certain algorithms. Those algorithms which are used
need to perform self-tests at runtime.
QEMU's built-in cryptography provider has no support for self-tests,
and neither does the nettle library.
If QEMU is required to be used in a FIPS enabled host, then it must be
built with the libgcrypt library enabled, which will unconditionally
enforce FIPS compliance in any algorithm usage.
Thus there is no need to keep either the -enable-fips option in QEMU, or
QEMU's internal FIPS checking methods.
Reviewed-by: John Snow <jsnow@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Kevin Wolf [Mon, 30 Nov 2020 10:56:14 +0000 (11:56 +0100)]
crypto: Fix memory leaks in set_loaded for tls-*
If you set the loaded property to true when it was already true, the
state is overwritten without freeing the old state first. Change the
set_loaded callback so that it always frees the old state (which is a
no-op if nothing was loaded) and only then load if requestsd.
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Kevin Wolf [Mon, 30 Nov 2020 10:56:13 +0000 (11:56 +0100)]
crypto: Forbid broken unloading of secrets
qcrypto_secret_prop_set_loaded() forgets to reset secret->rawdata after
unloading a secret, which will lead to a double free at some point.
Because there is no use case for unloading an already loaded secret
(apart from deleting the whole secret object) and we know that nobody
could use this because it would lead to crashes, let's just forbid the
operation instead of fixing the unloading.
Eventually, we'll want to get rid of 'loaded' in the external interface,
but for the meantime this is more consistent with rng, which has a
similar property 'opened' that also can't be reset to false after it
became true.
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Kevin Wolf [Mon, 30 Nov 2020 10:56:12 +0000 (11:56 +0100)]
crypto: Move USER_CREATABLE to secret_common base class
Instead of duplicating the code for user creatable objects in secret and
secret_keyring, move it to the common base clase secret_common. As the
base class is abstract, it won't become user creatable itself.
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
shiliyang [Mon, 7 Dec 2020 08:37:25 +0000 (16:37 +0800)]
crypto: Fix some code style problems, add spaces around operator
This patch fixes error style problems found by checkpatch.pl:
ERROR: spaces required around that '*'
ERROR: space required after that ','
ERROR: spaces required around that '|'
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Liyang Shi <shiliyang@huawei.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Peter Maydell [Fri, 29 Jan 2021 10:10:43 +0000 (10:10 +0000)]
Merge remote-tracking branch 'remotes/vivier2/tags/trivial-branch-for-6.0-pull-request' into staging
Trivial patches
20210129
# gpg: Signature made Fri 29 Jan 2021 08:32:17 GMT
# gpg: using RSA key
CD2F75DDC8E3A4DC2E4F5173F30C38BD3F2FBE3C
# gpg: issuer "laurent@vivier.eu"
# gpg: Good signature from "Laurent Vivier <lvivier@redhat.com>" [full]
# gpg: aka "Laurent Vivier <laurent@vivier.eu>" [full]
# gpg: aka "Laurent Vivier (Red Hat) <lvivier@redhat.com>" [full]
# Primary key fingerprint: CD2F 75DD C8E3 A4DC 2E4F 5173 F30C 38BD 3F2F BE3C
* remotes/vivier2/tags/trivial-branch-for-6.0-pull-request:
target/rx: Fix compiler errors for build with sanitizers
net/slirp.c: Fix spelling error in error message
tcg/tci: Restrict tci_write_reg16() to 64-bit hosts
MAINTAINERS: Update 9pfs tree URL
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Stefan Weil [Thu, 28 Jan 2021 17:21:27 +0000 (18:21 +0100)]
target/rx: Fix compiler errors for build with sanitizers
gcc (Debian 10.2.1-6) 10.2.1
20210110 aborts builds with enabled sanitizers:
../../../target/rx/op_helper.c: In function ‘helper_scmpu’:
../../../target/rx/op_helper.c:213:24: error: ‘tmp1’ may be used uninitialized in this function [-Werror=maybe-uninitialized]
213 | env->psw_c = (tmp0 >= tmp1);
| ~~~~~~^~~~~~~~
../../../target/rx/op_helper.c:213:24: error: ‘tmp0’ may be used uninitialized in this function [-Werror=maybe-uninitialized]
../../../target/rx/op_helper.c: In function ‘helper_suntil’:
../../../target/rx/op_helper.c:299:23: error: ‘tmp’ may be used uninitialized in this function [-Werror=maybe-uninitialized]
299 | env->psw_c = (tmp <= env->regs[2]);
| ~~~~~^~~~~~~~~~~~~~~~
../../../target/rx/op_helper.c: In function ‘helper_swhile’:
../../../target/rx/op_helper.c:318:23: error: ‘tmp’ may be used uninitialized in this function [-Werror=maybe-uninitialized]
318 | env->psw_c = (tmp <= env->regs[2]);
| ~~~~~^~~~~~~~~~~~~~~~
Rewriting the code fixes those errors.
Signed-off-by: Stefan Weil <sw@weilnetz.de>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-Id: <
20210128172127.46041-1-sw@weilnetz.de>
Signed-off-by: Laurent Vivier <laurent@vivier.eu>
Doug Evans [Fri, 22 Jan 2021 00:42:51 +0000 (16:42 -0800)]
net/slirp.c: Fix spelling error in error message
DNS should be DHCP
Signed-off-by: Doug Evans <dje@google.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Samuel Thibault <samuel.thibault@ens-lyon.org>
Message-Id: <
20210122004251.843837-1-dje@google.com>
Signed-off-by: Laurent Vivier <laurent@vivier.eu>
Peter Maydell [Thu, 28 Jan 2021 22:43:18 +0000 (22:43 +0000)]
Merge remote-tracking branch 'remotes/armbru/tags/pull-qapi-2021-01-28' into staging
QAPI patches patches for 2021-01-28
# gpg: Signature made Thu 28 Jan 2021 07:10:21 GMT
# gpg: using RSA key
354BC8B3D7EB2A6B68674E5F3870B400EB918653
# gpg: issuer "armbru@redhat.com"
# gpg: Good signature from "Markus Armbruster <armbru@redhat.com>" [full]
# gpg: aka "Markus Armbruster <armbru@pond.sub.org>" [full]
# Primary key fingerprint: 354B C8B3 D7EB 2A6B 6867 4E5F 3870 B400 EB91 8653
* remotes/armbru/tags/pull-qapi-2021-01-28:
qapi: More complex uses of QAPI_LIST_APPEND
qapi: Use QAPI_LIST_APPEND in trivial cases
qapi: Introduce QAPI_LIST_APPEND
qapi: A couple more QAPI_LIST_PREPEND() stragglers
net: Clarify early exit condition
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Peter Maydell [Thu, 28 Jan 2021 12:30:30 +0000 (12:30 +0000)]
Merge remote-tracking branch 'remotes/kevin/tags/for-upstream' into staging
Block layer patches:
- Fix crash on write to read-only devices
- iotests: Rewrite 'check' in Python, get rid of 'groups' and allow
non-numeric test case names
# gpg: Signature made Wed 27 Jan 2021 19:56:00 GMT
# gpg: using RSA key
DC3DEB159A9AF95D3D7456FE7F09B272C88F2FD6
# gpg: issuer "kwolf@redhat.com"
# gpg: Good signature from "Kevin Wolf <kwolf@redhat.com>" [full]
# Primary key fingerprint: DC3D EB15 9A9A F95D 3D74 56FE 7F09 B272 C88F 2FD6
* remotes/kevin/tags/for-upstream:
iotests: rename and move 169 and 199 tests
iotests: rewrite check into python
iotests: add testrunner.py
iotests: add testenv.py
iotests: add findtests.py
iotests: 146: drop extra whitespaces from .out file
virtio-scsi-test: Test writing to scsi-cd device
block: Separate blk_is_writable() and blk_supports_write_perm()
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Peter Maydell [Thu, 28 Jan 2021 10:47:00 +0000 (10:47 +0000)]
Merge remote-tracking branch 'remotes/kraxel/tags/ui-
20210127-pull-request' into staging
vnc: fix gtk-vnc compatibility issues.
vnc: vmware svga cleanup
# gpg: Signature made Wed 27 Jan 2021 11:00:59 GMT
# gpg: using RSA key
A0328CFFB93A17A79901FE7D4CB6D8EED3E87138
# gpg: Good signature from "Gerd Hoffmann (work) <kraxel@redhat.com>" [full]
# gpg: aka "Gerd Hoffmann <gerd@kraxel.org>" [full]
# gpg: aka "Gerd Hoffmann (private) <kraxel@gmail.com>" [full]
# Primary key fingerprint: A032 8CFF B93A 17A7 9901 FE7D 4CB6 D8EE D3E8 7138
* remotes/kraxel/tags/ui-
20210127-pull-request:
hw/display/vmware_vga: Remove dependency on VNC header
vnc: send extended desktop resize on update requests
Revert "vnc: move initialization to framebuffer_update_request"
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Eric Blake [Wed, 13 Jan 2021 22:10:13 +0000 (16:10 -0600)]
qapi: More complex uses of QAPI_LIST_APPEND
These cases require a bit more thought to review; in each case, the
code was appending to a list, but not with a FOOList **tail variable.
Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <
20210113221013.390592-6-eblake@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
[Flawed change to qmp_guest_network_get_interfaces() dropped]
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Eric Blake [Wed, 13 Jan 2021 22:10:12 +0000 (16:10 -0600)]
qapi: Use QAPI_LIST_APPEND in trivial cases
The easiest spots to use QAPI_LIST_APPEND are where we already have an
obvious pointer to the tail of a list. While at it, consistently use
the variable name 'tail' for that purpose.
Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <
20210113221013.390592-5-eblake@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Eric Blake [Wed, 13 Jan 2021 22:10:11 +0000 (16:10 -0600)]
qapi: Introduce QAPI_LIST_APPEND
Similar to the existing QAPI_LIST_PREPEND, but designed for use where
we want to preserve insertion order. Callers will be added in
upcoming patches. Note the difference in signature: PREPEND takes
List*, APPEND takes List**.
Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <
20210113221013.390592-4-eblake@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Eric Blake [Wed, 13 Jan 2021 22:10:10 +0000 (16:10 -0600)]
qapi: A couple more QAPI_LIST_PREPEND() stragglers
Commit
54aa3de72e switched multiple sites to use QAPI_LIST_PREPEND
instead of open-coding, but missed a couple of spots.
Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <
20210113221013.390592-3-eblake@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Eric Blake [Wed, 13 Jan 2021 22:10:09 +0000 (16:10 -0600)]
net: Clarify early exit condition
On first glance, the loop in qmp_query_rx_filter() has early return
paths that could leak any allocation of filter_list from a previous
iteration. But on closer inspection, it is obvious that all of the
early exits are guarded by has_name, and that the bulk of the loop
body can be executed at most once if the user is filtering by name,
thus, any early exit coincides with an empty list. Add asserts to
make this obvious.
Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <
20210113221013.390592-2-eblake@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Vladimir Sementsov-Ogievskiy [Mon, 25 Jan 2021 18:50:56 +0000 (21:50 +0300)]
iotests: rename and move 169 and 199 tests
Rename bitmaps migration tests and move them to tests subdirectory to
demonstrate new human-friendly test naming.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <
20210125185056.129513-7-vsementsov@virtuozzo.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Vladimir Sementsov-Ogievskiy [Mon, 25 Jan 2021 18:50:55 +0000 (21:50 +0300)]
iotests: rewrite check into python
Just use classes introduced in previous three commits. Behavior
difference is described in these three commits.
Drop group file, as it becomes unused.
Drop common.env: now check is in python, and for tests we use same
python interpreter that runs the check itself. Use build environment
PYTHON in check-block instead, to keep "make check" use the same
python.
Checking for virtio-blk moved to iotests.py, as it actually iotests.py
dependency. Actually not all python iotests depend on it, so in future
it may be refactored to checked only when really needed.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <
20210125185056.129513-6-vsementsov@virtuozzo.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Vladimir Sementsov-Ogievskiy [Mon, 25 Jan 2021 18:50:54 +0000 (21:50 +0300)]
iotests: add testrunner.py
Add TestRunner class, which will run tests in a new python iotests
running framework.
There are some differences with current ./check behavior, most
significant are:
- Consider all tests self-executable, just run them, don't run python
by hand.
- Elapsed time is cached in json file
- Elapsed time precision increased a bit
- Instead of using "diff -w" which ignores all whitespace differences,
manually strip whitespace at line end then use python difflib, which
no longer ignores spacing mid-line
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <
20210125185056.129513-5-vsementsov@virtuozzo.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Vladimir Sementsov-Ogievskiy [Mon, 25 Jan 2021 18:50:53 +0000 (21:50 +0300)]
iotests: add testenv.py
Add TestEnv class, which will handle test environment in a new python
iotests running framework.
Don't add compat=1.1 for qcow2 IMGOPTS, as v3 is default anyway.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <
20210125185056.129513-4-vsementsov@virtuozzo.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Vladimir Sementsov-Ogievskiy [Mon, 25 Jan 2021 18:50:52 +0000 (21:50 +0300)]
iotests: add findtests.py
Add python script with new logic of searching for tests:
Current ./check behavior:
- tests are named [0-9][0-9][0-9]
- tests must be registered in group file (even if test doesn't belong
to any group, like 142)
Behavior of findtests.py:
- group file is dropped
- tests are all files in tests/ subdirectory (except for .out files),
so it's not needed more to "register the test", just create it with
appropriate name in tests/ subdirectory. Old names like
[0-9][0-9][0-9] (in root iotests directory) are supported too, but
not recommended for new tests
- groups are parsed from '# group: ' line inside test files
- optional file group.local may be used to define some additional
groups for downstreams
- 'disabled' group is used to temporary disable tests. So instead of
commenting tests in old 'group' file you now can add them to
disabled group with help of 'group.local' file
- selecting test ranges like 5-15 are not supported more
(to support restarting failed ./check command from the middle of the
process, new argument is added: --start-from)
Benefits:
- no rebase conflicts in group file on patch porting from branch to
branch
- no conflicts in upstream, when different series want to occupy same
test number
- meaningful names for test files
For example, with digital number, when some person wants to add some
test about block-stream, he most probably will just create a new
test. But if there would be test-block-stream test already, he will
at first look at it and may be just add a test-case into it.
And anyway meaningful names are better.
This commit doesn't update check behavior (which will be done in
further commit), still, the documentation changed like new behavior is
already here. Let's live with this small inconsistency for the
following few commits, until final change.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <
20210125185056.129513-3-vsementsov@virtuozzo.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Vladimir Sementsov-Ogievskiy [Mon, 25 Jan 2021 18:50:51 +0000 (21:50 +0300)]
iotests: 146: drop extra whitespaces from .out file
check script will be stricter soon about whitespaces, so fix 146.out
now.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <
20210125185056.129513-2-vsementsov@virtuozzo.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Kevin Wolf [Mon, 18 Jan 2021 12:34:48 +0000 (13:34 +0100)]
virtio-scsi-test: Test writing to scsi-cd device
This tests that trying to write to a (read-only) scsi-cd device backed
by a read-write image file doesn't crash and results in the correct
error.
This is a regression test for https://bugs.launchpad.net/bugs/
1906693.
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Message-Id: <
20210118123448.307825-3-kwolf@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Kevin Wolf [Mon, 18 Jan 2021 12:34:47 +0000 (13:34 +0100)]
block: Separate blk_is_writable() and blk_supports_write_perm()
Currently, blk_is_read_only() tells whether a given BlockBackend can
only be used in read-only mode because its root node is read-only. Some
callers actually try to answer a slightly different question: Is the
BlockBackend configured to be writable, by taking write permissions on
the root node?
This can differ, for example, for CD-ROM devices which don't take write
permissions, but may be backed by a writable image file. scsi-cd allows
write requests to the drive if blk_is_read_only() returns false.
However, the write request will immediately run into an assertion
failure because the write permission is missing.
This patch introduces separate functions for both questions.
blk_supports_write_perm() answers the question whether the block
node/image file can support writable devices, whereas blk_is_writable()
tells whether the BlockBackend is currently configured to be writable.
All calls of blk_is_read_only() are converted to one of the two new
functions.
Fixes: https://bugs.launchpad.net/bugs/
1906693
Cc: qemu-stable@nongnu.org
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Message-Id: <
20210118123448.307825-2-kwolf@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Peter Maydell [Wed, 27 Jan 2021 19:02:03 +0000 (19:02 +0000)]
Merge remote-tracking branch 'remotes/huth-gitlab/tags/pull-request-2021-01-27' into staging
* Patches to speed up and improve the gitlab-CI
* Documentation for the decorators in the "acceptance" tests
* One small rework of a libqtest function
# gpg: Signature made Wed 27 Jan 2021 06:22:11 GMT
# gpg: using RSA key
27B88847EEE0250118F3EAB92ED9D774FE702DB5
# gpg: issuer "thuth@redhat.com"
# gpg: Good signature from "Thomas Huth <th.huth@gmx.de>" [full]
# gpg: aka "Thomas Huth <thuth@redhat.com>" [full]
# gpg: aka "Thomas Huth <huth@tuxfamily.org>" [full]
# gpg: aka "Thomas Huth <th.huth@posteo.de>" [unknown]
# Primary key fingerprint: 27B8 8847 EEE0 2501 18F3 EAB9 2ED9 D774 FE70 2DB5
* remotes/huth-gitlab/tags/pull-request-2021-01-27:
libqtest: Rework qtest_rsp()
docs/devel: Explain how acceptance tests can be skipped
gitlab-ci.yml: Avoid recompiling the sources in the test jobs
gitlab-ci.yml: Exclude some redundant targets in build-without-default-features
meson: Do not build optional libraries by default
configure: Only check for audio drivers if system-mode is selected
gitlab-ci.yml: Avoid some submodules to speed up the CI a little bit
gitlab-ci: Test building linux-user targets on CentOS 7
tests/docker: Install static libc package in CentOS 7
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Peter Maydell [Wed, 27 Jan 2021 17:40:24 +0000 (17:40 +0000)]
Merge remote-tracking branch 'remotes/edgar/tags/edgar/xilinx-next-2021-01-27.for-upstream' into staging
For upstream
# gpg: Signature made Wed 27 Jan 2021 07:41:20 GMT
# gpg: using RSA key
AC44FEDC14F7F1EBEDBF415129C596780F6BCA83
# gpg: Good signature from "Edgar E. Iglesias (Xilinx key) <edgar.iglesias@xilinx.com>" [unknown]
# gpg: aka "Edgar E. Iglesias <edgar.iglesias@gmail.com>" [full]
# Primary key fingerprint: AC44 FEDC 14F7 F1EB EDBF 4151 29C5 9678 0F6B CA83
* remotes/edgar/tags/edgar/xilinx-next-2021-01-27.for-upstream:
target/microblaze: Add security attributes on memory transactions
target/microblaze: use MMUAccessType instead of int in mmu_translate
target/microblaze: Add use-non-secure property
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Philippe Mathieu-Daudé [Sat, 23 Jan 2021 09:41:07 +0000 (10:41 +0100)]
tcg/tci: Restrict tci_write_reg16() to 64-bit hosts
Restrict tci_write_reg16() to 64-bit hosts to fix on 32-bit ones:
[520/1115] Compiling C object libqemu-arm-linux-user.fa.p/tcg_tci.c.o
FAILED: libqemu-arm-linux-user.fa.p/tcg_tci.c.o
tcg/tci.c:132:1: error: 'tci_write_reg16' defined but not used [-Werror=unused-function]
tci_write_reg16(tcg_target_ulong *regs, TCGReg index, uint16_t value)
^~~~~~~~~~~~~~~
Fixes:
2f160e0f979 ("tci: Add implementation for INDEX_op_ld16u_i64")
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Stefan Weil <sw@weilnetz.de>
Message-Id: <
20210123094107.
2340222-1-f4bug@amsat.org>
Signed-off-by: Laurent Vivier <laurent@vivier.eu>
Greg Kurz [Fri, 15 Jan 2021 13:42:24 +0000 (14:42 +0100)]
MAINTAINERS: Update 9pfs tree URL
I've already moved my repositories to gitlab for extra CI coverage,
and I won't use the ones at github anymore.
Signed-off-by: Greg Kurz <groug@kaod.org>
Reviewed-by: Christian Schoenebeck <qemu_oss@crudebyte.com>
Message-Id: <
161071814430.152031.
14540382419012818908.stgit@bahia.lan>
Signed-off-by: Laurent Vivier <laurent@vivier.eu>
Peter Maydell [Tue, 12 Jan 2021 16:16:08 +0000 (16:16 +0000)]
hw/display/vmware_vga: Remove dependency on VNC header
In commit
2f487a3d40faff1 we fixed a problem observed with using the
vmware-vga device and the VNC UI frontend in a belt-and-braces
manner:
* we made the VNC frontend handle non-multiple-of-16 surface widths
* we rounded up the vmware-vga display width to a multiple of 16
However this introduced a spurious dependency of a device model on a
UI frontend header. vmware-vga isn't special and should not care
about what UI frontend it is using, and the VNC frontend needs to
handle arbitrary surface widths because other display device models
could use them. Moreover, even if the maximum width in vmware-vga is
made a multiple of 16, the guest itself can always program a
different width.
Remove the dependency on the VNC header. Since we have been using
the rounded-up width value since 2014, stick with it rather than
introducing a behaviour change, but don't calculate it by rounding up
to VNC_DIRTY_BITS_PER_PIXEL any more.
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Message-Id: <
20210112161608.16055-1-peter.maydell@linaro.org>
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
Gerd Hoffmann [Mon, 25 Jan 2021 10:40:41 +0000 (11:40 +0100)]
vnc: send extended desktop resize on update requests
Unlike other pseudo-encodings these don't break gtk-vnc
because older versions don't suport the extended desktop
resize extension in the first place.
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
Tested-by: Laszlo Ersek <lersek@redhat.com>
Message-Id: <
20210125104041.495274-3-kraxel@redhat.com>
Gerd Hoffmann [Mon, 25 Jan 2021 10:40:40 +0000 (11:40 +0100)]
Revert "vnc: move initialization to framebuffer_update_request"
This reverts commit
9e1632ad07ca49de99da4bb231e9e2f22f2d8df5.
Older gtk-vnc versions can't deal with non-incremental update
requests sending pseudo-encodings, so trying to send full server
state (including desktop size, cursor etc. which is done using
pseudo-encodings) doesn't fly. Return to old behavior to send
those only for new connects and when changes happen.
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
Tested-by: Laszlo Ersek <lersek@redhat.com>
Message-Id: <
20210125104041.495274-2-kraxel@redhat.com>
Joe Komlodi [Fri, 22 Jan 2021 00:18:55 +0000 (16:18 -0800)]
target/microblaze: Add security attributes on memory transactions
Using the cfg.use_non_secure bitfield and the MMU access type, we can determine
if the access should be secure or not.
Signed-off-by: Joe Komlodi <komlodi@xilinx.com>
Reviewed-by: Edgar E. Iglesias <edgar.iglesias@xilinx.com>
Tested-by: Edgar E. Iglesias <edgar.iglesias@xilinx.com>
Message-Id: <
1611274735-303873-4-git-send-email-komlodi@xilinx.com>
Signed-off-by: Edgar E. Iglesias <edgar.iglesias@xilinx.com>
Joe Komlodi [Fri, 22 Jan 2021 00:18:54 +0000 (16:18 -0800)]
target/microblaze: use MMUAccessType instead of int in mmu_translate
Using MMUAccessType makes it more clear what the variable's use is.
No functional change.
Signed-off-by: Joe Komlodi <komlodi@xilinx.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Edgar E. Iglesias <edgar.iglesias@xilinx.com>
Tested-by: Edgar E. Iglesias <edgar.iglesias@xilinx.com>
Message-Id: <
1611274735-303873-3-git-send-email-komlodi@xilinx.com>
Signed-off-by: Edgar E. Iglesias <edgar.iglesias@xilinx.com>
Joe Komlodi [Fri, 22 Jan 2021 00:18:53 +0000 (16:18 -0800)]
target/microblaze: Add use-non-secure property
This property is used to control the security of the following interfaces
on MicroBlaze:
M_AXI_DP - data interface
M_AXI_IP - instruction interface
M_AXI_DC - dcache interface
M_AXI_IC - icache interface
It works by enabling or disabling the use of the non_secure[3:0] signals.
Interfaces and their corresponding values are taken from:
https://www.xilinx.com/support/documentation/sw_manuals/xilinx2020_2/ug984-vivado-microblaze-ref.pdf
page 153.
Signed-off-by: Joe Komlodi <komlodi@xilinx.com>
Reviewed-by: Edgar E. Iglesias <edgar.iglesias@xilinx.com>
Tested-by: Edgar E. Iglesias <edgar.iglesias@xilinx.com>
Message-Id: <
1611274735-303873-2-git-send-email-komlodi@xilinx.com>
Signed-off-by: Edgar E. Iglesias <edgar.iglesias@xilinx.com>
Markus Armbruster [Tue, 26 Jan 2021 15:16:49 +0000 (16:16 +0100)]
libqtest: Rework qtest_rsp()
qtest_rsp() is used in two different ways: (1) return some arguments
to caller, which the caller must free, and (2) return no arguments to
caller. Passing non-zero @expected_args gets you (1), and passing
zero gets you (2).
Having "the return value must be freed" depend on an argument this way
is less than ideal.
Provide separate functions for the two ways: (1) qtest_rsp_args()
takes @expected_args (possibly zero), and returns that number of
arguments. Caller must free the return value always. (2) qtest_rsp()
assumes zero, and returns nothing.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Message-Id: <
20210126151649.
2220902-1-armbru@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Wainer dos Santos Moschetta [Fri, 15 Jan 2021 21:00:22 +0000 (18:00 -0300)]
docs/devel: Explain how acceptance tests can be skipped
Documented under the "Acceptance tests using the Avocado Framework"
section in testing.rst how environment variables are used to skip tests.
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Message-Id: <
20210115210022.417996-1-wainersm@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Thomas Huth [Tue, 26 Jan 2021 06:57:57 +0000 (07:57 +0100)]
gitlab-ci.yml: Avoid recompiling the sources in the test jobs
Currently, our check-system-* jobs are recompiling the whole sources
again. This happens due to the fact that the jobs are checking out
the whole source tree and required submodules again, and only try
to use the "build" directory with the binaries and object files as an
artifact from the previous stage - which simply does not work right
anymore (with the current version of meson). Due to some changed
time stamps, meson/ninja are always trying to rebuild the whole tree.
In the long run, we could likely use "meson test --no-rebuild", but
there is still some work going on in that area to improve the user
experience. So until this has been done, simply avoid recompiling the
sources with a trick: pass NINJA=":" to the make process in the test
jobs. Also check out the submodules manually before updating the
timestamps in the build folder, so that the binaries are definitely
newer that all the source files.
This saves ca. 10 - 15 minutes of precious CI cycles in each run.
Suggested-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <
20210126065757.403853-1-thuth@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Thomas Huth [Tue, 26 Jan 2021 17:23:45 +0000 (18:23 +0100)]
gitlab-ci.yml: Exclude some redundant targets in build-without-default-features
The build-without-default-features job is running quite long and sometimes
already hits the 1h time limit. Exclude some targets which do not provide
additional test coverage here (since we e.g. also already test other targets
of the same type, just with different endianess, or a 64-bit superset) to
avoid that we hit the timeout here so easily.
Message-Id: <
20210126172345.15947-1-thuth@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Peter Maydell [Tue, 26 Jan 2021 21:08:13 +0000 (21:08 +0000)]
Merge remote-tracking branch 'remotes/maxreitz/tags/pull-block-2021-01-26' into staging
Block patches:
- Make backup block jobs use asynchronous requests with the block-copy
module
- Use COR filter node for stream block jobs
- Make coroutine-sigaltstack’s qemu_coroutine_new() function thread-safe
- Report error string when file locking fails with an unexpected errno
- iotest fixes, additions, and some refactoring
# gpg: Signature made Tue 26 Jan 2021 14:16:59 GMT
# gpg: using RSA key
91BEB60A30DB3E8857D11829F407DB0061D5CF40
# gpg: issuer "mreitz@redhat.com"
# gpg: Good signature from "Max Reitz <mreitz@redhat.com>" [full]
# Primary key fingerprint: 91BE B60A 30DB 3E88 57D1 1829 F407 DB00 61D5 CF40
* remotes/maxreitz/tags/pull-block-2021-01-26: (53 commits)
iotests/178: Pass value to invalid option
iotests/118: Drop 'change' test
iotests: Add test for the regression fixed in
c8bf9a9169
block: report errno when flock fcntl fails
simplebench: add bench-backup.py
simplebench: bench_block_job: add cmd_options argument
simplebench/bench_block_job: use correct shebang line with python3
block/block-copy: drop unused argument of block_copy()
block/block-copy: drop unused block_copy_set_progress_callback()
qapi: backup: disable copy_range by default
backup: move to block-copy
block/backup: drop extra gotos from backup_run()
block/block-copy: make progress_bytes_callback optional
iotests: 257: prepare for backup over block-copy
iotests: 219: prepare for backup over block-copy
iotests: 185: prepare for backup over block-copy
iotests/129: Limit backup's max-chunk/max-workers
iotests: 56: prepare for backup over block-copy
qapi: backup: add max-chunk and max-workers to x-perf struct
job: call job_enter from job_pause
...
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Philippe Mathieu-Daudé [Fri, 22 Jan 2021 20:44:33 +0000 (21:44 +0100)]
meson: Do not build optional libraries by default
The following libraries will be selected if a feature requires it:
- capstone
- fdt
- SLiRP
Suggested-by: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <
20210122204441.
2145197-5-philmd@redhat.com>
Acked-by: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Philippe Mathieu-Daudé [Fri, 22 Jan 2021 20:44:30 +0000 (21:44 +0100)]
configure: Only check for audio drivers if system-mode is selected
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Acked-by: Gerd Hoffmann <kraxel@redhat.com>
Message-Id: <
20210122204441.
2145197-2-philmd@redhat.com>
Acked-by: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Thomas Huth [Thu, 21 Jan 2021 17:44:51 +0000 (18:44 +0100)]
gitlab-ci.yml: Avoid some submodules to speed up the CI a little bit
Since the meson build system rework, the configure script prefers the
git submodules over the system libraries. So we are testing compilation
with capstone, fdt and libslirp as a submodule all over the place,
burning CPU cycles by recompiling these third party modules and wasting
some network bandwidth in the CI by cloning the submodules each time.
Let's stop doing that in at least a couple of jobs and use the system
libraries instead.
While we're at it, also install meson in the Fedora container, since
it is new enough already, so we do not need to check out the meson
submodule here.
Message-Id: <
20210121174451.658924-1-thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Philippe Mathieu-Daudé [Thu, 21 Jan 2021 17:28:29 +0000 (18:28 +0100)]
gitlab-ci: Test building linux-user targets on CentOS 7
Add a configuration tested by Peter Maydell (see [1] and [2])
but not covered in our CI [3]:
[705/2910] Compiling C object libqemu-arm-linux-user.fa.p/linux-user_strace.c.o
FAILED: libqemu-arm-linux-user.fa.p/linux-user_strace.c.o
../linux-user/strace.c: In function 'do_print_sockopt':
../linux-user/strace.c:2831:14: error: 'IPV6_ADDR_PREFERENCES' undeclared (first use in this function)
case IPV6_ADDR_PREFERENCES:
^
This job currently takes 31 minutes 32 seconds ([4]).
[1] https://lists.gnu.org/archive/html/qemu-devel/2021-01/msg05086.html
[2] https://lists.gnu.org/archive/html/qemu-devel/2021-01/msg05379.html
[3] https://gitlab.com/philmd/qemu/-/jobs/
977408284
[4] https://gitlab.com/philmd/qemu/-/jobs/
978223286
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Message-Id: <
20210121172829.
1643620-3-f4bug@amsat.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Philippe Mathieu-Daudé [Thu, 21 Jan 2021 17:28:28 +0000 (18:28 +0100)]
tests/docker: Install static libc package in CentOS 7
We need to install the static libc package to be able to run
the TCG tests:
$ make check-tcg
...
BUILD TCG tests for x86_64-softmmu
BUILD x86_64-softmmu guest-tests with cc
/usr/bin/ld: hello: warning: allocated section `.notes' not in segment
/usr/bin/ld: memory: warning: allocated section `.notes' not in segment
BUILD TCG tests for x86_64-linux-user
BUILD x86_64-linux-user guest-tests with cc
/usr/bin/ld: cannot find -lpthread
/usr/bin/ld: cannot find -lc
collect2: error: ld returned 1 exit status
make[2]: *** [threadcount] Error 1
make[1]: *** [cross-build-guest-tests] Error 2
make: *** [build-tcg-tests-x86_64-linux-user] Error 2
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Message-Id: <
20210121172829.
1643620-2-f4bug@amsat.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Max Reitz [Tue, 26 Jan 2021 12:38:34 +0000 (13:38 +0100)]
iotests/178: Pass value to invalid option
ccd3b3b8112 has deprecated short-hand boolean options (i.e., options
with values). All options without values are interpreted as boolean
options, so this includes the invalid option "snapshot.foo" used in
iotest 178.
So after
ccd3b3b8112, 178 fails with:
+qemu-img: warning: short-form boolean option 'snapshot.foo' deprecated
+Please use snapshot.foo=on instead
Suppress that deprecation warning by passing some value to it (it does
not matter which, because the option is invalid anyway).
Fixes:
ccd3b3b8112b670fdccf8a392b8419b173ffccb4
("qemu-option: warn for short-form boolean options")
Signed-off-by: Max Reitz <mreitz@redhat.com>
Message-Id: <
20210126123834.115915-1-mreitz@redhat.com>
Max Reitz [Tue, 26 Jan 2021 10:48:33 +0000 (11:48 +0100)]
iotests/118: Drop 'change' test
Commit
0afec75734331 removed the 'change' QMP command, so we can no
longer test it in 118.
Fixes:
0afec75734331a0b52fa3aa4235220eda8c7846f
('qmp: remove deprecated "change" command')
Signed-off-by: Max Reitz <mreitz@redhat.com>
Message-Id: <
20210126104833.57026-1-mreitz@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Alberto Garcia [Tue, 12 Jan 2021 17:05:40 +0000 (18:05 +0100)]
iotests: Add test for the regression fixed in
c8bf9a9169
Signed-off-by: Alberto Garcia <berto@igalia.com>
Suggested-by: Maxim Levitsky <mlevitsk@redhat.com>
Reviewed-by: Maxim Levitsky <mlevitsk@redhat.com>
Message-Id: <
20210112170540.2912-1-berto@igalia.com>
[mreitz: Add "# group:" line]
Signed-off-by: Max Reitz <mreitz@redhat.com>
David Edmondson [Wed, 13 Jan 2021 16:44:47 +0000 (16:44 +0000)]
block: report errno when flock fcntl fails
When a call to fcntl(2) for the purpose of adding file locks fails
with an error other than EAGAIN or EACCES, report the error returned
by fcntl.
EAGAIN or EACCES are elided as they are considered to be common
failures, indicating that a conflicting lock is held by another
process.
No errors are elided when removing file locks.
Signed-off-by: David Edmondson <david.edmondson@oracle.com>
Message-Id: <
20210113164447.
2545785-1-david.edmondson@oracle.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
Vladimir Sementsov-Ogievskiy [Sat, 16 Jan 2021 21:47:05 +0000 (00:47 +0300)]
simplebench: add bench-backup.py
Add script to benchmark new backup architecture.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <
20210116214705.822267-24-vsementsov@virtuozzo.com>
[mreitz: s/not unsupported/not supported/]
Signed-off-by: Max Reitz <mreitz@redhat.com>
Vladimir Sementsov-Ogievskiy [Sat, 16 Jan 2021 21:47:04 +0000 (00:47 +0300)]
simplebench: bench_block_job: add cmd_options argument
Add argument to allow additional block-job options.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-Id: <
20210116214705.822267-23-vsementsov@virtuozzo.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
Vladimir Sementsov-Ogievskiy [Sat, 16 Jan 2021 21:47:03 +0000 (00:47 +0300)]
simplebench/bench_block_job: use correct shebang line with python3
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-Id: <
20210116214705.822267-22-vsementsov@virtuozzo.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
Vladimir Sementsov-Ogievskiy [Sat, 16 Jan 2021 21:47:02 +0000 (00:47 +0300)]
block/block-copy: drop unused argument of block_copy()
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-Id: <
20210116214705.822267-21-vsementsov@virtuozzo.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
Vladimir Sementsov-Ogievskiy [Sat, 16 Jan 2021 21:47:01 +0000 (00:47 +0300)]
block/block-copy: drop unused block_copy_set_progress_callback()
Drop unused code.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-Id: <
20210116214705.822267-20-vsementsov@virtuozzo.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
Vladimir Sementsov-Ogievskiy [Sat, 16 Jan 2021 21:47:00 +0000 (00:47 +0300)]
qapi: backup: disable copy_range by default
Further commit will add a benchmark
(scripts/simplebench/bench-backup.py), which will show that backup
works better with async parallel requests (previous commit) and
disabled copy_range. So, let's disable copy_range by default.
Note: the option was added several commits ago with default to true,
to follow old behavior (the feature was enabled unconditionally), and
only now we are going to change the default behavior.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-Id: <
20210116214705.822267-19-vsementsov@virtuozzo.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
Vladimir Sementsov-Ogievskiy [Sat, 16 Jan 2021 21:46:59 +0000 (00:46 +0300)]
backup: move to block-copy
This brings async request handling and block-status driven chunk sizes
to backup out of the box, which improves backup performance.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-Id: <
20210116214705.822267-18-vsementsov@virtuozzo.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
Vladimir Sementsov-Ogievskiy [Sat, 16 Jan 2021 21:46:58 +0000 (00:46 +0300)]
block/backup: drop extra gotos from backup_run()
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-Id: <
20210116214705.822267-17-vsementsov@virtuozzo.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
Vladimir Sementsov-Ogievskiy [Sat, 16 Jan 2021 21:46:57 +0000 (00:46 +0300)]
block/block-copy: make progress_bytes_callback optional
We are going to stop use of this callback in the following commit.
Still the callback handling code will be dropped in a separate commit.
So, for now let's make it optional.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-Id: <
20210116214705.822267-16-vsementsov@virtuozzo.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
Vladimir Sementsov-Ogievskiy [Sat, 16 Jan 2021 21:46:56 +0000 (00:46 +0300)]
iotests: 257: prepare for backup over block-copy
Iotest 257 dumps a lot of in-progress information of backup job, such
as offset and bitmap dirtiness. Further commit will move backup to be
one block-copy call, which will introduce async parallel requests
instead of plain cluster-by-cluster copying. To keep things
deterministic, allow only one worker (only one copy request at a time)
for this test.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-Id: <
20210116214705.822267-15-vsementsov@virtuozzo.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
Vladimir Sementsov-Ogievskiy [Sat, 16 Jan 2021 21:46:55 +0000 (00:46 +0300)]
iotests: 219: prepare for backup over block-copy
The further change of moving backup to be a one block-copy call will
make copying chunk-size and cluster-size two separate things. So, even
with 64k cluster sized qcow2 image, default chunk would be 1M.
Test 219 depends on specified chunk-size. Update it for explicit
chunk-size for backup as for mirror.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-Id: <
20210116214705.822267-14-vsementsov@virtuozzo.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
Vladimir Sementsov-Ogievskiy [Sat, 16 Jan 2021 21:46:54 +0000 (00:46 +0300)]
iotests: 185: prepare for backup over block-copy
The further change of moving backup to be a one block-copy call will
make copying chunk-size and cluster-size two separate things. So, even
with 64k cluster sized qcow2 image, default chunk would be 1M.
185 test however assumes, that with speed limited to 64K, one iteration
would result in offset=64K. It will change, as first iteration would
result in offset=1M independently of speed.
So, let's explicitly specify, what test wants: set max-chunk to 64K, so
that one iteration is 64K. Note, that we don't need to limit
max-workers, as block-copy rate limiter will handle the situation and
wouldn't start new workers when speed limit is obviously reached.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-Id: <
20210116214705.822267-13-vsementsov@virtuozzo.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
Max Reitz [Wed, 20 Jan 2021 10:20:43 +0000 (11:20 +0100)]
iotests/129: Limit backup's max-chunk/max-workers
Right now, this does not change anything, because backup ignores
max-chunk and max-workers. However, as soon as backup is switched over
to block-copy for the background copying process, we will need it to
keep 129 passing.
Signed-off-by: Max Reitz <mreitz@redhat.com>
Message-Id: <
20210120102043.28346-1-mreitz@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Vladimir Sementsov-Ogievskiy [Sat, 16 Jan 2021 21:46:53 +0000 (00:46 +0300)]
iotests: 56: prepare for backup over block-copy
After introducing parallel async copy requests instead of plain
cluster-by-cluster copying loop, we'll have to wait for paused status,
as we need to wait for several parallel request. So, let's gently wait
instead of just asserting that job already paused.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-Id: <
20210116214705.822267-12-vsementsov@virtuozzo.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
Vladimir Sementsov-Ogievskiy [Sat, 16 Jan 2021 21:46:52 +0000 (00:46 +0300)]
qapi: backup: add max-chunk and max-workers to x-perf struct
Add new parameters to configure future backup features. The patch
doesn't introduce aio backup requests (so we actually have only one
worker) neither requests larger than one cluster. Still, formally we
satisfy these maximums anyway, so add the parameters now, to facilitate
further patch which will really change backup job behavior.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-Id: <
20210116214705.822267-11-vsementsov@virtuozzo.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
Vladimir Sementsov-Ogievskiy [Sat, 16 Jan 2021 21:46:51 +0000 (00:46 +0300)]
job: call job_enter from job_pause
If main job coroutine called job_yield (while some background process
is in progress), we should give it a chance to call job_pause_point().
It will be used in backup, when moved on async block-copy.
Note, that job_user_pause is not enough: we want to handle
child_job_drained_begin() as well, which call job_pause().
Still, if job is already in job_do_yield() in job_pause_point() we
should not enter it.
iotest 109 output is modified: on stop we do bdrv_drain_all() which now
triggers job pause immediately (and pause after ready is standby).
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <
20210116214705.822267-10-vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
Vladimir Sementsov-Ogievskiy [Sat, 16 Jan 2021 21:46:50 +0000 (00:46 +0300)]
blockjob: add set_speed to BlockJobDriver
We are going to use async block-copy call in backup, so we'll need to
passthrough setting backup speed to block-copy call.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-Id: <
20210116214705.822267-9-vsementsov@virtuozzo.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
Vladimir Sementsov-Ogievskiy [Sat, 16 Jan 2021 21:46:49 +0000 (00:46 +0300)]
block/block-copy: add block_copy_cancel
Add function to cancel running async block-copy call. It will be used
in backup.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-Id: <
20210116214705.822267-8-vsementsov@virtuozzo.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
Vladimir Sementsov-Ogievskiy [Sat, 16 Jan 2021 21:46:48 +0000 (00:46 +0300)]
block/block-copy: add ratelimit to block-copy
We are going to directly use one async block-copy operation for backup
job, so we need rate limiter.
We want to maintain current backup behavior: only background copying is
limited and copy-before-write operations only participate in limit
calculation. Therefore we need one rate limiter for block-copy state
and boolean flag for block-copy call state for actual limitation.
Note, that we can't just calculate each chunk in limiter after
successful copying: it will not save us from starting a lot of async
sub-requests which will exceed limit too much. Instead let's use the
following scheme on sub-request creation:
1. If at the moment limit is not exceeded, create the request and
account it immediately.
2. If at the moment limit is already exceeded, drop create sub-request
and handle limit instead (by sleep).
With this approach we'll never exceed the limit more than by one
sub-request (which pretty much matches current backup behavior).
Note also, that if there is in-flight block-copy async call,
block_copy_kick() should be used after set-speed to apply new setup
faster. For that block_copy_kick() published in this patch.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-Id: <
20210116214705.822267-7-vsementsov@virtuozzo.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
Vladimir Sementsov-Ogievskiy [Sat, 16 Jan 2021 21:46:47 +0000 (00:46 +0300)]
block/block-copy: add list of all call-states
It simplifies debugging.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-Id: <
20210116214705.822267-6-vsementsov@virtuozzo.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
Vladimir Sementsov-Ogievskiy [Sat, 16 Jan 2021 21:46:46 +0000 (00:46 +0300)]
block/block-copy: add max_chunk and max_workers parameters
They will be used for backup.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-Id: <
20210116214705.822267-5-vsementsov@virtuozzo.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
Vladimir Sementsov-Ogievskiy [Sat, 16 Jan 2021 21:46:45 +0000 (00:46 +0300)]
block/block-copy: implement block_copy_async
We'll need async block-copy invocation to use in backup directly.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-Id: <
20210116214705.822267-4-vsementsov@virtuozzo.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
Vladimir Sementsov-Ogievskiy [Sat, 16 Jan 2021 21:46:44 +0000 (00:46 +0300)]
block/block-copy: More explicit call_state
Refactor common path to use BlockCopyCallState pointer as parameter, to
prepare it for use in asynchronous block-copy (at least, we'll need to
run block-copy in a coroutine, passing the whole parameters as one
pointer).
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-Id: <
20210116214705.822267-3-vsementsov@virtuozzo.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
Vladimir Sementsov-Ogievskiy [Sat, 16 Jan 2021 21:46:43 +0000 (00:46 +0300)]
qapi: backup: add perf.use-copy-range parameter
Experiments show, that copy_range is not always making things faster.
So, to make experimentation simpler, let's add a parameter. Some more
perf parameters will be added soon, so here is a new struct.
For now, add new backup qmp parameter with x- prefix for the following
reasons:
- We are going to add more performance parameters, some will be
related to the whole block-copy process, some only to background
copying in backup (ignored for copy-before-write operations).
- On the other hand, we are going to use block-copy interface in other
block jobs, which will need performance options as well.. And it
should be the same structure or at least somehow related.
So, there are too much unclean things about how the interface and now
we need the new options mostly for testing. Let's keep them
experimental for a while.
In do_backup_common() new x-perf parameter handled in a way to
make further options addition simpler.
We add use-copy-range with default=true, and we'll change the default
in further patch, after moving backup to use block-copy.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-Id: <
20210116214705.822267-2-vsementsov@virtuozzo.com>
[mreitz: s/5\.2/6.0/]
Signed-off-by: Max Reitz <mreitz@redhat.com>
Max Reitz [Mon, 25 Jan 2021 12:03:05 +0000 (13:03 +0100)]
coroutine-sigaltstack: Add SIGUSR2 mutex
Disposition (action) for any given signal is global for the process.
When two threads run coroutine-sigaltstack's qemu_coroutine_new()
concurrently, they may interfere with each other: One of them may revert
the SIGUSR2 handler to SIG_DFL, between the other thread (a) setting up
coroutine_trampoline() as the handler and (b) raising SIGUSR2. That
SIGUSR2 will then terminate the QEMU process abnormally.
We have to ensure that only one thread at a time can modify the
process-global SIGUSR2 handler. To do so, wrap the whole section where
that is done in a mutex.
Alternatively, we could for example have the SIGUSR2 handler always be
coroutine_trampoline(), so there would be no need to invoke sigaction()
in qemu_coroutine_new(). Laszlo has posted a patch to do so here:
https://lists.nongnu.org/archive/html/qemu-devel/2021-01/msg05962.html
However, given that coroutine-sigaltstack is more of a fallback
implementation for platforms that do not support ucontext, that change
may be a bit too invasive to be comfortable with it. The mutex proposed
here may negatively impact performance, but the change is much simpler.
Signed-off-by: Max Reitz <mreitz@redhat.com>
Message-Id: <
20210125120305.19520-1-mreitz@redhat.com>
Reviewed-by: Laszlo Ersek <lersek@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Max Reitz [Mon, 18 Jan 2021 10:57:20 +0000 (11:57 +0100)]
iotests/300: Clean up pylint and mypy complaints
And consequentially drop it from 297's skip list.
Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Message-Id: <
20210118105720.14824-11-mreitz@redhat.com>
Max Reitz [Mon, 18 Jan 2021 10:57:19 +0000 (11:57 +0100)]
iotests/129: Clean up pylint and mypy complaints
And consequentially drop it from 297's skip list.
Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <
20210118105720.14824-10-mreitz@redhat.com>
Max Reitz [Mon, 18 Jan 2021 10:57:18 +0000 (11:57 +0100)]
iotests/129: Limit mirror job's buffer size
Issuing 'stop' on the VM drains all nodes. If the mirror job has many
large requests in flight, this may lead to significant I/O that looks a
bit like 'stop' would make the job try to complete (which is what 129
should verify not to happen).
We can limit the I/O in flight by limiting the buffer size, so mirror
will make very little progress during the 'stop' drain.
(We do not need to do anything about commit, which has a buffer size of
512 kB by default; or backup, which goes cluster by cluster. Once we
have asynchronous requests for backup, that will change, but then we can
fine-tune the backup job to only perform a single request on a very
small chunk, too.)
Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Message-Id: <
20210118105720.14824-9-mreitz@redhat.com>
Max Reitz [Mon, 18 Jan 2021 10:57:17 +0000 (11:57 +0100)]
iotests/129: Actually test a commit job
Before this patch, test_block_commit() performs an active commit, which
under the hood is a mirror job. If we want to test various different
block jobs, we should perhaps run an actual commit job instead.
Doing so requires adding an overlay above the source node before the
commit is done (and then specifying the source node as the top node for
the commit job).
Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Message-Id: <
20210118105720.14824-8-mreitz@redhat.com>
Max Reitz [Mon, 18 Jan 2021 10:57:16 +0000 (11:57 +0100)]
iotests/129: Use throttle node
Throttling on the BB has not affected block jobs in a while, so it is
possible that one of the jobs in 129 finishes before the VM is stopped.
We can fix that by running the job from a throttle node.
Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Message-Id: <
20210118105720.14824-7-mreitz@redhat.com>
Max Reitz [Mon, 18 Jan 2021 10:57:15 +0000 (11:57 +0100)]
iotests/129: Do not check @busy
@busy is false when the job is paused, which happens all the time
because that is how jobs yield (e.g. for mirror at least since commit
565ac01f8d3).
Back when 129 was added (2015), perhaps there was no better way of
checking whether the job was still actually running. Now we have the
@status field (as of
58b295ba52c, i.e. 2018), which can give us exactly
that information.
Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Message-Id: <
20210118105720.14824-6-mreitz@redhat.com>
Max Reitz [Mon, 18 Jan 2021 10:57:14 +0000 (11:57 +0100)]
iotests/129: Remove test images in tearDown()
Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Message-Id: <
20210118105720.14824-5-mreitz@redhat.com>
Max Reitz [Mon, 18 Jan 2021 10:57:13 +0000 (11:57 +0100)]
iotests: Move try_remove to iotests.py
Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Message-Id: <
20210118105720.14824-4-mreitz@redhat.com>
Max Reitz [Mon, 18 Jan 2021 10:57:12 +0000 (11:57 +0100)]
iotests/297: Rewrite in Python and extend reach
Instead of checking iotests.py only, check all Python files in the
qemu-iotests/ directory. Of course, most of them do not pass, so there
is an extensive skip list for now. (The only files that do pass are
209, 254, 283, and iotests.py.)
(Alternatively, we could have the opposite, i.e. an explicit list of
files that we do want to check, but I think it is better to check files
by default.)
Unless started in debug mode (./check -d), the output has no information
on which files are tested, so we will not have a problem e.g. with
backports, where some files may be missing when compared to upstream.
Besides the technical rewrite, some more things are changed:
- For the pylint invocation, PYTHONPATH is adjusted. This mirrors
setting MYPYPATH for mypy.
- Also, MYPYPATH is now derived from PYTHONPATH, so that we include
paths set by the environment. Maybe at some point we want to let the
check script add '../../python/' to PYTHONPATH so that iotests.py does
not need to do that.
- Passing --notes=FIXME,XXX to pylint suppresses warnings for TODO
comments. TODO is fine, we do not need 297 to complain about such
comments.
- The "Success" line from mypy's output is suppressed, because (A) it
does not add useful information, and (B) it would leak information
about the files having been tested to the reference output, which we
decidedly do not want.
Suggested-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
Message-Id: <
20210118105720.14824-3-mreitz@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Max Reitz [Mon, 18 Jan 2021 10:57:11 +0000 (11:57 +0100)]
iotests.py: Assume a couple of variables as given
There are a couple of environment variables that we fetch with
os.environ.get() without supplying a default. Clearly they are required
and expected to be set by the ./check script (as evidenced by
execute_setup_common(), which checks for test_dir and
qemu_default_machine to be set, and aborts if they are not).
Using .get() this way has the disadvantage of returning an Optional[str]
type, which mypy will complain about when tests just assume these values
to be str.
Use [] instead, which raises a KeyError for environment variables that
are not set. When this exception is raised, catch it and move the abort
code from execute_setup_common() there.
Drop the 'assert iotests.sock_dir is not None' from iotest 300, because
that sort of thing is precisely what this patch wants to prevent.
Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Message-Id: <
20210118105720.14824-2-mreitz@redhat.com>
Andrey Shinkevich [Wed, 16 Dec 2020 06:17:03 +0000 (09:17 +0300)]
block: apply COR-filter to block-stream jobs
This patch completes the series with the COR-filter applied to
block-stream operations.
Adding the filter makes it possible in future implement discarding
copied regions in backing files during the block-stream job, to reduce
the disk overuse (we need control on permissions).
Also, the filter now is smart enough to do copy-on-read with specified
base, so we have benefit on guest reads even when doing block-stream of
the part of the backing chain.
Several iotests are slightly modified due to filter insertion.
Signed-off-by: Andrey Shinkevich <andrey.shinkevich@virtuozzo.com>
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <
20201216061703.70908-14-vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
Vladimir Sementsov-Ogievskiy [Wed, 16 Dec 2020 06:17:02 +0000 (09:17 +0300)]
block/stream: add s->target_bs
Add a direct link to target bs for convenience and to simplify
following commit which will insert COR filter above target bs.
This is a part of original commit written by Andrey.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-Id: <
20201216061703.70908-13-vsementsov@virtuozzo.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
Vladimir Sementsov-Ogievskiy [Wed, 16 Dec 2020 06:17:01 +0000 (09:17 +0300)]
iotests: 30: prepare to COR filter insertion by stream job
test_stream_parallel run parallel stream jobs, intersecting so that top
of one is base of another. It's OK now, but it would be a problem if
insert the filter, as one job will want to use another job's filter as
above_base node.
Correct thing to do is move to new interface: "bottom" argument instead
of base. This guarantees that jobs don't intersect by their actions.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-Id: <
20201216061703.70908-12-vsementsov@virtuozzo.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
Vladimir Sementsov-Ogievskiy [Wed, 16 Dec 2020 06:17:00 +0000 (09:17 +0300)]
qapi: block-stream: add "bottom" argument
The code already don't freeze base node and we try to make it prepared
for the situation when base node is changed during the operation. In
other words, block-stream doesn't own base node.
Let's introduce a new interface which should replace the current one,
which will in better relations with the code. Specifying bottom node
instead of base, and requiring it to be non-filter gives us the
following benefits:
- drop difference between above_base and base_overlay, which will be
renamed to just bottom, when old interface dropped
- clean way to work with parallel streams/commits on the same backing
chain, which otherwise become a problem when we introduce a filter
for stream job
- cleaner interface. Nobody will surprised the fact that base node may
disappear during block-stream, when there is no word about "base" in
the interface.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <
20201216061703.70908-11-vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
Andrey Shinkevich [Wed, 16 Dec 2020 06:16:59 +0000 (09:16 +0300)]
stream: rework backing-file changing
Stream in stream_prepare calls bdrv_change_backing_file() to change
backing-file in the metadata of bs.
It may use either backing-file parameter given by user or just take
filename of base on job start.
Backing file format is determined by base on job finish.
There are some problems with this design, we solve only two by this
patch:
1. Consider scenario with backing-file unset. Current concept of stream
supports changing of the base during the job (we don't freeze link to
the base). So, we should not save base filename at job start,
- let's determine name of the base on job finish.
2. Using direct base to determine filename and format is not very good:
base node may be a filter, so its filename may be JSON, and format_name
is not good for storing into qcow2 metadata as backing file format.
- let's use unfiltered_base
Signed-off-by: Andrey Shinkevich <andrey.shinkevich@virtuozzo.com>
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
[vsementsov: change commit subject, change logic in stream_prepare]
Message-Id: <
20201216061703.70908-10-vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
Andrey Shinkevich [Wed, 16 Dec 2020 06:16:58 +0000 (09:16 +0300)]
copy-on-read: skip non-guest reads if no copy needed
If the flag BDRV_REQ_PREFETCH was set, skip idling read/write
operations in COR-driver. It can be taken into account for the
COR-algorithms optimization. That check is being made during the
block stream job by the moment.
Add the BDRV_REQ_PREFETCH flag to the supported_read_flags of the
COR-filter.
block: Modify the comment for the flag BDRV_REQ_PREFETCH as we are
going to use it alone and pass it to the COR-filter driver for further
processing.
Signed-off-by: Andrey Shinkevich <andrey.shinkevich@virtuozzo.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-Id: <
20201216061703.70908-9-vsementsov@virtuozzo.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
Andrey Shinkevich [Wed, 16 Dec 2020 06:16:57 +0000 (09:16 +0300)]
block: include supported_read_flags into BDS structure
Add the new member supported_read_flags to the BlockDriverState
structure. It will control the flags set for copy-on-read operations.
Make the block generic layer evaluate supported read flags before they
go to a block driver.
Suggested-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Signed-off-by: Andrey Shinkevich <andrey.shinkevich@virtuozzo.com>
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
[vsementsov: use assert instead of abort]
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-Id: <
20201216061703.70908-8-vsementsov@virtuozzo.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
Andrey Shinkevich [Wed, 16 Dec 2020 06:16:56 +0000 (09:16 +0300)]
iotests: add #310 to test bottom node in COR driver
The test case #310 is similar to #216 by Max Reitz. The difference is
that the test #310 involves a bottom node to the COR filter driver.
Signed-off-by: Andrey Shinkevich <andrey.shinkevich@virtuozzo.com>
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
[vsementsov: detach backing to test reads from top, limit to qcow2]
Message-Id: <
20201216061703.70908-7-vsementsov@virtuozzo.com>
[mreitz: Add "# group:" line]
Signed-off-by: Max Reitz <mreitz@redhat.com>
Andrey Shinkevich [Wed, 16 Dec 2020 06:16:55 +0000 (09:16 +0300)]
qapi: copy-on-read filter: add 'bottom' option
Add an option to limit copy-on-read operations to specified sub-chain
of backing-chain, to make copy-on-read filter useful for block-stream
job.
Suggested-by: Max Reitz <mreitz@redhat.com>
Suggested-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Signed-off-by: Andrey Shinkevich <andrey.shinkevich@virtuozzo.com>
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
[vsementsov: change subject, modified to freeze the chain,
do some fixes]
Message-Id: <
20201216061703.70908-6-vsementsov@virtuozzo.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
Andrey Shinkevich [Wed, 16 Dec 2020 06:16:54 +0000 (09:16 +0300)]
qapi: add filter-node-name to block-stream
Provide the possibility to pass the 'filter-node-name' parameter to the
block-stream job as it is done for the commit block job.
Signed-off-by: Andrey Shinkevich <andrey.shinkevich@virtuozzo.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
[vsementsov: comment indentation, s/Since: 5.2/Since: 6.0/]
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-Id: <
20201216061703.70908-5-vsementsov@virtuozzo.com>
[mreitz: s/commit/stream/]
Signed-off-by: Max Reitz <mreitz@redhat.com>
Andrey Shinkevich [Wed, 16 Dec 2020 06:16:53 +0000 (09:16 +0300)]
copy-on-read: add filter drop function
Provide API for the COR-filter removal. Also, drop the filter child
permissions for an inactive state when the filter node is being
removed.
To insert the filter, the block generic layer function
bdrv_insert_node() can be used.
The new function bdrv_cor_filter_drop() may be considered as an
intermediate solution before the QEMU permission update system has
overhauled. Then we are able to implement the API function
bdrv_remove_node() on the block generic layer.
Signed-off-by: Andrey Shinkevich <andrey.shinkevich@virtuozzo.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-Id: <
20201216061703.70908-4-vsementsov@virtuozzo.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>