OSDN Git Service
Paolo Bonzini [Thu, 18 May 2023 14:50:00 +0000 (16:50 +0200)]
meson: subprojects: replace submodules with wrap files
Compared to submodules, .wrap files have several advantages:
* option parsing and downloading is delegated to meson
* the commit is stored in a text file instead of a magic entry in the
git tree object
* we could stop shipping external dependencies that are only used as a
fallback, but not break compilation on platforms that lack them.
For example it may make sense to download dtc at build time, controlled
by --enable-download, even when building from a tarball. Right now,
this patch does the opposite: make-release treats dtc like libvfio-user
(which is not stable API and therefore hasn't found its way into any
distros) and keycodemap (which is a copylib, for better or worse).
dependency() can fall back to a wrap automatically. However, this
is only possible for libraries that come with a .pc file, and this
is not very common for libfdt even though the upstream project in
principle provides it; it also removes the control that we provide with
--enable-fdt={system,internal}. Therefore, the logic to pick system
vs. internal libfdt is left untouched.
--enable-fdt=git is removed; it was already a synonym for
--enable-fdt=internal.
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Paolo Bonzini [Tue, 30 May 2023 15:27:48 +0000 (17:27 +0200)]
build: log submodule update from git-submodule.sh
Print exactly which submodules have been updated, by reusing the logic of
"git-submodule.sh validate" after executing "git submodule update --init'.
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Paolo Bonzini [Tue, 30 May 2023 15:10:29 +0000 (17:10 +0200)]
git-submodule: allow partial update of .git-submodule-status
Allow a specific subdirectory to run git-submodule.sh with only a
subset of submodules, without removing the others from the
.git-submodule-status file.
This also allows scripts/git-submodule.sh to be more lenient:
validating an empty set of submodules is not a mistake.
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Paolo Bonzini [Thu, 18 May 2023 14:11:29 +0000 (16:11 +0200)]
configure: rename --enable-pypi to --enable-download, control subprojects too
The behavior of --{enable,disable}-pypi is similar to that of
-Dwrapmode={default,nodownload} respectively. In particular,
in both cases a feature needs to be explicitly enabled for the
dependency to be downloaded.
So, use a single option to control both cases. Now, --enable-slirp
will trigger cloning and building of libslirp if the .pc file
is not found on the machine.
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Paolo Bonzini [Thu, 25 May 2023 10:36:28 +0000 (12:36 +0200)]
configure: remove --with-git= option
The scenario for which --with-git= was introduced was to use a SOCKS proxy
such as tsocks. However, this was back in 2017 when QEMU's submodules
used the git:// protocol, and it is not as important when using the
"smart HTTP" backend; for example, neither "meson subprojects download"
nor scripts/checkpatch.pl obey the GIT environment variable.
So remove the knob, but test for the presence of git in the configure and
git-submodule.sh scripts, and suggest using --with-git-submodules=validate
+ a manual invocation of git-submodule.sh when git does not work. Hopefully
in the future the GIT environment variable will be supported by Meson.
Reviewed-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Paolo Bonzini [Tue, 6 Jun 2023 08:14:45 +0000 (10:14 +0200)]
mkvenv: always pass locally-installed packages to pip
Let pip decide whether a new version should be installed or the current
one is okay. This ensures that the virtual environment is updated
(either upgraded or downgraded) whenever a new version of a package is
requested.
The hardest part here is figuring out if a package is installed in
the venv (which also has to be done twice to account for the presence
of either setuptools in Python <3.8, or importlib in Python >=3.8).
Suggested-by: Peter Maydell <peter.maydell@linaro.org>
Cc: John Snow <jsnow@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Paolo Bonzini [Mon, 5 Jun 2023 15:30:38 +0000 (17:30 +0200)]
tests: Use separate virtual environment for avocado
This reverts commits
eea2d141179 ("Makefile: remove $(TESTS_PYTHON)",
2023-05-26) and
9c6692db550 ("tests: Use configure-provided pyvenv for
tests", 2023-05-18).
Right now, there is a conflict between wanting a ">=" constraint when
using a distro-provided package and wanting a "==" constraint when
installing Avocado from PyPI; this would provide the best of both worlds
in terms of resiliency for both distros that have required packages and
distros that don't.
The conflict is visible also for meson, where we would like to install
the latest 0.63.x version but also accept a distro 1.1.x version.
But it is worse for avocado, for two reasons:
1) we cannot use an "==" constraint to install avocado if the venv
includes a system avocado. The distro will package plugins that have
"==" constraints on the version that is included in the distro, and, using
"pip install avocado==88.1" on a venv that includes system packages will
result in this error:
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
avocado-framework-plugin-varianter-yaml-to-mux 98.0 requires avocado-framework==98.0, but you have avocado-framework 88.1 which is incompatible.
avocado-framework-plugin-result-html 98.0 requires avocado-framework==98.0, but you have avocado-framework 88.1 which is incompatible.
make[1]: Leaving directory '/home/berrange/src/virt/qemu/build'
2) we cannot use ">=" either if the venv does _not_ include a system
avocado, because that would result in the installation of v101.0 which
is the one we've just reverted.
So the idea is to encode the dependencies as an (acceptable, locked)
tuple, like this hypothetical TOML that would be committed inside
python/ and used by mkvenv.py:
[meson]
meson = { minimum = "0.63.0", install = "0.63.3", canary = "meson" }
[docs]
# 6.0 drops support for Python 3.7
sphinx = { minimum = "1.6", install = "<6.0", canary = "sphinx-build" }
sphinx_rtd_theme = { minimum = "0.5" }
[avocado]
avocado-framework = { minimum = "88.1", install = "88.1", canary = "avocado" }
Once this is implemented, it would also be possible to install avocado in
pyvenv/ using "mkvenv.py ensure", thus using the distro package on Fedora
and CentOS Stream (the only distros where it's available). But until
this is implemented, keep avocado in a separate venv. There is still the
benefit of using a single python for meson custom_targets and for sphinx.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Paolo Bonzini [Sat, 3 Jun 2023 21:54:57 +0000 (23:54 +0200)]
Revert "tests/requirements.txt: bump up avocado-framework version to 101.0"
This reverts commit
ec5ffa0056389c3c10ea2de1e78366f66f4e5abc.
Bumping avocado to version 101 has two issues. First, there are problems
where Avocado is not logging of command lines or terminal output, and not
collecting Python logs outside the avocado namespace.
Second, the recent changes to Python handling mean that there is a single
virtual environment for all the build, instead of a separate one for testing.
Requiring a too-new version of avocado causes conflicts with any avocado
plugins installed on the host:
$ make check-venv
make[1]: Entering directory '/home/berrange/src/virt/qemu/build'
GIT ui/keycodemapdb tests/fp/berkeley-testfloat-3 tests/fp/berkeley-softfloat-3 dtc
VENVPIP install -e /home/berrange/src/virt/qemu/python/
VENVPIP install -r /home/berrange/src/virt/qemu/tests/requirements.txt
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
avocado-framework-plugin-varianter-yaml-to-mux 98.0 requires avocado-framework==98.0, but you have avocado-framework 101.0 which is incompatible.
avocado-framework-plugin-result-html 98.0 requires avocado-framework==98.0, but you have avocado-framework 101.0 which is incompatible.
make[1]: Leaving directory '/home/berrange/src/virt/qemu/build'
To avoid this issue, tests/requirements.txt should use a ">=" constraint
and the version of Avocado should be limited to what distros provide
in the system packages. Only Fedora has Avocado, and more specifically
version 92.0 (though 98.0 is also available as a module). As a first
step, this patch reverts the introduction of a too-new Avocado.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Philippe Mathieu-Daudé [Mon, 5 Jun 2023 17:41:45 +0000 (19:41 +0200)]
scsi/qemu-pr-helper: Drop support for 'old' libmultipath API
Commit
1b0578f5c4 ("qemu-pr-helper: Fix build on CentOS 7") added
code to probe for 'old' libmultipath API on CentOS 7. However since
merge commit
8c345b3e6a (June 2021) we don't build/test CentOS 7 as
it felt out of our list of supported distrib versions. Therefore we
can safely remove the 'old' API check (mostly reverting commit
1b0578f5c4, except the code got converted to meson in commit
6ec0e15d95 "meson: move libmpathpersist test").
Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-Id: <
20230605174146.87440-2-philmd@linaro.org>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Thomas Huth [Mon, 5 Jun 2023 11:45:23 +0000 (13:45 +0200)]
meson.build: Use -Wno-undef only for SDL2 versions that need it
There is no need to disable this useful compiler warning for
all versions of the SDL. Unfortunately, various versions are
buggy (beside SDL 2.0.8, the version 2.26.0 and 2.26.1 are
broken, too, see https://github.com/libsdl-org/SDL/issues/6619 ),
but we can use a simple compiler check to see whether we need
the -Wno-undef or not.
This also enables the printing of the version number with
good versions of the SDL in the summary of the meson output
again.
Signed-off-by: Thomas Huth <thuth@redhat.com>
Message-Id: <
20230605114523.282987-1-thuth@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Thomas Huth [Fri, 2 Jun 2023 17:18:32 +0000 (19:18 +0200)]
meson.build: Group the audio backend entries in a separate summary section
Let's make it easier for the users to spot audio-related entries
in the summary of the meson output.
Signed-off-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <
20230602171832.533739-4-thuth@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Thomas Huth [Fri, 2 Jun 2023 17:18:31 +0000 (19:18 +0200)]
meson.build: Group the network backend entries in a separate summary section
Let's make it easier for the users to spot network-related entries
in the summary of the meson output.
Signed-off-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <
20230602171832.533739-3-thuth@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Thomas Huth [Fri, 2 Jun 2023 17:18:30 +0000 (19:18 +0200)]
meson.build: Group the UI entries in a separate summary section
Let's make it easier for the users to spot UI-related entries in
the summary of the meson output.
Signed-off-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <
20230602171832.533739-2-thuth@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Paolo Bonzini [Sat, 3 Jun 2023 21:09:32 +0000 (23:09 +0200)]
scripts: remove dead file
scripts/test-driver.py was used when "make check" was already using meson
introspection data, but it did not execute "meson test". It is dead since
commit
3d2f73ef75e ("build: use "meson test" as the test harness", 2021-12-23).
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Paolo Bonzini [Fri, 3 Mar 2023 13:37:51 +0000 (14:37 +0100)]
atomics: eliminate mb_read/mb_set
qatomic_mb_read and qatomic_mb_set were the very first atomic primitives
introduced for QEMU; their semantics are unclear and they provide a false
sense of safety.
The last use of qatomic_mb_read() has been removed, so delete it.
qatomic_mb_set() instead can survive as an optimized
qatomic_set()+smp_mb(), similar to Linux's smp_store_mb(), but
rename it to qatomic_set_mb() to match the order of the two
operations.
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Michal Privoznik [Tue, 30 May 2023 10:31:23 +0000 (12:31 +0200)]
meson: Avoid implicit declaration of absent functions
While detecting a presence of a function via 'cc.links()'
gives desired result (i.e. detects whether function is present),
it also produces a warning on systems where the function is not
present (into meson-log.txt), e.g.:
qemu.git/build/meson-private/tmph74x3p38/testfile.c:2:34: \
warning: implicit declaration of function 'malloc_trim' [-Wimplicit-function-declaration]
And some distributions (e.g. Gentoo) parse the meson log and
consider these erroneous because it can lead to feature
misdetection (see [1]).
We can check whether given function exists via
'cc.has_function()' or whether STATX_* macros exist via
'cc.has_header_symbol()'.
1: https://wiki.gentoo.org/wiki/Modern_C_porting
Resolves: https://bugs.gentoo.org/898810
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Message-Id: <
8e02776d18595a1c575c90a189ff65f1785f76ca.
1685442612.git.mprivozn@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Richard Henderson [Mon, 5 Jun 2023 20:16:56 +0000 (13:16 -0700)]
Merge tag 'pull-tcg-
20230605' of https://gitlab.com/rth7680/qemu into staging
Build tcg/ once for system and once for user.
Unmap perf_marker.
Remove left over _link_error() definitions.
# -----BEGIN PGP SIGNATURE-----
#
# iQFRBAABCgA7FiEEekgeeIaLTbaoWgXAZN846K9+IV8FAmR+QekdHHJpY2hhcmQu
# aGVuZGVyc29uQGxpbmFyby5vcmcACgkQZN846K9+IV8NJwf+PE6ShfN+N65hwt0w
# mgIrWNadgnXezEyaUdEuzLre9Ln9uD9Are948xIGPTVoLyr2wEr+Ma9dFI+GpTHY
# 8C0b5v0SfHBBIK+wJb7VNcEdssRXVTcpikmmYv9IRpFTT8349bCdvJhuwojU5cyx
# NGgq9ydJ/dEUDtmVU6EnOyLCDSHg2mc+KPVWhjXaVbVPTyP6Xmb0BwSSUt1t5NQK
# Zw7E6G4z7QYXa7GNPTtXSWzTH3y9bSLQNj3jzHfJ6DPW23zxhFjCPhJYl1ecOHVF
# pxXEEIjieHQxdlm3kg+tcCoZSJ6OXESNzu2D8zk8Kf6xJjMItTQOOuokWM6mBX4y
# VbQoXg==
# =8vrl
# -----END PGP SIGNATURE-----
# gpg: Signature made Mon 05 Jun 2023 01:13:29 PM PDT
# gpg: using RSA key
7A481E78868B4DB6A85A05C064DF38E8AF7E215F
# gpg: issuer "richard.henderson@linaro.org"
# gpg: Good signature from "Richard Henderson <richard.henderson@linaro.org>" [ultimate]
* tag 'pull-tcg-
20230605' of https://gitlab.com/rth7680/qemu: (52 commits)
tcg/tcg-op-vec: Remove left over _link_error() definitions
accel/tcg: Unmap perf_marker
tcg: Build once for system and once for user-only
exec/poison: Do not poison CONFIG_SOFTMMU
plugins: Drop unused headers from exec/plugin-gen.h
plugins: Move plugin_insn_append to translator.c
tcg: Remove target-specific headers from tcg.[ch]
tcg: Move env defines out of NEED_CPU_H in helper-head.h
tcg: Fix PAGE/PROT confusion
accel/tcg: Tidy includes for translator.[ch]
target/arm: Add missing include of exec/exec-all.h
target/*: Add missing includes of exec/translation-block.h
target/mips: Tidy helpers for translation
target/arm: Tidy helpers for translation
accel/tcg: Move translator_fake_ldb out of line
target/ppc: Inline gen_icount_io_start()
accel/tcg: Introduce translator_io_start
accel/tcg: Move most of gen-icount.h into translator.c
include/exec: Remove CODE_GEN_AVG_BLOCK_SIZE
tcg: Spit out exec/translation-block.h
...
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Philippe Mathieu-Daudé [Mon, 5 Jun 2023 17:56:44 +0000 (19:56 +0200)]
tcg/tcg-op-vec: Remove left over _link_error() definitions
In commit
d56fea79f9 ("tcg: Move TCG_{LOW,HIGH} to tcg-internal.h")
we replaced the "_link_error" definitions with modern QEMU_ERROR()
attribute markup. We covered tcg-op.c but forgot to completely
clean tcg-op-vec.c. Do it now.
Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-Id: <
20230605175647.88395-3-philmd@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Ilya Leoshkevich [Mon, 5 Jun 2023 11:41:34 +0000 (13:41 +0200)]
accel/tcg: Unmap perf_marker
Coverity complains that perf_marker is never unmapped.
Fix by unmapping it in perf_exit().
Fixes: Coverity CID
1507929
Fixes:
5584e2dbe8c9 ("tcg: add perfmap and jitdump")
Signed-off-by: Ilya Leoshkevich <iii@linux.ibm.com>
Message-Id: <
20230605114134.
1169974-1-iii@linux.ibm.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Richard Henderson [Tue, 23 May 2023 06:22:36 +0000 (23:22 -0700)]
tcg: Build once for system and once for user-only
Create two static libraries for use by each execution mode.
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Richard Henderson [Sun, 30 Apr 2023 07:54:23 +0000 (08:54 +0100)]
exec/poison: Do not poison CONFIG_SOFTMMU
If CONFIG_USER_ONLY is ok generically, so is CONFIG_SOFTMMU,
because they are exactly opposite.
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Richard Henderson [Sun, 2 Apr 2023 15:31:20 +0000 (08:31 -0700)]
plugins: Drop unused headers from exec/plugin-gen.h
Two headers are not required for the rest of the
contents of plugin-gen.h.
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Richard Henderson [Sun, 2 Apr 2023 15:27:22 +0000 (08:27 -0700)]
plugins: Move plugin_insn_append to translator.c
This function is only used in translator.c, and uses a
target-specific typedef: abi_ptr.
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Richard Henderson [Sun, 2 Apr 2023 04:22:06 +0000 (21:22 -0700)]
tcg: Remove target-specific headers from tcg.[ch]
This finally paves the way for tcg/ to be built once per mode.
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Richard Henderson [Sun, 2 Apr 2023 15:10:46 +0000 (08:10 -0700)]
tcg: Move env defines out of NEED_CPU_H in helper-head.h
Since the change to CPUArchState, we have a common typedef
that can always be used.
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Richard Henderson [Sun, 2 Apr 2023 15:13:50 +0000 (08:13 -0700)]
tcg: Fix PAGE/PROT confusion
The bug was hidden because they happen to have the same values.
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Richard Henderson [Sun, 2 Apr 2023 04:16:39 +0000 (21:16 -0700)]
accel/tcg: Tidy includes for translator.[ch]
Reduce the header to only bswap.h and cpu_ldst.h.
Move exec/translate-all.h to translator.c.
Reduce tcg.h and tcg-op.h to tcg-op-common.h.
Remove otherwise unused headers.
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Richard Henderson [Sun, 2 Apr 2023 04:12:50 +0000 (21:12 -0700)]
target/arm: Add missing include of exec/exec-all.h
This had been pulled in via exec/translator.h,
but the include of exec-all.h will be removed.
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Richard Henderson [Sun, 2 Apr 2023 04:11:29 +0000 (21:11 -0700)]
target/*: Add missing includes of exec/translation-block.h
This had been pulled in via exec/exec-all.h, via exec/translator.h,
but the include of exec-all.h will be removed.
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Richard Henderson [Sun, 2 Apr 2023 03:44:44 +0000 (20:44 -0700)]
target/mips: Tidy helpers for translation
Move most includes from *translate*.c to translate.h, ensuring
that we get the ordering correct. Ensure cpu.h is first.
Use disas/disas.h instead of exec/log.h.
Drop otherwise unused includes.
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Richard Henderson [Sun, 2 Apr 2023 03:38:32 +0000 (20:38 -0700)]
target/arm: Tidy helpers for translation
Move most includes from *translate*.c to translate.h, ensuring
that we get the ordering correct. Ensure cpu.h is first.
Use disas/disas.h instead of exec/log.h.
Drop otherwise unused includes.
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Richard Henderson [Sun, 2 Apr 2023 03:13:56 +0000 (20:13 -0700)]
accel/tcg: Move translator_fake_ldb out of line
This is used by exactly one host in extraordinary circumstances.
This means that translator.h need not include plugin-gen.h;
translator.c already includes plugin-gen.h.
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Philippe Mathieu-Daudé [Fri, 2 Jun 2023 09:54:39 +0000 (11:54 +0200)]
target/ppc: Inline gen_icount_io_start()
Now that gen_icount_io_start() is a simple wrapper to
translator_io_start(), inline it.
Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-Id: <
20230602095439.48102-1-philmd@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Richard Henderson [Tue, 23 May 2023 06:08:01 +0000 (23:08 -0700)]
accel/tcg: Introduce translator_io_start
New wrapper around gen_io_start which takes care of the USE_ICOUNT
check, as well as marking the DisasContext to end the TB.
Remove exec/gen-icount.h.
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Richard Henderson [Sun, 2 Apr 2023 00:09:47 +0000 (17:09 -0700)]
accel/tcg: Move most of gen-icount.h into translator.c
The only usage of gen_tb_start and gen_tb_end are here.
Move the static icount_start_insn variable into a local
within translator_loop. Simplify the two subroutines
by passing in the existing local cflags variable.
Leave only the declaration of gen_io_start in gen-icount.h.
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Richard Henderson [Mon, 1 May 2023 07:08:27 +0000 (08:08 +0100)]
include/exec: Remove CODE_GEN_AVG_BLOCK_SIZE
The last use was removed with
2ac01d6dafab.
Fixes:
2ac01d6dafab ("translate-all: use a binary search tree to track TBs in TBContext")
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Richard Henderson [Sat, 1 Apr 2023 22:47:00 +0000 (15:47 -0700)]
tcg: Spit out exec/translation-block.h
This is all that is required by tcg/ from exec-all.h.
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Richard Henderson [Sat, 1 Apr 2023 22:28:18 +0000 (15:28 -0700)]
exec-all: Widen TranslationBlock pc and cs_base to 64-bits
This makes TranslationBlock agnostic to the address size of the guest.
Use vaddr for pc, since that's always a virtual address.
Use uint64_t for cs_base, since usage varies between guests.
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Richard Henderson [Sat, 1 Apr 2023 21:35:46 +0000 (14:35 -0700)]
exec-all: Widen tb_page_addr_t for user-only
This is a step toward making TranslationBlock agnostic
to the address size of the guest.
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Richard Henderson [Sat, 1 Apr 2023 21:24:08 +0000 (14:24 -0700)]
tcg: Remove NO_CPU_IO_DEFS
From this remove, it's no longer clear what this is attempting
to protect. The last time a use of this define was added to
the source tree, as opposed to merely moved around, was 2008.
There have been many cleanups since that time and this is
no longer required for the build to succeed.
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Richard Henderson [Sat, 1 Apr 2023 17:15:11 +0000 (10:15 -0700)]
tcg: Split tcg/tcg-op-gvec.h
Create tcg/tcg-op-gvec-common.h, moving everything that does not
concern TARGET_LONG_BITS. Adjust tcg-op-gvec.c to use the new header.
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Richard Henderson [Sat, 1 Apr 2023 16:26:35 +0000 (09:26 -0700)]
tcg: Move TLB_FLAGS_MASK check out of get_alignment_bits
The replacement isn't ideal, as the raw count of bits
is not easily synced with exec/cpu-all.h, but it does
remove from tcg.h the target dependency on TARGET_PAGE_BITS_MIN
which is built into TLB_FLAGS_MASK.
Reviewed-by: Anton Johansson <anjo@rev.ng>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Richard Henderson [Sat, 1 Apr 2023 05:56:55 +0000 (22:56 -0700)]
tcg: Add guest_mo to TCGContext
This replaces of TCG_GUEST_DEFAULT_MO in tcg-op-ldst.c.
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Richard Henderson [Sat, 1 Apr 2023 04:30:31 +0000 (21:30 -0700)]
tcg: Add insn_start_words to TCGContext
This will enable replacement of TARGET_INSN_START_WORDS in tcg.c.
Split out "tcg/insn-start-words.h" and use it in target/.
Reviewed-by: Anton Johansson <anjo@rev.ng>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Richard Henderson [Sat, 3 Jun 2023 16:48:07 +0000 (09:48 -0700)]
target/sh4: Emit insn_start for each insn in gUSA region
Fixes an assert in tcg_gen_code that we don't accidentally
eliminate an insn_start during optimization.
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Richard Henderson [Sat, 1 Apr 2023 03:13:36 +0000 (20:13 -0700)]
tcg: Split helper-proto.h
Create helper-proto-common.h without the target specific portion.
Use that in tcg-op-common.h. Include helper-proto.h in target/arm
and target/hexagon before helper-info.c.inc; all other targets are
already correct in this regard.
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Richard Henderson [Sat, 1 Apr 2023 02:07:00 +0000 (19:07 -0700)]
tcg: Split helper-gen.h
Create helper-gen-common.h without the target specific portion.
Use that in tcg-op-common.h. Reorg headers in target/arm to
ensure that helper-gen.h is included before helper-info.c.inc.
All other targets are already correct in this regard.
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Richard Henderson [Thu, 30 Mar 2023 05:14:36 +0000 (22:14 -0700)]
tcg: Split tcg_gen_callN
Make tcg_gen_callN a static function. Create tcg_gen_call[0-7]
functions for use by helper-gen.h.inc.
Removes a multiplicty of calls to __stack_chk_fail, saving up
to 143kiB of .text space as measured on an x86_64 host.
Old New Less %Change
8888680 8741816 146864 1.65% qemu-system-aarch64
5911832 5856152 55680 0.94% qemu-system-riscv64
5816728 5767512 49216 0.85% qemu-system-mips64
6707832 6659144 48688 0.73% qemu-system-ppc64
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Richard Henderson [Thu, 30 Mar 2023 15:09:03 +0000 (08:09 -0700)]
tcg: Move temp_idx and tcgv_i32_temp debug out of line
Removes a multiplicity of calls to __assert_fail, saving up
to 360kiB of .text space as measured on an x86_64 host.
Old New Less %Change
9257272 8888680 368592 3.98% qemu-system-aarch64
6100968 5911832 189136 3.10% qemu-system-riscv64
5839112 5707032 132080 2.26% qemu-system-mips
4447608 4341752 105856 2.38% qemu-system-s390x
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Richard Henderson [Fri, 31 Mar 2023 17:37:04 +0000 (10:37 -0700)]
tcg: Pass TCGHelperInfo to tcg_gen_callN
In preparation for compiling tcg/ only once, eliminate
the all_helpers array. Instantiate the info structs for
the generic helpers in accel/tcg/, and the structs for
the target-specific helpers in each translate.c.
Since we don't see all of the info structs at startup,
initialize at first use, using g_once_init_* to make
sure we don't race while doing so.
Reviewed-by: Anton Johansson <anjo@rev.ng>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Richard Henderson [Thu, 30 Mar 2023 01:09:28 +0000 (18:09 -0700)]
tcg: Move TCGHelperInfo and dependencies to tcg/helper-info.h
This will be required outside of tcg-internal.h soon.
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Richard Henderson [Wed, 29 Mar 2023 18:55:33 +0000 (11:55 -0700)]
tcg: Remove outdated comments in helper-head.h
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Richard Henderson [Wed, 29 Mar 2023 18:44:06 +0000 (11:44 -0700)]
target/hexagon: Include helper-gen.h where needed
This had been included via tcg-op-common.h via tcg-op.h,
but that is going away. In idef-parser.y, shuffle some
tcg related includes into a more logical order.
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Richard Henderson [Wed, 29 Mar 2023 18:41:03 +0000 (11:41 -0700)]
target/arm: Include helper-gen.h in translator.h
This had been included via tcg-op-common.h via tcg-op.h,
but that is going away.
It is needed for inlines within translator.h, so we might as well
do it there and not individually in each translator c file.
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Richard Henderson [Wed, 29 Mar 2023 01:17:24 +0000 (18:17 -0700)]
tcg: Split tcg/tcg-op-common.h from tcg/tcg-op.h
Create tcg/tcg-op-common.h, moving everything that does not concern
TARGET_LONG_BITS or TCGv. Adjust tcg/*.c to use the new header
instead of tcg-op.h, in preparation for compiling tcg/ only once.
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Richard Henderson [Tue, 28 Mar 2023 01:44:05 +0000 (18:44 -0700)]
tcg: Move TCGv, dup_const_tl definitions to tcg-op.h
These two items are the last uses of TARGET_LONG_BITS within tcg.h,
and are more in common with the other "_tl" definitions within that file.
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Richard Henderson [Tue, 28 Mar 2023 01:32:36 +0000 (18:32 -0700)]
tcg: Split out tcg/oversized-guest.h
Move a use of TARGET_LONG_BITS out of tcg/tcg.h.
Include the new file only where required.
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Richard Henderson [Tue, 28 Mar 2023 01:30:15 +0000 (18:30 -0700)]
target/arm: Fix test of TCG_OVERSIZED_GUEST
The symbol is always defined, even if to 0. We wanted to test for
TCG_OVERSIZED_GUEST == 0.
This fixed, the #error is reached while building arm-softmmu, because
TCG_OVERSIZED_GUEST is not true (nor supposed to be true) for arm32
guest on a 32-bit host. But that's ok, because this feature doesn't
apply to arm32. Add an #ifdef for TARGET_AARCH64.
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Richard Henderson [Mon, 1 May 2023 09:57:11 +0000 (10:57 +0100)]
tcg: Split out tcg-target-reg-bits.h
Often, the only thing we need to know about the TCG host
is the register size.
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Richard Henderson [Tue, 28 Mar 2023 01:24:50 +0000 (18:24 -0700)]
*: Add missing includes of tcg/tcg.h
This had been pulled in from exec/cpu_ldst.h, via exec/exec-all.h,
but the include of tcg.h will be removed.
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Richard Henderson [Tue, 28 Mar 2023 01:23:15 +0000 (18:23 -0700)]
target/*: Add missing includes of tcg/debug-assert.h
This had been pulled in from tcg/tcg.h, via exec/cpu_ldst.h,
via exec/exec-all.h, but the include of tcg.h will be removed.
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Richard Henderson [Tue, 28 Mar 2023 01:21:42 +0000 (18:21 -0700)]
target/avr: Add missing includes of qemu/error-report.h
This had been pulled in from tcg/tcg.h, via exec/cpu_ldst.h,
via exec/exec-all.h, but the include of tcg.h will be removed.
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Richard Henderson [Mon, 27 Mar 2023 23:07:15 +0000 (16:07 -0700)]
tcg: Add tlb_fast_offset to TCGContext
Disconnect the layout of ArchCPU from TCG compilation.
Pass the relative offset of 'env' and 'neg.tlb.f' as a parameter.
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Richard Henderson [Fri, 24 Mar 2023 20:02:59 +0000 (13:02 -0700)]
tcg: Widen CPUTLBEntry comparators to 64-bits
This makes CPUTLBEntry agnostic to the address size of the guest.
When 32-bit addresses are in effect, we can simply read the low
32 bits of the 64-bit field. Similarly when we need to update
the field for setting TLB_NOTDIRTY.
For TCG backends that could in theory be big-endian, but in
practice are not (arm, loongarch, riscv), use QEMU_BUILD_BUG_ON
to document and ensure this is not accidentally missed.
For s390x, which is always big-endian, use HOST_BIG_ENDIAN anyway,
to document the reason for the adjustment.
For sparc64 and ppc64, always perform a 64-bit load, and rely on
the following 32-bit comparison to ignore the high bits.
Rearrange mips and ppc if ladders for clarity.
Reviewed-by: Anton Johansson <anjo@rev.ng>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Richard Henderson [Fri, 28 Apr 2023 08:16:01 +0000 (09:16 +0100)]
tcg: Move TCG_TYPE_TL from tcg.h to tcg-op.h
Removes the only use of TARGET_LONG_BITS from tcg.h, which is to be
target independent. Move the symbol to a define in tcg-op.h, which
will continue to be target dependent. Rather than complicate matters
for the use in tb_gen_code(), expand the definition there.
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Richard Henderson [Thu, 27 Apr 2023 17:15:07 +0000 (18:15 +0100)]
tcg/sparc64: Remove TARGET_LONG_BITS, TCG_TYPE_TL
All uses replaced with TCGContext.addr_type.
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Richard Henderson [Thu, 27 Apr 2023 15:39:09 +0000 (16:39 +0100)]
tcg/s390x: Remove TARGET_LONG_BITS, TCG_TYPE_TL
All uses replaced with TCGContext.addr_type.
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Richard Henderson [Thu, 27 Apr 2023 14:59:31 +0000 (15:59 +0100)]
tcg/riscv: Remove TARGET_LONG_BITS, TCG_TYPE_TL
All uses replaced with TCGContext.addr_type.
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Richard Henderson [Thu, 27 Apr 2023 14:45:50 +0000 (15:45 +0100)]
tcg/ppc: Remove TARGET_LONG_BITS, TCG_TYPE_TL
All uses replaced with TCGContext.addr_type.
Reviewed-by: Anton Johansson <anjo@rev.ng>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Richard Henderson [Mon, 5 Jun 2023 17:27:31 +0000 (10:27 -0700)]
Merge tag 'pull-block-2023-06-05' of https://gitlab.com/hreitz/qemu into staging
Block patches
- Fix padding of unaligned vectored requests to match the host alignment
for vectors with 1023 or 1024 buffers
- Refactor and fix bugs in parallels's image check functionality
- Add an option to the qcow2 driver to retain (qcow2-level) allocations
on discard requests from the guest (while still forwarding the discard
to the lower level and marking the range as zero)
# -----BEGIN PGP SIGNATURE-----
#
# iQJGBAABCAAwFiEEy2LXoO44KeRfAE00ofpA0JgBnN8FAmR+AT4SHGhyZWl0ekBy
# ZWRoYXQuY29tAAoJEKH6QNCYAZzfnboQAKD6YrreZLoseomRfqOAoApSf6yOdcHk
# 6kfsvzwzjosomsF1Pkzm4851vX5PyDqTdeu0iViM+pxanVO1b494q1P4VcAERqMB
# iZVs68R6M0l6HV9btWFGm+ibHJf4FapdntkIdwog1ka5TIhw5oDWCVNLigjhIoRv
# sM37Bgf14kC3sFTR++0HESsyU1eUP5gJjwJbPZ2IgJBmzYay0is1z5nHA/3VUswu
# 8dKnGQDsv62EtlK7PK8cU2BhLOeNi6Wr3bAb6Wf2QLB5e0qRb7oAkqNx5/UcTznk
# a3XMC1aiWhYvM/+DaYIpQUcIPgA8xQ1KHKeD6WjbGfLgZBqseX0aGWMByUsiY8Bo
# +BPIBnUDrbiPnAKB/XLQfnzlE+s7121/JpEbB7AkZqVFRGuw8Wur4tbc2fzvy8Pw
# x/uQfv3ZPi/2Lf6u7hv/TVHubXi8jucVgx3Ubu5Jeo3901S4/KOQBQ4BQ/GYIGQX
# 38ijSROcEd0eQJ1mTKPEctouxjSZCghNSbrn9DfsL1V3VWqWNKKGCU3hM+RQ1SJT
# 688qvnyYt8QZfTsiDSHR/GfKsufG0DkoqE7c9IhSEPohecAH8Rrc3HcLut7fuwD2
# gCFQhm68CPwwRmBjPCY6Zi1RDzeOyFBSWN31T6t0yTb4OHJ/3/cSZVBJtwwkOVbx
# zwabHDNdY5Kw
# =GuoL
# -----END PGP SIGNATURE-----
# gpg: Signature made Mon 05 Jun 2023 08:37:34 AM PDT
# gpg: using RSA key
CB62D7A0EE3829E45F004D34A1FA40D098019CDF
# gpg: issuer "hreitz@redhat.com"
# gpg: Good signature from "Hanna Reitz <hreitz@redhat.com>" [unknown]
# gpg: WARNING: This key is not certified with a trusted signature!
# gpg: There is no indication that the signature belongs to the owner.
# Primary key fingerprint: CB62 D7A0 EE38 29E4 5F00 4D34 A1FA 40D0 9801 9CDF
* tag 'pull-block-2023-06-05' of https://gitlab.com/hreitz/qemu:
qcow2: add discard-no-unref option
parallels: Incorrect condition in out-of-image check
parallels: Replace qemu_co_mutex_lock by WITH_QEMU_LOCK_GUARD
parallels: Move statistic collection to a separate function
parallels: Move check of leaks to a separate function
parallels: Fix statistics calculation
parallels: Move check of cluster outside image to a separate function
parallels: Move check of unclean image to a separate function
parallels: Use generic infrastructure for BAT writing in parallels_co_check()
parallels: create parallels_set_bat_entry_helper() to assign BAT value
parallels: Fix image_end_offset and data_end after out-of-image check
parallels: Fix high_off calculation in parallels_co_check()
parallels: Out of image offset in BAT leads to image inflation
iotests/iov-padding: New test
util/iov: Remove qemu_iovec_init_extended()
block: Collapse padded I/O vecs exceeding IOV_MAX
util/iov: Make qiov_slice() public
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Richard Henderson [Mon, 5 Jun 2023 14:20:45 +0000 (07:20 -0700)]
Merge tag 'qemu-sparc-
20230605' of https://github.com/mcayland/qemu into staging
qemu-sparc queue
# -----BEGIN PGP SIGNATURE-----
#
# iQFSBAABCgA8FiEEzGIauY6CIA2RXMnEW8LFb64PMh8FAmR9xHseHG1hcmsuY2F2
# ZS1heWxhbmRAaWxhbmRlLmNvLnVrAAoJEFvCxW+uDzIfFsgH/0lV2y9CphGdWwtV
# S/00CRhp5n0kv3P2m4yDvXBhBgMa8zS6m/ZHhcfuU5xOxkJAnPudvXiZucHP4TSy
# 6KFBgVi2+xBW1YJf2UEHlBG2cDnVDHtYQze6Ak3S+W5sVeZZU8n1k8k8kRwpuNDS
# io23oKXzzKMYErlwy12bEHEF7vpfcG9HktjpJxUtppkNDuPTysyKSA6BavEm3mJC
# MlIypbxqxgBvu2SJETeP/m1yLQOaswViCNkqNqUH+MgEc1ubtSCXEuXTfSHIsxLR
# 224kRPFciY1sm3NvG9yz75UYSq1JUvUC5mNIIqwjb4MZJ+YHqFb3S/6ilCYChQ8V
# IBPyvH0=
# =pxbI
# -----END PGP SIGNATURE-----
# gpg: Signature made Mon 05 Jun 2023 04:18:19 AM PDT
# gpg: using RSA key
CC621AB98E82200D915CC9C45BC2C56FAE0F321F
# gpg: issuer "mark.cave-ayland@ilande.co.uk"
# gpg: Good signature from "Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>" [unknown]
# gpg: WARNING: This key is not certified with a trusted signature!
# gpg: There is no indication that the signature belongs to the owner.
# Primary key fingerprint: CC62 1AB9 8E82 200D 915C C9C4 5BC2 C56F AE0F 321F
* tag 'qemu-sparc-
20230605' of https://github.com/mcayland/qemu:
hw/isa/i82378: Remove unused "io" attribute
hw/arm/omap: Remove unused omap_uart_attach()
hw/timer/i8254_common: Share "iobase" property via base class
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Jean-Louis Dupond [Mon, 5 Jun 2023 08:45:24 +0000 (10:45 +0200)]
qcow2: add discard-no-unref option
When we for example have a sparse qcow2 image and discard: unmap is enabled,
there can be a lot of fragmentation in the image after some time. Especially on VM's
that do a lot of writes/deletes.
This causes the qcow2 image to grow even over 110% of its virtual size,
because the free gaps in the image get too small to allocate new
continuous clusters. So it allocates new space at the end of the image.
Disabling discard is not an option, as discard is needed to keep the
incremental backup size as low as possible. Without discard, the
incremental backups would become large, as qemu thinks it's just dirty
blocks but it doesn't know the blocks are unneeded.
So we need to avoid fragmentation but also 'empty' the unneeded blocks in
the image to have a small incremental backup.
In addition, we also want to send the discards further down the stack, so
the underlying blocks are still discarded.
Therefor we introduce a new qcow2 option "discard-no-unref".
When setting this option to true, discards will no longer have the qcow2
driver relinquish cluster allocations. Other than that, the request is
handled as normal: All clusters in range are marked as zero, and, if
pass-discard-request is true, it is passed further down the stack.
The only difference is that the now-zero clusters are preallocated
instead of being unallocated.
This will avoid fragmentation on the qcow2 image.
Fixes: https://gitlab.com/qemu-project/qemu/-/issues/1621
Signed-off-by: Jean-Louis Dupond <jean-louis@dupond.be>
Message-Id: <
20230605084523.34134-2-jean-louis@dupond.be>
Reviewed-by: Hanna Czenczek <hreitz@redhat.com>
Signed-off-by: Hanna Czenczek <hreitz@redhat.com>
Alexander Ivanov [Mon, 24 Apr 2023 09:31:47 +0000 (11:31 +0200)]
parallels: Incorrect condition in out-of-image check
All the offsets in the BAT must be lower than the file size.
Fix the check condition for correct check.
Signed-off-by: Alexander Ivanov <alexander.ivanov@virtuozzo.com>
Reviewed-by: Denis V. Lunev <den@openvz.org>
Message-Id: <
20230424093147.197643-13-alexander.ivanov@virtuozzo.com>
Reviewed-by: Hanna Czenczek <hreitz@redhat.com>
Signed-off-by: Hanna Czenczek <hreitz@redhat.com>
Alexander Ivanov [Mon, 24 Apr 2023 09:31:46 +0000 (11:31 +0200)]
parallels: Replace qemu_co_mutex_lock by WITH_QEMU_LOCK_GUARD
Replace the way we use mutex in parallels_co_check() for simplier
and less error prone code.
Signed-off-by: Alexander Ivanov <alexander.ivanov@virtuozzo.com>
Reviewed-by: Denis V. Lunev <den@openvz.org>
Message-Id: <
20230424093147.197643-12-alexander.ivanov@virtuozzo.com>
Reviewed-by: Hanna Czenczek <hreitz@redhat.com>
Signed-off-by: Hanna Czenczek <hreitz@redhat.com>
Alexander Ivanov [Mon, 24 Apr 2023 09:31:45 +0000 (11:31 +0200)]
parallels: Move statistic collection to a separate function
We will add more and more checks so we need a better code structure
in parallels_co_check. Let each check performs in a separate loop
in a separate helper.
Signed-off-by: Alexander Ivanov <alexander.ivanov@virtuozzo.com>
Reviewed-by: Denis V. Lunev <den@openvz.org>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
Message-Id: <
20230424093147.197643-11-alexander.ivanov@virtuozzo.com>
Reviewed-by: Hanna Czenczek <hreitz@redhat.com>
Signed-off-by: Hanna Czenczek <hreitz@redhat.com>
Alexander Ivanov [Mon, 24 Apr 2023 09:31:44 +0000 (11:31 +0200)]
parallels: Move check of leaks to a separate function
We will add more and more checks so we need a better code structure
in parallels_co_check. Let each check performs in a separate loop
in a separate helper.
Signed-off-by: Alexander Ivanov <alexander.ivanov@virtuozzo.com>
Message-Id: <
20230424093147.197643-10-alexander.ivanov@virtuozzo.com>
Reviewed-by: Hanna Czenczek <hreitz@redhat.com>
Signed-off-by: Hanna Czenczek <hreitz@redhat.com>
Alexander Ivanov [Mon, 24 Apr 2023 09:31:43 +0000 (11:31 +0200)]
parallels: Fix statistics calculation
Exclude out-of-image clusters from allocated and fragmented clusters
calculation.
Signed-off-by: Alexander Ivanov <alexander.ivanov@virtuozzo.com>
Message-Id: <
20230424093147.197643-9-alexander.ivanov@virtuozzo.com>
Reviewed-by: Hanna Czenczek <hreitz@redhat.com>
Signed-off-by: Hanna Czenczek <hreitz@redhat.com>
Alexander Ivanov [Mon, 24 Apr 2023 09:31:42 +0000 (11:31 +0200)]
parallels: Move check of cluster outside image to a separate function
We will add more and more checks so we need a better code structure in
parallels_co_check. Let each check performs in a separate loop in a
separate helper.
Signed-off-by: Alexander Ivanov <alexander.ivanov@virtuozzo.com>
Reviewed-by: Denis V. Lunev <den@openvz.org>
Message-Id: <
20230424093147.197643-8-alexander.ivanov@virtuozzo.com>
Reviewed-by: Hanna Czenczek <hreitz@redhat.com>
Signed-off-by: Hanna Czenczek <hreitz@redhat.com>
Alexander Ivanov [Mon, 24 Apr 2023 09:31:41 +0000 (11:31 +0200)]
parallels: Move check of unclean image to a separate function
We will add more and more checks so we need a better code structure
in parallels_co_check. Let each check performs in a separate loop
in a separate helper.
Signed-off-by: Alexander Ivanov <alexander.ivanov@virtuozzo.com>
Reviewed-by: Denis V. Lunev <den@openvz.org>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
Message-Id: <
20230424093147.197643-7-alexander.ivanov@virtuozzo.com>
Reviewed-by: Hanna Czenczek <hreitz@redhat.com>
Signed-off-by: Hanna Czenczek <hreitz@redhat.com>
Alexander Ivanov [Mon, 24 Apr 2023 09:31:40 +0000 (11:31 +0200)]
parallels: Use generic infrastructure for BAT writing in parallels_co_check()
BAT is written in the context of conventional operations over the image
inside bdrv_co_flush() when it calls parallels_co_flush_to_os() callback.
Thus we should not modify BAT array directly, but call
parallels_set_bat_entry() helper and bdrv_co_flush() further on. After
that there is no need to manually write BAT and track its modification.
This makes code more generic and allows to split parallels_set_bat_entry()
for independent pieces.
Signed-off-by: Alexander Ivanov <alexander.ivanov@virtuozzo.com>
Reviewed-by: Denis V. Lunev <den@openvz.org>
Message-Id: <
20230424093147.197643-6-alexander.ivanov@virtuozzo.com>
Reviewed-by: Hanna Czenczek <hreitz@redhat.com>
Signed-off-by: Hanna Czenczek <hreitz@redhat.com>
Alexander Ivanov [Mon, 24 Apr 2023 09:31:39 +0000 (11:31 +0200)]
parallels: create parallels_set_bat_entry_helper() to assign BAT value
This helper will be reused in next patches during parallels_co_check
rework to simplify its code.
Signed-off-by: Alexander Ivanov <alexander.ivanov@virtuozzo.com>
Reviewed-by: Denis V. Lunev <den@openvz.org>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
Message-Id: <
20230424093147.197643-5-alexander.ivanov@virtuozzo.com>
Reviewed-by: Hanna Czenczek <hreitz@redhat.com>
Signed-off-by: Hanna Czenczek <hreitz@redhat.com>
Alexander Ivanov [Mon, 24 Apr 2023 09:31:38 +0000 (11:31 +0200)]
parallels: Fix image_end_offset and data_end after out-of-image check
Set data_end to the end of the last cluster inside the image. In such a
way we can be sure that corrupted offsets in the BAT can't affect on the
image size. If there are no allocated clusters set image_end_offset by
data_end.
Signed-off-by: Alexander Ivanov <alexander.ivanov@virtuozzo.com>
Reviewed-by: Denis V. Lunev <den@openvz.org>
Message-Id: <
20230424093147.197643-4-alexander.ivanov@virtuozzo.com>
Reviewed-by: Hanna Czenczek <hreitz@redhat.com>
Signed-off-by: Hanna Czenczek <hreitz@redhat.com>
Alexander Ivanov [Mon, 24 Apr 2023 09:31:37 +0000 (11:31 +0200)]
parallels: Fix high_off calculation in parallels_co_check()
Don't let high_off be more than the file size even if we don't fix the
image.
Signed-off-by: Alexander Ivanov <alexander.ivanov@virtuozzo.com>
Reviewed-by: Denis V. Lunev <den@openvz.org>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
Message-Id: <
20230424093147.197643-3-alexander.ivanov@virtuozzo.com>
Reviewed-by: Hanna Czenczek <hreitz@redhat.com>
Signed-off-by: Hanna Czenczek <hreitz@redhat.com>
Alexander Ivanov [Mon, 24 Apr 2023 09:31:36 +0000 (11:31 +0200)]
parallels: Out of image offset in BAT leads to image inflation
data_end field in BDRVParallelsState is set to the biggest offset present
in BAT. If this offset is outside of the image, any further write will
create the cluster at this offset and/or the image will be truncated to
this offset on close. This is definitely not correct.
Raise an error in parallels_open() if data_end points outside the image
and it is not a check (let the check to repaire the image). Set data_end
to the end of the cluster with the last correct offset.
Signed-off-by: Alexander Ivanov <alexander.ivanov@virtuozzo.com>
Message-Id: <
20230424093147.197643-2-alexander.ivanov@virtuozzo.com>
Reviewed-by: Hanna Czenczek <hreitz@redhat.com>
Signed-off-by: Hanna Czenczek <hreitz@redhat.com>
Hanna Czenczek [Tue, 11 Apr 2023 17:34:18 +0000 (19:34 +0200)]
iotests/iov-padding: New test
Test that even vectored IO requests with 1024 vector elements that are
not aligned to the device's request alignment will succeed.
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
Signed-off-by: Hanna Czenczek <hreitz@redhat.com>
Message-Id: <
20230411173418.19549-5-hreitz@redhat.com>
Hanna Czenczek [Tue, 11 Apr 2023 17:34:17 +0000 (19:34 +0200)]
util/iov: Remove qemu_iovec_init_extended()
bdrv_pad_request() was the main user of qemu_iovec_init_extended().
HEAD^ has removed that use, so we can remove qemu_iovec_init_extended()
now.
The only remaining user is qemu_iovec_init_slice(), which can easily
inline the small part it really needs.
Note that qemu_iovec_init_extended() offered a memcpy() optimization to
initialize the new I/O vector. qemu_iovec_concat_iov(), which is used
to replace its functionality, does not, but calls qemu_iovec_add() for
every single element. If we decide this optimization was important, we
will need to re-implement it in qemu_iovec_concat_iov(), which might
also benefit its pre-existing users.
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
Signed-off-by: Hanna Czenczek <hreitz@redhat.com>
Message-Id: <
20230411173418.19549-4-hreitz@redhat.com>
Hanna Czenczek [Tue, 11 Apr 2023 17:34:16 +0000 (19:34 +0200)]
block: Collapse padded I/O vecs exceeding IOV_MAX
When processing vectored guest requests that are not aligned to the
storage request alignment, we pad them by adding head and/or tail
buffers for a read-modify-write cycle.
The guest can submit I/O vectors up to IOV_MAX (1024) in length, but
with this padding, the vector can exceed that limit. As of
4c002cef0e9abe7135d7916c51abce47f7fc1ee2 ("util/iov: make
qemu_iovec_init_extended() honest"), we refuse to pad vectors beyond the
limit, instead returning an error to the guest.
To the guest, this appears as a random I/O error. We should not return
an I/O error to the guest when it issued a perfectly valid request.
Before
4c002cef0e9abe7135d7916c51abce47f7fc1ee2, we just made the vector
longer than IOV_MAX, which generally seems to work (because the guest
assumes a smaller alignment than we really have, file-posix's
raw_co_prw() will generally see bdrv_qiov_is_aligned() return false, and
so emulate the request, so that the IOV_MAX does not matter). However,
that does not seem exactly great.
I see two ways to fix this problem:
1. We split such long requests into two requests.
2. We join some elements of the vector into new buffers to make it
shorter.
I am wary of (1), because it seems like it may have unintended side
effects.
(2) on the other hand seems relatively simple to implement, with
hopefully few side effects, so this patch does that.
To do this, the use of qemu_iovec_init_extended() in bdrv_pad_request()
is effectively replaced by the new function bdrv_create_padded_qiov(),
which not only wraps the request IOV with padding head/tail, but also
ensures that the resulting vector will not have more than IOV_MAX
elements. Putting that functionality into qemu_iovec_init_extended() is
infeasible because it requires allocating a bounce buffer; doing so
would require many more parameters (buffer alignment, how to initialize
the buffer, and out parameters like the buffer, its length, and the
original elements), which is not reasonable.
Conversely, it is not difficult to move qemu_iovec_init_extended()'s
functionality into bdrv_create_padded_qiov() by using public
qemu_iovec_* functions, so that is what this patch does.
Because bdrv_pad_request() was the only "serious" user of
qemu_iovec_init_extended(), the next patch will remove the latter
function, so the functionality is not implemented twice.
Buglink: https://bugzilla.redhat.com/show_bug.cgi?id=2141964
Signed-off-by: Hanna Czenczek <hreitz@redhat.com>
Message-Id: <
20230411173418.19549-3-hreitz@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
Hanna Czenczek [Tue, 11 Apr 2023 17:34:15 +0000 (19:34 +0200)]
util/iov: Make qiov_slice() public
We want to inline qemu_iovec_init_extended() in block/io.c for padding
requests, and having access to qiov_slice() is useful for this. As a
public function, it is renamed to qemu_iovec_slice().
(We will need to count the number of I/O vector elements of a slice
there, and then later process this slice. Without qiov_slice(), we
would need to call qemu_iovec_subvec_niov(), and all further
IOV-processing functions may need to skip prefixing elements to
accomodate for a qiov_offset. Because qemu_iovec_subvec_niov()
internally calls qiov_slice(), we can just have the block/io.c code call
qiov_slice() itself, thus get the number of elements, and also create an
iovec array with the superfluous prefixing elements stripped, so the
following processing functions no longer need to skip them.)
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
Signed-off-by: Hanna Czenczek <hreitz@redhat.com>
Message-Id: <
20230411173418.19549-2-hreitz@redhat.com>
Bernhard Beschow [Tue, 23 May 2023 19:56:08 +0000 (21:56 +0200)]
hw/isa/i82378: Remove unused "io" attribute
The attribute isn't used since commit
5c9736789b79ea49cd236ac326f0a414f63b1015
"i82378: Cleanup implementation".
Signed-off-by: Bernhard Beschow <shentey@gmail.com>
Acked-by: Michael S. Tsirkin <mst@redhat.com>
Reviewed-by: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
Message-Id: <
20230523195608.125820-4-shentey@gmail.com>
Signed-off-by: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
Bernhard Beschow [Tue, 23 May 2023 19:56:07 +0000 (21:56 +0200)]
hw/arm/omap: Remove unused omap_uart_attach()
The function is unused since commit
bdad3654d3c55f478e538037d9eccd204e5fc8ee ('hw/arm/nseries: Remove
invalid/unnecessary n8x0_uart_setup()').
Signed-off-by: Bernhard Beschow <shentey@gmail.com>
Acked-by: Michael S. Tsirkin <mst@redhat.com>
Reviewed-by: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-Id: <
20230523195608.125820-3-shentey@gmail.com>
Signed-off-by: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
Bernhard Beschow [Tue, 23 May 2023 19:56:06 +0000 (21:56 +0200)]
hw/timer/i8254_common: Share "iobase" property via base class
Both TYPE_KVM_I8254 and TYPE_I8254 have their own but same implementation of
the "iobase" property. The storage for the property already resides in
PITCommonState, so also move the property definition there.
Signed-off-by: Bernhard Beschow <shentey@gmail.com>
Acked-by: Michael S. Tsirkin <mst@redhat.com>
Reviewed-by: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-Id: <
20230523195608.125820-2-shentey@gmail.com>
Signed-off-by: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
Richard Henderson [Mon, 5 Jun 2023 03:48:46 +0000 (20:48 -0700)]
Merge tag 'pull-loongarch-
20230605' of https://gitlab.com/gaosong/qemu into staging
Fixes Coverity CID:
1512452,
1512453
Fixes:
78464f023b54 ("hw/loongarch/virt: Modify ipi as percpu device")
# -----BEGIN PGP SIGNATURE-----
#
# iLMEAAEIAB0WIQS4/x2g0v3LLaCcbCxAov/yOSY+3wUCZH1THwAKCRBAov/yOSY+
# 37u0BADoodYhyV+jhfBCwKvZbE7yI3vvqcC1hdw8/jclLcIfYVsgUJ+qETmk9s/X
# h/WGevhwADb+fBcBIYEBWeFFfFdBlEDxua/ew0eFlRcB3HoIHRnJJT9L+rZ4ifNW
# b8HDhv/LjDj9dpQ449wpGlSOiSST3+AiCpKkH36KUOEfLP029Q==
# =3GIb
# -----END PGP SIGNATURE-----
# gpg: Signature made Sun 04 Jun 2023 08:14:39 PM PDT
# gpg: using RSA key
B8FF1DA0D2FDCB2DA09C6C2C40A2FFF239263EDF
# gpg: Good signature from "Song Gao <m17746591750@163.com>" [unknown]
# gpg: WARNING: This key is not certified with a trusted signature!
# gpg: There is no indication that the signature belongs to the owner.
# Primary key fingerprint: B8FF 1DA0 D2FD CB2D A09C 6C2C 40A2 FFF2 3926 3EDF
* tag 'pull-loongarch-
20230605' of https://gitlab.com/gaosong/qemu:
hw/intc/loongarch_ipi: Bring back all 4 IPI mailboxes
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Jiaxun Yang [Sun, 21 May 2023 10:23:04 +0000 (11:23 +0100)]
hw/intc/loongarch_ipi: Bring back all 4 IPI mailboxes
As per "Loongson 3A5000/3B5000 Processor Reference Manual",
Loongson 3A5000's IPI implementation have 4 mailboxes per
core.
However, in
78464f023b54 ("hw/loongarch/virt: Modify ipi as
percpu device"), the number of IPI mailboxes was reduced to
one, which mismatches actual hardware.
It won't affect LoongArch based system as LoongArch boot code
only uses the first mailbox, however MIPS based Loongson boot
code uses all 4 mailboxes.
Fixes Coverity CID:
1512452,
1512453
Fixes:
78464f023b54 ("hw/loongarch/virt: Modify ipi as percpu device")
Signed-off-by: Jiaxun Yang <jiaxun.yang@flygoat.com>
Reviewed-by: Song Gao <gaosong@loongson.cn>
Message-Id: <
20230521102307.87081-2-jiaxun.yang@flygoat.com>
Signed-off-by: Song Gao <gaosong@loongson.cn>
Richard Henderson [Sat, 3 Jun 2023 00:33:29 +0000 (17:33 -0700)]
Merge tag 'migration-
20230602-pull-request' of https://gitlab.com/juan.quintela/qemu into staging
Migration Pull request (
20230602 vintage)
This PULL request get:
- All migration-test patches except last one (daniel)
- Documentation about live test cases (peter)
Please apply.
# -----BEGIN PGP SIGNATURE-----
#
# iQIzBAABCAAdFiEEGJn/jt6/WMzuA0uC9IfvGFhy1yMFAmR5yRwACgkQ9IfvGFhy
# 1yOLQQ/+NsrXEj7Bwp2PdGo+wBRkq4Gah/mc9F00pqtJc2CGNWgfgDohhZjBrhRv
# cTABsfEcIKgCYqGYwVCklJGlUMzxlJPPcMfvou5SWN59E4FBFSg4DWaBfDPCS8LW
# yjnz0JcpxJ+Ge0eqP6xpTPKQ0YGisdav/PjF8GZewBCjyrhZop062a92B2t59D8Y
# shJYKaZZU/5/4zx6KqOm9OClD/yJ+w5q6cGn89/rFE0RMSVywZ3Y1O8/LwIAEP6U
# oj88rczh3geGlsmtPIeyhA3BdnYuPonmyLz8CINFH9+y2tR9l1dN59q1uwEOIvff
# BhJvxTNmkTvsi5zeAmbp2CYmRTwhBmlanh8v2OLNj8zlt0cHYNpiYUZO9qxCHIyT
# LnNTTYhrpqAqINdm+Z8c3ymDKkTz0KECBa45hdFtNB4ZOXPDQTHVqkQRfe3CxDKz
# f/WM4TxHEzVMw/Ow1K9Kbk7/AEwIV6Ol2BSf9D+ZcU4ydmu6ENhV9G4cQ9Orlv8I
# opychxf+O/b6yhVFq7J1ufDhfn3aWQmUQC06npEgfrIV/fLrXhYfs2CXkNZs78v6
# MTMNPNBN/UasM8hx+ldsjZEHf625lO3eNWoNY1Xxog5YICnNLA+tG6n69uybew2+
# UOVyoHwX7iqaToK6bQNCS4H/PjCp3v7fzw1Nsz48Pfaklpivz/k=
# =4exy
# -----END PGP SIGNATURE-----
# gpg: Signature made Fri 02 Jun 2023 03:49:00 AM PDT
# gpg: using RSA key
1899FF8EDEBF58CCEE034B82F487EF185872D723
# gpg: Good signature from "Juan Quintela <quintela@redhat.com>" [unknown]
# gpg: aka "Juan Quintela <quintela@trasno.org>" [unknown]
# gpg: WARNING: This key is not certified with a trusted signature!
# gpg: There is no indication that the signature belongs to the owner.
# Primary key fingerprint: 1899 FF8E DEBF 58CC EE03 4B82 F487 EF18 5872 D723
* tag 'migration-
20230602-pull-request' of https://gitlab.com/juan.quintela/qemu:
qtest/migration: Document live=true cases
tests/qtest: make more migration pre-copy scenarios run non-live
tests/qtest: distinguish src/dst migration VM stop/resume events
tests/qtest: capture RESUME events during migration
tests/qtest: replace wait_command() with qtest_qmp_assert_success
tests/qtest: switch to using event callbacks for STOP event
tests/qtest: get rid of some 'qtest_qmp' usage in migration test
tests/qtest: get rid of 'qmp_command' helper in migration test
tests/qtest: add support for callback to receive QMP events
tests/qtest: add various qtest_qmp_assert_success() variants
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Richard Henderson [Fri, 2 Jun 2023 21:57:22 +0000 (14:57 -0700)]
Merge tag 'pull-nbd-2023-06-01-v2' of https://repo.or.cz/qemu/ericb into staging
nbd and misc patches for 2023-06-01
- Eric Blake: Fix iotest 104 for NBD
- Eric Blake: Improve qcow2 spec on padding bytes
- Eric Blake: Fix read-beyond-bounds bug in qemu_strtosz
# -----BEGIN PGP SIGNATURE-----
#
# iQEzBAABCAAdFiEEccLMIrHEYCkn0vOqp6FrSiUnQ2oFAmR6JzEACgkQp6FrSiUn
# Q2oGwgf+PIaN8iedQo5KR08OEf9YxJXab7nL5Oh12+ZvrPOt8XoJcd585KblQ1YI
# 3bGC4CO1l4QO3xmKltVHi7hnlX+3/8WMEvh0jBQBG1AjjPCi5Y1A/gGTEJFX60Ux
# /ffEpo8+1vaHQ8srkxBMWIvpF/dYRaMXSm/CP5SNqTllTalTR46YHKL9odXTzIeN
# 0Zu9UQw/Jwp5A9/8KB+0M9SYXA6zOEmEqEyOwVESEAU2Lm7titwqdBny6GZc6DH6
# Sa2lKO0qQA/e9ya6jHm2c9ycoNCtQ/2VR8QuCd6WCf9DX8q/9RhdiJir+EK5gqp6
# 3JRUFtx783d0BwPnUDUqPawi4txFtw==
# =Cg4e
# -----END PGP SIGNATURE-----
# gpg: Signature made Fri 02 Jun 2023 10:30:25 AM PDT
# gpg: using RSA key
71C2CC22B1C4602927D2F3AAA7A16B4A2527436A
# gpg: Good signature from "Eric Blake <eblake@redhat.com>" [full]
# gpg: aka "Eric Blake (Free Software Programmer) <ebb9@byu.net>" [full]
# gpg: aka "[jpeg image of size 6874]" [full]
* tag 'pull-nbd-2023-06-01-v2' of https://repo.or.cz/qemu/ericb: (21 commits)
cutils: Improve qemu_strtosz handling of fractions
cutils: Improve qemu_strtod* error paths
cutils: Use parse_uint in qemu_strtosz for negative rejection
cutils: Set value in all integral qemu_strto* error paths
cutils: Set value in all qemu_strtosz* error paths
test-cutils: Add more coverage to qemu_strtosz
numa: Check for qemu_strtosz_MiB error
cutils: Allow NULL str in qemu_strtosz
test-cutils: Refactor qemu_strtosz tests for less boilerplate
test-cutils: Prepare for upcoming semantic change in qemu_strtosz
test-cutils: Add coverage of qemu_strtod
cutils: Allow NULL endptr in parse_uint()
cutils: Adjust signature of parse_uint[_full]
cutils: Document differences between parse_uint and qemu_strtou64
cutils: Fix wraparound parsing in qemu_strtoui
test-cutils: Test more integer corner cases
test-cutils: Test integral qemu_strto* value on failures
test-cutils: Use g_assert_cmpuint where appropriate
test-cutils: Avoid g_assert in unit tests
qcow2: Explicit mention of padding bytes
...
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Eric Blake [Mon, 22 May 2023 19:04:41 +0000 (14:04 -0500)]
cutils: Improve qemu_strtosz handling of fractions
We have several limitations and bugs worth fixing; they are
inter-related enough that it is not worth splitting this patch into
smaller pieces:
* ".5k" should work to specify 512, just as "0.5k" does
* "1.9999k" and "1." + "9"*50 + "k" should both produce the same
result of 2048 after rounding
* "1." + "0"*350 + "1B" should not be treated the same as "1.0B";
underflow in the fraction should not be lost
* "7.99e99" and "7.99e999" look similar, but our code was doing a
read-out-of-bounds on the latter because it was not expecting ERANGE
due to overflow. While we document that scientific notation is not
supported, and the previous patch actually fixed
qemu_strtod_finite() to no longer return ERANGE overflows, it is
easier to pre-filter than to try and determine after the fact if
strtod() consumed more than we wanted. Note that this is a
low-level semantic change (when endptr is not NULL, we can now
successfully parse with a scale of 'E' and then report trailing
junk, instead of failing outright with EINVAL); but an earlier
commit already argued that this is not a high-level semantic change
since the only caller passing in a non-NULL endptr also checks that
the tail is whitespace-only.
Fixes: https://gitlab.com/qemu-project/qemu/-/issues/1629
Fixes:
cf923b78 ("utils: Improve qemu_strtosz() to have 64 bits of precision", 6.0.0)
Fixes:
7625a1ed ("utils: Use fixed-point arithmetic in qemu_strtosz", 6.0.0)
Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Hanna Czenczek <hreitz@redhat.com>
Message-Id: <
20230522190441.64278-20-eblake@redhat.com>
[eblake: tweak function comment for accuracy]
Eric Blake [Mon, 22 May 2023 19:04:40 +0000 (14:04 -0500)]
cutils: Improve qemu_strtod* error paths
Previous patches changed all integral qemu_strto*() error paths to
guarantee that *value is never left uninitialized. Do likewise for
qemu_strtod. Also, tighten qemu_strtod_finite() to never return a
non-finite value (prior to this patch, we were rejecting "inf" with
-EINVAL and unspecified result 0.0, but failing "9e999" with -ERANGE
and HUGE_VAL - which is infinite on IEEE machines - despite our
function claiming to recognize only finite values).
Auditing callers, we have no external callers of qemu_strtod, and
among the callers of qemu_strtod_finite:
- qapi/qobject-input-visitor.c:qobject_input_type_number_keyval() and
qapi/string-input-visitor.c:parse_type_number() which reject all
errors (does not matter what we store)
- utils/cutils.c:do_strtosz() incorrectly assumes that *endptr points
to '.' on all failures (that is, it is not distinguishing between
EINVAL and ERANGE; and therefore still does the WRONG THING for
"9.9e999". The change here does not entirely fix that (a later
patch will tackle this more systematically), but at least it fixes
the read-out-of-bounds first diagnosed in
https://gitlab.com/qemu-project/qemu/-/issues/1629
- our testsuite, which we can update to match what we document
Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Hanna Czenczek <hreitz@redhat.com>
CC: qemu-stable@nongnu.org
Message-Id: <
20230522190441.64278-19-eblake@redhat.com>
Eric Blake [Mon, 22 May 2023 19:04:39 +0000 (14:04 -0500)]
cutils: Use parse_uint in qemu_strtosz for negative rejection
Rather than open-coding two different ways to check for an unwanted
negative sign, reuse the same code in both functions. That way, if we
decide down the road to accept "-0" instead of rejecting it, we have
fewer places to change. Also, it means we now get ERANGE instead of
EINVAL for negative values in qemu_strtosz, which is reasonable for
what it represents. This in turn changes the expected output of a
couple of iotests.
The change is not quite complete: negative fractional scaled values
can trip us up. This will be fixed in a later patch addressing other
issues with fractional scaled values.
Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Hanna Czenczek <hreitz@redhat.com>
Message-Id: <
20230522190441.64278-18-eblake@redhat.com>
Eric Blake [Mon, 22 May 2023 19:04:38 +0000 (14:04 -0500)]
cutils: Set value in all integral qemu_strto* error paths
Our goal in writing qemu_strtoi() and friends is to have an interface
harder to abuse than libc's strtol(). Leaving the return value
uninitialized on some but not all error paths does not lend itself
well to this goal; and our documentation wasn't helpful on what to
expect.
Note that the previous patch changed all qemu_strtosz() EINVAL error
paths to slam value to 0 rather than stay uninitialized, even when the
EINVAL eror occurs because of trailing junk. But for the remaining
integral qemu_strto*, it's easier to return the parsed value than to
force things back to zero, in part because of how check_strtox_error
works; in part because people expect that from libc strto* (while
there is no libc strtosz to compare to), and in part because doing so
creates less churn in the testsuite.
Here, the list of affected callers is much longer ('git grep
"qemu_strto[ui]" "*.c" "**/*.c" | grep -v tests/ |wc -l' outputs 107,
although a few of those are the implementation in in cutils.c), so
touching as little as possible is the wisest course of action.
Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Hanna Czenczek <hreitz@redhat.com>
Message-Id: <
20230522190441.64278-17-eblake@redhat.com>
Eric Blake [Mon, 22 May 2023 19:04:37 +0000 (14:04 -0500)]
cutils: Set value in all qemu_strtosz* error paths
Making callers determine whether or not *value was populated on error
is not nice for usability. Pre-patch, we have unit tests that check
that *result is left unchanged on most EINVAL errors and set to 0 on
many ERANGE errors. This is subtly different from libc strtoumax()
behavior which returns UINT64_MAX on ERANGE errors, as well as
different from our parse_uint() which slams to 0 on EINVAL on the
grounds that we want our functions to be harder to mis-use than
strtoumax().
Let's audit callers:
- hw/core/numa.c:parse_numa() fixed in the previous patch to check for
errors
- migration/migration-hmp-cmds.c:hmp_migrate_set_parameter(),
monitor/hmp.c:monitor_parse_arguments(),
qapi/opts-visitor.c:opts_type_size(),
qapi/qobject-input-visitor.c:qobject_input_type_size_keyval(),
qemu-img.c:cvtnum_full(), qemu-io-cmds.c:cvtnum(),
target/i386/cpu.c:x86_cpu_parse_featurestr(), and
util/qemu-option.c:parse_option_size() appear to reject all failures
(although some with distinct messages for ERANGE as opposed to
EINVAL), so it doesn't matter what is in the value parameter on
error.
- All remaining callers are in the testsuite, where we can tweak our
expectations to match our new desired behavior.
Advancing to the end of the string parsed on overflow (ERANGE), while
still returning 0, makes sense (UINT64_MAX as a size is unlikely to be
useful); likewise, our size parsing code is complex enough that it's
easier to always return 0 when endptr is NULL but trailing garbage was
found, rather than trying to return the value of the prefix actually
parsed (no current caller cared about the value of the prefix).
Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Hanna Czenczek <hreitz@redhat.com>
Message-Id: <
20230522190441.64278-16-eblake@redhat.com>