gitlab-ci: add manual job to run Coverity

Add a job that can be run, either manually or on a schedule, to upload a build to Coverity Scan. The job uses the run-coverity-scan script in multiple phases of check, download tools and upload, in order to avoid both wasting time (skip everything if you are above the upload quota) and avoid filling the log with the progress of downloading the tools. The job is intended to run on a scheduled pipeline run, and scheduled runs will not get any other job. It requires two variables to be in GitLab CI, COVERITY_TOKEN and COVERITY_EMAIL. Those are already set up in qemu-project's configuration as protected and masked variables. Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
run-coverity-scan: add --check-upload-only option
2024-03-05 20:07:30 +01:00 · 2024-03-05 12:32:09 +01:00 · 2024-03-05 12:32:09 +01:00 · 2024-03-05 12:32:09 +01:00 · 2024-03-05 12:32:09 +01:00 · 2024-03-05 12:32:09 +01:00
5887 changed files with 236424 additions and 341422 deletions
@@ -1,13 +0,0 @@
-#
-# Common b4 settings that can be used to send patches to QEMU upstream.
-# https://b4.docs.kernel.org/
-#
-
-[b4]
-    send-series-to = qemu-devel@nongnu.org
-    send-auto-to-cmd = echo
-    send-auto-cc-cmd = scripts/get_maintainer.pl --noroles --norolestats --nogit --nogit-fallback
-    am-perpatch-check-cmd = scripts/checkpatch.pl -q --terse --no-summary --mailback -
-    prep-perpatch-check-cmd = scripts/checkpatch.pl -q --terse --no-summary --mailback -
-    searchmask = https://lore.kernel.org/qemu-devel/?x=m&t=1&q=%s
-    linkmask = https://lore.kernel.org/qemu-devel/%s
@@ -47,16 +47,3 @@ emacs_mode = glsl
 [*.json]
 indent_style = space
 emacs_mode = python
-
-# by default follow QEMU's style
-[*.pl]
-indent_style = space
-indent_size = 4
-emacs_mode = perl
-
-# but user kernel "style" for imported scripts
-[scripts/{kernel-doc,get_maintainer.pl,checkpatch.pl}]
-indent_style = tab
-indent_size = 8
-emacs_mode = perl
-
@@ -2,8 +2,3 @@
 *.h.inc         diff=c
 *.m             diff=objc
 *.py            diff=python
-*.rs            diff=rust
-*.rs.inc        diff=rust
-Cargo.lock      diff=toml merge=binary
-
-*.patch         -text -whitespace
@@ -24,10 +24,6 @@ variables:
    # Each script line from will be in a collapsible section in the job output
    # and show the duration of each line.
    FF_SCRIPT_SECTIONS: 1
-    # The project has a fairly fat GIT repo so we try and avoid bringing in things
-    # we don't need. The --filter options avoid blobs and tree references we aren't going to use
-    # and we also avoid fetching tags.
-    GIT_FETCH_EXTRA_FLAGS: --filter=blob:none --filter=tree:0 --no-tags --prune --quiet

  interruptible: true

@@ -69,6 +65,10 @@ variables:
    - if: '$QEMU_CI != "1" && $QEMU_CI != "2" && $CI_PROJECT_NAMESPACE != $QEMU_CI_UPSTREAM'
      when: never

+    # Avocado jobs don't run in forks unless $QEMU_CI_AVOCADO_TESTING is set
+    - if: '$QEMU_JOB_AVOCADO && $QEMU_CI_AVOCADO_TESTING != "1" && $CI_PROJECT_NAMESPACE != $QEMU_CI_UPSTREAM'
+      when: never
+

    #############################################################
    # Stage 2: fine tune execution of jobs in specific scenarios
@@ -97,8 +97,8 @@ variables:
      when: manual
      allow_failure: true

-    # Functional jobs can be manually started in forks
-    - if: '$QEMU_JOB_FUNCTIONAL && $QEMU_CI_FUNCTIONAL != "1" && $CI_PROJECT_NAMESPACE != $QEMU_CI_UPSTREAM'
+    # Avocado jobs can be manually start in forks if $QEMU_CI_AVOCADO_TESTING is unset
+    - if: '$QEMU_JOB_AVOCADO && $CI_PROJECT_NAMESPACE != $QEMU_CI_UPSTREAM'
      when: manual
      allow_failure: true

@@ -124,7 +124,7 @@ variables:
      when: manual

    # Jobs can run if any jobs they depend on were successful
-    - if: '$CI_PROJECT_NAMESPACE == $QEMU_CI_UPSTREAM && $CI_COMMIT_BRANCH =~ /staging-[[:digit:]]+\.[[:digit:]]/'
+    - if: '$QEMU_JOB_SKIPPED && $CI_PROJECT_NAMESPACE == $QEMU_CI_UPSTREAM && $CI_COMMIT_BRANCH =~ /staging-[[:digit:]]+\.[[:digit:]]/'
      when: on_success
      variables:
        QEMU_CI_CONTAINER_TAG: $CI_COMMIT_REF_SLUG
@@ -8,23 +8,16 @@
    key: "$CI_JOB_NAME"
    when: always
  before_script:
-    - source scripts/ci/gitlab-ci-section
-    - section_start setup "Pre-script setup"
    - JOBS=$(expr $(nproc) + 1)
-    - cat /packages.txt
-    - section_end setup
  script:
    - export CCACHE_BASEDIR="$(pwd)"
    - export CCACHE_DIR="$CCACHE_BASEDIR/ccache"
    - export CCACHE_MAXSIZE="500M"
    - export PATH="$CCACHE_WRAPPERSDIR:$PATH"
-    - du -sh .git
    - mkdir build
    - cd build
    - ccache --zero-stats
-    - section_start configure "Running configure"
    - ../configure --enable-werror --disable-docs --enable-fdt=system
-          --disable-debug-info
          ${TARGETS:+--target-list="$TARGETS"}
          $CONFIGURE_ARGS ||
      { cat config.log meson-logs/meson-log.txt && exit 1; }
@@ -32,16 +25,11 @@
      then
        pyvenv/bin/meson configure . -Dbackend_max_links="$LD_JOBS" ;
      fi || exit 1;
-    - section_end configure
-    - section_start build "Building QEMU"
-    - $MAKE -j"$JOBS"
-    - section_end build
-    - section_start test "Running tests"
+    - make -j"$JOBS"
    - if test -n "$MAKE_CHECK_ARGS";
      then
-        $MAKE -j"$JOBS" $MAKE_CHECK_ARGS ;
+        make -j"$JOBS" $MAKE_CHECK_ARGS ;
      fi
-    - section_end test
    - ccache --show-stats

 # We jump some hoops in common_test_job_template to avoid
@@ -56,30 +44,22 @@
    exclude:
      - build/**/*.p
      - build/**/*.a.p
+      - build/**/*.fa.p
      - build/**/*.c.o
      - build/**/*.c.o.d
+      - build/**/*.fa

 .common_test_job_template:
  extends: .base_job_template
  stage: test
  image: $CI_REGISTRY_IMAGE/qemu/$IMAGE:$QEMU_CI_CONTAINER_TAG
  script:
-    - source scripts/ci/gitlab-ci-section
-    - section_start buildenv "Setting up to run tests"
    - scripts/git-submodule.sh update roms/SLOF
-    - build/pyvenv/bin/meson subprojects download $(cd build/subprojects && echo *)
+    - meson subprojects download $(cd build/subprojects && echo *)
    - cd build
    - find . -type f -exec touch {} +
    # Avoid recompiling by hiding ninja with NINJA=":"
-    # We also have to pre-cache the functional tests manually in this case
-    - if [ "x${QEMU_TEST_CACHE_DIR}" != "x" ]; then
-        $MAKE precache-functional ;
-      fi
-    - section_end buildenv
-    - section_start test "Running tests"
-    # doctests need all the compilation artifacts
-    - $MAKE NINJA=":" MTESTARGS="--no-suite doc" $MAKE_CHECK_ARGS
-    - section_end test
+    - make NINJA=":" $MAKE_CHECK_ARGS

 .native_test_job_template:
  extends: .common_test_job_template
@@ -92,12 +72,12 @@
    reports:
      junit: build/meson-logs/testlog.junit.xml

-.functional_test_job_template:
+.avocado_test_job_template:
  extends: .common_test_job_template
  cache:
    key: "${CI_JOB_NAME}-cache"
    paths:
-      - ${CI_PROJECT_DIR}/functional-cache
+      - ${CI_PROJECT_DIR}/avocado-cache
    policy: pull-push
  artifacts:
    name: "$CI_JOB_NAME-$CI_COMMIT_REF_SLUG"
@@ -106,41 +86,21 @@
    paths:
      - build/tests/results/latest/results.xml
      - build/tests/results/latest/test-results
-      - build/tests/functional/*/*/*.log
    reports:
      junit: build/tests/results/latest/results.xml
  before_script:
-    - export QEMU_TEST_ALLOW_UNTRUSTED_CODE=1
-    - export QEMU_TEST_CACHE_DIR=${CI_PROJECT_DIR}/functional-cache
+    - mkdir -p ~/.config/avocado
+    - echo "[datadir.paths]" > ~/.config/avocado/avocado.conf
+    - echo "cache_dirs = ['${CI_PROJECT_DIR}/avocado-cache']"
+           >> ~/.config/avocado/avocado.conf
+    - echo -e '[job.output.testlogs]\nstatuses = ["FAIL", "INTERRUPT"]'
+           >> ~/.config/avocado/avocado.conf
+    - if [ -d ${CI_PROJECT_DIR}/avocado-cache ]; then
+        du -chs ${CI_PROJECT_DIR}/avocado-cache ;
+      fi
+    - export AVOCADO_ALLOW_UNTRUSTED_CODE=1
  after_script:
    - cd build
-    - du -chs ${CI_PROJECT_DIR}/*-cache
+    - du -chs ${CI_PROJECT_DIR}/avocado-cache
  variables:
-    QEMU_JOB_FUNCTIONAL: 1
-
-.wasm_build_job_template:
-  extends: .base_job_template
-  stage: build
-  image: $CI_REGISTRY_IMAGE/qemu/$IMAGE:$QEMU_CI_CONTAINER_TAG
-  before_script:
-    - source scripts/ci/gitlab-ci-section
-    - section_start setup "Pre-script setup"
-    - JOBS=$(expr $(nproc) + 1)
-    - section_end setup
-  script:
-    - du -sh .git
-    - mkdir build
-    - cd build
-    - section_start configure "Running configure"
-    - emconfigure ../configure --disable-docs
-          ${TARGETS:+--target-list="$TARGETS"}
-          $CONFIGURE_ARGS ||
-      { cat config.log meson-logs/meson-log.txt && exit 1; }
-    - if test -n "$LD_JOBS";
-      then
-        pyvenv/bin/meson configure . -Dbackend_max_links="$LD_JOBS" ;
-      fi || exit 1;
-    - section_end configure
-    - section_start build "Building QEMU"
-    - emmake make -j"$JOBS"
-    - section_end build
+    QEMU_JOB_AVOCADO: 1
@@ -22,14 +22,15 @@ check-system-alpine:
    IMAGE: alpine
    MAKE_CHECK_ARGS: check-unit check-qtest

-functional-system-alpine:
-  extends: .functional_test_job_template
+avocado-system-alpine:
+  extends: .avocado_test_job_template
  needs:
    - job: build-system-alpine
      artifacts: true
  variables:
    IMAGE: alpine
-    MAKE_CHECK_ARGS: check-functional
+    MAKE_CHECK_ARGS: check-avocado
+    AVOCADO_TAGS: arch:avr arch:loongarch64 arch:mips64 arch:mipsel

 build-system-ubuntu:
  extends:
@@ -39,9 +40,9 @@ build-system-ubuntu:
    job: amd64-ubuntu2204-container
  variables:
    IMAGE: ubuntu2204
-    CONFIGURE_ARGS: --enable-docs --enable-rust
+    CONFIGURE_ARGS: --enable-docs
    TARGETS: alpha-softmmu microblazeel-softmmu mips64el-softmmu
-    MAKE_CHECK_ARGS: check-build check-doc
+    MAKE_CHECK_ARGS: check-build

 check-system-ubuntu:
  extends: .native_test_job_template
@@ -52,14 +53,15 @@ check-system-ubuntu:
    IMAGE: ubuntu2204
    MAKE_CHECK_ARGS: check

-functional-system-ubuntu:
-  extends: .functional_test_job_template
+avocado-system-ubuntu:
+  extends: .avocado_test_job_template
  needs:
    - job: build-system-ubuntu
      artifacts: true
  variables:
    IMAGE: ubuntu2204
-    MAKE_CHECK_ARGS: check-functional
+    MAKE_CHECK_ARGS: check-avocado
+    AVOCADO_TAGS: arch:alpha arch:microblazeel arch:mips64el

 build-system-debian:
  extends:
@@ -69,7 +71,7 @@ build-system-debian:
    job: amd64-debian-container
  variables:
    IMAGE: debian
-    CONFIGURE_ARGS: --with-coroutine=sigaltstack --enable-rust
+    CONFIGURE_ARGS: --with-coroutine=sigaltstack
    TARGETS: arm-softmmu i386-softmmu riscv64-softmmu sh4eb-softmmu
      sparc-softmmu xtensa-softmmu
    MAKE_CHECK_ARGS: check-build
@@ -83,14 +85,15 @@ check-system-debian:
    IMAGE: debian
    MAKE_CHECK_ARGS: check

-functional-system-debian:
-  extends: .functional_test_job_template
+avocado-system-debian:
+  extends: .avocado_test_job_template
  needs:
    - job: build-system-debian
      artifacts: true
  variables:
    IMAGE: debian
-    MAKE_CHECK_ARGS: check-functional
+    MAKE_CHECK_ARGS: check-avocado
+    AVOCADO_TAGS: arch:arm arch:i386 arch:riscv64 arch:sh4 arch:sparc arch:xtensa

 crash-test-debian:
  extends: .native_test_job_template
@@ -112,24 +115,10 @@ build-system-fedora:
    job: amd64-fedora-container
  variables:
    IMAGE: fedora
-    CONFIGURE_ARGS: --disable-gcrypt --enable-nettle --enable-docs --enable-crypto-afalg --enable-rust
+    CONFIGURE_ARGS: --disable-gcrypt --enable-nettle --enable-docs
    TARGETS: microblaze-softmmu mips-softmmu
      xtensa-softmmu m68k-softmmu riscv32-softmmu ppc-softmmu sparc64-softmmu
-    MAKE_CHECK_ARGS: check-build check-doc
-
-build-system-fedora-rust-nightly:
-  extends:
-    - .native_build_job_template
-    - .native_build_artifact_template
-  needs:
-    job: amd64-fedora-rust-nightly-container
-  variables:
-    IMAGE: fedora-rust-nightly
-    CONFIGURE_ARGS: --disable-docs --enable-rust --enable-strict-rust-lints
-    TARGETS: aarch64-softmmu
-    MAKE_CHECK_ARGS: check-build check-doc
-
-  allow_failure: true
+    MAKE_CHECK_ARGS: check-build

 check-system-fedora:
  extends: .native_test_job_template
@@ -140,14 +129,16 @@ check-system-fedora:
    IMAGE: fedora
    MAKE_CHECK_ARGS: check

-functional-system-fedora:
-  extends: .functional_test_job_template
+avocado-system-fedora:
+  extends: .avocado_test_job_template
  needs:
    - job: build-system-fedora
      artifacts: true
  variables:
    IMAGE: fedora
-    MAKE_CHECK_ARGS: check-functional
+    MAKE_CHECK_ARGS: check-avocado
+    AVOCADO_TAGS: arch:microblaze arch:mips arch:xtensa arch:m68k
+      arch:riscv32 arch:ppc arch:sparc64

 crash-test-fedora:
  extends: .native_test_job_template
@@ -167,13 +158,13 @@ build-system-centos:
    - .native_build_job_template
    - .native_build_artifact_template
  needs:
-    job: amd64-centos9-container
+    job: amd64-centos8-container
  variables:
-    IMAGE: centos9
+    IMAGE: centos8
    CONFIGURE_ARGS: --disable-nettle --enable-gcrypt --enable-vfio-user-server
      --enable-modules --enable-trace-backends=dtrace --enable-docs
    TARGETS: ppc64-softmmu or1k-softmmu s390x-softmmu
-      x86_64-softmmu rx-softmmu sh4-softmmu
+      x86_64-softmmu rx-softmmu sh4-softmmu nios2-softmmu
    MAKE_CHECK_ARGS: check-build

 # Previous QEMU release. Used for cross-version migration tests.
@@ -183,25 +174,20 @@ build-previous-qemu:
    when: on_success
    expire_in: 2 days
    paths:
-      - build-previous/qemu-bundle
-      - build-previous/qemu-system-aarch64
-      - build-previous/qemu-system-x86_64
-      - build-previous/tests/qtest/migration-test
-      - build-previous/scripts
+      - build-previous
+    exclude:
+      - build-previous/**/*.p
+      - build-previous/**/*.a.p
+      - build-previous/**/*.fa.p
+      - build-previous/**/*.c.o
+      - build-previous/**/*.c.o.d
+      - build-previous/**/*.fa
  needs:
    job: amd64-opensuse-leap-container
  variables:
    IMAGE: opensuse-leap
    TARGETS: x86_64-softmmu aarch64-softmmu
-    # Override the default flags as we need more to grab the old version
-    GIT_FETCH_EXTRA_FLAGS: --prune --quiet
  before_script:
-    - source scripts/ci/gitlab-ci-section
-    # Skip if this series contains the release bump commit. During the
-    # release process there might be a window of commits when the
-    # version tag is not yet present in the remote and git fetch would
-    # fail.
-    - if grep -q "\.0$" VERSION; then exit 0; fi
    - export QEMU_PREV_VERSION="$(sed 's/\([0-9.]*\)\.[0-9]*/v\1.0/' VERSION)"
    - git remote add upstream https://gitlab.com/qemu-project/qemu
    - git fetch upstream refs/tags/$QEMU_PREV_VERSION:refs/tags/$QEMU_PREV_VERSION
@@ -222,9 +208,6 @@ build-previous-qemu:
    IMAGE: opensuse-leap
    MAKE_CHECK_ARGS: check-build
  script:
-    # Skip for round release numbers, this job is only relevant for
-    # testing a development tree.
-    - if grep -q "\.0$" VERSION; then exit 0; fi
    # Use the migration-tests from the older QEMU tree. This avoids
    # testing an old QEMU against new features/tests that it is not
    # compatible with.
@@ -257,17 +240,19 @@ check-system-centos:
    - job: build-system-centos
      artifacts: true
  variables:
-    IMAGE: centos9
+    IMAGE: centos8
    MAKE_CHECK_ARGS: check

-functional-system-centos:
-  extends: .functional_test_job_template
+avocado-system-centos:
+  extends: .avocado_test_job_template
  needs:
    - job: build-system-centos
      artifacts: true
  variables:
-    IMAGE: centos9
-    MAKE_CHECK_ARGS: check-functional
+    IMAGE: centos8
+    MAKE_CHECK_ARGS: check-avocado
+    AVOCADO_TAGS: arch:ppc64 arch:or1k arch:s390x arch:x86_64 arch:rx
+      arch:sh4 arch:nios2

 build-system-opensuse:
  extends:
@@ -289,14 +274,15 @@ check-system-opensuse:
    IMAGE: opensuse-leap
    MAKE_CHECK_ARGS: check

-functional-system-opensuse:
-  extends: .functional_test_job_template
+avocado-system-opensuse:
+  extends: .avocado_test_job_template
  needs:
    - job: build-system-opensuse
      artifacts: true
  variables:
    IMAGE: opensuse-leap
-    MAKE_CHECK_ARGS: check-functional
+    MAKE_CHECK_ARGS: check-avocado
+    AVOCADO_TAGS: arch:s390x arch:x86_64 arch:aarch64

 #
 # Flaky tests. We don't run these by default and they are allow fail
@@ -316,17 +302,18 @@ build-system-flaky:
      ppc64-softmmu rx-softmmu s390x-softmmu sh4-softmmu x86_64-softmmu
    MAKE_CHECK_ARGS: check-build

-functional-system-flaky:
-  extends: .functional_test_job_template
+avocado-system-flaky:
+  extends: .avocado_test_job_template
  needs:
    - job: build-system-flaky
      artifacts: true
  allow_failure: true
  variables:
    IMAGE: debian
-    MAKE_CHECK_ARGS: check-functional
+    MAKE_CHECK_ARGS: check-avocado
    QEMU_JOB_OPTIONAL: 1
    QEMU_TEST_FLAKY_TESTS: 1
+    AVOCADO_TAGS: flaky

 # This jobs explicitly disable TCG (--disable-tcg), KVM is detected by
 # the configure script. The container doesn't contain Xen headers so
@@ -338,9 +325,9 @@ functional-system-flaky:
 build-tcg-disabled:
  extends: .native_build_job_template
  needs:
-    job: amd64-centos9-container
+    job: amd64-centos8-container
  variables:
-    IMAGE: centos9
+    IMAGE: centos8
  script:
    - mkdir build
    - cd build
@@ -353,13 +340,11 @@ build-tcg-disabled:
    - cd tests/qemu-iotests/
    - ./check -raw 001 002 003 004 005 008 009 010 011 012 021 025 032 033 048
            052 063 077 086 101 104 106 113 148 150 151 152 157 159 160 163
-            170 171 184 192 194 208 221 226 227 236 253 277 image-fleecing
+            170 171 183 184 192 194 208 221 226 227 236 253 277 image-fleecing
    - ./check -qcow2 028 051 056 057 058 065 068 082 085 091 095 096 102 122
            124 132 139 142 144 145 151 152 155 157 165 194 196 200 202
            208 209 216 218 227 234 246 247 248 250 254 255 257 258
            260 261 262 263 264 270 272 273 277 279 image-fleecing
-    - cd ../..
-    - make distclean

 build-user:
  extends: .native_build_job_template
@@ -443,8 +428,8 @@ clang-system:
    job: amd64-fedora-container
  variables:
    IMAGE: fedora
-    CONFIGURE_ARGS: --cc=clang --cxx=clang++ --enable-ubsan
-      --extra-cflags=-fno-sanitize-recover=undefined
+    CONFIGURE_ARGS: --cc=clang --cxx=clang++
+      --extra-cflags=-fsanitize=undefined --extra-cflags=-fno-sanitize-recover=undefined
    TARGETS: alpha-softmmu arm-softmmu m68k-softmmu mips64-softmmu s390x-softmmu
    MAKE_CHECK_ARGS: check-qtest check-tcg

@@ -455,9 +440,9 @@ clang-user:
  timeout: 70m
  variables:
    IMAGE: debian-all-test-cross
-    CONFIGURE_ARGS: --cc=clang --cxx=clang++ --disable-system --enable-ubsan
+    CONFIGURE_ARGS: --cc=clang --cxx=clang++ --disable-system
      --target-list-exclude=alpha-linux-user,microblazeel-linux-user,aarch64_be-linux-user,i386-linux-user,m68k-linux-user,mipsn32el-linux-user,xtensaeb-linux-user
-      --extra-cflags=-fno-sanitize-recover=undefined
+      --extra-cflags=-fsanitize=undefined --extra-cflags=-fno-sanitize-recover=undefined
    MAKE_CHECK_ARGS: check-unit check-tcg

 # Set LD_JOBS=1 because this requires LTO and ld consumes a large amount of memory.
@@ -467,8 +452,8 @@ clang-user:
 # Since slirp callbacks are used in QEMU Timers, we cannot use libslirp with
 # CFI builds, and thus have to disable it here.
 #
-# Split in three sets of build/check/functional to limit the execution time
-# of each job
+# Split in three sets of build/check/avocado to limit the execution time of each
+# job
 build-cfi-aarch64:
  extends:
    - .native_build_job_template
@@ -498,14 +483,14 @@ check-cfi-aarch64:
    IMAGE: fedora
    MAKE_CHECK_ARGS: check

-functional-cfi-aarch64:
-  extends: .functional_test_job_template
+avocado-cfi-aarch64:
+  extends: .avocado_test_job_template
  needs:
    - job: build-cfi-aarch64
      artifacts: true
  variables:
    IMAGE: fedora
-    MAKE_CHECK_ARGS: check-functional
+    MAKE_CHECK_ARGS: check-avocado

 build-cfi-ppc64-s390x:
  extends:
@@ -536,14 +521,14 @@ check-cfi-ppc64-s390x:
    IMAGE: fedora
    MAKE_CHECK_ARGS: check

-functional-cfi-ppc64-s390x:
-  extends: .functional_test_job_template
+avocado-cfi-ppc64-s390x:
+  extends: .avocado_test_job_template
  needs:
    - job: build-cfi-ppc64-s390x
      artifacts: true
  variables:
    IMAGE: fedora
-    MAKE_CHECK_ARGS: check-functional
+    MAKE_CHECK_ARGS: check-avocado

 build-cfi-x86_64:
  extends:
@@ -570,14 +555,14 @@ check-cfi-x86_64:
    IMAGE: fedora
    MAKE_CHECK_ARGS: check

-functional-cfi-x86_64:
-  extends: .functional_test_job_template
+avocado-cfi-x86_64:
+  extends: .avocado_test_job_template
  needs:
    - job: build-cfi-x86_64
      artifacts: true
  variables:
    IMAGE: fedora
-    MAKE_CHECK_ARGS: check-functional
+    MAKE_CHECK_ARGS: check-avocado

 tsan-build:
  extends: .native_build_job_template
@@ -588,9 +573,6 @@ tsan-build:
    CONFIGURE_ARGS: --enable-tsan --cc=clang --cxx=clang++
          --enable-trace-backends=ust --disable-slirp
    TARGETS: x86_64-softmmu ppc64-softmmu riscv64-softmmu x86_64-linux-user
-    # Remove when we switch to a distro with clang >= 18
-    # https://github.com/google/sanitizers/issues/1716
-    MAKE: setarch -R make

 # gcov is a GCC features
 gcov:
@@ -632,15 +614,12 @@ build-oss-fuzz:
    - CC="clang" CXX="clang++" CFLAGS="-fsanitize=address"
      ./scripts/oss-fuzz/build.sh
    - export ASAN_OPTIONS="fast_unwind_on_malloc=0"
-    - failures=0
    - for fuzzer in $(find ./build-oss-fuzz/DEST_DIR/ -executable -type f
                      | grep -v slirp); do
        grep "LLVMFuzzerTestOneInput" ${fuzzer} > /dev/null 2>&1 || continue ;
        echo Testing ${fuzzer} ... ;
-        "${fuzzer}" -runs=1 -seed=1 || { echo "FAILED:"" ${fuzzer} exit code is $?"; failures=$(($failures+1)); };
+        "${fuzzer}" -runs=1 -seed=1 || exit 1 ;
      done
-    - echo "Number of failures:"" $failures"
-    - test $failures = 0

 build-tci:
  extends: .native_build_job_template
@@ -652,7 +631,7 @@ build-tci:
    - TARGETS="aarch64 arm hppa m68k microblaze ppc64 s390x x86_64"
    - mkdir build
    - cd build
-    - ../configure --enable-tcg-interpreter --disable-kvm --disable-docs --disable-gtk --disable-vnc
+    - ../configure --enable-tcg-interpreter --disable-docs --disable-gtk --disable-vnc
        --target-list="$(for tg in $TARGETS; do echo -n ${tg}'-softmmu '; done)"
        || { cat config.log meson-logs/meson-log.txt && exit 1; }
    - make -j"$JOBS"
@@ -670,9 +649,9 @@ build-tci:
 build-without-defaults:
  extends: .native_build_job_template
  needs:
-    job: amd64-centos9-container
+    job: amd64-centos8-container
  variables:
-    IMAGE: centos9
+    IMAGE: centos8
    CONFIGURE_ARGS:
      --without-default-devices
      --without-default-features
@@ -680,7 +659,8 @@ build-without-defaults:
      --disable-pie
      --disable-qom-cast-debug
      --disable-strip
-      --target-list-exclude=aarch64-softmmu,microblaze-softmmu,mips64-softmmu,mipsel-softmmu,ppc64-softmmu,sh4el-softmmu,xtensa-softmmu,x86_64-softmmu
+    TARGETS: avr-softmmu s390x-softmmu sh4-softmmu
+      sparc64-softmmu hexagon-linux-user i386-linux-user s390x-linux-user
    MAKE_CHECK_ARGS: check

 build-libvhost-user:
@@ -786,12 +766,3 @@ coverity:
      when: never
    # Always manual on forks even if $QEMU_CI == "2"
    - when: manual
-
-build-wasm:
-  extends: .wasm_build_job_template
-  timeout: 2h
-  needs:
-    job: wasm-emsdk-cross-container
-  variables:
-    IMAGE: emsdk-wasm32-cross
-    CONFIGURE_ARGS: --static --disable-tools --enable-debug --enable-tcg-interpreter
@@ -19,9 +19,10 @@ cwd = os.getcwd()
 reponame = os.path.basename(cwd)
 repourl = "https://gitlab.com/%s/%s.git" % (namespace, reponame)

-print(f"adding upstream git repo @ {repourl}")
 subprocess.check_call(["git", "remote", "add", "check-dco", repourl])
-subprocess.check_call(["git", "fetch", "--refetch", "check-dco", "master"])
+subprocess.check_call(["git", "fetch", "check-dco", "master"],
+                      stdout=subprocess.DEVNULL,
+                      stderr=subprocess.DEVNULL)

 ancestor = subprocess.check_output(["git", "merge-base",
                                    "check-dco/master", "HEAD"],
@@ -78,10 +79,7 @@ of Origin 1.1 (DCO):

 To indicate acceptance of the DCO every commit must have a tag

-  Signed-off-by: YOUR NAME <EMAIL>
-
-where "YOUR NAME" is your commonly known identity in the context
-of the community.
+  Signed-off-by: REAL NAME <EMAIL>

 This can be achieved by passing the "-s" flag to the "git commit" command.

@@ -19,12 +19,13 @@ cwd = os.getcwd()
 reponame = os.path.basename(cwd)
 repourl = "https://gitlab.com/%s/%s.git" % (namespace, reponame)

-print(f"adding upstream git repo @ {repourl}")
 # GitLab CI environment does not give us any direct info about the
 # base for the user's branch. We thus need to figure out a common
 # ancestor between the user's branch and current git master.
 subprocess.check_call(["git", "remote", "add", "check-patch", repourl])
-subprocess.check_call(["git", "fetch", "--refetch", "check-patch", "master"])
+subprocess.check_call(["git", "fetch", "check-patch", "master"],
+                      stdout=subprocess.DEVNULL,
+                      stderr=subprocess.DEVNULL)

 ancestor = subprocess.check_output(["git", "merge-base",
                                    "check-patch/master", "HEAD"],
@@ -1,71 +0,0 @@
-#!/usr/bin/env python3
-#
-# check-units.py: check the number of compilation units and identify
-#                 those that are rebuilt multiple times
-#
-# Copyright (C) 2025 Linaro Ltd.
-#
-# SPDX-License-Identifier: GPL-2.0-or-later
-
-from os import access, R_OK, path
-from sys import exit
-import json
-import argparse
-from pathlib import Path
-from collections import Counter
-
-
-def extract_build_units(cc_path):
-    """
-    Extract the build units and their counds from compile_commands.json file.
-
-    Returns:
-        Hash table of ["unit"] = count
-    """
-
-    j = json.load(open(cc_path, 'r'))
-    files = [f['file'] for f in j]
-    build_units = Counter(files)
-
-    return build_units
-
-
-def analyse_units(build_units, top_n):
-    """
-    Analyse the build units and report stats and the top 10 rebuilds
-    """
-
-    print(f"Total source files: {len(build_units.keys())}")
-    print(f"Total build units: {sum(units.values())}")
-
-    # Create a sorted list by number of rebuilds
-    sorted_build_units = sorted(build_units.items(),
-                                key=lambda item: item[1],
-                                reverse=True)
-
-    print("Most rebuilt units:")
-    for unit, count in sorted_build_units[:top_n]:
-        print(f"  {unit} built {count} times")
-
-    print("Least rebuilt units:")
-    for unit, count in sorted_build_units[-10:]:
-        print(f"  {unit} built {count} times")
-
-
-if __name__ == "__main__":
-    parser = argparse.ArgumentParser(
-        description="analyse number of build units in compile_commands.json")
-    parser.add_argument("cc_path", type=Path, default=None,
-                        help="Path to compile_commands.json")
-    parser.add_argument("-n", type=int, default=20,
-                        help="Dump the top <n> entries")
-
-    args = parser.parse_args()
-
-    if path.isfile(args.cc_path) and access(args.cc_path, R_OK):
-        units = extract_build_units(args.cc_path)
-        analyse_units(units, args.n)
-        exit(0)
-    else:
-        print(f"{args.cc_path} doesn't exist or isn't readable")
-        exit(1)
@@ -13,48 +13,116 @@
 .cirrus_build_job:
  extends: .base_job_template
  stage: build
-  image: registry.gitlab.com/libvirt/libvirt-ci/cirrus-run:latest
+  image: registry.gitlab.com/libvirt/libvirt-ci/cirrus-run:master
  needs: []
-  allow_failure:
-    exit_codes: 3
  # 20 mins larger than "timeout_in" in cirrus/build.yml
  # as there's often a 5-10 minute delay before Cirrus CI
  # actually starts the task
  timeout: 80m
  script:
-    - set -o allexport
    - source .gitlab-ci.d/cirrus/$NAME.vars
-    - set +o allexport
-    - cirrus-vars <.gitlab-ci.d/cirrus/build.yml >.gitlab-ci.d/cirrus/$NAME.yml
+    - sed -e "s|[@]CI_REPOSITORY_URL@|$CI_REPOSITORY_URL|g"
+          -e "s|[@]CI_COMMIT_REF_NAME@|$CI_COMMIT_REF_NAME|g"
+          -e "s|[@]CI_COMMIT_SHA@|$CI_COMMIT_SHA|g"
+          -e "s|[@]CIRRUS_VM_INSTANCE_TYPE@|$CIRRUS_VM_INSTANCE_TYPE|g"
+          -e "s|[@]CIRRUS_VM_IMAGE_SELECTOR@|$CIRRUS_VM_IMAGE_SELECTOR|g"
+          -e "s|[@]CIRRUS_VM_IMAGE_NAME@|$CIRRUS_VM_IMAGE_NAME|g"
+          -e "s|[@]CIRRUS_VM_CPUS@|$CIRRUS_VM_CPUS|g"
+          -e "s|[@]CIRRUS_VM_RAM@|$CIRRUS_VM_RAM|g"
+          -e "s|[@]UPDATE_COMMAND@|$UPDATE_COMMAND|g"
+          -e "s|[@]INSTALL_COMMAND@|$INSTALL_COMMAND|g"
+          -e "s|[@]PATH@|$PATH_EXTRA${PATH_EXTRA:+:}\$PATH|g"
+          -e "s|[@]PKG_CONFIG_PATH@|$PKG_CONFIG_PATH|g"
+          -e "s|[@]PKGS@|$PKGS|g"
+          -e "s|[@]MAKE@|$MAKE|g"
+          -e "s|[@]PYTHON@|$PYTHON|g"
+          -e "s|[@]PIP3@|$PIP3|g"
+          -e "s|[@]PYPI_PKGS@|$PYPI_PKGS|g"
+          -e "s|[@]CONFIGURE_ARGS@|$CONFIGURE_ARGS|g"
+          -e "s|[@]TEST_TARGETS@|$TEST_TARGETS|g"
+      <.gitlab-ci.d/cirrus/build.yml >.gitlab-ci.d/cirrus/$NAME.yml
    - cat .gitlab-ci.d/cirrus/$NAME.yml
    - cirrus-run -v --show-build-log always .gitlab-ci.d/cirrus/$NAME.yml
  variables:
    QEMU_JOB_CIRRUS: 1

-x64-freebsd-14-build:
+x64-freebsd-13-build:
  extends: .cirrus_build_job
  variables:
-    NAME: freebsd-14
+    NAME: freebsd-13
    CIRRUS_VM_INSTANCE_TYPE: freebsd_instance
    CIRRUS_VM_IMAGE_SELECTOR: image_family
-    CIRRUS_VM_IMAGE_NAME: freebsd-14-2
+    CIRRUS_VM_IMAGE_NAME: freebsd-13-3
    CIRRUS_VM_CPUS: 8
    CIRRUS_VM_RAM: 8G
    UPDATE_COMMAND: pkg update; pkg upgrade -y
    INSTALL_COMMAND: pkg install -y
-    CONFIGURE_ARGS: --target-list-exclude=arm-softmmu,i386-softmmu,microblaze-softmmu,mips64el-softmmu,mipsel-softmmu,mips-softmmu,ppc-softmmu,sh4eb-softmmu,xtensa-softmmu
    TEST_TARGETS: check

-aarch64-macos-build:
+aarch64-macos-13-base-build:
+  extends: .cirrus_build_job
+  variables:
+    NAME: macos-13
+    CIRRUS_VM_INSTANCE_TYPE: macos_instance
+    CIRRUS_VM_IMAGE_SELECTOR: image
+    CIRRUS_VM_IMAGE_NAME: ghcr.io/cirruslabs/macos-ventura-base:latest
+    CIRRUS_VM_CPUS: 12
+    CIRRUS_VM_RAM: 24G
+    UPDATE_COMMAND: brew update
+    INSTALL_COMMAND: brew install
+    PATH_EXTRA: /opt/homebrew/ccache/libexec:/opt/homebrew/gettext/bin
+    PKG_CONFIG_PATH: /opt/homebrew/curl/lib/pkgconfig:/opt/homebrew/ncurses/lib/pkgconfig:/opt/homebrew/readline/lib/pkgconfig
+    TEST_TARGETS: check-unit check-block check-qapi-schema check-softfloat check-qtest-x86_64
+
+aarch64-macos-14-base-build:
  extends: .cirrus_build_job
  variables:
    NAME: macos-14
    CIRRUS_VM_INSTANCE_TYPE: macos_instance
    CIRRUS_VM_IMAGE_SELECTOR: image
-    CIRRUS_VM_IMAGE_NAME: ghcr.io/cirruslabs/macos-runner:sonoma
+    CIRRUS_VM_IMAGE_NAME: ghcr.io/cirruslabs/macos-sonoma-base:latest
+    CIRRUS_VM_CPUS: 12
+    CIRRUS_VM_RAM: 24G
    UPDATE_COMMAND: brew update
    INSTALL_COMMAND: brew install
    PATH_EXTRA: /opt/homebrew/ccache/libexec:/opt/homebrew/gettext/bin
    PKG_CONFIG_PATH: /opt/homebrew/curl/lib/pkgconfig:/opt/homebrew/ncurses/lib/pkgconfig:/opt/homebrew/readline/lib/pkgconfig
-    CONFIGURE_ARGS: --target-list-exclude=arm-softmmu,i386-softmmu,microblazeel-softmmu,mips64-softmmu,mipsel-softmmu,mips-softmmu,ppc-softmmu,sh4-softmmu,xtensaeb-softmmu
    TEST_TARGETS: check-unit check-block check-qapi-schema check-softfloat check-qtest-x86_64
+    QEMU_JOB_OPTIONAL: 1
+
+
+# The following jobs run VM-based tests via KVM on a Linux-based Cirrus-CI job
+.cirrus_kvm_job:
+  extends: .base_job_template
+  stage: build
+  image: registry.gitlab.com/libvirt/libvirt-ci/cirrus-run:master
+  needs: []
+  timeout: 80m
+  script:
+    - sed -e "s|[@]CI_REPOSITORY_URL@|$CI_REPOSITORY_URL|g"
+          -e "s|[@]CI_COMMIT_REF_NAME@|$CI_COMMIT_REF_NAME|g"
+          -e "s|[@]CI_COMMIT_SHA@|$CI_COMMIT_SHA|g"
+          -e "s|[@]NAME@|$NAME|g"
+          -e "s|[@]CONFIGURE_ARGS@|$CONFIGURE_ARGS|g"
+          -e "s|[@]TEST_TARGETS@|$TEST_TARGETS|g"
+      <.gitlab-ci.d/cirrus/kvm-build.yml >.gitlab-ci.d/cirrus/$NAME.yml
+    - cat .gitlab-ci.d/cirrus/$NAME.yml
+    - cirrus-run -v --show-build-log always .gitlab-ci.d/cirrus/$NAME.yml
+  variables:
+    QEMU_JOB_CIRRUS: 1
+    QEMU_JOB_OPTIONAL: 1
+
+
+x86-netbsd:
+  extends: .cirrus_kvm_job
+  variables:
+    NAME: netbsd
+    CONFIGURE_ARGS: --target-list=x86_64-softmmu,ppc64-softmmu,aarch64-softmmu
+    TEST_TARGETS: check
+
+x86-openbsd:
+  extends: .cirrus_kvm_job
+  variables:
+    NAME: openbsd
+    CONFIGURE_ARGS: --target-list=i386-softmmu,riscv64-softmmu,mips64-softmmu
+    TEST_TARGETS: check
@@ -8,7 +8,7 @@ env:
  CI_REPOSITORY_URL: "@CI_REPOSITORY_URL@"
  CI_COMMIT_REF_NAME: "@CI_COMMIT_REF_NAME@"
  CI_COMMIT_SHA: "@CI_COMMIT_SHA@"
-  PATH: "@PATH_EXTRA@:$PATH"
+  PATH: "@PATH@"
  PKG_CONFIG_PATH: "@PKG_CONFIG_PATH@"
  PYTHON: "@PYTHON@"
  MAKE: "@MAKE@"
@@ -26,7 +26,7 @@ build_task:
    - git clone --depth 100 "$CI_REPOSITORY_URL" .
    - git fetch origin "$CI_COMMIT_REF_NAME"
    - git reset --hard "$CI_COMMIT_SHA"
-  step_script:
+  build_script:
    - mkdir build
    - cd build
    - ../configure --enable-werror $CONFIGURE_ARGS
@@ -0,0 +1,16 @@
+# THIS FILE WAS AUTO-GENERATED
+#
+#  $ lcitool variables freebsd-13 qemu
+#
+# https://gitlab.com/libvirt/libvirt-ci
+
+CCACHE='/usr/local/bin/ccache'
+CPAN_PKGS=''
+CROSS_PKGS=''
+MAKE='/usr/local/bin/gmake'
+NINJA='/usr/local/bin/ninja'
+PACKAGING_COMMAND='pkg'
+PIP3='/usr/local/bin/pip-3.8'
+PKGS='alsa-lib bash bison bzip2 ca_root_nss capstone4 ccache cmocka ctags curl cyrus-sasl dbus diffutils dtc flex fusefs-libs3 gettext git glib gmake gnutls gsed gtk3 json-c libepoxy libffi libgcrypt libjpeg-turbo libnfs libslirp libspice-server libssh libtasn1 llvm lzo2 meson mtools ncurses nettle ninja opencv pixman pkgconf png py39-numpy py39-pillow py39-pip py39-sphinx py39-sphinx_rtd_theme py39-tomli py39-yaml python3 rpm2cpio sdl2 sdl2_image snappy sndio socat spice-protocol tesseract usbredir virglrenderer vte3 xorriso zstd'
+PYPI_PKGS=''
+PYTHON='/usr/local/bin/python3'
@@ -1,16 +0,0 @@
-# THIS FILE WAS AUTO-GENERATED
-#
-#  $ lcitool variables freebsd-14 qemu
-#
-# https://gitlab.com/libvirt/libvirt-ci
-
-CCACHE='/usr/local/bin/ccache'
-CPAN_PKGS=''
-CROSS_PKGS=''
-MAKE='/usr/local/bin/gmake'
-NINJA='/usr/local/bin/ninja'
-PACKAGING_COMMAND='pkg'
-PIP3='/usr/local/bin/pip'
-PKGS='alsa-lib bash bison bzip2 ca_root_nss capstone4 ccache4 cmocka ctags curl cyrus-sasl dbus diffutils dtc flex fusefs-libs3 gettext git glib gmake gnutls gsed gtk-vnc gtk3 json-c libepoxy libffi libgcrypt libjpeg-turbo libnfs libslirp libspice-server libssh libtasn1 llvm lzo2 meson mtools ncurses nettle ninja opencv pixman pkgconf png py311-numpy py311-pillow py311-pip py311-pyyaml py311-sphinx py311-sphinx_rtd_theme py311-tomli python3 rpm2cpio rust rust-bindgen-cli sdl2 sdl2_image snappy sndio socat spice-protocol tesseract usbredir virglrenderer vte3 vulkan-tools xorriso zstd'
-PYPI_PKGS=''
-PYTHON='/usr/local/bin/python3'
@@ -0,0 +1,31 @@
+container:
+  image: fedora:35
+  cpu: 4
+  memory: 8Gb
+  kvm: true
+
+env:
+  CIRRUS_CLONE_DEPTH: 1
+  CI_REPOSITORY_URL: "@CI_REPOSITORY_URL@"
+  CI_COMMIT_REF_NAME: "@CI_COMMIT_REF_NAME@"
+  CI_COMMIT_SHA: "@CI_COMMIT_SHA@"
+
+@NAME@_task:
+  @NAME@_vm_cache:
+    folder: $HOME/.cache/qemu-vm
+  install_script:
+    - dnf update -y
+    - dnf install -y git make openssh-clients qemu-img qemu-system-x86 wget meson
+  clone_script:
+    - git clone --depth 100 "$CI_REPOSITORY_URL" .
+    - git fetch origin "$CI_COMMIT_REF_NAME"
+    - git reset --hard "$CI_COMMIT_SHA"
+  build_script:
+    - if [ -f $HOME/.cache/qemu-vm/images/@NAME@.img ]; then
+        make vm-build-@NAME@ J=$(getconf _NPROCESSORS_ONLN)
+          EXTRA_CONFIGURE_OPTS="@CONFIGURE_ARGS@"
+          BUILD_TARGET="@TEST_TARGETS@" ;
+      else
+        make vm-build-@NAME@ J=$(getconf _NPROCESSORS_ONLN) BUILD_TARGET=help
+          EXTRA_CONFIGURE_OPTS="--disable-system --disable-user --disable-tools" ;
+      fi
@@ -0,0 +1,16 @@
+# THIS FILE WAS AUTO-GENERATED
+#
+#  $ lcitool variables macos-13 qemu
+#
+# https://gitlab.com/libvirt/libvirt-ci
+
+CCACHE='/opt/homebrew/bin/ccache'
+CPAN_PKGS=''
+CROSS_PKGS=''
+MAKE='/opt/homebrew/bin/gmake'
+NINJA='/opt/homebrew/bin/ninja'
+PACKAGING_COMMAND='brew'
+PIP3='/opt/homebrew/bin/pip3'
+PKGS='bash bc bison bzip2 capstone ccache cmocka ctags curl dbus diffutils dtc flex gcovr gettext git glib gnu-sed gnutls gtk+3 jemalloc jpeg-turbo json-c libepoxy libffi libgcrypt libiscsi libnfs libpng libslirp libssh libtasn1 libusb llvm lzo make meson mtools ncurses nettle ninja pixman pkg-config python3 rpm2cpio sdl2 sdl2_image snappy socat sparse spice-protocol swtpm tesseract usbredir vde vte3 xorriso zlib zstd'
+PYPI_PKGS='PyYAML numpy pillow sphinx sphinx-rtd-theme tomli'
+PYTHON='/opt/homebrew/bin/python3'
@@ -11,6 +11,6 @@ MAKE='/opt/homebrew/bin/gmake'
 NINJA='/opt/homebrew/bin/ninja'
 PACKAGING_COMMAND='brew'
 PIP3='/opt/homebrew/bin/pip3'
-PKGS='bash bc bindgen bison bzip2 capstone ccache cmocka ctags curl dbus diffutils dtc flex gcovr gettext git glib gnu-sed gnutls gtk+3 gtk-vnc jemalloc jpeg-turbo json-c libcbor libepoxy libffi libgcrypt libiscsi libnfs libpng libslirp libssh libtasn1 libusb llvm lzo make meson mtools ncurses nettle ninja pixman pkg-config python3 rpm2cpio rust sdl2 sdl2_image snappy socat sparse spice-protocol swtpm tesseract usbredir vde vte3 vulkan-tools xorriso zlib zstd'
+PKGS='bash bc bison bzip2 capstone ccache cmocka ctags curl dbus diffutils dtc flex gcovr gettext git glib gnu-sed gnutls gtk+3 jemalloc jpeg-turbo json-c libepoxy libffi libgcrypt libiscsi libnfs libpng libslirp libssh libtasn1 libusb llvm lzo make meson mtools ncurses nettle ninja pixman pkg-config python3 rpm2cpio sdl2 sdl2_image snappy socat sparse spice-protocol swtpm tesseract usbredir vde vte3 xorriso zlib zstd'
 PYPI_PKGS='PyYAML numpy pillow sphinx sphinx-rtd-theme tomli'
 PYTHON='/opt/homebrew/bin/python3'
@@ -1,10 +1,10 @@
 include:
  - local: '/.gitlab-ci.d/container-template.yml'

-amd64-centos9-container:
+amd64-centos8-container:
  extends: .container_job_template
  variables:
-    NAME: centos9
+    NAME: centos8

 amd64-fedora-container:
  extends: .container_job_template
@@ -22,6 +22,12 @@ arm64-debian-cross-container:
  variables:
    NAME: debian-arm64-cross

+armel-debian-cross-container:
+  extends: .container_job_template
+  stage: containers
+  variables:
+    NAME: debian-armel-cross
+
 armhf-debian-cross-container:
  extends: .container_job_template
  stage: containers
@@ -67,8 +73,11 @@ ppc64el-debian-cross-container:
 riscv64-debian-cross-container:
  extends: .container_job_template
  stage: containers
+  # as we are currently based on 'sid/unstable' we may break so...
+  allow_failure: true
  variables:
    NAME: debian-riscv64-cross
+    QEMU_JOB_OPTIONAL: 1

 s390x-debian-cross-container:
  extends: .container_job_template
@@ -87,12 +96,12 @@ xtensa-debian-cross-container:
  variables:
    NAME: debian-xtensa-cross

+cris-fedora-cross-container:
+  extends: .container_job_template
+  variables:
+    NAME: fedora-cris-cross
+
 win64-fedora-cross-container:
  extends: .container_job_template
  variables:
    NAME: fedora-win64-cross
-
-wasm-emsdk-cross-container:
-  extends: .container_job_template
-  variables:
-    NAME: emsdk-wasm32-cross
@@ -27,9 +27,3 @@ python-container:
  extends: .container_job_template
  variables:
    NAME: python
-
-amd64-fedora-rust-nightly-container:
-  extends: .container_job_template
-  variables:
-    NAME: fedora-rust-nightly
-  allow_failure: true
@@ -8,12 +8,6 @@
    key: "$CI_JOB_NAME"
    when: always
  timeout: 80m
-  before_script:
-    - source scripts/ci/gitlab-ci-section
-    - section_start setup "Pre-script setup"
-    - JOBS=$(expr $(nproc) + 1)
-    - cat /packages.txt
-    - section_end setup
  script:
    - export CCACHE_BASEDIR="$(pwd)"
    - export CCACHE_DIR="$CCACHE_BASEDIR/ccache"
@@ -22,30 +16,18 @@
    - mkdir build
    - cd build
    - ccache --zero-stats
-    - section_start configure "Running configure"
    - ../configure --enable-werror --disable-docs --enable-fdt=system
        --disable-user $QEMU_CONFIGURE_OPTS $EXTRA_CONFIGURE_OPTS
-        --target-list-exclude="arm-softmmu
+        --target-list-exclude="arm-softmmu cris-softmmu
          i386-softmmu microblaze-softmmu mips-softmmu mipsel-softmmu
          mips64-softmmu ppc-softmmu riscv32-softmmu sh4-softmmu
          sparc-softmmu xtensa-softmmu $CROSS_SKIP_TARGETS"
-    - section_end configure
-    - section_start build "Building QEMU"
-    - make -j"$JOBS" all check-build
-    - section_end build
-    - section_start test "Running tests"
-    - if test -n "$MAKE_CHECK_ARGS";
-      then
-        $MAKE -j"$JOBS" $MAKE_CHECK_ARGS ;
-      fi
-    - section_end test
-    - section_start installer "Building the installer"
+    - make -j$(expr $(nproc) + 1) all check-build $MAKE_CHECK_ARGS
    - if grep -q "EXESUF=.exe" config-host.mak;
      then make installer;
      version="$(git describe --match v[0-9]* 2>/dev/null || git rev-parse --short HEAD)";
      mv -v qemu-setup*.exe qemu-setup-${version}.exe;
      fi
-    - section_end installer
    - ccache --show-stats

 # Job to cross-build specific accelerators.
@@ -57,14 +39,11 @@
  extends: .base_job_template
  stage: build
  image: $CI_REGISTRY_IMAGE/qemu/$IMAGE:$QEMU_CI_CONTAINER_TAG
-  timeout: 60m
+  timeout: 30m
  cache:
    paths:
      - ccache/
    key: "$CI_JOB_NAME"
-  before_script:
-    - source scripts/ci/gitlab-ci-section
-    - JOBS=$(expr $(nproc) + 1)
  script:
    - export CCACHE_BASEDIR="$(pwd)"
    - export CCACHE_DIR="$CCACHE_BASEDIR/ccache"
@@ -72,19 +51,9 @@
    - export PATH="$CCACHE_WRAPPERSDIR:$PATH"
    - mkdir build
    - cd build
-    - section_start configure "Running configure"
    - ../configure --enable-werror --disable-docs $QEMU_CONFIGURE_OPTS
        --disable-tools --enable-${ACCEL:-kvm} $EXTRA_CONFIGURE_OPTS
-    - section_end configure
-    - section_start build "Building QEMU"
-    - make -j"$JOBS" all check-build
-    - section_end build
-    - section_start test "Running tests"
-    - if test -n "$MAKE_CHECK_ARGS";
-      then
-        $MAKE -j"$JOBS" $MAKE_CHECK_ARGS ;
-      fi
-    - section_end test
+    - make -j$(expr $(nproc) + 1) all check-build $MAKE_CHECK_ARGS

 .cross_user_build_job:
  extends: .base_job_template
@@ -94,31 +63,18 @@
    paths:
      - ccache/
    key: "$CI_JOB_NAME"
-  before_script:
-    - source scripts/ci/gitlab-ci-section
-    - JOBS=$(expr $(nproc) + 1)
  script:
    - export CCACHE_BASEDIR="$(pwd)"
    - export CCACHE_DIR="$CCACHE_BASEDIR/ccache"
    - export CCACHE_MAXSIZE="500M"
    - mkdir build
    - cd build
-    - section_start configure "Running configure"
    - ../configure --enable-werror --disable-docs $QEMU_CONFIGURE_OPTS
        --disable-system --target-list-exclude="aarch64_be-linux-user
-          alpha-linux-user m68k-linux-user microblazeel-linux-user
-          or1k-linux-user ppc-linux-user sparc-linux-user
+          alpha-linux-user cris-linux-user m68k-linux-user microblazeel-linux-user
+          nios2-linux-user or1k-linux-user ppc-linux-user sparc-linux-user
          xtensa-linux-user $CROSS_SKIP_TARGETS"
-    - section_end configure
-    - section_start build "Building QEMU"
-    - make -j"$JOBS" all check-build
-    - section_end build
-    - section_start test "Running tests"
-    - if test -n "$MAKE_CHECK_ARGS";
-      then
-        $MAKE -j"$JOBS" $MAKE_CHECK_ARGS ;
-      fi
-    - section_end test
+    - make -j$(expr $(nproc) + 1) all check-build $MAKE_CHECK_ARGS

 # We can still run some tests on some of our cross build jobs. They can add this
 # template to their extends to save the build logs and test results
@@ -1,6 +1,13 @@
 include:
  - local: '/.gitlab-ci.d/crossbuild-template.yml'

+cross-armel-user:
+  extends: .cross_user_build_job
+  needs:
+    job: armel-debian-cross-container
+  variables:
+    IMAGE: debian-armel-cross
+
 cross-armhf-user:
  extends: .cross_user_build_job
  needs:
@@ -30,17 +37,6 @@ cross-arm64-kvm-only:
    IMAGE: debian-arm64-cross
    EXTRA_CONFIGURE_OPTS: --disable-tcg --without-default-features

-cross-i686-system:
-  extends:
-    - .cross_system_build_job
-    - .cross_test_artifacts
-  needs:
-    job: i686-debian-cross-container
-  variables:
-    IMAGE: debian-i686-cross
-    EXTRA_CONFIGURE_OPTS: --disable-kvm
-    MAKE_CHECK_ARGS: check-qtest
-
 cross-i686-user:
  extends:
    - .cross_user_build_job
@@ -61,12 +57,8 @@ cross-i686-tci:
  variables:
    IMAGE: debian-i686-cross
    ACCEL: tcg-interpreter
-    EXTRA_CONFIGURE_OPTS: --target-list=i386-softmmu,i386-linux-user,arm-softmmu,arm-linux-user,ppc-softmmu,ppc-linux-user --disable-plugins --disable-kvm
-    # Force tests to run with reduced parallelism, to see whether this
-    # reduces the flakiness of this CI job. The CI
-    # environment by default shows us 8 CPUs and so we
-    # would otherwise be using a parallelism of 9.
-    MAKE_CHECK_ARGS: check check-tcg -j2
+    EXTRA_CONFIGURE_OPTS: --target-list=i386-softmmu,i386-linux-user,aarch64-softmmu,aarch64-linux-user,ppc-softmmu,ppc-linux-user --disable-plugins
+    MAKE_CHECK_ARGS: check check-tcg

 cross-mipsel-system:
  extends: .cross_system_build_job
@@ -118,8 +110,12 @@ cross-ppc64el-kvm-only:
    IMAGE: debian-ppc64el-cross
    EXTRA_CONFIGURE_OPTS: --disable-tcg --without-default-devices

+# The riscv64 cross-builds currently use a 'sid' container to get
+# compilers and libraries. Until something more stable is found we
+# allow_failure so as not to block CI.
 cross-riscv64-system:
  extends: .cross_system_build_job
+  allow_failure: true
  needs:
    job: riscv64-debian-cross-container
  variables:
@@ -127,6 +123,7 @@ cross-riscv64-system:

 cross-riscv64-user:
  extends: .cross_user_build_job
+  allow_failure: true
  needs:
    job: riscv64-debian-cross-container
  variables:
@@ -170,7 +167,7 @@ cross-win64-system:
    IMAGE: fedora-win64-cross
    EXTRA_CONFIGURE_OPTS: --enable-fdt=internal --disable-plugins
    CROSS_SKIP_TARGETS: alpha-softmmu avr-softmmu hppa-softmmu
-                        m68k-softmmu microblazeel-softmmu
+                        m68k-softmmu microblazeel-softmmu nios2-softmmu
                        or1k-softmmu rx-softmmu sh4eb-softmmu sparc64-softmmu
                        tricore-softmmu xtensaeb-softmmu
  artifacts:
@@ -10,14 +10,13 @@
 # gitlab-runner.  To avoid problems that gitlab-runner can cause while
 # reusing the GIT repository, let's enable the clone strategy, which
 # guarantees a fresh repository on each job run.
+variables:
+  GIT_STRATEGY: clone

 # All custom runners can extend this template to upload the testlog
 # data as an artifact and also feed the junit report
 .custom_runner_template:
  extends: .base_job_template
-  variables:
-    GIT_STRATEGY: clone
-    GIT_FETCH_EXTRA_FLAGS: --no-tags --prune --quiet
  artifacts:
    name: "$CI_JOB_NAME-$CI_COMMIT_REF_SLUG"
    expire_in: 7 days
@@ -29,6 +28,7 @@
      junit: build/meson-logs/testlog.junit.xml

 include:
-  - local: '/.gitlab-ci.d/custom-runners/ubuntu-22.04-s390x.yml'
+  - local: '/.gitlab-ci.d/custom-runners/ubuntu-20.04-s390x.yml'
  - local: '/.gitlab-ci.d/custom-runners/ubuntu-22.04-aarch64.yml'
  - local: '/.gitlab-ci.d/custom-runners/ubuntu-22.04-aarch32.yml'
+  - local: '/.gitlab-ci.d/custom-runners/centos-stream-8-x86_64.yml'
@@ -0,0 +1,24 @@
+# All centos-stream-8 jobs should run successfully in an environment
+# setup by the scripts/ci/setup/stream/8/build-environment.yml task
+# "Installation of extra packages to build QEMU"
+
+centos-stream-8-x86_64:
+ extends: .custom_runner_template
+ allow_failure: true
+ needs: []
+ stage: build
+ tags:
+ - centos_stream_8
+ - x86_64
+ rules:
+ - if: '$CI_PROJECT_NAMESPACE == "qemu-project" && $CI_COMMIT_BRANCH =~ /^staging/'
+ - if: "$CENTOS_STREAM_8_x86_64_RUNNER_AVAILABLE"
+ before_script:
+ - JOBS=$(expr $(nproc) + 1)
+ script:
+ - mkdir build
+ - cd build
+ - ../scripts/ci/org.centos/stream/8/x86_64/configure
+   || { cat config.log meson-logs/meson-log.txt; exit 1; }
+ - make -j"$JOBS"
+ - make NINJA=":" check check-avocado
@@ -1,53 +1,53 @@
-# All ubuntu-22.04 jobs should run successfully in an environment
-# setup by the scripts/ci/setup/ubuntu/build-environment.yml task
-# "Install basic packages to build QEMU on Ubuntu 22.04"
+# All ubuntu-20.04 jobs should run successfully in an environment
+# setup by the scripts/ci/setup/build-environment.yml task
+# "Install basic packages to build QEMU on Ubuntu 20.04/20.04"

-ubuntu-22.04-s390x-all-linux:
+ubuntu-20.04-s390x-all-linux-static:
 extends: .custom_runner_template
 needs: []
 stage: build
 tags:
- - ubuntu_22.04
+ - ubuntu_20.04
 - s390x
 rules:
 - if: '$CI_PROJECT_NAMESPACE == "qemu-project" && $CI_COMMIT_BRANCH =~ /^staging/'
 - if: "$S390X_RUNNER_AVAILABLE"
 script:
+ # --disable-libssh is needed because of https://bugs.launchpad.net/qemu/+bug/1838763
+ # --disable-glusterfs is needed because there's no static version of those libs in distro supplied packages
 - mkdir build
 - cd build
- - ../configure --enable-debug --disable-system --disable-tools --disable-docs
+ - ../configure --enable-debug --static --disable-system --disable-glusterfs --disable-libssh
   || { cat config.log meson-logs/meson-log.txt; exit 1; }
 - make --output-sync -j`nproc`
 - make --output-sync check-tcg
 - make --output-sync -j`nproc` check

-ubuntu-22.04-s390x-all-system:
+ubuntu-20.04-s390x-all:
 extends: .custom_runner_template
 needs: []
 stage: build
 tags:
- - ubuntu_22.04
+ - ubuntu_20.04
 - s390x
 timeout: 75m
 rules:
 - if: '$CI_PROJECT_NAMESPACE == "qemu-project" && $CI_COMMIT_BRANCH =~ /^staging/'
-   allow_failure: true
 - if: "$S390X_RUNNER_AVAILABLE"
-   allow_failure: true
 script:
 - mkdir build
 - cd build
- - ../configure --disable-user
+ - ../configure --disable-libssh
   || { cat config.log meson-logs/meson-log.txt; exit 1; }
 - make --output-sync -j`nproc`
 - make --output-sync -j`nproc` check

-ubuntu-22.04-s390x-alldbg:
+ubuntu-20.04-s390x-alldbg:
 extends: .custom_runner_template
 needs: []
 stage: build
 tags:
- - ubuntu_22.04
+ - ubuntu_20.04
 - s390x
 rules:
 - if: '$CI_PROJECT_NAMESPACE == "qemu-project" && $CI_COMMIT_BRANCH =~ /^staging/'
@@ -59,18 +59,18 @@ ubuntu-22.04-s390x-alldbg:
 script:
 - mkdir build
 - cd build
- - ../configure --enable-debug
+ - ../configure --enable-debug --disable-libssh
   || { cat config.log meson-logs/meson-log.txt; exit 1; }
 - make clean
 - make --output-sync -j`nproc`
 - make --output-sync -j`nproc` check

-ubuntu-22.04-s390x-clang:
+ubuntu-20.04-s390x-clang:
 extends: .custom_runner_template
 needs: []
 stage: build
 tags:
- - ubuntu_22.04
+ - ubuntu_20.04
 - s390x
 rules:
 - if: '$CI_PROJECT_NAMESPACE == "qemu-project" && $CI_COMMIT_BRANCH =~ /^staging/'
@@ -82,16 +82,16 @@ ubuntu-22.04-s390x-clang:
 script:
 - mkdir build
 - cd build
- - ../configure --cc=clang --cxx=clang++ --enable-ubsan
+ - ../configure --disable-libssh --cc=clang --cxx=clang++ --enable-sanitizers
   || { cat config.log meson-logs/meson-log.txt; exit 1; }
 - make --output-sync -j`nproc`
 - make --output-sync -j`nproc` check

-ubuntu-22.04-s390x-tci:
+ubuntu-20.04-s390x-tci:
 needs: []
 stage: build
 tags:
- - ubuntu_22.04
+ - ubuntu_20.04
 - s390x
 rules:
 - if: '$CI_PROJECT_NAMESPACE == "qemu-project" && $CI_COMMIT_BRANCH =~ /^staging/'
@@ -103,16 +103,16 @@ ubuntu-22.04-s390x-tci:
 script:
 - mkdir build
 - cd build
- - ../configure --enable-tcg-interpreter
+ - ../configure --disable-libssh --enable-tcg-interpreter
   || { cat config.log meson-logs/meson-log.txt; exit 1; }
 - make --output-sync -j`nproc`

-ubuntu-22.04-s390x-notcg:
+ubuntu-20.04-s390x-notcg:
 extends: .custom_runner_template
 needs: []
 stage: build
 tags:
- - ubuntu_22.04
+ - ubuntu_20.04
 - s390x
 rules:
 - if: '$CI_PROJECT_NAMESPACE == "qemu-project" && $CI_COMMIT_BRANCH =~ /^staging/'
@@ -124,7 +124,7 @@ ubuntu-22.04-s390x-notcg:
 script:
 - mkdir build
 - cd build
- - ../configure --disable-tcg
+ - ../configure --disable-libssh --disable-tcg
   || { cat config.log meson-logs/meson-log.txt; exit 1; }
 - make --output-sync -j`nproc`
 - make --output-sync -j`nproc` check
@@ -1,5 +1,5 @@
 # All ubuntu-22.04 jobs should run successfully in an environment
-# setup by the scripts/ci/setup/ubuntu/build-environment.yml task
+# setup by the scripts/ci/setup/qemu/build-environment.yml task
 # "Install basic packages to build QEMU on Ubuntu 22.04"

 ubuntu-22.04-aarch32-all:
@@ -1,5 +1,5 @@
 # All ubuntu-22.04 jobs should run successfully in an environment
-# setup by the scripts/ci/setup/ubuntu/build-environment.yml task
+# setup by the scripts/ci/setup/qemu/build-environment.yml task
 # "Install basic packages to build QEMU on Ubuntu 22.04"

 ubuntu-22.04-aarch64-all-linux-static:
@@ -103,7 +103,7 @@ ubuntu-22.04-aarch64-clang:
 script:
 - mkdir build
 - cd build
- - ../configure --disable-libssh --cc=clang --cxx=clang++ --enable-ubsan
+ - ../configure --disable-libssh --cc=clang --cxx=clang++ --enable-sanitizers
   || { cat config.log meson-logs/meson-log.txt; exit 1; }
 - make --output-sync -j`nproc --ignore=40`
 - make --output-sync -j`nproc --ignore=40` check
@@ -46,49 +46,3 @@ check-python-tox:
    QEMU_JOB_OPTIONAL: 1
  needs:
    job: python-container
-
-check-rust-tools-nightly:
-  extends: .base_job_template
-  stage: test
-  image: $CI_REGISTRY_IMAGE/qemu/fedora-rust-nightly:$QEMU_CI_CONTAINER_TAG
-  script:
-    - source scripts/ci/gitlab-ci-section
-    - section_start test "Running Rust code checks"
-    - cd build
-    - pyvenv/bin/meson devenv -w ../rust ${CARGO-cargo} fmt --check
-    - make clippy
-    - make rustdoc
-    - section_end test
-  variables:
-    GIT_DEPTH: 1
-  allow_failure: true
-  needs:
-    - job: build-system-fedora-rust-nightly
-      artifacts: true
-  artifacts:
-    when: on_success
-    expire_in: 2 days
-    paths:
-      - rust/target/doc
-
-check-build-units:
-  extends: .base_job_template
-  stage: build
-  image: $CI_REGISTRY_IMAGE/qemu/debian:$QEMU_CI_CONTAINER_TAG
-  needs:
-    job: amd64-debian-container
-  before_script:
-    - source scripts/ci/gitlab-ci-section
-    - section_start setup "Install Tools"
-    - apt install --assume-yes --no-install-recommends jq
-    - section_end setup
-  script:
-    - mkdir build
-    - cd build
-    - section_start configure "Running configure"
-    - ../configure
-    - cd ..
-    - section_end configure
-    - section_start analyse "Analyse"
-    - .gitlab-ci.d/check-units.py build/compile_commands.json
-    - section_end analyse
@@ -1,7 +1,9 @@
 msys2-64bit:
  extends: .base_job_template
  tags:
-  - saas-windows-medium-amd64
+  - shared-windows
+  - windows
+  - windows-1809
  cache:
    key: "$CI_JOB_NAME"
    paths:
@@ -17,9 +19,15 @@ msys2-64bit:
    # This feature doesn't (currently) work with PowerShell, it stops
    # the echo'ing of commands being run and doesn't show any timing
    FF_SCRIPT_SECTIONS: 0
-    CONFIGURE_ARGS: --disable-system --enable-tools -Ddebug=false -Doptimization=0
-    # The Windows git is a bit older so override the default
-    GIT_FETCH_EXTRA_FLAGS: --no-tags --prune --quiet
+    # do not remove "--without-default-devices"!
+    # commit 9f8e6cad65a6 ("gitlab-ci: Speed up the msys2-64bit job by using --without-default-devices"
+    # changed to compile QEMU with the --without-default-devices switch
+    # for this job, because otherwise the build could not complete within
+    # the project timeout.
+    CONFIGURE_ARGS:  --target-list=x86_64-softmmu --without-default-devices -Ddebug=false -Doptimization=0
+    # qTests don't run successfully with "--without-default-devices",
+    # so let's exclude the qtests from CI for now.
+    TEST_ARGS: --no-suite qtest
  artifacts:
    name: "$CI_JOB_NAME-$CI_COMMIT_REF_SLUG"
    expire_in: 7 days
@@ -76,19 +84,35 @@ msys2-64bit:
      bison diffutils flex
      git grep make sed
      mingw-w64-x86_64-binutils
+      mingw-w64-x86_64-capstone
      mingw-w64-x86_64-ccache
      mingw-w64-x86_64-curl
+      mingw-w64-x86_64-cyrus-sasl
+      mingw-w64-x86_64-dtc
      mingw-w64-x86_64-gcc
      mingw-w64-x86_64-glib2
+      mingw-w64-x86_64-gnutls
+      mingw-w64-x86_64-gtk3
+      mingw-w64-x86_64-libgcrypt
+      mingw-w64-x86_64-libjpeg-turbo
      mingw-w64-x86_64-libnfs
+      mingw-w64-x86_64-libpng
      mingw-w64-x86_64-libssh
+      mingw-w64-x86_64-libtasn1
+      mingw-w64-x86_64-libusb
+      mingw-w64-x86_64-lzo2
+      mingw-w64-x86_64-nettle
      mingw-w64-x86_64-ninja
      mingw-w64-x86_64-pixman
      mingw-w64-x86_64-pkgconf
      mingw-w64-x86_64-python
+      mingw-w64-x86_64-SDL2
+      mingw-w64-x86_64-SDL2_image
+      mingw-w64-x86_64-snappy
+      mingw-w64-x86_64-spice
+      mingw-w64-x86_64-usbredir
      mingw-w64-x86_64-zstd"
  - Write-Output "Running build at $(Get-Date -Format u)"
-  - $env:JOBS = $(.\msys64\usr\bin\bash -lc nproc)
  - $env:CHERE_INVOKING = 'yes'  # Preserve the current working directory
  - $env:MSYS = 'winsymlinks:native' # Enable native Windows symlink
  - $env:CCACHE_BASEDIR = "$env:CI_PROJECT_DIR"
@@ -99,8 +123,8 @@ msys2-64bit:
  - mkdir build
  - cd build
  - ..\msys64\usr\bin\bash -lc "ccache --zero-stats"
-  - ..\msys64\usr\bin\bash -lc "../configure $CONFIGURE_ARGS"
-  - ..\msys64\usr\bin\bash -lc "make -j$env:JOBS"
+  - ..\msys64\usr\bin\bash -lc "../configure --enable-fdt=system $CONFIGURE_ARGS"
+  - ..\msys64\usr\bin\bash -lc "make"
  - ..\msys64\usr\bin\bash -lc "make check MTESTARGS='$TEST_ARGS' || { cat meson-logs/testlog.txt; exit 1; } ;"
  - ..\msys64\usr\bin\bash -lc "ccache --show-stats"
  - Write-Output "Finished build at $(Get-Date -Format u)"
@@ -67,8 +67,6 @@ Andrey Drobyshev <andrey.drobyshev@virtuozzo.com> Andrey Drobyshev via <qemu-blo
 BALATON Zoltan <balaton@eik.bme.hu> BALATON Zoltan via <qemu-ppc@nongnu.org>

 # Next, replace old addresses by a more recent one.
-Akihiko Odaki <odaki@rsg.ci.i.u-tokyo.ac.jp> <akihiko.odaki@daynix.com>
-Akihiko Odaki <odaki@rsg.ci.i.u-tokyo.ac.jp> <akihiko.odaki@gmail.com>
 Aleksandar Markovic <aleksandar.qemu.devel@gmail.com> <aleksandar.markovic@mips.com>
 Aleksandar Markovic <aleksandar.qemu.devel@gmail.com> <aleksandar.markovic@imgtec.com>
 Aleksandar Markovic <aleksandar.qemu.devel@gmail.com> <amarkovic@wavecomp.com>
@@ -77,8 +75,6 @@ Aleksandar Rikalo <aleksandar.rikalo@syrmia.com> <aleksandar.rikalo@rt-rk.com>
 Alexander Graf <agraf@csgraf.de> <agraf@suse.de>
 Ani Sinha <anisinha@redhat.com> <ani@anisinha.ca>
 Anthony Liguori <anthony@codemonkey.ws> Anthony Liguori <aliguori@us.ibm.com>
-Brian Cain <brian.cain@oss.qualcomm.com> <bcain@quicinc.com>
-Brian Cain <brian.cain@oss.qualcomm.com> <quic_bcain@quicinc.com>
 Christian Borntraeger <borntraeger@linux.ibm.com> <borntraeger@de.ibm.com>
 Damien Hedde <damien.hedde@dahe.fr> <damien.hedde@greensocs.com>
 Filip Bozuta <filip.bozuta@syrmia.com> <filip.bozuta@rt-rk.com.com>
@@ -89,9 +85,8 @@ Huacai Chen <chenhuacai@kernel.org> <chenhc@lemote.com>
 Huacai Chen <chenhuacai@kernel.org> <chenhuacai@loongson.cn>
 James Hogan <jhogan@kernel.org> <james.hogan@imgtec.com>
 Juan Quintela <quintela@trasno.org> <quintela@redhat.com>
-Leif Lindholm <leif.lindholm@oss.qualcomm.com> <quic_llindhol@quicinc.com>
-Leif Lindholm <leif.lindholm@oss.qualcomm.com> <leif.lindholm@linaro.org>
-Leif Lindholm <leif.lindholm@oss.qualcomm.com> <leif@nuviainc.com>
+Leif Lindholm <quic_llindhol@quicinc.com> <leif.lindholm@linaro.org>
+Leif Lindholm <quic_llindhol@quicinc.com> <leif@nuviainc.com>
 Luc Michel <luc@lmichel.fr> <luc.michel@git.antfield.fr>
 Luc Michel <luc@lmichel.fr> <luc.michel@greensocs.com>
 Luc Michel <luc@lmichel.fr> <lmichel@kalray.eu>
@@ -105,7 +100,6 @@ Philippe Mathieu-Daudé <philmd@linaro.org> <f4bug@amsat.org>
 Philippe Mathieu-Daudé <philmd@linaro.org> <philmd@redhat.com>
 Philippe Mathieu-Daudé <philmd@linaro.org> <philmd@fungible.com>
 Roman Bolshakov <rbolshakov@ddn.com> <r.bolshakov@yadro.com>
-Sriram Yagnaraman <sriram.yagnaraman@ericsson.com> <sriram.yagnaraman@est.tech>
 Stefan Brankovic <stefan.brankovic@syrmia.com> <stefan.brankovic@rt-rk.com.com>
 Stefan Weil <sw@weilnetz.de> Stefan Weil <stefan@weilnetz.de>
 Taylor Simpson <ltaylorsimpson@gmail.com> <tsimpson@quicinc.com>
@@ -1,5 +1,5 @@
 os: linux
-dist: jammy
+dist: focal
 language: c
 compiler:
  - gcc
@@ -7,11 +7,13 @@ cache:
  # There is one cache per branch and compiler version.
  # characteristics of each job are used to identify the cache:
  # - OS name (currently only linux)
-  # - OS distribution (e.g. "jammy" for Linux)
+  # - OS distribution (for Linux, bionic or focal)
  # - Names and values of visible environment variables set in .travis.yml or Settings panel
  timeout: 1200
  ccache: true
  pip: true
+  directories:
+  - $HOME/avocado/data/cache


 # The channel name "irc.oftc.net#qemu" is encrypted against qemu/qemu
@@ -33,7 +35,7 @@ env:
    - TEST_BUILD_CMD=""
    - TEST_CMD="make check V=1"
    # This is broadly a list of "mainline" system targets which have support across the major distros
-    - MAIN_SYSTEM_TARGETS="aarch64-softmmu,mips64-softmmu,ppc64-softmmu,riscv64-softmmu,s390x-softmmu,x86_64-softmmu"
+    - MAIN_SOFTMMU_TARGETS="aarch64-softmmu,mips64-softmmu,ppc64-softmmu,riscv64-softmmu,s390x-softmmu,x86_64-softmmu"
    - CCACHE_SLOPPINESS="include_file_ctime,include_file_mtime"
    - CCACHE_MAXSIZE=1G
    - G_MESSAGES_DEBUG=error
@@ -79,9 +81,45 @@ after_script:
 jobs:
  include:

-    - name: "[ppc64] Clang check-tcg"
+    - name: "[aarch64] GCC check-tcg"
+      arch: arm64
+      dist: focal
+      addons:
+        apt_packages:
+          - libaio-dev
+          - libattr1-dev
+          - libbrlapi-dev
+          - libcacard-dev
+          - libcap-ng-dev
+          - libfdt-dev
+          - libgcrypt20-dev
+          - libgnutls28-dev
+          - libgtk-3-dev
+          - libiscsi-dev
+          - liblttng-ust-dev
+          - libncurses5-dev
+          - libnfs-dev
+          - libpixman-1-dev
+          - libpng-dev
+          - librados-dev
+          - libsdl2-dev
+          - libseccomp-dev
+          - liburcu-dev
+          - libusb-1.0-0-dev
+          - libvdeplug-dev
+          - libvte-2.91-dev
+          - ninja-build
+          # Tests dependencies
+          - genisoimage
+      env:
+        - TEST_CMD="make check check-tcg V=1"
+        - CONFIG="--disable-containers --enable-fdt=system
+                  --target-list=${MAIN_SOFTMMU_TARGETS} --cxx=/bin/false"
+        - UNRELIABLE=true
+
+    - name: "[ppc64] GCC check-tcg"
      arch: ppc64le
-      compiler: clang
+      dist: focal
      addons:
        apt_packages:
          - libaio-dev
@@ -107,7 +145,6 @@ jobs:
          - libvdeplug-dev
          - libvte-2.91-dev
          - ninja-build
-          - python3-tomli
          # Tests dependencies
          - genisoimage
      env:
@@ -117,6 +154,7 @@ jobs:

    - name: "[s390x] GCC check-tcg"
      arch: s390x
+      dist: focal
      addons:
        apt_packages:
          - libaio-dev
@@ -142,13 +180,13 @@ jobs:
          - libvdeplug-dev
          - libvte-2.91-dev
          - ninja-build
-          - python3-tomli
          # Tests dependencies
          - genisoimage
      env:
        - TEST_CMD="make check check-tcg V=1"
-        - CONFIG="--disable-containers
-            --target-list=hppa-softmmu,mips64-softmmu,ppc64-softmmu,riscv64-softmmu,s390x-softmmu,x86_64-softmmu"
+        - CONFIG="--disable-containers --enable-fdt=system
+                  --target-list=${MAIN_SOFTMMU_TARGETS},s390x-linux-user"
+        - UNRELIABLE=true
      script:
        - BUILD_RC=0 && make -j${JOBS} || BUILD_RC=$?
        - |
@@ -159,9 +197,9 @@ jobs:
              $(exit $BUILD_RC);
          fi

-    - name: "[s390x] Clang (other-system)"
+    - name: "[s390x] GCC (other-system)"
      arch: s390x
-      compiler: clang
+      dist: focal
      addons:
        apt_packages:
          - libaio-dev
@@ -182,16 +220,17 @@ jobs:
          - libsnappy-dev
          - libzstd-dev
          - nettle-dev
+          - xfslibs-dev
          - ninja-build
-          - python3-tomli
          # Tests dependencies
          - genisoimage
      env:
-        - CONFIG="--disable-containers --audio-drv-list=sdl --disable-user
-            --target-list=arm-softmmu,avr-softmmu,microblaze-softmmu,sh4eb-softmmu,sparc64-softmmu,xtensaeb-softmmu"
+        - CONFIG="--disable-containers --enable-fdt=system --audio-drv-list=sdl
+                  --disable-user --target-list-exclude=${MAIN_SOFTMMU_TARGETS}"

    - name: "[s390x] GCC (user)"
      arch: s390x
+      dist: focal
      addons:
        apt_packages:
          - libgcrypt20-dev
@@ -200,14 +239,13 @@ jobs:
          - ninja-build
          - flex
          - bison
-          - python3-tomli
      env:
-        - TEST_CMD="make check check-tcg V=1"
        - CONFIG="--disable-containers --disable-system"

    - name: "[s390x] Clang (disable-tcg)"
      arch: s390x
-      compiler: clang
+      dist: focal
+      compiler: clang-10
      addons:
        apt_packages:
          - libaio-dev
@@ -233,8 +271,9 @@ jobs:
          - libvdeplug-dev
          - libvte-2.91-dev
          - ninja-build
-          - python3-tomli
+          - clang-10
      env:
        - TEST_CMD="make check-unit"
        - CONFIG="--disable-containers --disable-tcg --enable-kvm --disable-tools
                  --enable-fdt=system --host-cc=clang --cxx=clang++"
+        - UNRELIABLE=true
@@ -2,7 +2,7 @@
 		       Version 2, June 1991

 Copyright (C) 1989, 1991 Free Software Foundation, Inc.,
- <https://fsf.org/>
+ 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
 Everyone is permitted to copy and distribute verbatim copies
 of this license document, but changing it is not allowed.

@@ -304,7 +304,8 @@ the "copyright" line and a pointer to where the full notice is found.
    GNU General Public License for more details.

    You should have received a copy of the GNU General Public License along
-    with this program; if not, see <https://www.gnu.org/licenses/>.
+    with this program; if not, write to the Free Software Foundation, Inc.,
+    51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.

 Also add information on how to contact you by electronic and paper mail.

@@ -2,7 +2,7 @@
                       Version 2.1, February 1999

 Copyright (C) 1991, 1999 Free Software Foundation, Inc.
- <https://fsf.org/>
+ 51 Franklin Street, Fifth Floor, Boston, MA  02110-1301  USA
 Everyone is permitted to copy and distribute verbatim copies
 of this license document, but changing it is not allowed.

@@ -484,7 +484,8 @@ convey the exclusion of warranty; and each file should have at least the
    Lesser General Public License for more details.

    You should have received a copy of the GNU Lesser General Public
-    License along with this library; if not, see <https://www.gnu.org/licenses/>.
+    License along with this library; if not, write to the Free Software
+    Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA  02110-1301  USA

 Also add information on how to contact you by electronic and paper mail.

@@ -4,4 +4,3 @@ source accel/Kconfig
 source target/Kconfig
 source hw/Kconfig
 source semihosting/Kconfig
-source rust/Kconfig
@@ -5,12 +5,6 @@
 config LINUX
    bool

-config LIBCBOR
-    bool
-
-config GNUTLS
-    bool
-
 config OPENGL
    bool

@@ -29,9 +23,6 @@ config IVSHMEM
 config TPM
    bool

-config FDT
-    bool
-
 config VHOST_USER
    bool

@@ -44,6 +35,9 @@ config VHOST_KERNEL
 config VIRTFS
    bool

+config PVRDMA
+    bool
+
 config MULTIPROCESS_ALLOWED
    bool
    imply MULTIPROCESS
@@ -58,9 +52,3 @@ config VFIO_USER_SERVER_ALLOWED

 config HV_BALLOON_POSSIBLE
    bool
-
-config HAVE_RUST
-    bool
-
-config MAC_PVG
-    bool
@@ -78,8 +78,7 @@ x := $(shell rm -rf meson-private meson-info meson-logs)
 endif

 # 1. ensure config-host.mak is up-to-date
-config-host.mak: $(SRC_PATH)/configure $(SRC_PATH)/scripts/meson-buildoptions.sh \
-		$(SRC_PATH)/pythondeps.toml $(SRC_PATH)/VERSION
+config-host.mak: $(SRC_PATH)/configure $(SRC_PATH)/scripts/meson-buildoptions.sh $(SRC_PATH)/VERSION
 	@echo config-host.mak is out-of-date, running configure
 	@if test -f meson-private/coredata.dat; then \
 	  ./config.status --skip-meson; \
@@ -142,13 +141,8 @@ MAKE.n = $(findstring n,$(firstword $(filter-out --%,$(MAKEFLAGS))))
 MAKE.k = $(findstring k,$(firstword $(filter-out --%,$(MAKEFLAGS))))
 MAKE.q = $(findstring q,$(firstword $(filter-out --%,$(MAKEFLAGS))))
 MAKE.nq = $(if $(word 2, $(MAKE.n) $(MAKE.q)),nq)
-NINJAFLAGS = \
-        $(if $V,-v) \
-        $(if $(MAKE.n), -n) \
-        $(if $(MAKE.k), -k0) \
-        $(filter-out -j, \
-          $(or $(filter -l% -j%, $(MAKEFLAGS)), \
-               $(if $(filter --jobserver-auth=%, $(MAKEFLAGS)),, -j1))) \
+NINJAFLAGS = $(if $V,-v) $(if $(MAKE.n), -n) $(if $(MAKE.k), -k0) \
+        $(filter-out -j, $(lastword -j1 $(filter -l% -j%, $(MAKEFLAGS)))) \
        -d keepdepfile
 ninja-cmd-goals = $(or $(MAKECMDGOALS), all)
 ninja-cmd-goals += $(foreach g, $(MAKECMDGOALS), $(.ninja-goals.$g))
@@ -187,6 +181,11 @@ SUBDIR_RULES=$(foreach t, all clean distclean, $(addsuffix /$(t), $(SUBDIRS)))
 $(SUBDIR_RULES):
 	$(call quiet-command,$(MAKE) $(SUBDIR_MAKEFLAGS) -C $(dir $@) V="$(V)" TARGET_DIR="$(dir $@)" $(notdir $@),)

+ifneq ($(filter contrib/plugins, $(SUBDIRS)),)
+.PHONY: plugins
+plugins: contrib/plugins/all
+endif
+
 .PHONY: recurse-all recurse-clean
 recurse-all: $(addsuffix /all, $(SUBDIRS))
 recurse-clean: $(addsuffix /clean, $(SUBDIRS))
@@ -207,10 +206,10 @@ clean: recurse-clean

 VERSION = $(shell cat $(SRC_PATH)/VERSION)

-dist: qemu-$(VERSION).tar.xz
+dist: qemu-$(VERSION).tar.bz2

-qemu-%.tar.xz:
-	$(SRC_PATH)/scripts/make-release "$(SRC_PATH)" "$(patsubst qemu-%.tar.xz,%,$@)"
+qemu-%.tar.bz2:
+	$(SRC_PATH)/scripts/make-release "$(SRC_PATH)" "$(patsubst qemu-%.tar.bz2,%,$@)"

 distclean: clean recurse-distclean
 	-$(quiet-@)test -f build.ninja && $(NINJA) $(NINJAFLAGS) -t clean -g || :
@@ -227,7 +226,6 @@ distclean: clean recurse-distclean
 	rm -Rf .sdk qemu-bundle

 find-src-path = find "$(SRC_PATH)" -path "$(SRC_PATH)/meson" -prune -o \
-	-path "$(SRC_PATH)/.pc" -prune -o \
 	-type l -prune -o \( -name "*.[chsS]" -o -name "*.[ch].inc" \)

 .PHONY: ctags
@@ -303,6 +301,11 @@ help:
 	$(call print-help,cscope,Generate cscope index)
 	$(call print-help,sparse,Run sparse on the QEMU source)
 	@echo  ''
+ifneq ($(filter contrib/plugins, $(SUBDIRS)),)
+	@echo  'Plugin targets:'
+	$(call print-help,plugins,Build the example TCG plugins)
+	@echo  ''
+endif
 	@echo  'Cleaning targets:'
 	$(call print-help,clean,Remove most generated files but keep the config)
 	$(call print-help,distclean,Remove all generated files)
@@ -82,7 +82,7 @@ guidelines set out in the `style section
 the Developers Guide.

 Additional information on submitting patches can be found online via
-the QEMU website:
+the QEMU website

 * `<https://wiki.qemu.org/Contribute/SubmitAPatch>`_
 * `<https://wiki.qemu.org/Contribute/TrivialPatches>`_
@@ -102,7 +102,7 @@ requires a working 'git send-email' setup, and by default doesn't
 automate everything, so you may want to go through the above steps
 manually for once.

-For installation instructions, please go to:
+For installation instructions, please go to

 *  `<https://github.com/stefanha/git-publish>`_

@@ -159,7 +159,7 @@ Contact
 =======

 The QEMU community can be contacted in a number of ways, with the two
-main methods being email and IRC:
+main methods being email and IRC

 * `<mailto:qemu-devel@nongnu.org>`_
 * `<https://lists.nongnu.org/mailman/listinfo/qemu-devel>`_
@@ -1 +1 @@
-10.0.50
+8.2.50
@@ -16,5 +16,4 @@ config KVM
 config XEN
    bool
    select FSDEV_9P if VIRTFS
-    select PCI_EXPRESS_GENERIC_BRIDGE
    select XEN_BUS
@@ -25,11 +25,10 @@
 */

 #include "qemu/osdep.h"
-#include "qemu/lockcnt.h"
 #include "qemu/thread.h"
 #include "qemu/main-loop.h"
 #include "hw/core/cpu.h"
-#include "system/accel-blocker.h"
+#include "sysemu/accel-blocker.h"

 static QemuLockCnt accel_in_ioctl_lock;
 static QemuEvent accel_in_ioctl_event;
@@ -1,144 +0,0 @@
-/*
- * QEMU accel class, components common to system emulation and user mode
- *
- * Copyright (c) 2003-2008 Fabrice Bellard
- * Copyright (c) 2014 Red Hat Inc.
- *
- * SPDX-License-Identifier: MIT
- */
-
-#include "qemu/osdep.h"
-#include "qemu/accel.h"
-#include "qemu/target-info.h"
-#include "accel/accel-ops.h"
-#include "accel/accel-cpu.h"
-#include "accel/accel-cpu-ops.h"
-#include "accel-internal.h"
-
-/* Lookup AccelClass from opt_name. Returns NULL if not found */
-AccelClass *accel_find(const char *opt_name)
-{
-    char *class_name = g_strdup_printf(ACCEL_CLASS_NAME("%s"), opt_name);
-    AccelClass *ac = ACCEL_CLASS(module_object_class_by_name(class_name));
-    g_free(class_name);
-    return ac;
-}
-
-/* Return the name of the current accelerator */
-const char *current_accel_name(void)
-{
-    AccelClass *ac = ACCEL_GET_CLASS(current_accel());
-
-    return ac->name;
-}
-
-static void accel_init_cpu_int_aux(ObjectClass *klass, void *opaque)
-{
-    CPUClass *cc = CPU_CLASS(klass);
-    AccelCPUClass *accel_cpu = opaque;
-
-    /*
-     * The first callback allows accel-cpu to run initializations
-     * for the CPU, customizing CPU behavior according to the accelerator.
-     *
-     * The second one allows the CPU to customize the accel-cpu
-     * behavior according to the CPU.
-     *
-     * The second is currently only used by TCG, to specialize the
-     * TCGCPUOps depending on the CPU type.
-     */
-    cc->accel_cpu = accel_cpu;
-    if (accel_cpu->cpu_class_init) {
-        accel_cpu->cpu_class_init(cc);
-    }
-    if (cc->init_accel_cpu) {
-        cc->init_accel_cpu(accel_cpu, cc);
-    }
-}
-
-/* initialize the arch-specific accel CpuClass interfaces */
-static void accel_init_cpu_interfaces(AccelClass *ac)
-{
-    const char *ac_name; /* AccelClass name */
-    char *acc_name;      /* AccelCPUClass name */
-    ObjectClass *acc;    /* AccelCPUClass */
-    const char *cpu_resolving_type = target_cpu_type();
-
-    ac_name = object_class_get_name(OBJECT_CLASS(ac));
-    g_assert(ac_name != NULL);
-
-    acc_name = g_strdup_printf("%s-%s", ac_name, cpu_resolving_type);
-    acc = object_class_by_name(acc_name);
-    g_free(acc_name);
-
-    if (acc) {
-        object_class_foreach(accel_init_cpu_int_aux,
-                             cpu_resolving_type, false, acc);
-    }
-}
-
-void accel_init_interfaces(AccelClass *ac)
-{
-    accel_init_ops_interfaces(ac);
-    accel_init_cpu_interfaces(ac);
-}
-
-void accel_cpu_instance_init(CPUState *cpu)
-{
-    if (cpu->cc->accel_cpu && cpu->cc->accel_cpu->cpu_instance_init) {
-        cpu->cc->accel_cpu->cpu_instance_init(cpu);
-    }
-}
-
-bool accel_cpu_common_realize(CPUState *cpu, Error **errp)
-{
-    AccelState *accel = current_accel();
-    AccelClass *acc = ACCEL_GET_CLASS(accel);
-
-    /* target specific realization */
-    if (cpu->cc->accel_cpu
-        && cpu->cc->accel_cpu->cpu_target_realize
-        && !cpu->cc->accel_cpu->cpu_target_realize(cpu, errp)) {
-        return false;
-    }
-
-    /* generic realization */
-    if (acc->cpu_common_realize && !acc->cpu_common_realize(cpu, errp)) {
-        return false;
-    }
-
-    return true;
-}
-
-void accel_cpu_common_unrealize(CPUState *cpu)
-{
-    AccelState *accel = current_accel();
-    AccelClass *acc = ACCEL_GET_CLASS(accel);
-
-    /* generic unrealization */
-    if (acc->cpu_common_unrealize) {
-        acc->cpu_common_unrealize(cpu);
-    }
-}
-
-int accel_supported_gdbstub_sstep_flags(void)
-{
-    AccelState *accel = current_accel();
-    AccelClass *acc = ACCEL_GET_CLASS(accel);
-    if (acc->gdbstub_supported_sstep_flags) {
-        return acc->gdbstub_supported_sstep_flags(accel);
-    }
-    return 0;
-}
-
-static const TypeInfo accel_types[] = {
-    {
-        .name           = TYPE_ACCEL,
-        .parent         = TYPE_OBJECT,
-        .class_size     = sizeof(AccelClass),
-        .instance_size  = sizeof(AccelState),
-        .abstract       = true,
-    },
-};
-
-DEFINE_TYPES(accel_types)
@@ -1,35 +0,0 @@
-/*
- * QMP commands related to accelerators
- *
- * Copyright (c) Linaro
- *
- * SPDX-License-Identifier: GPL-2.0-or-later
- */
-
-#include "qemu/osdep.h"
-#include "qemu/accel.h"
-#include "qapi/type-helpers.h"
-#include "qapi/qapi-commands-accelerator.h"
-#include "accel/accel-ops.h"
-#include "accel/accel-cpu-ops.h"
-#include "hw/core/cpu.h"
-
-HumanReadableText *qmp_x_accel_stats(Error **errp)
-{
-    AccelState *accel = current_accel();
-    AccelClass *acc = ACCEL_GET_CLASS(accel);
-    g_autoptr(GString) buf = g_string_new("");
-
-    if (acc->get_stats) {
-        acc->get_stats(accel, buf);
-    }
-    if (acc->ops->get_vcpu_stats) {
-        CPUState *cpu;
-
-        CPU_FOREACH(cpu) {
-            acc->ops->get_vcpu_stats(cpu, buf);
-        }
-    }
-
-    return human_readable_text_from_str(buf);
-}
@@ -25,15 +25,10 @@

 #include "qemu/osdep.h"
 #include "qemu/accel.h"
-#include "qapi/qapi-commands-accelerator.h"
-#include "monitor/monitor.h"
 #include "hw/boards.h"
-#include "hw/core/cpu.h"
-#include "accel/accel-ops.h"
-#include "accel/accel-cpu-ops.h"
-#include "system/cpus.h"
+#include "sysemu/cpus.h"
 #include "qemu/error-report.h"
-#include "accel-internal.h"
+#include "accel-system.h"

 int accel_init_machine(AccelState *accel, MachineState *ms)
 {
@@ -41,7 +36,7 @@ int accel_init_machine(AccelState *accel, MachineState *ms)
    int ret;
    ms->accelerator = accel;
    *(acc->allowed) = true;
-    ret = acc->init_machine(accel, ms);
+    ret = acc->init_machine(ms);
    if (ret < 0) {
        ms->accelerator = NULL;
        *(acc->allowed) = false;
@@ -62,21 +57,12 @@ void accel_setup_post(MachineState *ms)
    AccelState *accel = ms->accelerator;
    AccelClass *acc = ACCEL_GET_CLASS(accel);
    if (acc->setup_post) {
-        acc->setup_post(accel);
-    }
-}
-
-void accel_pre_resume(MachineState *ms, bool step_pending)
-{
-    AccelState *accel = ms->accelerator;
-    AccelClass *acc = ACCEL_GET_CLASS(accel);
-    if (acc->pre_resume_vm) {
-        acc->pre_resume_vm(accel, step_pending);
+        acc->setup_post(ms, accel);
    }
 }

 /* initialize the arch-independent accel operation interfaces */
-void accel_init_ops_interfaces(AccelClass *ac)
+void accel_system_init_ops_interfaces(AccelClass *ac)
 {
    const char *ac_name;
    char *ops_name;
@@ -87,35 +73,30 @@ void accel_init_ops_interfaces(AccelClass *ac)
    g_assert(ac_name != NULL);

    ops_name = g_strdup_printf("%s" ACCEL_OPS_SUFFIX, ac_name);
+    ops = ACCEL_OPS_CLASS(module_object_class_by_name(ops_name));
    oc = module_object_class_by_name(ops_name);
    if (!oc) {
        error_report("fatal: could not load module for type '%s'", ops_name);
        exit(1);
    }
    g_free(ops_name);
+    ops = ACCEL_OPS_CLASS(oc);
    /*
     * all accelerators need to define ops, providing at least a mandatory
     * non-NULL create_vcpu_thread operation.
     */
-    ops = ACCEL_OPS_CLASS(oc);
-    ac->ops = ops;
+    g_assert(ops != NULL);
    if (ops->ops_init) {
-        ops->ops_init(ac);
+        ops->ops_init(ops);
    }
    cpus_register_accel(ops);
 }

-static void accel_ops_class_init(ObjectClass *oc, const void *data)
-{
-    monitor_register_hmp_info_hrt("accel", qmp_x_accel_stats);
-}
-
 static const TypeInfo accel_ops_type_info = {
    .name = TYPE_ACCEL_OPS,
    .parent = TYPE_OBJECT,
    .abstract = true,
    .class_size = sizeof(AccelOpsClass),
-    .class_init = accel_ops_class_init,
 };

 static void accel_system_register_types(void)
@@ -1,5 +1,5 @@
 /*
- * QEMU accel internal functions
+ * QEMU System Emulation accel internal functions
 *
 * Copyright 2021 SUSE LLC
 *
@@ -7,11 +7,9 @@
 * See the COPYING file in the top-level directory.
 */

-#ifndef ACCEL_INTERNAL_H
-#define ACCEL_INTERNAL_H
+#ifndef ACCEL_SYSTEM_H
+#define ACCEL_SYSTEM_H

-#include "qemu/accel.h"
-
-void accel_init_ops_interfaces(AccelClass *ac);
+void accel_system_init_ops_interfaces(AccelClass *ac);

 #endif /* ACCEL_SYSTEM_H */
@@ -24,7 +24,141 @@
 */

 #include "qemu/osdep.h"
-#include "accel/accel-cpu-target.h"
+#include "qemu/accel.h"
+
+#include "cpu.h"
+#include "hw/core/accel-cpu.h"
+
+#ifndef CONFIG_USER_ONLY
+#include "accel-system.h"
+#endif /* !CONFIG_USER_ONLY */
+
+static const TypeInfo accel_type = {
+    .name = TYPE_ACCEL,
+    .parent = TYPE_OBJECT,
+    .class_size = sizeof(AccelClass),
+    .instance_size = sizeof(AccelState),
+};
+
+/* Lookup AccelClass from opt_name. Returns NULL if not found */
+AccelClass *accel_find(const char *opt_name)
+{
+    char *class_name = g_strdup_printf(ACCEL_CLASS_NAME("%s"), opt_name);
+    AccelClass *ac = ACCEL_CLASS(module_object_class_by_name(class_name));
+    g_free(class_name);
+    return ac;
+}
+
+/* Return the name of the current accelerator */
+const char *current_accel_name(void)
+{
+    AccelClass *ac = ACCEL_GET_CLASS(current_accel());
+
+    return ac->name;
+}
+
+static void accel_init_cpu_int_aux(ObjectClass *klass, void *opaque)
+{
+    CPUClass *cc = CPU_CLASS(klass);
+    AccelCPUClass *accel_cpu = opaque;
+
+    /*
+     * The first callback allows accel-cpu to run initializations
+     * for the CPU, customizing CPU behavior according to the accelerator.
+     *
+     * The second one allows the CPU to customize the accel-cpu
+     * behavior according to the CPU.
+     *
+     * The second is currently only used by TCG, to specialize the
+     * TCGCPUOps depending on the CPU type.
+     */
+    cc->accel_cpu = accel_cpu;
+    if (accel_cpu->cpu_class_init) {
+        accel_cpu->cpu_class_init(cc);
+    }
+    if (cc->init_accel_cpu) {
+        cc->init_accel_cpu(accel_cpu, cc);
+    }
+}
+
+/* initialize the arch-specific accel CpuClass interfaces */
+static void accel_init_cpu_interfaces(AccelClass *ac)
+{
+    const char *ac_name; /* AccelClass name */
+    char *acc_name;      /* AccelCPUClass name */
+    ObjectClass *acc;    /* AccelCPUClass */
+
+    ac_name = object_class_get_name(OBJECT_CLASS(ac));
+    g_assert(ac_name != NULL);
+
+    acc_name = g_strdup_printf("%s-%s", ac_name, CPU_RESOLVING_TYPE);
+    acc = object_class_by_name(acc_name);
+    g_free(acc_name);
+
+    if (acc) {
+        object_class_foreach(accel_init_cpu_int_aux,
+                             CPU_RESOLVING_TYPE, false, acc);
+    }
+}
+
+void accel_init_interfaces(AccelClass *ac)
+{
+#ifndef CONFIG_USER_ONLY
+    accel_system_init_ops_interfaces(ac);
+#endif /* !CONFIG_USER_ONLY */
+
+    accel_init_cpu_interfaces(ac);
+}
+
+void accel_cpu_instance_init(CPUState *cpu)
+{
+    CPUClass *cc = CPU_GET_CLASS(cpu);
+
+    if (cc->accel_cpu && cc->accel_cpu->cpu_instance_init) {
+        cc->accel_cpu->cpu_instance_init(cpu);
+    }
+}
+
+bool accel_cpu_common_realize(CPUState *cpu, Error **errp)
+{
+    CPUClass *cc = CPU_GET_CLASS(cpu);
+    AccelState *accel = current_accel();
+    AccelClass *acc = ACCEL_GET_CLASS(accel);
+
+    /* target specific realization */
+    if (cc->accel_cpu && cc->accel_cpu->cpu_target_realize
+        && !cc->accel_cpu->cpu_target_realize(cpu, errp)) {
+        return false;
+    }
+
+    /* generic realization */
+    if (acc->cpu_common_realize && !acc->cpu_common_realize(cpu, errp)) {
+        return false;
+    }
+
+    return true;
+}
+
+void accel_cpu_common_unrealize(CPUState *cpu)
+{
+    AccelState *accel = current_accel();
+    AccelClass *acc = ACCEL_GET_CLASS(accel);
+
+    /* generic unrealization */
+    if (acc->cpu_common_unrealize) {
+        acc->cpu_common_unrealize(cpu);
+    }
+}
+
+int accel_supported_gdbstub_sstep_flags(void)
+{
+    AccelState *accel = current_accel();
+    AccelClass *acc = ACCEL_GET_CLASS(accel);
+    if (acc->gdbstub_supported_sstep_flags) {
+        return acc->gdbstub_supported_sstep_flags();
+    }
+    return 0;
+}

 static const TypeInfo accel_cpu_type = {
    .name = TYPE_ACCEL_CPU,
@@ -35,6 +169,7 @@ static const TypeInfo accel_cpu_type = {

 static void register_accel_types(void)
 {
+    type_register_static(&accel_type);
    type_register_static(&accel_cpu_type);
 }

@@ -9,12 +9,6 @@

 #include "qemu/osdep.h"
 #include "qemu/accel.h"
-#include "accel-internal.h"
-
-void accel_init_ops_interfaces(AccelClass *ac)
-{
-    /* nothing */
-}

 AccelState *current_accel(void)
 {
@@ -13,11 +13,10 @@

 #include "qemu/osdep.h"
 #include "qemu/rcu.h"
-#include "system/cpus.h"
+#include "sysemu/cpus.h"
 #include "qemu/guest-random.h"
 #include "qemu/main-loop.h"
 #include "hw/core/cpu.h"
-#include "accel/dummy-cpus.h"

 static void *dummy_cpu_thread_fn(void *arg)
 {
@@ -69,6 +68,9 @@ void dummy_start_vcpu_thread(CPUState *cpu)
 {
    char thread_name[VCPU_THREAD_NAME_SIZE];

+    cpu->thread = g_malloc0(sizeof(QemuThread));
+    cpu->halt_cond = g_malloc0(sizeof(QemuCond));
+    qemu_cond_init(cpu->halt_cond);
    snprintf(thread_name, VCPU_THREAD_NAME_SIZE, "CPU %d/DUMMY",
             cpu->cpu_index);
    qemu_thread_create(cpu->thread, thread_name, dummy_cpu_thread_fn, cpu,
@@ -1,14 +0,0 @@
-/*
- * Dummy cpu thread code
- *
- * Copyright IBM, Corp. 2011
- *
- * SPDX-License-Identifier: GPL-2.0-or-later
- */
-
-#ifndef ACCEL_DUMMY_CPUS_H
-#define ACCEL_DUMMY_CPUS_H
-
-void dummy_start_vcpu_thread(CPUState *cpu);
-
-#endif
@@ -48,20 +48,23 @@
 */

 #include "qemu/osdep.h"
-#include "qemu/guest-random.h"
+#include "qemu/error-report.h"
 #include "qemu/main-loop.h"
-#include "qemu/queue.h"
-#include "gdbstub/enums.h"
-#include "exec/cpu-common.h"
-#include "hw/core/cpu.h"
-#include "accel/accel-cpu-ops.h"
-#include "system/cpus.h"
-#include "system/hvf.h"
-#include "system/hvf_int.h"
-#include <mach/mach_time.h>
+#include "exec/address-spaces.h"
+#include "exec/exec-all.h"
+#include "exec/gdbstub.h"
+#include "sysemu/cpus.h"
+#include "sysemu/hvf.h"
+#include "sysemu/hvf_int.h"
+#include "sysemu/runstate.h"
+#include "qemu/guest-random.h"

 HVFState *hvf_state;

+#ifdef __aarch64__
+#define HV_VM_DEFAULT NULL
+#endif
+
 /* Memory slots */

 hvf_slot *hvf_find_overlap_slot(uint64_t start, uint64_t size)
@@ -78,6 +81,127 @@ hvf_slot *hvf_find_overlap_slot(uint64_t start, uint64_t size)
    return NULL;
 }

+struct mac_slot {
+    int present;
+    uint64_t size;
+    uint64_t gpa_start;
+    uint64_t gva;
+};
+
+struct mac_slot mac_slots[32];
+
+static int do_hvf_set_memory(hvf_slot *slot, hv_memory_flags_t flags)
+{
+    struct mac_slot *macslot;
+    hv_return_t ret;
+
+    macslot = &mac_slots[slot->slot_id];
+
+    if (macslot->present) {
+        if (macslot->size != slot->size) {
+            macslot->present = 0;
+            ret = hv_vm_unmap(macslot->gpa_start, macslot->size);
+            assert_hvf_ok(ret);
+        }
+    }
+
+    if (!slot->size) {
+        return 0;
+    }
+
+    macslot->present = 1;
+    macslot->gpa_start = slot->start;
+    macslot->size = slot->size;
+    ret = hv_vm_map(slot->mem, slot->start, slot->size, flags);
+    assert_hvf_ok(ret);
+    return 0;
+}
+
+static void hvf_set_phys_mem(MemoryRegionSection *section, bool add)
+{
+    hvf_slot *mem;
+    MemoryRegion *area = section->mr;
+    bool writable = !area->readonly && !area->rom_device;
+    hv_memory_flags_t flags;
+    uint64_t page_size = qemu_real_host_page_size();
+
+    if (!memory_region_is_ram(area)) {
+        if (writable) {
+            return;
+        } else if (!memory_region_is_romd(area)) {
+            /*
+             * If the memory device is not in romd_mode, then we actually want
+             * to remove the hvf memory slot so all accesses will trap.
+             */
+             add = false;
+        }
+    }
+
+    if (!QEMU_IS_ALIGNED(int128_get64(section->size), page_size) ||
+        !QEMU_IS_ALIGNED(section->offset_within_address_space, page_size)) {
+        /* Not page aligned, so we can not map as RAM */
+        add = false;
+    }
+
+    mem = hvf_find_overlap_slot(
+            section->offset_within_address_space,
+            int128_get64(section->size));
+
+    if (mem && add) {
+        if (mem->size == int128_get64(section->size) &&
+            mem->start == section->offset_within_address_space &&
+            mem->mem == (memory_region_get_ram_ptr(area) +
+            section->offset_within_region)) {
+            return; /* Same region was attempted to register, go away. */
+        }
+    }
+
+    /* Region needs to be reset. set the size to 0 and remap it. */
+    if (mem) {
+        mem->size = 0;
+        if (do_hvf_set_memory(mem, 0)) {
+            error_report("Failed to reset overlapping slot");
+            abort();
+        }
+    }
+
+    if (!add) {
+        return;
+    }
+
+    if (area->readonly ||
+        (!memory_region_is_ram(area) && memory_region_is_romd(area))) {
+        flags = HV_MEMORY_READ | HV_MEMORY_EXEC;
+    } else {
+        flags = HV_MEMORY_READ | HV_MEMORY_WRITE | HV_MEMORY_EXEC;
+    }
+
+    /* Now make a new slot. */
+    int x;
+
+    for (x = 0; x < hvf_state->num_slots; ++x) {
+        mem = &hvf_state->slots[x];
+        if (!mem->size) {
+            break;
+        }
+    }
+
+    if (x == hvf_state->num_slots) {
+        error_report("No free slots");
+        abort();
+    }
+
+    mem->size = int128_get64(section->size);
+    mem->mem = memory_region_get_ram_ptr(area) + section->offset_within_region;
+    mem->start = section->offset_within_address_space;
+    mem->region = area;
+
+    if (do_hvf_set_memory(mem, flags)) {
+        error_report("Error registering new memory slot");
+        abort();
+    }
+}
+
 static void do_hvf_cpu_synchronize_state(CPUState *cpu, run_on_cpu_data arg)
 {
    if (!cpu->vcpu_dirty) {
@@ -115,16 +239,137 @@ static void hvf_cpu_synchronize_pre_loadvm(CPUState *cpu)
    run_on_cpu(cpu, do_hvf_cpu_synchronize_set_dirty, RUN_ON_CPU_NULL);
 }

+static void hvf_set_dirty_tracking(MemoryRegionSection *section, bool on)
+{
+    hvf_slot *slot;
+
+    slot = hvf_find_overlap_slot(
+            section->offset_within_address_space,
+            int128_get64(section->size));
+
+    /* protect region against writes; begin tracking it */
+    if (on) {
+        slot->flags |= HVF_SLOT_LOG;
+        hv_vm_protect((uintptr_t)slot->start, (size_t)slot->size,
+                      HV_MEMORY_READ | HV_MEMORY_EXEC);
+    /* stop tracking region*/
+    } else {
+        slot->flags &= ~HVF_SLOT_LOG;
+        hv_vm_protect((uintptr_t)slot->start, (size_t)slot->size,
+                      HV_MEMORY_READ | HV_MEMORY_WRITE | HV_MEMORY_EXEC);
+    }
+}
+
+static void hvf_log_start(MemoryListener *listener,
+                          MemoryRegionSection *section, int old, int new)
+{
+    if (old != 0) {
+        return;
+    }
+
+    hvf_set_dirty_tracking(section, 1);
+}
+
+static void hvf_log_stop(MemoryListener *listener,
+                         MemoryRegionSection *section, int old, int new)
+{
+    if (new != 0) {
+        return;
+    }
+
+    hvf_set_dirty_tracking(section, 0);
+}
+
+static void hvf_log_sync(MemoryListener *listener,
+                         MemoryRegionSection *section)
+{
+    /*
+     * sync of dirty pages is handled elsewhere; just make sure we keep
+     * tracking the region.
+     */
+    hvf_set_dirty_tracking(section, 1);
+}
+
+static void hvf_region_add(MemoryListener *listener,
+                           MemoryRegionSection *section)
+{
+    hvf_set_phys_mem(section, true);
+}
+
+static void hvf_region_del(MemoryListener *listener,
+                           MemoryRegionSection *section)
+{
+    hvf_set_phys_mem(section, false);
+}
+
+static MemoryListener hvf_memory_listener = {
+    .name = "hvf",
+    .priority = MEMORY_LISTENER_PRIORITY_ACCEL,
+    .region_add = hvf_region_add,
+    .region_del = hvf_region_del,
+    .log_start = hvf_log_start,
+    .log_stop = hvf_log_stop,
+    .log_sync = hvf_log_sync,
+};
+
 static void dummy_signal(int sig)
 {
 }

-static void do_hvf_get_vcpu_exec_time(CPUState *cpu, run_on_cpu_data arg)
+bool hvf_allowed;
+
+static int hvf_accel_init(MachineState *ms)
 {
-    int r = hv_vcpu_get_exec_time(cpu->accel->fd, arg.host_ptr);
-    assert_hvf_ok(r);
+    int x;
+    hv_return_t ret;
+    HVFState *s;
+
+    ret = hv_vm_create(HV_VM_DEFAULT);
+    assert_hvf_ok(ret);
+
+    s = g_new0(HVFState, 1);
+
+    s->num_slots = ARRAY_SIZE(s->slots);
+    for (x = 0; x < s->num_slots; ++x) {
+        s->slots[x].size = 0;
+        s->slots[x].slot_id = x;
+    }
+
+    QTAILQ_INIT(&s->hvf_sw_breakpoints);
+
+    hvf_state = s;
+    memory_listener_register(&hvf_memory_listener, &address_space_memory);
+
+    return hvf_arch_init();
 }

+static inline int hvf_gdbstub_sstep_flags(void)
+{
+    return SSTEP_ENABLE | SSTEP_NOIRQ;
+}
+
+static void hvf_accel_class_init(ObjectClass *oc, void *data)
+{
+    AccelClass *ac = ACCEL_CLASS(oc);
+    ac->name = "HVF";
+    ac->init_machine = hvf_accel_init;
+    ac->allowed = &hvf_allowed;
+    ac->gdbstub_supported_sstep_flags = hvf_gdbstub_sstep_flags;
+}
+
+static const TypeInfo hvf_accel_type = {
+    .name = TYPE_HVF_ACCEL,
+    .parent = TYPE_ACCEL,
+    .class_init = hvf_accel_class_init,
+};
+
+static void hvf_type_init(void)
+{
+    type_register_static(&hvf_accel_type);
+}
+
+type_init(hvf_type_init);
+
 static void hvf_vcpu_destroy(CPUState *cpu)
 {
    hv_return_t ret = hv_vcpu_destroy(cpu->accel->fd);
@@ -155,10 +400,10 @@ static int hvf_init_vcpu(CPUState *cpu)
    r = hv_vcpu_create(&cpu->accel->fd,
                       (hv_vcpu_exit_t **)&cpu->accel->exit, NULL);
 #else
-    r = hv_vcpu_create(&cpu->accel->fd, HV_VCPU_DEFAULT);
+    r = hv_vcpu_create((hv_vcpuid_t *)&cpu->accel->fd, HV_VCPU_DEFAULT);
 #endif
+    cpu->vcpu_dirty = 1;
    assert_hvf_ok(r);
-    cpu->vcpu_dirty = true;

    cpu->accel->guest_debug_enabled = false;

@@ -218,40 +463,16 @@ static void hvf_start_vcpu_thread(CPUState *cpu)
     */
    assert(hvf_enabled());

+    cpu->thread = g_malloc0(sizeof(QemuThread));
+    cpu->halt_cond = g_malloc0(sizeof(QemuCond));
+    qemu_cond_init(cpu->halt_cond);
+
    snprintf(thread_name, VCPU_THREAD_NAME_SIZE, "CPU %d/HVF",
             cpu->cpu_index);
    qemu_thread_create(cpu->thread, thread_name, hvf_cpu_thread_fn,
                       cpu, QEMU_THREAD_JOINABLE);
 }

-struct hvf_sw_breakpoint *hvf_find_sw_breakpoint(CPUState *cpu, vaddr pc)
-{
-    struct hvf_sw_breakpoint *bp;
-
-    QTAILQ_FOREACH(bp, &hvf_state->hvf_sw_breakpoints, entry) {
-        if (bp->pc == pc) {
-            return bp;
-        }
-    }
-    return NULL;
-}
-
-int hvf_sw_breakpoints_active(CPUState *cpu)
-{
-    return !QTAILQ_EMPTY(&hvf_state->hvf_sw_breakpoints);
-}
-
-static void do_hvf_update_guest_debug(CPUState *cpu, run_on_cpu_data arg)
-{
-    hvf_arch_update_guest_debug(cpu);
-}
-
-int hvf_update_guest_debug(CPUState *cpu)
-{
-    run_on_cpu(cpu, do_hvf_update_guest_debug, RUN_ON_CPU_NULL);
-    return 0;
-}
-
 static int hvf_insert_breakpoint(CPUState *cpu, int type, vaddr addr, vaddr len)
 {
    struct hvf_sw_breakpoint *bp;
@@ -354,28 +575,12 @@ static void hvf_remove_all_breakpoints(CPUState *cpu)
    }
 }

-static void hvf_get_vcpu_stats(CPUState *cpu, GString *buf)
-{
-    uint64_t time_mach; /* units of mach_absolute_time() */
-
-    run_on_cpu(cpu, do_hvf_get_vcpu_exec_time, RUN_ON_CPU_HOST_PTR(&time_mach));
-
-    mach_timebase_info_data_t timebase;
-    mach_timebase_info(&timebase);
-    uint64_t time_ns = time_mach * timebase.numer / timebase.denom;
-
-    g_string_append_printf(buf, "HVF cumulative execution time: %llu.%.3llus\n",
-                                 time_ns / 1000000000,
-                                (time_ns % 1000000000) / 1000000);
-}
-
-static void hvf_accel_ops_class_init(ObjectClass *oc, const void *data)
+static void hvf_accel_ops_class_init(ObjectClass *oc, void *data)
 {
    AccelOpsClass *ops = ACCEL_OPS_CLASS(oc);

    ops->create_vcpu_thread = hvf_start_vcpu_thread;
    ops->kick_vcpu_thread = hvf_kick_vcpu_thread;
-    ops->handle_interrupt = generic_handle_interrupt;

    ops->synchronize_post_reset = hvf_cpu_synchronize_post_reset;
    ops->synchronize_post_init = hvf_cpu_synchronize_post_init;
@@ -387,10 +592,7 @@ static void hvf_accel_ops_class_init(ObjectClass *oc, const void *data)
    ops->remove_all_breakpoints = hvf_remove_all_breakpoints;
    ops->update_guest_debug = hvf_update_guest_debug;
    ops->supports_guest_debug = hvf_arch_supports_guest_debug;
-
-    ops->get_vcpu_stats = hvf_get_vcpu_stats;
 };
-
 static const TypeInfo hvf_accel_ops_type = {
    .name = ACCEL_OPS_NAME("hvf"),

@@ -398,10 +600,8 @@ static const TypeInfo hvf_accel_ops_type = {
    .class_init = hvf_accel_ops_class_init,
    .abstract = true,
 };
-
 static void hvf_accel_ops_register_types(void)
 {
    type_register_static(&hvf_accel_ops_type);
 }
-
 type_init(hvf_accel_ops_register_types);
@@ -10,305 +10,66 @@

 #include "qemu/osdep.h"
 #include "qemu/error-report.h"
-#include "accel/accel-ops.h"
-#include "system/address-spaces.h"
-#include "system/memory.h"
-#include "system/hvf.h"
-#include "system/hvf_int.h"
-#include "hw/core/cpu.h"
-#include "hw/boards.h"
-#include "trace.h"
+#include "sysemu/hvf.h"
+#include "sysemu/hvf_int.h"

-bool hvf_allowed;
-
-struct mac_slot {
-    int present;
-    uint64_t size;
-    uint64_t gpa_start;
-    uint64_t gva;
-};
-
-struct mac_slot mac_slots[32];
-
-const char *hvf_return_string(hv_return_t ret)
-{
-    switch (ret) {
-    case HV_SUCCESS:      return "HV_SUCCESS";
-    case HV_ERROR:        return "HV_ERROR";
-    case HV_BUSY:         return "HV_BUSY";
-    case HV_BAD_ARGUMENT: return "HV_BAD_ARGUMENT";
-    case HV_NO_RESOURCES: return "HV_NO_RESOURCES";
-    case HV_NO_DEVICE:    return "HV_NO_DEVICE";
-    case HV_UNSUPPORTED:  return "HV_UNSUPPORTED";
-    case HV_DENIED:       return "HV_DENIED";
-    default:              return "[unknown hv_return value]";
-    }
-}
-
-void assert_hvf_ok_impl(hv_return_t ret, const char *file, unsigned int line,
-                        const char *exp)
+void assert_hvf_ok(hv_return_t ret)
 {
    if (ret == HV_SUCCESS) {
        return;
    }

-    error_report("Error: %s = %s (0x%x, at %s:%u)",
-        exp, hvf_return_string(ret), ret, file, line);
+    switch (ret) {
+    case HV_ERROR:
+        error_report("Error: HV_ERROR");
+        break;
+    case HV_BUSY:
+        error_report("Error: HV_BUSY");
+        break;
+    case HV_BAD_ARGUMENT:
+        error_report("Error: HV_BAD_ARGUMENT");
+        break;
+    case HV_NO_RESOURCES:
+        error_report("Error: HV_NO_RESOURCES");
+        break;
+    case HV_NO_DEVICE:
+        error_report("Error: HV_NO_DEVICE");
+        break;
+    case HV_UNSUPPORTED:
+        error_report("Error: HV_UNSUPPORTED");
+        break;
+#if defined(MAC_OS_VERSION_11_0) && \
+    MAC_OS_X_VERSION_MIN_REQUIRED >= MAC_OS_VERSION_11_0
+    case HV_DENIED:
+        error_report("Error: HV_DENIED");
+        break;
+#endif
+    default:
+        error_report("Unknown Error");
+    }

    abort();
 }

-static int do_hvf_set_memory(hvf_slot *slot, hv_memory_flags_t flags)
+struct hvf_sw_breakpoint *hvf_find_sw_breakpoint(CPUState *cpu, vaddr pc)
 {
-    struct mac_slot *macslot;
-    hv_return_t ret;
+    struct hvf_sw_breakpoint *bp;

-    macslot = &mac_slots[slot->slot_id];
-
-    if (macslot->present) {
-        if (macslot->size != slot->size) {
-            macslot->present = 0;
-            trace_hvf_vm_unmap(macslot->gpa_start, macslot->size);
-            ret = hv_vm_unmap(macslot->gpa_start, macslot->size);
-            assert_hvf_ok(ret);
+    QTAILQ_FOREACH(bp, &hvf_state->hvf_sw_breakpoints, entry) {
+        if (bp->pc == pc) {
+            return bp;
        }
    }
+    return NULL;
+}

-    if (!slot->size) {
-        return 0;
-    }
+int hvf_sw_breakpoints_active(CPUState *cpu)
+{
+    return !QTAILQ_EMPTY(&hvf_state->hvf_sw_breakpoints);
+}

-    macslot->present = 1;
-    macslot->gpa_start = slot->start;
-    macslot->size = slot->size;
-    trace_hvf_vm_map(slot->start, slot->size, slot->mem, flags,
-                     flags & HV_MEMORY_READ ?  'R' : '-',
-                     flags & HV_MEMORY_WRITE ? 'W' : '-',
-                     flags & HV_MEMORY_EXEC ?  'X' : '-');
-    ret = hv_vm_map(slot->mem, slot->start, slot->size, flags);
-    assert_hvf_ok(ret);
+int hvf_update_guest_debug(CPUState *cpu)
+{
+    hvf_arch_update_guest_debug(cpu);
    return 0;
 }
-
-static void hvf_set_phys_mem(MemoryRegionSection *section, bool add)
-{
-    hvf_slot *mem;
-    MemoryRegion *area = section->mr;
-    bool writable = !area->readonly && !area->rom_device;
-    hv_memory_flags_t flags;
-    uint64_t page_size = qemu_real_host_page_size();
-
-    if (!memory_region_is_ram(area)) {
-        if (writable) {
-            return;
-        } else if (!memory_region_is_romd(area)) {
-            /*
-             * If the memory device is not in romd_mode, then we actually want
-             * to remove the hvf memory slot so all accesses will trap.
-             */
-             add = false;
-        }
-    }
-
-    if (!QEMU_IS_ALIGNED(int128_get64(section->size), page_size) ||
-        !QEMU_IS_ALIGNED(section->offset_within_address_space, page_size)) {
-        /* Not page aligned, so we can not map as RAM */
-        add = false;
-    }
-
-    mem = hvf_find_overlap_slot(
-            section->offset_within_address_space,
-            int128_get64(section->size));
-
-    if (mem && add) {
-        if (mem->size == int128_get64(section->size) &&
-            mem->start == section->offset_within_address_space &&
-            mem->mem == (memory_region_get_ram_ptr(area) +
-            section->offset_within_region)) {
-            return; /* Same region was attempted to register, go away. */
-        }
-    }
-
-    /* Region needs to be reset. set the size to 0 and remap it. */
-    if (mem) {
-        mem->size = 0;
-        if (do_hvf_set_memory(mem, 0)) {
-            error_report("Failed to reset overlapping slot");
-            abort();
-        }
-    }
-
-    if (!add) {
-        return;
-    }
-
-    if (area->readonly ||
-        (!memory_region_is_ram(area) && memory_region_is_romd(area))) {
-        flags = HV_MEMORY_READ | HV_MEMORY_EXEC;
-    } else {
-        flags = HV_MEMORY_READ | HV_MEMORY_WRITE | HV_MEMORY_EXEC;
-    }
-
-    /* Now make a new slot. */
-    int x;
-
-    for (x = 0; x < hvf_state->num_slots; ++x) {
-        mem = &hvf_state->slots[x];
-        if (!mem->size) {
-            break;
-        }
-    }
-
-    if (x == hvf_state->num_slots) {
-        error_report("No free slots");
-        abort();
-    }
-
-    mem->size = int128_get64(section->size);
-    mem->mem = memory_region_get_ram_ptr(area) + section->offset_within_region;
-    mem->start = section->offset_within_address_space;
-    mem->region = area;
-
-    if (do_hvf_set_memory(mem, flags)) {
-        error_report("Error registering new memory slot");
-        abort();
-    }
-}
-
-static void hvf_set_dirty_tracking(MemoryRegionSection *section, bool on)
-{
-    hvf_slot *slot;
-
-    slot = hvf_find_overlap_slot(
-            section->offset_within_address_space,
-            int128_get64(section->size));
-
-    /* protect region against writes; begin tracking it */
-    if (on) {
-        slot->flags |= HVF_SLOT_LOG;
-        hv_vm_protect((uintptr_t)slot->start, (size_t)slot->size,
-                      HV_MEMORY_READ | HV_MEMORY_EXEC);
-    /* stop tracking region*/
-    } else {
-        slot->flags &= ~HVF_SLOT_LOG;
-        hv_vm_protect((uintptr_t)slot->start, (size_t)slot->size,
-                      HV_MEMORY_READ | HV_MEMORY_WRITE | HV_MEMORY_EXEC);
-    }
-}
-
-static void hvf_log_start(MemoryListener *listener,
-                          MemoryRegionSection *section, int old, int new)
-{
-    if (old != 0) {
-        return;
-    }
-
-    hvf_set_dirty_tracking(section, 1);
-}
-
-static void hvf_log_stop(MemoryListener *listener,
-                         MemoryRegionSection *section, int old, int new)
-{
-    if (new != 0) {
-        return;
-    }
-
-    hvf_set_dirty_tracking(section, 0);
-}
-
-static void hvf_log_sync(MemoryListener *listener,
-                         MemoryRegionSection *section)
-{
-    /*
-     * sync of dirty pages is handled elsewhere; just make sure we keep
-     * tracking the region.
-     */
-    hvf_set_dirty_tracking(section, 1);
-}
-
-static void hvf_region_add(MemoryListener *listener,
-                           MemoryRegionSection *section)
-{
-    hvf_set_phys_mem(section, true);
-}
-
-static void hvf_region_del(MemoryListener *listener,
-                           MemoryRegionSection *section)
-{
-    hvf_set_phys_mem(section, false);
-}
-
-static MemoryListener hvf_memory_listener = {
-    .name = "hvf",
-    .priority = MEMORY_LISTENER_PRIORITY_ACCEL,
-    .region_add = hvf_region_add,
-    .region_del = hvf_region_del,
-    .log_start = hvf_log_start,
-    .log_stop = hvf_log_stop,
-    .log_sync = hvf_log_sync,
-};
-
-static int hvf_accel_init(AccelState *as, MachineState *ms)
-{
-    int x;
-    hv_return_t ret;
-    HVFState *s = HVF_STATE(as);
-    int pa_range = 36;
-    MachineClass *mc = MACHINE_GET_CLASS(ms);
-
-    if (mc->hvf_get_physical_address_range) {
-        pa_range = mc->hvf_get_physical_address_range(ms);
-        if (pa_range < 0) {
-            return -EINVAL;
-        }
-    }
-
-    ret = hvf_arch_vm_create(ms, (uint32_t)pa_range);
-    if (ret == HV_DENIED) {
-        error_report("Could not access HVF. Is the executable signed"
-                     " with com.apple.security.hypervisor entitlement?");
-        exit(1);
-    }
-    assert_hvf_ok(ret);
-
-    s->num_slots = ARRAY_SIZE(s->slots);
-    for (x = 0; x < s->num_slots; ++x) {
-        s->slots[x].size = 0;
-        s->slots[x].slot_id = x;
-    }
-
-    QTAILQ_INIT(&s->hvf_sw_breakpoints);
-
-    hvf_state = s;
-    memory_listener_register(&hvf_memory_listener, &address_space_memory);
-
-    return hvf_arch_init();
-}
-
-static int hvf_gdbstub_sstep_flags(AccelState *as)
-{
-    return SSTEP_ENABLE | SSTEP_NOIRQ;
-}
-
-static void hvf_accel_class_init(ObjectClass *oc, const void *data)
-{
-    AccelClass *ac = ACCEL_CLASS(oc);
-    ac->name = "HVF";
-    ac->init_machine = hvf_accel_init;
-    ac->allowed = &hvf_allowed;
-    ac->gdbstub_supported_sstep_flags = hvf_gdbstub_sstep_flags;
-}
-
-static const TypeInfo hvf_accel_type = {
-    .name = TYPE_HVF_ACCEL,
-    .parent = TYPE_ACCEL,
-    .instance_size = sizeof(HVFState),
-    .class_init = hvf_accel_class_init,
-};
-
-static void hvf_type_init(void)
-{
-    type_register_static(&hvf_accel_type);
-}
-
-type_init(hvf_type_init);
@@ -1,7 +0,0 @@
-# SPDX-License-Identifier: GPL-2.0-or-later
-#
-# See docs/devel/tracing.rst for syntax documentation.
-
-# hvf-accel-ops.c
-hvf_vm_map(uint64_t paddr, uint64_t size, void *vaddr, uint8_t flags, const char r, const char w, const char e) "paddr:0x%016"PRIx64" size:0x%08"PRIx64" vaddr:%p flags:0x%02x/%c%c%c"
-hvf_vm_unmap(uint64_t paddr, uint64_t size) "paddr:0x%016"PRIx64" size:0x%08"PRIx64
@@ -1,2 +0,0 @@
-/* SPDX-License-Identifier: GPL-2.0-or-later */
-#include "trace/trace-accel_hvf.h"
@@ -16,11 +16,10 @@
 #include "qemu/osdep.h"
 #include "qemu/error-report.h"
 #include "qemu/main-loop.h"
-#include "accel/accel-cpu-ops.h"
-#include "system/kvm.h"
-#include "system/kvm_int.h"
-#include "system/runstate.h"
-#include "system/cpus.h"
+#include "sysemu/kvm.h"
+#include "sysemu/kvm_int.h"
+#include "sysemu/runstate.h"
+#include "sysemu/cpus.h"
 #include "qemu/guest-random.h"
 #include "qapi/error.h"

@@ -67,6 +66,9 @@ static void kvm_start_vcpu_thread(CPUState *cpu)
 {
    char thread_name[VCPU_THREAD_NAME_SIZE];

+    cpu->thread = g_malloc0(sizeof(QemuThread));
+    cpu->halt_cond = g_malloc0(sizeof(QemuCond));
+    qemu_cond_init(cpu->halt_cond);
    snprintf(thread_name, VCPU_THREAD_NAME_SIZE, "CPU %d/KVM",
             cpu->cpu_index);
    qemu_thread_create(cpu->thread, thread_name, kvm_vcpu_thread_fn,
@@ -80,17 +82,17 @@ static bool kvm_vcpu_thread_is_idle(CPUState *cpu)

 static bool kvm_cpus_are_resettable(void)
 {
-    return !kvm_enabled() || !kvm_state->guest_state_protected;
+    return !kvm_enabled() || kvm_cpu_check_are_resettable();
 }

-#ifdef TARGET_KVM_HAVE_GUEST_DEBUG
+#ifdef KVM_CAP_SET_GUEST_DEBUG
 static int kvm_update_guest_debug_ops(CPUState *cpu)
 {
    return kvm_update_guest_debug(cpu, 0);
 }
 #endif

-static void kvm_accel_ops_class_init(ObjectClass *oc, const void *data)
+static void kvm_accel_ops_class_init(ObjectClass *oc, void *data)
 {
    AccelOpsClass *ops = ACCEL_OPS_CLASS(oc);

@@ -101,9 +103,8 @@ static void kvm_accel_ops_class_init(ObjectClass *oc, const void *data)
    ops->synchronize_post_init = kvm_cpu_synchronize_post_init;
    ops->synchronize_state = kvm_cpu_synchronize_state;
    ops->synchronize_pre_loadvm = kvm_cpu_synchronize_pre_loadvm;
-    ops->handle_interrupt = generic_handle_interrupt;

-#ifdef TARGET_KVM_HAVE_GUEST_DEBUG
+#ifdef KVM_CAP_SET_GUEST_DEBUG
    ops->update_guest_debug = kvm_update_guest_debug_ops;
    ops->supports_guest_debug = kvm_supports_guest_debug;
    ops->insert_breakpoint = kvm_insert_breakpoint;
@@ -10,6 +10,8 @@
 #ifndef KVM_CPUS_H
 #define KVM_CPUS_H

+#include "sysemu/cpus.h"
+
 int kvm_init_vcpu(CPUState *cpu, Error **errp);
 int kvm_cpu_exec(CPUState *cpu);
 void kvm_destroy_vcpu(CPUState *cpu);
@@ -20,4 +22,5 @@ bool kvm_supports_guest_debug(void);
 int kvm_insert_breakpoint(CPUState *cpu, int type, vaddr addr, vaddr len);
 int kvm_remove_breakpoint(CPUState *cpu, int type, vaddr addr, vaddr len);
 void kvm_remove_all_breakpoints(CPUState *cpu);
+
 #endif /* KVM_CPUS_H */
@@ -1,25 +1,21 @@
 # See docs/devel/tracing.rst for syntax documentation.

 # kvm-all.c
-kvm_ioctl(unsigned long type, void *arg) "type 0x%lx, arg %p"
-kvm_vm_ioctl(unsigned long type, void *arg) "type 0x%lx, arg %p"
-kvm_vcpu_ioctl(int cpu_index, unsigned long type, void *arg) "cpu_index %d, type 0x%lx, arg %p"
+kvm_ioctl(int type, void *arg) "type 0x%x, arg %p"
+kvm_vm_ioctl(int type, void *arg) "type 0x%x, arg %p"
+kvm_vcpu_ioctl(int cpu_index, int type, void *arg) "cpu_index %d, type 0x%x, arg %p"
 kvm_run_exit(int cpu_index, uint32_t reason) "cpu_index %d, reason %d"
-kvm_device_ioctl(int fd, unsigned long type, void *arg) "dev fd %d, type 0x%lx, arg %p"
+kvm_device_ioctl(int fd, int type, void *arg) "dev fd %d, type 0x%x, arg %p"
 kvm_failed_reg_get(uint64_t id, const char *msg) "Warning: Unable to retrieve ONEREG %" PRIu64 " from KVM: %s"
 kvm_failed_reg_set(uint64_t id, const char *msg) "Warning: Unable to set ONEREG %" PRIu64 " to KVM: %s"
 kvm_init_vcpu(int cpu_index, unsigned long arch_cpu_id) "index: %d id: %lu"
-kvm_create_vcpu(int cpu_index, unsigned long arch_cpu_id, int kvm_fd) "index: %d, id: %lu, kvm fd: %d"
-kvm_destroy_vcpu(int cpu_index, unsigned long arch_cpu_id) "index: %d id: %lu"
-kvm_park_vcpu(int cpu_index, unsigned long arch_cpu_id) "index: %d id: %lu"
-kvm_unpark_vcpu(unsigned long arch_cpu_id, const char *msg) "id: %lu %s"
 kvm_irqchip_commit_routes(void) ""
 kvm_irqchip_add_msi_route(char *name, int vector, int virq) "dev %s vector %d virq %d"
 kvm_irqchip_update_msi_route(int virq) "Updating MSI route virq=%d"
 kvm_irqchip_release_virq(int virq) "virq %d"
 kvm_set_ioeventfd_mmio(int fd, uint64_t addr, uint32_t val, bool assign, uint32_t size, bool datamatch) "fd: %d @0x%" PRIx64 " val=0x%x assign: %d size: %d match: %d"
 kvm_set_ioeventfd_pio(int fd, uint16_t addr, uint32_t val, bool assign, uint32_t size, bool datamatch) "fd: %d @0x%x val=0x%x assign: %d size: %d match: %d"
-kvm_set_user_memory(uint16_t as, uint16_t slot, uint32_t flags, uint64_t guest_phys_addr, uint64_t memory_size, uint64_t userspace_addr, uint32_t fd, uint64_t fd_offset, int ret) "AddrSpace#%d Slot#%d flags=0x%x gpa=0x%"PRIx64 " size=0x%"PRIx64 " ua=0x%"PRIx64 " guest_memfd=%d" " guest_memfd_offset=0x%" PRIx64 " ret=%d"
+kvm_set_user_memory(uint32_t slot, uint32_t flags, uint64_t guest_phys_addr, uint64_t memory_size, uint64_t userspace_addr, int ret) "Slot#%d flags=0x%x gpa=0x%"PRIx64 " size=0x%"PRIx64 " ua=0x%"PRIx64 " ret=%d"
 kvm_clear_dirty_log(uint32_t slot, uint64_t start, uint32_t size) "slot#%"PRId32" start 0x%"PRIx64" size 0x%"PRIx32
 kvm_resample_fd_notify(int gsi) "gsi %d"
 kvm_dirty_ring_full(int id) "vcpu %d"
@@ -29,11 +25,9 @@ kvm_dirty_ring_reaper(const char *s) "%s"
 kvm_dirty_ring_reap(uint64_t count, int64_t t) "reaped %"PRIu64" pages (took %"PRIi64" us)"
 kvm_dirty_ring_reaper_kick(const char *reason) "%s"
 kvm_dirty_ring_flush(int finished) "%d"
+kvm_destroy_vcpu(void) ""
 kvm_failed_get_vcpu_mmap_size(void) ""
 kvm_cpu_exec(void) ""
 kvm_interrupt_exit_request(void) ""
 kvm_io_window_exit(void) ""
 kvm_run_exit_system_event(int cpu_index, uint32_t event_type) "cpu_index %d, system_even_type %"PRIu32
-kvm_convert_memory(uint64_t start, uint64_t size, const char *msg) "start 0x%" PRIx64 " size 0x%" PRIx64 " %s"
-kvm_memory_fault(uint64_t start, uint64_t size, uint64_t flags) "start 0x%" PRIx64 " size 0x%" PRIx64 " flags 0x%" PRIx64
-kvm_slots_grow(unsigned int old, unsigned int new) "%u -> %u"
@@ -1,6 +1,5 @@
-common_ss.add(files('accel-common.c'))
 specific_ss.add(files('accel-target.c'))
-system_ss.add(files('accel-system.c', 'accel-blocker.c', 'accel-qmp.c'))
+system_ss.add(files('accel-system.c', 'accel-blocker.c'))
 user_ss.add(files('accel-user.c'))

 subdir('tcg')
@@ -18,33 +18,18 @@
 #include "qemu/option.h"
 #include "qemu/config-file.h"
 #include "qemu/accel.h"
-#include "accel/accel-ops.h"
-#include "accel/accel-cpu-ops.h"
-#include "system/qtest.h"
-#include "system/cpus.h"
+#include "sysemu/qtest.h"
+#include "sysemu/cpus.h"
 #include "qemu/guest-random.h"
 #include "qemu/main-loop.h"
 #include "hw/core/cpu.h"
-#include "accel/dummy-cpus.h"

-static int64_t qtest_clock_counter;
-
-static int64_t qtest_get_virtual_clock(void)
-{
-    return qatomic_read_i64(&qtest_clock_counter);
-}
-
-static void qtest_set_virtual_clock(int64_t count)
-{
-    qatomic_set_i64(&qtest_clock_counter, count);
-}
-
-static int qtest_init_accel(AccelState *as, MachineState *ms)
+static int qtest_init_accel(MachineState *ms)
 {
    return 0;
 }

-static void qtest_accel_class_init(ObjectClass *oc, const void *data)
+static void qtest_accel_class_init(ObjectClass *oc, void *data)
 {
    AccelClass *ac = ACCEL_CLASS(oc);
    ac->name = "QTest";
@@ -61,14 +46,12 @@ static const TypeInfo qtest_accel_type = {
 };
 module_obj(TYPE_QTEST_ACCEL);

-static void qtest_accel_ops_class_init(ObjectClass *oc, const void *data)
+static void qtest_accel_ops_class_init(ObjectClass *oc, void *data)
 {
    AccelOpsClass *ops = ACCEL_OPS_CLASS(oc);

    ops->create_vcpu_thread = dummy_start_vcpu_thread;
    ops->get_virtual_clock = qtest_get_virtual_clock;
-    ops->set_virtual_clock = qtest_set_virtual_clock;
-    ops->handle_interrupt = generic_handle_interrupt;
 };

 static const TypeInfo qtest_accel_ops_type = {
@@ -1,12 +0,0 @@
-/*
- * HVF stubs for QEMU
- *
- *  Copyright (c) Linaro
- *
- * SPDX-License-Identifier: GPL-2.0-or-later
- */
-
-#include "qemu/osdep.h"
-#include "system/hvf.h"
-
-bool hvf_allowed;
@@ -11,7 +11,7 @@
 */

 #include "qemu/osdep.h"
-#include "system/kvm.h"
+#include "sysemu/kvm.h"
 #include "hw/pci/msi.h"

 KVMState *kvm_state;
@@ -29,6 +29,10 @@ void kvm_flush_coalesced_mmio_buffer(void)
 {
 }

+void kvm_cpu_synchronize_state(CPUState *cpu)
+{
+}
+
 bool kvm_has_sync_mmu(void)
 {
    return false;
@@ -101,6 +105,11 @@ unsigned int kvm_get_free_memslots(void)
    return 0;
 }

+void kvm_init_cpu_signals(CPUState *cpu)
+{
+    abort();
+}
+
 bool kvm_arm_supports_user_irq(void)
 {
    return false;
@@ -120,8 +129,3 @@ bool kvm_hwpoisoned_mem(void)
 {
    return false;
 }
-
-int kvm_create_guest_memfd(uint64_t size, uint64_t flags, Error **errp)
-{
-    return -ENOSYS;
-}
@@ -2,8 +2,5 @@ system_stubs_ss = ss.source_set()
 system_stubs_ss.add(when: 'CONFIG_XEN', if_false: files('xen-stub.c'))
 system_stubs_ss.add(when: 'CONFIG_KVM', if_false: files('kvm-stub.c'))
 system_stubs_ss.add(when: 'CONFIG_TCG', if_false: files('tcg-stub.c'))
-system_stubs_ss.add(when: 'CONFIG_HVF', if_false: files('hvf-stub.c'))
-system_stubs_ss.add(when: 'CONFIG_NVMM', if_false: files('nvmm-stub.c'))
-system_stubs_ss.add(when: 'CONFIG_WHPX', if_false: files('whpx-stub.c'))

 specific_ss.add_all(when: ['CONFIG_SYSTEM_ONLY'], if_true: system_stubs_ss)
@@ -1,12 +0,0 @@
-/*
- * NVMM stubs for QEMU
- *
- *  Copyright (c) Linaro
- *
- * SPDX-License-Identifier: GPL-2.0-or-later
- */
-
-#include "qemu/osdep.h"
-#include "system/nvmm.h"
-
-bool nvmm_allowed;
@@ -11,7 +11,30 @@
 */

 #include "qemu/osdep.h"
-#include "exec/cpu-common.h"
+#include "exec/tb-flush.h"
+#include "exec/exec-all.h"
+
+void tb_flush(CPUState *cpu)
+{
+}
+
+void tlb_set_dirty(CPUState *cpu, vaddr vaddr)
+{
+}
+
+int probe_access_flags(CPUArchState *env, vaddr addr, int size,
+                       MMUAccessType access_type, int mmu_idx,
+                       bool nonfault, void **phost, uintptr_t retaddr)
+{
+     g_assert_not_reached();
+}
+
+void *probe_access(CPUArchState *env, vaddr addr, int size,
+                   MMUAccessType access_type, int mmu_idx, uintptr_t retaddr)
+{
+     /* Handled by hardware accelerator. */
+     g_assert_not_reached();
+}

 G_NORETURN void cpu_loop_exit(CPUState *cpu)
 {
@@ -1,12 +0,0 @@
-/*
- * WHPX stubs for QEMU
- *
- *  Copyright (c) Linaro
- *
- * SPDX-License-Identifier: GPL-2.0-or-later
- */
-
-#include "qemu/osdep.h"
-#include "system/whpx.h"
-
-bool whpx_allowed;
@@ -6,7 +6,7 @@
 */

 #include "qemu/osdep.h"
-#include "system/xen.h"
+#include "sysemu/xen.h"
 #include "qapi/qapi-commands-migration.h"

 bool xen_allowed;
@@ -14,20 +14,9 @@
 */

 static void atomic_trace_rmw_post(CPUArchState *env, uint64_t addr,
-                                  uint64_t read_value_low,
-                                  uint64_t read_value_high,
-                                  uint64_t write_value_low,
-                                  uint64_t write_value_high,
                                  MemOpIdx oi)
 {
-    if (cpu_plugin_mem_cbs_enabled(env_cpu(env))) {
-        qemu_plugin_vcpu_mem_cb(env_cpu(env), addr,
-                                read_value_low, read_value_high,
-                                oi, QEMU_PLUGIN_MEM_R);
-        qemu_plugin_vcpu_mem_cb(env_cpu(env), addr,
-                                write_value_low, write_value_high,
-                                oi, QEMU_PLUGIN_MEM_W);
-    }
+    qemu_plugin_vcpu_mem_cb(env_cpu(env), addr, oi, QEMU_PLUGIN_MEM_RW);
 }

 /*
@@ -53,14 +53,6 @@
 # error unsupported data size
 #endif

-#if DATA_SIZE == 16
-# define VALUE_LOW(val) int128_getlo(val)
-# define VALUE_HIGH(val) int128_gethi(val)
-#else
-# define VALUE_LOW(val) val
-# define VALUE_HIGH(val) 0
-#endif
-
 #if DATA_SIZE >= 4
 # define ABI_TYPE  DATA_TYPE
 #else
@@ -77,7 +69,7 @@
 # define END  _le
 #endif

-ABI_TYPE ATOMIC_NAME(cmpxchg)(CPUArchState *env, vaddr addr,
+ABI_TYPE ATOMIC_NAME(cmpxchg)(CPUArchState *env, abi_ptr addr,
                              ABI_TYPE cmpv, ABI_TYPE newv,
                              MemOpIdx oi, uintptr_t retaddr)
 {
@@ -91,17 +83,12 @@ ABI_TYPE ATOMIC_NAME(cmpxchg)(CPUArchState *env, vaddr addr,
    ret = qatomic_cmpxchg__nocheck(haddr, cmpv, newv);
 #endif
    ATOMIC_MMU_CLEANUP;
-    atomic_trace_rmw_post(env, addr,
-                          VALUE_LOW(ret),
-                          VALUE_HIGH(ret),
-                          VALUE_LOW(newv),
-                          VALUE_HIGH(newv),
-                          oi);
+    atomic_trace_rmw_post(env, addr, oi);
    return ret;
 }

 #if DATA_SIZE < 16
-ABI_TYPE ATOMIC_NAME(xchg)(CPUArchState *env, vaddr addr, ABI_TYPE val,
+ABI_TYPE ATOMIC_NAME(xchg)(CPUArchState *env, abi_ptr addr, ABI_TYPE val,
                           MemOpIdx oi, uintptr_t retaddr)
 {
    DATA_TYPE *haddr = atomic_mmu_lookup(env_cpu(env), addr, oi,
@@ -110,29 +97,19 @@ ABI_TYPE ATOMIC_NAME(xchg)(CPUArchState *env, vaddr addr, ABI_TYPE val,

    ret = qatomic_xchg__nocheck(haddr, val);
    ATOMIC_MMU_CLEANUP;
-    atomic_trace_rmw_post(env, addr,
-                          VALUE_LOW(ret),
-                          VALUE_HIGH(ret),
-                          VALUE_LOW(val),
-                          VALUE_HIGH(val),
-                          oi);
+    atomic_trace_rmw_post(env, addr, oi);
    return ret;
 }

 #define GEN_ATOMIC_HELPER(X)                                        \
-ABI_TYPE ATOMIC_NAME(X)(CPUArchState *env, vaddr addr,              \
+ABI_TYPE ATOMIC_NAME(X)(CPUArchState *env, abi_ptr addr,            \
                        ABI_TYPE val, MemOpIdx oi, uintptr_t retaddr) \
 {                                                                   \
    DATA_TYPE *haddr, ret;                                          \
    haddr = atomic_mmu_lookup(env_cpu(env), addr, oi, DATA_SIZE, retaddr);   \
    ret = qatomic_##X(haddr, val);                                  \
    ATOMIC_MMU_CLEANUP;                                             \
-    atomic_trace_rmw_post(env, addr,                                \
-                          VALUE_LOW(ret),                           \
-                          VALUE_HIGH(ret),                          \
-                          VALUE_LOW(val),                           \
-                          VALUE_HIGH(val),                          \
-                          oi);                                      \
+    atomic_trace_rmw_post(env, addr, oi);                           \
    return ret;                                                     \
 }

@@ -156,7 +133,7 @@ GEN_ATOMIC_HELPER(xor_fetch)
 * of CF_PARALLEL's value, we'll trace just a read and a write.
 */
 #define GEN_ATOMIC_HELPER_FN(X, FN, XDATA_TYPE, RET)                \
-ABI_TYPE ATOMIC_NAME(X)(CPUArchState *env, vaddr addr,              \
+ABI_TYPE ATOMIC_NAME(X)(CPUArchState *env, abi_ptr addr,            \
                        ABI_TYPE xval, MemOpIdx oi, uintptr_t retaddr) \
 {                                                                   \
    XDATA_TYPE *haddr, cmp, old, new, val = xval;                   \
@@ -168,12 +145,7 @@ ABI_TYPE ATOMIC_NAME(X)(CPUArchState *env, vaddr addr,              \
        cmp = qatomic_cmpxchg__nocheck(haddr, old, new);            \
    } while (cmp != old);                                           \
    ATOMIC_MMU_CLEANUP;                                             \
-    atomic_trace_rmw_post(env, addr,                                \
-                          VALUE_LOW(old),                           \
-                          VALUE_HIGH(old),                          \
-                          VALUE_LOW(xval),                          \
-                          VALUE_HIGH(xval),                         \
-                          oi);                                      \
+    atomic_trace_rmw_post(env, addr, oi);                           \
    return RET;                                                     \
 }

@@ -202,7 +174,7 @@ GEN_ATOMIC_HELPER_FN(umax_fetch, MAX,  DATA_TYPE, new)
 # define END  _be
 #endif

-ABI_TYPE ATOMIC_NAME(cmpxchg)(CPUArchState *env, vaddr addr,
+ABI_TYPE ATOMIC_NAME(cmpxchg)(CPUArchState *env, abi_ptr addr,
                              ABI_TYPE cmpv, ABI_TYPE newv,
                              MemOpIdx oi, uintptr_t retaddr)
 {
@@ -216,17 +188,12 @@ ABI_TYPE ATOMIC_NAME(cmpxchg)(CPUArchState *env, vaddr addr,
    ret = qatomic_cmpxchg__nocheck(haddr, BSWAP(cmpv), BSWAP(newv));
 #endif
    ATOMIC_MMU_CLEANUP;
-    atomic_trace_rmw_post(env, addr,
-                          VALUE_LOW(ret),
-                          VALUE_HIGH(ret),
-                          VALUE_LOW(newv),
-                          VALUE_HIGH(newv),
-                          oi);
+    atomic_trace_rmw_post(env, addr, oi);
    return BSWAP(ret);
 }

 #if DATA_SIZE < 16
-ABI_TYPE ATOMIC_NAME(xchg)(CPUArchState *env, vaddr addr, ABI_TYPE val,
+ABI_TYPE ATOMIC_NAME(xchg)(CPUArchState *env, abi_ptr addr, ABI_TYPE val,
                           MemOpIdx oi, uintptr_t retaddr)
 {
    DATA_TYPE *haddr = atomic_mmu_lookup(env_cpu(env), addr, oi,
@@ -235,29 +202,19 @@ ABI_TYPE ATOMIC_NAME(xchg)(CPUArchState *env, vaddr addr, ABI_TYPE val,

    ret = qatomic_xchg__nocheck(haddr, BSWAP(val));
    ATOMIC_MMU_CLEANUP;
-    atomic_trace_rmw_post(env, addr,
-                          VALUE_LOW(ret),
-                          VALUE_HIGH(ret),
-                          VALUE_LOW(val),
-                          VALUE_HIGH(val),
-                          oi);
+    atomic_trace_rmw_post(env, addr, oi);
    return BSWAP(ret);
 }

 #define GEN_ATOMIC_HELPER(X)                                        \
-ABI_TYPE ATOMIC_NAME(X)(CPUArchState *env, vaddr addr,              \
+ABI_TYPE ATOMIC_NAME(X)(CPUArchState *env, abi_ptr addr,            \
                        ABI_TYPE val, MemOpIdx oi, uintptr_t retaddr) \
 {                                                                   \
    DATA_TYPE *haddr, ret;                                          \
    haddr = atomic_mmu_lookup(env_cpu(env), addr, oi, DATA_SIZE, retaddr);   \
    ret = qatomic_##X(haddr, BSWAP(val));                           \
    ATOMIC_MMU_CLEANUP;                                             \
-    atomic_trace_rmw_post(env, addr,                                \
-                          VALUE_LOW(ret),                           \
-                          VALUE_HIGH(ret),                          \
-                          VALUE_LOW(val),                           \
-                          VALUE_HIGH(val),                          \
-                          oi);                                      \
+    atomic_trace_rmw_post(env, addr, oi);                           \
    return BSWAP(ret);                                              \
 }

@@ -278,7 +235,7 @@ GEN_ATOMIC_HELPER(xor_fetch)
 * of CF_PARALLEL's value, we'll trace just a read and a write.
 */
 #define GEN_ATOMIC_HELPER_FN(X, FN, XDATA_TYPE, RET)                \
-ABI_TYPE ATOMIC_NAME(X)(CPUArchState *env, vaddr addr,              \
+ABI_TYPE ATOMIC_NAME(X)(CPUArchState *env, abi_ptr addr,            \
                        ABI_TYPE xval, MemOpIdx oi, uintptr_t retaddr) \
 {                                                                   \
    XDATA_TYPE *haddr, ldo, ldn, old, new, val = xval;              \
@@ -290,12 +247,7 @@ ABI_TYPE ATOMIC_NAME(X)(CPUArchState *env, vaddr addr,              \
        ldn = qatomic_cmpxchg__nocheck(haddr, ldo, BSWAP(new));     \
    } while (ldo != ldn);                                           \
    ATOMIC_MMU_CLEANUP;                                             \
-    atomic_trace_rmw_post(env, addr,                                \
-                          VALUE_LOW(old),                           \
-                          VALUE_HIGH(old),                          \
-                          VALUE_LOW(xval),                          \
-                          VALUE_HIGH(xval),                         \
-                          oi);                                      \
+    atomic_trace_rmw_post(env, addr, oi);                           \
    return RET;                                                     \
 }

@@ -329,5 +281,3 @@ GEN_ATOMIC_HELPER_FN(add_fetch, ADD, DATA_TYPE, new)
 #undef SUFFIX
 #undef DATA_SIZE
 #undef SHIFT
-#undef VALUE_LOW
-#undef VALUE_HIGH
@@ -1,41 +0,0 @@
-/*
- * Internal memory barrier helpers for QEMU (target agnostic)
- *
- *  Copyright (c) 2003 Fabrice Bellard
- *
- * SPDX-License-Identifier: LGPL-2.1-or-later
- */
-
-#ifndef ACCEL_TCG_BACKEND_LDST_H
-#define ACCEL_TCG_BACKEND_LDST_H
-
-#include "tcg-target-mo.h"
-
-/**
- * tcg_req_mo:
- * @guest_mo: Guest default memory order
- * @type: TCGBar
- *
- * Filter @type to the barrier that is required for the guest
- * memory ordering vs the host memory ordering.  A non-zero
- * result indicates that some barrier is required.
- */
-#define tcg_req_mo(guest_mo, type) \
-    ((type) & guest_mo & ~TCG_TARGET_DEFAULT_MO)
-
-/**
- * cpu_req_mo:
- * @cpu: CPUState
- * @type: TCGBar
- *
- * If tcg_req_mo indicates a barrier for @type is required
- * for the guest memory model, issue a host memory barrier.
- */
-#define cpu_req_mo(cpu, type)     \
-    do {                          \
-        if (tcg_req_mo(cpu->cc->tcg_ops->guest_default_memory_order, type)) { \
-            smp_mb();             \
-        }                         \
-    } while (0)
-
-#endif
@@ -18,45 +18,13 @@
 */

 #include "qemu/osdep.h"
-#include "exec/log.h"
-#include "system/tcg.h"
+#include "sysemu/cpus.h"
+#include "sysemu/tcg.h"
 #include "qemu/plugin.h"
 #include "internal-common.h"

 bool tcg_allowed;

-bool tcg_cflags_has(CPUState *cpu, uint32_t flags)
-{
-    return cpu->tcg_cflags & flags;
-}
-
-void tcg_cflags_set(CPUState *cpu, uint32_t flags)
-{
-    cpu->tcg_cflags |= flags;
-}
-
-uint32_t curr_cflags(CPUState *cpu)
-{
-    uint32_t cflags = cpu->tcg_cflags;
-
-    /*
-     * Record gdb single-step.  We should be exiting the TB by raising
-     * EXCP_DEBUG, but to simplify other tests, disable chaining too.
-     *
-     * For singlestep and -d nochain, suppress goto_tb so that
-     * we can log -d cpu,exec after every TB.
-     */
-    if (unlikely(cpu->singlestep_enabled)) {
-        cflags |= CF_NO_GOTO_TB | CF_NO_GOTO_PTR | CF_SINGLE_STEP | 1;
-    } else if (qatomic_read(&one_insn_per_tb)) {
-        cflags |= CF_NO_GOTO_TB | 1;
-    } else if (qemu_loglevel_mask(CPU_LOG_TB_NOCHAIN)) {
-        cflags |= CF_NO_GOTO_TB;
-    }
-
-    return cflags;
-}
-
 /* exit the current TB, but without causing any exception to be raised */
 void cpu_loop_exit_noexc(CPUState *cpu)
 {
@@ -21,30 +21,26 @@
 #include "qemu/qemu-print.h"
 #include "qapi/error.h"
 #include "qapi/type-helpers.h"
-#include "hw/core/cpu.h"
-#include "accel/tcg/cpu-ops.h"
-#include "accel/tcg/helper-retaddr.h"
+#include "hw/core/tcg-cpu-ops.h"
 #include "trace.h"
 #include "disas/disas.h"
-#include "exec/cpu-common.h"
-#include "exec/cpu-interrupt.h"
-#include "exec/page-protection.h"
-#include "exec/mmap-lock.h"
-#include "exec/translation-block.h"
+#include "exec/exec-all.h"
 #include "tcg/tcg.h"
 #include "qemu/atomic.h"
 #include "qemu/rcu.h"
 #include "exec/log.h"
 #include "qemu/main-loop.h"
-#include "exec/icount.h"
+#include "sysemu/cpus.h"
+#include "exec/cpu-all.h"
+#include "sysemu/cpu-timers.h"
 #include "exec/replay-core.h"
-#include "system/tcg.h"
+#include "sysemu/tcg.h"
 #include "exec/helper-proto-common.h"
 #include "tb-jmp-cache.h"
 #include "tb-hash.h"
 #include "tb-context.h"
-#include "tb-internal.h"
 #include "internal-common.h"
+#include "internal-target.h"

 /* -icount align implementation. */

@@ -148,10 +144,35 @@ static void init_delay_params(SyncClocks *sc, const CPUState *cpu)
 }
 #endif /* CONFIG USER ONLY */

+uint32_t curr_cflags(CPUState *cpu)
+{
+    uint32_t cflags = cpu->tcg_cflags;
+
+    /*
+     * Record gdb single-step.  We should be exiting the TB by raising
+     * EXCP_DEBUG, but to simplify other tests, disable chaining too.
+     *
+     * For singlestep and -d nochain, suppress goto_tb so that
+     * we can log -d cpu,exec after every TB.
+     */
+    if (unlikely(cpu->singlestep_enabled)) {
+        cflags |= CF_NO_GOTO_TB | CF_NO_GOTO_PTR | CF_SINGLE_STEP | 1;
+    } else if (qatomic_read(&one_insn_per_tb)) {
+        cflags |= CF_NO_GOTO_TB | 1;
+    } else if (qemu_loglevel_mask(CPU_LOG_TB_NOCHAIN)) {
+        cflags |= CF_NO_GOTO_TB;
+    }
+
+    return cflags;
+}
+
 struct tb_desc {
-    TCGTBCPUState s;
+    vaddr pc;
+    uint64_t cs_base;
    CPUArchState *env;
    tb_page_addr_t page_addr0;
+    uint32_t flags;
+    uint32_t cflags;
 };

 static bool tb_lookup_cmp(const void *p, const void *d)
@@ -159,11 +180,11 @@ static bool tb_lookup_cmp(const void *p, const void *d)
    const TranslationBlock *tb = p;
    const struct tb_desc *desc = d;

-    if ((tb_cflags(tb) & CF_PCREL || tb->pc == desc->s.pc) &&
+    if ((tb_cflags(tb) & CF_PCREL || tb->pc == desc->pc) &&
        tb_page_addr0(tb) == desc->page_addr0 &&
-        tb->cs_base == desc->s.cs_base &&
-        tb->flags == desc->s.flags &&
-        tb_cflags(tb) == desc->s.cflags) {
+        tb->cs_base == desc->cs_base &&
+        tb->flags == desc->flags &&
+        tb_cflags(tb) == desc->cflags) {
        /* check next page if needed */
        tb_page_addr_t tb_phys_page1 = tb_page_addr1(tb);
        if (tb_phys_page1 == -1) {
@@ -181,7 +202,7 @@ static bool tb_lookup_cmp(const void *p, const void *d)
             * is different for the new TB.  Therefore any exception raised
             * here by the faulting lookup is not premature.
             */
-            virt_page1 = TARGET_PAGE_ALIGN(desc->s.pc);
+            virt_page1 = TARGET_PAGE_ALIGN(desc->pc);
            phys_page1 = get_page_addr_code(desc->env, virt_page1);
            if (tb_phys_page1 == phys_page1) {
                return true;
@@ -191,65 +212,59 @@ static bool tb_lookup_cmp(const void *p, const void *d)
    return false;
 }

-static TranslationBlock *tb_htable_lookup(CPUState *cpu, TCGTBCPUState s)
+static TranslationBlock *tb_htable_lookup(CPUState *cpu, vaddr pc,
+                                          uint64_t cs_base, uint32_t flags,
+                                          uint32_t cflags)
 {
    tb_page_addr_t phys_pc;
    struct tb_desc desc;
    uint32_t h;

-    desc.s = s;
    desc.env = cpu_env(cpu);
-    phys_pc = get_page_addr_code(desc.env, s.pc);
+    desc.cs_base = cs_base;
+    desc.flags = flags;
+    desc.cflags = cflags;
+    desc.pc = pc;
+    phys_pc = get_page_addr_code(desc.env, pc);
    if (phys_pc == -1) {
        return NULL;
    }
    desc.page_addr0 = phys_pc;
-    h = tb_hash_func(phys_pc, (s.cflags & CF_PCREL ? 0 : s.pc),
-                     s.flags, s.cs_base, s.cflags);
+    h = tb_hash_func(phys_pc, (cflags & CF_PCREL ? 0 : pc),
+                     flags, cs_base, cflags);
    return qht_lookup_custom(&tb_ctx.htable, &desc, h, tb_lookup_cmp);
 }

-/**
- * tb_lookup:
- * @cpu: CPU that will execute the returned translation block
- * @pc: guest PC
- * @cs_base: arch-specific value associated with translation block
- * @flags: arch-specific translation block flags
- * @cflags: CF_* flags
- *
- * Look up a translation block inside the QHT using @pc, @cs_base, @flags and
- * @cflags. Uses @cpu's tb_jmp_cache. Might cause an exception, so have a
- * longjmp destination ready.
- *
- * Returns: an existing translation block or NULL.
- */
-static inline TranslationBlock *tb_lookup(CPUState *cpu, TCGTBCPUState s)
+/* Might cause an exception, so have a longjmp destination ready */
+static inline TranslationBlock *tb_lookup(CPUState *cpu, vaddr pc,
+                                          uint64_t cs_base, uint32_t flags,
+                                          uint32_t cflags)
 {
    TranslationBlock *tb;
    CPUJumpCache *jc;
    uint32_t hash;

    /* we should never be trying to look up an INVALID tb */
-    tcg_debug_assert(!(s.cflags & CF_INVALID));
+    tcg_debug_assert(!(cflags & CF_INVALID));

-    hash = tb_jmp_cache_hash_func(s.pc);
+    hash = tb_jmp_cache_hash_func(pc);
    jc = cpu->tb_jmp_cache;

    tb = qatomic_read(&jc->array[hash].tb);
    if (likely(tb &&
-               jc->array[hash].pc == s.pc &&
-               tb->cs_base == s.cs_base &&
-               tb->flags == s.flags &&
-               tb_cflags(tb) == s.cflags)) {
+               jc->array[hash].pc == pc &&
+               tb->cs_base == cs_base &&
+               tb->flags == flags &&
+               tb_cflags(tb) == cflags)) {
        goto hit;
    }

-    tb = tb_htable_lookup(cpu, s);
+    tb = tb_htable_lookup(cpu, pc, cs_base, flags, cflags);
    if (tb == NULL) {
        return NULL;
    }

-    jc->array[hash].pc = s.pc;
+    jc->array[hash].pc = pc;
    qatomic_set(&jc->array[hash].tb, tb);

 hit:
@@ -257,7 +272,7 @@ hit:
     * As long as tb is not NULL, the contents are consistent.  Therefore,
     * the virtual PC has to match for non-CF_PCREL translations.
     */
-    assert((tb_cflags(tb) & CF_PCREL) || tb->pc == s.pc);
+    assert((tb_cflags(tb) & CF_PCREL) || tb->pc == pc);
    return tb;
 }

@@ -274,11 +289,14 @@ static void log_cpu_exec(vaddr pc, CPUState *cpu,
        if (qemu_loglevel_mask(CPU_LOG_TB_CPU)) {
            FILE *logfile = qemu_log_trylock();
            if (logfile) {
-                int flags = CPU_DUMP_CCOP;
+                int flags = 0;

                if (qemu_loglevel_mask(CPU_LOG_TB_FPU)) {
                    flags |= CPU_DUMP_FPU;
                }
+#if defined(TARGET_I386)
+                flags |= CPU_DUMP_CCOP;
+#endif
                if (qemu_loglevel_mask(CPU_LOG_TB_VPU)) {
                    flags |= CPU_DUMP_VPU;
                }
@@ -350,7 +368,7 @@ static bool check_for_breakpoints_slow(CPUState *cpu, vaddr pc,
     * breakpoints are removed.
     */
    if (match_page) {
-        *cflags = (*cflags & ~CF_COUNT_MASK) | CF_NO_GOTO_TB | CF_BP_PAGE | 1;
+        *cflags = (*cflags & ~CF_COUNT_MASK) | CF_NO_GOTO_TB | 1;
    }
    return false;
 }
@@ -374,6 +392,9 @@ const void *HELPER(lookup_tb_ptr)(CPUArchState *env)
 {
    CPUState *cpu = env_cpu(env);
    TranslationBlock *tb;
+    vaddr pc;
+    uint64_t cs_base;
+    uint32_t flags, cflags;

    /*
     * By definition we've just finished a TB, so I/O is OK.
@@ -383,36 +404,25 @@ const void *HELPER(lookup_tb_ptr)(CPUArchState *env)
     * The next TB, if we chain to it, will clear the flag again.
     */
    cpu->neg.can_do_io = true;
+    cpu_get_tb_cpu_state(env, &pc, &cs_base, &flags);

-    TCGTBCPUState s = cpu->cc->tcg_ops->get_tb_cpu_state(cpu);
-    s.cflags = curr_cflags(cpu);
-
-    if (check_for_breakpoints(cpu, s.pc, &s.cflags)) {
+    cflags = curr_cflags(cpu);
+    if (check_for_breakpoints(cpu, pc, &cflags)) {
        cpu_loop_exit(cpu);
    }

-    tb = tb_lookup(cpu, s);
+    tb = tb_lookup(cpu, pc, cs_base, flags, cflags);
    if (tb == NULL) {
        return tcg_code_gen_epilogue;
    }

    if (qemu_loglevel_mask(CPU_LOG_TB_CPU | CPU_LOG_EXEC)) {
-        log_cpu_exec(s.pc, cpu, tb);
+        log_cpu_exec(pc, cpu, tb);
    }

    return tb->tc.ptr;
 }

-/* Return the current PC from CPU, which may be cached in TB. */
-static vaddr log_pc(CPUState *cpu, const TranslationBlock *tb)
-{
-    if (tb_cflags(tb) & CF_PCREL) {
-        return cpu->cc->get_pc(cpu);
-    } else {
-        return tb->pc;
-    }
-}
-
 /* Execute a TB, and fix up the CPU state afterwards if necessary */
 /*
 * Disable CFI checks.
@@ -426,6 +436,7 @@ static vaddr log_pc(CPUState *cpu, const TranslationBlock *tb)
 static inline TranslationBlock * QEMU_DISABLE_CFI
 cpu_tb_exec(CPUState *cpu, TranslationBlock *itb, int *tb_exit)
 {
+    CPUArchState *env = cpu_env(cpu);
    uintptr_t ret;
    TranslationBlock *last_tb;
    const void *tb_ptr = itb->tc.ptr;
@@ -435,7 +446,7 @@ cpu_tb_exec(CPUState *cpu, TranslationBlock *itb, int *tb_exit)
    }

    qemu_thread_jit_execute();
-    ret = tcg_qemu_tb_exec(cpu_env(cpu), tb_ptr);
+    ret = tcg_qemu_tb_exec(env, tb_ptr);
    cpu->neg.can_do_io = true;
    qemu_plugin_disable_mem_helpers(cpu);
    /*
@@ -547,7 +558,11 @@ static void cpu_exec_longjmp_cleanup(CPUState *cpu)

 void cpu_exec_step_atomic(CPUState *cpu)
 {
+    CPUArchState *env = cpu_env(cpu);
    TranslationBlock *tb;
+    vaddr pc;
+    uint64_t cs_base;
+    uint32_t flags, cflags;
    int tb_exit;

    if (sigsetjmp(cpu->jmp_env, 0) == 0) {
@@ -556,13 +571,13 @@ void cpu_exec_step_atomic(CPUState *cpu)
        g_assert(!cpu->running);
        cpu->running = true;

-        TCGTBCPUState s = cpu->cc->tcg_ops->get_tb_cpu_state(cpu);
-        s.cflags = curr_cflags(cpu);
+        cpu_get_tb_cpu_state(env, &pc, &cs_base, &flags);

+        cflags = curr_cflags(cpu);
        /* Execute in a serial context. */
-        s.cflags &= ~CF_PARALLEL;
+        cflags &= ~CF_PARALLEL;
        /* After 1 insn, return and release the exclusive lock. */
-        s.cflags |= CF_NO_GOTO_TB | CF_NO_GOTO_PTR | 1;
+        cflags |= CF_NO_GOTO_TB | CF_NO_GOTO_PTR | 1;
        /*
         * No need to check_for_breakpoints here.
         * We only arrive in cpu_exec_step_atomic after beginning execution
@@ -570,16 +585,16 @@ void cpu_exec_step_atomic(CPUState *cpu)
         * Any breakpoint for this insn will have been recognized earlier.
         */

-        tb = tb_lookup(cpu, s);
+        tb = tb_lookup(cpu, pc, cs_base, flags, cflags);
        if (tb == NULL) {
            mmap_lock();
-            tb = tb_gen_code(cpu, s);
+            tb = tb_gen_code(cpu, pc, cs_base, flags, cflags);
            mmap_unlock();
        }

        cpu_exec_enter(cpu);
        /* execute the generated code */
-        trace_exec_tb(tb, s.pc);
+        trace_exec_tb(tb, pc);
        cpu_tb_exec(cpu, tb, &tb_exit);
        cpu_exec_exit(cpu);
    } else {
@@ -647,6 +662,7 @@ static inline void tb_add_jump(TranslationBlock *tb, int n,

 out_unlock_next:
    qemu_spin_unlock(&tb_next->jmp_lock);
+    return;
 }

 static inline bool cpu_handle_halt(CPUState *cpu)
@@ -654,9 +670,11 @@ static inline bool cpu_handle_halt(CPUState *cpu)
 #ifndef CONFIG_USER_ONLY
    if (cpu->halted) {
        const TCGCPUOps *tcg_ops = cpu->cc->tcg_ops;
-        bool leave_halt = tcg_ops->cpu_exec_halt(cpu);

-        if (!leave_halt) {
+        if (tcg_ops->cpu_exec_halt) {
+            tcg_ops->cpu_exec_halt(cpu);
+        }
+        if (!cpu_has_work(cpu)) {
            return true;
        }

@@ -712,10 +730,10 @@ static inline bool cpu_handle_exception(CPUState *cpu, int *ret)
     * If user mode only, we simulate a fake exception which will be
     * handled outside the cpu execution loop.
     */
+#if defined(TARGET_I386)
    const TCGCPUOps *tcg_ops = cpu->cc->tcg_ops;
-    if (tcg_ops->fake_user_interrupt) {
-        tcg_ops->fake_user_interrupt(cpu);
-    }
+    tcg_ops->fake_user_interrupt(cpu);
+#endif /* TARGET_I386 */
    *ret = cpu->exception_index;
    cpu->exception_index = -1;
    return true;
@@ -802,23 +820,35 @@ static inline bool cpu_handle_interrupt(CPUState *cpu,
            cpu->exception_index = EXCP_HLT;
            bql_unlock();
            return true;
-        } else {
+        }
+#if defined(TARGET_I386)
+        else if (interrupt_request & CPU_INTERRUPT_INIT) {
+            X86CPU *x86_cpu = X86_CPU(cpu);
+            CPUArchState *env = &x86_cpu->env;
+            replay_interrupt();
+            cpu_svm_check_intercept_param(env, SVM_EXIT_INIT, 0, 0);
+            do_cpu_init(x86_cpu);
+            cpu->exception_index = EXCP_HALTED;
+            bql_unlock();
+            return true;
+        }
+#else
+        else if (interrupt_request & CPU_INTERRUPT_RESET) {
+            replay_interrupt();
+            cpu_reset(cpu);
+            bql_unlock();
+            return true;
+        }
+#endif /* !TARGET_I386 */
+        /* The target hook has 3 exit conditions:
+           False when the interrupt isn't processed,
+           True when it is, and we should restart on a new TB,
+           and via longjmp via cpu_loop_exit.  */
+        else {
            const TCGCPUOps *tcg_ops = cpu->cc->tcg_ops;

-            if (interrupt_request & CPU_INTERRUPT_RESET) {
-                replay_interrupt();
-                tcg_ops->cpu_exec_reset(cpu);
-                bql_unlock();
-                return true;
-            }
-
-            /*
-             * The target hook has 3 exit conditions:
-             * False when the interrupt isn't processed,
-             * True when it is, and we should restart on a new TB,
-             * and via longjmp via cpu_loop_exit.
-             */
-            if (tcg_ops->cpu_exec_interrupt(cpu, interrupt_request)) {
+            if (tcg_ops->cpu_exec_interrupt &&
+                tcg_ops->cpu_exec_interrupt(cpu, interrupt_request)) {
                if (!tcg_ops->need_replay_interrupt ||
                    tcg_ops->need_replay_interrupt(interrupt_request)) {
                    replay_interrupt();
@@ -868,6 +898,8 @@ static inline void cpu_loop_exec_tb(CPUState *cpu, TranslationBlock *tb,
                                    vaddr pc, TranslationBlock **last_tb,
                                    int *tb_exit)
 {
+    int32_t insns_left;
+
    trace_exec_tb(tb, pc);
    tb = cpu_tb_exec(cpu, tb, tb_exit);
    if (*tb_exit != TB_EXIT_REQUESTED) {
@@ -876,7 +908,8 @@ static inline void cpu_loop_exec_tb(CPUState *cpu, TranslationBlock *tb,
    }

    *last_tb = NULL;
-    if (cpu_loop_exit_requested(cpu)) {
+    insns_left = qatomic_read(&cpu->neg.icount_decr.u32);
+    if (insns_left < 0) {
        /* Something asked us to stop executing chained TBs; just
         * continue round the main loop. Whatever requested the exit
         * will also have set something else (eg exit_request or
@@ -893,7 +926,7 @@ static inline void cpu_loop_exec_tb(CPUState *cpu, TranslationBlock *tb,
    /* Ensure global icount has gone forward */
    icount_update(cpu);
    /* Refill decrementer and continue execution.  */
-    int32_t insns_left = MIN(0xffff, cpu->icount_budget);
+    insns_left = MIN(0xffff, cpu->icount_budget);
    cpu->neg.icount_decr.u16.low = insns_left;
    cpu->icount_extra = cpu->icount_budget - insns_left;

@@ -924,8 +957,11 @@ cpu_exec_loop(CPUState *cpu, SyncClocks *sc)

        while (!cpu_handle_interrupt(cpu, &last_tb)) {
            TranslationBlock *tb;
-            TCGTBCPUState s = cpu->cc->tcg_ops->get_tb_cpu_state(cpu);
-            s.cflags = cpu->cflags_next_tb;
+            vaddr pc;
+            uint64_t cs_base;
+            uint32_t flags, cflags;
+
+            cpu_get_tb_cpu_state(cpu_env(cpu), &pc, &cs_base, &flags);

            /*
             * When requested, use an exact setting for cflags for the next
@@ -934,32 +970,33 @@ cpu_exec_loop(CPUState *cpu, SyncClocks *sc)
             * have CF_INVALID set, -1 is a convenient invalid value that
             * does not require tcg headers for cpu_common_reset.
             */
-            if (s.cflags == -1) {
-                s.cflags = curr_cflags(cpu);
+            cflags = cpu->cflags_next_tb;
+            if (cflags == -1) {
+                cflags = curr_cflags(cpu);
            } else {
                cpu->cflags_next_tb = -1;
            }

-            if (check_for_breakpoints(cpu, s.pc, &s.cflags)) {
+            if (check_for_breakpoints(cpu, pc, &cflags)) {
                break;
            }

-            tb = tb_lookup(cpu, s);
+            tb = tb_lookup(cpu, pc, cs_base, flags, cflags);
            if (tb == NULL) {
                CPUJumpCache *jc;
                uint32_t h;

                mmap_lock();
-                tb = tb_gen_code(cpu, s);
+                tb = tb_gen_code(cpu, pc, cs_base, flags, cflags);
                mmap_unlock();

                /*
                 * We add the TB in the virtual pc hash table
                 * for the fast lookup
                 */
-                h = tb_jmp_cache_hash_func(s.pc);
+                h = tb_jmp_cache_hash_func(pc);
                jc = cpu->tb_jmp_cache;
-                jc->array[h].pc = s.pc;
+                jc->array[h].pc = pc;
                qatomic_set(&jc->array[h].tb, tb);
            }

@@ -979,7 +1016,7 @@ cpu_exec_loop(CPUState *cpu, SyncClocks *sc)
                tb_add_jump(last_tb, tb_exit, tb);
            }

-            cpu_loop_exec_tb(cpu, tb, s.pc, &last_tb, &tb_exit);
+            cpu_loop_exec_tb(cpu, tb, pc, &last_tb, &tb_exit);

            /* Try to align the host and virtual clocks
               if the guest is in advance */
@@ -1033,18 +1070,7 @@ bool tcg_exec_realizefn(CPUState *cpu, Error **errp)
    static bool tcg_target_initialized;

    if (!tcg_target_initialized) {
-        /* Check mandatory TCGCPUOps handlers */
-        const TCGCPUOps *tcg_ops = cpu->cc->tcg_ops;
-#ifndef CONFIG_USER_ONLY
-        assert(tcg_ops->cpu_exec_halt);
-        assert(tcg_ops->cpu_exec_interrupt);
-        assert(tcg_ops->cpu_exec_reset);
-        assert(tcg_ops->pointer_wrap);
-#endif /* !CONFIG_USER_ONLY */
-        assert(tcg_ops->translate_code);
-        assert(tcg_ops->get_tb_cpu_state);
-        assert(tcg_ops->mmu_index);
-        tcg_ops->initialize();
+        cpu->cc->tcg_ops->initialize();
        tcg_target_initialized = true;
    }

@@ -19,39 +19,30 @@

 #include "qemu/osdep.h"
 #include "qemu/main-loop.h"
-#include "qemu/target-info.h"
-#include "accel/tcg/cpu-ops.h"
-#include "accel/tcg/iommu.h"
-#include "accel/tcg/probe.h"
-#include "exec/page-protection.h"
-#include "system/memory.h"
-#include "accel/tcg/cpu-ldst-common.h"
-#include "accel/tcg/cpu-mmu-index.h"
+#include "hw/core/tcg-cpu-ops.h"
+#include "exec/exec-all.h"
+#include "exec/memory.h"
+#include "exec/cpu_ldst.h"
 #include "exec/cputlb.h"
 #include "exec/tb-flush.h"
-#include "system/ram_addr.h"
-#include "exec/mmu-access-type.h"
-#include "exec/tlb-common.h"
-#include "exec/vaddr.h"
+#include "exec/memory-internal.h"
+#include "exec/ram_addr.h"
 #include "tcg/tcg.h"
 #include "qemu/error-report.h"
 #include "exec/log.h"
 #include "exec/helper-proto-common.h"
-#include "exec/tlb-flags.h"
 #include "qemu/atomic.h"
 #include "qemu/atomic128.h"
-#include "tb-internal.h"
+#include "exec/translate-all.h"
 #include "trace.h"
 #include "tb-hash.h"
-#include "tb-internal.h"
-#include "tlb-bounds.h"
 #include "internal-common.h"
+#include "internal-target.h"
 #ifdef CONFIG_PLUGIN
 #include "qemu/plugin-memory.h"
 #endif
 #include "tcg/tcg-ldst.h"
-#include "backend-ldst.h"
-
+#include "tcg/oversized-guest.h"

 /* DEBUG defines, enable DEBUG_TLB_LOG to log to the CPU_LOG_MMU target */
 /* #define DEBUG_TLB */
@@ -104,43 +95,6 @@ static inline size_t sizeof_tlb(CPUTLBDescFast *fast)
    return fast->mask + (1 << CPU_TLB_ENTRY_BITS);
 }

-static inline uint64_t tlb_read_idx(const CPUTLBEntry *entry,
-                                    MMUAccessType access_type)
-{
-    /* Do not rearrange the CPUTLBEntry structure members. */
-    QEMU_BUILD_BUG_ON(offsetof(CPUTLBEntry, addr_read) !=
-                      MMU_DATA_LOAD * sizeof(uintptr_t));
-    QEMU_BUILD_BUG_ON(offsetof(CPUTLBEntry, addr_write) !=
-                      MMU_DATA_STORE * sizeof(uintptr_t));
-    QEMU_BUILD_BUG_ON(offsetof(CPUTLBEntry, addr_code) !=
-                      MMU_INST_FETCH * sizeof(uintptr_t));
-
-    const uintptr_t *ptr = &entry->addr_idx[access_type];
-    /* ofs might correspond to .addr_write, so use qatomic_read */
-    return qatomic_read(ptr);
-}
-
-static inline uint64_t tlb_addr_write(const CPUTLBEntry *entry)
-{
-    return tlb_read_idx(entry, MMU_DATA_STORE);
-}
-
-/* Find the TLB index corresponding to the mmu_idx + address pair.  */
-static inline uintptr_t tlb_index(CPUState *cpu, uintptr_t mmu_idx,
-                                  vaddr addr)
-{
-    uintptr_t size_mask = cpu->neg.tlb.f[mmu_idx].mask >> CPU_TLB_ENTRY_BITS;
-
-    return (addr >> TARGET_PAGE_BITS) & size_mask;
-}
-
-/* Find the TLB entry corresponding to the mmu_idx + address pair.  */
-static inline CPUTLBEntry *tlb_entry(CPUState *cpu, uintptr_t mmu_idx,
-                                     vaddr addr)
-{
-    return &cpu->neg.tlb.f[mmu_idx].table[tlb_index(cpu, mmu_idx, addr)];
-}
-
 static void tlb_window_reset(CPUTLBDesc *desc, int64_t ns,
                             size_t max_entries)
 {
@@ -412,9 +366,12 @@ void tlb_flush_by_mmuidx(CPUState *cpu, uint16_t idxmap)
 {
    tlb_debug("mmu_idx: 0x%" PRIx16 "\n", idxmap);

-    assert_cpu_is_self(cpu);
-
-    tlb_flush_by_mmuidx_async_work(cpu, RUN_ON_CPU_HOST_INT(idxmap));
+    if (cpu->created && !qemu_cpu_is_self(cpu)) {
+        async_run_on_cpu(cpu, tlb_flush_by_mmuidx_async_work,
+                         RUN_ON_CPU_HOST_INT(idxmap));
+    } else {
+        tlb_flush_by_mmuidx_async_work(cpu, RUN_ON_CPU_HOST_INT(idxmap));
+    }
 }

 void tlb_flush(CPUState *cpu)
@@ -422,6 +379,21 @@ void tlb_flush(CPUState *cpu)
    tlb_flush_by_mmuidx(cpu, ALL_MMUIDX_BITS);
 }

+void tlb_flush_by_mmuidx_all_cpus(CPUState *src_cpu, uint16_t idxmap)
+{
+    const run_on_cpu_func fn = tlb_flush_by_mmuidx_async_work;
+
+    tlb_debug("mmu_idx: 0x%"PRIx16"\n", idxmap);
+
+    flush_all_helper(src_cpu, fn, RUN_ON_CPU_HOST_INT(idxmap));
+    fn(src_cpu, RUN_ON_CPU_HOST_INT(idxmap));
+}
+
+void tlb_flush_all_cpus(CPUState *src_cpu)
+{
+    tlb_flush_by_mmuidx_all_cpus(src_cpu, ALL_MMUIDX_BITS);
+}
+
 void tlb_flush_by_mmuidx_all_cpus_synced(CPUState *src_cpu, uint16_t idxmap)
 {
    const run_on_cpu_func fn = tlb_flush_by_mmuidx_async_work;
@@ -603,12 +575,28 @@ void tlb_flush_page_by_mmuidx(CPUState *cpu, vaddr addr, uint16_t idxmap)
 {
    tlb_debug("addr: %016" VADDR_PRIx " mmu_idx:%" PRIx16 "\n", addr, idxmap);

-    assert_cpu_is_self(cpu);
-
    /* This should already be page aligned */
    addr &= TARGET_PAGE_MASK;

-    tlb_flush_page_by_mmuidx_async_0(cpu, addr, idxmap);
+    if (qemu_cpu_is_self(cpu)) {
+        tlb_flush_page_by_mmuidx_async_0(cpu, addr, idxmap);
+    } else if (idxmap < TARGET_PAGE_SIZE) {
+        /*
+         * Most targets have only a few mmu_idx.  In the case where
+         * we can stuff idxmap into the low TARGET_PAGE_BITS, avoid
+         * allocating memory for this operation.
+         */
+        async_run_on_cpu(cpu, tlb_flush_page_by_mmuidx_async_1,
+                         RUN_ON_CPU_TARGET_PTR(addr | idxmap));
+    } else {
+        TLBFlushPageByMMUIdxData *d = g_new(TLBFlushPageByMMUIdxData, 1);
+
+        /* Otherwise allocate a structure, freed by the worker.  */
+        d->addr = addr;
+        d->idxmap = idxmap;
+        async_run_on_cpu(cpu, tlb_flush_page_by_mmuidx_async_2,
+                         RUN_ON_CPU_HOST_PTR(d));
+    }
 }

 void tlb_flush_page(CPUState *cpu, vaddr addr)
@@ -616,6 +604,46 @@ void tlb_flush_page(CPUState *cpu, vaddr addr)
    tlb_flush_page_by_mmuidx(cpu, addr, ALL_MMUIDX_BITS);
 }

+void tlb_flush_page_by_mmuidx_all_cpus(CPUState *src_cpu, vaddr addr,
+                                       uint16_t idxmap)
+{
+    tlb_debug("addr: %016" VADDR_PRIx " mmu_idx:%"PRIx16"\n", addr, idxmap);
+
+    /* This should already be page aligned */
+    addr &= TARGET_PAGE_MASK;
+
+    /*
+     * Allocate memory to hold addr+idxmap only when needed.
+     * See tlb_flush_page_by_mmuidx for details.
+     */
+    if (idxmap < TARGET_PAGE_SIZE) {
+        flush_all_helper(src_cpu, tlb_flush_page_by_mmuidx_async_1,
+                         RUN_ON_CPU_TARGET_PTR(addr | idxmap));
+    } else {
+        CPUState *dst_cpu;
+
+        /* Allocate a separate data block for each destination cpu.  */
+        CPU_FOREACH(dst_cpu) {
+            if (dst_cpu != src_cpu) {
+                TLBFlushPageByMMUIdxData *d
+                    = g_new(TLBFlushPageByMMUIdxData, 1);
+
+                d->addr = addr;
+                d->idxmap = idxmap;
+                async_run_on_cpu(dst_cpu, tlb_flush_page_by_mmuidx_async_2,
+                                 RUN_ON_CPU_HOST_PTR(d));
+            }
+        }
+    }
+
+    tlb_flush_page_by_mmuidx_async_0(src_cpu, addr, idxmap);
+}
+
+void tlb_flush_page_all_cpus(CPUState *src, vaddr addr)
+{
+    tlb_flush_page_by_mmuidx_all_cpus(src, addr, ALL_MMUIDX_BITS);
+}
+
 void tlb_flush_page_by_mmuidx_all_cpus_synced(CPUState *src_cpu,
                                              vaddr addr,
                                              uint16_t idxmap)
@@ -771,21 +799,19 @@ void tlb_flush_range_by_mmuidx(CPUState *cpu, vaddr addr,
 {
    TLBFlushRangeData d;

-    assert_cpu_is_self(cpu);
-
-    /* If no page bits are significant, this devolves to tlb_flush. */
-    if (bits < TARGET_PAGE_BITS) {
-        tlb_flush_by_mmuidx(cpu, idxmap);
-        return;
-    }
    /*
     * If all bits are significant, and len is small,
     * this devolves to tlb_flush_page.
     */
-    if (len <= TARGET_PAGE_SIZE && bits >= target_long_bits()) {
+    if (bits >= TARGET_LONG_BITS && len <= TARGET_PAGE_SIZE) {
        tlb_flush_page_by_mmuidx(cpu, addr, idxmap);
        return;
    }
+    /* If no page bits are significant, this devolves to tlb_flush. */
+    if (bits < TARGET_PAGE_BITS) {
+        tlb_flush_by_mmuidx(cpu, idxmap);
+        return;
+    }

    /* This should already be page aligned */
    d.addr = addr & TARGET_PAGE_MASK;
@@ -793,7 +819,14 @@ void tlb_flush_range_by_mmuidx(CPUState *cpu, vaddr addr,
    d.idxmap = idxmap;
    d.bits = bits;

-    tlb_flush_range_by_mmuidx_async_0(cpu, d);
+    if (qemu_cpu_is_self(cpu)) {
+        tlb_flush_range_by_mmuidx_async_0(cpu, d);
+    } else {
+        /* Otherwise allocate a structure, freed by the worker.  */
+        TLBFlushRangeData *p = g_memdup(&d, sizeof(d));
+        async_run_on_cpu(cpu, tlb_flush_range_by_mmuidx_async_1,
+                         RUN_ON_CPU_HOST_PTR(p));
+    }
 }

 void tlb_flush_page_bits_by_mmuidx(CPUState *cpu, vaddr addr,
@@ -802,6 +835,54 @@ void tlb_flush_page_bits_by_mmuidx(CPUState *cpu, vaddr addr,
    tlb_flush_range_by_mmuidx(cpu, addr, TARGET_PAGE_SIZE, idxmap, bits);
 }

+void tlb_flush_range_by_mmuidx_all_cpus(CPUState *src_cpu,
+                                        vaddr addr, vaddr len,
+                                        uint16_t idxmap, unsigned bits)
+{
+    TLBFlushRangeData d;
+    CPUState *dst_cpu;
+
+    /*
+     * If all bits are significant, and len is small,
+     * this devolves to tlb_flush_page.
+     */
+    if (bits >= TARGET_LONG_BITS && len <= TARGET_PAGE_SIZE) {
+        tlb_flush_page_by_mmuidx_all_cpus(src_cpu, addr, idxmap);
+        return;
+    }
+    /* If no page bits are significant, this devolves to tlb_flush. */
+    if (bits < TARGET_PAGE_BITS) {
+        tlb_flush_by_mmuidx_all_cpus(src_cpu, idxmap);
+        return;
+    }
+
+    /* This should already be page aligned */
+    d.addr = addr & TARGET_PAGE_MASK;
+    d.len = len;
+    d.idxmap = idxmap;
+    d.bits = bits;
+
+    /* Allocate a separate data block for each destination cpu.  */
+    CPU_FOREACH(dst_cpu) {
+        if (dst_cpu != src_cpu) {
+            TLBFlushRangeData *p = g_memdup(&d, sizeof(d));
+            async_run_on_cpu(dst_cpu,
+                             tlb_flush_range_by_mmuidx_async_1,
+                             RUN_ON_CPU_HOST_PTR(p));
+        }
+    }
+
+    tlb_flush_range_by_mmuidx_async_0(src_cpu, d);
+}
+
+void tlb_flush_page_bits_by_mmuidx_all_cpus(CPUState *src_cpu,
+                                            vaddr addr, uint16_t idxmap,
+                                            unsigned bits)
+{
+    tlb_flush_range_by_mmuidx_all_cpus(src_cpu, addr, TARGET_PAGE_SIZE,
+                                       idxmap, bits);
+}
+
 void tlb_flush_range_by_mmuidx_all_cpus_synced(CPUState *src_cpu,
                                               vaddr addr,
                                               vaddr len,
@@ -811,19 +892,19 @@ void tlb_flush_range_by_mmuidx_all_cpus_synced(CPUState *src_cpu,
    TLBFlushRangeData d, *p;
    CPUState *dst_cpu;

-    /* If no page bits are significant, this devolves to tlb_flush. */
-    if (bits < TARGET_PAGE_BITS) {
-        tlb_flush_by_mmuidx_all_cpus_synced(src_cpu, idxmap);
-        return;
-    }
    /*
     * If all bits are significant, and len is small,
     * this devolves to tlb_flush_page.
     */
-    if (len <= TARGET_PAGE_SIZE && bits >= target_long_bits()) {
+    if (bits >= TARGET_LONG_BITS && len <= TARGET_PAGE_SIZE) {
        tlb_flush_page_by_mmuidx_all_cpus_synced(src_cpu, addr, idxmap);
        return;
    }
+    /* If no page bits are significant, this devolves to tlb_flush. */
+    if (bits < TARGET_PAGE_BITS) {
+        tlb_flush_by_mmuidx_all_cpus_synced(src_cpu, idxmap);
+        return;
+    }

    /* This should already be page aligned */
    d.addr = addr & TARGET_PAGE_MASK;
@@ -887,17 +968,26 @@ void tlb_unprotect_code(ram_addr_t ram_addr)
 *
 * Called with tlb_c.lock held.
 */
-static void tlb_reset_dirty_range_locked(CPUTLBEntryFull *full, CPUTLBEntry *ent,
+static void tlb_reset_dirty_range_locked(CPUTLBEntry *tlb_entry,
                                         uintptr_t start, uintptr_t length)
 {
-    const uintptr_t addr = ent->addr_write;
-    int flags = addr | full->slow_flags[MMU_DATA_STORE];
+    uintptr_t addr = tlb_entry->addr_write;

-    flags &= TLB_INVALID_MASK | TLB_MMIO | TLB_DISCARD_WRITE | TLB_NOTDIRTY;
-    if (flags == 0) {
-        uintptr_t host = (addr & TARGET_PAGE_MASK) + ent->addend;
-        if ((host - start) < length) {
-            qatomic_set(&ent->addr_write, addr | TLB_NOTDIRTY);
+    if ((addr & (TLB_INVALID_MASK | TLB_MMIO |
+                 TLB_DISCARD_WRITE | TLB_NOTDIRTY)) == 0) {
+        addr &= TARGET_PAGE_MASK;
+        addr += tlb_entry->addend;
+        if ((addr - start) < length) {
+#if TARGET_LONG_BITS == 32
+            uint32_t *ptr_write = (uint32_t *)&tlb_entry->addr_write;
+            ptr_write += HOST_BIG_ENDIAN;
+            qatomic_set(ptr_write, *ptr_write | TLB_NOTDIRTY);
+#elif TCG_OVERSIZED_GUEST
+            tlb_entry->addr_write |= TLB_NOTDIRTY;
+#else
+            qatomic_set(&tlb_entry->addr_write,
+                        tlb_entry->addr_write | TLB_NOTDIRTY);
+#endif
        }
    }
 }
@@ -916,25 +1006,23 @@ static inline void copy_tlb_helper_locked(CPUTLBEntry *d, const CPUTLBEntry *s)
 * We must take tlb_c.lock to avoid racing with another vCPU update. The only
 * thing actually updated is the target TLB entry ->addr_write flags.
 */
-void tlb_reset_dirty(CPUState *cpu, uintptr_t start, uintptr_t length)
+void tlb_reset_dirty(CPUState *cpu, ram_addr_t start1, ram_addr_t length)
 {
    int mmu_idx;

    qemu_spin_lock(&cpu->neg.tlb.c.lock);
    for (mmu_idx = 0; mmu_idx < NB_MMU_MODES; mmu_idx++) {
-        CPUTLBDesc *desc = &cpu->neg.tlb.d[mmu_idx];
-        CPUTLBDescFast *fast = &cpu->neg.tlb.f[mmu_idx];
-        unsigned int n = tlb_n_entries(fast);
        unsigned int i;
+        unsigned int n = tlb_n_entries(&cpu->neg.tlb.f[mmu_idx]);

        for (i = 0; i < n; i++) {
-            tlb_reset_dirty_range_locked(&desc->fulltlb[i], &fast->table[i],
-                                         start, length);
+            tlb_reset_dirty_range_locked(&cpu->neg.tlb.f[mmu_idx].table[i],
+                                         start1, length);
        }

        for (i = 0; i < CPU_VTLB_SIZE; i++) {
-            tlb_reset_dirty_range_locked(&desc->vfulltlb[i], &desc->vtable[i],
-                                         start, length);
+            tlb_reset_dirty_range_locked(&cpu->neg.tlb.d[mmu_idx].vtable[i],
+                                         start1, length);
        }
    }
    qemu_spin_unlock(&cpu->neg.tlb.c.lock);
@@ -951,7 +1039,7 @@ static inline void tlb_set_dirty1_locked(CPUTLBEntry *tlb_entry,

 /* update the TLB corresponding to virtual page vaddr
   so that it is no longer dirty */
-static void tlb_set_dirty(CPUState *cpu, vaddr addr)
+void tlb_set_dirty(CPUState *cpu, vaddr addr)
 {
    int mmu_idx;

@@ -1057,11 +1145,14 @@ void tlb_set_page_full(CPUState *cpu, int mmu_idx,
              " prot=%x idx=%d\n",
              addr, full->phys_addr, prot, mmu_idx);

-    read_flags = full->tlb_fill_flags;
+    read_flags = 0;
    if (full->lg_page_size < TARGET_PAGE_BITS) {
        /* Repeat the MMU check and TLB fill on every access.  */
        read_flags |= TLB_INVALID_MASK;
    }
+    if (full->attrs.byte_swap) {
+        read_flags |= TLB_BSWAP;
+    }

    is_ram = memory_region_is_ram(section->mr);
    is_romd = memory_region_is_romd(section->mr);
@@ -1186,7 +1277,7 @@ void tlb_set_page_full(CPUState *cpu, int mmu_idx,

 void tlb_set_page_with_attrs(CPUState *cpu, vaddr addr,
                             hwaddr paddr, MemTxAttrs attrs, int prot,
-                             int mmu_idx, vaddr size)
+                             int mmu_idx, uint64_t size)
 {
    CPUTLBEntryFull full = {
        .phys_addr = paddr,
@@ -1201,65 +1292,29 @@ void tlb_set_page_with_attrs(CPUState *cpu, vaddr addr,

 void tlb_set_page(CPUState *cpu, vaddr addr,
                  hwaddr paddr, int prot,
-                  int mmu_idx, vaddr size)
+                  int mmu_idx, uint64_t size)
 {
    tlb_set_page_with_attrs(cpu, addr, paddr, MEMTXATTRS_UNSPECIFIED,
                            prot, mmu_idx, size);
 }

-/**
- * tlb_hit_page: return true if page aligned @addr is a hit against the
- * TLB entry @tlb_addr
- *
- * @addr: virtual address to test (must be page aligned)
- * @tlb_addr: TLB entry address (a CPUTLBEntry addr_read/write/code value)
- */
-static inline bool tlb_hit_page(uint64_t tlb_addr, vaddr addr)
-{
-    return addr == (tlb_addr & (TARGET_PAGE_MASK | TLB_INVALID_MASK));
-}
-
-/**
- * tlb_hit: return true if @addr is a hit against the TLB entry @tlb_addr
- *
- * @addr: virtual address to test (need not be page aligned)
- * @tlb_addr: TLB entry address (a CPUTLBEntry addr_read/write/code value)
- */
-static inline bool tlb_hit(uint64_t tlb_addr, vaddr addr)
-{
-    return tlb_hit_page(tlb_addr, addr & TARGET_PAGE_MASK);
-}
-
 /*
- * Note: tlb_fill_align() can trigger a resize of the TLB.
- * This means that all of the caller's prior references to the TLB table
- * (e.g. CPUTLBEntry pointers) must be discarded and looked up again
- * (e.g. via tlb_entry()).
+ * Note: tlb_fill() can trigger a resize of the TLB. This means that all of the
+ * caller's prior references to the TLB table (e.g. CPUTLBEntry pointers) must
+ * be discarded and looked up again (e.g. via tlb_entry()).
 */
-static bool tlb_fill_align(CPUState *cpu, vaddr addr, MMUAccessType type,
-                           int mmu_idx, MemOp memop, int size,
-                           bool probe, uintptr_t ra)
+static void tlb_fill(CPUState *cpu, vaddr addr, int size,
+                     MMUAccessType access_type, int mmu_idx, uintptr_t retaddr)
 {
-    const TCGCPUOps *ops = cpu->cc->tcg_ops;
-    CPUTLBEntryFull full;
+    bool ok;

-    if (ops->tlb_fill_align) {
-        if (ops->tlb_fill_align(cpu, &full, addr, type, mmu_idx,
-                                memop, size, probe, ra)) {
-            tlb_set_page_full(cpu, mmu_idx, addr, &full);
-            return true;
-        }
-    } else {
-        /* Legacy behaviour is alignment before paging. */
-        if (addr & ((1u << memop_alignment_bits(memop)) - 1)) {
-            ops->do_unaligned_access(cpu, addr, type, mmu_idx, ra);
-        }
-        if (ops->tlb_fill(cpu, addr, size, type, mmu_idx, probe, ra)) {
-            return true;
-        }
-    }
-    assert(probe);
-    return false;
+    /*
+     * This is not a probe, so only valid return is success; failure
+     * should result in exception + longjmp to the cpu loop.
+     */
+    ok = cpu->cc->tcg_ops->tlb_fill(cpu, addr, size,
+                                    access_type, mmu_idx, false, retaddr);
+    assert(ok);
 }

 static inline void cpu_unaligned_access(CPUState *cpu, vaddr addr,
@@ -1342,7 +1397,7 @@ static void notdirty_write(CPUState *cpu, vaddr mem_vaddr, unsigned size,
    trace_memory_notdirty_write_access(mem_vaddr, ram_addr, size);

    if (!cpu_physical_memory_get_dirty_flag(ram_addr, DIRTY_MEMORY_CODE)) {
-        tb_invalidate_phys_range_fast(cpu, ram_addr, size, retaddr);
+        tb_invalidate_phys_range_fast(ram_addr, size, retaddr);
    }

    /*
@@ -1374,22 +1429,22 @@ static int probe_access_internal(CPUState *cpu, vaddr addr,

    if (!tlb_hit_page(tlb_addr, page_addr)) {
        if (!victim_tlb_hit(cpu, mmu_idx, index, access_type, page_addr)) {
-            if (!tlb_fill_align(cpu, addr, access_type, mmu_idx,
-                                0, fault_size, nonfault, retaddr)) {
+            if (!cpu->cc->tcg_ops->tlb_fill(cpu, addr, fault_size, access_type,
+                                            mmu_idx, nonfault, retaddr)) {
                /* Non-faulting page table read failed.  */
                *phost = NULL;
                *pfull = NULL;
                return TLB_INVALID_MASK;
            }

-            /* TLB resize via tlb_fill_align may have moved the entry.  */
+            /* TLB resize via tlb_fill may have moved the entry.  */
            index = tlb_index(cpu, mmu_idx, addr);
            entry = tlb_entry(cpu, mmu_idx, addr);

            /*
             * With PAGE_WRITE_INV, we set TLB_INVALID_MASK immediately,
-             * to force the next access through tlb_fill_align.  We've just
-             * called tlb_fill_align, so we know that this entry *is* valid.
+             * to force the next access through tlb_fill.  We've just
+             * called tlb_fill, so we know that this entry *is* valid.
             */
            flags &= ~TLB_INVALID_MASK;
        }
@@ -1401,8 +1456,9 @@ static int probe_access_internal(CPUState *cpu, vaddr addr,
    flags |= full->slow_flags[access_type];

    /* Fold all "mmio-like" bits into TLB_MMIO.  This is not RAM.  */
-    if (unlikely(flags & ~(TLB_WATCHPOINT | TLB_NOTDIRTY | TLB_CHECK_ALIGNED))
-        || (access_type != MMU_INST_FETCH && force_mmio)) {
+    if (unlikely(flags & ~(TLB_WATCHPOINT | TLB_NOTDIRTY))
+        ||
+        (access_type != MMU_INST_FETCH && force_mmio)) {
        *phost = NULL;
        return TLB_MMIO;
    }
@@ -1514,7 +1570,7 @@ void *probe_access(CPUArchState *env, vaddr addr, int size,
    return host;
 }

-void *tlb_vaddr_to_host(CPUArchState *env, vaddr addr,
+void *tlb_vaddr_to_host(CPUArchState *env, abi_ptr addr,
                        MMUAccessType access_type, int mmu_idx)
 {
    CPUTLBEntryFull *full;
@@ -1630,17 +1686,16 @@ typedef struct MMULookupLocals {
 * mmu_lookup1: translate one page
 * @cpu: generic cpu state
 * @data: lookup parameters
- * @memop: memory operation for the access, or 0
 * @mmu_idx: virtual address context
 * @access_type: load/store/code
 * @ra: return address into tcg generated code, or 0
 *
 * Resolve the translation for the one page at @data.addr, filling in
 * the rest of @data with the results.  If the translation fails,
- * tlb_fill_align will longjmp out.  Return true if the softmmu tlb for
+ * tlb_fill will longjmp out.  Return true if the softmmu tlb for
 * @mmu_idx may have resized.
 */
-static bool mmu_lookup1(CPUState *cpu, MMULookupPageData *data, MemOp memop,
+static bool mmu_lookup1(CPUState *cpu, MMULookupPageData *data,
                        int mmu_idx, MMUAccessType access_type, uintptr_t ra)
 {
    vaddr addr = data->addr;
@@ -1655,8 +1710,7 @@ static bool mmu_lookup1(CPUState *cpu, MMULookupPageData *data, MemOp memop,
    if (!tlb_hit(tlb_addr, addr)) {
        if (!victim_tlb_hit(cpu, mmu_idx, index, access_type,
                            addr & TARGET_PAGE_MASK)) {
-            tlb_fill_align(cpu, addr, access_type, mmu_idx,
-                           memop, data->size, false, ra);
+            tlb_fill(cpu, addr, data->size, access_type, mmu_idx, ra);
            maybe_resized = true;
            index = tlb_index(cpu, mmu_idx, addr);
            entry = tlb_entry(cpu, mmu_idx, addr);
@@ -1668,25 +1722,6 @@ static bool mmu_lookup1(CPUState *cpu, MMULookupPageData *data, MemOp memop,
    flags = tlb_addr & (TLB_FLAGS_MASK & ~TLB_FORCE_SLOW);
    flags |= full->slow_flags[access_type];

-    if (likely(!maybe_resized)) {
-        /* Alignment has not been checked by tlb_fill_align. */
-        int a_bits = memop_alignment_bits(memop);
-
-        /*
-         * This alignment check differs from the one above, in that this is
-         * based on the atomicity of the operation. The intended use case is
-         * the ARM memory type field of each PTE, where access to pages with
-         * Device memory type require alignment.
-         */
-        if (unlikely(flags & TLB_CHECK_ALIGNED)) {
-            int at_bits = memop_atomicity_bits(memop);
-            a_bits = MAX(a_bits, at_bits);
-        }
-        if (unlikely(addr & ((1 << a_bits) - 1))) {
-            cpu_unaligned_access(cpu, addr, access_type, mmu_idx, ra);
-        }
-    }
-
    data->full = full;
    data->flags = flags;
    /* Compute haddr speculatively; depending on flags it might be invalid. */
@@ -1743,6 +1778,7 @@ static void mmu_watch_or_dirty(CPUState *cpu, MMULookupPageData *data,
 static bool mmu_lookup(CPUState *cpu, vaddr addr, MemOpIdx oi,
                       uintptr_t ra, MMUAccessType type, MMULookupLocals *l)
 {
+    unsigned a_bits;
    bool crosspage;
    int flags;

@@ -1751,6 +1787,12 @@ static bool mmu_lookup(CPUState *cpu, vaddr addr, MemOpIdx oi,

    tcg_debug_assert(l->mmu_idx < NB_MMU_MODES);

+    /* Handle CPU specific unaligned behaviour */
+    a_bits = get_alignment_bits(l->memop);
+    if (addr & ((1 << a_bits) - 1)) {
+        cpu_unaligned_access(cpu, addr, type, l->mmu_idx, ra);
+    }
+
    l->page[0].addr = addr;
    l->page[0].size = memop_size(l->memop);
    l->page[1].addr = (addr + l->page[0].size - 1) & TARGET_PAGE_MASK;
@@ -1758,7 +1800,7 @@ static bool mmu_lookup(CPUState *cpu, vaddr addr, MemOpIdx oi,
    crosspage = (addr ^ l->page[1].addr) & TARGET_PAGE_MASK;

    if (likely(!crosspage)) {
-        mmu_lookup1(cpu, &l->page[0], l->memop, l->mmu_idx, type, ra);
+        mmu_lookup1(cpu, &l->page[0], l->mmu_idx, type, ra);

        flags = l->page[0].flags;
        if (unlikely(flags & (TLB_WATCHPOINT | TLB_NOTDIRTY))) {
@@ -1773,15 +1815,12 @@ static bool mmu_lookup(CPUState *cpu, vaddr addr, MemOpIdx oi,
        l->page[1].size = l->page[0].size - size0;
        l->page[0].size = size0;

-        l->page[1].addr = cpu->cc->tcg_ops->pointer_wrap(cpu, l->mmu_idx,
-                                                         l->page[1].addr, addr);
-
        /*
         * Lookup both pages, recognizing exceptions from either.  If the
         * second lookup potentially resized, refresh first CPUTLBEntryFull.
         */
-        mmu_lookup1(cpu, &l->page[0], l->memop, l->mmu_idx, type, ra);
-        if (mmu_lookup1(cpu, &l->page[1], 0, l->mmu_idx, type, ra)) {
+        mmu_lookup1(cpu, &l->page[0], l->mmu_idx, type, ra);
+        if (mmu_lookup1(cpu, &l->page[1], l->mmu_idx, type, ra)) {
            uintptr_t index = tlb_index(cpu, l->mmu_idx, addr);
            l->page[0].full = &cpu->neg.tlb.d[l->mmu_idx].fulltlb[index];
        }
@@ -1812,18 +1851,34 @@ static void *atomic_mmu_lookup(CPUState *cpu, vaddr addr, MemOpIdx oi,
 {
    uintptr_t mmu_idx = get_mmuidx(oi);
    MemOp mop = get_memop(oi);
+    int a_bits = get_alignment_bits(mop);
    uintptr_t index;
    CPUTLBEntry *tlbe;
    vaddr tlb_addr;
    void *hostaddr;
    CPUTLBEntryFull *full;
-    bool did_tlb_fill = false;

    tcg_debug_assert(mmu_idx < NB_MMU_MODES);

    /* Adjust the given return address.  */
    retaddr -= GETPC_ADJ;

+    /* Enforce guest required alignment.  */
+    if (unlikely(a_bits > 0 && (addr & ((1 << a_bits) - 1)))) {
+        /* ??? Maybe indicate atomic op to cpu_unaligned_access */
+        cpu_unaligned_access(cpu, addr, MMU_DATA_STORE,
+                             mmu_idx, retaddr);
+    }
+
+    /* Enforce qemu required alignment.  */
+    if (unlikely(addr & (size - 1))) {
+        /* We get here if guest alignment was not requested,
+           or was not enforced by cpu_unaligned_access above.
+           We might widen the access and emulate, but for now
+           mark an exception and exit the cpu loop.  */
+        goto stop_the_world;
+    }
+
    index = tlb_index(cpu, mmu_idx, addr);
    tlbe = tlb_entry(cpu, mmu_idx, addr);

@@ -1832,9 +1887,8 @@ static void *atomic_mmu_lookup(CPUState *cpu, vaddr addr, MemOpIdx oi,
    if (!tlb_hit(tlb_addr, addr)) {
        if (!victim_tlb_hit(cpu, mmu_idx, index, MMU_DATA_STORE,
                            addr & TARGET_PAGE_MASK)) {
-            tlb_fill_align(cpu, addr, MMU_DATA_STORE, mmu_idx,
-                           mop, size, false, retaddr);
-            did_tlb_fill = true;
+            tlb_fill(cpu, addr, size,
+                     MMU_DATA_STORE, mmu_idx, retaddr);
            index = tlb_index(cpu, mmu_idx, addr);
            tlbe = tlb_entry(cpu, mmu_idx, addr);
        }
@@ -1848,38 +1902,17 @@ static void *atomic_mmu_lookup(CPUState *cpu, vaddr addr, MemOpIdx oi,
     * but addr_read will only be -1 if PAGE_READ was unset.
     */
    if (unlikely(tlbe->addr_read == -1)) {
-        tlb_fill_align(cpu, addr, MMU_DATA_LOAD, mmu_idx,
-                       0, size, false, retaddr);
+        tlb_fill(cpu, addr, size, MMU_DATA_LOAD, mmu_idx, retaddr);
        /*
         * Since we don't support reads and writes to different
         * addresses, and we do have the proper page loaded for
-         * write, this shouldn't ever return.
-         */
-        g_assert_not_reached();
-    }
-
-    /* Enforce guest required alignment, if not handled by tlb_fill_align. */
-    if (!did_tlb_fill && (addr & ((1 << memop_alignment_bits(mop)) - 1))) {
-        cpu_unaligned_access(cpu, addr, MMU_DATA_STORE, mmu_idx, retaddr);
-    }
-
-    /* Enforce qemu required alignment.  */
-    if (unlikely(addr & (size - 1))) {
-        /*
-         * We get here if guest alignment was not requested, or was not
-         * enforced by cpu_unaligned_access or tlb_fill_align above.
-         * We might widen the access and emulate, but for now
-         * mark an exception and exit the cpu loop.
+         * write, this shouldn't ever return.  But just in case,
+         * handle via stop-the-world.
         */
        goto stop_the_world;
    }
-
-    /* Finish collecting tlb flags for both read and write. */
-    full = &cpu->neg.tlb.d[mmu_idx].fulltlb[index];
+    /* Collect tlb flags for read. */
    tlb_addr |= tlbe->addr_read;
-    tlb_addr &= TLB_FLAGS_MASK & ~TLB_FORCE_SLOW;
-    tlb_addr |= full->slow_flags[MMU_DATA_STORE];
-    tlb_addr |= full->slow_flags[MMU_DATA_LOAD];

    /* Notice an IO access or a needs-MMU-lookup access */
    if (unlikely(tlb_addr & (TLB_MMIO | TLB_DISCARD_WRITE))) {
@@ -1889,12 +1922,13 @@ static void *atomic_mmu_lookup(CPUState *cpu, vaddr addr, MemOpIdx oi,
    }

    hostaddr = (void *)((uintptr_t)addr + tlbe->addend);
+    full = &cpu->neg.tlb.d[mmu_idx].fulltlb[index];

    if (unlikely(tlb_addr & TLB_NOTDIRTY)) {
        notdirty_write(cpu, addr, size, full, retaddr);
    }

-    if (unlikely(tlb_addr & TLB_WATCHPOINT)) {
+    if (unlikely(tlb_addr & TLB_FORCE_SLOW)) {
        int wp_flags = 0;

        if (full->slow_flags[MMU_DATA_STORE] & TLB_WATCHPOINT) {
@@ -1903,8 +1937,10 @@ static void *atomic_mmu_lookup(CPUState *cpu, vaddr addr, MemOpIdx oi,
        if (full->slow_flags[MMU_DATA_LOAD] & TLB_WATCHPOINT) {
            wp_flags |= BP_MEM_READ;
        }
-        cpu_check_watchpoint(cpu, addr, size,
-                             full->attrs, wp_flags, retaddr);
+        if (wp_flags) {
+            cpu_check_watchpoint(cpu, addr, size,
+                                 full->attrs, wp_flags, retaddr);
+        }
    }

    return hostaddr;
@@ -2331,7 +2367,7 @@ static uint8_t do_ld1_mmu(CPUState *cpu, vaddr addr, MemOpIdx oi,
    MMULookupLocals l;
    bool crosspage;

-    cpu_req_mo(cpu, TCG_MO_LD_LD | TCG_MO_ST_LD);
+    cpu_req_mo(TCG_MO_LD_LD | TCG_MO_ST_LD);
    crosspage = mmu_lookup(cpu, addr, oi, ra, access_type, &l);
    tcg_debug_assert(!crosspage);

@@ -2346,7 +2382,7 @@ static uint16_t do_ld2_mmu(CPUState *cpu, vaddr addr, MemOpIdx oi,
    uint16_t ret;
    uint8_t a, b;

-    cpu_req_mo(cpu, TCG_MO_LD_LD | TCG_MO_ST_LD);
+    cpu_req_mo(TCG_MO_LD_LD | TCG_MO_ST_LD);
    crosspage = mmu_lookup(cpu, addr, oi, ra, access_type, &l);
    if (likely(!crosspage)) {
        return do_ld_2(cpu, &l.page[0], l.mmu_idx, access_type, l.memop, ra);
@@ -2370,7 +2406,7 @@ static uint32_t do_ld4_mmu(CPUState *cpu, vaddr addr, MemOpIdx oi,
    bool crosspage;
    uint32_t ret;

-    cpu_req_mo(cpu, TCG_MO_LD_LD | TCG_MO_ST_LD);
+    cpu_req_mo(TCG_MO_LD_LD | TCG_MO_ST_LD);
    crosspage = mmu_lookup(cpu, addr, oi, ra, access_type, &l);
    if (likely(!crosspage)) {
        return do_ld_4(cpu, &l.page[0], l.mmu_idx, access_type, l.memop, ra);
@@ -2391,7 +2427,7 @@ static uint64_t do_ld8_mmu(CPUState *cpu, vaddr addr, MemOpIdx oi,
    bool crosspage;
    uint64_t ret;

-    cpu_req_mo(cpu, TCG_MO_LD_LD | TCG_MO_ST_LD);
+    cpu_req_mo(TCG_MO_LD_LD | TCG_MO_ST_LD);
    crosspage = mmu_lookup(cpu, addr, oi, ra, access_type, &l);
    if (likely(!crosspage)) {
        return do_ld_8(cpu, &l.page[0], l.mmu_idx, access_type, l.memop, ra);
@@ -2414,7 +2450,7 @@ static Int128 do_ld16_mmu(CPUState *cpu, vaddr addr,
    Int128 ret;
    int first;

-    cpu_req_mo(cpu, TCG_MO_LD_LD | TCG_MO_ST_LD);
+    cpu_req_mo(TCG_MO_LD_LD | TCG_MO_ST_LD);
    crosspage = mmu_lookup(cpu, addr, oi, ra, MMU_DATA_LOAD, &l);
    if (likely(!crosspage)) {
        if (unlikely(l.page[0].flags & TLB_MMIO)) {
@@ -2742,7 +2778,7 @@ static void do_st1_mmu(CPUState *cpu, vaddr addr, uint8_t val,
    MMULookupLocals l;
    bool crosspage;

-    cpu_req_mo(cpu, TCG_MO_LD_ST | TCG_MO_ST_ST);
+    cpu_req_mo(TCG_MO_LD_ST | TCG_MO_ST_ST);
    crosspage = mmu_lookup(cpu, addr, oi, ra, MMU_DATA_STORE, &l);
    tcg_debug_assert(!crosspage);

@@ -2756,7 +2792,7 @@ static void do_st2_mmu(CPUState *cpu, vaddr addr, uint16_t val,
    bool crosspage;
    uint8_t a, b;

-    cpu_req_mo(cpu, TCG_MO_LD_ST | TCG_MO_ST_ST);
+    cpu_req_mo(TCG_MO_LD_ST | TCG_MO_ST_ST);
    crosspage = mmu_lookup(cpu, addr, oi, ra, MMU_DATA_STORE, &l);
    if (likely(!crosspage)) {
        do_st_2(cpu, &l.page[0], val, l.mmu_idx, l.memop, ra);
@@ -2778,7 +2814,7 @@ static void do_st4_mmu(CPUState *cpu, vaddr addr, uint32_t val,
    MMULookupLocals l;
    bool crosspage;

-    cpu_req_mo(cpu, TCG_MO_LD_ST | TCG_MO_ST_ST);
+    cpu_req_mo(TCG_MO_LD_ST | TCG_MO_ST_ST);
    crosspage = mmu_lookup(cpu, addr, oi, ra, MMU_DATA_STORE, &l);
    if (likely(!crosspage)) {
        do_st_4(cpu, &l.page[0], val, l.mmu_idx, l.memop, ra);
@@ -2799,7 +2835,7 @@ static void do_st8_mmu(CPUState *cpu, vaddr addr, uint64_t val,
    MMULookupLocals l;
    bool crosspage;

-    cpu_req_mo(cpu, TCG_MO_LD_ST | TCG_MO_ST_ST);
+    cpu_req_mo(TCG_MO_LD_ST | TCG_MO_ST_ST);
    crosspage = mmu_lookup(cpu, addr, oi, ra, MMU_DATA_STORE, &l);
    if (likely(!crosspage)) {
        do_st_8(cpu, &l.page[0], val, l.mmu_idx, l.memop, ra);
@@ -2822,7 +2858,7 @@ static void do_st16_mmu(CPUState *cpu, vaddr addr, Int128 val,
    uint64_t a, b;
    int first;

-    cpu_req_mo(cpu, TCG_MO_LD_ST | TCG_MO_ST_ST);
+    cpu_req_mo(TCG_MO_LD_ST | TCG_MO_ST_ST);
    crosspage = mmu_lookup(cpu, addr, oi, ra, MMU_DATA_STORE, &l);
    if (likely(!crosspage)) {
        if (unlikely(l.page[0].flags & TLB_MMIO)) {
@@ -2907,45 +2943,54 @@ static void do_st16_mmu(CPUState *cpu, vaddr addr, Int128 val,

 /* Code access functions.  */

-uint8_t cpu_ldb_code_mmu(CPUArchState *env, vaddr addr,
+uint32_t cpu_ldub_code(CPUArchState *env, abi_ptr addr)
+{
+    CPUState *cs = env_cpu(env);
+    MemOpIdx oi = make_memop_idx(MO_UB, cpu_mmu_index(cs, true));
+    return do_ld1_mmu(cs, addr, oi, 0, MMU_INST_FETCH);
+}
+
+uint32_t cpu_lduw_code(CPUArchState *env, abi_ptr addr)
+{
+    CPUState *cs = env_cpu(env);
+    MemOpIdx oi = make_memop_idx(MO_TEUW, cpu_mmu_index(cs, true));
+    return do_ld2_mmu(cs, addr, oi, 0, MMU_INST_FETCH);
+}
+
+uint32_t cpu_ldl_code(CPUArchState *env, abi_ptr addr)
+{
+    CPUState *cs = env_cpu(env);
+    MemOpIdx oi = make_memop_idx(MO_TEUL, cpu_mmu_index(cs, true));
+    return do_ld4_mmu(cs, addr, oi, 0, MMU_INST_FETCH);
+}
+
+uint64_t cpu_ldq_code(CPUArchState *env, abi_ptr addr)
+{
+    CPUState *cs = env_cpu(env);
+    MemOpIdx oi = make_memop_idx(MO_TEUQ, cpu_mmu_index(cs, true));
+    return do_ld8_mmu(cs, addr, oi, 0, MMU_INST_FETCH);
+}
+
+uint8_t cpu_ldb_code_mmu(CPUArchState *env, abi_ptr addr,
                         MemOpIdx oi, uintptr_t retaddr)
 {
    return do_ld1_mmu(env_cpu(env), addr, oi, retaddr, MMU_INST_FETCH);
 }

-uint16_t cpu_ldw_code_mmu(CPUArchState *env, vaddr addr,
+uint16_t cpu_ldw_code_mmu(CPUArchState *env, abi_ptr addr,
                          MemOpIdx oi, uintptr_t retaddr)
 {
    return do_ld2_mmu(env_cpu(env), addr, oi, retaddr, MMU_INST_FETCH);
 }

-uint32_t cpu_ldl_code_mmu(CPUArchState *env, vaddr addr,
+uint32_t cpu_ldl_code_mmu(CPUArchState *env, abi_ptr addr,
                          MemOpIdx oi, uintptr_t retaddr)
 {
    return do_ld4_mmu(env_cpu(env), addr, oi, retaddr, MMU_INST_FETCH);
 }

-uint64_t cpu_ldq_code_mmu(CPUArchState *env, vaddr addr,
+uint64_t cpu_ldq_code_mmu(CPUArchState *env, abi_ptr addr,
                          MemOpIdx oi, uintptr_t retaddr)
 {
    return do_ld8_mmu(env_cpu(env), addr, oi, retaddr, MMU_INST_FETCH);
 }
-
-/*
- * Common pointer_wrap implementations.
- */
-
-/*
- * To be used for strict alignment targets.
- * Because no accesses are unaligned, no accesses wrap either.
- */
-vaddr cpu_pointer_wrap_notreached(CPUState *cs, int idx, vaddr res, vaddr base)
-{
-    g_assert_not_reached();
-}
-
-/* To be used for strict 32-bit targets. */
-vaddr cpu_pointer_wrap_uint32(CPUState *cs, int idx, vaddr res, vaddr base)
-{
-    return (uint32_t)res;
-}
@@ -27,16 +27,17 @@
 #include "migration/vmstate.h"
 #include "qapi/error.h"
 #include "qemu/error-report.h"
-#include "system/cpus.h"
-#include "system/qtest.h"
+#include "sysemu/cpus.h"
+#include "sysemu/qtest.h"
 #include "qemu/main-loop.h"
 #include "qemu/option.h"
 #include "qemu/seqlock.h"
-#include "system/replay.h"
-#include "system/runstate.h"
+#include "sysemu/replay.h"
+#include "sysemu/runstate.h"
 #include "hw/core/cpu.h"
-#include "exec/icount.h"
-#include "system/cpu-timers-internal.h"
+#include "sysemu/cpu-timers.h"
+#include "sysemu/cpu-throttle.h"
+#include "sysemu/cpu-timers-internal.h"

 /*
 * ICOUNT: Instruction Counter
@@ -48,8 +49,6 @@ static bool icount_sleep = true;
 /* Arbitrarily pick 1MIPS as the minimum allowable speed.  */
 #define MAX_ICOUNT_SHIFT 10

-bool icount_align_option;
-
 /* Do not count executed instructions */
 ICountMode use_icount = ICOUNT_DISABLED;

@@ -337,8 +336,10 @@ void icount_start_warp_timer(void)
    deadline = qemu_clock_deadline_ns_all(QEMU_CLOCK_VIRTUAL,
                                          ~QEMU_TIMER_ATTR_EXTERNAL);
    if (deadline < 0) {
-        if (!icount_sleep) {
-            warn_report_once("icount sleep disabled and no active timers");
+        static bool notified;
+        if (!icount_sleep && !notified) {
+            warn_report("icount sleep disabled and no active timers");
+            notified = true;
        }
        return;
    }
@@ -9,136 +9,18 @@
 #ifndef ACCEL_TCG_INTERNAL_COMMON_H
 #define ACCEL_TCG_INTERNAL_COMMON_H

-#include "exec/cpu-common.h"
 #include "exec/translation-block.h"
-#include "exec/mmap-lock.h"
-#include "accel/tcg/tb-cpu-state.h"

 extern int64_t max_delay;
 extern int64_t max_advance;

-extern bool one_insn_per_tb;
-
-extern bool icount_align_option;
-
 /*
 * Return true if CS is not running in parallel with other cpus, either
 * because there are no other cpus or we are within an exclusive context.
 */
 static inline bool cpu_in_serial_context(CPUState *cs)
 {
-    return !tcg_cflags_has(cs, CF_PARALLEL) || cpu_in_exclusive_context(cs);
+    return !(cs->tcg_cflags & CF_PARALLEL) || cpu_in_exclusive_context(cs);
 }

-/**
- * cpu_plugin_mem_cbs_enabled() - are plugin memory callbacks enabled?
- * @cs: CPUState pointer
- *
- * The memory callbacks are installed if a plugin has instrumented an
- * instruction for memory. This can be useful to know if you want to
- * force a slow path for a series of memory accesses.
- */
-static inline bool cpu_plugin_mem_cbs_enabled(const CPUState *cpu)
-{
-#ifdef CONFIG_PLUGIN
-    return !!cpu->neg.plugin_mem_cbs;
-#else
-    return false;
-#endif
-}
-
-TranslationBlock *tb_gen_code(CPUState *cpu, TCGTBCPUState s);
-void page_init(void);
-void tb_htable_init(void);
-void tb_reset_jump(TranslationBlock *tb, int n);
-TranslationBlock *tb_link_page(TranslationBlock *tb);
-void cpu_restore_state_from_tb(CPUState *cpu, TranslationBlock *tb,
-                               uintptr_t host_pc);
-
-/**
- * tlb_init - initialize a CPU's TLB
- * @cpu: CPU whose TLB should be initialized
- */
-void tlb_init(CPUState *cpu);
-/**
- * tlb_destroy - destroy a CPU's TLB
- * @cpu: CPU whose TLB should be destroyed
- */
-void tlb_destroy(CPUState *cpu);
-
-bool tcg_exec_realizefn(CPUState *cpu, Error **errp);
-void tcg_exec_unrealizefn(CPUState *cpu);
-
-/* current cflags for hashing/comparison */
-uint32_t curr_cflags(CPUState *cpu);
-
-void tb_check_watchpoint(CPUState *cpu, uintptr_t retaddr);
-
-/**
- * get_page_addr_code_hostp()
- * @env: CPUArchState
- * @addr: guest virtual address of guest code
- *
- * See get_page_addr_code() (full-system version) for documentation on the
- * return value.
- *
- * Sets *@hostp (when @hostp is non-NULL) as follows.
- * If the return value is -1, sets *@hostp to NULL. Otherwise, sets *@hostp
- * to the host address where @addr's content is kept.
- *
- * Note: this function can trigger an exception.
- */
-tb_page_addr_t get_page_addr_code_hostp(CPUArchState *env, vaddr addr,
-                                        void **hostp);
-
-/**
- * get_page_addr_code()
- * @env: CPUArchState
- * @addr: guest virtual address of guest code
- *
- * If we cannot translate and execute from the entire RAM page, or if
- * the region is not backed by RAM, returns -1. Otherwise, returns the
- * ram_addr_t corresponding to the guest code at @addr.
- *
- * Note: this function can trigger an exception.
- */
-static inline tb_page_addr_t get_page_addr_code(CPUArchState *env,
-                                                vaddr addr)
-{
-    return get_page_addr_code_hostp(env, addr, NULL);
-}
-
-/*
- * Access to the various translations structures need to be serialised
- * via locks for consistency.  In user-mode emulation access to the
- * memory related structures are protected with mmap_lock.
- * In !user-mode we use per-page locks.
- */
-#ifdef CONFIG_USER_ONLY
-#define assert_memory_lock() tcg_debug_assert(have_mmap_lock())
-#else
-#define assert_memory_lock()
-#endif
-
-#if defined(CONFIG_SOFTMMU) && defined(CONFIG_DEBUG_TCG)
-void assert_no_pages_locked(void);
-#else
-static inline void assert_no_pages_locked(void) { }
-#endif
-
-#ifdef CONFIG_USER_ONLY
-static inline void page_table_config_init(void) { }
-#else
-void page_table_config_init(void);
-#endif
-
-#ifndef CONFIG_USER_ONLY
-G_NORETURN void cpu_io_recompile(CPUState *cpu, uintptr_t retaddr);
-#endif /* CONFIG_USER_ONLY */
-
-void tb_phys_invalidate(TranslationBlock *tb, tb_page_addr_t page_addr);
-void tb_set_jmp_target(TranslationBlock *tb, int n, uintptr_t addr);
-
-void tcg_get_stats(AccelState *accel, GString *buf);
-
 #endif
@@ -0,0 +1,132 @@
+/*
+ * Internal execution defines for qemu (target specific)
+ *
+ *  Copyright (c) 2003 Fabrice Bellard
+ *
+ * SPDX-License-Identifier: LGPL-2.1-or-later
+ */
+
+#ifndef ACCEL_TCG_INTERNAL_TARGET_H
+#define ACCEL_TCG_INTERNAL_TARGET_H
+
+#include "exec/exec-all.h"
+#include "exec/translate-all.h"
+
+/*
+ * Access to the various translations structures need to be serialised
+ * via locks for consistency.  In user-mode emulation access to the
+ * memory related structures are protected with mmap_lock.
+ * In !user-mode we use per-page locks.
+ */
+#ifdef CONFIG_USER_ONLY
+#define assert_memory_lock() tcg_debug_assert(have_mmap_lock())
+#else
+#define assert_memory_lock()
+#endif
+
+#if defined(CONFIG_SOFTMMU) && defined(CONFIG_DEBUG_TCG)
+void assert_no_pages_locked(void);
+#else
+static inline void assert_no_pages_locked(void) { }
+#endif
+
+#ifdef CONFIG_USER_ONLY
+static inline void page_table_config_init(void) { }
+#else
+void page_table_config_init(void);
+#endif
+
+#ifdef CONFIG_USER_ONLY
+/*
+ * For user-only, page_protect sets the page read-only.
+ * Since most execution is already on read-only pages, and we'd need to
+ * account for other TBs on the same page, defer undoing any page protection
+ * until we receive the write fault.
+ */
+static inline void tb_lock_page0(tb_page_addr_t p0)
+{
+    page_protect(p0);
+}
+
+static inline void tb_lock_page1(tb_page_addr_t p0, tb_page_addr_t p1)
+{
+    page_protect(p1);
+}
+
+static inline void tb_unlock_page1(tb_page_addr_t p0, tb_page_addr_t p1) { }
+static inline void tb_unlock_pages(TranslationBlock *tb) { }
+#else
+void tb_lock_page0(tb_page_addr_t);
+void tb_lock_page1(tb_page_addr_t, tb_page_addr_t);
+void tb_unlock_page1(tb_page_addr_t, tb_page_addr_t);
+void tb_unlock_pages(TranslationBlock *);
+#endif
+
+#ifdef CONFIG_SOFTMMU
+void tb_invalidate_phys_range_fast(ram_addr_t ram_addr,
+                                   unsigned size,
+                                   uintptr_t retaddr);
+G_NORETURN void cpu_io_recompile(CPUState *cpu, uintptr_t retaddr);
+#endif /* CONFIG_SOFTMMU */
+
+TranslationBlock *tb_gen_code(CPUState *cpu, vaddr pc,
+                              uint64_t cs_base, uint32_t flags,
+                              int cflags);
+void page_init(void);
+void tb_htable_init(void);
+void tb_reset_jump(TranslationBlock *tb, int n);
+TranslationBlock *tb_link_page(TranslationBlock *tb);
+bool tb_invalidate_phys_page_unwind(tb_page_addr_t addr, uintptr_t pc);
+void cpu_restore_state_from_tb(CPUState *cpu, TranslationBlock *tb,
+                               uintptr_t host_pc);
+
+bool tcg_exec_realizefn(CPUState *cpu, Error **errp);
+void tcg_exec_unrealizefn(CPUState *cpu);
+
+/* Return the current PC from CPU, which may be cached in TB. */
+static inline vaddr log_pc(CPUState *cpu, const TranslationBlock *tb)
+{
+    if (tb_cflags(tb) & CF_PCREL) {
+        return cpu->cc->get_pc(cpu);
+    } else {
+        return tb->pc;
+    }
+}
+
+extern bool one_insn_per_tb;
+
+/**
+ * tcg_req_mo:
+ * @type: TCGBar
+ *
+ * Filter @type to the barrier that is required for the guest
+ * memory ordering vs the host memory ordering.  A non-zero
+ * result indicates that some barrier is required.
+ *
+ * If TCG_GUEST_DEFAULT_MO is not defined, assume that the
+ * guest requires strict ordering.
+ *
+ * This is a macro so that it's constant even without optimization.
+ */
+#ifdef TCG_GUEST_DEFAULT_MO
+# define tcg_req_mo(type) \
+    ((type) & TCG_GUEST_DEFAULT_MO & ~TCG_TARGET_DEFAULT_MO)
+#else
+# define tcg_req_mo(type) ((type) & ~TCG_TARGET_DEFAULT_MO)
+#endif
+
+/**
+ * cpu_req_mo:
+ * @type: TCGBar
+ *
+ * If tcg_req_mo indicates a barrier for @type is required
+ * for the guest memory model, issue a host memory barrier.
+ */
+#define cpu_req_mo(type)          \
+    do {                          \
+        if (tcg_req_mo(type)) {   \
+            smp_mb();             \
+        }                         \
+    } while (0)
+
+#endif /* ACCEL_TCG_INTERNAL_H */
@@ -9,8 +9,8 @@
 * See the COPYING file in the top-level directory.
 */

-#include "host/load-extract-al16-al8.h.inc"
-#include "host/store-insert-al16.h.inc"
+#include "host/load-extract-al16-al8.h"
+#include "host/store-insert-al16.h"

 #ifdef CONFIG_ATOMIC64
 # define HAVE_al8          true
@@ -168,7 +168,6 @@ static uint64_t load_atomic8_or_exit(CPUState *cpu, uintptr_t ra, void *pv)
 #endif

    /* Ultimate fallback: re-execute in serial context. */
-    trace_load_atom8_or_exit_fallback(ra);
    cpu_loop_exit_atomic(cpu, ra);
 }

@@ -213,7 +212,6 @@ static Int128 load_atomic16_or_exit(CPUState *cpu, uintptr_t ra, void *pv)
    }

    /* Ultimate fallback: re-execute in serial context. */
-    trace_load_atom16_or_exit_fallback(ra);
    cpu_loop_exit_atomic(cpu, ra);
 }

@@ -521,7 +519,6 @@ static uint64_t load_atom_8(CPUState *cpu, uintptr_t ra,
        if (HAVE_al8) {
            return load_atom_extract_al8x2(pv);
        }
-        trace_load_atom8_fallback(memop, ra);
        cpu_loop_exit_atomic(cpu, ra);
    default:
        g_assert_not_reached();
@@ -566,7 +563,6 @@ static Int128 load_atom_16(CPUState *cpu, uintptr_t ra,
        break;
    case MO_64:
        if (!HAVE_al8) {
-            trace_load_atom16_fallback(memop, ra);
            cpu_loop_exit_atomic(cpu, ra);
        }
        a = load_atomic8(pv);
@@ -574,7 +570,6 @@ static Int128 load_atom_16(CPUState *cpu, uintptr_t ra,
        break;
    case -MO_64:
        if (!HAVE_al8) {
-            trace_load_atom16_fallback(memop, ra);
            cpu_loop_exit_atomic(cpu, ra);
        }
        a = load_atom_extract_al8x2(pv);
@@ -902,7 +897,6 @@ static void store_atom_2(CPUState *cpu, uintptr_t ra,
        g_assert_not_reached();
    }

-    trace_store_atom2_fallback(memop, ra);
    cpu_loop_exit_atomic(cpu, ra);
 }

@@ -967,7 +961,6 @@ static void store_atom_4(CPUState *cpu, uintptr_t ra,
                return;
            }
        }
-        trace_store_atom4_fallback(memop, ra);
        cpu_loop_exit_atomic(cpu, ra);
    default:
        g_assert_not_reached();
@@ -1036,7 +1029,6 @@ static void store_atom_8(CPUState *cpu, uintptr_t ra,
    default:
        g_assert_not_reached();
    }
-    trace_store_atom8_fallback(memop, ra);
    cpu_loop_exit_atomic(cpu, ra);
 }

@@ -1115,6 +1107,5 @@ static void store_atom_16(CPUState *cpu, uintptr_t ra,
    default:
        g_assert_not_reached();
    }
-    trace_store_atom16_fallback(memop, ra);
    cpu_loop_exit_atomic(cpu, ra);
 }
@@ -123,69 +123,62 @@ void helper_st_i128(CPUArchState *env, uint64_t addr, Int128 val, MemOpIdx oi)
 * Load helpers for cpu_ldst.h
 */

-static void plugin_load_cb(CPUArchState *env, vaddr addr,
-                           uint64_t value_low,
-                           uint64_t value_high,
-                           MemOpIdx oi)
+static void plugin_load_cb(CPUArchState *env, abi_ptr addr, MemOpIdx oi)
 {
-    if (cpu_plugin_mem_cbs_enabled(env_cpu(env))) {
-        qemu_plugin_vcpu_mem_cb(env_cpu(env), addr,
-                                value_low, value_high,
-                                oi, QEMU_PLUGIN_MEM_R);
-    }
+    qemu_plugin_vcpu_mem_cb(env_cpu(env), addr, oi, QEMU_PLUGIN_MEM_R);
 }

-uint8_t cpu_ldb_mmu(CPUArchState *env, vaddr addr, MemOpIdx oi, uintptr_t ra)
+uint8_t cpu_ldb_mmu(CPUArchState *env, abi_ptr addr, MemOpIdx oi, uintptr_t ra)
 {
    uint8_t ret;

    tcg_debug_assert((get_memop(oi) & MO_SIZE) == MO_UB);
    ret = do_ld1_mmu(env_cpu(env), addr, oi, ra, MMU_DATA_LOAD);
-    plugin_load_cb(env, addr, ret, 0, oi);
+    plugin_load_cb(env, addr, oi);
    return ret;
 }

-uint16_t cpu_ldw_mmu(CPUArchState *env, vaddr addr,
+uint16_t cpu_ldw_mmu(CPUArchState *env, abi_ptr addr,
                     MemOpIdx oi, uintptr_t ra)
 {
    uint16_t ret;

    tcg_debug_assert((get_memop(oi) & MO_SIZE) == MO_16);
    ret = do_ld2_mmu(env_cpu(env), addr, oi, ra, MMU_DATA_LOAD);
-    plugin_load_cb(env, addr, ret, 0, oi);
+    plugin_load_cb(env, addr, oi);
    return ret;
 }

-uint32_t cpu_ldl_mmu(CPUArchState *env, vaddr addr,
+uint32_t cpu_ldl_mmu(CPUArchState *env, abi_ptr addr,
                     MemOpIdx oi, uintptr_t ra)
 {
    uint32_t ret;

    tcg_debug_assert((get_memop(oi) & MO_SIZE) == MO_32);
    ret = do_ld4_mmu(env_cpu(env), addr, oi, ra, MMU_DATA_LOAD);
-    plugin_load_cb(env, addr, ret, 0, oi);
+    plugin_load_cb(env, addr, oi);
    return ret;
 }

-uint64_t cpu_ldq_mmu(CPUArchState *env, vaddr addr,
+uint64_t cpu_ldq_mmu(CPUArchState *env, abi_ptr addr,
                     MemOpIdx oi, uintptr_t ra)
 {
    uint64_t ret;

    tcg_debug_assert((get_memop(oi) & MO_SIZE) == MO_64);
    ret = do_ld8_mmu(env_cpu(env), addr, oi, ra, MMU_DATA_LOAD);
-    plugin_load_cb(env, addr, ret, 0, oi);
+    plugin_load_cb(env, addr, oi);
    return ret;
 }

-Int128 cpu_ld16_mmu(CPUArchState *env, vaddr addr,
+Int128 cpu_ld16_mmu(CPUArchState *env, abi_ptr addr,
                    MemOpIdx oi, uintptr_t ra)
 {
    Int128 ret;

    tcg_debug_assert((get_memop(oi) & MO_SIZE) == MO_128);
    ret = do_ld16_mmu(env_cpu(env), addr, oi, ra);
-    plugin_load_cb(env, addr, int128_getlo(ret), int128_gethi(ret), oi);
+    plugin_load_cb(env, addr, oi);
    return ret;
 }

@@ -193,53 +186,361 @@ Int128 cpu_ld16_mmu(CPUArchState *env, vaddr addr,
 * Store helpers for cpu_ldst.h
 */

-static void plugin_store_cb(CPUArchState *env, vaddr addr,
-                            uint64_t value_low,
-                            uint64_t value_high,
-                            MemOpIdx oi)
+static void plugin_store_cb(CPUArchState *env, abi_ptr addr, MemOpIdx oi)
 {
-    if (cpu_plugin_mem_cbs_enabled(env_cpu(env))) {
-        qemu_plugin_vcpu_mem_cb(env_cpu(env), addr,
-                                value_low, value_high,
-                                oi, QEMU_PLUGIN_MEM_W);
-    }
+    qemu_plugin_vcpu_mem_cb(env_cpu(env), addr, oi, QEMU_PLUGIN_MEM_W);
 }

-void cpu_stb_mmu(CPUArchState *env, vaddr addr, uint8_t val,
+void cpu_stb_mmu(CPUArchState *env, abi_ptr addr, uint8_t val,
                 MemOpIdx oi, uintptr_t retaddr)
 {
    helper_stb_mmu(env, addr, val, oi, retaddr);
-    plugin_store_cb(env, addr, val, 0, oi);
+    plugin_store_cb(env, addr, oi);
 }

-void cpu_stw_mmu(CPUArchState *env, vaddr addr, uint16_t val,
+void cpu_stw_mmu(CPUArchState *env, abi_ptr addr, uint16_t val,
                 MemOpIdx oi, uintptr_t retaddr)
 {
    tcg_debug_assert((get_memop(oi) & MO_SIZE) == MO_16);
    do_st2_mmu(env_cpu(env), addr, val, oi, retaddr);
-    plugin_store_cb(env, addr, val, 0, oi);
+    plugin_store_cb(env, addr, oi);
 }

-void cpu_stl_mmu(CPUArchState *env, vaddr addr, uint32_t val,
+void cpu_stl_mmu(CPUArchState *env, abi_ptr addr, uint32_t val,
                    MemOpIdx oi, uintptr_t retaddr)
 {
    tcg_debug_assert((get_memop(oi) & MO_SIZE) == MO_32);
    do_st4_mmu(env_cpu(env), addr, val, oi, retaddr);
-    plugin_store_cb(env, addr, val, 0, oi);
+    plugin_store_cb(env, addr, oi);
 }

-void cpu_stq_mmu(CPUArchState *env, vaddr addr, uint64_t val,
+void cpu_stq_mmu(CPUArchState *env, abi_ptr addr, uint64_t val,
                 MemOpIdx oi, uintptr_t retaddr)
 {
    tcg_debug_assert((get_memop(oi) & MO_SIZE) == MO_64);
    do_st8_mmu(env_cpu(env), addr, val, oi, retaddr);
-    plugin_store_cb(env, addr, val, 0, oi);
+    plugin_store_cb(env, addr, oi);
 }

-void cpu_st16_mmu(CPUArchState *env, vaddr addr, Int128 val,
+void cpu_st16_mmu(CPUArchState *env, abi_ptr addr, Int128 val,
                  MemOpIdx oi, uintptr_t retaddr)
 {
    tcg_debug_assert((get_memop(oi) & MO_SIZE) == MO_128);
    do_st16_mmu(env_cpu(env), addr, val, oi, retaddr);
-    plugin_store_cb(env, addr, int128_getlo(val), int128_gethi(val), oi);
+    plugin_store_cb(env, addr, oi);
+}
+
+/*
+ * Wrappers of the above
+ */
+
+uint32_t cpu_ldub_mmuidx_ra(CPUArchState *env, abi_ptr addr,
+                            int mmu_idx, uintptr_t ra)
+{
+    MemOpIdx oi = make_memop_idx(MO_UB, mmu_idx);
+    return cpu_ldb_mmu(env, addr, oi, ra);
+}
+
+int cpu_ldsb_mmuidx_ra(CPUArchState *env, abi_ptr addr,
+                       int mmu_idx, uintptr_t ra)
+{
+    return (int8_t)cpu_ldub_mmuidx_ra(env, addr, mmu_idx, ra);
+}
+
+uint32_t cpu_lduw_be_mmuidx_ra(CPUArchState *env, abi_ptr addr,
+                               int mmu_idx, uintptr_t ra)
+{
+    MemOpIdx oi = make_memop_idx(MO_BEUW | MO_UNALN, mmu_idx);
+    return cpu_ldw_mmu(env, addr, oi, ra);
+}
+
+int cpu_ldsw_be_mmuidx_ra(CPUArchState *env, abi_ptr addr,
+                          int mmu_idx, uintptr_t ra)
+{
+    return (int16_t)cpu_lduw_be_mmuidx_ra(env, addr, mmu_idx, ra);
+}
+
+uint32_t cpu_ldl_be_mmuidx_ra(CPUArchState *env, abi_ptr addr,
+                              int mmu_idx, uintptr_t ra)
+{
+    MemOpIdx oi = make_memop_idx(MO_BEUL | MO_UNALN, mmu_idx);
+    return cpu_ldl_mmu(env, addr, oi, ra);
+}
+
+uint64_t cpu_ldq_be_mmuidx_ra(CPUArchState *env, abi_ptr addr,
+                              int mmu_idx, uintptr_t ra)
+{
+    MemOpIdx oi = make_memop_idx(MO_BEUQ | MO_UNALN, mmu_idx);
+    return cpu_ldq_mmu(env, addr, oi, ra);
+}
+
+uint32_t cpu_lduw_le_mmuidx_ra(CPUArchState *env, abi_ptr addr,
+                               int mmu_idx, uintptr_t ra)
+{
+    MemOpIdx oi = make_memop_idx(MO_LEUW | MO_UNALN, mmu_idx);
+    return cpu_ldw_mmu(env, addr, oi, ra);
+}
+
+int cpu_ldsw_le_mmuidx_ra(CPUArchState *env, abi_ptr addr,
+                          int mmu_idx, uintptr_t ra)
+{
+    return (int16_t)cpu_lduw_le_mmuidx_ra(env, addr, mmu_idx, ra);
+}
+
+uint32_t cpu_ldl_le_mmuidx_ra(CPUArchState *env, abi_ptr addr,
+                              int mmu_idx, uintptr_t ra)
+{
+    MemOpIdx oi = make_memop_idx(MO_LEUL | MO_UNALN, mmu_idx);
+    return cpu_ldl_mmu(env, addr, oi, ra);
+}
+
+uint64_t cpu_ldq_le_mmuidx_ra(CPUArchState *env, abi_ptr addr,
+                              int mmu_idx, uintptr_t ra)
+{
+    MemOpIdx oi = make_memop_idx(MO_LEUQ | MO_UNALN, mmu_idx);
+    return cpu_ldq_mmu(env, addr, oi, ra);
+}
+
+void cpu_stb_mmuidx_ra(CPUArchState *env, abi_ptr addr, uint32_t val,
+                       int mmu_idx, uintptr_t ra)
+{
+    MemOpIdx oi = make_memop_idx(MO_UB, mmu_idx);
+    cpu_stb_mmu(env, addr, val, oi, ra);
+}
+
+void cpu_stw_be_mmuidx_ra(CPUArchState *env, abi_ptr addr, uint32_t val,
+                          int mmu_idx, uintptr_t ra)
+{
+    MemOpIdx oi = make_memop_idx(MO_BEUW | MO_UNALN, mmu_idx);
+    cpu_stw_mmu(env, addr, val, oi, ra);
+}
+
+void cpu_stl_be_mmuidx_ra(CPUArchState *env, abi_ptr addr, uint32_t val,
+                          int mmu_idx, uintptr_t ra)
+{
+    MemOpIdx oi = make_memop_idx(MO_BEUL | MO_UNALN, mmu_idx);
+    cpu_stl_mmu(env, addr, val, oi, ra);
+}
+
+void cpu_stq_be_mmuidx_ra(CPUArchState *env, abi_ptr addr, uint64_t val,
+                          int mmu_idx, uintptr_t ra)
+{
+    MemOpIdx oi = make_memop_idx(MO_BEUQ | MO_UNALN, mmu_idx);
+    cpu_stq_mmu(env, addr, val, oi, ra);
+}
+
+void cpu_stw_le_mmuidx_ra(CPUArchState *env, abi_ptr addr, uint32_t val,
+                          int mmu_idx, uintptr_t ra)
+{
+    MemOpIdx oi = make_memop_idx(MO_LEUW | MO_UNALN, mmu_idx);
+    cpu_stw_mmu(env, addr, val, oi, ra);
+}
+
+void cpu_stl_le_mmuidx_ra(CPUArchState *env, abi_ptr addr, uint32_t val,
+                          int mmu_idx, uintptr_t ra)
+{
+    MemOpIdx oi = make_memop_idx(MO_LEUL | MO_UNALN, mmu_idx);
+    cpu_stl_mmu(env, addr, val, oi, ra);
+}
+
+void cpu_stq_le_mmuidx_ra(CPUArchState *env, abi_ptr addr, uint64_t val,
+                          int mmu_idx, uintptr_t ra)
+{
+    MemOpIdx oi = make_memop_idx(MO_LEUQ | MO_UNALN, mmu_idx);
+    cpu_stq_mmu(env, addr, val, oi, ra);
+}
+
+/*--------------------------*/
+
+uint32_t cpu_ldub_data_ra(CPUArchState *env, abi_ptr addr, uintptr_t ra)
+{
+    int mmu_index = cpu_mmu_index(env_cpu(env), false);
+    return cpu_ldub_mmuidx_ra(env, addr, mmu_index, ra);
+}
+
+int cpu_ldsb_data_ra(CPUArchState *env, abi_ptr addr, uintptr_t ra)
+{
+    return (int8_t)cpu_ldub_data_ra(env, addr, ra);
+}
+
+uint32_t cpu_lduw_be_data_ra(CPUArchState *env, abi_ptr addr, uintptr_t ra)
+{
+    int mmu_index = cpu_mmu_index(env_cpu(env), false);
+    return cpu_lduw_be_mmuidx_ra(env, addr, mmu_index, ra);
+}
+
+int cpu_ldsw_be_data_ra(CPUArchState *env, abi_ptr addr, uintptr_t ra)
+{
+    return (int16_t)cpu_lduw_be_data_ra(env, addr, ra);
+}
+
+uint32_t cpu_ldl_be_data_ra(CPUArchState *env, abi_ptr addr, uintptr_t ra)
+{
+    int mmu_index = cpu_mmu_index(env_cpu(env), false);
+    return cpu_ldl_be_mmuidx_ra(env, addr, mmu_index, ra);
+}
+
+uint64_t cpu_ldq_be_data_ra(CPUArchState *env, abi_ptr addr, uintptr_t ra)
+{
+    int mmu_index = cpu_mmu_index(env_cpu(env), false);
+    return cpu_ldq_be_mmuidx_ra(env, addr, mmu_index, ra);
+}
+
+uint32_t cpu_lduw_le_data_ra(CPUArchState *env, abi_ptr addr, uintptr_t ra)
+{
+    int mmu_index = cpu_mmu_index(env_cpu(env), false);
+    return cpu_lduw_le_mmuidx_ra(env, addr, mmu_index, ra);
+}
+
+int cpu_ldsw_le_data_ra(CPUArchState *env, abi_ptr addr, uintptr_t ra)
+{
+    return (int16_t)cpu_lduw_le_data_ra(env, addr, ra);
+}
+
+uint32_t cpu_ldl_le_data_ra(CPUArchState *env, abi_ptr addr, uintptr_t ra)
+{
+    int mmu_index = cpu_mmu_index(env_cpu(env), false);
+    return cpu_ldl_le_mmuidx_ra(env, addr, mmu_index, ra);
+}
+
+uint64_t cpu_ldq_le_data_ra(CPUArchState *env, abi_ptr addr, uintptr_t ra)
+{
+    int mmu_index = cpu_mmu_index(env_cpu(env), false);
+    return cpu_ldq_le_mmuidx_ra(env, addr, mmu_index, ra);
+}
+
+void cpu_stb_data_ra(CPUArchState *env, abi_ptr addr,
+                     uint32_t val, uintptr_t ra)
+{
+    int mmu_index = cpu_mmu_index(env_cpu(env), false);
+    cpu_stb_mmuidx_ra(env, addr, val, mmu_index, ra);
+}
+
+void cpu_stw_be_data_ra(CPUArchState *env, abi_ptr addr,
+                        uint32_t val, uintptr_t ra)
+{
+    int mmu_index = cpu_mmu_index(env_cpu(env), false);
+    cpu_stw_be_mmuidx_ra(env, addr, val, mmu_index, ra);
+}
+
+void cpu_stl_be_data_ra(CPUArchState *env, abi_ptr addr,
+                        uint32_t val, uintptr_t ra)
+{
+    int mmu_index = cpu_mmu_index(env_cpu(env), false);
+    cpu_stl_be_mmuidx_ra(env, addr, val, mmu_index, ra);
+}
+
+void cpu_stq_be_data_ra(CPUArchState *env, abi_ptr addr,
+                        uint64_t val, uintptr_t ra)
+{
+    int mmu_index = cpu_mmu_index(env_cpu(env), false);
+    cpu_stq_be_mmuidx_ra(env, addr, val, mmu_index, ra);
+}
+
+void cpu_stw_le_data_ra(CPUArchState *env, abi_ptr addr,
+                        uint32_t val, uintptr_t ra)
+{
+    int mmu_index = cpu_mmu_index(env_cpu(env), false);
+    cpu_stw_le_mmuidx_ra(env, addr, val, mmu_index, ra);
+}
+
+void cpu_stl_le_data_ra(CPUArchState *env, abi_ptr addr,
+                        uint32_t val, uintptr_t ra)
+{
+    int mmu_index = cpu_mmu_index(env_cpu(env), false);
+    cpu_stl_le_mmuidx_ra(env, addr, val, mmu_index, ra);
+}
+
+void cpu_stq_le_data_ra(CPUArchState *env, abi_ptr addr,
+                        uint64_t val, uintptr_t ra)
+{
+    int mmu_index = cpu_mmu_index(env_cpu(env), false);
+    cpu_stq_le_mmuidx_ra(env, addr, val, mmu_index, ra);
+}
+
+/*--------------------------*/
+
+uint32_t cpu_ldub_data(CPUArchState *env, abi_ptr addr)
+{
+    return cpu_ldub_data_ra(env, addr, 0);
+}
+
+int cpu_ldsb_data(CPUArchState *env, abi_ptr addr)
+{
+    return (int8_t)cpu_ldub_data(env, addr);
+}
+
+uint32_t cpu_lduw_be_data(CPUArchState *env, abi_ptr addr)
+{
+    return cpu_lduw_be_data_ra(env, addr, 0);
+}
+
+int cpu_ldsw_be_data(CPUArchState *env, abi_ptr addr)
+{
+    return (int16_t)cpu_lduw_be_data(env, addr);
+}
+
+uint32_t cpu_ldl_be_data(CPUArchState *env, abi_ptr addr)
+{
+    return cpu_ldl_be_data_ra(env, addr, 0);
+}
+
+uint64_t cpu_ldq_be_data(CPUArchState *env, abi_ptr addr)
+{
+    return cpu_ldq_be_data_ra(env, addr, 0);
+}
+
+uint32_t cpu_lduw_le_data(CPUArchState *env, abi_ptr addr)
+{
+    return cpu_lduw_le_data_ra(env, addr, 0);
+}
+
+int cpu_ldsw_le_data(CPUArchState *env, abi_ptr addr)
+{
+    return (int16_t)cpu_lduw_le_data(env, addr);
+}
+
+uint32_t cpu_ldl_le_data(CPUArchState *env, abi_ptr addr)
+{
+    return cpu_ldl_le_data_ra(env, addr, 0);
+}
+
+uint64_t cpu_ldq_le_data(CPUArchState *env, abi_ptr addr)
+{
+    return cpu_ldq_le_data_ra(env, addr, 0);
+}
+
+void cpu_stb_data(CPUArchState *env, abi_ptr addr, uint32_t val)
+{
+    cpu_stb_data_ra(env, addr, val, 0);
+}
+
+void cpu_stw_be_data(CPUArchState *env, abi_ptr addr, uint32_t val)
+{
+    cpu_stw_be_data_ra(env, addr, val, 0);
+}
+
+void cpu_stl_be_data(CPUArchState *env, abi_ptr addr, uint32_t val)
+{
+    cpu_stl_be_data_ra(env, addr, val, 0);
+}
+
+void cpu_stq_be_data(CPUArchState *env, abi_ptr addr, uint64_t val)
+{
+    cpu_stq_be_data_ra(env, addr, val, 0);
+}
+
+void cpu_stw_le_data(CPUArchState *env, abi_ptr addr, uint32_t val)
+{
+    cpu_stw_le_data_ra(env, addr, val, 0);
+}
+
+void cpu_stl_le_data(CPUArchState *env, abi_ptr addr, uint32_t val)
+{
+    cpu_stl_le_data_ra(env, addr, val, 0);
+}
+
+void cpu_stq_le_data(CPUArchState *env, abi_ptr addr, uint64_t val)
+{
+    cpu_stq_le_data_ra(env, addr, val, 0);
 }
@@ -1,39 +1,36 @@
-if not have_tcg
-   subdir_done()
-endif
-
-tcg_ss = ss.source_set()
-
-tcg_ss.add(files(
-  'cpu-exec.c',
+common_ss.add(when: 'CONFIG_TCG', if_true: files(
  'cpu-exec-common.c',
-  'tcg-runtime.c',
-  'tcg-runtime-gvec.c',
-  'tb-maint.c',
+))
+tcg_specific_ss = ss.source_set()
+tcg_specific_ss.add(files(
  'tcg-all.c',
-  'tcg-stats.c',
+  'cpu-exec.c',
+  'tb-maint.c',
+  'tcg-runtime-gvec.c',
+  'tcg-runtime.c',
  'translate-all.c',
  'translator.c',
 ))
+tcg_specific_ss.add(when: 'CONFIG_USER_ONLY', if_true: files('user-exec.c'))
+tcg_specific_ss.add(when: 'CONFIG_SYSTEM_ONLY', if_false: files('user-exec-stub.c'))
 if get_option('plugins')
-  tcg_ss.add(files('plugin-gen.c'))
+  tcg_specific_ss.add(files('plugin-gen.c'))
 endif
+specific_ss.add_all(when: 'CONFIG_TCG', if_true: tcg_specific_ss)

-user_ss.add_all(tcg_ss)
-system_ss.add_all(tcg_ss)
-
-user_ss.add(files(
-  'user-exec.c',
-  'user-exec-stub.c',
-))
-
-system_ss.add(files(
+specific_ss.add(when: ['CONFIG_SYSTEM_ONLY', 'CONFIG_TCG'], if_true: files(
  'cputlb.c',
-  'icount-common.c',
-  'monitor.c',
-  'tcg-accel-ops.c',
-  'tcg-accel-ops-icount.c',
-  'tcg-accel-ops-mttcg.c',
-  'tcg-accel-ops-rr.c',
  'watchpoint.c',
 ))
+
+system_ss.add(when: ['CONFIG_TCG'], if_true: files(
+  'icount-common.c',
+  'monitor.c',
+))
+
+tcg_module_ss.add(when: ['CONFIG_SYSTEM_ONLY', 'CONFIG_TCG'], if_true: files(
+  'tcg-accel-ops.c',
+  'tcg-accel-ops-mttcg.c',
+  'tcg-accel-ops-icount.c',
+  'tcg-accel-ops-rr.c',
+))
@@ -7,13 +7,197 @@
 */

 #include "qemu/osdep.h"
+#include "qemu/accel.h"
+#include "qemu/qht.h"
 #include "qapi/error.h"
 #include "qapi/type-helpers.h"
 #include "qapi/qapi-commands-machine.h"
 #include "monitor/monitor.h"
-#include "system/tcg.h"
+#include "sysemu/cpus.h"
+#include "sysemu/cpu-timers.h"
+#include "sysemu/tcg.h"
 #include "tcg/tcg.h"
 #include "internal-common.h"
+#include "tb-context.h"
+
+
+static void dump_drift_info(GString *buf)
+{
+    if (!icount_enabled()) {
+        return;
+    }
+
+    g_string_append_printf(buf, "Host - Guest clock  %"PRIi64" ms\n",
+                           (cpu_get_clock() - icount_get()) / SCALE_MS);
+    if (icount_align_option) {
+        g_string_append_printf(buf, "Max guest delay     %"PRIi64" ms\n",
+                               -max_delay / SCALE_MS);
+        g_string_append_printf(buf, "Max guest advance   %"PRIi64" ms\n",
+                               max_advance / SCALE_MS);
+    } else {
+        g_string_append_printf(buf, "Max guest delay     NA\n");
+        g_string_append_printf(buf, "Max guest advance   NA\n");
+    }
+}
+
+static void dump_accel_info(GString *buf)
+{
+    AccelState *accel = current_accel();
+    bool one_insn_per_tb = object_property_get_bool(OBJECT(accel),
+                                                    "one-insn-per-tb",
+                                                    &error_fatal);
+
+    g_string_append_printf(buf, "Accelerator settings:\n");
+    g_string_append_printf(buf, "one-insn-per-tb: %s\n\n",
+                           one_insn_per_tb ? "on" : "off");
+}
+
+static void print_qht_statistics(struct qht_stats hst, GString *buf)
+{
+    uint32_t hgram_opts;
+    size_t hgram_bins;
+    char *hgram;
+
+    if (!hst.head_buckets) {
+        return;
+    }
+    g_string_append_printf(buf, "TB hash buckets     %zu/%zu "
+                           "(%0.2f%% head buckets used)\n",
+                           hst.used_head_buckets, hst.head_buckets,
+                           (double)hst.used_head_buckets /
+                           hst.head_buckets * 100);
+
+    hgram_opts =  QDIST_PR_BORDER | QDIST_PR_LABELS;
+    hgram_opts |= QDIST_PR_100X   | QDIST_PR_PERCENT;
+    if (qdist_xmax(&hst.occupancy) - qdist_xmin(&hst.occupancy) == 1) {
+        hgram_opts |= QDIST_PR_NODECIMAL;
+    }
+    hgram = qdist_pr(&hst.occupancy, 10, hgram_opts);
+    g_string_append_printf(buf, "TB hash occupancy   %0.2f%% avg chain occ. "
+                           "Histogram: %s\n",
+                           qdist_avg(&hst.occupancy) * 100, hgram);
+    g_free(hgram);
+
+    hgram_opts = QDIST_PR_BORDER | QDIST_PR_LABELS;
+    hgram_bins = qdist_xmax(&hst.chain) - qdist_xmin(&hst.chain);
+    if (hgram_bins > 10) {
+        hgram_bins = 10;
+    } else {
+        hgram_bins = 0;
+        hgram_opts |= QDIST_PR_NODECIMAL | QDIST_PR_NOBINRANGE;
+    }
+    hgram = qdist_pr(&hst.chain, hgram_bins, hgram_opts);
+    g_string_append_printf(buf, "TB hash avg chain   %0.3f buckets. "
+                           "Histogram: %s\n",
+                           qdist_avg(&hst.chain), hgram);
+    g_free(hgram);
+}
+
+struct tb_tree_stats {
+    size_t nb_tbs;
+    size_t host_size;
+    size_t target_size;
+    size_t max_target_size;
+    size_t direct_jmp_count;
+    size_t direct_jmp2_count;
+    size_t cross_page;
+};
+
+static gboolean tb_tree_stats_iter(gpointer key, gpointer value, gpointer data)
+{
+    const TranslationBlock *tb = value;
+    struct tb_tree_stats *tst = data;
+
+    tst->nb_tbs++;
+    tst->host_size += tb->tc.size;
+    tst->target_size += tb->size;
+    if (tb->size > tst->max_target_size) {
+        tst->max_target_size = tb->size;
+    }
+    if (tb->page_addr[1] != -1) {
+        tst->cross_page++;
+    }
+    if (tb->jmp_reset_offset[0] != TB_JMP_OFFSET_INVALID) {
+        tst->direct_jmp_count++;
+        if (tb->jmp_reset_offset[1] != TB_JMP_OFFSET_INVALID) {
+            tst->direct_jmp2_count++;
+        }
+    }
+    return false;
+}
+
+static void tlb_flush_counts(size_t *pfull, size_t *ppart, size_t *pelide)
+{
+    CPUState *cpu;
+    size_t full = 0, part = 0, elide = 0;
+
+    CPU_FOREACH(cpu) {
+        full += qatomic_read(&cpu->neg.tlb.c.full_flush_count);
+        part += qatomic_read(&cpu->neg.tlb.c.part_flush_count);
+        elide += qatomic_read(&cpu->neg.tlb.c.elide_flush_count);
+    }
+    *pfull = full;
+    *ppart = part;
+    *pelide = elide;
+}
+
+static void tcg_dump_info(GString *buf)
+{
+    g_string_append_printf(buf, "[TCG profiler not compiled]\n");
+}
+
+static void dump_exec_info(GString *buf)
+{
+    struct tb_tree_stats tst = {};
+    struct qht_stats hst;
+    size_t nb_tbs, flush_full, flush_part, flush_elide;
+
+    tcg_tb_foreach(tb_tree_stats_iter, &tst);
+    nb_tbs = tst.nb_tbs;
+    /* XXX: avoid using doubles ? */
+    g_string_append_printf(buf, "Translation buffer state:\n");
+    /*
+     * Report total code size including the padding and TB structs;
+     * otherwise users might think "-accel tcg,tb-size" is not honoured.
+     * For avg host size we use the precise numbers from tb_tree_stats though.
+     */
+    g_string_append_printf(buf, "gen code size       %zu/%zu\n",
+                           tcg_code_size(), tcg_code_capacity());
+    g_string_append_printf(buf, "TB count            %zu\n", nb_tbs);
+    g_string_append_printf(buf, "TB avg target size  %zu max=%zu bytes\n",
+                           nb_tbs ? tst.target_size / nb_tbs : 0,
+                           tst.max_target_size);
+    g_string_append_printf(buf, "TB avg host size    %zu bytes "
+                           "(expansion ratio: %0.1f)\n",
+                           nb_tbs ? tst.host_size / nb_tbs : 0,
+                           tst.target_size ?
+                           (double)tst.host_size / tst.target_size : 0);
+    g_string_append_printf(buf, "cross page TB count %zu (%zu%%)\n",
+                           tst.cross_page,
+                           nb_tbs ? (tst.cross_page * 100) / nb_tbs : 0);
+    g_string_append_printf(buf, "direct jump count   %zu (%zu%%) "
+                           "(2 jumps=%zu %zu%%)\n",
+                           tst.direct_jmp_count,
+                           nb_tbs ? (tst.direct_jmp_count * 100) / nb_tbs : 0,
+                           tst.direct_jmp2_count,
+                           nb_tbs ? (tst.direct_jmp2_count * 100) / nb_tbs : 0);
+
+    qht_statistics_init(&tb_ctx.htable, &hst);
+    print_qht_statistics(hst, buf);
+    qht_statistics_destroy(&hst);
+
+    g_string_append_printf(buf, "\nStatistics:\n");
+    g_string_append_printf(buf, "TB flush count      %u\n",
+                           qatomic_read(&tb_ctx.tb_flush_count));
+    g_string_append_printf(buf, "TB invalidate count %u\n",
+                           qatomic_read(&tb_ctx.tb_phys_invalidate_count));
+
+    tlb_flush_counts(&flush_full, &flush_part, &flush_elide);
+    g_string_append_printf(buf, "TLB full flushes    %zu\n", flush_full);
+    g_string_append_printf(buf, "TLB partial flushes %zu\n", flush_part);
+    g_string_append_printf(buf, "TLB elided flushes  %zu\n", flush_elide);
+    tcg_dump_info(buf);
+}

 HumanReadableText *qmp_x_query_jit(Error **errp)
 {
@@ -24,7 +208,29 @@ HumanReadableText *qmp_x_query_jit(Error **errp)
        return NULL;
    }

-    tcg_dump_stats(buf);
+    dump_accel_info(buf);
+    dump_exec_info(buf);
+    dump_drift_info(buf);
+
+    return human_readable_text_from_str(buf);
+}
+
+static void tcg_dump_op_count(GString *buf)
+{
+    g_string_append_printf(buf, "[TCG profiler not compiled]\n");
+}
+
+HumanReadableText *qmp_x_query_opcount(Error **errp)
+{
+    g_autoptr(GString) buf = g_string_new("");
+
+    if (!tcg_enabled()) {
+        error_setg(errp,
+                   "Opcode count information is only available with accel=tcg");
+        return NULL;
+    }
+
+    tcg_dump_op_count(buf);

    return human_readable_text_from_str(buf);
 }
@@ -32,6 +238,7 @@ HumanReadableText *qmp_x_query_jit(Error **errp)
 static void hmp_tcg_register(void)
 {
    monitor_register_hmp_info_hrt("jit", qmp_x_query_jit);
+    monitor_register_hmp_info_hrt("opcount", qmp_x_query_opcount);
 }

 type_init(hmp_tcg_register);
@@ -0,0 +1,5 @@
+#ifdef CONFIG_PLUGIN
+DEF_HELPER_FLAGS_2(plugin_vcpu_udata_cb_no_wg, TCG_CALL_NO_WG | TCG_CALL_PLUGIN, void, i32, ptr)
+DEF_HELPER_FLAGS_2(plugin_vcpu_udata_cb_no_rwg, TCG_CALL_NO_RWG | TCG_CALL_PLUGIN, void, i32, ptr)
+DEF_HELPER_FLAGS_4(plugin_vcpu_mem_cb, TCG_CALL_NO_RWG | TCG_CALL_PLUGIN, void, i32, i32, i64, ptr)
+#endif
@@ -20,9 +20,8 @@
 #ifndef EXEC_TB_HASH_H
 #define EXEC_TB_HASH_H

-#include "exec/vaddr.h"
-#include "exec/target_page.h"
-#include "exec/translation-block.h"
+#include "exec/cpu-defs.h"
+#include "exec/exec-all.h"
 #include "qemu/xxhash.h"
 #include "tb-jmp-cache.h"

@@ -1,55 +0,0 @@
-/*
- * TranslationBlock internal declarations (target specific)
- *
- *  Copyright (c) 2003 Fabrice Bellard
- *
- * SPDX-License-Identifier: LGPL-2.1-or-later
- */
-
-#ifndef ACCEL_TCG_TB_INTERNAL_TARGET_H
-#define ACCEL_TCG_TB_INTERNAL_TARGET_H
-
-#include "exec/translation-block.h"
-
-/*
- * The true return address will often point to a host insn that is part of
- * the next translated guest insn.  Adjust the address backward to point to
- * the middle of the call insn.  Subtracting one would do the job except for
- * several compressed mode architectures (arm, mips) which set the low bit
- * to indicate the compressed mode; subtracting two works around that.  It
- * is also the case that there are no host isas that contain a call insn
- * smaller than 4 bytes, so we don't worry about special-casing this.
- */
-#define GETPC_ADJ   2
-
-void tb_lock_page0(tb_page_addr_t);
-
-#ifdef CONFIG_USER_ONLY
-/*
- * For user-only, page_protect sets the page read-only.
- * Since most execution is already on read-only pages, and we'd need to
- * account for other TBs on the same page, defer undoing any page protection
- * until we receive the write fault.
- */
-static inline void tb_lock_page1(tb_page_addr_t p0, tb_page_addr_t p1)
-{
-    tb_lock_page0(p1);
-}
-
-static inline void tb_unlock_page1(tb_page_addr_t p0, tb_page_addr_t p1) { }
-static inline void tb_unlock_pages(TranslationBlock *tb) { }
-#else
-void tb_lock_page1(tb_page_addr_t, tb_page_addr_t);
-void tb_unlock_page1(tb_page_addr_t, tb_page_addr_t);
-void tb_unlock_pages(TranslationBlock *);
-#endif
-
-#ifdef CONFIG_SOFTMMU
-void tb_invalidate_phys_range_fast(CPUState *cpu, ram_addr_t ram_addr,
-                                   unsigned size, uintptr_t retaddr);
-#endif /* CONFIG_SOFTMMU */
-
-bool tb_invalidate_phys_page_unwind(CPUState *cpu, tb_page_addr_t addr,
-                                    uintptr_t pc);
-
-#endif
@@ -9,9 +9,6 @@
 #ifndef ACCEL_TCG_TB_JMP_CACHE_H
 #define ACCEL_TCG_TB_JMP_CACHE_H

-#include "qemu/rcu.h"
-#include "exec/cpu-common.h"
-
 #define TB_JMP_CACHE_BITS 12
 #define TB_JMP_CACHE_SIZE (1 << TB_JMP_CACHE_BITS)

@@ -22,12 +19,12 @@
 * non-NULL value of 'tb'.  Strictly speaking pc is only needed for
 * CF_PCREL, but it's used always for simplicity.
 */
-typedef struct CPUJumpCache {
+struct CPUJumpCache {
    struct rcu_head rcu;
    struct {
        TranslationBlock *tb;
        vaddr pc;
    } array[TB_JMP_CACHE_SIZE];
-} CPUJumpCache;
+};

 #endif /* ACCEL_TCG_TB_JMP_CACHE_H */
@@ -22,21 +22,15 @@
 #include "qemu/qtree.h"
 #include "exec/cputlb.h"
 #include "exec/log.h"
-#include "exec/page-protection.h"
-#include "exec/mmap-lock.h"
+#include "exec/exec-all.h"
 #include "exec/tb-flush.h"
-#include "exec/target_page.h"
-#include "accel/tcg/cpu-ops.h"
-#include "tb-internal.h"
-#include "system/tcg.h"
+#include "exec/translate-all.h"
+#include "sysemu/tcg.h"
 #include "tcg/tcg.h"
 #include "tb-hash.h"
 #include "tb-context.h"
-#include "tb-internal.h"
 #include "internal-common.h"
-#ifdef CONFIG_USER_ONLY
-#include "user/page-protection.h"
-#endif
+#include "internal-target.h"


 /* List iterators for lists of tagged pointers in TranslationBlock. */
@@ -157,7 +151,11 @@ static PageForEachNext foreach_tb_next(PageForEachNext tb,
 /*
 * In system mode we want L1_MAP to be based on ram offsets.
 */
-#define L1_MAP_ADDR_SPACE_BITS  HOST_LONG_BITS
+#if HOST_LONG_BITS < TARGET_PHYS_ADDR_SPACE_BITS
+# define L1_MAP_ADDR_SPACE_BITS  HOST_LONG_BITS
+#else
+# define L1_MAP_ADDR_SPACE_BITS  TARGET_PHYS_ADDR_SPACE_BITS
+#endif

 /* Size of the L2 (and L3, etc) page tables.  */
 #define V_L2_BITS 10
@@ -714,7 +712,7 @@ static void tb_record(TranslationBlock *tb)
    tb_page_addr_t paddr0 = tb_page_addr0(tb);
    tb_page_addr_t paddr1 = tb_page_addr1(tb);
    tb_page_addr_t pindex0 = paddr0 >> TARGET_PAGE_BITS;
-    tb_page_addr_t pindex1 = paddr1 >> TARGET_PAGE_BITS;
+    tb_page_addr_t pindex1 = paddr0 >> TARGET_PAGE_BITS;

    assert(paddr0 != -1);
    if (unlikely(paddr1 != -1) && pindex0 != pindex1) {
@@ -746,7 +744,7 @@ static void tb_remove(TranslationBlock *tb)
    tb_page_addr_t paddr0 = tb_page_addr0(tb);
    tb_page_addr_t paddr1 = tb_page_addr1(tb);
    tb_page_addr_t pindex0 = paddr0 >> TARGET_PAGE_BITS;
-    tb_page_addr_t pindex1 = paddr1 >> TARGET_PAGE_BITS;
+    tb_page_addr_t pindex1 = paddr0 >> TARGET_PAGE_BITS;

    assert(paddr0 != -1);
    if (unlikely(paddr1 != -1) && pindex0 != pindex1) {
@@ -1006,8 +1004,7 @@ TranslationBlock *tb_link_page(TranslationBlock *tb)
 * Called with mmap_lock held for user-mode emulation.
 * NOTE: this function must not be called while a TB is running.
 */
-void tb_invalidate_phys_range(CPUState *cpu, tb_page_addr_t start,
-                              tb_page_addr_t last)
+void tb_invalidate_phys_range(tb_page_addr_t start, tb_page_addr_t last)
 {
    TranslationBlock *tb;
    PageForEachNext n;
@@ -1030,16 +1027,17 @@ static void tb_invalidate_phys_page(tb_page_addr_t addr)

    start = addr & TARGET_PAGE_MASK;
    last = addr | ~TARGET_PAGE_MASK;
-    tb_invalidate_phys_range(NULL, start, last);
+    tb_invalidate_phys_range(start, last);
 }

 /*
 * Called with mmap_lock held. If pc is not 0 then it indicates the
 * host PC of the faulting store instruction that caused this invalidate.
- * Returns true if the caller needs to abort execution of the current TB.
+ * Returns true if the caller needs to abort execution of the current
+ * TB (because it was modified by this store and the guest CPU has
+ * precise-SMC semantics).
 */
-bool tb_invalidate_phys_page_unwind(CPUState *cpu, tb_page_addr_t addr,
-                                    uintptr_t pc)
+bool tb_invalidate_phys_page_unwind(tb_page_addr_t addr, uintptr_t pc)
 {
    TranslationBlock *current_tb;
    bool current_tb_modified;
@@ -1051,7 +1049,10 @@ bool tb_invalidate_phys_page_unwind(CPUState *cpu, tb_page_addr_t addr,
     * Without precise smc semantics, or when outside of a TB,
     * we can skip to invalidate.
     */
-    if (!pc || !cpu || !cpu->cc->tcg_ops->precise_smc) {
+#ifndef TARGET_HAS_PRECISE_SMC
+    pc = 0;
+#endif
+    if (!pc) {
        tb_invalidate_phys_page(addr);
        return false;
    }
@@ -1074,14 +1075,15 @@ bool tb_invalidate_phys_page_unwind(CPUState *cpu, tb_page_addr_t addr,
             * the CPU state.
             */
            current_tb_modified = true;
-            cpu_restore_state_from_tb(cpu, current_tb, pc);
+            cpu_restore_state_from_tb(current_cpu, current_tb, pc);
        }
        tb_phys_invalidate__locked(tb);
    }

    if (current_tb_modified) {
        /* Force execution of one insn next time.  */
-        cpu->cflags_next_tb = 1 | CF_NOIRQ | curr_cflags(cpu);
+        CPUState *cpu = current_cpu;
+        cpu->cflags_next_tb = 1 | CF_NOIRQ | curr_cflags(current_cpu);
        return true;
    }
    return false;
@@ -1090,28 +1092,23 @@ bool tb_invalidate_phys_page_unwind(CPUState *cpu, tb_page_addr_t addr,
 /*
 * @p must be non-NULL.
 * Call with all @pages locked.
- * (@cpu, @retaddr) may be (NULL, 0) outside of a cpu context,
- * in which case precise_smc need not be detected.
 */
 static void
-tb_invalidate_phys_page_range__locked(CPUState *cpu,
-                                      struct page_collection *pages,
+tb_invalidate_phys_page_range__locked(struct page_collection *pages,
                                      PageDesc *p, tb_page_addr_t start,
                                      tb_page_addr_t last,
                                      uintptr_t retaddr)
 {
    TranslationBlock *tb;
    PageForEachNext n;
+#ifdef TARGET_HAS_PRECISE_SMC
    bool current_tb_modified = false;
-    TranslationBlock *current_tb = NULL;
+    TranslationBlock *current_tb = retaddr ? tcg_tb_lookup(retaddr) : NULL;
+#endif /* TARGET_HAS_PRECISE_SMC */

    /* Range may not cross a page. */
    tcg_debug_assert(((start ^ last) & TARGET_PAGE_MASK) == 0);

-    if (retaddr && cpu && cpu->cc->tcg_ops->precise_smc) {
-        current_tb = tcg_tb_lookup(retaddr);
-    }
-
    /*
     * We remove all the TBs in the range [start, last].
     * XXX: see if in some cases it could be faster to invalidate all the code
@@ -1129,7 +1126,8 @@ tb_invalidate_phys_page_range__locked(CPUState *cpu,
            tb_last = tb_start + (tb_last & ~TARGET_PAGE_MASK);
        }
        if (!(tb_last < start || tb_start > last)) {
-            if (unlikely(current_tb == tb) &&
+#ifdef TARGET_HAS_PRECISE_SMC
+            if (current_tb == tb &&
                (tb_cflags(current_tb) & CF_COUNT_MASK) != 1) {
                /*
                 * If we are modifying the current TB, we must stop
@@ -1139,8 +1137,9 @@ tb_invalidate_phys_page_range__locked(CPUState *cpu,
                 * restore the CPU state.
                 */
                current_tb_modified = true;
-                cpu_restore_state_from_tb(cpu, current_tb, retaddr);
+                cpu_restore_state_from_tb(current_cpu, current_tb, retaddr);
            }
+#endif /* TARGET_HAS_PRECISE_SMC */
            tb_phys_invalidate__locked(tb);
        }
    }
@@ -1150,13 +1149,15 @@ tb_invalidate_phys_page_range__locked(CPUState *cpu,
        tlb_unprotect_code(start);
    }

-    if (unlikely(current_tb_modified)) {
+#ifdef TARGET_HAS_PRECISE_SMC
+    if (current_tb_modified) {
        page_collection_unlock(pages);
        /* Force execution of one insn next time.  */
-        cpu->cflags_next_tb = 1 | CF_NOIRQ | curr_cflags(cpu);
+        current_cpu->cflags_next_tb = 1 | CF_NOIRQ | curr_cflags(current_cpu);
        mmap_unlock();
-        cpu_loop_exit_noexc(cpu);
+        cpu_loop_exit_noexc(current_cpu);
    }
+#endif
 }

 /*
@@ -1166,8 +1167,7 @@ tb_invalidate_phys_page_range__locked(CPUState *cpu,
 * access: the virtual CPU will exit the current TB if code is modified inside
 * this TB.
 */
-void tb_invalidate_phys_range(CPUState *cpu, tb_page_addr_t start,
-                              tb_page_addr_t last)
+void tb_invalidate_phys_range(tb_page_addr_t start, tb_page_addr_t last)
 {
    struct page_collection *pages;
    tb_page_addr_t index, index_last;
@@ -1186,30 +1186,44 @@ void tb_invalidate_phys_range(CPUState *cpu, tb_page_addr_t start,
        page_start = index << TARGET_PAGE_BITS;
        page_last = page_start | ~TARGET_PAGE_MASK;
        page_last = MIN(page_last, last);
-        tb_invalidate_phys_page_range__locked(cpu, pages, pd,
+        tb_invalidate_phys_page_range__locked(pages, pd,
                                              page_start, page_last, 0);
    }
    page_collection_unlock(pages);
 }

+/*
+ * Call with all @pages in the range [@start, @start + len[ locked.
+ */
+static void tb_invalidate_phys_page_fast__locked(struct page_collection *pages,
+                                                 tb_page_addr_t start,
+                                                 unsigned len, uintptr_t ra)
+{
+    PageDesc *p;
+
+    p = page_find(start >> TARGET_PAGE_BITS);
+    if (!p) {
+        return;
+    }
+
+    assert_page_locked(p);
+    tb_invalidate_phys_page_range__locked(pages, p, start, start + len - 1, ra);
+}
+
 /*
 * len must be <= 8 and start must be a multiple of len.
 * Called via softmmu_template.h when code areas are written to with
 * iothread mutex not held.
 */
-void tb_invalidate_phys_range_fast(CPUState *cpu, ram_addr_t start,
-                                   unsigned len, uintptr_t ra)
+void tb_invalidate_phys_range_fast(ram_addr_t ram_addr,
+                                   unsigned size,
+                                   uintptr_t retaddr)
 {
-    PageDesc *p = page_find(start >> TARGET_PAGE_BITS);
+    struct page_collection *pages;

-    if (p) {
-        ram_addr_t last = start + len - 1;
-        struct page_collection *pages = page_collection_lock(start, last);
-
-        tb_invalidate_phys_page_range__locked(cpu, pages, p,
-                                              start, last, ra);
-        page_collection_unlock(pages);
-    }
+    pages = page_collection_lock(ram_addr, ram_addr + size - 1);
+    tb_invalidate_phys_page_fast__locked(pages, ram_addr, size, retaddr);
+    page_collection_unlock(pages);
 }

 #endif /* CONFIG_USER_ONLY */
@@ -24,11 +24,11 @@
 */

 #include "qemu/osdep.h"
-#include "system/replay.h"
-#include "exec/icount.h"
+#include "sysemu/replay.h"
+#include "sysemu/cpu-timers.h"
 #include "qemu/main-loop.h"
 #include "qemu/guest-random.h"
-#include "hw/core/cpu.h"
+#include "exec/exec-all.h"

 #include "tcg-accel-ops.h"
 #include "tcg-accel-ops-icount.h"
@@ -24,12 +24,13 @@
 */

 #include "qemu/osdep.h"
-#include "system/tcg.h"
-#include "system/replay.h"
-#include "exec/icount.h"
+#include "sysemu/tcg.h"
+#include "sysemu/replay.h"
+#include "sysemu/cpu-timers.h"
 #include "qemu/main-loop.h"
 #include "qemu/notify.h"
 #include "qemu/guest-random.h"
+#include "exec/exec-all.h"
 #include "hw/boards.h"
 #include "tcg/startup.h"
 #include "tcg-accel-ops.h"
@@ -113,6 +114,7 @@ static void *mttcg_cpu_thread_fn(void *arg)
            }
        }

+        qatomic_set_mb(&cpu->exit_request, 0);
        qemu_wait_io_event(cpu);
    } while (!cpu->unplug || cpu_can_run(cpu));

@@ -135,6 +137,10 @@ void mttcg_start_vcpu_thread(CPUState *cpu)
    g_assert(tcg_enabled());
    tcg_cpu_init_cflags(cpu, current_machine->smp.max_cpus > 1);

+    cpu->thread = g_new0(QemuThread, 1);
+    cpu->halt_cond = g_malloc0(sizeof(QemuCond));
+    qemu_cond_init(cpu->halt_cond);
+
    /* create a thread per vCPU with TCG (MTTCG) */
    snprintf(thread_name, VCPU_THREAD_NAME_SIZE, "CPU %d/TCG",
             cpu->cpu_index);
@@ -25,13 +25,13 @@

 #include "qemu/osdep.h"
 #include "qemu/lockable.h"
-#include "system/tcg.h"
-#include "system/replay.h"
-#include "exec/icount.h"
+#include "sysemu/tcg.h"
+#include "sysemu/replay.h"
+#include "sysemu/cpu-timers.h"
 #include "qemu/main-loop.h"
 #include "qemu/notify.h"
 #include "qemu/guest-random.h"
-#include "exec/cpu-common.h"
+#include "exec/exec-all.h"
 #include "tcg/startup.h"
 #include "tcg-accel-ops.h"
 #include "tcg-accel-ops-rr.h"
@@ -109,7 +109,7 @@ static void rr_wait_io_event(void)
 {
    CPUState *cpu;

-    while (all_cpu_threads_idle()) {
+    while (all_cpu_threads_idle() && replay_can_wait()) {
        rr_stop_kick_timer();
        qemu_cond_wait_bql(first_cpu->halt_cond);
    }
@@ -302,7 +302,9 @@ static void *rr_cpu_thread_fn(void *arg)
        rr_deal_with_unplugged_cpus();
    }

-    g_assert_not_reached();
+    rcu_remove_force_rcu_notifier(&force_rcu);
+    rcu_unregister_thread();
+    return NULL;
 }

 void rr_start_vcpu_thread(CPUState *cpu)
@@ -315,23 +317,22 @@ void rr_start_vcpu_thread(CPUState *cpu)
    tcg_cpu_init_cflags(cpu, false);

    if (!single_tcg_cpu_thread) {
-        single_tcg_halt_cond = cpu->halt_cond;
-        single_tcg_cpu_thread = cpu->thread;
+        cpu->thread = g_new0(QemuThread, 1);
+        cpu->halt_cond = g_new0(QemuCond, 1);
+        qemu_cond_init(cpu->halt_cond);

        /* share a single thread for all cpus with TCG */
        snprintf(thread_name, VCPU_THREAD_NAME_SIZE, "ALL CPUs/TCG");
        qemu_thread_create(cpu->thread, thread_name,
                           rr_cpu_thread_fn,
                           cpu, QEMU_THREAD_JOINABLE);
+
+        single_tcg_halt_cond = cpu->halt_cond;
+        single_tcg_cpu_thread = cpu->thread;
    } else {
-        /* we share the thread, dump spare data */
-        g_free(cpu->thread);
-        qemu_cond_destroy(cpu->halt_cond);
-        g_free(cpu->halt_cond);
+        /* we share the thread */
        cpu->thread = single_tcg_cpu_thread;
        cpu->halt_cond = single_tcg_halt_cond;
-
-        /* copy the stuff done at start of rr_cpu_thread_fn */
        cpu->thread_id = first_cpu->thread_id;
        cpu->neg.can_do_io = 1;
        cpu->created = true;
@@ -26,22 +26,16 @@
 */

 #include "qemu/osdep.h"
-#include "accel/accel-ops.h"
-#include "accel/accel-cpu-ops.h"
-#include "system/tcg.h"
-#include "system/replay.h"
-#include "exec/icount.h"
+#include "sysemu/tcg.h"
+#include "sysemu/replay.h"
+#include "sysemu/cpu-timers.h"
 #include "qemu/main-loop.h"
 #include "qemu/guest-random.h"
 #include "qemu/timer.h"
-#include "exec/cputlb.h"
+#include "exec/exec-all.h"
 #include "exec/hwaddr.h"
 #include "exec/tb-flush.h"
-#include "exec/translation-block.h"
-#include "exec/watchpoint.h"
-#include "gdbstub/enums.h"
-
-#include "hw/core/cpu.h"
+#include "exec/gdbstub.h"

 #include "tcg-accel-ops.h"
 #include "tcg-accel-ops-mttcg.h"
@@ -66,7 +60,7 @@ void tcg_cpu_init_cflags(CPUState *cpu, bool parallel)

    cflags |= parallel ? CF_PARALLEL : 0;
    cflags |= icount_enabled() ? CF_USE_ICOUNT : 0;
-    tcg_cflags_set(cpu, cflags);
+    cpu->tcg_cflags |= cflags;
 }

 void tcg_cpu_destroy(CPUState *cpu)
@@ -81,9 +75,6 @@ int tcg_cpu_exec(CPUState *cpu)
    cpu_exec_start(cpu);
    ret = cpu_exec(cpu);
    cpu_exec_end(cpu);
-
-    qatomic_set_mb(&cpu->exit_request, 0);
-
    return ret;
 }

@@ -97,6 +88,8 @@ static void tcg_cpu_reset_hold(CPUState *cpu)
 /* mask must never be zero, except for A20 change call */
 void tcg_handle_interrupt(CPUState *cpu, int mask)
 {
+    g_assert(bql_locked());
+
    cpu->interrupt_request |= mask;

    /*
@@ -124,9 +117,10 @@ static inline int xlat_gdb_type(CPUState *cpu, int gdbtype)
        [GDB_WATCHPOINT_ACCESS] = BP_GDB | BP_MEM_ACCESS,
    };

+    CPUClass *cc = CPU_GET_CLASS(cpu);
    int cputype = xlat[gdbtype];

-    if (cpu->cc->gdb_stop_before_watchpoint) {
+    if (cc->gdb_stop_before_watchpoint) {
        cputype |= BP_STOP_BEFORE_ACCESS;
    }
    return cputype;
@@ -200,10 +194,8 @@ static inline void tcg_remove_all_breakpoints(CPUState *cpu)
    cpu_watchpoint_remove_all(cpu, BP_GDB);
 }

-static void tcg_accel_ops_init(AccelClass *ac)
+static void tcg_accel_ops_init(AccelOpsClass *ops)
 {
-    AccelOpsClass *ops = ac->ops;
-
    if (qemu_tcg_mttcg_enabled()) {
        ops->create_vcpu_thread = mttcg_start_vcpu_thread;
        ops->kick_vcpu_thread = mttcg_kick_vcpu_thread;
@@ -228,7 +220,7 @@ static void tcg_accel_ops_init(AccelClass *ac)
    ops->remove_all_breakpoints = tcg_remove_all_breakpoints;
 }

-static void tcg_accel_ops_class_init(ObjectClass *oc, const void *data)
+static void tcg_accel_ops_class_init(ObjectClass *oc, void *data)
 {
    AccelOpsClass *ops = ACCEL_OPS_CLASS(oc);

@@ -12,7 +12,7 @@
 #ifndef TCG_ACCEL_OPS_H
 #define TCG_ACCEL_OPS_H

-#include "system/cpus.h"
+#include "sysemu/cpus.h"

 void tcg_cpu_destroy(CPUState *cpu);
 int tcg_cpu_exec(CPUState *cpu);
@@ -24,31 +24,26 @@
 */

 #include "qemu/osdep.h"
-#include "system/tcg.h"
+#include "sysemu/tcg.h"
 #include "exec/replay-core.h"
-#include "exec/icount.h"
+#include "sysemu/cpu-timers.h"
 #include "tcg/startup.h"
+#include "tcg/oversized-guest.h"
 #include "qapi/error.h"
 #include "qemu/error-report.h"
 #include "qemu/accel.h"
 #include "qemu/atomic.h"
-#include "qapi/qapi-types-common.h"
 #include "qapi/qapi-builtin-visit.h"
 #include "qemu/units.h"
-#include "qemu/target-info.h"
-#ifndef CONFIG_USER_ONLY
+#if !defined(CONFIG_USER_ONLY)
 #include "hw/boards.h"
 #endif
-#include "accel/accel-ops.h"
-#include "accel/accel-cpu-ops.h"
-#include "accel/tcg/cpu-ops.h"
-#include "internal-common.h"
-
+#include "internal-target.h"

 struct TCGState {
    AccelState parent_obj;

-    OnOffAuto mttcg_enabled;
+    bool mttcg_enabled;
    bool one_insn_per_tb;
    int splitwx_enabled;
    unsigned long tb_size;
@@ -60,18 +55,40 @@ typedef struct TCGState TCGState;
 DECLARE_INSTANCE_CHECKER(TCGState, TCG_STATE,
                         TYPE_TCG_ACCEL)

-#ifndef CONFIG_USER_ONLY
-bool qemu_tcg_mttcg_enabled(void)
+/*
+ * We default to false if we know other options have been enabled
+ * which are currently incompatible with MTTCG. Otherwise when each
+ * guest (target) has been updated to support:
+ *   - atomic instructions
+ *   - memory ordering primitives (barriers)
+ * they can set the appropriate CONFIG flags in ${target}-softmmu.mak
+ *
+ * Once a guest architecture has been converted to the new primitives
+ * there is one remaining limitation to check:
+ *   - The guest can't be oversized (e.g. 64 bit guest on 32 bit host)
+ */
+
+static bool default_mttcg_enabled(void)
 {
-    TCGState *s = TCG_STATE(current_accel());
-    return s->mttcg_enabled == ON_OFF_AUTO_ON;
+    if (icount_enabled() || TCG_OVERSIZED_GUEST) {
+        return false;
+    }
+#ifdef TARGET_SUPPORTS_MTTCG
+# ifndef TCG_GUEST_DEFAULT_MO
+#  error "TARGET_SUPPORTS_MTTCG without TCG_GUEST_DEFAULT_MO"
+# endif
+    return true;
+#else
+    return false;
+#endif
 }
-#endif /* !CONFIG_USER_ONLY */

 static void tcg_accel_instance_init(Object *obj)
 {
    TCGState *s = TCG_STATE(obj);

+    s->mttcg_enabled = default_mttcg_enabled();
+
    /* If debugging enabled, default "auto on", otherwise off. */
 #if defined(CONFIG_DEBUG_TCG) && !defined(CONFIG_USER_ONLY)
    s->splitwx_enabled = -1;
@@ -80,57 +97,24 @@ static void tcg_accel_instance_init(Object *obj)
 #endif
 }

+bool mttcg_enabled;
 bool one_insn_per_tb;

-static int tcg_init_machine(AccelState *as, MachineState *ms)
+static int tcg_init_machine(MachineState *ms)
 {
-    TCGState *s = TCG_STATE(as);
-    unsigned max_threads = 1;
-
-#ifndef CONFIG_USER_ONLY
-    CPUClass *cc = CPU_CLASS(object_class_by_name(target_cpu_type()));
-    bool mttcg_supported = cc->tcg_ops->mttcg_supported;
-
-    switch (s->mttcg_enabled) {
-    case ON_OFF_AUTO_AUTO:
-        /*
-         * We default to false if we know other options have been enabled
-         * which are currently incompatible with MTTCG. Otherwise when each
-         * guest (target) has been updated to support:
-         *   - atomic instructions
-         *   - memory ordering primitives (barriers)
-         * they can set the appropriate CONFIG flags in ${target}-softmmu.mak
-         *
-         * Once a guest architecture has been converted to the new primitives
-         * there is one remaining limitation to check:
-         *   - The guest can't be oversized (e.g. 64 bit guest on 32 bit host)
-         */
-        if (mttcg_supported && !icount_enabled()) {
-            s->mttcg_enabled = ON_OFF_AUTO_ON;
-            max_threads = ms->smp.max_cpus;
-        } else {
-            s->mttcg_enabled = ON_OFF_AUTO_OFF;
-        }
-        break;
-    case ON_OFF_AUTO_ON:
-        if (!mttcg_supported) {
-            warn_report("Guest not yet converted to MTTCG - "
-                        "you may get unexpected results");
-        }
-        max_threads = ms->smp.max_cpus;
-        break;
-    case ON_OFF_AUTO_OFF:
-        break;
-    default:
-        g_assert_not_reached();
-    }
+    TCGState *s = TCG_STATE(current_accel());
+#ifdef CONFIG_USER_ONLY
+    unsigned max_cpus = 1;
+#else
+    unsigned max_cpus = ms->smp.max_cpus;
 #endif

    tcg_allowed = true;
+    mttcg_enabled = s->mttcg_enabled;

    page_init();
    tb_htable_init();
-    tcg_init(s->tb_size * MiB, s->splitwx_enabled, max_threads);
+    tcg_init(s->tb_size * MiB, s->splitwx_enabled, max_cpus);

 #if defined(CONFIG_SOFTMMU)
    /*
@@ -140,10 +124,6 @@ static int tcg_init_machine(AccelState *as, MachineState *ms)
    tcg_prologue_init();
 #endif

-#ifdef CONFIG_USER_ONLY
-    qdev_create_fake_machine();
-#endif
-
    return 0;
 }

@@ -151,7 +131,7 @@ static char *tcg_get_thread(Object *obj, Error **errp)
 {
    TCGState *s = TCG_STATE(obj);

-    return g_strdup(s->mttcg_enabled == ON_OFF_AUTO_ON ? "multi" : "single");
+    return g_strdup(s->mttcg_enabled ? "multi" : "single");
 }

 static void tcg_set_thread(Object *obj, const char *value, Error **errp)
@@ -159,13 +139,19 @@ static void tcg_set_thread(Object *obj, const char *value, Error **errp)
    TCGState *s = TCG_STATE(obj);

    if (strcmp(value, "multi") == 0) {
-        if (icount_enabled()) {
+        if (TCG_OVERSIZED_GUEST) {
+            error_setg(errp, "No MTTCG when guest word size > hosts");
+        } else if (icount_enabled()) {
            error_setg(errp, "No MTTCG when icount is enabled");
        } else {
-            s->mttcg_enabled = ON_OFF_AUTO_ON;
+#ifndef TARGET_SUPPORTS_MTTCG
+            warn_report("Guest not yet converted to MTTCG - "
+                        "you may get unexpected results");
+#endif
+            s->mttcg_enabled = true;
        }
    } else if (strcmp(value, "single") == 0) {
-        s->mttcg_enabled = ON_OFF_AUTO_OFF;
+        s->mttcg_enabled = false;
    } else {
        error_setg(errp, "Invalid 'thread' setting %s", value);
    }
@@ -221,7 +207,7 @@ static void tcg_set_one_insn_per_tb(Object *obj, bool value, Error **errp)
    qatomic_set(&one_insn_per_tb, value);
 }

-static int tcg_gdbstub_supported_sstep_flags(AccelState *as)
+static int tcg_gdbstub_supported_sstep_flags(void)
 {
    /*
     * In replay mode all events will come from the log and can't be
@@ -236,14 +222,13 @@ static int tcg_gdbstub_supported_sstep_flags(AccelState *as)
    }
 }

-static void tcg_accel_class_init(ObjectClass *oc, const void *data)
+static void tcg_accel_class_init(ObjectClass *oc, void *data)
 {
    AccelClass *ac = ACCEL_CLASS(oc);
    ac->name = "tcg";
    ac->init_machine = tcg_init_machine;
    ac->cpu_common_realize = tcg_exec_realizefn;
    ac->cpu_common_unrealize = tcg_exec_unrealizefn;
-    ac->get_stats = tcg_get_stats;
    ac->allowed = &tcg_allowed;
    ac->gdbstub_supported_sstep_flags = tcg_gdbstub_supported_sstep_flags;

@@ -19,6 +19,7 @@

 #include "qemu/osdep.h"
 #include "qemu/host-utils.h"
+#include "cpu.h"
 #include "exec/helper-proto-common.h"
 #include "tcg/tcg-gvec-desc.h"

@@ -23,9 +23,13 @@
 */
 #include "qemu/osdep.h"
 #include "qemu/host-utils.h"
-#include "exec/cpu-common.h"
+#include "cpu.h"
 #include "exec/helper-proto-common.h"
-#include "accel/tcg/getpc.h"
+#include "exec/cpu_ldst.h"
+#include "exec/exec-all.h"
+#include "disas/disas.h"
+#include "exec/log.h"
+#include "tcg/tcg.h"

 #define HELPER_H  "accel/tcg/tcg-runtime.h"
 #include "exec/helper-info.c.inc"
@@ -1,219 +0,0 @@
-/*
- * SPDX-License-Identifier: LGPL-2.1-or-later
- *
- *  QEMU TCG statistics
- *
- *  Copyright (c) 2003-2005 Fabrice Bellard
- */
-
-#include "qemu/osdep.h"
-#include "qemu/accel.h"
-#include "qemu/qht.h"
-#include "qapi/error.h"
-#include "system/cpu-timers.h"
-#include "exec/icount.h"
-#include "hw/core/cpu.h"
-#include "tcg/tcg.h"
-#include "internal-common.h"
-#include "tb-context.h"
-#include <math.h>
-
-static void dump_drift_info(GString *buf)
-{
-    if (!icount_enabled()) {
-        return;
-    }
-
-    g_string_append_printf(buf, "Host - Guest clock  %"PRIi64" ms\n",
-                           (cpu_get_clock() - icount_get()) / SCALE_MS);
-    if (icount_align_option) {
-        g_string_append_printf(buf, "Max guest delay     %"PRIi64" ms\n",
-                               -max_delay / SCALE_MS);
-        g_string_append_printf(buf, "Max guest advance   %"PRIi64" ms\n",
-                               max_advance / SCALE_MS);
-    } else {
-        g_string_append_printf(buf, "Max guest delay     NA\n");
-        g_string_append_printf(buf, "Max guest advance   NA\n");
-    }
-}
-
-static void dump_accel_info(AccelState *accel, GString *buf)
-{
-    bool one_insn_per_tb = object_property_get_bool(OBJECT(accel),
-                                                    "one-insn-per-tb",
-                                                    &error_fatal);
-
-    g_string_append_printf(buf, "Accelerator settings:\n");
-    g_string_append_printf(buf, "one-insn-per-tb: %s\n\n",
-                           one_insn_per_tb ? "on" : "off");
-}
-
-static void print_qht_statistics(struct qht_stats hst, GString *buf)
-{
-    uint32_t hgram_opts;
-    size_t hgram_bins;
-    char *hgram;
-    double avg;
-
-    if (!hst.head_buckets) {
-        return;
-    }
-    g_string_append_printf(buf, "TB hash buckets     %zu/%zu "
-                           "(%0.2f%% head buckets used)\n",
-                           hst.used_head_buckets, hst.head_buckets,
-                           (double)hst.used_head_buckets /
-                           hst.head_buckets * 100);
-
-    hgram_opts =  QDIST_PR_BORDER | QDIST_PR_LABELS;
-    hgram_opts |= QDIST_PR_100X   | QDIST_PR_PERCENT;
-    if (qdist_xmax(&hst.occupancy) - qdist_xmin(&hst.occupancy) == 1) {
-        hgram_opts |= QDIST_PR_NODECIMAL;
-    }
-    hgram = qdist_pr(&hst.occupancy, 10, hgram_opts);
-    avg = qdist_avg(&hst.occupancy);
-    if (!isnan(avg)) {
-        g_string_append_printf(buf, "TB hash occupancy   "
-                                    "%0.2f%% avg chain occ. "
-                                    "Histogram: %s\n",
-                               avg * 100, hgram);
-    }
-    g_free(hgram);
-
-    hgram_opts = QDIST_PR_BORDER | QDIST_PR_LABELS;
-    hgram_bins = qdist_xmax(&hst.chain) - qdist_xmin(&hst.chain);
-    if (hgram_bins > 10) {
-        hgram_bins = 10;
-    } else {
-        hgram_bins = 0;
-        hgram_opts |= QDIST_PR_NODECIMAL | QDIST_PR_NOBINRANGE;
-    }
-    hgram = qdist_pr(&hst.chain, hgram_bins, hgram_opts);
-    avg = qdist_avg(&hst.chain);
-    if (!isnan(avg)) {
-        g_string_append_printf(buf, "TB hash avg chain   %0.3f buckets. "
-                               "Histogram: %s\n",
-                               avg, hgram);
-    }
-    g_free(hgram);
-}
-
-struct tb_tree_stats {
-    size_t nb_tbs;
-    size_t host_size;
-    size_t target_size;
-    size_t max_target_size;
-    size_t direct_jmp_count;
-    size_t direct_jmp2_count;
-    size_t cross_page;
-};
-
-static gboolean tb_tree_stats_iter(gpointer key, gpointer value, gpointer data)
-{
-    const TranslationBlock *tb = value;
-    struct tb_tree_stats *tst = data;
-
-    tst->nb_tbs++;
-    tst->host_size += tb->tc.size;
-    tst->target_size += tb->size;
-    if (tb->size > tst->max_target_size) {
-        tst->max_target_size = tb->size;
-    }
-#ifndef CONFIG_USER_ONLY
-    if (tb->page_addr[1] != -1) {
-        tst->cross_page++;
-    }
-#endif
-    if (tb->jmp_reset_offset[0] != TB_JMP_OFFSET_INVALID) {
-        tst->direct_jmp_count++;
-        if (tb->jmp_reset_offset[1] != TB_JMP_OFFSET_INVALID) {
-            tst->direct_jmp2_count++;
-        }
-    }
-    return false;
-}
-
-static void tlb_flush_counts(size_t *pfull, size_t *ppart, size_t *pelide)
-{
-    CPUState *cpu;
-    size_t full = 0, part = 0, elide = 0;
-
-    CPU_FOREACH(cpu) {
-        full += qatomic_read(&cpu->neg.tlb.c.full_flush_count);
-        part += qatomic_read(&cpu->neg.tlb.c.part_flush_count);
-        elide += qatomic_read(&cpu->neg.tlb.c.elide_flush_count);
-    }
-    *pfull = full;
-    *ppart = part;
-    *pelide = elide;
-}
-
-static void tcg_dump_flush_info(GString *buf)
-{
-    size_t flush_full, flush_part, flush_elide;
-
-    g_string_append_printf(buf, "TB flush count      %u\n",
-                           qatomic_read(&tb_ctx.tb_flush_count));
-    g_string_append_printf(buf, "TB invalidate count %u\n",
-                           qatomic_read(&tb_ctx.tb_phys_invalidate_count));
-
-    tlb_flush_counts(&flush_full, &flush_part, &flush_elide);
-    g_string_append_printf(buf, "TLB full flushes    %zu\n", flush_full);
-    g_string_append_printf(buf, "TLB partial flushes %zu\n", flush_part);
-    g_string_append_printf(buf, "TLB elided flushes  %zu\n", flush_elide);
-}
-
-static void dump_exec_info(GString *buf)
-{
-    struct tb_tree_stats tst = {};
-    struct qht_stats hst;
-    size_t nb_tbs;
-
-    tcg_tb_foreach(tb_tree_stats_iter, &tst);
-    nb_tbs = tst.nb_tbs;
-    /* XXX: avoid using doubles ? */
-    g_string_append_printf(buf, "Translation buffer state:\n");
-    /*
-     * Report total code size including the padding and TB structs;
-     * otherwise users might think "-accel tcg,tb-size" is not honoured.
-     * For avg host size we use the precise numbers from tb_tree_stats though.
-     */
-    g_string_append_printf(buf, "gen code size       %zu/%zu\n",
-                           tcg_code_size(), tcg_code_capacity());
-    g_string_append_printf(buf, "TB count            %zu\n", nb_tbs);
-    g_string_append_printf(buf, "TB avg target size  %zu max=%zu bytes\n",
-                           nb_tbs ? tst.target_size / nb_tbs : 0,
-                           tst.max_target_size);
-    g_string_append_printf(buf, "TB avg host size    %zu bytes "
-                           "(expansion ratio: %0.1f)\n",
-                           nb_tbs ? tst.host_size / nb_tbs : 0,
-                           tst.target_size ?
-                           (double)tst.host_size / tst.target_size : 0);
-    g_string_append_printf(buf, "cross page TB count %zu (%zu%%)\n",
-                           tst.cross_page,
-                           nb_tbs ? (tst.cross_page * 100) / nb_tbs : 0);
-    g_string_append_printf(buf, "direct jump count   %zu (%zu%%) "
-                           "(2 jumps=%zu %zu%%)\n",
-                           tst.direct_jmp_count,
-                           nb_tbs ? (tst.direct_jmp_count * 100) / nb_tbs : 0,
-                           tst.direct_jmp2_count,
-                           nb_tbs ? (tst.direct_jmp2_count * 100) / nb_tbs : 0);
-
-    qht_statistics_init(&tb_ctx.htable, &hst);
-    print_qht_statistics(hst, buf);
-    qht_statistics_destroy(&hst);
-
-    g_string_append_printf(buf, "\nStatistics:\n");
-    tcg_dump_flush_info(buf);
-}
-
-void tcg_get_stats(AccelState *accel, GString *buf)
-{
-    dump_accel_info(accel, buf);
-    dump_exec_info(buf);
-    dump_drift_info(buf);
-}
-
-void tcg_dump_stats(GString *buf)
-{
-    tcg_get_stats(current_accel(), buf);
-}
@@ -1,13 +0,0 @@
-/*
- * softmmu size bounds
- * SPDX-License-Identifier: LGPL-2.1-or-later
- */
-
-#ifndef ACCEL_TCG_TLB_BOUNDS_H
-#define ACCEL_TCG_TLB_BOUNDS_H
-
-#define CPU_TLB_DYN_MIN_BITS 6
-#define CPU_TLB_DYN_MAX_BITS (32 - TARGET_PAGE_BITS)
-#define CPU_TLB_DYN_DEFAULT_BITS 8
-
-#endif /* ACCEL_TCG_TLB_BOUNDS_H */
@@ -12,15 +12,3 @@ memory_notdirty_set_dirty(uint64_t vaddr) "0x%" PRIx64

 # translate-all.c
 translate_block(void *tb, uintptr_t pc, const void *tb_code) "tb:%p, pc:0x%"PRIxPTR", tb_code:%p"
-
-# ldst_atomicity
-load_atom2_fallback(uint32_t memop, uintptr_t ra) "mop:0x%"PRIx32", ra:0x%"PRIxPTR""
-load_atom4_fallback(uint32_t memop, uintptr_t ra) "mop:0x%"PRIx32", ra:0x%"PRIxPTR""
-load_atom8_or_exit_fallback(uintptr_t ra) "ra:0x%"PRIxPTR""
-load_atom8_fallback(uint32_t memop, uintptr_t ra) "mop:0x%"PRIx32", ra:0x%"PRIxPTR""
-load_atom16_fallback(uint32_t memop, uintptr_t ra) "mop:0x%"PRIx32", ra:0x%"PRIxPTR""
-load_atom16_or_exit_fallback(uintptr_t ra) "ra:0x%"PRIxPTR""
-store_atom2_fallback(uint32_t memop, uintptr_t ra) "mop:0x%"PRIx32", ra:0x%"PRIxPTR""
-store_atom4_fallback(uint32_t memop, uintptr_t ra) "mop:0x%"PRIx32", ra:0x%"PRIxPTR""
-store_atom8_fallback(uint32_t memop, uintptr_t ra) "mop:0x%"PRIx32", ra:0x%"PRIxPTR""
-store_atom16_fallback(uint32_t memop, uintptr_t ra) "mop:0x%"PRIx32", ra:0x%"PRIxPTR""
@@ -21,20 +21,48 @@

 #include "trace.h"
 #include "disas/disas.h"
+#include "exec/exec-all.h"
 #include "tcg/tcg.h"
-#include "exec/mmap-lock.h"
-#include "tb-internal.h"
+#if defined(CONFIG_USER_ONLY)
+#include "qemu.h"
+#if defined(__FreeBSD__) || defined(__FreeBSD_kernel__)
+#include <sys/param.h>
+#if __FreeBSD_version >= 700104
+#define HAVE_KINFO_GETVMMAP
+#define sigqueue sigqueue_freebsd  /* avoid redefinition */
+#include <sys/proc.h>
+#include <machine/profile.h>
+#define _KERNEL
+#include <sys/user.h>
+#undef _KERNEL
+#undef sigqueue
+#include <libutil.h>
+#endif
+#endif
+#else
+#include "exec/ram_addr.h"
+#endif
+
+#include "exec/cputlb.h"
+#include "exec/translate-all.h"
+#include "exec/translator.h"
 #include "exec/tb-flush.h"
+#include "qemu/bitmap.h"
+#include "qemu/qemu-print.h"
+#include "qemu/main-loop.h"
 #include "qemu/cacheinfo.h"
-#include "qemu/target-info.h"
+#include "qemu/timer.h"
 #include "exec/log.h"
-#include "exec/icount.h"
-#include "accel/tcg/cpu-ops.h"
+#include "sysemu/cpus.h"
+#include "sysemu/cpu-timers.h"
+#include "sysemu/tcg.h"
+#include "qapi/error.h"
+#include "hw/core/tcg-cpu-ops.h"
 #include "tb-jmp-cache.h"
 #include "tb-hash.h"
 #include "tb-context.h"
-#include "tb-internal.h"
 #include "internal-common.h"
+#include "internal-target.h"
 #include "tcg/perf.h"
 #include "tcg/insn-start-words.h"

@@ -77,7 +105,7 @@ static int64_t decode_sleb128(const uint8_t **pp)
        val |= (int64_t)(byte & 0x7f) << shift;
        shift += 7;
    } while (byte & 0x80);
-    if (shift < 64 && (byte & 0x40)) {
+    if (shift < TARGET_LONG_BITS && (byte & 0x40)) {
        val |= -(int64_t)1 << shift;
    }

@@ -88,7 +116,7 @@ static int64_t decode_sleb128(const uint8_t **pp)
 /* Encode the data collected about the instructions while compiling TB.
   Place the data at BLOCK, and return the number of bytes consumed.

-   The logical table consists of INSN_START_WORDS uint64_t's,
+   The logical table consists of TARGET_INSN_START_WORDS target_ulong's,
   which come from the target's insn_start data, followed by a uintptr_t
   which comes from the host pc of the end of the code implementing the insn.

@@ -108,13 +136,13 @@ static int encode_search(TranslationBlock *tb, uint8_t *block)
    for (i = 0, n = tb->icount; i < n; ++i) {
        uint64_t prev, curr;

-        for (j = 0; j < INSN_START_WORDS; ++j) {
+        for (j = 0; j < TARGET_INSN_START_WORDS; ++j) {
            if (i == 0) {
                prev = (!(tb_cflags(tb) & CF_PCREL) && j == 0 ? tb->pc : 0);
            } else {
-                prev = insn_data[(i - 1) * INSN_START_WORDS + j];
+                prev = insn_data[(i - 1) * TARGET_INSN_START_WORDS + j];
            }
-            curr = insn_data[i * INSN_START_WORDS + j];
+            curr = insn_data[i * TARGET_INSN_START_WORDS + j];
            p = encode_sleb128(p, curr - prev);
        }
        prev = (i == 0 ? 0 : insn_end_off[i - 1]);
@@ -146,7 +174,7 @@ static int cpu_unwind_data_from_tb(TranslationBlock *tb, uintptr_t host_pc,
        return -1;
    }

-    memset(data, 0, sizeof(uint64_t) * INSN_START_WORDS);
+    memset(data, 0, sizeof(uint64_t) * TARGET_INSN_START_WORDS);
    if (!(tb_cflags(tb) & CF_PCREL)) {
        data[0] = tb->pc;
    }
@@ -156,7 +184,7 @@ static int cpu_unwind_data_from_tb(TranslationBlock *tb, uintptr_t host_pc,
     * at which the end of the insn exceeds host_pc.
     */
    for (i = 0; i < num_insns; ++i) {
-        for (j = 0; j < INSN_START_WORDS; ++j) {
+        for (j = 0; j < TARGET_INSN_START_WORDS; ++j) {
            data[j] += decode_sleb128(&p);
        }
        iter_pc += decode_sleb128(&p);
@@ -174,7 +202,7 @@ static int cpu_unwind_data_from_tb(TranslationBlock *tb, uintptr_t host_pc,
 void cpu_restore_state_from_tb(CPUState *cpu, TranslationBlock *tb,
                               uintptr_t host_pc)
 {
-    uint64_t data[INSN_START_WORDS];
+    uint64_t data[TARGET_INSN_START_WORDS];
    int insns_left = cpu_unwind_data_from_tb(tb, host_pc, data);

    if (insns_left < 0) {
@@ -246,10 +274,8 @@ static int setjmp_gen_code(CPUArchState *env, TranslationBlock *tb,

    tcg_func_start(tcg_ctx);

-    CPUState *cs = env_cpu(env);
-    tcg_ctx->cpu = cs;
-    cs->cc->tcg_ops->translate_code(cs, tb, max_insns, pc, host_pc);
-
+    tcg_ctx->cpu = env_cpu(env);
+    gen_intermediate_code(env_cpu(env), tb, max_insns, pc, host_pc);
    assert(tb->size != 0);
    tcg_ctx->cpu = NULL;
    *max_insns = tb->icount;
@@ -258,7 +284,9 @@ static int setjmp_gen_code(CPUArchState *env, TranslationBlock *tb,
 }

 /* Called with mmap_lock held for user mode emulation.  */
-TranslationBlock *tb_gen_code(CPUState *cpu, TCGTBCPUState s)
+TranslationBlock *tb_gen_code(CPUState *cpu,
+                              vaddr pc, uint64_t cs_base,
+                              uint32_t flags, int cflags)
 {
    CPUArchState *env = cpu_env(cpu);
    TranslationBlock *tb, *existing_tb;
@@ -271,14 +299,14 @@ TranslationBlock *tb_gen_code(CPUState *cpu, TCGTBCPUState s)
    assert_memory_lock();
    qemu_thread_jit_write();

-    phys_pc = get_page_addr_code_hostp(env, s.pc, &host_pc);
+    phys_pc = get_page_addr_code_hostp(env, pc, &host_pc);

    if (phys_pc == -1) {
        /* Generate a one-shot TB with 1 insn in it */
-        s.cflags = (s.cflags & ~CF_COUNT_MASK) | 1;
+        cflags = (cflags & ~CF_COUNT_MASK) | 1;
    }

-    max_insns = s.cflags & CF_COUNT_MASK;
+    max_insns = cflags & CF_COUNT_MASK;
    if (max_insns == 0) {
        max_insns = TCG_MAX_INSNS;
    }
@@ -298,12 +326,12 @@ TranslationBlock *tb_gen_code(CPUState *cpu, TCGTBCPUState s)

    gen_code_buf = tcg_ctx->code_gen_ptr;
    tb->tc.ptr = tcg_splitwx_to_rx(gen_code_buf);
-    if (!(s.cflags & CF_PCREL)) {
-        tb->pc = s.pc;
+    if (!(cflags & CF_PCREL)) {
+        tb->pc = pc;
    }
-    tb->cs_base = s.cs_base;
-    tb->flags = s.flags;
-    tb->cflags = s.cflags;
+    tb->cs_base = cs_base;
+    tb->flags = flags;
+    tb->cflags = cflags;
    tb_set_page_addr0(tb, phys_pc);
    tb_set_page_addr1(tb, -1);
    if (phys_pc != -1) {
@@ -311,20 +339,30 @@ TranslationBlock *tb_gen_code(CPUState *cpu, TCGTBCPUState s)
    }

    tcg_ctx->gen_tb = tb;
-    tcg_ctx->addr_type = target_long_bits() == 32 ? TCG_TYPE_I32 : TCG_TYPE_I64;
-    tcg_ctx->guest_mo = cpu->cc->tcg_ops->guest_default_memory_order;
+    tcg_ctx->addr_type = TARGET_LONG_BITS == 32 ? TCG_TYPE_I32 : TCG_TYPE_I64;
+#ifdef CONFIG_SOFTMMU
+    tcg_ctx->page_bits = TARGET_PAGE_BITS;
+    tcg_ctx->page_mask = TARGET_PAGE_MASK;
+    tcg_ctx->tlb_dyn_max_bits = CPU_TLB_DYN_MAX_BITS;
+#endif
+    tcg_ctx->insn_start_words = TARGET_INSN_START_WORDS;
+#ifdef TCG_GUEST_DEFAULT_MO
+    tcg_ctx->guest_mo = TCG_GUEST_DEFAULT_MO;
+#else
+    tcg_ctx->guest_mo = TCG_MO_ALL;
+#endif

 restart_translate:
-    trace_translate_block(tb, s.pc, tb->tc.ptr);
+    trace_translate_block(tb, pc, tb->tc.ptr);

-    gen_code_size = setjmp_gen_code(env, tb, s.pc, host_pc, &max_insns, &ti);
+    gen_code_size = setjmp_gen_code(env, tb, pc, host_pc, &max_insns, &ti);
    if (unlikely(gen_code_size < 0)) {
        switch (gen_code_size) {
        case -1:
            /*
             * Overflow of code_gen_buffer, or the current slice of it.
             *
-             * TODO: We don't need to re-do tcg_ops->translate_code, nor
+             * TODO: We don't need to re-do gen_intermediate_code, nor
             * should we re-do the tcg optimization currently hidden
             * inside tcg_gen_code.  All that should be required is to
             * flush the TBs, allocate a new TB, re-initialize it per
@@ -394,10 +432,10 @@ TranslationBlock *tb_gen_code(CPUState *cpu, TCGTBCPUState s)
     * For CF_PCREL, attribute all executions of the generated code
     * to its first mapping.
     */
-    perf_report_code(s.pc, tb, tcg_splitwx_to_rx(gen_code_buf));
+    perf_report_code(pc, tb, tcg_splitwx_to_rx(gen_code_buf));

    if (qemu_loglevel_mask(CPU_LOG_TB_OUT_ASM) &&
-        qemu_log_in_addr_range(s.pc)) {
+        qemu_log_in_addr_range(pc)) {
        FILE *logfile = qemu_log_trylock();
        if (logfile) {
            int code_size, data_size;
@@ -419,7 +457,7 @@ TranslationBlock *tb_gen_code(CPUState *cpu, TCGTBCPUState s)
            fprintf(logfile, "OUT: [size=%d]\n", gen_code_size);
            fprintf(logfile,
                    "  -- guest addr 0x%016" PRIx64 " + tb prologue\n",
-                    tcg_ctx->gen_insn_data[insn * INSN_START_WORDS]);
+                    tcg_ctx->gen_insn_data[insn * TARGET_INSN_START_WORDS]);
            chunk_start = tcg_ctx->gen_insn_end_off[insn];
            disas(logfile, tb->tc.ptr, chunk_start);

@@ -432,7 +470,7 @@ TranslationBlock *tb_gen_code(CPUState *cpu, TCGTBCPUState s)
                size_t chunk_end = tcg_ctx->gen_insn_end_off[insn];
                if (chunk_end > chunk_start) {
                    fprintf(logfile, "  -- guest addr 0x%016" PRIx64 "\n",
-                            tcg_ctx->gen_insn_data[insn * INSN_START_WORDS]);
+                            tcg_ctx->gen_insn_data[insn * TARGET_INSN_START_WORDS]);
                    disas(logfile, tb->tc.ptr + chunk_start,
                          chunk_end - chunk_start);
                    chunk_start = chunk_end;
@@ -489,32 +527,23 @@ TranslationBlock *tb_gen_code(CPUState *cpu, TCGTBCPUState s)
        tb_reset_jump(tb, 1);
    }

-    /*
-     * Insert TB into the corresponding region tree before publishing it
-     * through QHT. Otherwise rewinding happened in the TB might fail to
-     * lookup itself using host PC.
-     */
-    tcg_tb_insert(tb);
-
    /*
     * If the TB is not associated with a physical RAM page then it must be
-     * a temporary one-insn TB.
-     *
-     * Such TBs must be added to region trees in order to make sure that
-     * restore_state_to_opc() - which on some architectures is not limited to
-     * rewinding, but also affects exception handling! - is called when such a
-     * TB causes an exception.
-     *
-     * At the same time, temporary one-insn TBs must be executed at most once,
-     * because subsequent reads from, e.g., I/O memory may return different
-     * values. So return early before attempting to link to other TBs or add
-     * to the QHT.
+     * a temporary one-insn TB, and we have nothing left to do. Return early
+     * before attempting to link to other TBs or add to the lookup table.
     */
    if (tb_page_addr0(tb) == -1) {
        assert_no_pages_locked();
        return tb;
    }

+    /*
+     * Insert TB into the corresponding region tree before publishing it
+     * through QHT. Otherwise rewinding happened in the TB might fail to
+     * lookup itself using host PC.
+     */
+    tcg_tb_insert(tb);
+
    /*
     * No explicit memory barrier is required -- tb_link_page() makes the
     * TB visible in a consistent state.
@@ -550,11 +579,15 @@ void tb_check_watchpoint(CPUState *cpu, uintptr_t retaddr)
        /* The exception probably happened in a helper.  The CPU state should
           have been saved before calling it. Fetch the PC from there.  */
        CPUArchState *env = cpu_env(cpu);
-        TCGTBCPUState s = cpu->cc->tcg_ops->get_tb_cpu_state(cpu);
-        tb_page_addr_t addr = get_page_addr_code(env, s.pc);
+        vaddr pc;
+        uint64_t cs_base;
+        tb_page_addr_t addr;
+        uint32_t flags;

+        cpu_get_tb_cpu_state(env, &pc, &cs_base, &flags);
+        addr = get_page_addr_code(env, pc);
        if (addr != -1) {
-            tb_invalidate_phys_range(cpu, addr, addr);
+            tb_invalidate_phys_range(addr, addr);
        }
    }
 }
@@ -585,7 +618,7 @@ void cpu_io_recompile(CPUState *cpu, uintptr_t retaddr)
     * to account for the re-execution of the branch.
     */
    n = 1;
-    cc = cpu->cc;
+    cc = CPU_GET_CLASS(cpu);
    if (cc->tcg_ops->io_recompile_replay_branch &&
        cc->tcg_ops->io_recompile_replay_branch(cpu, tb)) {
        cpu->neg.icount_decr.u16.low++;
@@ -596,13 +629,12 @@ void cpu_io_recompile(CPUState *cpu, uintptr_t retaddr)
     * Exit the loop and potentially generate a new TB executing the
     * just the I/O insns. We also limit instrumentation to memory
     * operations only (which execute after completion) so we don't
-     * double instrument the instruction. Also don't let an IRQ sneak
-     * in before we execute it.
+     * double instrument the instruction.
     */
-    cpu->cflags_next_tb = curr_cflags(cpu) | CF_MEMI_ONLY | CF_NOIRQ | n;
+    cpu->cflags_next_tb = curr_cflags(cpu) | CF_MEMI_ONLY | n;

    if (qemu_loglevel_mask(CPU_LOG_EXEC)) {
-        vaddr pc = cpu->cc->get_pc(cpu);
+        vaddr pc = log_pc(cpu, tb);
        if (qemu_log_in_addr_range(pc)) {
            qemu_log("cpu_io_recompile: rewound execution of TB to %016"
                     VADDR_PRIx "\n", pc);
@@ -612,6 +644,15 @@ void cpu_io_recompile(CPUState *cpu, uintptr_t retaddr)
    cpu_loop_exit_noexc(cpu);
 }

+#else /* CONFIG_USER_ONLY */
+
+void cpu_interrupt(CPUState *cpu, int mask)
+{
+    g_assert(bql_locked());
+    cpu->interrupt_request |= mask;
+    qatomic_set(&cpu->neg.icount_decr.u16.high, -1);
+}
+
 #endif /* CONFIG_USER_ONLY */

 /*
@@ -8,28 +8,30 @@
 */

 #include "qemu/osdep.h"
-#include "qemu/bswap.h"
 #include "qemu/log.h"
 #include "qemu/error-report.h"
-#include "accel/tcg/cpu-ldst-common.h"
-#include "accel/tcg/cpu-mmu-index.h"
-#include "exec/target_page.h"
+#include "exec/exec-all.h"
 #include "exec/translator.h"
 #include "exec/plugin-gen.h"
 #include "tcg/tcg-op-common.h"
-#include "internal-common.h"
-#include "disas/disas.h"
-#include "tb-internal.h"
+#include "internal-target.h"

 static void set_can_do_io(DisasContextBase *db, bool val)
 {
-    QEMU_BUILD_BUG_ON(sizeof_field(CPUState, neg.can_do_io) != 1);
-    tcg_gen_st8_i32(tcg_constant_i32(val), tcg_env,
-                    offsetof(CPUState, neg.can_do_io) - sizeof(CPUState));
+    if (db->saved_can_do_io != val) {
+        db->saved_can_do_io = val;
+
+        QEMU_BUILD_BUG_ON(sizeof_field(CPUState, neg.can_do_io) != 1);
+        tcg_gen_st8_i32(tcg_constant_i32(val), tcg_env,
+                        offsetof(ArchCPU, parent_obj.neg.can_do_io) -
+                        offsetof(ArchCPU, env));
+    }
 }

 bool translator_io_start(DisasContextBase *db)
 {
+    set_can_do_io(db, true);
+
    /*
     * Ensure that this instruction will be the last in the TB.
     * The target may override this to something more forceful.
@@ -48,8 +50,8 @@ static TCGOp *gen_tb_start(DisasContextBase *db, uint32_t cflags)
    if ((cflags & CF_USE_ICOUNT) || !(cflags & CF_NOIRQ)) {
        count = tcg_temp_new_i32();
        tcg_gen_ld_i32(count, tcg_env,
-                       offsetof(CPUState, neg.icount_decr.u32) -
-                       sizeof(CPUState));
+                       offsetof(ArchCPU, parent_obj.neg.icount_decr.u32)
+                       - offsetof(ArchCPU, env));
    }

    if (cflags & CF_USE_ICOUNT) {
@@ -78,10 +80,17 @@ static TCGOp *gen_tb_start(DisasContextBase *db, uint32_t cflags)

    if (cflags & CF_USE_ICOUNT) {
        tcg_gen_st16_i32(count, tcg_env,
-                         offsetof(CPUState, neg.icount_decr.u16.low) -
-                         sizeof(CPUState));
+                         offsetof(ArchCPU, parent_obj.neg.icount_decr.u16.low)
+                         - offsetof(ArchCPU, env));
    }

+    /*
+     * cpu->neg.can_do_io is set automatically here at the beginning of
+     * each translation block.  The cost is minimal, plus it would be
+     * very easy to forget doing it in the translator.
+     */
+    set_can_do_io(db, db->max_insns == 1);
+
    return icount_start_insn;
 }

@@ -103,11 +112,6 @@ static void gen_tb_end(const TranslationBlock *tb, uint32_t cflags,
    }
 }

-bool translator_is_same_page(const DisasContextBase *db, vaddr addr)
-{
-    return ((addr ^ db->pc_first) & TARGET_PAGE_MASK) == 0;
-}
-
 bool translator_use_goto_tb(DisasContextBase *db, vaddr dest)
 {
    /* Suppress goto_tb if requested. */
@@ -116,7 +120,7 @@ bool translator_use_goto_tb(DisasContextBase *db, vaddr dest)
    }

    /* Check for the dest on the same page as the start of the TB.  */
-    return translator_is_same_page(db, dest);
+    return ((db->pc_first ^ dest) & TARGET_PAGE_MASK) == 0;
 }

 void translator_loop(CPUState *cpu, TranslationBlock *tb, int *max_insns,
@@ -125,7 +129,6 @@ void translator_loop(CPUState *cpu, TranslationBlock *tb, int *max_insns,
 {
    uint32_t cflags = tb_cflags(tb);
    TCGOp *icount_start_insn;
-    TCGOp *first_insn_start = NULL;
    bool plugin_enabled;

    /* Initialize DisasContext */
@@ -135,13 +138,10 @@ void translator_loop(CPUState *cpu, TranslationBlock *tb, int *max_insns,
    db->is_jmp = DISAS_NEXT;
    db->num_insns = 0;
    db->max_insns = *max_insns;
-    db->insn_start = NULL;
-    db->fake_insn = false;
+    db->singlestep_enabled = cflags & CF_SINGLE_STEP;
+    db->saved_can_do_io = -1;
    db->host_addr[0] = host_pc;
    db->host_addr[1] = NULL;
-    db->record_start = 0;
-    db->record_len = 0;
-    db->code_mmuidx = cpu_mmu_index(cpu, true);

    ops->init_disas_context(db, cpu);
    tcg_debug_assert(db->is_jmp == DISAS_NEXT);  /* no early exit */
@@ -151,16 +151,12 @@ void translator_loop(CPUState *cpu, TranslationBlock *tb, int *max_insns,
    ops->tb_start(db, cpu);
    tcg_debug_assert(db->is_jmp == DISAS_NEXT);  /* no early exit */

-    plugin_enabled = plugin_gen_tb_start(cpu, db);
+    plugin_enabled = plugin_gen_tb_start(cpu, db, cflags & CF_MEMI_ONLY);
    db->plugin_enabled = plugin_enabled;

    while (true) {
        *max_insns = ++db->num_insns;
        ops->insn_start(db, cpu);
-        db->insn_start = tcg_last_op();
-        if (first_insn_start == NULL) {
-            first_insn_start = db->insn_start;
-        }
        tcg_debug_assert(db->is_jmp == DISAS_NEXT);  /* no early exit */

        if (plugin_enabled) {
@@ -173,6 +169,10 @@ void translator_loop(CPUState *cpu, TranslationBlock *tb, int *max_insns,
         * done next -- either exiting this loop or locate the start of
         * the next instruction.
         */
+        if (db->num_insns == db->max_insns) {
+            /* Accept I/O on the last instruction.  */
+            set_can_do_io(db, true);
+        }
        ops->translate_insn(db, cpu);

        /*
@@ -205,318 +205,172 @@ void translator_loop(CPUState *cpu, TranslationBlock *tb, int *max_insns,
    ops->tb_stop(db, cpu);
    gen_tb_end(tb, cflags, icount_start_insn, db->num_insns);

-    /*
-     * Manage can_do_io for the translation block: set to false before
-     * the first insn and set to true before the last insn.
-     */
-    if (db->num_insns == 1) {
-        tcg_debug_assert(first_insn_start == db->insn_start);
-    } else {
-        tcg_debug_assert(first_insn_start != db->insn_start);
-        tcg_ctx->emit_before_op = first_insn_start;
-        set_can_do_io(db, false);
-    }
-    tcg_ctx->emit_before_op = db->insn_start;
-    set_can_do_io(db, true);
-    tcg_ctx->emit_before_op = NULL;
-
-    /* May be used by disas_log or plugin callbacks. */
-    tb->size = db->pc_next - db->pc_first;
-    tb->icount = db->num_insns;
-
    if (plugin_enabled) {
        plugin_gen_tb_end(cpu, db->num_insns);
    }

+    /* The disas_log hook may use these values rather than recompute.  */
+    tb->size = db->pc_next - db->pc_first;
+    tb->icount = db->num_insns;
+
    if (qemu_loglevel_mask(CPU_LOG_TB_IN_ASM)
        && qemu_log_in_addr_range(db->pc_first)) {
        FILE *logfile = qemu_log_trylock();
        if (logfile) {
            fprintf(logfile, "----------------\n");
-
-            if (!ops->disas_log ||
-                !ops->disas_log(db, cpu, logfile)) {
-                fprintf(logfile, "IN: %s\n", lookup_symbol(db->pc_first));
-                target_disas(logfile, cpu, db);
-            }
+            ops->disas_log(db, cpu, logfile);
            fprintf(logfile, "\n");
            qemu_log_unlock(logfile);
        }
    }
 }

-static bool translator_ld(CPUArchState *env, DisasContextBase *db,
-                          void *dest, vaddr pc, size_t len)
+static void *translator_access(CPUArchState *env, DisasContextBase *db,
+                               vaddr pc, size_t len)
 {
-    TranslationBlock *tb = db->tb;
-    vaddr last = pc + len - 1;
    void *host;
-    vaddr base;
+    vaddr base, end;
+    TranslationBlock *tb;
+
+    tb = db->tb;

    /* Use slow path if first page is MMIO. */
    if (unlikely(tb_page_addr0(tb) == -1)) {
-        /* We capped translation with first page MMIO in tb_gen_code. */
-        tcg_debug_assert(db->max_insns == 1);
-        return false;
+        return NULL;
    }

-    host = db->host_addr[0];
-    base = db->pc_first;
-
-    if (likely(((base ^ last) & TARGET_PAGE_MASK) == 0)) {
-        /* Entire read is from the first page. */
-        goto do_read;
-    }
-
-    if (unlikely(((base ^ pc) & TARGET_PAGE_MASK) == 0)) {
-        /*
-         * Read begins on the first page and extends to the second.
-         * The unaligned read is never atomic.
-         */
-        size_t len0 = -(pc | TARGET_PAGE_MASK);
-        memcpy(dest, host + (pc - base), len0);
-        pc += len0;
-        dest += len0;
-        len -= len0;
-    }
-
-    /*
-     * The read must conclude on the second page and not extend to a third.
-     *
-     * TODO: We could allow the two pages to be virtually discontiguous,
-     * since we already allow the two pages to be physically discontiguous.
-     * The only reasonable use case would be executing an insn at the end
-     * of the address space wrapping around to the beginning.  For that,
-     * we would need to know the current width of the address space.
-     * In the meantime, assert.
-     */
-    base = (base & TARGET_PAGE_MASK) + TARGET_PAGE_SIZE;
-    assert(((base ^ pc) & TARGET_PAGE_MASK) == 0);
-    assert(((base ^ last) & TARGET_PAGE_MASK) == 0);
-    host = db->host_addr[1];
-
-    if (host == NULL) {
-        tb_page_addr_t page0, old_page1, new_page1;
-
-        new_page1 = get_page_addr_code_hostp(env, base, &db->host_addr[1]);
-
-        /*
-         * If the second page is MMIO, treat as if the first page
-         * was MMIO as well, so that we do not cache the TB.
-         */
-        if (unlikely(new_page1 == -1)) {
-            tb_unlock_pages(tb);
-            tb_set_page_addr0(tb, -1);
-            /* Require that this be the final insn. */
-            db->max_insns = db->num_insns;
-            return false;
-        }
-
-        /*
-         * If this is not the first time around, and page1 matches,
-         * then we already have the page locked.  Alternately, we're
-         * not doing anything to prevent the PTE from changing, so
-         * we might wind up with a different page, requiring us to
-         * re-do the locking.
-         */
-        old_page1 = tb_page_addr1(tb);
-        if (likely(new_page1 != old_page1)) {
-            page0 = tb_page_addr0(tb);
-            if (unlikely(old_page1 != -1)) {
-                tb_unlock_page1(page0, old_page1);
-            }
-            tb_set_page_addr1(tb, new_page1);
-            tb_lock_page1(page0, new_page1);
-        }
+    end = pc + len - 1;
+    if (likely(is_same_page(db, end))) {
+        host = db->host_addr[0];
+        base = db->pc_first;
+    } else {
        host = db->host_addr[1];
+        base = TARGET_PAGE_ALIGN(db->pc_first);
+        if (host == NULL) {
+            tb_page_addr_t page0, old_page1, new_page1;
+
+            new_page1 = get_page_addr_code_hostp(env, base, &db->host_addr[1]);
+
+            /*
+             * If the second page is MMIO, treat as if the first page
+             * was MMIO as well, so that we do not cache the TB.
+             */
+            if (unlikely(new_page1 == -1)) {
+                tb_unlock_pages(tb);
+                tb_set_page_addr0(tb, -1);
+                return NULL;
+            }
+
+            /*
+             * If this is not the first time around, and page1 matches,
+             * then we already have the page locked.  Alternately, we're
+             * not doing anything to prevent the PTE from changing, so
+             * we might wind up with a different page, requiring us to
+             * re-do the locking.
+             */
+            old_page1 = tb_page_addr1(tb);
+            if (likely(new_page1 != old_page1)) {
+                page0 = tb_page_addr0(tb);
+                if (unlikely(old_page1 != -1)) {
+                    tb_unlock_page1(page0, old_page1);
+                }
+                tb_set_page_addr1(tb, new_page1);
+                tb_lock_page1(page0, new_page1);
+            }
+            host = db->host_addr[1];
+        }
+
+        /* Use slow path when crossing pages. */
+        if (is_same_page(db, pc)) {
+            return NULL;
+        }
    }

- do_read:
-    /*
-     * Assume aligned reads should be atomic, if possible.
-     * We're not in a position to jump out with EXCP_ATOMIC.
-     */
-    host += pc - base;
-    switch (len) {
-    case 2:
-        if (QEMU_IS_ALIGNED(pc, 2)) {
-            uint16_t t = qatomic_read((uint16_t *)host);
-            stw_he_p(dest, t);
-            return true;
-        }
-        break;
-    case 4:
-        if (QEMU_IS_ALIGNED(pc, 4)) {
-            uint32_t t = qatomic_read((uint32_t *)host);
-            stl_he_p(dest, t);
-            return true;
-        }
-        break;
-#ifdef CONFIG_ATOMIC64
-    case 8:
-        if (QEMU_IS_ALIGNED(pc, 8)) {
-            uint64_t t = qatomic_read__nocheck((uint64_t *)host);
-            stq_he_p(dest, t);
-            return true;
-        }
-        break;
-#endif
-    }
-    /* Unaligned or partial read from the second page is not atomic. */
-    memcpy(dest, host, len);
-    return true;
+    tcg_debug_assert(pc >= base);
+    return host + (pc - base);
 }

-static void record_save(DisasContextBase *db, vaddr pc,
-                        const void *from, int size)
+static void plugin_insn_append(abi_ptr pc, const void *from, size_t size)
 {
-    int offset;
+#ifdef CONFIG_PLUGIN
+    struct qemu_plugin_insn *insn = tcg_ctx->plugin_insn;
+    abi_ptr off;

-    /* Do not record probes before the start of TB. */
-    if (pc < db->pc_first) {
+    if (insn == NULL) {
        return;
    }
-
-    /*
-     * In translator_access, we verified that pc is within 2 pages
-     * of pc_first, thus this will never overflow.
-     */
-    offset = pc - db->pc_first;
-
-    /*
-     * Either the first or second page may be I/O.  If it is the second,
-     * then the first byte we need to record will be at a non-zero offset.
-     * In either case, we should not need to record but a single insn.
-     */
-    if (db->record_len == 0) {
-        db->record_start = offset;
-        db->record_len = size;
-    } else {
-        assert(offset == db->record_start + db->record_len);
-        assert(db->record_len + size <= sizeof(db->record));
-        db->record_len += size;
+    off = pc - insn->vaddr;
+    if (off < insn->data->len) {
+        g_byte_array_set_size(insn->data, off);
+    } else if (off > insn->data->len) {
+        /* we have an unexpected gap */
+        g_assert_not_reached();
    }

-    memcpy(db->record + (offset - db->record_start), from, size);
+    insn->data = g_byte_array_append(insn->data, from, size);
+#endif
 }

-size_t translator_st_len(const DisasContextBase *db)
+uint8_t translator_ldub(CPUArchState *env, DisasContextBase *db, abi_ptr pc)
 {
-    return db->fake_insn ? db->record_len : db->tb->size;
+    uint8_t ret;
+    void *p = translator_access(env, db, pc, sizeof(ret));
+
+    if (p) {
+        plugin_insn_append(pc, p, sizeof(ret));
+        return ldub_p(p);
+    }
+    ret = cpu_ldub_code(env, pc);
+    plugin_insn_append(pc, &ret, sizeof(ret));
+    return ret;
 }

-bool translator_st(const DisasContextBase *db, void *dest,
-                   vaddr addr, size_t len)
+uint16_t translator_lduw(CPUArchState *env, DisasContextBase *db, abi_ptr pc)
 {
-    size_t offset, offset_end;
+    uint16_t ret, plug;
+    void *p = translator_access(env, db, pc, sizeof(ret));

-    if (addr < db->pc_first) {
-        return false;
+    if (p) {
+        plugin_insn_append(pc, p, sizeof(ret));
+        return lduw_p(p);
    }
-    offset = addr - db->pc_first;
-    offset_end = offset + len;
-    if (offset_end > translator_st_len(db)) {
-        return false;
-    }
-
-    if (!db->fake_insn) {
-        size_t offset_page1 = -(db->pc_first | TARGET_PAGE_MASK);
-
-        /* Get all the bytes from the first page. */
-        if (db->host_addr[0]) {
-            if (offset_end <= offset_page1) {
-                memcpy(dest, db->host_addr[0] + offset, len);
-                return true;
-            }
-            if (offset < offset_page1) {
-                size_t len0 = offset_page1 - offset;
-                memcpy(dest, db->host_addr[0] + offset, len0);
-                offset += len0;
-                dest += len0;
-            }
-        }
-
-        /* Get any bytes from the second page. */
-        if (db->host_addr[1] && offset >= offset_page1) {
-            memcpy(dest, db->host_addr[1] + (offset - offset_page1),
-                   offset_end - offset);
-            return true;
-        }
-    }
-
-    /* Else get recorded bytes. */
-    if (db->record_len != 0 &&
-        offset >= db->record_start &&
-        offset_end <= db->record_start + db->record_len) {
-        memcpy(dest, db->record + (offset - db->record_start),
-               offset_end - offset);
-        return true;
-    }
-    return false;
+    ret = cpu_lduw_code(env, pc);
+    plug = tswap16(ret);
+    plugin_insn_append(pc, &plug, sizeof(ret));
+    return ret;
 }

-uint8_t translator_ldub(CPUArchState *env, DisasContextBase *db, vaddr pc)
+uint32_t translator_ldl(CPUArchState *env, DisasContextBase *db, abi_ptr pc)
 {
-    uint8_t val;
+    uint32_t ret, plug;
+    void *p = translator_access(env, db, pc, sizeof(ret));

-    if (!translator_ld(env, db, &val, pc, sizeof(val))) {
-        MemOpIdx oi = make_memop_idx(MO_UB, db->code_mmuidx);
-        val = cpu_ldb_code_mmu(env, pc, oi, 0);
-        record_save(db, pc, &val, sizeof(val));
+    if (p) {
+        plugin_insn_append(pc, p, sizeof(ret));
+        return ldl_p(p);
    }
-    return val;
+    ret = cpu_ldl_code(env, pc);
+    plug = tswap32(ret);
+    plugin_insn_append(pc, &plug, sizeof(ret));
+    return ret;
 }

-uint16_t translator_lduw_end(CPUArchState *env, DisasContextBase *db,
-                             vaddr pc, MemOp endian)
+uint64_t translator_ldq(CPUArchState *env, DisasContextBase *db, abi_ptr pc)
 {
-    uint16_t val;
+    uint64_t ret, plug;
+    void *p = translator_access(env, db, pc, sizeof(ret));

-    if (!translator_ld(env, db, &val, pc, sizeof(val))) {
-        MemOpIdx oi = make_memop_idx(MO_UW, db->code_mmuidx);
-        val = cpu_ldw_code_mmu(env, pc, oi, 0);
-        record_save(db, pc, &val, sizeof(val));
+    if (p) {
+        plugin_insn_append(pc, p, sizeof(ret));
+        return ldq_p(p);
    }
-    if (endian & MO_BSWAP) {
-        val = bswap16(val);
-    }
-    return val;
+    ret = cpu_ldq_code(env, pc);
+    plug = tswap64(ret);
+    plugin_insn_append(pc, &plug, sizeof(ret));
+    return ret;
 }

-uint32_t translator_ldl_end(CPUArchState *env, DisasContextBase *db,
-                            vaddr pc, MemOp endian)
+void translator_fake_ldb(uint8_t insn8, abi_ptr pc)
 {
-    uint32_t val;
-
-    if (!translator_ld(env, db, &val, pc, sizeof(val))) {
-        MemOpIdx oi = make_memop_idx(MO_UL, db->code_mmuidx);
-        val = cpu_ldl_code_mmu(env, pc, oi, 0);
-        record_save(db, pc, &val, sizeof(val));
-    }
-    if (endian & MO_BSWAP) {
-        val = bswap32(val);
-    }
-    return val;
-}
-
-uint64_t translator_ldq_end(CPUArchState *env, DisasContextBase *db,
-                            vaddr pc, MemOp endian)
-{
-    uint64_t val;
-
-    if (!translator_ld(env, db, &val, pc, sizeof(val))) {
-        MemOpIdx oi = make_memop_idx(MO_UQ, db->code_mmuidx);
-        val = cpu_ldq_code_mmu(env, pc, oi, 0);
-        record_save(db, pc, &val, sizeof(val));
-    }
-    if (endian & MO_BSWAP) {
-        val = bswap64(val);
-    }
-    return val;
-}
-
-void translator_fake_ld(DisasContextBase *db, const void *data, size_t len)
-{
-    db->fake_insn = true;
-    record_save(db, db->pc_first, data, len);
+    plugin_insn_append(pc, &insn8, sizeof(insn8));
 }
@@ -1,7 +1,6 @@
 #include "qemu/osdep.h"
 #include "hw/core/cpu.h"
 #include "exec/replay-core.h"
-#include "internal-common.h"

 void cpu_resume(CPUState *cpu)
 {
@@ -19,16 +18,6 @@ void cpu_exec_reset_hold(CPUState *cpu)
 {
 }

-/* User mode emulation does not support softmmu yet.  */
-
-void tlb_init(CPUState *cpu)
-{
-}
-
-void tlb_destroy(CPUState *cpu)
-{
-}
-
 /* User mode emulation does not support record/replay yet.  */

 bool replay_exception(void)
@@ -17,42 +17,25 @@
 * License along with this library; if not, see <http://www.gnu.org/licenses/>.
 */
 #include "qemu/osdep.h"
-#include "accel/tcg/cpu-ops.h"
+#include "hw/core/tcg-cpu-ops.h"
 #include "disas/disas.h"
-#include "exec/vaddr.h"
-#include "exec/tlb-flags.h"
+#include "exec/exec-all.h"
 #include "tcg/tcg.h"
 #include "qemu/bitops.h"
 #include "qemu/rcu.h"
-#include "accel/tcg/cpu-ldst-common.h"
-#include "accel/tcg/helper-retaddr.h"
-#include "accel/tcg/probe.h"
-#include "user/cpu_loop.h"
-#include "user/guest-host.h"
-#include "qemu/main-loop.h"
-#include "user/page-protection.h"
-#include "exec/page-protection.h"
-#include "exec/helper-proto-common.h"
+#include "exec/cpu_ldst.h"
+#include "exec/translate-all.h"
+#include "exec/helper-proto.h"
 #include "qemu/atomic128.h"
-#include "qemu/bswap.h"
-#include "qemu/int128.h"
-#include "trace.h"
+#include "trace/trace-root.h"
 #include "tcg/tcg-ldst.h"
-#include "backend-ldst.h"
 #include "internal-common.h"
-#include "tb-internal.h"
+#include "internal-target.h"

 __thread uintptr_t helper_retaddr;

 //#define DEBUG_SIGNAL

-void cpu_interrupt(CPUState *cpu, int mask)
-{
-    g_assert(bql_locked());
-    cpu->interrupt_request |= mask;
-    qatomic_set(&cpu->neg.icount_decr.u16.high, -1);
-}
-
 /*
 * Adjust the pc to pass to cpu_restore_state; return the memop type.
 */
@@ -126,9 +109,9 @@ MMUAccessType adjust_signal_pc(uintptr_t *pc, bool is_write)
 * guest, we'd end up in an infinite loop of retrying the faulting access.
 */
 bool handle_sigsegv_accerr_write(CPUState *cpu, sigset_t *old_set,
-                                 uintptr_t host_pc, vaddr guest_addr)
+                                 uintptr_t host_pc, abi_ptr guest_addr)
 {
-    switch (page_unprotect(cpu, guest_addr, host_pc)) {
+    switch (page_unprotect(guest_addr, host_pc)) {
    case 0:
        /*
         * Fault not caused by a page marked unwritable to protect
@@ -162,7 +145,7 @@ typedef struct PageFlagsNode {

 static IntervalTreeRoot pageflags_root;

-static PageFlagsNode *pageflags_find(vaddr start, vaddr last)
+static PageFlagsNode *pageflags_find(target_ulong start, target_ulong last)
 {
    IntervalTreeNode *n;

@@ -170,7 +153,8 @@ static PageFlagsNode *pageflags_find(vaddr start, vaddr last)
    return n ? container_of(n, PageFlagsNode, itree) : NULL;
 }

-static PageFlagsNode *pageflags_next(PageFlagsNode *p, vaddr start, vaddr last)
+static PageFlagsNode *pageflags_next(PageFlagsNode *p, target_ulong start,
+                                     target_ulong last)
 {
    IntervalTreeNode *n;

@@ -199,22 +183,13 @@ int walk_memory_regions(void *priv, walk_memory_regions_fn fn)
    return rc;
 }

-static int dump_region(void *opaque, vaddr start, vaddr end, int prot)
+static int dump_region(void *priv, target_ulong start,
+                       target_ulong end, unsigned long prot)
 {
-    FILE *f = opaque;
-    uint64_t mask;
-    int width;
+    FILE *f = (FILE *)priv;

-    if (guest_addr_max <= UINT32_MAX) {
-        mask = UINT32_MAX, width = 8;
-    } else {
-        mask = UINT64_MAX, width = 16;
-    }
-
-    fprintf(f, "%0*" PRIx64 "-%0*" PRIx64 " %0*" PRIx64 " %c%c%c\n",
-            width, start & mask,
-            width, end & mask,
-            width, (end - start) & mask,
+    fprintf(f, TARGET_FMT_lx"-"TARGET_FMT_lx" "TARGET_FMT_lx" %c%c%c\n",
+            start, end, end - start,
            ((prot & PAGE_READ) ? 'r' : '-'),
            ((prot & PAGE_WRITE) ? 'w' : '-'),
            ((prot & PAGE_EXEC) ? 'x' : '-'));
@@ -224,14 +199,14 @@ static int dump_region(void *opaque, vaddr start, vaddr end, int prot)
 /* dump memory mappings */
 void page_dump(FILE *f)
 {
-    int width = guest_addr_max <= UINT32_MAX ? 8 : 16;
+    const int length = sizeof(target_ulong) * 2;

    fprintf(f, "%-*s %-*s %-*s %s\n",
-            width, "start", width, "end", width, "size", "prot");
+            length, "start", length, "end", length, "size", "prot");
    walk_memory_regions(f, dump_region);
 }

-int page_get_flags(vaddr address)
+int page_get_flags(target_ulong address)
 {
    PageFlagsNode *p = pageflags_find(address, address);

@@ -254,7 +229,7 @@ int page_get_flags(vaddr address)
 }

 /* A subroutine of page_set_flags: insert a new node for [start,last]. */
-static void pageflags_create(vaddr start, vaddr last, int flags)
+static void pageflags_create(target_ulong start, target_ulong last, int flags)
 {
    PageFlagsNode *p = g_new(PageFlagsNode, 1);

@@ -265,13 +240,13 @@ static void pageflags_create(vaddr start, vaddr last, int flags)
 }

 /* A subroutine of page_set_flags: remove everything in [start,last]. */
-static bool pageflags_unset(vaddr start, vaddr last)
+static bool pageflags_unset(target_ulong start, target_ulong last)
 {
    bool inval_tb = false;

    while (true) {
        PageFlagsNode *p = pageflags_find(start, last);
-        vaddr p_last;
+        target_ulong p_last;

        if (!p) {
            break;
@@ -310,7 +285,8 @@ static bool pageflags_unset(vaddr start, vaddr last)
 * A subroutine of page_set_flags: nothing overlaps [start,last],
 * but check adjacent mappings and maybe merge into a single range.
 */
-static void pageflags_create_merge(vaddr start, vaddr last, int flags)
+static void pageflags_create_merge(target_ulong start, target_ulong last,
+                                   int flags)
 {
    PageFlagsNode *next = NULL, *prev = NULL;

@@ -361,11 +337,11 @@ static void pageflags_create_merge(vaddr start, vaddr last, int flags)
 #define PAGE_STICKY  (PAGE_ANON | PAGE_PASSTHROUGH | PAGE_TARGET_STICKY)

 /* A subroutine of page_set_flags: add flags to [start,last]. */
-static bool pageflags_set_clear(vaddr start, vaddr last,
+static bool pageflags_set_clear(target_ulong start, target_ulong last,
                                int set_flags, int clear_flags)
 {
    PageFlagsNode *p;
-    vaddr p_start, p_last;
+    target_ulong p_start, p_last;
    int p_flags, merge_flags;
    bool inval_tb = false;

@@ -500,7 +476,12 @@ static bool pageflags_set_clear(vaddr start, vaddr last,
    return inval_tb;
 }

-void page_set_flags(vaddr start, vaddr last, int flags)
+/*
+ * Modify the flags of a page and invalidate the code if necessary.
+ * The flag PAGE_WRITE_ORG is positioned automatically depending
+ * on PAGE_WRITE.  The mmap_lock should already be held.
+ */
+void page_set_flags(target_ulong start, target_ulong last, int flags)
 {
    bool reset = false;
    bool inval_tb = false;
@@ -509,7 +490,7 @@ void page_set_flags(vaddr start, vaddr last, int flags)
       guest address space.  If this assert fires, it probably indicates
       a missing call to h2g_valid.  */
    assert(start <= last);
-    assert(last <= guest_addr_max);
+    assert(last <= GUEST_ADDR_MAX);
    /* Only set PAGE_ANON with new mappings. */
    assert(!(flags & PAGE_ANON) || (flags & PAGE_RESET));
    assert_memory_lock();
@@ -536,13 +517,13 @@ void page_set_flags(vaddr start, vaddr last, int flags)
                                        ~(reset ? 0 : PAGE_STICKY));
    }
    if (inval_tb) {
-        tb_invalidate_phys_range(NULL, start, last);
+        tb_invalidate_phys_range(start, last);
    }
 }

-bool page_check_range(vaddr start, vaddr len, int flags)
+bool page_check_range(target_ulong start, target_ulong len, int flags)
 {
-    vaddr last;
+    target_ulong last;
    int locked;  /* tri-state: =0: unlocked, +1: global, -1: local */
    bool ret;

@@ -591,7 +572,7 @@ bool page_check_range(vaddr start, vaddr len, int flags)
                break;
            }
            /* Asking about writable, but has been protected: undo. */
-            if (!page_unprotect(NULL, start, 0)) {
+            if (!page_unprotect(start, 0)) {
                ret = false;
                break;
            }
@@ -618,19 +599,20 @@ bool page_check_range(vaddr start, vaddr len, int flags)
    return ret;
 }

-bool page_check_range_empty(vaddr start, vaddr last)
+bool page_check_range_empty(target_ulong start, target_ulong last)
 {
    assert(last >= start);
    assert_memory_lock();
    return pageflags_find(start, last) == NULL;
 }

-vaddr page_find_range_empty(vaddr min, vaddr max, vaddr len, vaddr align)
+target_ulong page_find_range_empty(target_ulong min, target_ulong max,
+                                   target_ulong len, target_ulong align)
 {
-    vaddr len_m1, align_m1;
+    target_ulong len_m1, align_m1;

    assert(min <= max);
-    assert(max <= guest_addr_max);
+    assert(max <= GUEST_ADDR_MAX);
    assert(len != 0);
    assert(is_power_of_2(align));
    assert_memory_lock();
@@ -665,10 +647,10 @@ vaddr page_find_range_empty(vaddr min, vaddr max, vaddr len, vaddr align)
    }
 }

-void tb_lock_page0(tb_page_addr_t address)
+void page_protect(tb_page_addr_t address)
 {
    PageFlagsNode *p;
-    vaddr start, last;
+    target_ulong start, last;
    int host_page_size = qemu_real_host_page_size();
    int prot;

@@ -710,13 +692,11 @@ void tb_lock_page0(tb_page_addr_t address)
 * immediately exited. (We can only return 2 if the 'pc' argument is
 * non-zero.)
 */
-int page_unprotect(CPUState *cpu, tb_page_addr_t address, uintptr_t pc)
+int page_unprotect(target_ulong address, uintptr_t pc)
 {
    PageFlagsNode *p;
    bool current_tb_invalidated;

-    assert((cpu == NULL) == (pc == 0));
-
    /*
     * Technically this isn't safe inside a signal handler.  However we
     * know this only ever happens in a synchronous SEGV handler, so in
@@ -739,15 +719,15 @@ int page_unprotect(CPUState *cpu, tb_page_addr_t address, uintptr_t pc)
         * this thread raced with another one which got here first and
         * set the page to PAGE_WRITE and did the TB invalidate for us.
         */
-        if (pc && cpu->cc->tcg_ops->precise_smc) {
-            TranslationBlock *current_tb = tcg_tb_lookup(pc);
-            if (current_tb) {
-                current_tb_invalidated = tb_cflags(current_tb) & CF_INVALID;
-            }
+#ifdef TARGET_HAS_PRECISE_SMC
+        TranslationBlock *current_tb = tcg_tb_lookup(pc);
+        if (current_tb) {
+            current_tb_invalidated = tb_cflags(current_tb) & CF_INVALID;
        }
+#endif
    } else {
        int host_page_size = qemu_real_host_page_size();
-        vaddr start, len, i;
+        target_ulong start, len, i;
        int prot;

        if (host_page_size <= TARGET_PAGE_SIZE) {
@@ -755,15 +735,14 @@ int page_unprotect(CPUState *cpu, tb_page_addr_t address, uintptr_t pc)
            len = TARGET_PAGE_SIZE;
            prot = p->flags | PAGE_WRITE;
            pageflags_set_clear(start, start + len - 1, PAGE_WRITE, 0);
-            current_tb_invalidated =
-                tb_invalidate_phys_page_unwind(cpu, start, pc);
+            current_tb_invalidated = tb_invalidate_phys_page_unwind(start, pc);
        } else {
            start = address & -host_page_size;
            len = host_page_size;
            prot = 0;

            for (i = 0; i < len; i += TARGET_PAGE_SIZE) {
-                vaddr addr = start + i;
+                target_ulong addr = start + i;

                p = pageflags_find(addr, addr);
                if (p) {
@@ -779,13 +758,13 @@ int page_unprotect(CPUState *cpu, tb_page_addr_t address, uintptr_t pc)
                 * the corresponding translated code.
                 */
                current_tb_invalidated |=
-                    tb_invalidate_phys_page_unwind(cpu, addr, pc);
+                    tb_invalidate_phys_page_unwind(addr, pc);
            }
        }
        if (prot & PAGE_EXEC) {
            prot = (prot & ~PAGE_EXEC) | PAGE_READ;
        }
-        mprotect((void *)g2h_untagged(start), len, prot & PAGE_RWX);
+        mprotect((void *)g2h_untagged(start), len, prot & PAGE_BITS);
    }
    mmap_unlock();

@@ -817,7 +796,7 @@ static int probe_access_internal(CPUArchState *env, vaddr addr,
    if (guest_addr_valid_untagged(addr)) {
        int page_flags = page_get_flags(addr);
        if (page_flags & acc_flag) {
-            if (access_type != MMU_INST_FETCH
+            if ((acc_flag == PAGE_READ || acc_flag == PAGE_WRITE)
                && cpu_plugin_mem_cbs_enabled(env_cpu(env))) {
                return TLB_MMIO;
            }
@@ -859,12 +838,6 @@ void *probe_access(CPUArchState *env, vaddr addr, int size,
    return size ? g2h(env_cpu(env), addr) : NULL;
 }

-void *tlb_vaddr_to_host(CPUArchState *env, vaddr addr,
-                        MMUAccessType access_type, int mmu_idx)
-{
-    return g2h(env_cpu(env), addr);
-}
-
 tb_page_addr_t get_page_addr_code_hostp(CPUArchState *env, vaddr addr,
                                        void **hostp)
 {
@@ -879,6 +852,7 @@ tb_page_addr_t get_page_addr_code_hostp(CPUArchState *env, vaddr addr,
    return addr;
 }

+#ifdef TARGET_PAGE_DATA_SIZE
 /*
 * Allocate chunks of target data together.  For the only current user,
 * if we allocate one hunk per page, we have overhead of 40/128 or 40%.
@@ -894,16 +868,10 @@ typedef struct TargetPageDataNode {
 } TargetPageDataNode;

 static IntervalTreeRoot targetdata_root;
-static size_t target_page_data_size;

-void page_reset_target_data(vaddr start, vaddr last)
+void page_reset_target_data(target_ulong start, target_ulong last)
 {
    IntervalTreeNode *n, *next;
-    size_t size = target_page_data_size;
-
-    if (likely(size == 0)) {
-        return;
-    }

    assert_memory_lock();

@@ -915,7 +883,7 @@ void page_reset_target_data(vaddr start, vaddr last)
         n != NULL;
         n = next,
         next = next ? interval_tree_iter_next(n, start, last) : NULL) {
-        vaddr n_start, n_last, p_ofs, p_len;
+        target_ulong n_start, n_last, p_ofs, p_len;
        TargetPageDataNode *t = container_of(n, TargetPageDataNode, itree);

        if (n->start >= start && n->last <= last) {
@@ -934,21 +902,16 @@ void page_reset_target_data(vaddr start, vaddr last)
        n_last = MIN(last, n->last);
        p_len = (n_last + 1 - n_start) >> TARGET_PAGE_BITS;

-        memset(t->data + p_ofs * size, 0, p_len * size);
+        memset(t->data + p_ofs * TARGET_PAGE_DATA_SIZE, 0,
+               p_len * TARGET_PAGE_DATA_SIZE);
    }
 }

-void *page_get_target_data(vaddr address, size_t size)
+void *page_get_target_data(target_ulong address)
 {
    IntervalTreeNode *n;
    TargetPageDataNode *t;
-    vaddr page, region, p_ofs;
-
-    /* Remember the size from the first call, and it should be constant. */
-    if (unlikely(target_page_data_size != size)) {
-        assert(target_page_data_size == 0);
-        target_page_data_size = size;
-    }
+    target_ulong page, region, p_ofs;

    page = address & TARGET_PAGE_MASK;
    region = address & TBD_MASK;
@@ -964,7 +927,8 @@ void *page_get_target_data(vaddr address, size_t size)
        mmap_lock();
        n = interval_tree_iter_first(&targetdata_root, page, page);
        if (!n) {
-            t = g_malloc0(sizeof(TargetPageDataNode) + TPD_PAGES * size);
+            t = g_malloc0(sizeof(TargetPageDataNode)
+                          + TPD_PAGES * TARGET_PAGE_DATA_SIZE);
            n = &t->itree;
            n->start = region;
            n->last = region | ~TBD_MASK;
@@ -975,15 +939,18 @@ void *page_get_target_data(vaddr address, size_t size)

    t = container_of(n, TargetPageDataNode, itree);
    p_ofs = (page - region) >> TARGET_PAGE_BITS;
-    return t->data + p_ofs * size;
+    return t->data + p_ofs * TARGET_PAGE_DATA_SIZE;
 }
+#else
+void page_reset_target_data(target_ulong start, target_ulong last) { }
+#endif /* TARGET_PAGE_DATA_SIZE */

 /* The system-mode versions of these helpers are in cputlb.c.  */

 static void *cpu_mmu_lookup(CPUState *cpu, vaddr addr,
                            MemOp mop, uintptr_t ra, MMUAccessType type)
 {
-    int a_bits = memop_alignment_bits(mop);
+    int a_bits = get_alignment_bits(mop);
    void *ret;

    /* Enforce guest required alignment.  */
@@ -996,85 +963,6 @@ static void *cpu_mmu_lookup(CPUState *cpu, vaddr addr,
    return ret;
 }

-/* physical memory access (slow version, mainly for debug) */
-int cpu_memory_rw_debug(CPUState *cpu, vaddr addr,
-                        void *ptr, size_t len, bool is_write)
-{
-    int flags;
-    vaddr l, page;
-    uint8_t *buf = ptr;
-    ssize_t written;
-    int ret = -1;
-    int fd = -1;
-
-    mmap_lock();
-
-    while (len > 0) {
-        page = addr & TARGET_PAGE_MASK;
-        l = (page + TARGET_PAGE_SIZE) - addr;
-        if (l > len) {
-            l = len;
-        }
-        flags = page_get_flags(page);
-        if (!(flags & PAGE_VALID)) {
-            goto out_close;
-        }
-        if (is_write) {
-            if (flags & PAGE_WRITE) {
-                memcpy(g2h(cpu, addr), buf, l);
-            } else {
-                /* Bypass the host page protection using ptrace. */
-                if (fd == -1) {
-                    fd = open("/proc/self/mem", O_WRONLY);
-                    if (fd == -1) {
-                        goto out;
-                    }
-                }
-                /*
-                 * If there is a TranslationBlock and we weren't bypassing the
-                 * host page protection, the memcpy() above would SEGV,
-                 * ultimately leading to page_unprotect(). So invalidate the
-                 * translations manually. Both invalidation and pwrite() must
-                 * be under mmap_lock() in order to prevent the creation of
-                 * another TranslationBlock in between.
-                 */
-                tb_invalidate_phys_range(NULL, addr, addr + l - 1);
-                written = pwrite(fd, buf, l,
-                                 (off_t)(uintptr_t)g2h_untagged(addr));
-                if (written != l) {
-                    goto out_close;
-                }
-            }
-        } else if (flags & PAGE_READ) {
-            memcpy(buf, g2h(cpu, addr), l);
-        } else {
-            /* Bypass the host page protection using ptrace. */
-            if (fd == -1) {
-                fd = open("/proc/self/mem", O_RDONLY);
-                if (fd == -1) {
-                    goto out;
-                }
-            }
-            if (pread(fd, buf, l,
-                      (off_t)(uintptr_t)g2h_untagged(addr)) != l) {
-                goto out_close;
-            }
-        }
-        len -= l;
-        buf += l;
-        addr += l;
-    }
-    ret = 0;
-out_close:
-    if (fd != -1) {
-        close(fd);
-    }
-out:
-    mmap_unlock();
-
-    return ret;
-}
-
 #include "ldst_atomicity.c.inc"

 static uint8_t do_ld1_mmu(CPUState *cpu, vaddr addr, MemOpIdx oi,
@@ -1083,7 +971,7 @@ static uint8_t do_ld1_mmu(CPUState *cpu, vaddr addr, MemOpIdx oi,
    void *haddr;
    uint8_t ret;

-    cpu_req_mo(cpu, TCG_MO_LD_LD | TCG_MO_ST_LD);
+    cpu_req_mo(TCG_MO_LD_LD | TCG_MO_ST_LD);
    haddr = cpu_mmu_lookup(cpu, addr, get_memop(oi), ra, access_type);
    ret = ldub_p(haddr);
    clear_helper_retaddr();
@@ -1097,7 +985,7 @@ static uint16_t do_ld2_mmu(CPUState *cpu, vaddr addr, MemOpIdx oi,
    uint16_t ret;
    MemOp mop = get_memop(oi);

-    cpu_req_mo(cpu, TCG_MO_LD_LD | TCG_MO_ST_LD);
+    cpu_req_mo(TCG_MO_LD_LD | TCG_MO_ST_LD);
    haddr = cpu_mmu_lookup(cpu, addr, mop, ra, access_type);
    ret = load_atom_2(cpu, ra, haddr, mop);
    clear_helper_retaddr();
@@ -1115,7 +1003,7 @@ static uint32_t do_ld4_mmu(CPUState *cpu, vaddr addr, MemOpIdx oi,
    uint32_t ret;
    MemOp mop = get_memop(oi);

-    cpu_req_mo(cpu, TCG_MO_LD_LD | TCG_MO_ST_LD);
+    cpu_req_mo(TCG_MO_LD_LD | TCG_MO_ST_LD);
    haddr = cpu_mmu_lookup(cpu, addr, mop, ra, access_type);
    ret = load_atom_4(cpu, ra, haddr, mop);
    clear_helper_retaddr();
@@ -1133,7 +1021,7 @@ static uint64_t do_ld8_mmu(CPUState *cpu, vaddr addr, MemOpIdx oi,
    uint64_t ret;
    MemOp mop = get_memop(oi);

-    cpu_req_mo(cpu, TCG_MO_LD_LD | TCG_MO_ST_LD);
+    cpu_req_mo(TCG_MO_LD_LD | TCG_MO_ST_LD);
    haddr = cpu_mmu_lookup(cpu, addr, mop, ra, access_type);
    ret = load_atom_8(cpu, ra, haddr, mop);
    clear_helper_retaddr();
@@ -1144,7 +1032,7 @@ static uint64_t do_ld8_mmu(CPUState *cpu, vaddr addr, MemOpIdx oi,
    return ret;
 }

-static Int128 do_ld16_mmu(CPUState *cpu, vaddr addr,
+static Int128 do_ld16_mmu(CPUState *cpu, abi_ptr addr,
                          MemOpIdx oi, uintptr_t ra)
 {
    void *haddr;
@@ -1152,7 +1040,7 @@ static Int128 do_ld16_mmu(CPUState *cpu, vaddr addr,
    MemOp mop = get_memop(oi);

    tcg_debug_assert((mop & MO_SIZE) == MO_128);
-    cpu_req_mo(cpu, TCG_MO_LD_LD | TCG_MO_ST_LD);
+    cpu_req_mo(TCG_MO_LD_LD | TCG_MO_ST_LD);
    haddr = cpu_mmu_lookup(cpu, addr, mop, ra, MMU_DATA_LOAD);
    ret = load_atom_16(cpu, ra, haddr, mop);
    clear_helper_retaddr();
@@ -1168,7 +1056,7 @@ static void do_st1_mmu(CPUState *cpu, vaddr addr, uint8_t val,
 {
    void *haddr;

-    cpu_req_mo(cpu, TCG_MO_LD_ST | TCG_MO_ST_ST);
+    cpu_req_mo(TCG_MO_LD_ST | TCG_MO_ST_ST);
    haddr = cpu_mmu_lookup(cpu, addr, get_memop(oi), ra, MMU_DATA_STORE);
    stb_p(haddr, val);
    clear_helper_retaddr();
@@ -1180,7 +1068,7 @@ static void do_st2_mmu(CPUState *cpu, vaddr addr, uint16_t val,
    void *haddr;
    MemOp mop = get_memop(oi);

-    cpu_req_mo(cpu, TCG_MO_LD_ST | TCG_MO_ST_ST);
+    cpu_req_mo(TCG_MO_LD_ST | TCG_MO_ST_ST);
    haddr = cpu_mmu_lookup(cpu, addr, mop, ra, MMU_DATA_STORE);

    if (mop & MO_BSWAP) {
@@ -1196,7 +1084,7 @@ static void do_st4_mmu(CPUState *cpu, vaddr addr, uint32_t val,
    void *haddr;
    MemOp mop = get_memop(oi);

-    cpu_req_mo(cpu, TCG_MO_LD_ST | TCG_MO_ST_ST);
+    cpu_req_mo(TCG_MO_LD_ST | TCG_MO_ST_ST);
    haddr = cpu_mmu_lookup(cpu, addr, mop, ra, MMU_DATA_STORE);

    if (mop & MO_BSWAP) {
@@ -1212,7 +1100,7 @@ static void do_st8_mmu(CPUState *cpu, vaddr addr, uint64_t val,
    void *haddr;
    MemOp mop = get_memop(oi);

-    cpu_req_mo(cpu, TCG_MO_LD_ST | TCG_MO_ST_ST);
+    cpu_req_mo(TCG_MO_LD_ST | TCG_MO_ST_ST);
    haddr = cpu_mmu_lookup(cpu, addr, mop, ra, MMU_DATA_STORE);

    if (mop & MO_BSWAP) {
@@ -1228,7 +1116,7 @@ static void do_st16_mmu(CPUState *cpu, vaddr addr, Int128 val,
    void *haddr;
    MemOpIdx mop = get_memop(oi);

-    cpu_req_mo(cpu, TCG_MO_LD_ST | TCG_MO_ST_ST);
+    cpu_req_mo(TCG_MO_LD_ST | TCG_MO_ST_ST);
    haddr = cpu_mmu_lookup(cpu, addr, mop, ra, MMU_DATA_STORE);

    if (mop & MO_BSWAP) {
@@ -1238,28 +1126,101 @@ static void do_st16_mmu(CPUState *cpu, vaddr addr, Int128 val,
    clear_helper_retaddr();
 }

-uint8_t cpu_ldb_code_mmu(CPUArchState *env, vaddr addr,
+uint32_t cpu_ldub_code(CPUArchState *env, abi_ptr ptr)
+{
+    uint32_t ret;
+
+    set_helper_retaddr(1);
+    ret = ldub_p(g2h_untagged(ptr));
+    clear_helper_retaddr();
+    return ret;
+}
+
+uint32_t cpu_lduw_code(CPUArchState *env, abi_ptr ptr)
+{
+    uint32_t ret;
+
+    set_helper_retaddr(1);
+    ret = lduw_p(g2h_untagged(ptr));
+    clear_helper_retaddr();
+    return ret;
+}
+
+uint32_t cpu_ldl_code(CPUArchState *env, abi_ptr ptr)
+{
+    uint32_t ret;
+
+    set_helper_retaddr(1);
+    ret = ldl_p(g2h_untagged(ptr));
+    clear_helper_retaddr();
+    return ret;
+}
+
+uint64_t cpu_ldq_code(CPUArchState *env, abi_ptr ptr)
+{
+    uint64_t ret;
+
+    set_helper_retaddr(1);
+    ret = ldq_p(g2h_untagged(ptr));
+    clear_helper_retaddr();
+    return ret;
+}
+
+uint8_t cpu_ldb_code_mmu(CPUArchState *env, abi_ptr addr,
                         MemOpIdx oi, uintptr_t ra)
 {
-    return do_ld1_mmu(env_cpu(env), addr, oi, ra ? ra : 1, MMU_INST_FETCH);
+    void *haddr;
+    uint8_t ret;
+
+    haddr = cpu_mmu_lookup(env_cpu(env), addr, oi, ra, MMU_INST_FETCH);
+    ret = ldub_p(haddr);
+    clear_helper_retaddr();
+    return ret;
 }

-uint16_t cpu_ldw_code_mmu(CPUArchState *env, vaddr addr,
+uint16_t cpu_ldw_code_mmu(CPUArchState *env, abi_ptr addr,
                          MemOpIdx oi, uintptr_t ra)
 {
-    return do_ld2_mmu(env_cpu(env), addr, oi, ra ? ra : 1, MMU_INST_FETCH);
+    void *haddr;
+    uint16_t ret;
+
+    haddr = cpu_mmu_lookup(env_cpu(env), addr, oi, ra, MMU_INST_FETCH);
+    ret = lduw_p(haddr);
+    clear_helper_retaddr();
+    if (get_memop(oi) & MO_BSWAP) {
+        ret = bswap16(ret);
+    }
+    return ret;
 }

-uint32_t cpu_ldl_code_mmu(CPUArchState *env, vaddr addr,
+uint32_t cpu_ldl_code_mmu(CPUArchState *env, abi_ptr addr,
                          MemOpIdx oi, uintptr_t ra)
 {
-    return do_ld4_mmu(env_cpu(env), addr, oi, ra ? ra : 1, MMU_INST_FETCH);
+    void *haddr;
+    uint32_t ret;
+
+    haddr = cpu_mmu_lookup(env_cpu(env), addr, oi, ra, MMU_INST_FETCH);
+    ret = ldl_p(haddr);
+    clear_helper_retaddr();
+    if (get_memop(oi) & MO_BSWAP) {
+        ret = bswap32(ret);
+    }
+    return ret;
 }

-uint64_t cpu_ldq_code_mmu(CPUArchState *env, vaddr addr,
+uint64_t cpu_ldq_code_mmu(CPUArchState *env, abi_ptr addr,
                          MemOpIdx oi, uintptr_t ra)
 {
-    return do_ld8_mmu(env_cpu(env), addr, oi, ra ? ra : 1, MMU_INST_FETCH);
+    void *haddr;
+    uint64_t ret;
+
+    haddr = cpu_mmu_lookup(env_cpu(env), addr, oi, ra, MMU_DATA_LOAD);
+    ret = ldq_p(haddr);
+    clear_helper_retaddr();
+    if (get_memop(oi) & MO_BSWAP) {
+        ret = bswap64(ret);
+    }
+    return ret;
 }

 #include "ldst_common.c.inc"
@@ -1271,7 +1232,7 @@ static void *atomic_mmu_lookup(CPUState *cpu, vaddr addr, MemOpIdx oi,
                               int size, uintptr_t retaddr)
 {
    MemOp mop = get_memop(oi);
-    int a_bits = memop_alignment_bits(mop);
+    int a_bits = get_alignment_bits(mop);
    void *ret;

    /* Enforce guest required alignment.  */
@@ -1,23 +0,0 @@
-/*
- * TaskState helpers for QEMU
- *
- * Copyright (c) 2023 Linaro Ltd.
- *
- * Authors:
- *   Philippe Mathieu-Daudé
- *
- * SPDX-License-Identifier: GPL-2.0-or-later
- */
-#ifndef ACCEL_TCG_VCPU_STATE_H
-#define ACCEL_TCG_VCPU_STATE_H
-
-#include "hw/core/cpu.h"
-
-#ifdef CONFIG_USER_ONLY
-static inline TaskState *get_task_state(const CPUState *cs)
-{
-    return cs->opaque;
-}
-#endif
-
-#endif
--- a/Show More
+++ b/Show More
Author	SHA1	Message	Date
Paolo Bonzini	9173d5d084	gitlab-ci: add manual job to run Coverity Add a job that can be run, either manually or on a schedule, to upload a build to Coverity Scan. The job uses the run-coverity-scan script in multiple phases of check, download tools and upload, in order to avoid both wasting time (skip everything if you are above the upload quota) and avoid filling the log with the progress of downloading the tools. The job is intended to run on a scheduled pipeline run, and scheduled runs will not get any other job. It requires two variables to be in GitLab CI, COVERITY_TOKEN and COVERITY_EMAIL. Those are already set up in qemu-project's configuration as protected and masked variables. Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>	2024-03-05 20:07:30 +01:00
Paolo Bonzini	d8f4a7a80e	run-coverity-scan: add --check-upload-only option Add an option to check if upload is permitted without actually attempting a build. This can be useful to add a third outcome beyond success and failure---namely, a CI job can self-cancel if the uploading quota has been reached. Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>	2024-03-05 12:32:09 +01:00
Paolo Bonzini	c80a048f38	target/i386: remove mask from CCPrepare With the introduction of TSTEQ and TSTNE the .mask field is always -1, so remove all the now-unnecessary code. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>	2024-03-05 12:32:09 +01:00
Paolo Bonzini	af87044127	target/i386: use TSTEQ/TSTNE to check flags The new conditions obviously come in handy when testing individual bits of EFLAGS, and they make it possible to remove the .mask field of CCPrepare. Lowering to shift+and is done by the optimizer if necessary. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>	2024-03-05 12:32:09 +01:00
Paolo Bonzini	395d5d09c7	target/i386: use TSTEQ/TSTNE to test low bits When testing the sign bit or equality to zero of a partial register, it is useful to use a single TSTEQ or TSTNE operation. It can also be used to test the parity flag, using bit 0 of the population count. Do not do this for target_ulong-sized values however; the optimizer would produce a comparison against zero anyway, and it avoids shifts by 64 which are undefined behavior. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>	2024-03-05 12:32:09 +01:00
Paolo Bonzini	ac056cbb54	mips: do not list individual devices from configs/ Add new "select" and "imply" directives if needed. The resulting config-devices.mak files are the same as before. Builds without default devices will become much smaller than before, and qtests fail (as expected, though suboptimal) for mips64-softmmu because most tests do not use -nodefaults, so remove it from build-without-defaults Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>	2024-03-05 12:32:09 +01:00
Paolo Bonzini	f0060d4691	oslib-posix: fix memory leak in touch_all_pages touch_all_pages() can return early, before creating threads. In this case, however, it leaks the MemsetContext that it has allocated at the beginning of the function. Reported by Coverity as CID 1534922. Fixes: `04accf43df` ("oslib-posix: initialize backend memory objects in parallel", 2024-02-06) Reviewed-by: Mark Kanda <mark.kanda@oracle.com> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>	2024-03-05 12:32:09 +01:00
Paolo Bonzini	0170f3ea3d	hw/intc/apic: fix memory leak deliver_bitmask is allocated on the heap in apic_deliver(), but there are many paths in the function that return before the corresponding g_free() is reached. Fix this by switching to g_autofree and, while at it, also switch to g_new. Do the same in apic_deliver_irq() as well for consistency. Fixes: `b5ee0468e9` ("apic: add support for x2APIC mode", 2024-02-14) Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org> Reviewed-by: Bui Quang Minh <minhquangbui99@gmail.com> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>	2024-03-05 12:32:09 +01:00
Sven Schnelle	9ae56d2e46	hw/scsi/lsi53c895a: stop script on phase mismatch Netbsd isn't happy with qemu lsi53c895a emulation: cd0(esiop0:0:2:0): command with tag id 0 reset esiop0: autoconfiguration error: phase mismatch without command esiop0: autoconfiguration error: unhandled scsi interrupt, sist=0x80 sstat1=0x0 DSA=0x23a64b1 DSP=0x50 This is because lsi_bad_phase() triggers a phase mismatch, which stops SCRIPT processing. However, after returning to lsi_command_complete(), SCRIPT is restarted with lsi_resume_script(). Fix this by adding a return value to lsi_bad_phase(), and only resume script processing when lsi_bad_phase() didn't trigger a host interrupt. Signed-off-by: Sven Schnelle <svens@stackframe.org> Tested-by: Helge Deller <deller@gmx.de> Message-ID: <20240302214453.2071388-1-svens@stackframe.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>	2024-03-05 10:46:48 +01:00
Akihiko Odaki	6ed0b8431c	meson: Remove --warn-common ldflag --warn-common ldflag causes warnings for multiple definitions of ___asan_globals_registered when enabling AddressSanitizer with clang. The warning is somewhat obsolete so just remove it. The common block is used to allow duplicate definitions of uninitialized global variables. In the past, GCC and clang used to place such variables in a common block by default, which prevented programmers for noticing accidental duplicate definitions. Commit `49237acdb7` ("Enable ld flag --warn-common") added --warn-common ldflag so that ld warns in such a case. Today, both of GCC and clang don't use common blocks by default[1][2] so any remaining use of common blocks should be intentional. Remove --warn-common ldflag to suppress warnings for intentional use of common blocks. [1]: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=85678 [2]: https://reviews.llvm.org/D75056 Signed-off-by: Akihiko Odaki <akihiko.odaki@daynix.com> Message-ID: <20240304-common-v1-1-1a2005d1f350@daynix.com> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>	2024-03-05 10:46:48 +01:00
Dmitrii Gavrilov	50a715c77c	system/qdev-monitor: move drain_call_rcu call under if (!dev) in qmp_device_add() Original goal of addition of drain_call_rcu to qmp_device_add was to cover the failure case of qdev_device_add. It seems call of drain_call_rcu was misplaced in `7bed89958b` what led to waiting for pending RCU callbacks under happy path too. What led to overall performance degradation of qmp_device_add. In this patch call of drain_call_rcu moved under handling of failure of qdev_device_add. Signed-off-by: Dmitrii Gavrilov <ds-gavr@yandex-team.ru> Message-ID: <20231103105602.90475-1-ds-gavr@yandex-team.ru> Fixes: `7bed89958b` ("device_core: use drain_call_rcu in in qmp_device_add", 2020-10-12) Cc: qemu-stable@nongnu.org Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>	2024-03-05 10:46:48 +01:00
Sven Schnelle	efb1c1d294	hw/scsi/lsi53c895a: add timer to scripts processing HP-UX 10.20 seems to make the lsi53c895a spinning on a memory location under certain circumstances. As the SCSI controller and CPU are not running at the same time this loop will never finish. After some time, the check loop interrupts with a unexpected device disconnect. This works, but is slow because the kernel resets the scsi controller. Instead of signaling UDC, start a timer and exit the loop. Until the timer fires, the CPU can process instructions which might changes the memory location. The limit of instructions is also reduced because scripts running on the SCSI processor are usually very short. This keeps the time until the loop is exit short. Suggested-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Sven Schnelle <svens@stackframe.org> Message-ID: <20240229204407.1699260-1-svens@stackframe.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>	2024-03-05 10:46:47 +01:00
@@ -1 +1 @@
 .0.50
 .2.50