Compare commits

...

71 Commits
v4 ... v23

Author SHA1 Message Date
William Douglas 936534a889 abireport: run nm output through c++filt
Signed-off-by: William Douglas <william.douglas@intel.com>
2025-04-02 14:36:34 -07:00
William Douglas d73a1e70d8 Move hwdb.d files to -config subpackage
These are udev files that really belong with the rules.d files so put
them in config as well.

Signed-off-by: William Douglas <william.douglas@intel.com>
2025-03-28 14:35:20 -07:00
William Douglas f07e061437 Allow requires_ban to override defaults
In cases where a built-in default for a requirement exists (bin
requires conf for example) the bin_requires_ban was not honored. This
change enables the ban to go into effect.

Signed-off-by: William Douglas <william.douglas@intel.com>
2025-03-26 11:00:57 -07:00
William Douglas d4144f5efd Ignore extraction failures on absolute symlinks
Don't attempt to extract these but let the build continue when source
archives have absolute symlinks.

Signed-off-by: William Douglas <william.douglas@intel.com>
2025-03-14 18:52:01 -07:00
William Douglas 07a959cc83 Ensure content separation for cargo's config.toml
The .cargo/config.toml file may exist for a project and the source
content configuration that we append needs to have a newline
separating it from the existing content.

Signed-off-by: William Douglas <william.douglas@intel.com>
2025-02-28 12:47:03 -08:00
Brett T. Warden e36a856c50 Add test URL for libopenmpt 2025-02-12 10:19:56 -08:00
Brett T. Warden 2618dc3eb1 Fix version parsing for libopenmpt
Given tarball libopenmpt-0.7.13+release.autotools.tar.gz, ban
'autotools' from the version number, and replace '+' with '.', so
instead of 0.7.13+.autotools, generate simply 0.7.13.
2025-02-12 10:19:56 -08:00
William Douglas 8b9384758b When copying archives, also copy any hidden files
Previously copying archives would miss hidden files as '*' doesn't
match files with '.' prefix.

Signed-off-by: William Douglas <william.douglas@intel.com>
2025-02-07 11:45:25 -08:00
William Douglas 94c6be068b Add file to skip verification for a single version
Sometimes a package will make a release without verification we
support, in these cases it is nice to be able to toggle off
verification just for that release. Add a new configuration file that
contains the version to not validate.

This file could be extended in the future to support adding specific
keys not to use in the verification process in cases where that key
has problems (though this might be better left to another
configuration file).

Signed-off-by: William Douglas <william.douglas@intel.com>
2025-01-21 09:57:09 -08:00
William Douglas f4a13a5a93 Rework cargo pattern use
In cases where a package has both a Makefile and a Cargo.toml try and
use the make pattern (the options.conf flag for cargo vendor still
needs to be changed). When the cargo pattern is being used, use cargo
install as well. More complex cases likely should just be patched into
a Makefile and perhaps submitted upstream.

Signed-off-by: William Douglas <william.douglas@intel.com>
2025-01-02 16:49:26 -08:00
Brett T. Warden 5424026110 Add R-project mirror domain for build pattern detection
We use ftp.osuosl.org/pub/cran/ as a CRAN mirror, and autospec uses the
URL to select the R build pattern, so make sure that URL is also
included.
2024-11-14 15:56:24 -08:00
William Douglas e822d6e48d Set extractall filter for tarfile
See
https://docs.python.org/3/library/tarfile.html#tarfile.TarFile.extractall

Signed-off-by: William Douglas <william.douglas@intel.com>
2024-11-04 11:40:24 -08:00
William Douglas 9bfe801c96 Fix subdir for pyproject
When using optimized builds for pyproject patterns that also need a
subdir, the path needs to be reset before attempting to change to the
optimized directory.

Signed-off-by: William Douglas <william.douglas@intel.com>
2024-11-04 11:13:46 -08:00
K1ngfish3r 412ce5ee2e i blame isort setting(?) 2024-10-27 22:39:42 -07:00
K1ngfish3r 1fa3bdd6e0 add subdir 2024-10-27 22:39:42 -07:00
William Douglas 4ea76c998e Remove %check for setup.py test
setup.py test is gone in 3.13 and projects seem to be doing custom
things for replacements. Until wider per project detection is in
place, just remove the %check section for setup.py.

Signed-off-by: William Douglas <william.douglas@intel.com>
2024-10-27 20:28:28 -07:00
Brett T. Warden b2d28bb55a Add path to Qt6 build tools in qmake build_pattern
When using the qmake build pattern, add the path /usr/lib64/qt6/bin/
that contains qmake6 and other Qt6 build tools.
2024-10-14 15:56:05 -07:00
Brett T. Warden 4d029647d7 parse_cmake: Also match hypen in pkg_check_modules(X)
If a cmake file defines a pkgconfig dependency with a hypen in it, make
sure we can handle it:
pkg_check_modules(DBUS-1 REQUIRED dbus-1 IMPORTED_TARGET)

Previously we used \w to match the first term (DBUS-1), which does *not*
match hyphens.
2024-10-01 11:40:23 -07:00
Brett T. Warden 5279a11b53 Split packages from cmake_modules when adding them as buildreqs
When parsing cmake files for find_package dependencies, we match against
entries in cmake_modules. Many of the entries of this file list
multiple packages, separated by space. Split on whitespace so we
actually feed only individual package names to each add_buildreq call.

Otherwise, if cmake_modules provides "extra-cmake-modules png2ico", for
example, and you have "png2ico" in buildreq_ban, the specfile would still
list both extra-cmake-modules and png2ico as build dependencies, because
add_buildreq only compared exact matches.
2024-10-01 11:40:23 -07:00
William Douglas a19cdc79b4 Always handle the common tmepfile prefix for license_skips
Modify license skips to handle the common tempfile prefix of '/tmp/*'
and normalize the case where lines lead with a slash vs not.

Also remove a duplicate test for globlike match.

Signed-off-by: William Douglas <william.douglas@intel.com>
2024-09-19 15:09:41 -07:00
K1ngfish3r 185099bba8 use_ninja for %check 2024-09-19 13:11:21 -07:00
William Douglas 7b01f0ba82 Add support for skipping license paths
Add support for a new config file with glob like handling of paths as
some projects contain license files that are not the license of the
project.

scan_for_licenses now looks at the configuration and does glob like
matching for all the potential license files it tries to add.

This change caused a few structural adjustments to the code and a
slight style change for importing just the toplevel util submodule
that are superficial.

Signed-off-by: William Douglas <william.douglas@intel.com>
2024-09-19 12:02:52 -07:00
William Douglas 6a4b23bb3e Update configs
Add new values needed for various package builds to the configuration
files.

Signed-off-by: William Douglas <william.douglas@intel.com>
2024-09-12 09:27:28 -07:00
William Douglas c6d7bdb41c Add new failed pattern
Signed-off-by: William Douglas <william.douglas@intel.com>
2024-09-12 09:27:28 -07:00
William Douglas a16ede7440 Add new test for _process_build_log
Adds the first example test to _process_build_log verifying it can
handle a bad patch.

Signed-off-by: William Douglas <william.douglas@intel.com>
2024-09-12 09:27:28 -07:00
William Douglas a62a849262 Add github url handling for tags with slashes
vectorscan uses 'vectorscan/' as a tag prefix which confuses our
version parsing (even more so with v being the tag prefix). Add
support for matching any '*/version' tag names as the top priority.

Signed-off-by: William Douglas <william.douglas@intel.com>
2024-09-09 12:42:57 -07:00
William Douglas eaa4f711da Fix type mismatch
current_patch is modified in the called as an array so make sure it is
created as one.

Signed-off-by: William Douglas <william.douglas@intel.com>
2024-08-01 16:49:45 -07:00
William Douglas f35655a0cc More error report matching
Signed-off-by: William Douglas <william.douglas@intel.com>
2024-08-01 11:54:35 -07:00
William Douglas 0c573b604b Fix setup line for packages without a prefix folder
Previously in a cleanup commit 74c0833c the support for packages that
didn't have a prefix folder was unintentionally removed. This change
updates the setup call for non-R packages to add the support back.

Signed-off-by: William Douglas <william.douglas@intel.com>
2024-07-31 14:59:15 -07:00
William Douglas 356da62750 Enable working with Jinja2 spec templates
Allow autospec to handle building packages that use a
Jinja2 formatted *.spec.template file. Currently only package_name,
package_version, package_release and package_url fields are supported
but others can be added as needed (patches and archives are good next
steps).

Signed-off-by: William Douglas <william.douglas@intel.com>
2024-07-25 13:06:51 -07:00
William Douglas b5caddc404 Add options flag for avoiding full rebuilds
Add an options.conf flag for allowing builds to use mock's
--short-circuit for reducing rebuild time.

This change moves the feature from default enabling with no way to
turn off to default disable with a configure to turn on.

Signed-off-by: William Douglas <william.douglas@intel.com>
2024-07-22 16:46:29 -07:00
William Douglas 2181c1fe68 Add new error logging
Once we know autospec is going to exit unsuccessfully, send out a
hopefully useful error from print_fatal or scan the build.log one last
time for useful data and add that content to an output file.

This is intended to only be done in update flows and the file should
be cleaned up prior to autospec being run again. Note that without a
special environment variable active nothing will be written out to the
file.

Signed-off-by: William Douglas <william.douglas@intel.com>
2024-07-22 16:46:29 -07:00
William Douglas d6606ad5a8 Ensure print_fatal is always fatal
print_fatal calls need to be followed up by an exit consistently and
exit's need to use print_fatal before being called.

Signed-off-by: William Douglas <william.douglas@intel.com>
2024-07-22 16:46:29 -07:00
William Douglas 4d708b6fe2 Update gpg keyserver
Signed-off-by: William Douglas <william.douglas@intel.com>
2024-07-22 16:46:29 -07:00
Brett T. Warden 1bec16fc2e Remove quotation marks from package modules
For PKG modules required by cmake or autoconf scripts, remove any
quotation marks before using them in pkgconfig().
2024-07-18 11:22:28 -07:00
Brett T. Warden 9f33e630cd Add another failure pattern for Perl dependencies 2024-07-17 16:59:17 -07:00
William Douglas b858a2a990 Add cargo vendor drop in style file
Add support for cargo_vendors which should be pretty much the output
of a 'cargo vendor' run. This will be appended to any existing
.cargo/config.toml file in the sources. It is intended to mostly come
from the vendor.py in common but useful for other builds as well which
don't need to enable cargo_vendor in options.conf.

Signed-off-by: William Douglas <william.douglas@intel.com>
2024-07-08 15:09:34 -07:00
William Douglas 43d564b0b7 Make autoupdate default for new packages
Things should be autoupdating by default for new content at this point
as being held back for no reason isn't helpful.

Signed-off-by: William Douglas <william.douglas@intel.com>
2024-07-08 12:24:12 -07:00
William Douglas 2659038eaa Fix meson check builddir typo
Signed-off-by: William Douglas <william.douglas@intel.com>
2024-07-02 12:17:59 -07:00
William Douglas dc0ff31b43 Add support for a copy_prepend config file
This patch has a few interrelated changes in it but primarily it is
supporting a new copy_prepend configuration file for autospec. This is
intended to support cases where changes should be made to the source
directory prior to the source directory being copied for different
builds (avx2, 32bit, etc).

Also with this change some tweaks to how cmake builds are handled to
be more aligned with other build systems. Primarily that the source
directoy is now fully copied rather than just creating a cmake build
directory per build in the same source directory.

Finally check support has been updated to account for the new path and
update support for meson and apx.

Signed-off-by: William Douglas <william.douglas@intel.com>
2024-07-01 01:36:31 -07:00
William Douglas 3f1fa8e70b Fix mpi cmake Unix Makefiles argument
Signed-off-by: William Douglas <william.douglas@intel.com>
2024-07-01 01:36:31 -07:00
William Douglas 840d2ca0e2 Add new failed pattern detection
Signed-off-by: William Douglas <william.douglas@intel.com>
2024-07-01 01:36:31 -07:00
Brett T. Warden a5d3013703 Add pkgconfig detection pattern for rust dependencies 2024-06-25 13:13:03 -07:00
K1ngfish3r 381dfd88cc make check! 2024-06-20 14:55:48 -07:00
K1ngfish3r f483b68c90 make check? 2024-06-20 14:55:48 -07:00
K1ngfish3r 5d6bcfe2f7 zstd support 2024-06-20 14:55:48 -07:00
Brett T. Warden f9eab4897e Try a little bit harder to find licenses
Look in directories named 'licensing'.
2024-06-18 13:57:32 -07:00
William Douglas fbcebd0b3d Add missing install handling for ninja
For cmake builds, use ninja install if use_ninja is set.

Signed-off-by: William Douglas <william.douglas@intel.com>
2024-05-30 01:15:04 -07:00
William Douglas 1f398f5e7b Add config for using ninja instead of make
Given more packages are using ninja as the build system of choice
instead of make, add flag to enable ninja usage.

Signed-off-by: William Douglas <william.douglas@intel.com>
2024-05-29 16:02:05 -07:00
Arjan van de Ven 8e89d0cb0e no gcc-14 2024-05-29 17:44:43 +00:00
William Douglas 6fa3d52db4 Remove Ofast from flags
This caused some incorrect behavior for numpy. Generally disabling to
avoid problematic behavior.

Signed-off-by: William Douglas <william.douglas@intel.com>
2024-05-14 13:03:20 -07:00
Brett T. Warden 5905be97e8 Fix version parsing for x265 2024-04-12 09:23:30 -07:00
Brett T. Warden 81e1eebe28 Extend key ID matching to expired keys
gpg accepts signatures with expired keys as long as the signature was
made prior to key expiration. But it also changes the status-fd output
format that we grep for the expected key ID. Make sure we look for the
alternate EXPKEYSIG line in the output in that case to find the key ID.
2024-04-12 09:08:10 -07:00
Brett T. Warden 658bd0de10 Add gnupg as a buildreq if we'll need it during build 2024-04-11 16:10:02 -07:00
Brett T. Warden b628caf931 Add Config fields for pkey and signature macros 2024-04-11 12:37:09 -07:00
Brett T. Warden 8142032e7c Fix escaping to satisfy flake 2024-04-11 12:22:59 -07:00
Brett T. Warden 5a302d6c91 Check GPG package signatures during build
Add gpg commands to the specfile so we verify the package signature
during every rpmbuild. Also ensure that the signature key ID matches
what we expect.
2024-04-11 12:22:59 -07:00
William Douglas f56f1fa18d Don't autodetect xz build requirements
Signed-off-by: William Douglas <william.douglas@intel.com>
2024-03-29 15:10:55 -07:00
Brett T. Warden 0c408e7089 Yell louder about known bad signing keys 2024-03-29 11:43:38 -07:00
Brett T. Warden 5471f10f61 Implement blocklist for signing key IDs
Implement a blocklist, keyid_blocklist. If a keyid (without leading 0x)
in this file matches the signing key for a release, terminate autospec
with an error.

This is intended as a mechanism to block building releases signed with a
known compromised key.
2024-03-29 11:43:38 -07:00
William Douglas c02b2fec6a Add etc_files log
When a package has content in /etc, it is removed silently, this
change adds an etc_files log file in git that shows what files were in
/etc that weren't packaged.

Signed-off-by: William Douglas <william.douglas@intel.com>
2024-03-22 14:11:36 -07:00
William Douglas 312e1714f2 Update failed patterns and cmake patterns for qt6
Signed-off-by: William Douglas <william.douglas@intel.com>
2024-03-22 12:42:28 -07:00
William Douglas 652616b1b6 Add path handling for qt6
Signed-off-by: William Douglas <william.douglas@intel.com>
2024-03-22 12:42:28 -07:00
William Douglas f4bef72a70 Add handling for multiline find_package
Previously support for cmake's find_package parsing was only able to
handle single line entries. This change adds support for multiline
versions and allows for module namespacing (currently qt6 and kf6
support is available).

The change for cmake_modules shows this namespacing in use.

Signed-off-by: William Douglas <william.douglas@intel.com>
2024-03-15 14:20:20 -07:00
William Douglas 6095ef84a0 Migrate configs for qt6
Partial migration of patterns to qt6 for failed_commands and
qt_modules. Still more changes needed here but this is a start.

Signed-off-by: William Douglas <william.douglas@intel.com>
2024-03-15 14:20:20 -07:00
William Douglas fa36e95657 Stop using the %qmake macro
Rather than using %qmake, add the definition into autospec. This is
done to allow for using make_command to specify which version of qmake
to use.

Note for now qmake6 is not in PATH and needs a build_prepend addition.

Signed-off-by: William Douglas <william.douglas@intel.com>
2024-03-15 14:20:20 -07:00
William Douglas 3d985ebaf9 Allow optimization level specification in the R pattern
Signed-off-by: William Douglas <william.douglas@intel.com>
2024-03-11 16:25:34 -07:00
Brett T. Warden 1ab68caa3d Search licenses/*.txt
We search for licenses as *.txt in subdirectories "LICENSES" and "license". Also search "licenses".
2024-03-04 16:21:34 -08:00
Brett T. Warden e738c511eb Add support for non-tarball bz2 archives
Some projects require additional bz2 downloads that are single
compressed files, not tar archives. Detect and extract those with
appropriate bz2 tools instead of tar.

For a given entry like this:
ARCHIVES = https://example.com/download/foo.dat.bz2 src/data

...upstream file is downloaded to:
builddir/build/SOURCES/foo.dat.bz2

...initially extracted as:
builddir/build/BUILD/foo.dat/foo.dat

...and finally copied to
builddir/build/BUILD/<pkg>/src/data/foo.dat
2024-02-26 09:12:10 -08:00
William Douglas da8b975a56 Add --host to autogen pattern for apx
autogen uses autotools and also needs a --host parameter to build apx.

Signed-off-by: William Douglas <william.douglas@intel.com>
2024-02-07 14:06:26 -08:00
William Douglas 213bb01a2f Make use of GOAMD64
Add export for GOAMD64 and modify where needed in order to support
building optimized binaries using golang.

Signed-off-by: William Douglas <william.douglas@intel.com>
2024-02-05 13:14:23 -08:00
31 changed files with 1097 additions and 361 deletions
+20
View File
@@ -128,6 +128,14 @@ release
``.spec``. This is also bumped and generated on existing and new packages,
respectively. This results in less manual work via automatic management.
license_skips
Each line in the file should be the path to a license file. That path needs
to account for the package tarfile prefix. Likely requires using '*' to be
effective (e.g. ``pkgname-*/path/to/license`` where ``*`` handles the version).
Files paths can contain a single '*' per directory such that
a line of ``/foo*/bar*`` is allowed but ``/foo*bar*`` is not.
$package.license
In certain cases, the package license may not be automatically discovered. In
this instance, ``autospec`` will exit with an error. Update this file to
@@ -204,6 +212,12 @@ ${custom}_provides_ban
Controlling the build process
------------------------------
invalid_release_sig
This file contains the current version that will **not** have its package
file be processed for signature verification (overriding the config_opt).
This file will be automatically deleted after a new release and is intended
to override a single bad signed release.
extra_sources
This file contains a list of extra files to be added to the ``.spec`` and
optionally installed as well. Each non-blank and non-comment line should start
@@ -268,6 +282,12 @@ prep_prepend
resulting ``.spec``, and is used for situations where fine-grained
control is required.
copy_prepend
Additional actions that should take place directly before the source
directory is copied for other builds (32bit, avx2, etc). This will be
placed in the resulting ``.spec``, and is used for situations where
fine-grained control is required.
build_prepend
Additional actions that should take place after ``%build`` and before
the ``%configure`` macro or equivalent (``%cmake``, etc.). If autospec
+4 -2
View File
@@ -155,7 +155,7 @@ def is_file_valid(path):
def dump_symbols(path):
"""Get symbols from a file."""
cmd = "nm --defined-only -g --dynamic \"{}\"".format(path)
cmd = "nm --defined-only -g --dynamic \"{}\" | c++filt".format(path)
lines = None
ret = set()
@@ -163,7 +163,7 @@ def dump_symbols(path):
try:
lines = get_output(cmd)
except Exception as e:
print("Fatal error inspecting {}: {}".format(path, e))
util.print_fatal("Fatal error inspecting {}: {}".format(path, e))
sys.exit(1)
for line in lines.split("\n"):
line = line.strip()
@@ -236,6 +236,7 @@ def examine_abi_host(download_path, results_dir, name):
cwd=download_path)
except Exception as e:
util.print_fatal("Error invoking abireport: {}".format(e))
sys.exit(1)
def examine_abi_fallback(download_path, results_dir, name):
@@ -272,6 +273,7 @@ def examine_abi_fallback(download_path, results_dir, name):
subprocess.check_call(cmd, shell=True)
except Exception as e:
util.print_fatal("Error extracting RPMS: {}".format(e))
sys.exit(1)
os.chdir(download_path)
collected_files = set()
+9 -4
View File
@@ -38,7 +38,7 @@ import specfiles
import tarball
from abireport import examine_abi
from logcheck import logcheck
from util import binary_in_path, print_fatal, write_out
from util import binary_in_path, print_build_failed, print_fatal, write_out
sys.path.append(os.path.dirname(__file__))
@@ -269,9 +269,13 @@ def package(args, url, name, archives, workingdir):
pkg_integrity.check(url, conf, interactive=interactive_mode)
pkg_integrity.load_specfile(specfile)
specfile.write_spec()
spec_type = specfile.write_spec()
while 1:
package.package(filemanager, args.mock_config, args.mock_opts, conf, requirements, content, args.cleanup)
if spec_type == "template":
# specfile template is assumed "correct" and any failures need to be manually addressed
break
filemanager.load_specfile(specfile)
specfile.write_spec()
filemanager.newfiles_printed = 0
@@ -291,7 +295,7 @@ def package(args, url, name, archives, workingdir):
if package.success == 0:
conf.create_buildreq_cache(content.version, requirements.buildreqs_cache)
print_fatal("Build failed, aborting")
print_build_failed()
sys.exit(1)
elif os.path.isfile("README.clear"):
try:
@@ -304,7 +308,8 @@ def package(args, url, name, archives, workingdir):
except Exception:
pass
check.check_regression(conf.download_path, conf.config_opts['skip_tests'], package.round - 1)
if spec_type == "generate":
check.check_regression(conf.download_path, conf.config_opts['skip_tests'], package.round - 1)
examine_abi(conf.download_path, content.name)
if os.path.exists("/var/lib/rpm"):
+10 -9
View File
@@ -22,6 +22,7 @@
import os
import re
import shutil
import sys
import util
@@ -153,7 +154,7 @@ class Build(object):
return True
self.must_restart = 0
self.file_restart = 0
is_clean = True
fatals = []
util.call("sync")
with util.open_auto(filename, "r") as rootlog:
loglines = rootlog.readlines()
@@ -161,9 +162,10 @@ class Build(object):
for line in loglines:
match = missing_pat.match(line)
if match is not None:
util.print_fatal("Cannot resolve dependency name: {}".format(match.group(1)))
is_clean = False
return is_clean
fatals.append(f"Cannot resolve dependency name: {match.group(1)}")
if fatals:
util.print_fatal('\n'.join(fatals))
sys.exit(1)
def parse_build_results(self, filename, returncode, filemanager, config, requirements, content):
"""Handle build log contents."""
@@ -283,7 +285,7 @@ class Build(object):
mockopts,
]
if not cleanup and self.must_restart == 0 and self.file_restart > 0 and set(filemanager.excludes) == set(filemanager.manual_excludes):
if config.config_opts.get('avoid_rebuild') and not cleanup and self.must_restart == 0 and self.file_restart > 0 and set(filemanager.excludes) == set(filemanager.manual_excludes):
cmd_args.append("--no-clean")
cmd_args.append("--short-circuit=binary")
@@ -295,12 +297,11 @@ class Build(object):
# sanity check the build log
if not os.path.exists(config.download_path + "/results/build.log"):
util.print_fatal("Mock command failed, results log does not exist. User may not have correct permissions.")
exit(1)
sys.exit(1)
if not self.parse_buildroot_log(config.download_path + "/results/root.log", ret):
return
self.parse_buildroot_log(config.download_path + "/results/root.log", ret)
self.parse_build_results(config.download_path + "/results/build.log", ret, filemanager, config, requirements, content)
if filemanager.has_banned:
util.print_fatal("Content in banned paths found, aborting build")
exit(1)
sys.exit(1)
+39 -6
View File
@@ -147,7 +147,7 @@ def parse_modules_list(modules_string, is_cmake=False):
modules = [m for m in re.split(r'\s*([><]?=|\${?[^}]*}?)\s*', modules_string)]
modules = filter(None, modules)
else:
modules = [m.strip('[]') for m in modules_string.split()]
modules = [m.strip('[]').strip('"') for m in modules_string.split()]
res = []
next_is_ver = False
for mod in modules:
@@ -483,21 +483,50 @@ class Requirements(object):
def set_build_req(self, config):
"""Add build requirements based on the build pattern."""
def findpackage_parse_lines(self, fp_line, line_iter, cmake_modules):
"""Parse find_package multiline segment of the line_iter."""
qt6module = re.compile(r"^[^#]*find_package\(\s*\bQt6.*", re.I)
kf6module = re.compile(r"^[^#]*find_package\(\s*\bKF6.*", re.I)
ns = ''
if qt6module.search(fp_line):
ns = 'qt6'
elif kf6module.search(fp_line):
ns = 'kf6'
while True:
ln = next(line_iter).strip()
if not ln:
continue
modules = ln.strip(')').split(' ')
for module in modules:
if module:
if pkg := cmake_modules.get(f"{ns}.{module}"):
self.add_buildreq(pkg)
if ')' in ln:
break
def parse_cmake(self, filename, cmake_modules, conf32):
"""Scan a .cmake or CMakeLists.txt file for what's it's actually looking for."""
findpackage = re.compile(r"^[^#]*find_package\((\w+)\b.*\)", re.I)
pkgconfig = re.compile(r"^[^#]*pkg_check_modules\s*\(\w+ (.*)\)", re.I)
findpackage_multiline = re.compile(r"^[^#]*find_package\((\w+)\b.*", re.I)
pkgconfig = re.compile(r"^[^#]*pkg_check_modules\s*\([\w\-]+ (.*)\)", re.I)
pkg_search_modifiers = {'REQUIRED', 'QUIET', 'NO_CMAKE_PATH',
'NO_CMAKE_ENVIRONMENT_PATH', 'IMPORTED_TARGET'}
extractword = re.compile(r'(?:"([^"]+)"|(\S+))(.*)')
with util.open_auto(filename, "r") as f:
lines = f.readlines()
lines = iter(f.readlines())
for line in lines:
if match := findpackage.search(line):
module = match.group(1)
if pkg := cmake_modules.get(module):
self.add_buildreq(pkg)
if pkgs := cmake_modules.get(module):
# Some of the entries in cmake_modules list multiple packages, space-separated, so we need to split.
# Otherwise, anything in buildreq_ban would have to match the entire string, not just a single package name.
# For example: Png2Ico, extra-cmake-modules png2ico
# buildreq_ban would have to contain "extra-cmake-modules png2ico" to match, instead of just "png2ico"
for pkg in pkgs.split():
self.add_buildreq(pkg)
elif findpackage_multiline.search(line):
self.findpackage_parse_lines(line, lines, cmake_modules)
if match := pkgconfig.search(line):
rest = match.group(1)
@@ -727,10 +756,14 @@ class Requirements(object):
configure_ac_files = []
qmake_profiles = []
cmake_files = []
if config.config_opts['use_ninja']:
self.add_buildreq('ninja')
for dirpath, _, files in os.walk(dirn):
default_score = 2 if dirpath == dirn else 1
if "Cargo.toml" in files:
if "Cargo.toml" in files and 'Makefile' not in files:
config.set_build_pattern('cargo', default_score)
if "CMakeLists.txt" in files and "configure.ac" not in files:
+18 -21
View File
@@ -64,50 +64,54 @@ def scan_for_tests(src_dir, config, requirements, content):
if config.config_opts.get('skip_tests') or tests_config:
return
make_command = "ninja" if config.config_opts.get('use_ninja') else "make"
makeflags = "%{?_smp_mflags} " if config.parallel_build else ""
make_check = "make {}check".format(makeflags)
cmake_check = "make test"
make_check = "{} {}check".format(make_command, makeflags)
cmake_check = "{} test".format(make_command)
make_check_openmpi = "module load openmpi\nexport OMPI_MCA_rmaps_base_oversubscribe=1\n" \
"make {}check\nmodule unload openmpi".format(makeflags)
"{} {}check\nmodule unload openmpi".format(make_command, makeflags)
cmake_check_openmpi = "module load openmpi\nexport OMPI_MCA_rmaps_base_oversubscribe=1\n" \
"make test\nmodule unload openmpi"
"{} test\nmodule unload openmpi".format(make_command)
if config.config_opts.get('allow_test_failures'):
make_check_openmpi = "module load openmpi\nexport OMPI_MCA_rmaps_base_oversubscribe=1\n" \
"make {}check || :\nmodule unload openmpi".format(makeflags)
"{} {}check || :\nmodule unload openmpi".format(make_command, makeflags)
cmake_check_openmpi = "module load openmpi\nexport OMPI_MCA_rmaps_base_oversubscribe=1\n" \
"make test || :\nmodule unload openmpi"
"{} test || :\nmodule unload openmpi".format(make_command)
perl_check = "make TEST_VERBOSE=1 test"
setup_check = """PYTHONPATH=%{buildroot}$(python -c "import sys; print(sys.path[-1])") python setup.py test"""
perl_check = "{} TEST_VERBOSE=1 test".format(make_command)
meson_check = "meson test -C builddir --print-errorlogs"
if config.config_opts.get('allow_test_failures'):
make_check += " || :"
cmake_check += " || :"
perl_check += " || :"
setup_check += " || :"
meson_check += " || :"
testsuites = {
"makecheck": make_check,
"perlcheck": perl_check,
"setup.py": setup_check,
"cmake": "cd clr-build; " + cmake_check,
"meson": meson_check,
}
if config.config_opts.get('32bit'):
testsuites["makecheck"] += "\ncd ../build32;\n" + make_check + " || :"
testsuites["cmake"] += "\ncd ../clr-build32;\n" + cmake_check + " || :"
testsuites["cmake"] += "\ncd ../../build32/clr-build32;\n" + cmake_check + " || :"
testsuites["meson"] += "\ncd ../build32;\n" + meson_check + " || :"
if config.config_opts.get('use_avx2'):
testsuites["makecheck"] += "\ncd ../buildavx2;\n" + make_check + " || :"
testsuites["cmake"] += "\ncd ../clr-build-avx2;\n" + cmake_check + " || :"
testsuites["cmake"] += "\ncd ../../buildavx2/clr-build-avx2;\n" + cmake_check + " || :"
testsuites["meson"] += "\ncd ../buildavx2;\n" + meson_check + " || :"
if config.config_opts.get('use_avx512'):
testsuites["makecheck"] += "\ncd ../buildavx512;\n" + make_check + " || :"
testsuites["cmake"] += "\ncd ../clr-build-avx512;\n" + cmake_check + " || :"
testsuites["cmake"] += "\ncd ../../buildavx512/clr-build-avx512;\n" + cmake_check + " || :"
testsuites["meson"] += "\ncd ../buildavx512;\n" + meson_check + " || :"
if config.config_opts.get('use_apx'):
testsuites["makecheck"] += "\ncd ../buildapx;\n" + make_check + " || :"
testsuites["cmake"] += "\ncd ../../buildapx/clr-build-apx;\n" + cmake_check + " || :"
testsuites["meson"] += "\ncd ../buildapx;\n" + meson_check + " || :"
if config.config_opts.get('openmpi'):
testsuites["makecheck"] += "\ncd ../build-openmpi;\n" + make_check_openmpi
testsuites["cmake"] += "\ncd ../clr-build-openmpi;\n" + cmake_check_openmpi
testsuites["cmake"] += "\ncd ../../build-openmpi/clr-build-openmpi;\n" + cmake_check_openmpi
files = os.listdir(src_dir)
@@ -138,13 +142,6 @@ def scan_for_tests(src_dir, config, requirements, content):
elif config.default_pattern in ["cpan"] and "Makefile.PL" in files:
tests_config = testsuites["perlcheck"]
elif config.default_pattern == "distutils3" and "setup.py" in files:
with util.open_auto(os.path.join(src_dir, "setup.py"), 'r') as setup_fp:
setup_contents = setup_fp.read()
if "test_suite" in setup_contents or "pbr=True" in setup_contents:
tests_config = testsuites["setup.py"]
elif config.default_pattern == "R":
tests_config = "export _R_CHECK_FORCE_SUGGESTS_=false\n" \
"R CMD check --no-manual --no-examples --no-codoc . " \
+19 -2
View File
@@ -266,6 +266,7 @@ KSMServerDBusInterface, plasma-workspace-dev
KSaneCore, ksanecore-dev
KScreenLocker, kscreenlocker-dev
KSysGuard, libksysguard-dev
KUserFeedback, kuserfeedback-dev
KWaylandServer, kwayland-server-dev
KWinDBusInterface, kwin-dev
KWinEffects, kwin-dev
@@ -288,7 +289,6 @@ LibIcal, libical-dev
LibKEduVocDocument, libkeduvocdocument-dev
LibKWorkspace, plasma-workspace-dev
LibKompareDiff2, libkomparediff2-dev
LibLZMA, xz-dev
LibNotificationManager, plasma-workspace-dev
LibObs, obs-studio-dev
LibTaskManager, plasma-workspace-dev
@@ -329,6 +329,7 @@ PerlLibs, perl
Phonon4Qt5, phonon-dev
Phonon4Qt5Experimental, phonon-dev
PkgConfig, pkg-config
PlasmaActivities, plasma-activities-dev
PlasmaPotdProvider, kdeplasma-addons-dev
PlasmaWaylandProtocols, plasma-wayland-protocols-dev
Png2Ico, extra-cmake-modules png2ico
@@ -346,7 +347,7 @@ Python3, python3-dev
PythonInterp, python3
PythonLibs, python3-dev
QGpgme, gpgme-dev gpgme-extras
QHelpGenerator, extra-cmake-modules qttools-dev
QHelpGenerator, extra-cmake-modules qt6tools-dev
QMobipocket, kdegraphics-mobipocket-dev
Qca-qt5, qca-qt5-dev
Qt3DTests, qt6base-dev
@@ -901,6 +902,19 @@ igsc, igsc-dev
json-c, json-c-dev
kColorPicker, kcolorpicker-dev
kImageAnnotator, kimageannotator-dev
kf6.Config, kconfig-dev
kf6.CoreAddons, kcoreaddons-dev
kf6.Crash, kcrash-dev
kf6.DBusAddons, kdbusaddons-dev
kf6.FileMetaData, kfilemetadata-dev
kf6.I18n, ki18n-dev
kf6.JobWidgets, kjobwidgets-dev
kf6.KIO, kio-dev
kf6.Notifications, knotifications-dev
kf6.Purpose, purpose-dev
kf6.Runner, krunner-dev
kf6.Service, kservice-dev
kf6.StatusNotifierItem, kstatusnotifieritem-dev
kim-api, kim-api-data
leveldb, leveldb-dev
libavif, libavif-dev
@@ -926,6 +940,9 @@ pugixml, pugixml-dev
pybind11, pypi(pybind11)
qt5xdg, libqtxdg-data
qt5xdgiconloader, libqtxdg-data
qt6.DBus, qt6base-dev
qt6.Gui, qt6base-dev
qt6.Widgets, qt6base-dev
qtxdg-tools, qtxdg-tools-data
rabbitmq-c, rabbitmq-c-dev
realsense2, librealsense-dev
+1 -1
View File
@@ -49,7 +49,7 @@ def scan_for_changes(download_path, directory, transforms):
shutil.copy(source, target)
os.chmod(target, 0o644)
except Exception as e:
print("Error copying file: {}".format(e))
util.print_fatal("Error copying file: {}".format(e))
sys.exit(1)
found.append(item)
+48 -8
View File
@@ -28,7 +28,8 @@ from collections import OrderedDict
import check
import license
from util import call, open_auto, print_info, print_warning, write_out
from util import (call, open_auto, print_fatal, print_info, print_warning,
write_out)
def read_pattern_conf(filename, dest, list_format=False, path=None):
@@ -68,6 +69,7 @@ class Config(object):
def __init__(self, download_path):
"""Initialize Default configuration settings."""
self.cargo_vendors = ""
self.content = None # hack to avoid circular init dependency
self.extra_configure = ""
self.extra_configure32 = ""
@@ -91,6 +93,7 @@ class Config(object):
self.install_macro = "%make_install"
self.disable_static = "--disable-static"
self.prep_prepend = []
self.copy_prepend = []
self.build_prepend = []
self.build_prepend_once = []
self.build_append = []
@@ -107,6 +110,7 @@ class Config(object):
self.custom_summ = ""
self.license_fetch = None
self.license_show = None
self.license_skips = []
self.git_uri = None
self.os_packages = set()
self.config_file = None
@@ -115,12 +119,15 @@ class Config(object):
self.old_keyid = None
self.profile_payload = None
self.signature = None
self.signature_macro = None
self.pkey_macro = None
self.yum_conf = None
self.failed_pattern_dir = None
self.alias = None
self.failed_commands = {}
self.ignored_commands = {}
self.gems = {}
self.keyid_blocklist = {}
self.license_hashes = {}
self.license_translations = {}
self.license_blacklist = {}
@@ -151,6 +158,7 @@ class Config(object):
}
self.config_opts = {}
self.config_options = {
"avoid_rebuild": "Try to use mock short circuit to avoid full rebuilds",
"broken_c++": "extend flags with '-std=gnu++98",
"cargo_vendor": "create vendor archive with cargo",
"use_lto": "configure build for lto",
@@ -184,6 +192,7 @@ class Config(object):
"server": "Package is only used by servers",
"no_glob": "Do not use the replacement pattern for file matching",
"allow_exe": "Allow Windows executables (*.exe, *.dll) to be packaged",
"use_ninja": "Use ninja build files",
}
# simple_pattern_pkgconfig patterns
# contains patterns for parsing build.log for missing dependencies
@@ -263,6 +272,7 @@ class Config(object):
(r"Can't locate [a-zA-Z0-9_\-\/\.]+ in @INC \(you may need to install the ([a-zA-Z0-9_\-:]+) module\)", 0, 'perl'),
(r"Cannot find ([a-zA-Z0-9\-_\.]*)", 1, None),
(r"Checking for (.*?)\.\.\.no", 0, None),
(r"checking for (.*?) \(using pkg-config\)\.\.\. no", 0, None),
(r"Checking for (.*?)\s*: not found", 0, None),
(r"Checking for (.*?)\s>=.*\s*: not found", 0, None),
(r"Could not find suitable distribution for Requirement.parse\('([a-zA-Z\-\.]*)", 0, None),
@@ -285,10 +295,12 @@ class Config(object):
(r"No rule to make target `(.*)',", 0, None),
(r"Package (.*) was not found in the pkg-config search path.", 0, 'pkgconfig'),
(r"Package '([a-zA-Z0-9\-:]*)', required by '.*', not found", 0, 'pkgconfig'),
(r"The file `([a-zA-Z0-9\-:]*)\.pc` needs to be installed and the PKG_CONFIG_PATH environment variable must contain its parent directory\.", 0, 'pkgconfig'),
(r"Package which this enhances but not available for checking: [']([a-zA-Z0-9\-]*)[']", 0, 'R'),
(r"Perhaps you should add the directory containing `([a-zA-Z0-9\-:]*)\.pc'", 0, 'pkgconfig'),
(r"Program (.*) found: NO", 0, None),
(r"Target '[a-zA-Z0-9\-]' can't be generated as '(.*)' could not be found", 0, None),
(r"The missing Perl modules are:\s*(\S+)", 0, 'perl'),
(r"Unable to `import (.*)`", 0, None),
(r"Unable to find '(.*)'", 0, None),
(r"Warning: prerequisite ([a-zA-Z:]+) [0-9\.]+ not found.", 0, 'perl'),
@@ -309,6 +321,7 @@ class Config(object):
(r"checking for [a-zA-Z0-9\_\-]+ in (.*?)\.\.\. no", 0, None),
(r"checking for library containing (.*)... no", 0, None),
(r"checking for perl module ([a-zA-Z:]+) [0-9\.]+... no", 0, 'perl'),
("checking if (.*) is available...", 0, None),
(r"configure: error: (?:pkg-config missing|Unable to locate) (.*)", 0, None),
(r"configure: error: ([a-zA-Z0-9]+) (?:is required to build|not found)", 0, None),
(r"configure: error: Cannot find (.*)\. Make sure", 0, None),
@@ -334,7 +347,7 @@ class Config(object):
def detect_build_from_url(self, url):
"""Detect build patterns and build requirements from the patterns detected in the url."""
# R package
if "cran.r-project.org" in url or "cran.rstudio.com" in url:
if "cran.r-project.org" in url or "cran.rstudio.com" in url or "/pub/cran/" in url:
self.set_build_pattern("R", 10)
# python
@@ -431,20 +444,24 @@ class Config(object):
# next the options
config_f['autospec'] = {}
for fname, comment in sorted(self.config_options.items()):
fpath = os.path.join(self.download_path, fname)
config_f.set('autospec', '# {}'.format(comment))
if os.path.exists(fname):
if os.path.isfile(fpath):
config_f['autospec'][fname] = 'true'
os.remove(fname)
os.remove(fpath)
else:
config_f['autospec'][fname] = 'false'
# default lto to true for new things
config_f['autospec']['use_lto'] = 'true'
# default autoupdate to true for new things
config_f['autospec']['autoupdate'] = 'true'
# renamed options need special care
if os.path.exists("skip_test_suite"):
skip_path = os.path.join(self.download_path, "skip_test_suite")
if os.path.exists(skip_path):
config_f['autospec']['skip_tests'] = 'true'
os.remove("skip_test_suite")
os.remove(skip_path)
self.write_config(config_f)
def create_buildreq_cache(self, version, buildreqs_cache):
@@ -475,7 +492,7 @@ class Config(object):
config_f = configparser.ConfigParser(interpolation=None)
config_f.read(opts_path)
if "autospec" not in config_f.sections():
print("Missing autospec section in options.conf")
print_fatal("Missing autospec section in options.conf")
sys.exit(1)
if 'package' in config_f.sections() and config_f['package'].get('alias'):
@@ -603,6 +620,7 @@ class Config(object):
read_pattern_conf("license_blacklist", self.license_blacklist, list_format=True, path=path)
read_pattern_conf("qt_modules", self.qt_modules, path=path)
read_pattern_conf("cmake_modules", self.cmake_modules, path=path)
read_pattern_conf("keyid_blocklist", self.keyid_blocklist, list_format=True, path=path)
def parse_existing_spec(self, name):
"""Determine the old version, old patch list, old keyid, and cves from old spec file."""
@@ -696,7 +714,7 @@ class Config(object):
config.read(self.config_file)
if "autospec" not in config.sections():
print("Missing autospec section..")
print_fatal("Missing autospec section..")
sys.exit(1)
self.git_uri = config['autospec'].get('git', None)
@@ -819,6 +837,18 @@ class Config(object):
except Exception as e:
print_warning(f"Unable to remove buildreq_cache file: {e}")
invalid_release_sig_file = os.path.join(self.download_path, "invalid_release_sig")
content = self.read_conf_file(invalid_release_sig_file)
if content and content[0] == version:
self.config_opts['verify_required'] = False
else:
try:
os.unlink(invalid_release_sig_file)
except FileNotFoundError:
pass
except Exception as e:
print_warning(f"Unable to remove invalid_release_sig file: {e}")
content = self.read_conf_file(os.path.join(self.download_path, "pkgconfig_add"))
for extra in content:
extra = pkgconfig_re.sub(r'\1', extra)
@@ -979,6 +1009,15 @@ class Config(object):
if not license.add_license(word, self.license_translations, self.license_blacklist):
print_warning("{}: blacklisted license {} ignored.".format(self.content.name + ".license", word))
content = self.read_conf_file(os.path.join(self.download_path, "license_skips"))
if content:
self.license_skips = self.validate_extras_content(content, "license_skips")
# cargo_vendors is the output of 'cargo vendor' and should be read as is
content = self.read_file(os.path.join(self.download_path, "cargo_vendors"), track=True)
if content:
self.cargo_vendors = "".join(content)
if self.config_opts['use_clang']:
self.config_opts['funroll-loops'] = False
requirements.add_buildreq("llvm")
@@ -997,6 +1036,7 @@ class Config(object):
requirements.add_buildreq("openssh")
self.prep_prepend = self.read_script_file(os.path.join(self.download_path, "prep_prepend"))
self.copy_prepend = self.read_script_file(os.path.join(self.download_path, "copy_prepend"))
if os.path.isfile(os.path.join(self.download_path, "prep_append")):
os.rename(os.path.join(self.download_path, "prep_append"), os.path.join(self.download_path, "build_prepend"))
self.make_prepend = self.read_script_file(os.path.join(self.download_path, "make_prepend"))
+109 -20
View File
@@ -12,6 +12,7 @@
-lICE, pkgconfig(ice)
-lIlmThread, ilmbase-dev
-lSM, pkgconfig(sm)
-lTerminfo, ncurses-dev
-lX11, pkgconfig(x11)
-lXau, libXau-dev
-lXaw, pkgconfig(xaw)
@@ -75,7 +76,6 @@
-lkrb5, krb5-dev
-lldap, openldap-dev
-lldb, ldb-dev
-llzma, xz-dev
-llzo, lzo-dev
-lmagic, file-dev
-lmenu, ncurses-dev
@@ -99,11 +99,11 @@
-lrdmacm, rdma-core-dev
-lreadline, readline-dev
-lsecret, libsecret-dev
-lsecret, pkgconfig(libsecret-1)
-lsensors, lm-sensors-dev
-lsqlite3, sqlite-autoconf-dev
-lssh, libssh-dev
-lssl, openssl-dev
-lTerminfo, ncurses-dev
-ltermcap, ncurses-dev
-ltinfo, ncurses-dev
-ltinfow, ncurses-dev
@@ -124,11 +124,14 @@ ACL_TYPE_EXTENDED, acl-dev
ALSA, alsa-lib-dev
APR, apr-dev
APR-util, apr-util-dev
ARPACK-NG, arpack-ng-dev
ASSIMP, assimp-dev
Adwaita, libadwaita-dev
Analitza, analitza-dev
Analitza5, analitza-dev
BABL, babl-dev
BISON, bison
BROTLI, brotli-dev
BZ2_bzCompress, bzip2-dev
BZip2, bzip2-dev
Berkeley DB, db
@@ -138,6 +141,7 @@ Boost, boost-dev
Breeze, breeze-dev
CAIRO, cairo-dev
CARES, c-ares-dev
CFITSIO, cfitsio-dev
CGAL, CGAL-dev
CHECK, check
CLUTTER, clutter-dev
@@ -173,7 +177,9 @@ EXPAT, expat-dev
Eet.h, eet-dev
Eigen/Core, eigen-dev
Eigen/Dense, eigen-dev
Eigen/Sparse, eigen-dev
Eigen3, eigen-dev
Epoxy, pkgconfig(epoxy)
Exiv2, exiv2-dev
Expat 1.95.x, expat-dev
Expat XML Parser, expat-dev
@@ -254,6 +260,7 @@ Fontconfig, pkgconfig(fontconfig)
FreeType 2, freetype-dev
Freetype, freetype-dev
GCRYPT, libgcrypt-dev
GCrypt, libgcrypt-dev
GD, gd-dev
GDBM, gdbm-dev
GDM_PAM_EXTENSIONS, gdm-dev
@@ -280,9 +287,11 @@ GSTREAMER, gstreamer-dev
GSettingSchemas, gsettings-desktop-schemas
GSettingSchemas, gsettings-desktop-schemas-dev
GTK, gtk+-dev
GTK4, gtk4-dev
GnuTLS, gnutls-dev
GooglePerftools, gperftools-dev
Gperf, gperf
Gpgme, pkgconfig(gpgme)
Gpgmepp, gpgme-dev
Gphoto2, libgphoto2-dev
Grantlee5, grantlee-dev
@@ -375,6 +384,7 @@ KF5KMahjongglib, libkmahjongg-dev
KF5KdepimDBusInterfaces, kdepim-apps-libs-dev
KF5Kipi, libkipi-dev
KF5Kirigami2, kirigami2-dev
KF5KirigamiAddons, kirigami2-dev
KF5KontactInterface, kontactinterface-dev
KF5Kross, kross-dev
KF5Ldap, kldap-dev
@@ -416,6 +426,8 @@ KF5Su, kdesu-dev
KF5Syndication, syndication-dev
KF5SyntaxHighlighting, syntax-highlighting-dev
KF5SysGuard, libksysguard-dev
KF5TextAutoCorrection, ktextaddons-dev
KF5TextAutoCorrectionWidgets, ktextaddons-dev
KF5TextEditor, ktexteditor-dev
KF5TextWidgets, ktextwidgets-dev
KF5ThreadWeaver, threadweaver-dev
@@ -428,16 +440,75 @@ KF5WidgetsAddons, kwidgetsaddons-dev
KF5WindowSystem, kwindowsystem-dev
KF5XmlGui, kxmlgui-dev
KF5XmlRpcClient, kxmlrpcclient-dev
KF6Attica, attica-dev
KF6Baloo, baloo-dev
KF6BalooWidgets, baloo-widgets-dev
KF6ColorScheme, kcolorscheme-dev
KF6Config, kconfig-dev
KF6Contacts, kcontacts-dev
KF6DNSSD, kdnssd-dev
KF6FileMetaData, kfilemetadata-dev
KF6GlobalAccel, kglobalaccel-dev
KF6Holidays, kholidays-dev
KF6IdleTime, kidletime-dev
KF6JobWidgets, kjobwidgets-dev
KF6KCMUtils, kcmutils-dev
KF6Kirigami, kirigami-dev
KF6Kirigami2, kirigami-dev
KF6KirigamiAddons, kirigami-addons-dev
KF6KirigamiAddons2, kirigami-addons-dev
KF6KirigamiPlatform, kirigami-dev
KF6NetworkManagerQt, networkmanager-qt-dev
KF6NewStuff, knewstuff-def
KF6Notifications, knotifications-dev
KF6NotifyConfig, knotifyconfig-dev
KF6Package, kpackage-dev
KF6Prison, kprison-dev
KF6Purpose, purpose-dev
KF6QuickCharts, kquickcharts-dev
KF6Runner, krunner-dev
KF6Screen, libkscreen-dev
KF6StatusNotifierItem, kstatusnotifieritem-dev
KF6SyntaxHighlighting, syntax-highlighting-dev
KF6TextEditor, ktexteditor-dev
KGantt, kdiagram-dev
KPim5Akonadi, akonadi-dev
KPim5AkonadiContact, akonadi-contacts-dev
KPim5AkonadiMime, akonadi-mime-dev
KPim5AkonadiNotes, akonadi-notes-dev
KPim5GrantleeTheme, grantleetheme-dev
KPim5Gravatar, libgravatar-dev
KPim5IMAP, kimap-staticdev
KPim5Ldap, kldap-dev
KPim5LibKSieve, libksieve-dev
KPim5Libkdepim, libkdepim-dev
KPim5Libkleo, libkleo-dev
KPim5MailImporter, mailimporter-dev
KPim5MailTransport, kmailtransport-dev
KPim5MailTransportAkonadi, kmailtransport-dev
KPim5MessageComposer, messagelib-dev
KPim5MessageCore, messagelib-dev
KPim5Mime, kmime-dev
KPim5PimCommon, pimcommon-dev
KPim5TextEdit, kpimtextedit-dev
KPim6IMAP, kimap-dev
KPim6Libkdepim, libkdepim-dev
KPim6Libkleo, libkleo-dev
KPim6MailTransport, kmailtransport-dev
KPim6MessageCore, messagelib-dev
KPim6Mime, kmime-dev
KPim6PimCommon, pimcommon-dev
KPimGAPI, libkgapi-dev
KPimKDAV, kdav-dev
KPimPkPass, kpkpass-dev
KPimSMTP, ksmtp-dev
KPimSMTPConfig, ksmtp-dev
KPipeWire, kpipewire-dev
KScreenLocker, kscreenlocker-dev
KTP, ktp-common-internals-dev
KdepimLibs, kdepimlibs-dev
Kerberos 5, krb5-dev
KirigamiAddons, kirigami-addons-dev
LEX, flex bison
LIBCAP_NG_PATH, libcap-ng-dev
LIBEXIF, libexif-dev
@@ -445,7 +516,6 @@ LIBGCRYPT - version >= 1.5.0, libgcrypt-dev
LIBGD, libgd-dev
LIBICAL, libical-dev
LIBKONQ, kde-baseapps-dev
LIBLZMA, xz-dev
LIBNOTIFY, libnotify-dev
LIBPCREVERSION, pcre-dev
LIBRSVG, librsvg-dev
@@ -467,12 +537,12 @@ LibExiv2, pkgconfig(exiv2)
LibKEduVocDocument, libkeduvocdocument-dev
LibKWorkspace, plasma-workspace-dev
LibKrb5, krb5-dev
LibLZMA, xz-dev
LibR, R-dev
LibSSH, libssh-dev
LibXml2, libxml2-dev
LibXslt, libxslt-bin
Libcap, libcap-dev
Libev, libev-dev
Lua51, lua-dev
LuaJIT, LuaJIT-dev
MDDS, mdds-dev
@@ -483,6 +553,7 @@ NETTLE, nettle-dev
NUMA, numactl-dev
Nepomuk, nepomuk-core-dev
NetworkManager, NetworkManager-dev
Nice, libnice-dev
NumPy, pypi-numpy
Numa, numactl-dev
OPENEXR, openexr-dev
@@ -503,12 +574,16 @@ PAM, Linux-PAM-dev
PCAP, libpcap-dev
PCIUTILS, pciutils-dev
PCRE, pcre-dev
PCRE2, pcre2-dev
PNG, pkgconfig(libpng)
POPT, popt-dev
POSIX ACL, acl-dev
PULSEAUDIO, pulseaudio-dev
Phonon, phonon-dev
Phonon4Qt5, phonon-dev
Phonon4Qt6, phonon-dev
PlasmaActivities, plasma-activities-dev
PlasmaActivitiesStats, plasma-activities-stats-dev
PolkitQt5-1, polkit-qt-dev
Poppler, poppler-dev
PopplerQt4, poppler-dev
@@ -522,13 +597,12 @@ Python, python3-dev
Python.h, python3-dev
PythonLibs, python3-dev
PythonLibsUnix, python3-dev
QCoro5, qcoro-dev
QGpgme, gpgme-dev
QImageBlitz, qimageblitz-dev
QREncode, qrencode-dev
QRencode, qrencode-dev
QT, buildreq-kde
Qt::Quick, qt6declarative-dev
Qt::Svg, qt6svg-dev
Qalculate, libqalculate-dev
Qca-qt5, qca-qt5-dev
Qt4, qt-everywhere-opensource-src-dev
@@ -539,6 +613,7 @@ Qt5Keychain, qtkeychain-dev
Qt5LinguistTools, qttools-dev
Qt5Multimedia, qtmultimedia-dev
Qt5Network, qtbase-dev
Qt5NetworkAuth, pkgconfig(Qt5NetworkAuth)
Qt5QuickControls2, pkgconfig(Qt5QuickControls2)
Qt5QuickTemplates2, pkgconfig(Qt5QuickTemplates2)
Qt5QuickTest, pkgconfig(Qt5QuickTest)
@@ -552,9 +627,13 @@ Qt5Xdg, libqtxdg-dev
Qt5XdgIconLoader, libqtxdg-dev
Qt5XmlPatterns, qtxmlpatterns-dev
Qt6Positioning, qt6positioning-dev
Qt6Qml, qt6declarative-dev
Qt6Quick, qt6declarative-dev
Qt6QuickTimeline, qt6quicktimeline-dev
Qt6ShaderTools, qt6shadertools-dev
Qt6WaylandClient, qt6wayland-dev
Qt::Quick, qt6declarative-dev
Qt::Svg, qt6svg-dev
R, R
READLINE, readline-dev
RSVG, librsvg-dev
@@ -578,6 +657,7 @@ Soprano, soprano-dev
SpatialIndex, libspatialindex-dev
Sphinx, pypi-sphinx
Sqlite, sqlite-autoconf-dev
Srtp2, libsrtp-dev
Startup notification library, libnotify-dev
TBB, tbb-dev
TIFF, tiff-dev
@@ -598,6 +678,7 @@ WEBKIT, webkitgtk-dev
WEBPMUX, libwebp-dev
WSME, WSME
Wayland, wayland-dev
WaylandProtocols, wayland-protocols-dev plasma-wayland-protocols-dev
WaylandScanner, wayland
WebOb, pypi-webob
X, pkgconfig(x11)
@@ -778,7 +859,7 @@ debtcollector, debtcollector
decorator, pypi-decorator
deflate, zlib-dev
deflateBound in -lz, zlib-dev
designer, qttools-dev
designer, qt6tools-dev
desktop-file-install, desktop-file-utils
desktop-file-validate, desktop-file-utils
dialog, dialog
@@ -817,6 +898,7 @@ ez_setup, pypi-setuptools
ffi.h, libffi-dev
ffi_call, libffi-dev
fftw3.h, fftw-dev
file, file
find_packages, pypi-setuptools
flatbuffers/flatbuffers.h, FlatBuffers-dev
flatpak, flatpak-dev
@@ -829,6 +911,7 @@ fontconfig.h, fontconfig-dev
fontconfig/fontconfig.h, fontconfig-dev
freerdp2, freerdp-dev
freetype-config, freetype-dev
frozen/unordered_map.h, frozen-dev
fuse (via pkg-config), compat-fuse-soname2-dev fuse-dev
fuse, fuse-dev
fuse.h, fuse-dev
@@ -842,6 +925,7 @@ gawk, gawk
gc/gc.h, gc-dev
gcj, gcj
gconftool-2, GConf-dev
gcr-3, pkgconfig(gcr-3)
gcrypt.h, libgcrypt-dev
gdal-config, gdal-dev
gdbm.h, gdbm-dev
@@ -858,6 +942,7 @@ gevent, pypi-gevent
gflags, gflags-dev
gfortran, gfortran
gi, pygobject
gi-docgen, gi-docgen
gif_lib.h, giflib-dev
gio-2.0, glib-dev
giomm-2.4, glibmm-dev
@@ -979,7 +1064,7 @@ lber.h, openldap-dev
lcms2, lcms2-dev
lcms2.h, lcms2-dev
lcms2/lcms2.h, lcms2-dev
lconvert, qttools-dev
lconvert, qt6tools-dev
ldap.h, openldap-dev
ldap_parse_sort_control, openldap-dev
ldap_search in -lldap, openldap-dev
@@ -1020,7 +1105,6 @@ libhandy-0.0, libhandy-dev
libiberty.h, binutils-dev
libkmod.h, kmod-dev
libksba, libksba-dev
liblzma, xz-dev
libmnl/libmnl.h, libmnl-dev
libmpfr, mpfr-dev
libmspack >= 0.0.20040308alpha (via pkg-config), libmspack-dev
@@ -1029,6 +1113,7 @@ libnettle, nettle-dev nettle-lib
libnftnl, libnftnl-dev
libnl-1, libnl-dev
libnotify, libnotify-dev
libnova, libnova-dev
libpeas-1.0, libpeas-dev
libpipewire-0.2, pipewire-dev
libpng, pkgconfig(libpng)
@@ -1047,6 +1132,7 @@ libsecret-unstable, libsecret-dev
libsoup-2.4, libsoup-dev
libssh, libssh-dev
libssh2.h, libssh2-dev
libssl, openssl-dev
libsystemd, pkgconfig(systemd)
libtirpc (via pkg-config), pkgconfig(libtirpc)
libudev.h, systemd-dev
@@ -1065,14 +1151,14 @@ libxml2 devel libraries, libxml2-dev
libxml2 library, libxml2-dev
libxslt/pattern.h, libxslt-dev
libz, zlib-dev
linguist, qttools-dev
linguist, qt6tools-dev
linux/xattr.h, attr-dev
llvm-config, llvm-dev
loadkeys, kbd
lrelease, qttools-dev
lrelease, qt6tools-dev
ltdl.h, libtool-dev
lua, lua-dev
lupdate, qttools-dev
lupdate, qt6tools-dev
lvm2app.h, LVM2-dev
lxml, pypi-lxml
lxqt, liblxqt-dev
@@ -1080,8 +1166,6 @@ lxqt-build-tools, lxqt-build-tools
lxqt-globalkeys, lxqt-globalkeys-dev
lxqt-globalkeys-ui, lxqt-globalkeys-dev
lz4.h, lz4-dev
lzma, xz-dev
lzma.h, xz-dev
lzo/lzoconf.h, lzo-dev
magic, file-dev
magic.h, file-dev
@@ -1194,7 +1278,7 @@ pg_config, postgresql-dev
php, php
php-config7, php
pip, pypi-pip
pixeltool, qttools-dev
pixeltool, qt6tools-dev
pixman, pkgconfig(pixman-1)
pkg-config, pkg-config
pkg.m4, pkg-config-dev
@@ -1223,6 +1307,8 @@ pycparser, pypi(pycparser)
pycurl, pypi(pycurl)
pylint, pypi(pylint)
pyperclip, pypi(pyperclip)
pypi(openvino), openvino-python3
pypi(pytorch), pytorch-python3
pytest, pypi-pytest
pytest-cov, pypi(pytest_cov)
python lxml, pypi(lxml)
@@ -1232,11 +1318,11 @@ python-pcre, python-pcre
python-subunit, pypi(python_subunit)
pytz, pypi(pytz)
pyyaml, PyYAML
qcollectiongenerator, qttools-dev
qcollectiongenerator, qt6tools-dev
qdbuscpp2xml, qtbase-dev
qdbusxml2cpp, qtbase-dev
qhelpconverter, qttools-dev
qhelpgenerator, qttools-dev
qhelpconverter, qt6tools-dev
qhelpgenerator, qt6tools-dev
qhull/libqhull.h, qhull-dev
qjson/parser.h, qjson-dev
qlalr, qtbase-dev
@@ -1250,8 +1336,8 @@ qmlplugindump, qtdeclarative-dev
qmltestrunner, qtdeclarative-dev
qscxmlc, qtscxml-dev
qt5, buildreq-kde
qtattributionsscanner, qttools-dev
qtplugininfo, qttools-dev
qtattributionsscanner, qt6tools-dev
qtplugininfo, qt6tools-dev
qtwaylandscanner, qtwayland-dev
qvkgen, qtbase-dev
raptor, raptor2-dev
@@ -1267,6 +1353,7 @@ requests, pypi-requests
rest-0.7, rest-dev
retrying, retrying
rpm, pkgconfig(rpm)
rst2man rst2man.py, pypi-docutils
rst2man, pypi-docutils
rsvg-convert, librsvg
ruby, ruby
@@ -1279,6 +1366,7 @@ sbc, sbc-dev
scrollkeeper-config, rarian
scrollkeeper-preinstall, rarian
sdl2, SDL2-dev
sdl2-config, SDL2-dev
security/pam_appl.h, Linux-PAM-dev
selenium, pypi-selenium
setcap, libcap-dev
@@ -1415,6 +1503,7 @@ yaml-cpp, yaml-cpp-dev
yarg, pypi-yarg
yasm, yasm
yytext is a pointer, flex
zconf.h, zlib-dev
zenity, zenity
zip, zip
zlib, zlib-dev
+9 -28
View File
@@ -197,33 +197,6 @@ class FileManager(object):
return removed
def globlike_match(self, filename, match_name):
"""Compare the filename to the match_name in a way that simulates the shell glob '*'."""
fsplit = filename.split('/')
if len(fsplit) != len(match_name):
return False
match = True
for fpart, mpart in zip(fsplit, match_name):
if fpart != mpart:
if '*' not in mpart:
match = False
break
if len(mpart) > len(fpart) + 1:
match = False
break
mpl, mpr = mpart.split('*')
try:
if fpart.index(mpl) != 0:
match = False
break
if fpart.rindex(mpr) != len(fpart) - len(mpr):
match = False
break
except ValueError:
match = False
break
return match
def push_file(self, filename, pkg_name):
"""Perform a number of checks against the filename and push the filename if appropriate."""
if filename in self.files or filename in self.files_blacklist:
@@ -245,7 +218,7 @@ class FileManager(object):
elif len('/'.join(match_name)) <= (len(norm_filename) + 1):
# the match name may be 1 longer due to a glob
# being able to match an empty string
if self.globlike_match(norm_filename, match_name):
if util.globlike_match(norm_filename, match_name):
path_prefix = '/' if not match else match.group()
self.push_package_file(os.path.join(path_prefix, *match_name), k)
return
@@ -301,6 +274,7 @@ class FileManager(object):
(r"^/usr/share/info/", "info"),
(r"^/usr/share/abi/", "abi"),
(r"^/usr/share/qt5/examples/", "examples"),
(r"^/usr/share/qt6/examples/", "examples"),
(r"^/usr/share/omf", "main", "/usr/share/omf/*"),
(r"^/usr/share/installed-tests/", "tests"),
(r"^/usr/libexec/installed-tests/", "tests"),
@@ -349,11 +323,17 @@ class FileManager(object):
(r"^/usr/lib64/cmake/", "dev"),
(r"^/usr/lib32/cmake/", "dev32"),
(r"^/usr/lib/qt5/mkspecs/", "dev"),
(r"^/usr/lib/qt6/mkspecs/", "dev"),
(r"^/usr/lib64/qt5/mkspecs/", "dev"),
(r"^/usr/lib32/qt5/mkspecs/", "dev32"),
(r"^/usr/lib64/qt6/mkspecs/", "dev"),
(r"^/usr/lib32/qt6/mkspecs/", "dev32"),
(r"^/usr/lib/qt5/", "lib"),
(r"^/usr/lib/qt6/", "lib"),
(r"^/usr/lib64/qt5/", "lib"),
(r"^/usr/lib32/qt5/", "lib32"),
(r"^/usr/lib64/qt6/", "lib"),
(r"^/usr/lib32/qt6/", "lib32"),
(r"^/usr/lib/[a-zA-Z0-9._+-]*\.so$", so_dest),
(r"^/usr/lib64/libkdeinit5_[a-zA-Z0-9._+-]*\.so$", "lib"),
(r"^/usr/lib32/libkdeinit5_[a-zA-Z0-9._+-]*\.so$", "lib32"),
@@ -397,6 +377,7 @@ class FileManager(object):
(r"^/lib/systemd/user/", "services"),
(r"^/usr/lib/systemd/system/", "services"),
(r"^/usr/lib/systemd/user/", "services"),
(r"^/usr/lib/udev/hwdb.d", "config"),
(r"^/usr/lib/udev/rules.d", "config"),
(r"^/usr/lib/modules-load.d", "config"),
(r"^/usr/lib/tmpfiles.d", "config"),
+1
View File
@@ -109,6 +109,7 @@ def commit_to_git(config, name, success):
call("git add profile_payload", check=False, stderr=subprocess.DEVNULL, cwd=path)
call("git add options.conf", check=False, stderr=subprocess.DEVNULL, cwd=path)
call("git add configure_misses", check=False, stderr=subprocess.DEVNULL, cwd=path)
call("git add etc_files", check=False, stderr=subprocess.DEVNULL, cwd=path)
call("git add whatrequires", check=False, stderr=subprocess.DEVNULL, cwd=path)
call("git add description", check=False, stderr=subprocess.DEVNULL, cwd=path)
call("git add attrs", check=False, stderr=subprocess.DEVNULL, cwd=path)
+1
View File
@@ -0,0 +1 @@
59FCF207FEA7F445
+28 -11
View File
@@ -29,7 +29,7 @@ import urllib.parse
import chardet
import download
from util import get_contents, get_sha1sum, print_fatal, print_warning
import util
default_license = "TO BE DETERMINED"
@@ -96,7 +96,7 @@ def decode_license(license):
def license_from_copying_hash(copying, srcdir, config, name):
"""Add licenses based on the hash of the copying file."""
try:
data = get_contents(copying)
data = util.get_contents(copying)
except FileNotFoundError:
# LICENSE file is a bad symlink (qemu-4.2.0!)
return
@@ -109,7 +109,7 @@ def license_from_copying_hash(copying, srcdir, config, name):
if not data:
return
hash_sum = get_sha1sum(copying)
hash_sum = util.get_sha1sum(copying)
if config.license_fetch:
values = {'hash': hash_sum, 'text': data, 'package': name}
@@ -142,9 +142,24 @@ def license_from_copying_hash(copying, srcdir, config, name):
else:
if not config.license_show:
return
print_warning("Unknown license {0} with hash {1}".format(copying, hash_sum))
util.print_warning("Unknown license {0} with hash {1}".format(copying, hash_sum))
hash_url = config.license_show % {'HASH': hash_sum}
print_warning("Visit {0} to enter".format(hash_url))
util.print_warning("Visit {0} to enter".format(hash_url))
def skip_license(license_path, config):
"""Check if a given license file path should be skipped."""
skip_name = False
for skip in config.license_skips:
# handle the common tempfile prefix and normalize for
# skip lines without a starting '/'
skip = skip if skip[0] != '' else skip[1:]
skip_path = ['', 'tmp', '*'] + skip
if util.globlike_match(license_path, skip_path):
util.print_warning(f"Skip license detected for file at {license_path}")
skip_name = True
break
return skip_name
def scan_for_licenses(srcdir, config, pkg_name):
@@ -166,8 +181,9 @@ def scan_for_licenses(srcdir, config, pkg_name):
for dirpath, dirnames, files in os.walk(srcdir):
for name in files:
if name.lower() in targets or target_pat.search(name.lower()):
license_from_copying_hash(os.path.join(dirpath, name),
srcdir, config, pkg_name)
license_path = os.path.join(dirpath, name)
if not skip_license(license_path, config):
license_from_copying_hash(license_path, srcdir, config, pkg_name)
# Also search for license texts in project trees that are
# REUSE-compliant, or are in process of adopting this standard (for
# example, KDE ecosystem packages). See https://reuse.software for
@@ -178,12 +194,13 @@ def scan_for_licenses(srcdir, config, pkg_name):
# seen in the DPDK 20.11.3 tree, where the `LICENSES` directory is
# named `license` instead.
dirbase = os.path.basename(dirpath)
if re.search(r'^(LICENSES|license)$', dirbase) and re.search(r'\.txt$', name):
license_from_copying_hash(os.path.join(dirpath, name),
srcdir, config, pkg_name)
if re.search(r'^(LICENSES|licenses?|licensing)$', dirbase) and re.search(r'\.txt$', name):
license_path = os.path.join(dirpath, name)
if not skip_license(license_path, config):
license_from_copying_hash(license_path, srcdir, config, pkg_name)
if not licenses:
print_fatal(" Cannot find any license or a valid {}.license file!\n".format(pkg_name))
util.print_fatal(" Cannot find any license or a valid {}.license file!\n".format(pkg_name))
sys.exit(1)
print("Licenses : ", " ".join(sorted(licenses)))
+43 -11
View File
@@ -3,12 +3,12 @@
# This file is sorted with LC_COLLATE=C
# Lines beginning with '#' are ignored.
# For strings that start with '#', escape the '#' as '\#'.
=========================
%
%license
(LGPL)
(new)
(specified
*
+
-
-MIT
@@ -26,6 +26,8 @@
3-Clause
3-clause
3BSD
=========================
>=-2
@CPACK_RPM_PACKAGE_LICENSE@
AGPLv3+
ALv2
@@ -35,7 +37,12 @@ APPLICATION__TYPE
ASL
Artistic
BSD
BSD(2
BSD(3
BSD-2-Clause-Views
BSD-3
BSD-compatible
BSD-derived(Repoze)
BSD-like
BSD-style
BSDL
@@ -43,6 +50,7 @@ BSD_3_clause
BSDish
BSL
CC-BY
Clause
Commercial
Corp.
Distribution
@@ -53,6 +61,7 @@ EPL
Eclipse
Expat(MIT/X11)
Expat/MIT
FOUNDATION
Foundation
FreeBSD
GENERAL
@@ -60,30 +69,39 @@ GFDL
GNU
GPL
GPL+
GPL-1.0-only
GPL/BSD
GPL/BSD/CPL
GPLv2.1
General
IBM
Jupyter
LESSER
LGPL
LGPL+BSD
LGPL/MIT
LICENCE
Lesser
Library
Licences
License
License(2.0)
License(==-2.0)
License(>=-2)
License(>=-2.0)
License(LGPL)
License(MIT)
License,
License-2
License-2.0
License-2.0(MPL-2.0)
License-3(GPLv3)
Licensing
Lucent
MIT-0
MIT/Expat
MPL
Minpack
Modified
Mozilla
Muddy-MIT
@@ -95,9 +113,13 @@ Open
PIL
PSF-2+
PUBLIC
PYTHON
Permission
Public
Revised
SIL
SIP
SOFTWARE
See
Set
Software
@@ -111,31 +133,46 @@ UNKNOWN
Unknown
Unkown
Unlimited
VERSION-2
Version
Version-2.0
Version-3
WITH
What
\#
a
advertising
and
any
bsd
charge
classifiers)
clause)
copy
cryptsetup-OpenSSL-exception
details
domain
domain.
dual
exceptions
for
free
gpl
granted
hereby
http://nmap.org/man/man-legal.html
into
is
it
later
later(LGPLv2+)
license
licensing
ndg/httpsclient/LICENCE
new
none
obtaining
of
on
open_source
option
@@ -143,11 +180,16 @@ option)
or
or(at
others
person
public
released
software
style
terms
the
to
under?
unencumbered
unknown
unrestricted
uses
@@ -160,13 +202,3 @@ with
your
|
~
Permission
a
charge
copy
granted
hereby
obtaining
of
person
to
+2
View File
@@ -16,9 +16,11 @@ Apache License 2.0, Apache-2.0
Apache License, Version 2.0, Apache-2.0
Apache, Apache-2.0
Apache-2, Apache-2.0
Apache2, Apache-2.0
Apache2.0, Apache-2.0
Apachev2, Apache-2.0
Artistic-1.0+GPL-1.0, Artistic-1.0 GPL-1.0
Artistic/GPL, Artistic-1.0-Perl GPL-1.0-or-later
Artistic_2, Artistic-2.0
BSD(3-clause), BSD-3-Clause
BSD-2-clause, BSD-2-Clause
+27 -7
View File
@@ -18,10 +18,25 @@
import os
import re
import sys
from util import print_fatal, write_out
def log_etc(lines):
"""Return the content of the START/etc ... END/etc section."""
etc = []
while True:
line = next(lines)
line = line.strip()
if line == 'END/etc':
break
if line.startswith('+'):
continue
etc.append(line)
return etc
def logcheck(pkg_loc):
"""Try to discover configuration options that were automatically switched off."""
log = os.path.join(pkg_loc, 'results', 'build.log')
@@ -52,7 +67,12 @@ def logcheck(pkg_loc):
pat = re.compile(r"^checking (?:for )?(.*?)\.\.\. no")
misses = []
for line in lines:
iter_lines = iter(lines)
for line in iter_lines:
if line.strip() == "START/etc":
etc = log_etc(iter_lines)
if etc:
write_log(pkg_loc, "etc_files", etc)
match = None
m = pat.search(line)
if m:
@@ -70,8 +90,8 @@ def logcheck(pkg_loc):
if match in blacklist:
print_fatal("Blacklisted configure-miss is forbidden: " + match)
misses.append("Blacklisted configure-miss is forbidden: " + match)
write_misses(pkg_loc, misses)
exit(1)
write_log(pkg_loc, 'configure_misses', misses)
sys.exit(1)
print("Configure miss: " + match)
misses.append("Configure miss: " + match)
@@ -79,9 +99,9 @@ def logcheck(pkg_loc):
if not misses:
return
write_misses(pkg_loc, misses)
write_log(pkg_loc, 'configure_misses', misses)
def write_misses(pkg_loc, misses):
"""Create configure_misses file with automatically disabled configuration options."""
write_out(os.path.join(pkg_loc, 'configure_misses'), '\n'.join(sorted(misses)))
def write_log(pkg_loc, fname, content):
"""Create log file with content."""
write_out(os.path.join(pkg_loc, fname), '\n'.join(sorted(content)))
+10 -6
View File
@@ -42,7 +42,7 @@ KEYID_TRY = ""
KEYID = ""
IMPORTED = ""
EMAIL = ""
GNUPGCONF = """keyserver keys.gnupg.net"""
GNUPGCONF = """keyserver keyserver.ubuntu.com"""
CMD_TIMEOUT = 20
ENV = os.environ
INPUT_GETTER_TIMEOUT = 60
@@ -205,9 +205,9 @@ class Verifier(object):
@staticmethod
def quit():
"""Stop verification."""
print('Critical error quitting...')
util.print_fatal("Verification required for build (verify_required option set)")
print(SEPT)
exit(1)
sys.exit(1)
@staticmethod
def calc_sum(filepath, digest_algo):
@@ -264,12 +264,12 @@ def get_signature_file(package_url, package_path):
def compare_keys(newkey, oldkey):
"""Key comparison to check against key tampering."""
if newkey != oldkey:
util.print_error('Public key has changed:\n'
util.print_fatal('Public key has changed:\n'
' old key: {}\n'
' new key: {}\n'
'this is a critical security error, quitting...'
.format(oldkey, newkey))
exit(1)
sys.exit(1)
# sha256sum Verifier
@@ -483,6 +483,10 @@ class GPGVerifier(Verifier):
return None
# valid signature exists at package_sign_path, operate on it now
keyid = get_keyid(self.package_sign_path)
if keyid in self.config.keyid_blocklist:
self.print_result(False, err_msg='KNOWNBADACTOR: {}'.format(keyid))
self.quit()
# default location first
pubkey_loc = self.pubkey_path.format(keyid)
cache_key = os.path.join(KEY_CACHE_DIR, pubkey_loc)
@@ -517,7 +521,6 @@ class GPGVerifier(Verifier):
def quit_verify():
"""Halt build due to verification being required."""
util.print_error("Verification required for build (verify_required option set)")
Verifier.quit()
@@ -528,6 +531,7 @@ VERIFIER_TYPES = {
'.bz2': GPGVerifier,
'.xz': GPGVerifier,
'.zip': GPGVerifier,
'.zst': GPGVerifier,
}
+1 -1
View File
@@ -110,7 +110,7 @@ def main():
pkg_name = sys.argv[1]
pypi_name = get_pypi_name(pkg_name)
if not pypi_name:
print(f"Couldn't find {pkg_name} in pypi")
util.print_fatal(f"Couldn't find {pkg_name} in pypi")
sys.exit(1)
pypi_metadata = get_pypi_metadata(pypi_name)
print(pypi_metadata)
+64 -64
View File
@@ -2,67 +2,67 @@
# pkgconfig equivalent, to be added to BuildRequires. Please keep sorted.
# This file is sorted with LC_COLLATE=C
# Format: <qmake key>, <pkgconfig value>
3danimation, Qt53DAnimation
3dcore, Qt53DCore
3dextras, Qt53DExtras
3dinput, Qt53DInput
3dlogic, Qt53DLogic
3dquick, Qt53DQuick
3dquickanimation, Qt53DQuickAnimation
3dquickextras, Qt53DQuickExtras
3dquickinput, Qt53DQuickInput
3dquickrender, Qt53DQuickRender
3dquickscene2d, Qt53DQuickScene2D
3drender, Qt53DRender
bluetooth, Qt5Bluetooth
charts, Qt5Charts
concurrent, Qt5Concurrent
core, Qt5Core
datavisualization, Qt5DataVisualization
dbus, Qt5DBus
declarative, Qt5Declarative
designer, Qt5Designer
gamepad, Qt5Gamepad
gui, Qt5Gui
help, Qt5Help
location, Qt5Location
multimedia, Qt5Multimedia
multimediawidgets, Qt5MultimediaWidgets
network, Qt5Network
networkauth, Qt5NetworkAuth
nfc, Qt5Nfc
opengl, Qt5OpenGL
openglextensions, Qt5OpenGLExtensions
positioning, Qt5Positioning
printsupport, Qt5PrintSupport
purchasing, Qt5Purchasing
qml, Qt5Qml
qmltest, Qt5QuickTest
quick, Qt5Quick
quickcontrols2, Qt5QuickControls2
quickwidgets, Qt5QuickWidgets
remoteobjects, Qt5RemoteObjects
script, Qt5Script
scripttools, Qt5ScriptTools
scxml, Qt5Scxml
sensors, Qt5Sensors
serialbus, Qt5SerialBus
serialport, Qt5SerialPort
sql, Qt5Sql
svg, Qt5Svg
testlib, Qt5Test
texttospeech, Qt5TextToSpeech
uitools, Qt5UiTools
waylandclient, Qt5WaylandClient
waylandcompositor, Qt5WaylandCompositor
webchannel, Qt5WebChannel
webengine, Qt5WebEngine
webenginecore, Qt5WebEngineCore
webenginewidgets, Qt5WebEngineWidgets
webkit, Qt5WebKit
webkitwidgets, Qt5WebKitWidgets
websockets, Qt5WebSockets
widgets, Qt5Widgets
x11extras, Qt5X11Extras
xml, Qt5Xml
xmlpatterns, Qt5XmlPatterns
3danimation, Qt63DAnimation
3dcore, Qt63DCore
3dextras, Qt63DExtras
3dinput, Qt63DInput
3dlogic, Qt63DLogic
3dquick, Qt63DQuick
3dquickanimation, Qt63DQuickAnimation
3dquickextras, Qt63DQuickExtras
3dquickinput, Qt63DQuickInput
3dquickrender, Qt63DQuickRender
3dquickscene2d, Qt63DQuickScene2D
3drender, Qt63DRender
bluetooth, Qt6Bluetooth
charts, Qt6Charts
concurrent, Qt6Concurrent
core, Qt6Core
datavisualization, Qt6DataVisualization
dbus, Qt6DBus
declarative, Qt6Declarative
designer, Qt6Designer
gamepad, Qt6Gamepad
gui, Qt6Gui
help, Qt6Help
location, Qt6Location
multimedia, Qt6Multimedia
multimediawidgets, Qt6MultimediaWidgets
network, Qt6Network
networkauth, Qt6NetworkAuth
nfc, Qt6Nfc
opengl, Qt6OpenGL
openglextensions, Qt6OpenGLExtensions
positioning, Qt6Positioning
printsupport, Qt6PrintSupport
purchasing, Qt6Purchasing
qml, Qt6Qml
qmltest, Qt6QuickTest
quick, Qt6Quick
quickcontrols2, Qt6QuickControls2
quickwidgets, Qt6QuickWidgets
remoteobjects, Qt6RemoteObjects
script, Qt6Script
scripttools, Qt6ScriptTools
scxml, Qt6Scxml
sensors, Qt6Sensors
serialbus, Qt6SerialBus
serialport, Qt6SerialPort
sql, Qt6Sql
svg, Qt6Svg
testlib, Qt6Test
texttospeech, Qt6TextToSpeech
uitools, Qt6UiTools
waylandclient, Qt6WaylandClient
waylandcompositor, Qt6WaylandCompositor
webchannel, Qt6WebChannel
webengine, Qt6WebEngine
webenginecore, Qt6WebEngineCore
webenginewidgets, Qt6WebEngineWidgets
webkit, Qt6WebKit
webkitwidgets, Qt6WebKitWidgets
websockets, Qt6WebSockets
widgets, Qt6Widgets
x11extras, Qt6X11Extras
xml, Qt6Xml
xmlpatterns, Qt6XmlPatterns
+260 -109
View File
@@ -26,6 +26,8 @@ import types
from collections import OrderedDict
import git
from jinja2 import Environment
from jinja2.loaders import DictLoader
from util import _file_write, open_auto
AVX2_CFLAGS = "-march=x86-64-v3"
@@ -73,12 +75,42 @@ class Specfile(object):
def write_spec(self):
"""Write spec file."""
self.specfile = open_auto("{}/{}.spec".format(self.config.download_path, self.name), "w")
spec_path = f"{os.path.join(self.config.download_path, self.name)}.spec"
self.specfile = open_auto(spec_path, "w")
self.specfile.write_strip = types.MethodType(_file_write, self.specfile)
# last chance to sanitize url for template and build types
if self.config.urlban:
clean_url = re.sub(self.config.urlban, "localhost", self.url)
# Duplicate prefixes entry before we change the url
self.content.prefixes[clean_url] = self.content.prefixes.get(self.url)
self.url = clean_url
template_path = f"{spec_path}.template"
if os.path.isfile(template_path):
# make templates have a template build pattern
self.config.default_pattern = "template"
# spec file comment header
self.write_comment_header()
if os.path.isfile(template_path):
with open_auto(template_path) as tfile:
template_content = tfile.read()
template = Environment(loader=DictLoader({'spec': template_content})).get_template('spec')
kw = {
'package_name': self.name,
'package_version': self.version,
'package_url': self.url,
'package_release': self.release,
}
self.specfile.write(template.render(**kw))
self.specfile.write_strip('\n')
self.specfile.close()
# return specfile type built so autospec knows how to
# handle build results (template should only builds once)
return "template"
if self.config.config_opts.get('keepstatic'):
self._write("%define keepstatic 1\n")
@@ -110,6 +142,10 @@ class Specfile(object):
self.specfile.close()
# return specfile type built so autospec knows how to
# handle build results (generate has multiple builds)
return "generate"
def write_comment_header(self):
"""Write comment header to spec file."""
self._write("#\n")
@@ -132,11 +168,6 @@ class Specfile(object):
def write_nvr(self):
"""Write name, version, and release information."""
if self.config.urlban:
clean_url = re.sub(self.config.urlban, "localhost", self.url)
# Duplicate prefixes entry before we change the url
self.content.prefixes[clean_url] = self.content.prefixes.get(self.url)
self.url = clean_url
self._write("Name : {}\n".format(self.name))
self._write("Version : {}\n".format(self.version))
self._write("Release : {}\n".format(str(self.release)))
@@ -159,8 +190,17 @@ class Specfile(object):
# if package is verified, include the signature in the source tarball
if self.keyid and self.config.signature:
# We'll need gnupg to verify the signature. Need to add it here so it's ready before write_buildreq
self.requirements.add_buildreq("gnupg")
count += 1
self._write_strip(f"Source{count} : {self.config.signature}")
self.config.signature_macro = f"%{{SOURCE{count}}}"
# Also include the public key in the source tarball.
count += 1
self._write_strip(f"Source{count} : {self.keyid}.pkey")
self.config.pkey_macro = f"%{{SOURCE{count}}}"
for source in self.config.extra_sources:
count += 1
@@ -286,6 +326,9 @@ class Specfile(object):
self._write("Group: Default\n")
for dep in deps.get(pkg, []):
# honor requires_ban for manual overrides
if f"{self.name}-{dep}" in self.requirements.banned_requires.get(pkg, []):
continue
if dep in self.packages:
self._write("Requires: {}-{} = %{{version}}-%{{release}}\n".format(self.name, dep))
@@ -405,6 +448,8 @@ class Specfile(object):
self._write_strip("## make_prepend end")
if self.config.make_command:
make = self.config.make_command
elif self.config.config_opts['use_ninja']:
make = "ninja"
else:
make = "make"
if build32:
@@ -421,7 +466,11 @@ class Specfile(object):
def write_cmake_line_openmpi(self):
"""Write cmake line (openmpi) to spec file."""
cmake_string = 'cmake -G "Unix Makefiles" -DCMAKE_INSTALL_PREFIX=$MPI_ROOT -DCMAKE_INSTALL_SBINDIR=$MPI_BIN \\\n' \
if self.config.config_opts['use_ninja']:
cmake_type = "Ninja"
else:
cmake_type = "Unix Makefiles"
cmake_string = f"cmake -G '{cmake_type}' -DCMAKE_INSTALL_PREFIX=$MPI_ROOT -DCMAKE_INSTALL_SBINDIR=$MPI_BIN \\\n" \
'-DCMAKE_INSTALL_LIBDIR=$MPI_LIB -DCMAKE_INSTALL_INCLUDEDIR=$MPI_INCLUDE -DLIB_INSTALL_DIR=$MPI_LIB \\\n' \
'-DBUILD_SHARED_LIBS:BOOL=ON -DLIB_SUFFIX=64 \\\n' \
'-DCMAKE_AR=/usr/bin/gcc-ar -DCMAKE_BUILD_TYPE=RelWithDebInfo -DCMAKE_RANLIB=/usr/bin/gcc-ranlib \\\n'
@@ -430,18 +479,35 @@ class Specfile(object):
def write_prep(self):
"""Write prep section to spec file."""
self._write_strip("%prep")
if self.keyid and self.config.signature:
self._write_strip("mkdir .gnupg")
self._write_strip("chmod 700 .gnupg")
self._write_strip(f"gpg --homedir .gnupg --import {self.config.pkey_macro}")
self._write_strip(f"gpg --homedir .gnupg --status-fd 1 --verify {self.config.signature_macro} %{{SOURCE0}} > gpg.status")
self._write_strip(f"grep -E '^\\[GNUPG:\\] (GOODSIG|EXPKEYSIG) {self.keyid}' gpg.status")
self.write_prep_prepend()
prefix = self.content.prefixes[self.url]
if self.config.default_pattern == 'R':
prefix = self.content.tarball_prefix
self._write_strip("%setup -q -n " + prefix)
else:
self._write_strip("%setup -q -n " + prefix)
prefix = self.content.prefixes[self.url]
if not prefix:
prefix = os.path.splitext(os.path.basename(self.url))[0]
self._write_strip("%setup -q -c -n " + prefix)
else:
self._write_strip("%setup -q -n " + prefix)
for archive in self.config.sources["archive"]:
# Handle various archive types
extract_cmd = 'tar xf {}'
if archive.endswith('.zip'):
extract_cmd = 'unzip -q {}'
if archive.endswith('.bz2') and not archive.endswith('.tar.bz2'):
extract_cmd = 'bzcat {0} > $(basename "{0}" .bz2)'
if archive.endswith('.zst'):
if archive.endswith('.tar.zst'):
extract_cmd = 'tar -I zstd xf {}'
else:
extract_cmd = 'zstd -dqc {0} > $(basename "{0}" .zst)'
self._write_strip('cd %{_builddir}')
archive_file = os.path.basename(archive)
if self.config.archive_details.get(archive + "prefix"):
@@ -474,45 +540,43 @@ class Specfile(object):
if not archive_prefix:
# Make it up
archive_prefix = os.path.splitext(os.path.basename(archive))[0]
self._write_strip("cp -r %{{_builddir}}/{0}/* %{{_builddir}}/{1}/{2}"
self._write_strip("cp -r %{{_builddir}}/{0}/. %{{_builddir}}/{1}/{2}"
.format(archive_prefix,
self.content.tarball_prefix,
destination))
self.apply_patches()
# setup cargo.toml vendoring if needed
if self.config.config_opts['cargo_vendor']:
if self.config.cargo_vendors:
if self.config.subdir:
self._write_strip("pushd " + self.config.subdir)
self._write_strip("mkdir -p .cargo")
self._write_strip("echo '[source.crates-io]' >> .cargo/config.toml")
self._write_strip("""echo 'replace-with = "vendored-sources"' >> .cargo/config.toml""")
self._write_strip("echo '[source.vendored-sources]' >> .cargo/config.toml")
self._write_strip("""echo 'directory = "vendor"' >> .cargo/config.toml""")
self._write_strip(f"echo '\n{self.config.cargo_vendors}' >> .cargo/config.toml")
if self.config.subdir:
self._write_strip("popd")
if self.config.default_pattern != 'cmake':
if self.config.config_opts['32bit']:
self._write_strip("pushd ..")
self._write_strip("cp -a {} build32".format(self.content.tarball_prefix))
self._write_strip("popd")
if self.config.config_opts['use_avx2']:
self._write_strip("pushd ..")
self._write_strip("cp -a {} buildavx2".format(self.content.tarball_prefix))
self._write_strip("popd")
if self.config.config_opts['use_avx512']:
self._write_strip("pushd ..")
self._write_strip("cp -a {} buildavx512".format(self.content.tarball_prefix))
self._write_strip("popd")
if self.config.config_opts['use_apx']:
self._write_strip("pushd ..")
self._write_strip("cp -a {} buildapx".format(self.content.tarball_prefix))
self._write_strip("popd")
if self.config.config_opts['openmpi']:
self._write_strip("pushd ..")
self._write_strip("cp -a {} build-openmpi".format(self.content.tarball_prefix))
self._write_strip("popd")
self.write_copy_prepend()
if self.config.config_opts['32bit']:
self._write_strip("pushd ..")
self._write_strip("cp -a {} build32".format(self.content.tarball_prefix))
self._write_strip("popd")
if self.config.config_opts['use_avx2']:
self._write_strip("pushd ..")
self._write_strip("cp -a {} buildavx2".format(self.content.tarball_prefix))
self._write_strip("popd")
if self.config.config_opts['use_avx512']:
self._write_strip("pushd ..")
self._write_strip("cp -a {} buildavx512".format(self.content.tarball_prefix))
self._write_strip("popd")
if self.config.config_opts['use_apx']:
self._write_strip("pushd ..")
self._write_strip("cp -a {} buildapx".format(self.content.tarball_prefix))
self._write_strip("popd")
if self.config.config_opts['openmpi']:
self._write_strip("pushd ..")
self._write_strip("cp -a {} build-openmpi".format(self.content.tarball_prefix))
self._write_strip("popd")
self._write_strip("\n")
def write_32bit_exports(self):
@@ -574,10 +638,10 @@ class Specfile(object):
if self.config.config_opts['use_clang']:
flags.extend(["-O3"])
else:
flags.extend(["-Ofast", "-fno-semantic-interposition", "-falign-functions=32"])
flags.extend(["-fno-semantic-interposition", "-falign-functions=32"])
if not self.config.config_opts['full-debug-info'] and not self.config.config_opts['use_clang']:
flags.extend(["-gno-variable-location-views", "-gno-column-info", "-femit-struct-debug-baseonly", "-fdebug-types-section", "-gz=zstd", "-g1"])
if self.config.default_pattern != 'qmake':
if self.config.default_pattern != 'qmake' or self.config.default_pattern != 'qmake6':
if self.config.config_opts['use_lto']:
flags.extend(["-O3", lto, "-ffat-lto-objects"])
if self.config.config_opts['use_clang']:
@@ -710,6 +774,7 @@ class Specfile(object):
self.write_license_files()
self._write_strip("export GOAMD64=v2")
if self.config.config_opts['32bit']:
self._write_strip("pushd ../build32/" + self.config.subdir)
self._write_strip("%make_install32 {} {}".format(self.config.extra_make_install,
@@ -729,21 +794,25 @@ class Specfile(object):
self._write_strip("popd")
if self.config.config_opts['use_avx2']:
self._write_strip("GOAMD64=v3")
self._write_strip("pushd ../buildavx2/" + self.config.subdir)
self._write_strip("%s_v3 %s\n" % (self.config.install_macro, self.config.extra_make_install))
self._write_strip("popd")
if self.config.config_opts['use_avx512']:
self._write_strip("GOAMD64=v4")
self._write_strip("pushd ../buildavx512/" + self.config.subdir)
self._write_strip("%s_v4 %s\n" % (self.config.install_macro, self.config.extra_make_install))
self._write_strip("popd")
if self.config.config_opts['use_apx']:
self._write_strip("GOAMD64=v3")
self._write_strip("pushd ../buildapx/" + self.config.subdir)
self._write_strip("%s_va %s\n" % (self.config.install_macro, self.config.extra_make_install))
self._write_strip("popd")
if self.config.config_opts['openmpi']:
self._write_strip("GOAMD64=v3")
self._write_strip("pushd ../build-openmpi/" + self.config.subdir)
self.write_install_openmpi()
self._write_strip("popd")
@@ -751,6 +820,7 @@ class Specfile(object):
if self.config.subdir:
self._write_strip("pushd " + self.config.subdir)
self._write_strip("GOAMD64=v2")
self._write_strip("%s %s\n" % (self.config.install_macro, self.config.extra_make_install))
if self.config.subdir:
@@ -766,6 +836,14 @@ class Specfile(object):
self._write_strip("{}\n".format(line))
self._write_strip("## prep_prepend end")
def write_copy_prepend(self):
"""Write out any custom supplied commands prior to creating source copies for avx, etc builds."""
if self.config.copy_prepend:
self._write_strip("## copy_prepend content")
for line in self.config.copy_prepend:
self._write_strip("{}\n".format(line))
self._write_strip("## copy_prepend end")
def write_build_prepend(self):
"""Write out any custom supplied commands at the start of the %build section and every build type."""
if self.config.build_prepend:
@@ -879,10 +957,13 @@ class Specfile(object):
self.write_license_files()
if self.config.subdir:
self._write_strip("pushd " + self.config.subdir)
self._write_strip("export GOAMD64=v2")
if self.config.config_opts['use_ninja'] and self.config.install_macro == '%make_install':
self.config.install_macro = '%ninja_install'
if self.config.config_opts['32bit']:
self._write_strip("pushd ../build32/" + self.config.subdir)
self._write_strip("pushd clr-build32")
self._write_strip("{}32 {} {}".format(self.config.install_macro,
self.config.extra_make_install,
@@ -900,27 +981,44 @@ class Specfile(object):
self._write_strip(" popd")
self._write_strip("fi")
self._write_strip("popd")
self._write_strip("popd")
if self.config.config_opts['use_avx2']:
self._write_strip("pushd ../buildavx2/" + self.config.subdir)
self._write_strip("GOAMD64=v3")
self._write_strip("pushd clr-build-avx2")
self._write_strip("%s_v3 %s || :\n" % (self.config.install_macro, self.config.extra_make_install))
self._write_strip("popd")
self._write_strip("popd")
if self.config.config_opts['use_avx512']:
self._write_strip("pushd ../buildavx512/" + self.config.subdir)
self._write_strip("GOAMD64=v4")
self._write_strip("pushd clr-build-avx512")
self._write_strip("%s_v4 %s || :\n" % (self.config.install_macro, self.config.extra_make_install))
self._write_strip("popd")
self._write_strip("popd")
if self.config.config_opts['use_apx']:
self._write_strip("pushd ../buildapx/" + self.config.subdir)
self._write_strip("GOAMD64=v3")
self._write_strip("pushd clr-build-apx")
self._write_strip("%s_va %s || :\n" % (self.config.install_macro, self.config.extra_make_install))
self._write_strip("popd")
self._write_strip("popd")
if self.config.config_opts['openmpi']:
self._write_strip("pushd ../build-openmpi/" + self.config.subdir)
self._write_strip("GOAMD64=v3")
self._write_strip("pushd clr-build-openmpi")
self.write_install_openmpi()
self._write_strip("popd")
self._write_strip("popd")
if self.config.subdir:
self._write_strip("pushd " + self.config.subdir)
self._write_strip("GOAMD64=v2")
self._write_strip("pushd clr-build")
self._write_strip("%s %s\n" % (self.config.install_macro, self.config.extra_make_install))
self._write_strip("popd")
@@ -1003,6 +1101,7 @@ class Specfile(object):
self.write_profile_payload("configure")
if self.config.subdir:
self._write_strip("pushd {}".format(self.config.subdir))
self._write_strip("export GOAMD64=v2")
self._write_strip("{0}%configure {1} {2} {3}"
.format(self.get_profile_use_flags(),
self.config.disable_static,
@@ -1031,6 +1130,7 @@ class Specfile(object):
self._write_strip("unset PKG_CONFIG_PATH")
self._write_strip("pushd ../buildavx2/" + self.config.subdir)
self.write_build_prepend()
self._write_strip("GOAMD64=v3")
self._write_strip(f'CFLAGS="$CLEAR_INTERMEDIATE_CFLAGS {AVX2_CFLAGS} {AVX2_LFLAGS} "')
self._write_strip(f'CXXFLAGS="$CLEAR_INTERMEDIATE_CXXFLAGS {AVX2_CFLAGS} {AVX2_LFLAGS} "')
self._write_strip(f'FFLAGS="$CLEAR_INTERMEDIATE_FFLAGS {AVX2_CFLAGS} {AVX2_LFLAGS} "')
@@ -1047,6 +1147,7 @@ class Specfile(object):
self._write_strip("unset PKG_CONFIG_PATH")
self._write_strip("pushd ../buildavx512/" + self.config.subdir)
self.write_build_prepend()
self._write_strip("GOAMD64=v4")
self._write_strip(f'CFLAGS="$CLEAR_INTERMEDIATE_CFLAGS {AVX512_CFLAGS} {AVX512_LFLAGS} "')
self._write_strip(f'CXXFLAGS="$CLEAR_INTERMEDIATE_CXXFLAGS {AVX512_CFLAGS} {AVX512_LFLAGS} "')
self._write_strip(f'FFLAGS="$CLEAR_INTERMEDIATE_FFLAGS {AVX512_CFLAGS} {AVX512_LFLAGS} "')
@@ -1063,7 +1164,7 @@ class Specfile(object):
self._write_strip("unset PKG_CONFIG_PATH")
self._write_strip("pushd ../buildapx/" + self.config.subdir)
self.write_build_prepend()
self._write_strip('CC="gcc-14"')
self._write_strip("GOAMD64=v3")
self._write_strip(f'CFLAGS="$CLEAR_INTERMEDIATE_CFLAGS {APX_CFLAGS} {APX_LFLAGS} "')
self._write_strip(f'CXXFLAGS="$CLEAR_INTERMEDIATE_CXXFLAGS {AVX2_CFLAGS} {AVX2_LFLAGS} "')
self._write_strip(f'FFLAGS="$CLEAR_INTERMEDIATE_FFLAGS {APX_CFLAGS} {APX_LFLAGS} "')
@@ -1081,6 +1182,7 @@ class Specfile(object):
self._write_strip(". /usr/share/defaults/etc/profile.d/modules.sh")
self._write_strip("module load openmpi")
self.write_build_prepend()
self._write_strip("GOAMD64=v3")
self._write_strip(f'CFLAGS="$CLEAR_INTERMEDIATE_CFLAGS {AVX2_CFLAGS} {AVX2_LFLAGS} "')
self._write_strip(f'CXXFLAGS="$CLEAR_INTERMEDIATE_CXXFLAGS {AVX2_CFLAGS} {AVX2_LFLAGS} "')
self._write_strip(f'FFLAGS="$CLEAR_INTERMEDIATE_FFLAGS {AVX2_CFLAGS} {AVX2_LFLAGS} "')
@@ -1105,6 +1207,7 @@ class Specfile(object):
self.write_profile_payload("configure_ac")
if self.config.subdir:
self._write_strip("pushd " + self.config.subdir)
self._write_strip("export GOAMD64=v2")
self._write_strip("{0}%reconfigure {1} {2} {3}"
.format(self.get_profile_use_flags(),
self.config.disable_static,
@@ -1132,6 +1235,7 @@ class Specfile(object):
self._write_strip("unset PKG_CONFIG_PATH")
self._write_strip("pushd ../buildavx2/" + self.config.subdir)
self.write_build_prepend()
self._write_strip("GOAMD64=v3")
self._write_strip(f'CFLAGS="$CLEAR_INTERMEDIATE_CFLAGS {AVX2_CFLAGS} {AVX2_LFLAGS} "')
self._write_strip(f'CXXFLAGS="$CLEAR_INTERMEDIATE_CXXFLAGS {AVX2_CFLAGS} {AVX2_LFLAGS} "')
self._write_strip(f'FFLAGS="$CLEAR_INTERMEDIATE_FFLAGS {AVX2_CFLAGS} {AVX2_LFLAGS} "')
@@ -1148,6 +1252,7 @@ class Specfile(object):
self._write_strip("unset PKG_CONFIG_PATH")
self._write_strip("pushd ../buildavx512/" + self.config.subdir)
self.write_build_prepend()
self._write_strip("GOAMD64=v4")
self._write_strip(f'CFLAGS="$CLEAR_INTERMEDIATE_CFLAGS {AVX512_CFLAGS} {AVX512_LFLAGS} "')
self._write_strip(f'CXXFLAGS="$CLEAR_INTERMEDIATE_CXXFLAGS {AVX512_CFLAGS} {AVX512_LFLAGS} "')
self._write_strip(f'FFLAGS="$CLEAR_INTERMEDIATE_FFLAGS {AVX512_CFLAGS} {AVX512_LFLAGS} "')
@@ -1164,7 +1269,7 @@ class Specfile(object):
self._write_strip("unset PKG_CONFIG_PATH")
self._write_strip("pushd ../buildapx/" + self.config.subdir)
self.write_build_prepend()
self._write_strip('CC="gcc-14"')
self._write_strip("GOAMD64=v3")
self._write_strip(f'CFLAGS="$CLEAR_INTERMEDIATE_CFLAGS {APX_CFLAGS} {APX_LFLAGS} "')
self._write_strip(f'CXXFLAGS="$CLEAR_INTERMEDIATE_CXXFLAGS {AVX2_CFLAGS} {AVX2_LFLAGS} "')
self._write_strip(f'FFLAGS="$CLEAR_INTERMEDIATE_FFLAGS {APX_CFLAGS} {APX_LFLAGS} "')
@@ -1189,6 +1294,7 @@ class Specfile(object):
self.write_profile_payload("make")
if self.config.subdir:
self._write_strip("pushd " + self.config.subdir)
self._write_strip("export GOAMD64=v2")
self.write_make_line()
if self.config.subdir:
self._write_strip("popd")
@@ -1201,6 +1307,7 @@ class Specfile(object):
if self.config.config_opts['use_avx2']:
self._write_strip("pushd ../buildavx2" + self.config.subdir)
self.write_build_prepend()
self._write_strip("GOAMD64=v3")
self._write_strip(f'CFLAGS="$CLEAR_INTERMEDIATE_CFLAGS {AVX2_CFLAGS} {AVX2_LFLAGS} "')
self._write_strip(f'CXXFLAGS="$CLEAR_INTERMEDIATE_CXXFLAGS {AVX2_CFLAGS} {AVX2_LFLAGS} "')
self._write_strip(f'FFLAGS="$CLEAR_INTERMEDIATE_FFLAGS {AVX2_CFLAGS} {AVX2_LFLAGS} "')
@@ -1211,6 +1318,7 @@ class Specfile(object):
if self.config.config_opts['use_avx512']:
self._write_strip("pushd ../buildavx512" + self.config.subdir)
self.write_build_prepend()
self._write_strip("GOAMD64=v4")
self._write_strip(f'CFLAGS="$CLEAR_INTERMEDIATE_CFLAGS {AVX512_CFLAGS} {AVX512_LFLAGS} "')
self._write_strip(f'CXXFLAGS="$CLEAR_INTERMEDIATE_CXXFLAGS {AVX512_CFLAGS} {AVX512_LFLAGS} "')
self._write_strip(f'FFLAGS="$CLEAR_INTERMEDIATE_FFLAGS {AVX512_CFLAGS} {AVX512_LFLAGS} "')
@@ -1221,7 +1329,7 @@ class Specfile(object):
if self.config.config_opts['use_apx'] and not self.config.config_opts['use_clang']:
self._write_strip("pushd ../buildapx" + self.config.subdir)
self.write_build_prepend()
self._write_strip('CC=gcc-14')
self._write_strip("GOAMD64=v3")
self._write_strip(f'CFLAGS="$CLEAR_INTERMEDIATE_CFLAGS {APX_CFLAGS} {APX_LFLAGS} "')
self._write_strip(f'CXXFLAGS="$CLEAR_INTERMEDIATE_CXXFLAGS {AVX2_CFLAGS} {AVX2_LFLAGS} "')
self._write_strip(f'FFLAGS="$CLEAR_INTERMEDIATE_FFLAGS {APX_CFLAGS} {APX_LFLAGS} "')
@@ -1240,6 +1348,9 @@ class Specfile(object):
self.write_lang_c(export_epoch=True)
self.write_variables()
self.write_profile_payload("autogen")
if self.config.subdir:
self._write_strip("pushd " + self.config.subdir)
self._write_strip("export GOAMD64=v2")
self._write_strip("{0}%autogen {1} {2} {3}"
.format(self.get_profile_use_flags(),
self.config.disable_static,
@@ -1247,6 +1358,8 @@ class Specfile(object):
self.config.extra_configure64))
self.write_make_line()
self._write_strip("\n")
if self.config.subdir:
self._write_strip("popd")
if self.config.config_opts['32bit']:
self._write_strip("pushd ../build32/" + self.config.subdir)
self.write_build_prepend()
@@ -1265,6 +1378,7 @@ class Specfile(object):
if self.config.config_opts['use_avx2']:
self._write_strip("pushd ../buildavx2/" + self.config.subdir)
self.write_build_prepend()
self._write_strip("GOAMD64=v3")
self._write_strip(f'CFLAGS="$CLEAR_INTERMEDIATE_CFLAGS {AVX2_CFLAGS} {AVX2_LFLAGS} "')
self._write_strip(f'CXXFLAGS="$CLEAR_INTERMEDIATE_CXXFLAGS {AVX2_CFLAGS} {AVX2_LFLAGS} "')
self._write_strip(f'FFLAGS="$CLEAR_INTERMEDIATE_FFLAGS {AVX2_CFLAGS} {AVX2_LFLAGS} "')
@@ -1280,6 +1394,7 @@ class Specfile(object):
if self.config.config_opts['use_avx512']:
self._write_strip("pushd ../buildavx512/" + self.config.subdir)
self.write_build_prepend()
self._write_strip("GOAMD64=v4")
self._write_strip(f'CFLAGS="$CLEAR_INTERMEDIATE_CFLAGS {AVX512_CFLAGS} {AVX512_LFLAGS} "')
self._write_strip(f'CXXFLAGS="$CLEAR_INTERMEDIATE_CXXFLAGS {AVX512_CFLAGS} {AVX512_LFLAGS} "')
self._write_strip(f'FFLAGS="$CLEAR_INTERMEDIATE_FFLAGS {AVX512_CFLAGS} {AVX512_LFLAGS} "')
@@ -1295,13 +1410,13 @@ class Specfile(object):
if self.config.config_opts['use_apx'] and not self.config.config_opts['use_clang']:
self._write_strip("pushd ../buildapx/" + self.config.subdir)
self.write_build_prepend()
self._write_strip('CC=gcc-14')
self._write_strip("GOAMD64=v3")
self._write_strip(f'CFLAGS="$CLEAR_INTERMEDIATE_CFLAGS {APX_CFLAGS} {APX_LFLAGS} "')
self._write_strip(f'CXXFLAGS="$CLEAR_INTERMEDIATE_CXXFLAGS {AVX2_CFLAGS} {AVX2_LFLAGS} "')
self._write_strip(f'FFLAGS="$CLEAR_INTERMEDIATE_FFLAGS {APX_CFLAGS} {APX_LFLAGS} "')
self._write_strip(f'FCFLAGS="$CLEAR_INTERMEDIATE_FCFLAGS {APX_CFLAGS} "')
self._write_strip(f'LDFLAGS="$CLEAR_INTERMEDIATE_LDFLAGS {APX_LCFLAGS} "')
self._write_strip("%autogen {0} {1} {2} "
self._write_strip("%autogen --host=x86_64-clr-linux-gnu {0} {1} {2} "
.format(self.config.disable_static,
self.config.extra_configure,
self.config.extra_configure_apx))
@@ -1322,6 +1437,9 @@ class Specfile(object):
for module in self.config.pypi_overrides:
self._write_strip(f"pypi-dep-fix.py . {module}")
self._write_strip("python3 -m build --wheel --skip-dependency-check --no-isolation " + self.config.extra_configure)
self._write_strip("\n")
if self.config.subdir:
self._write_strip("popd")
if self.config.config_opts['use_avx2']:
self._write_strip("pushd ../buildavx2/" + self.config.subdir)
@@ -1340,7 +1458,6 @@ class Specfile(object):
if self.config.config_opts['use_apx'] and not self.config.config_opts['use_clang']:
self._write_strip("pushd ../buildapx/" + self.config.subdir)
self.write_build_prepend()
self._write_strip('CC=gcc-14')
self._write_strip(f'CFLAGS="$CLEAR_INTERMEDIATE_CFLAGS {APX_CFLAGS} {APX_LFLAGS} "')
self._write_strip(f'CXXFLAGS="$CLEAR_INTERMEDIATE_CXXFLAGS {AVX2_CFLAGS} {AVX2_LFLAGS} "')
self._write_strip(f'FFLAGS="$CLEAR_INTERMEDIATE_FFLAGS {APX_CFLAGS} {APX_LFLAGS} "')
@@ -1353,8 +1470,6 @@ class Specfile(object):
self._write_strip("popd")
self._write_strip("\n")
if self.config.subdir:
self._write_strip("popd")
self.write_build_append()
self.write_check()
self._write_strip("%install")
@@ -1388,7 +1503,6 @@ class Specfile(object):
if self.config.config_opts['use_apx'] and not self.config.config_opts['use_clang']:
self._write_strip("pushd ../buildapx/" + self.config.subdir)
self._write_strip('CC=gcc-14')
self._write_strip(f'CFLAGS="$CLEAR_INTERMEDIATE_CFLAGS {APX_CFLAGS} {APX_LFLAGS} "')
self._write_strip(f'CXXFLAGS="$CLEAR_INTERMEDIATE_CXXFLAGS {AVX2_CFLAGS} {AVX2_LFLAGS} "')
self._write_strip(f'FFLAGS="$CLEAR_INTERMEDIATE_FFLAGS {APX_CFLAGS} {APX_LFLAGS} "')
@@ -1431,7 +1545,6 @@ class Specfile(object):
if self.config.config_opts['use_apx'] and not self.config.config_opts['use_clang']:
self._write_strip("pushd ../buildapx/" + self.config.subdir)
self.write_build_prepend()
self._write_strip('CC=gcc-14')
self._write_strip(f'CFLAGS="$CLEAR_INTERMEDIATE_CFLAGS {APX_CFLAGS} {APX_LFLAGS} "')
self._write_strip(f'CXXFLAGS="$CLEAR_INTERMEDIATE_CXXFLAGS {AVX2_CFLAGS} {AVX2_LFLAGS} "')
self._write_strip(f'FFLAGS="$CLEAR_INTERMEDIATE_FFLAGS {APX_CFLAGS} {APX_LFLAGS} "')
@@ -1476,7 +1589,6 @@ class Specfile(object):
if self.config.config_opts['use_apx'] and not self.config.config_opts['use_clang']:
self._write_strip("pushd ../buildapx/" + self.config.subdir)
self._write_strip('CC=gcc-14')
self._write_strip(f'CFLAGS="$CLEAR_INTERMEDIATE_CFLAGS {APX_CFLAGS} {APX_LFLAGS} "')
self._write_strip(f'CXXFLAGS="$CLEAR_INTERMEDIATE_CXXFLAGS {AVX2_CFLAGS} {AVX2_LFLAGS} "')
self._write_strip(f'FFLAGS="$CLEAR_INTERMEDIATE_FFLAGS {APX_CFLAGS} {APX_LFLAGS} "')
@@ -1514,49 +1626,52 @@ class Specfile(object):
self._write_strip("\n")
self._write_strip("mkdir -p ~/.R")
self._write_strip(f"echo \"CFLAGS = $CFLAGS {AVX2_CFLAGS} -ftree-vectorize -mno-vzeroupper\" > ~/.R/Makevars")
self._write_strip(f"echo \"FFLAGS = $FFLAGS {AVX2_CFLAGS} -ftree-vectorize -mno-vzeroupper \" >> ~/.R/Makevars")
self._write_strip(f"echo \"CXXFLAGS = $CXXFLAGS {AVX2_CFLAGS} -ftree-vectorize -mno-vzeroupper \" >> ~/.R/Makevars")
if self.config.config_opts['use_avx2']:
self._write_strip(f"echo \"CFLAGS = $CFLAGS {AVX2_CFLAGS} -ftree-vectorize -mno-vzeroupper\" > ~/.R/Makevars")
self._write_strip(f"echo \"FFLAGS = $FFLAGS {AVX2_CFLAGS} -ftree-vectorize -mno-vzeroupper \" >> ~/.R/Makevars")
self._write_strip(f"echo \"CXXFLAGS = $CXXFLAGS {AVX2_CFLAGS} -ftree-vectorize -mno-vzeroupper \" >> ~/.R/Makevars")
self._write_strip("R CMD INSTALL "
f"{self.config.extra_configure} "
"--install-tests "
"--use-LTO "
"--built-timestamp=${SOURCE_DATE_EPOCH} "
"--data-compress=none "
"--compress=none "
"--build -l "
"%{buildroot}-v3/usr/lib64/R/library .")
self._write_strip("R CMD INSTALL "
f"{self.config.extra_configure} "
"--install-tests "
"--use-LTO "
"--built-timestamp=${SOURCE_DATE_EPOCH} "
"--data-compress=none "
"--compress=none "
"--build -l "
"%{buildroot}-v3/usr/lib64/R/library .")
self._write_strip(f"echo \"CFLAGS = $CFLAGS {AVX512_CFLAGS} -ftree-vectorize -mno-vzeroupper \" > ~/.R/Makevars")
self._write_strip(f"echo \"FFLAGS = $FFLAGS {AVX512_CFLAGS} -ftree-vectorize -mno-vzeroupper \" >> ~/.R/Makevars")
self._write_strip(f"echo \"CXXFLAGS = $CXXFLAGS {AVX512_CFLAGS} -ftree-vectorize -mno-vzeroupper \" >> ~/.R/Makevars")
if self.config.config_opts['use_avx512']:
self._write_strip(f"echo \"CFLAGS = $CFLAGS {AVX512_CFLAGS} -ftree-vectorize -mno-vzeroupper \" > ~/.R/Makevars")
self._write_strip(f"echo \"FFLAGS = $FFLAGS {AVX512_CFLAGS} -ftree-vectorize -mno-vzeroupper \" >> ~/.R/Makevars")
self._write_strip(f"echo \"CXXFLAGS = $CXXFLAGS {AVX512_CFLAGS} -ftree-vectorize -mno-vzeroupper \" >> ~/.R/Makevars")
self._write_strip("R CMD INSTALL "
"--preclean "
f"{self.config.extra_configure} "
"--install-tests "
"--use-LTO "
"--no-test-load "
"--data-compress=none "
"--compress=none "
"--built-timestamp=${SOURCE_DATE_EPOCH} "
"--build -l "
"%{buildroot}-v4/usr/lib64/R/library .")
self._write_strip("R CMD INSTALL "
"--preclean "
f"{self.config.extra_configure} "
"--install-tests "
"--use-LTO "
"--no-test-load "
"--data-compress=none "
"--compress=none "
"--built-timestamp=${SOURCE_DATE_EPOCH} "
"--build -l "
"%{buildroot}-v4/usr/lib64/R/library .")
self._write_strip(f"echo \"CFLAGS = $CFLAGS {APX_CFLAGS} -ftree-vectorize -mno-vzeroupper\" > ~/.R/Makevars")
self._write_strip(f"echo \"FFLAGS = $FFLAGS {APX_CFLAGS} -ftree-vectorize -mno-vzeroupper \" >> ~/.R/Makevars")
self._write_strip(f"echo \"CXXFLAGS = $CXXFLAGS {AVX2_CFLAGS} -ftree-vectorize -mno-vzeroupper \" >> ~/.R/Makevars")
if self.config.config_opts['use_apx']:
self._write_strip(f"echo \"CFLAGS = $CFLAGS {APX_CFLAGS} -ftree-vectorize -mno-vzeroupper\" > ~/.R/Makevars")
self._write_strip(f"echo \"FFLAGS = $FFLAGS {APX_CFLAGS} -ftree-vectorize -mno-vzeroupper \" >> ~/.R/Makevars")
self._write_strip(f"echo \"CXXFLAGS = $CXXFLAGS {AVX2_CFLAGS} -ftree-vectorize -mno-vzeroupper \" >> ~/.R/Makevars")
self._write_strip("R CMD INSTALL "
f"{self.config.extra_configure} "
"--install-tests "
"--use-LTO "
"--built-timestamp=${SOURCE_DATE_EPOCH} "
"--data-compress=none "
"--compress=none "
"--build -l "
"%{buildroot}-va/usr/lib64/R/library .")
self._write_strip("R CMD INSTALL "
f"{self.config.extra_configure} "
"--install-tests "
"--use-LTO "
"--built-timestamp=${SOURCE_DATE_EPOCH} "
"--data-compress=none "
"--compress=none "
"--build -l "
"%{buildroot}-va/usr/lib64/R/library .")
self._write_strip("echo \"CFLAGS = $CFLAGS -ftree-vectorize \" > ~/.R/Makevars")
self._write_strip("echo \"FFLAGS = $FFLAGS -ftree-vectorize \" >> ~/.R/Makevars")
@@ -1588,54 +1703,71 @@ class Specfile(object):
self._write_strip("mkdir -p clr-build")
self._write_strip("pushd clr-build")
self.write_variables()
self._write_strip("%cmake {} {}".format(self.config.cmake_srcdir, self.extra_cmake))
self._write_strip("export GOAMD64=v2")
if self.config.config_opts['use_ninja']:
cmake_type = "-G Ninja"
else:
cmake_type = "-G 'Unix Makefiles'"
self._write_strip(f"%cmake {self.config.cmake_srcdir} {self.extra_cmake} {cmake_type}")
self.write_profile_payload("cmake")
self.write_make_line()
self._write_strip("popd")
if self.config.subdir:
self._write_strip("popd")
if self.config.config_opts['use_avx2']:
self._write_strip("pushd ../buildavx2/" + self.config.subdir)
self._write_strip("mkdir -p clr-build-avx2")
self._write_strip("pushd clr-build-avx2")
self.write_build_prepend()
self.write_variables()
self._write_strip("GOAMD64=v3")
self._write_strip(f'CFLAGS="$CLEAR_INTERMEDIATE_CFLAGS {AVX2_CFLAGS} {AVX2_LFLAGS} "')
self._write_strip(f'CXXFLAGS="$CLEAR_INTERMEDIATE_CXXFLAGS {AVX2_CFLAGS} {AVX2_LFLAGS} "')
self._write_strip(f'FFLAGS="$CLEAR_INTERMEDIATE_FFLAGS {AVX2_CFLAGS} {AVX2_LFLAGS} "')
self._write_strip(f'FCFLAGS="$CLEAR_INTERMEDIATE_FCFLAGS {AVX2_CFLAGS} "')
self._write_strip("%cmake {} {}".format(self.config.cmake_srcdir, self.extra_cmake))
self._write_strip(f"%cmake {self.config.cmake_srcdir} {self.extra_cmake} {cmake_type}")
self.write_make_line()
self._write_strip("popd")
self._write_strip("popd")
if self.config.config_opts['use_avx512']:
self._write_strip("pushd ../buildavx512/" + self.config.subdir)
self._write_strip("mkdir -p clr-build-avx512")
self._write_strip("pushd clr-build-avx512")
self.write_build_prepend()
self.write_variables()
self._write_strip("GOAMD64=v4")
self._write_strip(f'CFLAGS="$CLEAR_INTERMEDIATE_CFLAGS {AVX512_CFLAGS} {AVX512_LFLAGS} "')
self._write_strip(f'CXXFLAGS="$CLEAR_INTERMEDIATE_CXXFLAGS {AVX512_CFLAGS} {AVX512_LFLAGS} "')
self._write_strip(f'FFLAGS="$CLEAR_INTERMEDIATE_FFLAGS {AVX512_CFLAGS} {AVX512_LFLAGS} "')
self._write_strip(f'FCFLAGS="$CLEAR_INTERMEDIATE_FCFLAGS {AVX512_CFLAGS} "')
self._write_strip("%cmake {} {}".format(self.config.cmake_srcdir, self.extra_cmake))
self._write_strip(f"%cmake {self.config.cmake_srcdir} {self.extra_cmake} {cmake_type}")
self.write_make_line()
self._write_strip("popd")
self._write_strip("popd")
if self.config.config_opts['use_apx'] and not self.config.config_opts['use_clang']:
self._write_strip("pushd ../buildapx/" + self.config.subdir)
self._write_strip("mkdir -p clr-build-apx")
self._write_strip("pushd clr-build-apx")
self.write_build_prepend()
self.write_variables()
self._write_strip('CC=gcc-14')
self._write_strip("GOAMD64=v3")
self._write_strip(f'CFLAGS="$CLEAR_INTERMEDIATE_CFLAGS {APX_CFLAGS} {APX_LFLAGS} "')
self._write_strip(f'CXXFLAGS="$CLEAR_INTERMEDIATE_CXXFLAGS {AVX2_CFLAGS} {AVX2_LFLAGS} "')
self._write_strip(f'FFLAGS="$CLEAR_INTERMEDIATE_FFLAGS {APX_CFLAGS} {APX_LFLAGS} "')
self._write_strip(f'FCFLAGS="$CLEAR_INTERMEDIATE_FCFLAGS {APX_CFLAGS} "')
self._write_strip("%cmake {} {}".format(self.config.cmake_srcdir, self.extra_cmake))
self._write_strip(f"%cmake {self.config.cmake_srcdir} {self.extra_cmake} {cmake_type}")
self.write_make_line()
self._write_strip("popd")
self._write_strip("popd")
if self.config.config_opts['32bit']:
self._write_strip("pushd ../build32/" + self.config.subdir)
self._write_strip("mkdir -p clr-build32")
self._write_strip("pushd clr-build32")
self.write_build_prepend()
@@ -1644,18 +1776,21 @@ class Specfile(object):
self._write_strip("%cmake -DLIB_INSTALL_DIR:PATH=/usr/lib32 "
"-DCMAKE_INSTALL_LIBDIR=/usr/lib32 "
"-DLIB_SUFFIX=32 "
"{} {} ".format(self.config.cmake_srcdir, self.extra_cmake))
f"{self.config.cmake_srcdir} {self.extra_cmake} {cmake_type}")
self.write_make_line()
self._write_strip("unset PKG_CONFIG_PATH")
self._write_strip("popd")
self._write_strip("popd")
if self.config.config_opts['openmpi']:
self._write_strip("pushd ../build-openmpi/" + self.config.subdir)
self._write_strip("mkdir -p clr-build-openmpi")
self._write_strip("pushd clr-build-openmpi")
self._write_strip(". /usr/share/defaults/etc/profile.d/modules.sh")
self._write_strip("module load openmpi")
self.write_build_prepend()
self.write_variables()
self._write_strip("GOAMD64=v3")
self._write_strip(f'CFLAGS="$CLEAR_INTERMEDIATE_CFLAGS {AVX2_CFLAGS} {AVX2_LFLAGS} "')
self._write_strip(f'CXXFLAGS="$CLEAR_INTERMEDIATE_CXXFLAGS {AVX2_CFLAGS} {AVX2_LFLAGS} "')
self._write_strip(f'FFLAGS="$CLEAR_INTERMEDIATE_FFLAGS {AVX2_CFLAGS} {AVX2_LFLAGS} "')
@@ -1664,8 +1799,6 @@ class Specfile(object):
self.write_make_line()
self._write_strip("module unload openmpi")
self._write_strip("popd")
if self.config.subdir:
self._write_strip("popd")
self._write_strip("\n")
@@ -1693,7 +1826,21 @@ class Specfile(object):
if self.config.subdir:
self._write_strip("pushd " + self.config.subdir)
self._write_strip("%qmake {} {}".format(extra_qmake_args, self.config.extra_configure))
self._write_strip('export QMAKE_CFLAGS="$CFLAGS"')
self._write_strip('export QMAKE_CXXFLAGS="$CXXFLAGS"')
self._write_strip('export QMAKE_LFLAGS="$LDFLAGS"')
self._write_strip('export QMAKE_LIBDIR=/usr/lib64')
self._write_strip('export QMAKE_CFLAGS_RELEASE=')
self._write_strip('export QMAKE_CXXFLAGS_RELEASE=')
# Add the qt6base tools to the path
self._write_strip('export PATH=/usr/lib64/qt6/bin:$PATH')
if self.config.make_command:
qmake = self.config.make_command
else:
qmake = "qmake6"
self._write_strip(f"{qmake} {extra_qmake_args} {self.config.extra_configure}")
self._write_strip("test -r config.log && cat config.log")
self.write_make_line()
@@ -1702,7 +1849,7 @@ class Specfile(object):
if self.config.config_opts['use_avx2']:
self._write_strip("pushd ../buildavx2/" + self.config.subdir)
self._write("%qmake 'QT_CPU_FEATURES.x86_64 += avx avx2 bmi bmi2 f16c fma lzcnt popcnt'\\\n")
self._write(f"{qmake} 'QT_CPU_FEATURES.x86_64 += avx avx2 bmi bmi2 f16c fma lzcnt popcnt'\\\n")
self._write(f' QMAKE_CFLAGS+="{AVX2_CFLAGS} {AVX2_LFLAGS}" QMAKE_CXXFLAGS+="{AVX2_CFLAGS} {AVX2_LFLAGS}" \\\n')
self._write(f' QMAKE_LFLAGS+="{AVX2_LCFLAGS}" {extra_qmake_args} {self.config.extra_configure}\n')
self.write_make_line()
@@ -1767,11 +1914,13 @@ class Specfile(object):
self.write_variables()
if self.config.subdir:
self._write_strip("pushd " + self.config.subdir)
self._write_strip("export GOAMD64=v2")
self._write_strip('meson --libdir=lib64 --prefix=/usr --buildtype=plain {0} {1} builddir'
.format(self.config.extra_configure,
self.config.extra_configure64))
self._write_strip("ninja -v -C builddir")
if self.config.config_opts['use_avx2']:
self._write_strip("GOAMD64=v3")
if self.config.config_opts['pgo'] and self.config.profile_payload != "":
self._write_strip(f'CFLAGS="$CFLAGS_GENERATE {AVX2_CFLAGS} {AVX2_LFLAGS} " CXXFLAGS="$CXXFLAGS_GENERATE '
f'{AVX2_CFLAGS} {AVX2_LFLAGS} " LDFLAGS="$LDFLAGS_GENERATE {AVX2_LCFLAGS} " '
@@ -1794,6 +1943,7 @@ class Specfile(object):
'{1} builddiravx2'.format(self.config.extra_configure, self.config.extra_configure64))
self._write_strip('ninja -v -C builddiravx2')
if self.config.config_opts['use_avx512']:
self._write_strip("GOAMD64=v4")
if self.config.config_opts['pgo'] and self.config.profile_payload != "":
self._write_strip(f'CFLAGS="$CFLAGS_GENERATE {AVX512_CFLAGS} {AVX512_LFLAGS} " CXXFLAGS="$CXXFLAGS_GENERATE '
f'{AVX512_CFLAGS} {AVX512_LFLAGS} " LDFLAGS="$LDFLAGS_GENERATE {AVX512_LCFLAGS} " '
@@ -1816,7 +1966,7 @@ class Specfile(object):
'{1} builddiravx512'.format(self.config.extra_configure, self.config.extra_configure64))
self._write_strip('ninja -v -C builddiravx512')
if self.config.config_opts['use_apx'] and not self.config.config_opts['use_clang']:
self._write_strip('CC=gcc-14')
self._write_strip("GOAMD64=v3")
if self.config.config_opts['pgo'] and self.config.profile_payload != "":
self._write_strip(f'CFLAGS="$CFLAGS_GENERATE {APX_CFLAGS} {APX_LFLAGS} "'
f' CXXFLAGS="$CXXFLAGS_GENERATE {AVX2_CFLAGS} {AVX2_LFLAGS} "'
@@ -1859,6 +2009,7 @@ class Specfile(object):
self._write_strip("%install")
self.write_variables()
self.write_install_prepend()
self._write_strip("export GOAMD64=v2")
self.write_license_files()
if self.config.config_opts['32bit']:
self._write_strip('pushd ../build32/' + self.config.subdir)
@@ -1879,13 +2030,16 @@ class Specfile(object):
if self.config.subdir:
self._write_strip("pushd " + self.config.subdir)
if self.config.config_opts['use_avx2']:
self._write_strip("GOAMD64=v3")
self._write_strip('DESTDIR=%{buildroot}-v3 ninja -C builddiravx2 install')
if self.config.config_opts['use_avx512']:
self._write_strip("GOAMD64=v4")
self._write_strip('DESTDIR=%{buildroot}-v4 ninja -C builddiravx512 install')
if self.config.config_opts['use_apx'] and not self.config.config_opts['use_clang']:
self._write_strip('CC=gcc-14')
self._write_strip("GOAMD64=v3")
self._write_strip('DESTDIR=%{buildroot}-va ninja -C builddirapx install')
self._write_strip("GOAMD64=v2")
self._write_strip("DESTDIR=%{buildroot} ninja -C builddir install")
if self.config.subdir:
self._write_strip("popd")
@@ -1916,14 +2070,11 @@ class Specfile(object):
self.write_license_files()
if self.config.subdir:
self._write_strip("pushd " + self.config.subdir)
if self.config.install_macro:
self._write_strip(self.config.install_macro)
else:
self._write_strip("cargo install --path .")
self._write_strip("mkdir -p %{buildroot}/usr/bin")
self._write_strip('pushd "${HOME}/.cargo/bin/"')
self._write_strip("mv * %{buildroot}/usr/bin/")
self._write_strip("popd")
self._write_strip("cargo install --path .")
self._write_strip("mkdir -p %{buildroot}/usr/bin")
self._write_strip('pushd "${HOME}/.cargo/bin/"')
self._write_strip("mv * %{buildroot}/usr/bin/")
self._write_strip("popd")
if self.config.subdir:
self._write_strip("popd")
self.write_install_append()
+42 -5
View File
@@ -17,6 +17,7 @@
# along with this program. If not, see <http://www.gnu.org/licenses/>.
#
import bz2
import configparser
import os
import re
@@ -25,6 +26,7 @@ import tarfile
import zipfile
import download
import zstandard as zstd
from util import do_regex, get_sha1sum, print_fatal, write_out
@@ -50,6 +52,10 @@ class Source():
"""Determine compression type."""
if self.url.lower().endswith(('.zip', 'jar')):
self.type = 'zip'
elif self.url.lower().endswith(('.bz2')) and not self.url.lower().endswith(('.tar.bz2')):
self.type = 'bz2'
elif self.url.lower().endswith('.zst'):
self.type = 'zst'
else:
self.type = 'tar'
@@ -78,6 +84,19 @@ class Source():
print_fatal("Not a valid tar file.")
sys.exit(1)
def set_zst_prefix(self):
"""Determine prefix folder name of tar.zst file."""
with tarfile.open(fileobj=zstd.open(self.path, 'rb'), mode='r|') as content:
lines = content.getnames()
if len(lines) == 0:
print_fatal("Zstd compressed tar file doesn't appear to have any content")
sys.exit(1)
elif len(lines) > 1:
self.prefix = os.path.commonpath(lines)
def set_bz2_prefix(self):
"""No prefix for plain bz2 archives."""
def set_zip_prefix(self):
"""Determine prefix folder name of zip file."""
if zipfile.is_zipfile(self.path):
@@ -101,18 +120,35 @@ class Source():
extraction_path = base_path
extract_method = getattr(self, 'extract_{}'.format(self.type))
extract_method(extraction_path)
try:
extract_method(extraction_path)
except tarfile.AbsoluteLinkError:
pass
def extract_tar(self, extraction_path):
"""Extract tar in path."""
with tarfile.open(self.path) as content:
content.extractall(path=extraction_path)
content.extractall(path=extraction_path, filter='data')
def extract_bz2(self, extraction_path):
"""Extract plain bz2 file in path."""
with bz2.BZ2File(self.path, 'rb') as content:
data = content.read()
newfile = self.path.rsplit('/', 1)[1]
newfile = newfile.rsplit('.bz2', 1)[0]
with open(os.path.join(extraction_path), mode='wb') as f:
f.write(data)
def extract_zip(self, extraction_path):
"""Extract zip in path."""
with zipfile.ZipFile(self.path, 'r') as content:
content.extractall(path=extraction_path)
def extract_zst(self, extraction_path):
"""Extract zst in path."""
with tarfile.open(fileobj=zstd.open(self.path, 'rb'), mode='r|') as content:
content.extractall(path=extraction_path, filter='data')
def convert_version(ver_str, name):
"""Remove disallowed characters from the version."""
@@ -120,7 +156,7 @@ def convert_version(ver_str, name):
# them out with expensive regular expressions
banned_subs = ["x86.64", "source", "src", "all", "bin", "release", "rh",
"ga", ".ce", "lcms", "onig", "linux", "gc", "sdk", "orig",
"jurko", "%2f", "%2F", "%20"]
"jurko", "%2f", "%2F", "%20", "x265", "autotools"]
# package names may be modified in the version string by adding "lib" for
# example. Remove these from the name before trying to remove the name from
@@ -136,7 +172,7 @@ def convert_version(ver_str, name):
ver_str = ver_str.replace(name.replace(mod, ""), "")
# replace illegal characters
ver_str = ver_str.strip().replace('-', '.').replace('_', '.')
ver_str = ver_str.strip().replace('-', '.').replace('_', '.').replace('+', '.')
# remove banned substrings
for sub in banned_subs:
@@ -262,7 +298,8 @@ class Content():
if "github.com" in self.url:
# define regex accepted for valid packages, important for specific
# patterns to come before general ones
github_patterns = [r"https?://github.com/(.*)/(.*?)/archive/refs/tags/[vVrR]?(.*)\.tar",
github_patterns = [r"https?://github.com/(.*)/(.*?)/archive/refs/tags/.*/(.*).tar",
r"https?://github.com/(.*)/(.*?)/archive/refs/tags/[vVrR]?(.*)\.tar",
r"https?://github.com/(.*)/(.*?)/archive/[v|r]?.*/(.*).tar",
r"https?://github.com/(.*)/(.*?)/archive/[-a-zA-Z_]*-(.*).tar",
r"https?://github.com/(.*)/(.*?)/archive/[vVrR]?(.*).tar",
+136
View File
@@ -27,6 +27,101 @@ import sys
dictionary_filename = os.path.dirname(__file__) + "/translate.dic"
dictionary = [line.strip() for line in open(dictionary_filename, 'r')]
os_paths = None
ERROR_FILE = 'pumpAutospec'
ERROR_ENV = 'AUTOSPEC_UPDATE'
def _log_error(error):
write_out(ERROR_FILE, f"{error}\n", mode='a')
def _commit_result():
if not os.path.isfile(ERROR_FILE):
return
call(f"git add {ERROR_FILE}", check=False, stderr=subprocess.DEVNULL)
call(f"git commit {ERROR_FILE} -m 'Notes update'", check=False, stderr=subprocess.DEVNULL)
call("git push", check=False, stderr=subprocess.DEVNULL)
def _process_line(line, prev_line, current_patch, reported_patches, error):
if m := re.match('^Patch #[0-9]+ .(?P<patch>.*).:', line):
current_patch[0] = m.group('patch')
if m := re.match('Hunk #[0-9]+ FAILED at [0-9]+', line):
if current_patch[0] not in reported_patches:
_log_error("Patch " + current_patch[0] + " does not apply")
reported_patches[current_patch[0]] = True
return True
if m := re.match(".*can't find file to patch at input line ", line):
if current_patch[0] not in reported_patches:
_log_error("Patch " + current_patch[0] + " does not apply")
reported_patches[current_patch[0]] = True
return True
if m := re.match('.*meson.build:[0-9]+:[0-9]+: ERROR: Unknown options: "(?P<option>.*)"', line):
_log_error("Unknown meson option: " + m.group('option'))
return True
if m := re.match('Error: package (?P<module>.*) .* was found, but', line):
_log_error("R package " + m.group('module') + " not found")
return True
if m := re.match('.*CMake Error at .*/CMake', prev_line):
if m := re.match('(?P<module>.*) not found', line):
_log_error("CMake module " + m.group('module') + "not found")
return True
if m := re.match(r'go: download.*connect: connection refused', line):
_log_error("Go online update")
return True
if 'Updating crates.io index' in line:
_log_error("Rust crates.io online update")
return True
if "error: '__builtin_ctzs' needs isa option -mbmi" in line:
_log_error(" error: '__builtin_ctzs' needs isa option -mbmi")
return True
if "error:" in line and 'Bad exit status from' not in line:
m = re.match('.*error:(?P<error>.*)', line)
if m and not error:
_log_error("Compiler: " + m.group('error'))
return True
if m := re.match(r'Could NOT find (?P<package>.*) .missing', line):
_log_error("CMake module " + m.group('package') + " not found")
return True
if m := re.match(r'Could not find a package configuration file provided by (?P<package>.*) with', line):
_log_error("CMake module " + m.group('package') + " not found")
return True
# Unable to find program 'gperf'
if m := re.match(r"Failed to find program (?P<module>.*)", line):
_log_error("Failed to find " + m.group('module'))
return True
if m := re.match(r"Failed to find (?P<module>.*)", line):
_log_error("Failed to find " + m.group('module'))
return True
return False
def _process_build_log(filename):
with open_auto(filename, "r") as lfile:
lines = lfile.readlines()
prev_line = ''
current_patch = ['']
reported_patches = {}
error = False
for line in lines:
if _process_line(line, prev_line, current_patch, reported_patches, error):
error = True
prev_line = line
if error:
_commit_result()
def call(command, logfile=None, check=True, **kwargs):
@@ -116,9 +211,22 @@ def print_error(message):
_print_message(message, 'ERROR', 'red')
def print_build_failed():
"""Print final fatal error, color coded for TTYs."""
_print_message('Build failed, aborting', 'FATAL', 'red')
try:
if os.environ.get(ERROR_ENV):
_process_build_log('results/build.log')
except Exception:
pass
def print_fatal(message):
"""Print fatal error, color coded for TTYs."""
_print_message(message, 'FATAL', 'red')
if os.environ.get(ERROR_ENV):
write_out(ERROR_FILE, f"{message}\n", mode='a')
_commit_result()
def print_warning(message):
@@ -166,3 +274,31 @@ def open_auto(*args, **kwargs):
assert 'encoding' not in kwargs
assert 'errors' not in kwargs
return open(*args, encoding="utf-8", errors="surrogateescape", **kwargs)
def globlike_match(filename, match_name):
"""Compare the filename to the match_name in a way that simulates the shell glob '*'."""
fsplit = filename.split('/')
if len(fsplit) != len(match_name):
return False
match = True
for fpart, mpart in zip(fsplit, match_name):
if fpart != mpart:
if '*' not in mpart:
match = False
break
if len(mpart) > len(fpart) + 1:
match = False
break
mpl, mpr = mpart.split('*')
try:
if fpart.index(mpl) != 0:
match = False
break
if fpart.rindex(mpr) != len(fpart) - len(mpr):
match = False
break
except ValueError:
match = False
break
return match
+2
View File
@@ -1612,3 +1612,5 @@ http://sourceforge.net/projects/zsh/files/zsh/5.4.2/zsh-5.4.2.tar.gz,zsh,5.4.2
https://pigeonhole.dovecot.org/releases/2.3/dovecot-2.3.11-pigeonhole-0.5.11.tar.gz,pigeonhole,0.5.11
https://pigeonhole.dovecot.org/releases/2.3/dovecot-2.3-pigeonhole-0.5.20.tar.gz,pigeonhole,0.5.20
https://www.ezix.org/software/files/lshw-B.02.19.2.tar.gz,lshw,02.19.2
https://github.com/VectorCamp/vectorscan/archive/refs/tags/vectorscan/5.4.11.tar.gz,vectorscan,5.4.11
https://lib.openmpt.org/files/libopenmpt/src/libopenmpt-0.7.13+release.autotools.tar.gz,libopenmpt,0.7.13
+19 -9
View File
@@ -190,25 +190,36 @@ class TestBuildpattern(unittest.TestCase):
def mock_util_call(cmd):
del cmd
def mock_wrap_fatal():
output = ['']
def mock_print_fatal(msg):
output[0] = msg
return output, mock_print_fatal
result, mock_print_fatal = mock_wrap_fatal()
exit_backup = build.sys.exit
call_backup = build.util.call
print_backup = build.util.print_fatal
build.sys.exit = MagicMock()
build.util.call = mock_util_call
build.util.print_fatal = mock_print_fatal
open_name = 'build.util.open_auto'
content = "line1\nDEBUG util.py:399: No matching package to install: 'foobar'\nDEBUG util.py:399: No matching package to install: 'foobarbaz'\nline 4"
m_open = mock_open(read_data=content)
pkg = build.Build()
pkg.must_restart = 1
pkg.file_restart = 1
result = True
with patch(open_name, m_open, create=True):
result = pkg.parse_buildroot_log('testname', 1)
pkg.parse_buildroot_log('testname', 1)
build.util.print_fatal = print_backup
build.util.call = call_backup
mock_exit = build.sys.exit
build.sys.exit = exit_backup
self.assertFalse(result)
self.assertEqual(pkg.must_restart, 0)
self.assertEqual(pkg.file_restart, 0)
self.assertEqual(result[0], 'Cannot resolve dependency name: foobar\nCannot resolve dependency name: foobarbaz')
mock_exit.assert_called_once()
def test_parse_buildroot_log_pass(self):
"""
@@ -227,11 +238,10 @@ class TestBuildpattern(unittest.TestCase):
result = True
with patch(open_name, m_open, create=True):
result = pkg.parse_buildroot_log('testname', 1)
pkg.parse_buildroot_log('testname', 1)
build.util.call = call_backup
self.assertTrue(result)
self.assertEqual(pkg.must_restart, 0)
def test_parse_buildroot_log_noop(self):
+48 -1
View File
@@ -402,6 +402,7 @@ class TestBuildreq(unittest.TestCase):
should be sufficient.
"""
conf = config.Config("")
conf.config_opts['use_ninja'] = False
with tempfile.TemporaryDirectory() as tmpd:
os.mkdir(os.path.join(tmpd, 'subdir'))
open(os.path.join(tmpd, 'setup.py'), 'w').close()
@@ -418,6 +419,7 @@ class TestBuildreq(unittest.TestCase):
should be sufficient.
"""
conf = config.Config("")
conf.config_opts['use_ninja'] = False
with tempfile.TemporaryDirectory() as tmpd:
os.mkdir(os.path.join(tmpd, 'subdir'))
open(os.path.join(tmpd, 'CMakeLists.txt'), 'w').close()
@@ -434,6 +436,7 @@ class TestBuildreq(unittest.TestCase):
should be sufficient.
"""
conf = config.Config("")
conf.config_opts['use_ninja'] = False
with tempfile.TemporaryDirectory() as tmpd:
os.mkdir(os.path.join(tmpd, 'subdir'))
open(os.path.join(tmpd, 'SConstruct'), 'w').close()
@@ -450,6 +453,7 @@ class TestBuildreq(unittest.TestCase):
should be sufficient.
"""
conf = config.Config("")
conf.config_opts['use_ninja'] = False
with tempfile.TemporaryDirectory() as tmpd:
os.mkdir(os.path.join(tmpd, 'subdir'))
open(os.path.join(tmpd, 'meson.build'), 'w').close()
@@ -480,6 +484,7 @@ class TestBuildreq(unittest.TestCase):
buildreq.pypidata.get_pypi_metadata = MagicMock(return_value=content)
with tempfile.TemporaryDirectory() as tmpd:
conf = config.Config(tmpd)
conf.config_opts['use_ninja'] = False
os.mkdir(os.path.join(tmpd, 'subdir'))
open(os.path.join(tmpd, 'subdir', 'pyproject.toml'), 'w').close()
self.reqs.scan_for_configure(os.path.join(tmpd, 'subdir'), "", conf)
@@ -512,6 +517,7 @@ class TestBuildreq(unittest.TestCase):
m_open = mock_open(read_data=content)
with tempfile.TemporaryDirectory() as tmpd:
conf = config.Config(tmpd)
conf.config_opts['use_ninja'] = False
os.mkdir(os.path.join(tmpd, 'subdir'))
open(os.path.join(tmpd, 'subdir', 'pyproject.toml'), 'w').close()
open(os.path.join(tmpd, 'pypi.json'), 'w').close()
@@ -537,6 +543,7 @@ class TestBuildreq(unittest.TestCase):
with tempfile.TemporaryDirectory() as tmpd:
conf = config.Config(tmpd)
conf.config_opts['use_ninja'] = False
os.mkdir(os.path.join(tmpd, 'subdir'))
open(os.path.join(tmpd, 'subdir', 'setup.py'), 'w').close()
open(os.path.join(tmpd, 'subdir', 'requires.txt'), 'w').close()
@@ -546,6 +553,21 @@ class TestBuildreq(unittest.TestCase):
self.reqs.add_setup_py_requires.assert_called_once()
self.reqs.grab_python_requirements.assert_called_once()
def test_scan_for_configure_ninja(self):
"""
Test scan_for_configure when ninja is enabled.
"""
conf = config.Config("")
conf.config_opts['use_ninja'] = True
with tempfile.TemporaryDirectory() as tmpd:
os.mkdir(os.path.join(tmpd, 'subdir'))
open(os.path.join(tmpd, 'setup.py'), 'w').close()
self.reqs.scan_for_configure(tmpd, "", conf)
self.assertEqual(self.reqs.buildreqs,
set(['buildreq-distutils3', 'ninja']))
def test_parse_cmake_pkg_check_modules(self):
"""
Test parse_cmake to ensure accurate detection of versioned and
@@ -621,12 +643,30 @@ class TestBuildreq(unittest.TestCase):
"valid": "valid",
"valid_but_commented": "valid_but_commented",
"different_name": "another_name",
"qt6.module1": "qt6module1",
"qt6.module2": "qt6module2",
"kf6.module3": "kf6module3",
"kf6.module4": "kf6module4",
".module5": "namodule5",
".module6": "namodule6"
}
content = '''
find_package(valid)
#find_package(foo)
# find_package(valid_but_commented)
find_package(different_name)
find_package(Qt6 stuff
module1
module2)
find_package(KF6 stuff
module3
module4
)
find_package(NOT_HANDLED_NAMESPACE stuff
module5
module6
)
'''
with tempfile.TemporaryDirectory() as tmpd:
with open(os.path.join(tmpd, 'fname'), 'w') as f:
@@ -634,7 +674,14 @@ find_package(different_name)
self.reqs.parse_cmake(os.path.join(tmpd, 'fname'), cmake_modules, False)
self.assertEqual(self.reqs.buildreqs,
set(['valid', 'another_name']))
set(['valid',
'another_name',
'qt6module1',
'qt6module2',
'kf6module3',
'kf6module4',
'namodule5',
'namodule6']))
def test_r_desc_field_begin(self):
"""Test parsing of the first R description field."""
-20
View File
@@ -152,26 +152,6 @@ class TestTest(unittest.TestCase):
check.os.listdir = listdir_backup
self.assertEqual(check.tests_config, 'make TEST_VERBOSE=1 test')
def test_scan_for_tests_setup(self):
"""
Test scan_for_tests with setup.py suite
"""
reqs = buildreq.Requirements("")
conf = config.Config("")
tcontent = tarball.Content("", "", "", [], conf, "")
listdir_backup = os.listdir
check.os.listdir = mock_generator(['setup.py'])
content = 'test_suite'
m_open = mock_open(read_data=content)
with patch(self.open_name, m_open, create=True):
conf.default_pattern = "distutils3"
check.scan_for_tests('pkgdir', conf, reqs, tcontent)
check.os.listdir = listdir_backup
self.assertEqual(check.tests_config,
'PYTHONPATH=%{buildroot}$(python -c "import sys; print(sys.path[-1])") '
'python setup.py test')
def test_scan_for_tests_cmake(self):
"""
Test scan_for_tests with cmake suite
+1
View File
@@ -6,6 +6,7 @@ import config
# Structure: (url, build_pattern)
BUILD_PAT_URL = [
("https://cran.r-project.org/src/contrib/raster_3.0-12.tar.gz", "R"),
("https://ftp.osuosl.org/pub/cran/src/contrib/hexbin_1.28.5.tar.gz", "R"),
("http://pypi.debian.net/argparse/argparse-1.4.0.tar.gz", "distutils3"),
("https://pypi.python.org/packages/source/T/Tempita/Tempita-0.5.2.tar.gz", "distutils3"),
("https://cpan.metacpan.org/authors/id/T/TO/TODDR/IO-Tty-1.14.tar.gz", "cpan"),
+57 -1
View File
@@ -71,7 +71,7 @@ class TestLicense(unittest.TestCase):
conf = config.Config("")
conf.setup_patterns()
# remove the hash from license_hashes
del(conf.license_hashes[license.get_sha1sum('tests/COPYING_TEST')])
del(conf.license_hashes[license.util.get_sha1sum('tests/COPYING_TEST')])
conf.license_show = "license.show.url"
license.license_from_copying_hash('tests/COPYING_TEST', '', conf, '')
@@ -233,6 +233,62 @@ class TestLicense(unittest.TestCase):
self.assertIn("Cannot find any license", out.getvalue())
self.assertEqual(license.licenses, [])
def test_scan_for_licenses_skip(self):
"""
Test scan_for_licenses in temporary directory with licenses to skip
"""
conf = config.Config("")
conf.setup_patterns()
conf.license_skips = [['COPYING']]
with open('tests/COPYING_TEST', 'rb') as copyingf:
content = copyingf.read()
with tempfile.TemporaryDirectory() as tmpd:
# create the copying file
with open(os.path.join(tmpd, 'COPYING'), 'w') as newcopyingf:
newcopyingf.write(content.decode('utf-8'))
# create some cruft for testing
for testf in ['testlib.c', 'testmain.c', 'testheader.h']:
with open(os.path.join(tmpd, testf), 'w') as newtestf:
newtestf.write('test content')
# let's check that the proper thing is being printed as well
out = StringIO()
with redirect_stdout(out):
with self.assertRaises(SystemExit) as thread:
license.scan_for_licenses(tmpd, conf, '')
self.assertEqual(thread.exception.code, 1)
self.assertIn("Cannot find any license", out.getvalue())
self.assertEqual(license.licenses, [])
def test_scan_for_licenses_skip_prefix_slash(self):
"""
Test scan_for_licenses in temporary directory with licenses to skip
"""
conf = config.Config("")
conf.setup_patterns()
conf.license_skips = [['', 'COPYING']]
with open('tests/COPYING_TEST', 'rb') as copyingf:
content = copyingf.read()
with tempfile.TemporaryDirectory() as tmpd:
# create the copying file
with open(os.path.join(tmpd, 'COPYING'), 'w') as newcopyingf:
newcopyingf.write(content.decode('utf-8'))
# create some cruft for testing
for testf in ['testlib.c', 'testmain.c', 'testheader.h']:
with open(os.path.join(tmpd, testf), 'w') as newtestf:
newtestf.write('test content')
# let's check that the proper thing is being printed as well
out = StringIO()
with redirect_stdout(out):
with self.assertRaises(SystemExit) as thread:
license.scan_for_licenses(tmpd, conf, '')
self.assertEqual(thread.exception.code, 1)
self.assertIn("Cannot find any license", out.getvalue())
self.assertEqual(license.licenses, [])
def test_load_specfile(self):
"""
Test load_specfile with populated license list. This method is not
+1 -14
View File
@@ -47,7 +47,7 @@ class TestSpecfileWrite(unittest.TestCase):
self.WRITES = self.WRITES[:4] + self.WRITES[6:]
self.assertEqual(expect, self.WRITES)
def test_write_nvr_no_urlban(self):
def test_write_nvr(self):
"""
test Specfile.write_nvr with no urlban set
"""
@@ -59,19 +59,6 @@ class TestSpecfileWrite(unittest.TestCase):
"Source0 : http://www.testpkg.com/testpkg/pkg-1.0.tar.gz\n"]
self.assertEqual(expect, self.WRITES)
def test_write_nvr_urlban(self):
"""
test Specfile.write_nvr with urlban set
"""
self.specfile.config.urlban = "www.testpkg.com"
self.specfile.write_nvr()
expect = ["Name : pkg\n",
"Version : 1.0\n",
"Release : 2\n",
"URL : http://localhost/testpkg/pkg-1.0.tar.gz\n",
"Source0 : http://localhost/testpkg/pkg-1.0.tar.gz\n"]
self.assertEqual(expect, self.WRITES)
def test_write_sources(self):
"""
test write_sources with all Specfile.sources set.
+68 -1
View File
@@ -2,7 +2,8 @@ import subprocess
import os
import tempfile
import unittest
import unittest.mock
from unittest.mock import MagicMock, mock_open, patch
import util
@@ -64,5 +65,71 @@ class TestUtil(unittest.TestCase):
self.assertTrue(util.binary_in_path('testbin'))
self.assertEqual(util.os_paths, [tmpd])
def test__process_build_log_bad_patch(self):
"""
Test _process_build_log with a bad patch
"""
def isfile_mock(_):
return True
isfile_backup = util.os.path.isfile
util.os.path.isfile = isfile_mock
call_backup = util.call
util.call = MagicMock()
open_name = 'util.open_auto'
content = "Patch #1 (bad.patch):\nHunk #1 FAILED at 1."
m_open = mock_open(read_data=content)
with patch(open_name, m_open, create=True):
util._process_build_log('filename')
util.os.path.isfile = isfile_backup
mock_call = util.call
util.call = call_backup
self.assertTrue(len(mock_call.mock_calls) == 3)
def test_globlike_match(self):
"""
Test globlike_match
"""
match_name = ['a', 'b', 'c']
file_path = 'a/b'
self.assertFalse(util.globlike_match(file_path, match_name))
match_name = ['a', 'c']
file_path = 'a/b'
self.assertFalse(util.globlike_match(file_path, match_name))
match_name = ['a', 'bb*']
file_path = 'a/b'
self.assertFalse(util.globlike_match(file_path, match_name))
match_name = ['a', 'b*']
file_path = 'a/ab'
self.assertFalse(util.globlike_match(file_path, match_name))
match_name = ['a', '*a']
file_path = 'a/ab'
self.assertFalse(util.globlike_match(file_path, match_name))
match_name = ['a', 'c*']
file_path = 'a/b'
self.assertFalse(util.globlike_match(file_path, match_name))
match_name = ['a', '*c']
file_path = 'a/b'
self.assertFalse(util.globlike_match(file_path, match_name))
match_name = ['a', 'b*']
file_path = 'a/b'
self.assertTrue(util.globlike_match(file_path, match_name))
match_name = ['a', '*b']
file_path = 'a/b'
self.assertTrue(util.globlike_match(file_path, match_name))
match_name = ['a', 'b']
file_path = 'a/b'
self.assertTrue(util.globlike_match(file_path, match_name))
if __name__ == '__main__':
unittest.main(buffer=True)