80 Commits
Author SHA1 Message Date
Robert Dower b4dbb0c4f0 archive repository 2025-08-07 13:28:13 -07:00
Arjan van de Ven 3550db8413 add libdeps.py to this git tree as well 2024-03-21 00:42:30 +00:00
William Douglas 931e37281b Update regex syntax
Python no longer accepts a '\' escape so remove it as it isn't
necessary.

Signed-off-by: William Douglas <william.douglas@intel.com>
v56
2024-02-06 13:31:21 -08:00
William Douglas 8f91ff6c0a Add APX support
Signed-off-by: William Douglas <william.douglas@intel.com>
v55
2024-01-31 16:56:15 -08:00
William Douglas 1f9391c02e Add symlinks test
Signed-off-by: William Douglas <william.douglas@intel.com>
2023-10-10 15:46:42 -07:00
William Douglas c77ec44f28 Add another file to fixup pypi deps on
Signed-off-by: William Douglas <william.douglas@intel.com>
v54
2023-09-21 09:08:21 -07:00
William Douglas 5effa819d4 Whitespace fixup
Signed-off-by: William Douglas <william.douglas@intel.com>
2023-06-08 09:09:50 -07:00
William Douglas ee142404df Do not copy symlinks with elf-move.py
Symlinks aren't intended to be different between optimization levels
so they are not needed to be installed in the optimized chroots.

Signed-off-by: William Douglas <william.douglas@intel.com>
v53
2023-05-30 12:10:47 -07:00
William Douglas e3989a4e46 Force path correction
Same files in optimized paths would be under a symlink in the standard
path so try and just correct them during the move.

Signed-off-by: William Douglas <william.douglas@intel.com>
v52
2023-05-17 18:34:25 -07:00
William Douglas 58a5bebf1f Stop moving library content off to hwcaps
The libraries being in hwcaps is going to cause extra space usage more
heavily when enabling avx2 broadly in the ecosystem. To avoid this,
use the binary replacement that's already in place for the rest of the
ecosystem instead.

Signed-off-by: William Douglas <william.douglas@intel.com>
v51
2023-04-28 08:39:22 -07:00
William Douglas b476a39f52 Rework elf-move to move content into a path prefix
The packages now are expected to have content placed in a optimized
path prefix, V3 for AVX2 and V4 for AVX512 (with additional prefixes
as new options are added). Update elf-move.py to move content in the
installdir to the targetdir under a prefix based on the btype.

Library files that are moved to the hwcaps directory are not impacted
by this change. All other files that are moved will now be found in
the targetdir with the remaining fullpath name of prefix + virtpath.

Signed-off-by: William Douglas <william.douglas@intel.com>
v50
2023-04-12 11:05:32 -07:00
William Douglas abc7f5b6d8 Fix check for elf paths with /usr/lib64 at the end
In some cases paths end with /usr/lib64 but are not exactly /usr/lib64
and elf-move was incorrectly classifying them as libraries. Fix this
by checking against the exact virt path dirname for /usr/lib64
instead.

Signed-off-by: William Douglas <william.douglas@intel.com>
v49
2023-01-24 09:12:37 -08:00
William Douglas bf39523aea Add filemap verification tool 2022-09-30 14:52:08 -07:00
William Douglas f123935135 Add original file to the filemap
The original file being missing from the filemap makes it hard to know
what all the files in the optimized-elf directory actually correspond
to. This change allows better validation and tracking to happen in
case there are issues in either the content creation or deployment
side of the operation.

Signed-off-by: William Douglas <william.r.douglas@gmail.com>
v48
2022-09-23 12:48:31 -07:00
William Douglas 5d2d9c93e5 Add testing for elf-move.py
Also add a make test (that currently only runs elf-move tests).

Signed-off-by: William Douglas <william.douglas@intel.com>
2022-08-23 13:06:28 -07:00
William Douglas 9be6f1ad5f Fixup comparison test
Make comparison more pythonic.

Signed-off-by: William Douglas <william.douglas@intel.com>
2022-08-23 13:04:08 -07:00
William Douglas 2d2f6fef13 Try and read the elf header
Accidently removed file reads to check elf headers, add code to do so
back.

Signed-off-by: William Douglas <william.douglas@intel.com>
v47
2022-08-22 15:36:19 -07:00
William Douglas eecb3f5713 Only operate on elf files
Make the content of the filemap clearly indicate if a file is an elf
or not and use that to determine if the file is added to the
optimized-elf contents.

Signed-off-by: William Douglas <william.douglas@intel.com>
v46
2022-08-22 13:33:46 -07:00
William Douglas d3c422486c Refactor work
Remove a global, fixup parser and remove some unused code.

Signed-off-by: William Douglas <william.douglas@intel.com>
v45
2022-08-22 11:22:34 -07:00
Arjan van de Ven 44e442795d key bugfixes v44 2022-08-18 18:32:32 +00:00
Arjan van de Ven 7be7096e18 fix filename goof v43 2022-08-16 14:47:02 +00:00
Arjan van de Ven f43d3f90a4 more delta friendly hash encodings v42 2022-08-12 19:16:58 +00:00
Arjan van de Ven 5582af06bd ignore errors v41 2022-08-09 17:45:18 +00:00
William Douglas e8a7688cd6 Make hardlink for original file
Save space in the rpm by making a hardlink.

Signed-off-by: William Douglas <william.douglas@intel.com>
v40
2022-08-08 15:02:28 -07:00
William Douglas 14eef6fe3d Rework copy_original placement and usage
copy_original should only run when the filemap entry is added to the
optimized_dir.

Also use the targetdir file not the installdir file (as that is the
sse version).

Signed-off-by: William Douglas <william.douglas@intel.com>
v39
2022-08-08 11:21:27 -07:00
William Douglas 7a25432aff Fixups for copy_original
Signed-off-by: William Douglas <william.douglas@intel.com>
2022-08-08 09:17:57 -07:00
Arjan van de Ven 0c25551df3 add a copy_original function 2022-08-08 16:00:26 +00:00
William Douglas 72834e1f65 Fix missing plugin content
For content that isn't directly in /usr/lib64 but is under that
path (typically plugins) it wasn't added to the optimized-elf
directory incorrectly. Resolve this by fixing the test against
/usr/lib64.

Signed-off-by: William Douglas <william.douglas@intel.com>
v38
2022-06-30 11:16:33 -07:00
William Douglas 1093a655a5 Error out if optimized-elf path is passed to targetdir
Callers previously would set the full optimized-elf path and need to
be updated. To ensure they are updated rather than build incorrectly,
check if the full path is being used for targetdir and error out.

Signed-off-by: William Douglas <william.douglas@intel.com>
v37
2022-06-27 11:17:01 -07:00
Arjan van de Ven 01d1149f4a endswidth v36 2022-06-24 16:32:21 +00:00
Arjan van de Ven ebc2ac52ba one more fix v35 v34 2022-06-24 15:46:44 +00:00
Arjan van de Ven 22f0cb1c0d typo v33 2022-06-24 14:53:43 +00:00
Arjan van de Ven 28eb559aa5 skip plugins v32 2022-06-24 14:14:36 +00:00
William Douglas 415a400edd Directly install lib types
The libraries do not need to conditionally installed as they are used
conditionally already. While this takes up additional disk space it is
preferrable for now due to swupd not handling the missing files well.

This requires a change in how targetpath is interpretted. Now the
targetpath is no longer the path given to the optimized file directory
but instead the path to the base of the target and the optimzied file
directory is now fixed ('/usr/share/clear/optimized-elf').

Signed-off-by: William Douglas <william.douglas@intel.com>
v31
2022-06-20 11:51:05 -07:00
Patrick McCarty b573085227 avxjudge: adjust regex for matching function names
Golang binaries have been observed to contain function names with many
other ASCII characters not captured by the previous regex.

Adjust that regex by permitting arbitrary characters for the function
name component of the line (between the ` <` and `>:`) and declaring the
expected content for the rest of the line.

Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>
2022-06-07 13:36:14 -07:00
Patrick McCarty 603f8d2354 avxjudge: fix function summary reporting
Refactor the output parsing of `objdump -d ...` so that function
summaries are properly reported.

Previously, the correspondence of results and functions were off-by-one:
the results were mistakenly associated with the next function to parse
instead of the previous function that was just parsed.

Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>
2022-06-07 13:36:14 -07:00
Patrick McCarty 9c58781110 avxjudge: call wrapper to print function summary
Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>
2022-06-07 13:36:14 -07:00
William Douglas f76aa44d80 Don't leave around empty filemaps
In cases where nothing was written to the filemap, delete it as it
just creates extra package waste if it is left behind.

Signed-off-by: William Douglas <william.douglas@intel.com>
v30
2022-05-19 13:47:50 -07:00
Arjan van de Ven 6eff659245 stat can fail on symlinks to nothing v29 2022-02-16 16:23:06 +00:00
Arjan van de Ven 6b171ba3bf symlink less v28 2022-01-31 15:54:48 +00:00
Arjan van de Ven e7848c7068 also move library symlinks v27 2022-01-29 22:27:41 +00:00
William Douglas 37633c34ea Add skip-path option
Enable skipping a specific file (this will override the include if
both paths are set to the same file).

Current usage is excluding setuid files that are only known about by
autospec and are otherwise tricky to exclude after.

Signed-off-by: William Douglas <william.douglas@intel.com>
v26
2022-01-21 13:25:39 -08:00
William Douglas d4084e61c2 Skip setuid files
The setuid files need to be specially annotated in the spec
file in order for the setuid attribute to be handled correctly by
rpm. Because the optimized binaries are not easily able to be
identified this way in the spec file, just skip setuid binaries from
being added to the optimized list.

Signed-off-by: William Douglas <william.douglas@intel.com>
v25
2022-01-21 10:46:12 -08:00
William Douglas 75fb0b530e Update regex dep matches
It is possible for matches to have the form:

rfc3986[idna2008]<2,>=1.3

So account for the bracket as part of the regex.

Signed-off-by: William Douglas <william.douglas@intel.com>
v24
2022-01-12 04:32:28 -08:00
William Douglas 3f524a463c Add new dep fix file
Also modify the pattern to account for the new file's format for
specifying dependencies.

Signed-off-by: William Douglas <william.douglas@intel.com>
v23
2022-01-09 13:21:05 -08:00
William Douglas 593a8c119b pypi-dep-fix: Add missing comparison regex patterns
Signed-off-by: William Douglas <william.douglas@intel.com>
v22
2021-11-22 07:55:15 -08:00
William Douglas 013a989f5a Add new python pypi dependency replacement tool
Add tooling to remove version dependencies for a given list of python
modules found in a subset of files in a given directory. This is
useful for cases where packages in pypi have out of date dependencies
relative to the test of the ecoysystem.

Signed-off-by: William Douglas <william.douglas@intel.com>
v21
2021-10-19 11:42:45 -07:00
William Douglas 7dfbf641ad elf-move: Include filepath in hash
Because miltiple binaries might share the same exact contents, include
the filepath as part of the hash.

Signed-off-by: William Douglas <william.douglas@intel.com>
v20
2021-10-14 12:30:27 -07:00
William Douglas 7c60cf719a elf-move: Fix --path behavior and crash
If --path was not passed previously elf-move would crash, it also was
incorrectly handling the filepath instead of the virtpath (only care
about the path after the DESTDIR for --path, makes handling easier
too).

Finally make --path override the --skip behavior so they can work
together to append new content to a file.
v19
2021-10-12 15:45:35 -07:00
William Douglas bbb7ff5e70 elf-move: Add option to always process a given path 2021-10-12 15:20:55 -07:00