This release adds several new features:
* Bitbake infile parsing using the --infile flag (see README)
* Support override file for %description section
* Default python pattern now distutils3 instead of distutils23
* Improved Rust/Cargo support
* Improved 32bit make/make_install args support
* Print fingerprint of imported pubkey
* Change default keyring to package repository with fallback the
autospec repository.
* Improved output
...makes some configuration updates:
* Adds several new public keys
* Adds lines to configure_whitelist
* Adds patterns for failed_patterns
...and makes several fixes as well:
* Print guessed commitmessage as string instead of list
* Fix several issues with catkin support
* Follow redirects by the license server
* Add a couple new URL patterns
* Remove unneeded V and VERBOSE flags for make lines
* Remove no-longer-used functional test suite
* Turn off string interpolation for URLs containing '%'
* Respect [autospec][urlban] for archive URLs
Signed-off-by: Matthew Johnson <matthew.johnson@intel.com>
When the user chooses to import a public key during the autospec run
display information on that key in the commit message.
Signed-off-by: Matthew Johnson <matthew.johnson@intel.com>
Instead of relying on a centralized keyring that resides in the autospec
repository, make the default keyring the package directory itself. Any
key imports will be to the package repository and will be committed as
part of the automatic autospec commit.
Retain the autospec repo keyring as a fallback keyring when the public
key does not exist in the package repo.
Signed-off-by: Matthew Johnson <matthew.johnson@intel.com>
Fixes#79
Print the key's fingerprint so the user can cross-reference a PGP key
server and make a more informed decision to import the public key.
Signed-off-by: Matthew Johnson <matthew.johnson@intel.com>
When skipping automatic git commit due to --skip-git flag print a
message informing the user they must commit their changes themselves.
Fixes#89
Signed-off-by: Matthew Johnson <matthew.johnson@intel.com>
Improve rust/cargo support by:
1) Better handling only toplevel Cargo configuration files for dependency
lookup.
2) Adding runtime requirements for buildreq packages in order to get
dependencies from non directly dependent packages resolved.
3) Add cargo as a buildreq for rust packages to handle the rustc/cargo
split.
4) Add cargo detection from failed_commands.
5) Fix rust source directory destination to not append version in case
the package name already had version information in it.
6) Get name and version information from crates.io package files.
The intention here is to use the self.cmake_srcdir variable, not print
the literal string to the file...
Signed-off-by: Matthew Johnson <matthew.johnson@intel.com>
configparser.ConfigParser attempts to interpolate "%" characters in the
values of the key-value pairs when parsing. This is absolutely unwanted
behavior, especially when parsing URLs such as
<sourceforge url>/joe-editor/files/JOE%20sources/<joe tar>
which cause an uncaught exception.
Turn off interpolation by passing the interpolation=None argument to
ConfigParser.
Add a unit tests to report when ConfigParser(...) is called in the
codebase without setting interpolation to None.
Signed-off-by: Matthew Johnson <matthew.johnson@intel.com>
This test suite is extremely hard to keep up-to-date and is no longer as
valuable now that unit tests have been written for much of the code
base. Actual functional testing can be done against package repos
without requiring them to be included in the autospec repository.
Signed-off-by: Matthew Johnson <matthew.johnson@intel.com>
Finish the work of 9bb89743d4 and remove
remaining VERBOSE flags for make. As mentioned in the previous commit
autospec does not parse the verbose output from make, so the flag is not
necessary and causes a considerable slowdown when parsing large build
logs.
Signed-off-by: Matthew Johnson <matthew.johnson@intel.com>
Suffice to get 302 or 200 on HEAD. Subsequent locations may point to
arbitrary servers which may return 403 on HEAD (while allow GET). This
issue happens with releases hosted on github.com, for example.
Depending on the dependencies name catkin will break the line or not,
this new patterns will catch more errors.
Signed-off-by: Leandro Dorileo <leandro.maciel.dorileo@intel.com>
Change default python buildpattern from distutils23 (both python2 and
python3) to distutils3 (python3 only). This is a step toward deprecating
python2 usage in distro packages. Any packages that rely on python2
should have "distutils23" added to the build_pattern configuration file
to maintain their python2 subpackage (or, preferrably, built with
python3 only as is default).
Signed-off-by: Matthew Johnson <matthew.johnson@intel.com>
Autospec does not parse the verbose output from make, so V=1 is not
necessary and slows down autospec build log parsing considerably. If
users want to debug a build at that level they should add the V=1 back
manually.
Signed-off-by: Matthew Johnson <matthew.johnson@intel.com>
We want to add a buildreq of the package providing the detected module'
pkgconfig (-lib/-dev), but, the package providing it will not install
the expected cmake files then make sure we also add it.
Signed-off-by: Leandro Dorileo <leandro.maciel.dorileo@intel.com>
Only try to parse CMakeLists.txt (looking for catkin' deps) only if
the CMakeLists.txt being processed is in the root tarball dir or in
the cmake_srcdir directory. With this we prevent to detect
dependencies on undesired sub components.
Signed-off-by: Leandro Dorileo <leandro.maciel.dorileo@intel.com>
Resolves#34
In case the user wants to override the base package %description content
autodetected by autospec, add support for a config file named
"description" to provide the content instead.
Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>
commitmessage is a list and is being printed as a string representation
of a list. Join it to a string to the print looks sane.
Signed-off-by: Matthew Johnson <matthew.johnson@intel.com>
Add print statements with the INFILE header to show when a value
from the infile has been incorporated into the autospec build
of a package.
Signed-off-by: Gabi Beyer <gabib@live.com>
Bitbake files contain executable tasks like configure and install.
Additionally, some have operations like append and prepend with
the task names. Scrape these commands, perform the operations and
store them in the dictionary as appropriate.
Then, after updating the specfile, write these patterns, as
comments to their mapped file name.
Signed-off-by: Gabi Beyer <gabib@live.com>
To use the infile_dict after the creation of the specfile, it
must be passed into the package function.
Also the requirement request for python is needed.
Signed-off-by: Gabi Beyer <gabib@live.com>
Add the license from the recipe file to the spec licenses
prior to the specfile creation, so that tarball analysis
that does not find a license file will get one from the
bitbake file.
Signed-off-by: Gabi Beyer <gabib@live.com>
Get the source url to set the url if one is not passed into
autospec. This allows a user to use an infile file as the
source for the tarball, and prevents them having to manually
find the url. It also requires that the variable PV and ROS_SPN
are replaced with their correct strings (version and name,
respectively).
Changed the name of the infile_parsers.py file to infile_bb_parser
since it is specific to recipe files.
Have string read into lists and joined to fix spacing issues.
Signed-off-by: Gabi Beyer <gabib@live.com>
Create 3 separate infile files, the infile_handler reads in the
inputs and determnes how to parse them. The --infile argument now
allows a url, file, or directory of files to be passed as the
input. The infile_handler determines that type of input, calls the
correct parser depending on the filetype, and calls the update to
the specfile.
The infile parsers contains the scraping and parsing for all file
types. Currently it scrapes both .bb and .inc files and stores the
data into one dictionary. Because these files are both for recipes
their data corresponds and the .inc file takes presedence.
The infile update spec file udpates aspects of the specfile that
have been scraped and stored into a dict from the infile parsers
file. If there exists a value for ROS_SPN in the .bb file, use that
when replace the ROS_SNP value in the replace PV function.
Signed-off-by: Gabi Beyer <gabib@live.com>
Add comments to the update functions (summary, license, build
deps) within the funcion.
Add an update_license function that appends any licenses scraped
from the bb file to the autospec licenses list if they don't
already exist.
Also added unittests for functionality
Signed-off-by: Gabi Beyer <gabib@live.com>
The specfile dependencies are a set, so create a set from the
deps scraped from the bb file, union the two sets, and write
it to the specfile.
Signed-off-by: Gabi Beyer <gabib@live.com>
If the specfile doesn't have a summary in the specfile, it
is overwritten by the summary provided by the bitbake recipe.
Signed-off-by: Gabi Beyer <gabib@live.com>
Add --infile argument that takes a url, path, or directory
to a bitbake/recipe file. It scrapes the .bb/.inc file for
configuration data and stores it in a dictionary. This
architecture can be used for multiple file formats.
Also adds some initial unit tests for scraping functionality.
Signed-off-by: Gabi Beyer <gabib@live.com>
In this case, the project uses one GitHub repo to distribute different
packages. The existing extraction rules are not enough
Signed-off-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Catkin is based on cmake and is a way to package/build modules as a
cmake components. This patch recognizes the patterns and handle the
dependencies, every catkin package provides a package-config descriptor
so it's pretty safe to rely on that.
Matthew Johnson