Implement the logging package for mixer.
A common log file can be set for all the mixer commands in the
builder.conf. E.g. `mixer config set Mixer.LOG <filepath>`
The log file and level can also be set for individual mixer
commands using the `--log` and `--log-level` flags respectively.
The various log levels are:
ERROR (1), WARNING (2), INFO (3), DEBUG (4) and VERBOSE (5).
Default log level is 4.
Fixes#666
Signed-off-by: Ashlesha Atrey ashlesha.atrey@intel.com
When calling the copyFileWithFlags function with the preserveSrc flag
set, file permissions must be set after creating the destination file
and setting the file owner. This is necessary because umask can prevent
newly created files from acquiring the source file's permissions and
file ownership changes can impact the setuid/setgid permissions.
Similarly directory permissions are set after creating the directory to
avoid unexpected permission behavior caused by umask.
Signed-off-by: John Akre <john.w.akre@intel.com>
When using content chroots, file uid/gid values should be preserved when
copying them to the full chroot.
Fixes#727
Signed-off-by: John Akre <john.w.akre@intel.com>
By default, files with the prefix "/bin/", "/usr/bin/" and "/usr/local/bin/"
will have a new export flag "x" in the 4th byte of the flag section of the
bundle Manifest.
The user can override this by using a new keyword "un-export()" in
the bundle definition file.
E.g. un-export(/usr/bin/file2)
Note: The export flag is at a bundle-file level and hence does not get
written to the Full manifest.
Fixes#708
Includes bat tests for the export flag feature.
Also, adds an entry for the content-chroot tests to Travis.
Signed-off-by: Reagan Lopez <reagan.lopez@intel.com>
Existing image and output directories for the current mix version will
be removed when building a mix. This prevents mixer from building on top
of directories contaminated by a prior mix.
Fixes#669
Signed-off-by: John Akre <john.w.akre@intel.com>
BUILD_ID field in os-release file specifies the upstream Clear
Linux version from which the mix is being created.
If upstream Clear Linux is not found, BUILD_ID is empty.
fixes#706
Signed-off-by: Ashlesha Atrey <ashlesha.atrey@intel.com>
Validate that upstream and local bundles are correctly deleted when
crossing an upstream format as a downstream mix.
Signed-off-by: John Akre <john.w.akre@intel.com>
Currently, for repos with "file" baseurl scheme, mixer expects the
rpms to be directly within the baseurl path.
But it is possible that rpms can be in sub directories within the repo baseurl.
In order to handle this, if an rpm is not found directly within the repo baseurl,
run repoquery to determine the actual location of the rpm.
This is expensive and it is recommended that rpms are stored directly
within the repo baseurl.
Includes a bat test.
Fixes#674
Signed-off-by: Reagan Lopez <reagan.lopez@intel.com>
Switch from ister.py to clr-installer for generating OS images.
Will migrate existing ister JSON file if it exist.
Both ISO generate and Raw Image retention enabled by default.
Signed-off-by: Mark D Horn <mark.d.horn@intel.com>
Currently, the bundle defintions are downloaded from the default
clr-bundles repo https://github.com/clearlinux/clr-bundles/archive/.
The user can now specify a different upstream bundles url using a mixer init flag or mixer
config parameter.
E.g. mixer init --upstream-bundles-url=https://github.com/reaganlo/clr-bundles/archive/
E.g. mixer config set Swupd.UPSTREAM_BUNDLES_URL https://github.com/reaganlo/clr-bundles/archive/
Since there is a tight coupling between the --upstream-version and bundles
repo, the user should ensure that the bundles repo has the same structure as the
clr-bundles repo and the tar.gz files should be named after an upstream
version. i.e. If the mix is based on an upstream-version 31300, the bundles repo
should have a file 31300.tar.gz
Fixes#614
Note: Setting this value using `mixer config set` will only update it in builder.conf.
It does not fetch the upstream bundles. A dedicated command for that purpose
will have to be implemented instead.
Signed-off-by: Reagan Lopez <reagan.lopez@intel.com>
This patch allows users to configure the compression methods that will
be used for fullfiles creation. The compression method can be set in the
Swupd.COMPRESSION property in builder.conf. Instead of enabling all
methods as previosly, xz is the only method that comes enabled by
default.
Note: internal gzip is still used for Link and Directory fullfile
creation.
Signed-off-by: Rodrigo Chiossi <rodrigo.chiossi@intel.com>
Added testcase for generating deltapacks with missing files for
different versions. If file is missing, delta pack for that
bundle should be ignored as delta pack generation is optional.
Signed-off-by: Ashlesha Atrey <ashlesha.atrey@intel.com>
The following minor statistics reporting changes were added to improve
statistics readability:
* Add colons to the added/deleted package sections
* Add space between '|' character and bundle name
Signed-off-by: John Akre <john.w.akre@intel.com>
In some cases a single DNF config file is not sufficient to correctly
download packages for the mix versions specified by the from and to
flags. The --from-repo-url and --to-repo-url flags were added to
temporarily override the baseurl value for the specified repo in the DNF
config when downloading packages for the from and to versions respectively.
Multiple instances of these flags can be used to override additional
repo/baseurl pairs. An example flag format is shown below:
--from-repo-url <repo>=<URL>
--to-repo-url <repo>=<URL>
Signed-off-by: John Akre <john.w.akre@intel.com>
When doing manual format bumps, the upstream version for +20 needs to be
updated to the upstream version in the new format, but since the format
differs, the build will fail.
This PR adds a new flag to `mixer versions update` and to `mixer build`
subcommands that allows this check to be skipped.
Signed-off-by: Rodrigo Chiossi <rodrigo.chiossi@intel.com>
Directories can be installed with inconsistent sizes. To make mixes
reproducible, the size of directories is omitted from the contentsize
calculation.
Signed-off-by: John Akre <john.w.akre@intel.com>
The mixer.state file now tracks the previous mix version with the
PREVIOUS_MIX_VERSION field. When this value is not set, it will default
to the LAST_VER or 0 when LAST_VER is invalid.
Format bumps create an important exception for the value of
PREVIOUS_MIX_VERSION. During a format bump, PREVIOUS_MIX_VERSION will be
overridden by the LAST_VER which will prevent PREVIOUS_MIX_VERSION from
altering format bump behavior.
Additionally, the mixer.state version was incremented and tests for
state file version conversions were added.
Signed-off-by: John Akre <john.w.akre@intel.com>
New param added in builder.conf Mixer section to support
customize os-release file.
Example to specify modified os-release file path in Mixer section:
[Mixer]
OS_RELEASE_PATH = "/tmp/modified-os-release"
Note: Either use empty string or do not specify the OS_RELEASE_PATH indicated
Mixer to use default os-release file.
Signed-off-by: sweeaun <swee.aun.khor@intel.com>
The bundle validation count only needs to account for the bundles that
have been directly added to the mix. With the fragmentation of the
default bundles into smaller bundles that are used as includes, the test
must ignore the included bundles and just count the ones that were
directly added.
Signed-off-by: Rodrigo Chiossi <rodrigo.chiossi@intel.com>
swupd-server used to support building bsdiff based deltas of manifest
files. This change adds that ability back for bundle manifests with
the 'mixer build delta-manifests' command.
Since the default Clear repo mirror changed to the cdn, dnf timeouts
occur more frequently. Increasing the timeout to 45 seconds from the
default timeout of 30 seconds.
Signed-off-by: John Akre <john.w.akre@intel.com>
Use cdn instead of download.clearlinux.org as the standard upstream URL,
because it's more reliable. Other usages of download.clearlinux.org were
also changed with the exception of version number downloads. Downloading
version numbers from download.clearlinux.org is still preferred because
we don't need to be concerned with caching.
Signed-off-by: Otavio Pontes <otavio.pontes@intel.com>
Version 27310 of clearlinux included os-core-webproxy as dependency of
os-core-update. This adds one extra bundle to the default bundle list.
Since bundle tests will count the number of lines in the output for
validation, this extra default bundle must be accounted for.
Signed-off-by: Rodrigo Chiossi <rodrigo.chiossi@intel.com>
The clear repo is mandatory for mixin. This change prevents mixin from
removing the clear repo.
Fixes#326
Signed-off-by: John Akre <john.w.akre@intel.com>
`bundle add` will now add valid bundles and skip only the invalid ones,
thereby fixing issue #489.
Signed-off-by: Reagan Lopez <reagan.lopez@intel.com>
- Reordered the tests to reduce code change impacts
- Added more test coverage for `bundle remove`
Signed-off-by: Reagan Lopez <reagan.lopez@intel.com>
With the removal of the editor feature in `bundle edit`,
the term `edit` is not relevant anymore. Hence, changing it to `create`.
In order to prevent breakage of existing use-cases,
`edit` is aliased to `create`and `--suppress-editor` is made hidden and deprecated.
Signed-off-by: Reagan Lopez <reagan.lopez@intel.com>
`bundle edit` will no longer open the editor. It will just create new bundles or copy existing bundles.
This command will locate the bundle by first looking in local-bundles, and then in upstream-bundles.
If the bundle is only found upstream, the bundle file will be copied to the local-bundles directory.
If the bundle is not found anywhere, a blank template will be created with the correct name.
This commit also fixes issue #448 and increases test coverage for `bundle validate`.
Signed-off-by: Reagan Lopez <reagan.lopez@intel.com>
There is no reason to number these tests as bat does not respect the
numbering system and it was obnoxious to try to keep collisions from
happening and keep the descriptions up-to-date.
Signed-off-by: Matthew Johnson <matthew.johnson@intel.com>
We had three 09-* tests, renumber them so we can prove we actually know
how to count when people look at our test directory.
Signed-off-by: Matthew Johnson <matthew.johnson@intel.com>
Add a bat test that creates a format bump then attempts to create delta
packs over that format bump. Make sure that the delta packs are not
created and that they are not even attempted to be created.
Signed-off-by: Matthew Johnson <matthew.johnson@intel.com>
This commit fulfills enhancement #58. When building an image without providing a template,
if the default image template is not present in the mix directory, mixer will do the following:
- Copy image template from ister template "/usr/share/defaults/ister/release-image-config.json".
- Update image template with the bundle list from "mixbundles" file.
In case the "mixbundles" file is empty or not present, use ister template as is.
- Inform the user about the above steps.
- Continue the existing build process.
Signed-off-by: Reagan Lopez <reagan.lopez@intel.com>
William Douglas