Compare commits

..

79 Commits

Author SHA1 Message Date
clrbuilder 865997ad69 Update packages file for version 30450
Signed-off-by: clrbuilder <clrbuilder@intel.com>
2019-07-19 08:02:20 +00:00
clrbuilder 70bfbe8c17 Update packages file for version 30440
Signed-off-by: clrbuilder <clrbuilder@intel.com>
2019-07-18 08:02:19 +00:00
Patrick McCarty 3b1e4e5558 pullrebase: skip rebase if current branch is up-to-date
Any attempt at rebasing is unnecessary when the current branch is
up-to-date with the remote, so skip the rebase in this case.

Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>
2019-07-17 12:54:04 -07:00
clrbuilder 2465219907 Update packages file for version 30420
Signed-off-by: clrbuilder <clrbuilder@intel.com>
2019-07-17 16:15:58 +00:00
clrbuilder 77775f060c Update packages file for version 30410
Signed-off-by: clrbuilder <clrbuilder@intel.com>
2019-07-16 18:03:37 +00:00
clrbuilder 67406f7be2 Update packages file for version 30400
Signed-off-by: clrbuilder <clrbuilder@intel.com>
2019-07-16 10:01:43 +00:00
clrbuilder 9bece241c0 Update packages file for version 30380
Signed-off-by: clrbuilder <clrbuilder@intel.com>
2019-07-15 10:02:02 +00:00
clrbuilder 602cc9563f Update packages file for version 30360
Signed-off-by: clrbuilder <clrbuilder@intel.com>
2019-07-13 17:54:20 +00:00
George T Kramer 02b5ad93e3 Place koji user configuration in user folder
Avoid placing koji.conf in a system-level directory

Signed-off-by: George T Kramer <george.t.kramer@intel.com>
2019-07-11 17:10:09 -07:00
clrbuilder fb40533923 Update packages file for version 30320
Signed-off-by: clrbuilder <clrbuilder@intel.com>
2019-07-11 12:02:13 +00:00
clrbuilder 06dd86a3b9 Update packages file for version 30290
Signed-off-by: clrbuilder <clrbuilder@intel.com>
2019-07-10 00:47:12 +00:00
clrbuilder 9db26c3041 Update packages file for version 30280
Signed-off-by: clrbuilder <clrbuilder@intel.com>
2019-07-09 18:12:07 +00:00
clrbuilder ade6194a0c Update packages file for version 30270
Signed-off-by: clrbuilder <clrbuilder@intel.com>
2019-07-09 12:02:10 +00:00
clrbuilder a8d18b2f92 Update packages file for version 30260
Signed-off-by: clrbuilder <clrbuilder@intel.com>
2019-07-07 17:52:48 +00:00
Arjan van de Ven c2d637b438 don't shhow version changes in the 32 bit / avx copies 2019-07-05 13:02:00 +00:00
clrbuilder 48d06e458d Update packages file for version 30230
Signed-off-by: clrbuilder <clrbuilder@intel.com>
2019-07-05 05:53:08 +00:00
clrbuilder 0ba3b95dad Update packages file for version 30200
Signed-off-by: clrbuilder <clrbuilder@intel.com>
2019-07-04 08:19:58 +00:00
Miguel Bernal Marin 9749d00c56 update OVMF* files
Clear Linux edk2 package was updated to stable201905

https://github.com/clearlinux-pkgs/edk2/commit/7b191c6ae982a550f2a335cd4cd4cf2857d94b49

Let's update OVMF.fd and OVMF_CODE.fd

Note:  OVMF_VARS.fd has the same hash and is not updated.

Note2: that these version of OVMF* are a copy from /usr/share/qemu
       in Clear Linux OS 30170.

$ sha512sum /usr/share/qemu/OVMF*
c6630296e4a82dc4a24b20522e815171fd9c1689403a36abf7ec070c7fce85af63b88b328721a94c415f513ffd2423f71e591ae9713e65850b5a29ebd1edac89  /usr/share/qemu/OVMF_CODE.fd
f48de15ce6dbc19c6a0770677a874f289a69df813f31add873d30324df6b96ae0a0756af52fef3101f5077280851f095120663ec68bb6d19b12a6b0937f86df8  /usr/share/qemu/OVMF.fd
448412fd7ba267b4180db8ade6edb67af467e5b9b3e3ff8dfd583a2fded4070f6951667297e6896ce8bd9f4d2ec3dd8a5a70b6e9a2f686efec9a57124fec512a  /usr/share/qemu/OVMF_VARS.fd
2019-07-03 14:22:51 -05:00
clrbuilder 7841d4dd5c Update packages file for version 30190
Signed-off-by: clrbuilder <clrbuilder@intel.com>
2019-07-03 18:04:39 +00:00
clrbuilder 91c89fe47d Update packages file for version 30180
Signed-off-by: clrbuilder <clrbuilder@intel.com>
2019-07-03 12:02:25 +00:00
Patrick McCarty 9d696fee5a Add R-R2jags and R-R2WinBUGS packages
Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>
2019-07-02 11:48:05 -07:00
clrbuilder 54aea8a696 Update packages file for version 30160
Signed-off-by: clrbuilder <clrbuilder@intel.com>
2019-07-02 12:02:28 +00:00
clrbuilder a4545eb4b0 Update packages file for version 30140
Signed-off-by: clrbuilder <clrbuilder@intel.com>
2019-07-01 12:02:28 +00:00
clrbuilder 35d4f2c00f Update packages file for version 30130
Signed-off-by: clrbuilder <clrbuilder@intel.com>
2019-06-28 18:01:41 +00:00
clrbuilder 15c22ac4eb Update packages file for version 30110
Signed-off-by: clrbuilder <clrbuilder@intel.com>
2019-06-26 18:08:11 +00:00
clrbuilder 1c672f7638 Update packages file for version 30100
Signed-off-by: clrbuilder <clrbuilder@intel.com>
2019-06-26 12:02:25 +00:00
clrbuilder d0832c89f4 Update packages file for version 30090
Signed-off-by: clrbuilder <clrbuilder@intel.com>
2019-06-25 18:03:08 +00:00
clrbuilder 3a87e928c5 Update packages file for version 30080
Signed-off-by: clrbuilder <clrbuilder@intel.com>
2019-06-25 12:02:17 +00:00
clrbuilder 5ba1a7e1e3 Update packages file for version 30070
Signed-off-by: clrbuilder <clrbuilder@intel.com>
2019-06-24 18:03:08 +00:00
clrbuilder ba02bc2a4f Update packages file for version 30060
Signed-off-by: clrbuilder <clrbuilder@intel.com>
2019-06-24 12:02:30 +00:00
clrbuilder 0ef90a4704 Update packages file for version 30050
Signed-off-by: clrbuilder <clrbuilder@intel.com>
2019-06-23 15:45:31 +00:00
clrbuilder 0b39553aa6 Update packages file for version 30040
Signed-off-by: clrbuilder <clrbuilder@intel.com>
2019-06-21 18:03:19 +00:00
Tan, Yew Wayne 7cd52ed2fd ltsutils: Fix PackageRepo.getNVR for package names that contain '-'
Refactor unit tests, add test case for MySQL-python package
2019-06-21 09:12:15 -07:00
Tan, Yew Wayne a379bdd1e5 ltsutils: prompt: Timeout after 60 seconds
Signed-off-by: Tan, Yew Wayne <yew.wayne.tan@intel.com>
2019-06-21 09:12:15 -07:00
Tan, Yew Wayne b8eaa3c103 ltsutils: make lts-koji: Modify koji target for building on LTS infrastructure
Differences:
- Restrict to submit from an active branch instead of master branch
- git pull --rebase from matching remote LTS branch if available
- Add "-lts" suffix to git tag
- Assume koji buildtag name is identical to branch name

Signed-off-by: Tan, Yew Wayne <yew.wayne.tan@intel.com>
2019-06-21 09:12:15 -07:00
Tan, Yew Wayne 0664274e0a ltsutils: make lts-build: Build RPM or reuse existing build
Todo: can-reuse-binary simply compares package versions to determine if
Koji builds can be shared between branches. Work is in progress to
implement ABI compatiblity testing.

Signed-off-by: Tan, Yew Wayne <yew.wayne.tan@intel.com>
2019-06-21 09:12:15 -07:00
Tan, Yew Wayne 69cee2fcd7 LTS package maintenance utility (ltsutils): initial commit
This tooling is designed to automate 2 main tasks that are part of
the package maintenance workflow of Clear Linux LTS. These tasks are:
- Back-porting of a patch (e.g. security fix) to older branches.
- (Not implemented yet) Building RPMs with the intent of sharing binaries of
  older LTS branches to newer branches whenever possible.

2 new targets are defined in Makefile.common.lts:
- lts-show: Show a summary of active LTS branches
- lts-backport: Attempt to fast-forward the previous active branch to the current branch

"Active" branches correspond to LTS releases that currently have support.
They are listed in a flat file "active-branches" in "lts" directory, from
oldest to newest. New entries are added by Clear Linux LTS developers as
new releases become available, and entries removed as releases become
obsolete.

Note: For CVE patching, the tool is not aware of CVE severity levels or
the minimum supported severity level of each LTS branch. For now it is
the user's responsibility to know when a CVE does not apply to older
branches and stop calling "make lts-backport".

Signed-off-by: Tan, Yew Wayne <yew.wayne.tan@intel.com>
2019-06-21 09:12:15 -07:00
clrbuilder 19d7dfcc56 Update packages file for version 30030
Signed-off-by: clrbuilder <clrbuilder@intel.com>
2019-06-21 12:02:15 +00:00
clrbuilder 25e60b8d7f Update packages file for version 30020
Signed-off-by: clrbuilder <clrbuilder@intel.com>
2019-06-20 18:02:35 +00:00
clrbuilder c6749e0569 Update packages file for version 30010
Signed-off-by: clrbuilder <clrbuilder@intel.com>
2019-06-19 20:21:46 +00:00
Geoffroy Van Cutsem 79b17335fa Really set 'latest_builds' to the latest value of the day
Consider the following scenario:
* System being set-up on day 0 (by running user-setup.sh script)
* The github.com/clearlinux/common repo is cloned under projects/common
* Fast-forward in the future
* Clone a package for which there is no reachable upstream URL
	(e.g. clr-power-tweaks or clr-systemd-config
* Run 'make sources'
That operation will fail because it will search for the src.rpm files starting
with the Clear Linux build determined by looking up the latest tag in the
project/common repo (and the latest is the one from day 0). So it will only find
older versions of those src.rpm, and will eventually fail.

This patch adds a line that fetches all the latest tags from the upstream
projects/common repo so the 'latest_builds' variable is *really* set to the
latest of the day.

Signed-off-by: Geoffroy Van Cutsem <geoffroy.vancutsem@intel.com>
2019-06-19 11:03:00 -07:00
clrbuilder 18503d7ff9 Update packages file for version 30000
Signed-off-by: clrbuilder <clrbuilder@intel.com>
2019-06-19 15:32:37 +00:00
clrbuilder d4a14d4b51 Update packages file for version 29990
Signed-off-by: clrbuilder <clrbuilder@intel.com>
2019-06-19 12:02:34 +00:00
Patrick McCarty 67f9a85781 toplevel: simplify clean-pkgs-dir command
Create a PKGS_ORPHANED list containing all orphaned package repos (i.e.
package repos for packages not listed in projects/common/packages,
excluding the local compatibility packages/common directory), and use
that list for the `clean-pkgs-dir` command.

Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>
2019-06-18 17:30:19 -07:00
clrbuilder 2ab93bc546 Update packages file for version 29980
Signed-off-by: clrbuilder <clrbuilder@intel.com>
2019-06-18 18:06:21 +00:00
clrbuilder 0697a48265 Update packages file for version 29970
Signed-off-by: clrbuilder <clrbuilder@intel.com>
2019-06-18 12:02:24 +00:00
clrbuilder 3488d9df0b Update packages file for version 29960
Signed-off-by: clrbuilder <clrbuilder@intel.com>
2019-06-17 17:58:32 +00:00
clrbuilder 7eda6b13d5 Update packages file for version 29950
Signed-off-by: clrbuilder <clrbuilder@intel.com>
2019-06-17 11:58:06 +00:00
clrbuilder 22cf4d9aaa Update packages file for version 29940
Signed-off-by: clrbuilder <clrbuilder@intel.com>
2019-06-14 17:57:53 +00:00
Arjan van de Ven abf88fe302 add some more boilerplate patterns 2019-06-14 14:48:13 +00:00
Arjan van de Ven e7d86e5ace make it clear there's no point in submitting routine stuff for review
as it only slows things down
2019-06-14 14:41:28 +00:00
clrbuilder 0ca8a71a6a Update packages file for version 29930
Signed-off-by: clrbuilder <clrbuilder@intel.com>
2019-06-14 11:57:34 +00:00
clrbuilder c594d50d97 Update packages file for version 29920
Signed-off-by: clrbuilder <clrbuilder@intel.com>
2019-06-13 17:58:36 +00:00
clrbuilder 923b813833 Update packages file for version 29910
Signed-off-by: clrbuilder <clrbuilder@intel.com>
2019-06-13 11:57:26 +00:00
Thiago Macieira facfc57d55 Update message for the for-review.txt on how to recreate
People SHOULD update commit messages if it's not a simple bump or
version update.

Signed-off-by: Thiago Macieira <thiago.macieira@intel.com>
2019-06-12 16:05:52 -07:00
clrbuilder 9e0d404e52 Update packages file for version 29900
Signed-off-by: clrbuilder <clrbuilder@intel.com>
2019-06-12 17:58:01 +00:00
clrbuilder 9d954cb33a Update packages file for version 29890
Signed-off-by: clrbuilder <clrbuilder@intel.com>
2019-06-12 11:57:25 +00:00
Patrick McCarty 158c743fde Rename typed-ast -> typed_ast
Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>
2019-06-11 20:48:41 -07:00
clrbuilder c5969bbd83 Update packages file for version 29880
Signed-off-by: clrbuilder <clrbuilder@intel.com>
2019-06-11 17:58:21 +00:00
clrbuilder cb01f2b9ef Update packages file for version 29850
Signed-off-by: clrbuilder <clrbuilder@intel.com>
2019-06-10 11:58:23 +00:00
clrbuilder c58e39da50 Update packages file for version 29840
Signed-off-by: clrbuilder <clrbuilder@intel.com>
2019-06-07 17:58:47 +00:00
clrbuilder 26dea91314 Update packages file for version 29830
Signed-off-by: clrbuilder <clrbuilder@intel.com>
2019-06-07 12:01:05 +00:00
clrbuilder efa4e1bcf8 Update packages file for version 29820
Signed-off-by: clrbuilder <clrbuilder@intel.com>
2019-06-06 17:59:00 +00:00
clrbuilder 9fba303a7b Update packages file for version 29800
Signed-off-by: clrbuilder <clrbuilder@intel.com>
2019-06-05 20:28:24 +00:00
clrbuilder eefffcb4cd Update packages file for version 29780
Signed-off-by: clrbuilder <clrbuilder@intel.com>
2019-06-05 04:24:22 +00:00
clrbuilder ae59dd4aa9 Update packages file for version 29770
Signed-off-by: clrbuilder <clrbuilder@intel.com>
2019-06-04 21:01:00 +00:00
Patrick McCarty 9ed6cf64eb Add notice about for-review.txt; make compatible with git send-email
The for-review.txt files were being generated with `git show` and thus
not compatible with `git send-email`. Use `git format-patch` instead to
fix this issue.

Also add a more prominent notice about the existence of for-review.txt
and its intended purpose.

Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>
2019-06-03 15:17:49 -07:00
clrbuilder 1e1a52e948 Update packages file for version 29680
Signed-off-by: clrbuilder <clrbuilder@intel.com>
2019-05-29 13:05:05 +00:00
clrbuilder b9d3725172 Update packages file for version 29660
Signed-off-by: clrbuilder <clrbuilder@intel.com>
2019-05-28 13:41:13 +00:00
clrbuilder d2c0eea1d2 Update packages file for version 29620
Signed-off-by: clrbuilder <clrbuilder@intel.com>
2019-05-24 11:58:40 +00:00
clrbuilder 038007f64a Update packages file for version 29600
Signed-off-by: clrbuilder <clrbuilder@intel.com>
2019-05-23 11:58:43 +00:00
clrbuilder 930a8bd3f7 Update packages file for version 29590
Signed-off-by: clrbuilder <clrbuilder@intel.com>
2019-05-22 19:40:13 +00:00
clrbuilder 33c27808a6 Update packages file for version 29570
Signed-off-by: clrbuilder <clrbuilder@intel.com>
2019-05-22 11:59:13 +00:00
Patrick McCarty f9b7f544be scratch-wait: explicitly specify the wait option
We always want the koji CLI to wait for `make scratch-wait`, regardless
of the environment koji detects it is being run under.

Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>
2019-05-22 00:44:24 -07:00
clrbuilder 6fe60137f4 Update packages file for version 29530
Signed-off-by: clrbuilder <clrbuilder@intel.com>
2019-05-21 11:58:35 +00:00
clrbuilder b32e466635 Update packages file for version 29520
Signed-off-by: clrbuilder <clrbuilder@intel.com>
2019-05-20 17:59:07 +00:00
clrbuilder 755bd91716 Update packages file for version 29510
Signed-off-by: clrbuilder <clrbuilder@intel.com>
2019-05-20 11:59:04 +00:00
clrbuilder d2b615d14b Update packages file for version 29500
Signed-off-by: clrbuilder <clrbuilder@intel.com>
2019-05-18 19:01:47 +00:00
clrbuilder 71b5075a7b Update packages file for version 29480
Signed-off-by: clrbuilder <clrbuilder@intel.com>
2019-05-17 18:36:33 +00:00
17 changed files with 891 additions and 29 deletions
+22 -3
View File
@@ -143,6 +143,10 @@ pullrebase:
echo "No commits exist in remote. Continuing."; \
exit 0; \
fi; \
if [ "$$(git rev-list FETCH_HEAD | head -n 1)" = "$$(git rev-list HEAD | head -n 1)" ]; then \
echo "Current branch up-to-date. Continuing."; \
exit 0; \
fi; \
echo "Rebasing to origin/master"; \
if git status --porcelain | grep -q '^.[^?]'; \
then \
@@ -186,8 +190,19 @@ autospec: preautospec-checks pullrebase localreponotice clean-old-content
bash update_changelog.sh ; \
git commit --amend --no-edit Change* ; \
fi
git diff | grep -q index || python3 $(TOPLVL)/projects/common/patchfilter.py > for-review.txt ;
git diff | grep -q index || python3 $(TOPLVL)/projects/common/patchfilter.py ;
@$(MAKE) -s for-review.txt
diffstat for-review.txt
@printf "\n**\n"
@printf "** NOTICE: A patch with changes is available in the file for-review.txt\n"
@printf "** To recreate (e.g., after git commit --amend), run make for-review.txt\n"
@printf "** To submit for review: git send-email --to <recipient> for-review.txt\n"
@printf "**\n\n"
#help for-review.txt: Creates the for-review.txt file, which is
#help a minimized version of the Git commit, suitable for code review.
for-review.txt:
git diff | grep -q index || python3 $(TOPLVL)/projects/common/patchfilter.py > for-review.txt
.PHONY: for-review.txt
#help autospecnogit: Runs autospec, but does not create a commit
autospecnogit:
@@ -269,6 +284,7 @@ sources:
@$(MAKE) generateupstream; \
[ $$? -eq 0 ] && exit 0; \
nvr="$$(rpmspec --srpm -q --qf '%{NAME}\t%{VERSION}\t%{RELEASE}\n' $(SPECFILE))"; \
git -C $(TOPLVL)/projects/common fetch --tags >/dev/null 2>&1; \
latest_builds=$$(git -C $(TOPLVL)/projects/common tag -l | sort -rn); \
for b in $$latest_builds; do \
url="$(DOWNLOAD_MIRROR)/releases/$$b/clear/source/package-sources"; \
@@ -320,7 +336,7 @@ scratch: $(SPECFILE) upstream $(SRPMFILE) kojidef
#help scratch-wait: Performs scratch-build on package.
scratch-wait: $(SPECFILE) upstream $(SRPMFILE) kojidef
$(KOJI_CMD) build --scratch $(KOJI_TAG) $(SRPMFILE)
$(KOJI_CMD) build --scratch --wait $(KOJI_TAG) $(SRPMFILE)
#help koji-nowait: Same as 'make koji', but do not block
koji-nowait:
@@ -492,6 +508,9 @@ cloc: $(SRPMFILE)
@$(MOCK) --clean --scrub=chroot --uniqueext=$(PKG_NAME)
cat results/cloc.txt
# Define LTS-specific targets in a separate makefile
-include $(TOPLVL)/projects/common/Makefile.common.lts
# Define site local common targets in a separate makefile
-include $(TOPLVL)/projects/common/Makefile.common.site_local
+90
View File
@@ -0,0 +1,90 @@
#-*-makefile-*-
LTSUTILS = python $(TOPLVL)/projects/common/lts/main.py $(PKG_NAME)
#help lts-show: Display a summary of active LTS branches.
lts-show:
@$(LTSUTILS) sanity-check
@while read b; do \
if git show-ref $$b > /dev/null; then \
echo $$b $$(git log --oneline -1 $$b); \
else \
echo $$b Not found; \
fi; \
done < $(TOPLVL)/projects/common/lts/active-branches
#help lts-backport: Fast-forward the previous active branch to the current
#help branch.
lts-backport:
@$(LTSUTILS) sanity-check
@newer=$$(git symbolic-ref HEAD); \
newer=$${newer#refs/heads/}; \
if ! $(LTSUTILS) prev-branch --checkout; then \
echo Could not check out previous active branch.; \
exit 0; \
fi; \
if $(LTSUTILS) is-same-version $$newer; then \
$(LTSUTILS) fast-forward $$newer; \
else \
echo Most likely a patch needs to be manually re-applied for this version.; \
echo Alternatively, use \"git merge --ff-only $$newer\" to upgrade the package version.; \
fi
#help lts-build: Build RPM in Koji, or reuse existing build from older LTS
#help branch.
lts-build:
@$(LTSUTILS) sanity-check
@tag=$$(git tag --points-at); \
if [[ -z $$tag ]]; then \
echo This commit does not have a tag.; \
echo Proceeding to run \"make lts-koji\".; \
$(LTSUTILS) prompt || exit; \
$(MAKE) lts-koji; \
exit; \
fi; \
older=$$($(LTSUTILS) prev-branch); \
if [[ -z $$older ]]; then \
echo Could not determine previous active branch.; \
exit 1; \
fi; \
current=$$($(LTSUTILS) current-branch); \
if [[ $$(git rev-parse $$older) != $$(git rev-parse $$current) ]]; then \
echo Previous branch and current branch have different commits.; \
for b in $$older $$current; do echo $$b $$(git log --oneline -1 $$b); done; \
echo Proceeding to run \"make lts-koji\".; \
$(LTSUTILS) prompt || exit; \
$(MAKE) lts-koji; \
exit; \
fi; \
if $(LTSUTILS) can-reuse-binary $$older; then \
echo Reusing binary from $$older.; \
$(KOJI_CMD) tag-build $$current $(SRPMVERS); \
else \
echo Could not reuse binary from $$older.; \
echo Proceeding to run \"make bump lts-koji\".; \
$(LTSUTILS) prompt || exit; \
$(MAKE) bump lts-koji; \
fi
@$(LTSUTILS) next-branch --checkout
#help lts-koji: Same as "koji" but for working on LTS branches. Should not
#help be called directly, use lts-build instead.
lts-koji: prekoji-checks kojidef
@$(LTSUTILS) sanity-check
@$(MAKE) spdxcheck
@$(MAKE) checkblacklist
@current=$$($(LTSUTILS) current-branch); \
if ! grep $$current $(TOPLVL)/projects/common/lts/active-branches; then \
echo "Error: Must be on an active branch to submit to koji" >&2; \
exit 1; \
fi; \
if ! git diff --quiet HEAD ${SPECFILE}; then \
echo "Error: All changes to ${SPECFILE} must be committed first" >&2; \
exit 1; \
fi; \
if git rev-parse --verify --quiet origin/$$current > /dev/null; then \
git pull --rebase; \
fi
git tag $(SRPMVERS)-lts
git push origin $$($(LTSUTILS) current-branch) refs/tags/$(SRPMVERS)-lts
$(KOJI_CMD) build $$KOJI_NOWAIT $$($(LTSUTILS) current-branch) $(PKG_BASE_URL)/$(PKG_NAME)?#$(SRPMVERS)-lts
+9 -7
View File
@@ -11,6 +11,7 @@ SHELL = /bin/bash
PKGS = $(shell cat $(PACKAGES_FILE))
PKGS_LOCAL = $(sort $(filter ${PKGS},$(subst packages/,,$(wildcard packages/*))))
PKGS_ORPHANED = $(sort $(filter-out common ${PKGS},$(subst packages/,,$(wildcard packages/*))))
proj_PKGS = $(addprefix proj_,$(PROJ_PKGS))
clone_PKGS = $(addprefix clone_,$(PKGS))
@@ -128,11 +129,9 @@ pull: ${PULL_PKGS}
#help clean-pkgs-dir: For packages that are no longer present in the distro,
#help removes the associated package repos from the ./packages tree
clean-pkgs-dir: $(PACKAGES_FILE)
@for p in `ls packages`; do \
if ! grep -wq "^$$p$$" "$<" && [ "$$p" != common ]; then \
echo "Removing $$p from packages, it is no longer in common/packages."; \
rm -rf packages/$$p; \
fi \
@for p in ${PKGS_ORPHANED}; do \
echo "Removing $$p from packages, it is no longer in common/packages."; \
rm -rf packages/$$p; \
done
@echo "packages directory cleaned";
@@ -229,8 +228,11 @@ autospecnew: preautospecnew-checks localreponotice
if [ $$? -eq 0 ]; then \
$(MAKE) link-new-rpms PKG_REPO_DIR="${TOPLVL}/packages/${NAME}"; \
$(TOPLVL)/projects/common/checkblacklist.sh $(TOPLVL)/projects/common/blacklist $(TOPLVL)/packages/${NAME}/results/*.rpm; \
python3 $(TOPLVL)/projects/common/patchfilter.py <(git -C $(TOPLVL)/packages/$(NAME) show) > $(TOPLVL)/packages/$(NAME)/for-review.txt; \
python3 $(TOPLVL)/projects/common/patchfilter.py <(git -C $(TOPLVL)/packages/$(NAME) show); \
python3 $(TOPLVL)/projects/common/patchfilter.py <(git -C $(TOPLVL)/packages/$(NAME) format-patch -1 --stdout) > $(TOPLVL)/packages/$(NAME)/for-review.txt; \
printf "\n**\n"; \
printf "** NOTICE: A patch with changes is available in the file $(TOPLVL)/packages/$(NAME)/for-review.txt\n"; \
printf "** Please look through this file and if you are unsure, please submit for code review with git send-email\n"; \
printf "**\n\n"; \
fi; \
else \
echo "$(NAME) already exists at $(TOPLVL)/packages/$(NAME)"; \
BIN
View File
Binary file not shown.
BIN
View File
Binary file not shown.
+5
View File
@@ -0,0 +1,5 @@
TEST = test.test_ltsutils
.PHONY: test
test:
PYTHONPATH=. python -m unittest -v -k .Test $(TEST)
+19
View File
@@ -0,0 +1,19 @@
# LTS package maintenance utility
This tooling is designed to automate 2 main tasks that are part of
the package maintenance workflow of Clear Linux LTS. These tasks are:
- Back-porting of a patch (e.g. security fix) to older branches.
- Building RPMs with the intent of sharing binaries of older LTS branches to
newer branches whenever possible.
There should be no need to run this tool directly. Instead use the following
targets defined in Makefile.common.lts:
- lts-show: Show a summary of active LTS branches
- lts-backport: Attempt to fast-forward the previous active branch to the current branch
- lts-build: Build RPM in Koji, or reuse existing build from older branch
"Active" branches correspond to LTS releases that currently have support.
They are listed in a flat file "active-branches" in "lts" directory, from
oldest to newest. New entries are added by Clear Linux LTS developers as
new releases become available, and entries removed as releases become
obsolete.
View File
View File
+52
View File
@@ -0,0 +1,52 @@
import pathlib
from subprocess import PIPE, CalledProcessError
from .shell import Shell
class PackageRepo:
'''Represents a package repository. Most methods are wrappers of git commands.'''
class UnknownCurrentBranchException(Exception): pass
class InvalidBranchException(Exception): pass
def __init__(self, name, path):
self.name = name
self.path = pathlib.Path(path)
self.sh = Shell(self.path)
def getNVR(self, commit='HEAD'):
with self.sh.popen(['git', 'show', '{}:{}.spec'.format(commit, self.name)], stdout=PIPE) as specfile:
nvr = self.sh.run('rpmspec --srpm -q --queryformat %{NVR} /dev/stdin', stdin=specfile.stdout)
return tuple(nvr.stdout.strip().rsplit('-', maxsplit=2))
def checkoutBranch(self, branch, allow_remote=False):
# allow_remote=True allows checking out a new remote-tracking branch
if not allow_remote and not self.hasBranch(branch):
raise self.InvalidBranchException(branch)
self.sh.run_args(['git', 'checkout', branch], capture_output=False)
def fastForwardBranch(self, old, new):
self.checkoutBranch(old)
if not self.hasBranch(new):
raise self.InvalidBranchException(new)
self.sh.run_args(['git', 'merge', '--ff-only', new], capture_output=False)
def getActiveBranches(self):
toplvl = pathlib.Path(self.path) / '../..'
common = toplvl / 'projects/common'
active_branches = common / 'lts/active-branches'
with active_branches.open() as f:
return [line.rstrip() for line in f]
def getCurrentBranch(self):
try:
head = self.sh.run('git symbolic-ref HEAD').stdout.strip()
except CalledProcessError:
raise self.UnknownCurrentBranchException
refs_heads = 'refs/heads/'
assert head.startswith(refs_heads)
head = head[len(refs_heads):]
return head
def hasBranch(self, branch):
p = self.sh.run_args(['git', 'rev-parse', 'refs/heads/'+branch], check=False)
return p.returncode == 0
+27
View File
@@ -0,0 +1,27 @@
import subprocess, shlex
class Shell:
# Default options passed to subprocess.run. May be customized per-instance.
cwd = None
check = True
capture_output = True
text = True
def __init__(self, cwd=None):
if cwd: self.cwd = cwd
def run_args(self, args, **kwargs):
kwargs1 = {
'check': self.check,
'capture_output': self.capture_output,
'text': self.text,
'cwd': self.cwd
}
kwargs1.update(kwargs)
return subprocess.run(args, **kwargs1)
def run(self, cmd, **kwargs):
return self.run_args(shlex.split(cmd), **kwargs)
def popen(self, args, **kwargs):
return subprocess.Popen(args, cwd=self.cwd, **kwargs)
+161
View File
@@ -0,0 +1,161 @@
#!/usr/bin/python
import sys,argparse
from ltsutils.package_repo import PackageRepo
def log(msg, **kwargs):
print(msg, file=sys.stderr)
def init_parser():
main = argparse.ArgumentParser()
main.add_argument('package_name', nargs=1)
subparsers = main.add_subparsers(dest='command', metavar='command', required=True)
# Package maintenance commands
p = subparsers.add_parser('prev-branch',
help='show previous branch')
p.add_argument('--checkout', action='store_true', help='checkout the branch')
p = subparsers.add_parser('next-branch',
help='show next branch')
p.add_argument('--checkout', action='store_true', help='checkout the branch')
p = subparsers.add_parser('current-branch',
help='show current branch')
p = subparsers.add_parser('is-same-version',
help='return true if package version is the same as the given branch')
p.add_argument('branch', nargs=1)
p = subparsers.add_parser('fast-forward',
help='fast-forward current branch to a newer branch')
p.add_argument('branch', nargs=1)
# RPM build commands
p = subparsers.add_parser('can-reuse-binary',
help='check if binary from another branch can be used in current branch')
p.add_argument('branch', nargs=1)
# Other commands
p = subparsers.add_parser('prompt',
help='prompt user to continue and return appropriate exit code')
p = subparsers.add_parser('sanity-check',
help='run sanity checks for data consistency')
return main
def prev_branch(args, repo):
active_branches = repo.getActiveBranches()
current = repo.getCurrentBranch()
i = active_branches.index(current)
if i == 0:
log('Already on oldest active branch.')
return False
prev = active_branches[i-1]
print(prev)
if args.checkout:
repo.checkoutBranch(prev, allow_remote=True)
def next_branch(args, repo):
active_branches = repo.getActiveBranches()
current = repo.getCurrentBranch()
i = active_branches.index(current)
if i == len(active_branches)-1:
log('Already on newest active branch.')
return False
next_ = active_branches[i+1]
print(next_)
if args.checkout:
repo.checkoutBranch(next_, allow_remote=False)
def current_branch(args, repo):
current = repo.getCurrentBranch()
print(current)
def is_same_version(args, other):
current = repo.getCurrentBranch()
other = args.branch[0]
assert repo.hasBranch(other), 'Branch %s not found' % other
v1, v2 = [repo.getNVR('refs/heads/'+b)[1] for b in (current, other)]
if v1 != v2:
log('Current version {} does not match version {} on branch {}'.format(v1, v2, other))
return v1 == v2
def fast_forward(args, repo):
current = repo.getCurrentBranch()
newer = args.branch[0]
log('Fast-forwarding {} to {}'.format(current, newer))
repo.fastForwardBranch(current, newer)
def sanity_check(args, repo):
ok = True
# HEAD must point to a branch
try:
current = repo.getCurrentBranch()
except PackageRepo.UnknownCurrentBranchException:
log('Unknown current branch. Has a branch been checked out?')
current = None
ok = False
# active-branches file must not be empty
active_branches = repo.getActiveBranches()
if not len(active_branches):
log('No active branches defined. Is active-branches file empty?')
ok = False
# current branch must be an active branch
elif current and current not in active_branches:
log('%s is not an active LTS branch.' % current)
ok = False
return ok
def can_reuse_binary(args, repo):
# Just compare versions for now
# TODO: ABI compatibility testing
current = repo.getCurrentBranch()
older = args.branch[0]
v1, v2 = [repo.getNVR('refs/heads/'+b)[1] for b in (older, current)]
return v1 == v2
def prompt(args, repo):
import selectors
timeout = 60
while True:
print('Continue? (y/N): ', end='', flush=True)
with selectors.DefaultSelector() as sel:
sel.register(sys.stdin, selectors.EVENT_READ)
events = sel.select(timeout)
if not len(events):
print('Timed out after {}s.'.format(timeout))
return False
else:
s = sys.stdin.readline().rstrip('\n')
if s in ('Y', 'y', 'N', 'n', ''):
break
if s in ('Y', 'y'):
return True
else:
print('Cancelled.')
return False
if __name__=='__main__':
args = init_parser().parse_args()
repo = PackageRepo(args.package_name[0], '.')
commands = {
'prev-branch': prev_branch,
'next-branch': next_branch,
'current-branch': current_branch,
'is-same-version': is_same_version,
'fast-forward': fast_forward,
'can-reuse-binary': can_reuse_binary,
'prompt': prompt,
'sanity-check': sanity_check,
}
ret = commands[args.command](args, repo)
if ret is not None:
exit(0 if ret else 1)
View File
+97
View File
@@ -0,0 +1,97 @@
import unittest
import os, pathlib, tempfile, logging
import subprocess, functools
from ltsutils.package_repo import PackageRepo
import ltsutils.shell
run = functools.partial(subprocess.run, check=True)
class Package:
def __init__(self, name):
self.name = name
self.repo_url = 'https://github.com/clearlinux-pkgs/{}.git'.format(name)
class LTSUtilsTestCase(unittest.TestCase):
toplvl = pathlib.Path('../../..')
packages = toplvl / 'packages'
def cloneOrExtractRepo(self):
tmpdir = pathlib.Path('/var/tmp/common-lts-test')
tmpdir.mkdir(mode=0o700, exist_ok=True)
tarball = tmpdir / '{}.tar.gz'.format(self.package.name)
if tarball.exists():
run(['tar', 'xf', tarball, '-C', self.workdir])
else:
run(['git', 'clone', self.package.repo_url, self.workdir])
run(['tar', 'czf', tarball, '-C', self.workdir, '.'])
def setUp(self):
self._tmpdir = tempfile.TemporaryDirectory(prefix='test-{}-'.format(self.package.name), dir=self.packages)
self.workdir = pathlib.Path(self._tmpdir.name)
self.cloneOrExtractRepo()
self.repo = PackageRepo(self.package.name, self.workdir)
self._sh = ltsutils.shell.Shell(self.workdir)
self._sh.capture_output = False
def sh(self, cmd):
return self._sh.run(cmd)
def sh_stdout(self, cmd):
return self._sh.run(cmd, capture_output=True).stdout.rstrip()
def tearDown(self):
self._tmpdir.cleanup()
class PackageRepoTestCase(LTSUtilsTestCase):
def setUp(self):
super().setUp()
self.L1 = self.package.L1
self.L2 = self.package.L2
self.sh('git branch L1 %s' % self.L1)
self.sh('git branch L2 %s' % self.L2)
def testGetNVR(self):
raise NotImplementedError
def testHasBranch(self):
self.assertTrue(self.repo.hasBranch('L2'))
self.assertFalse(self.repo.hasBranch('L3'))
def testCheckoutBranch(self):
self.repo.checkoutBranch('L2')
self.assertEqual(self.sh_stdout('git rev-parse HEAD'), self.L2)
self.assertRaises(PackageRepo.InvalidBranchException, self.repo.checkoutBranch, 'L3')
def testFastForward(self):
self.repo.fastForwardBranch('L1', 'L2')
self.assertEqual(self.sh_stdout('git rev-parse L1'), self.L2)
self.assertEqual(self.sh_stdout('git rev-parse L2'), self.L2)
def testGetCurrentBranch(self):
self.repo.checkoutBranch('L2')
b = self.repo.getCurrentBranch()
self.assertEqual(b, 'L2')
self.sh('git checkout --detach L2')
self.assertRaises(PackageRepo.UnknownCurrentBranchException, self.repo.getCurrentBranch)
class TestNano(PackageRepoTestCase):
package = Package('nano')
package.L1 = '3dcfa09f5217eedf6ec7539af7e243655d3abdb6' # 3.2-54
package.L2 = 'b8243dd54e8feb16a11474f848b8735f5591cf12' # 3.2-55
def testGetNVR(self):
nvr = self.repo.getNVR(self.L2)
self.assertEqual(nvr, ('nano', '3.2', '55'))
class TestMySQL_Python(PackageRepoTestCase):
package = Package('MySQL-python')
package.L1 = '386163d8fc9c857c7194c4e958374af4c4f071ed' # 1.2.5-31
package.L2 = 'f85bc5ec2141384f45f224d4464a0a44a981a4d4' # 1.2.5-33
def testGetNVR(self):
nvr = self.repo.getNVR(self.L2)
self.assertEqual(nvr, ('MySQL-python', '1.2.5', '33'))
+393 -16
View File
File diff suppressed because it is too large Load Diff
+14 -1
View File
@@ -195,7 +195,7 @@ def main():
with open (filename, "r") as myfile:
lines = myfile.readlines()
else:
output = subprocess.check_output("git show", shell=True).decode("latin-1")
output = subprocess.check_output("git format-patch -1 --stdout", shell=True).decode("latin-1")
lines = output.split("\n")
parse_patch(lines)
@@ -225,6 +225,14 @@ def main():
zap_line_in_file_start(".spec", "Source99 :")
zap_line_in_file_start(".spec", "Source0 :")
zap_line_in_file_start(".spec", "export SOURCE_DATE_EPOCH")
zap_line_in_file_start(".spec", "export GCC_IGNORE_WERROR=1")
zap_line_in_file_start(".spec", "export AR=gcc-ar")
zap_line_in_file_start(".spec", "export RANLIB=gcc-ranlib")
zap_line_in_file_start(".spec", "export NM=gcc-nm")
zap_line_in_file_start(".spec", "export CFLAGS=\"$CFLAGS -O3 -ffat-lto-objects -flto=4 \"")
zap_line_in_file_start(".spec", "export FCFLAGS=\"$CFLAGS -O3 -ffat-lto-objects -flto=4 \"")
zap_line_in_file_start(".spec", "export FFLAGS=\"$CFLAGS -O3 -ffat-lto-objects -flto=4 \"")
zap_line_in_file_start(".spec", "export CXXFLAGS=\"$CXXFLAGS -O3 -ffat-lto-objects -flto=4 \"")
zap_line_in_file_start(".spec", "%setup -q -n ")
zap_line_in_file_start(".spec", "URL :")
zap_line_in_file_start(".spec", "Version :")
@@ -236,6 +244,7 @@ def main():
zap_line_in_file_start(".spec", "data components for the")
zap_line_in_file_start(".spec", "locales components for the")
zap_line_in_file_start(".spec", "license components for the")
zap_line_in_file_start(".spec", "doc components for the")
zap_line_in_file_start(".spec", "Group: Default")
zap_line_in_file_start(".spec", "export http_proxy=http://127.0.0.1:9/")
zap_line_in_file_start(".spec", "export https_proxy=http://127.0.0.1:9/")
@@ -246,13 +255,17 @@ def main():
zap_line_in_file_start(".spec", "Group: Binaries")
zap_line_in_file_start(".spec", "Group: Data")
zap_line_in_file_start(".spec", "Group: Libraries")
zap_line_in_file_start(".spec", "Group: Documentation")
zap_line_in_file_start(".spec", "export LANG=C")
zap_line_in_file_start(".spec", "%description lib")
zap_line_in_file_start(".spec", "%description bin")
zap_line_in_file_start(".spec", "%description data")
zap_line_in_file_start(".spec", "%description doc")
zap_line_in_file_start(".spec", "%description locales")
zap_line_in_file_start(".spec", "%description dev")
zap_line_in_file_start(".spec", "%description license")
zap_line_in_file_start(".spec", "%package doc")
zap_line_in_file_start(".spec", "cp -a ")
zap_line_in_file_start("b/Makefile", "URL =")
zap_line_in_file_start("b/Makefile", "PKG_NAME :=")
+2 -2
View File
@@ -145,9 +145,9 @@ if [ "$USE_KOJI" ]; then
cp "$CLIENTCA" ~/.koji/clientca.crt
cp "$SERVERCA" ~/.koji/serverca.crt
if [ ! -f /etc/koji.conf ]; then
if [ ! -f ~/.koji/config ]; then
echo "Setting up koji config . . ."
sudo cp projects/common/conf/koji.conf /etc
cp -f projects/common/conf/koji.conf ~/.koji/config
fi
fi