Commit Graph

99974 Commits

Author SHA1 Message Date
Simon Glass
ba0c4008f2 patman: Add a simple database implementation
For recording series information, patman needs a database. Add a module
which uses sqlite3 for this. It has a basic schema, enough to support a
series subcommand.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-05-27 10:07:42 +01:00
Simon Glass
c9efedfc2f patman: Avoid touching the system or local config
Set the git variables to ensure that any local gitconfig is not used or
affected by the tests.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-05-27 10:07:42 +01:00
Simon Glass
dd16f0e262 patman: Add a fourth series for testing
Create one more series to use for testing. This one has four separate
patches.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-05-27 10:07:42 +01:00
Simon Glass
1be26d78cf patman: Move common test code into a new module
The func_test file is quite large. In order to allow new tests to be
added to a separate file, move the common test code into a separate
class, to be inherited by other classes.

Drop unnecessary imports in func_test

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-05-27 10:07:42 +01:00
Simon Glass
d5675b84c4 patman: Add a test for collecting a PATCH prefix
Add one more test for completeness, since this occurs commonly.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-05-27 10:07:42 +01:00
Simon Glass
77177ba82f patman: Use python3 with /usr/bin/env
If python2 is available then we must specify python3 here. Update the
test to handle this.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-05-27 10:07:42 +01:00
Simon Glass
31a1c4c9d8 u_boot_pylib: Add more functions to gitutil
Add functions for checking a branch, showing a commit, etc. to support
the new functionality.

Git version 2.34.1 ignores --stat if --quiet is given, so adjust the
args so that this performs as expected.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-05-27 10:07:42 +01:00
Simon Glass
212ed6bdb7 patman: Deal with git safe-directory warning
When running tests where the .git directory is not owned by the current
user, various warnings are produced and the tests fail. This happens in
CI.

For patman itself, modify the gitutil.get_top_level() function to return
None in this case. Ensure that the warning is not shown, since it creates
about 1000 lines of output.

For checkpatch, the same warning is produced even though --no-tree is
given. Suppress that as well.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-05-27 10:07:42 +01:00
Simon Glass
45f239afa1 u_boot_pylib: Correct pylint warnings in gitutil
Correct various pylint warnings in this file.

The remaining ones are three functions with too many arguments (R0913
and R0918) and use of global (W0603).

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-05-27 10:07:42 +01:00
Simon Glass
2e680f6765 patman: Support collecting all lines in the commit message
In some cases we want to collect all lines in the commit message so that
the commit can be recreated with the same message as before, or perhaps
with light filtering.

Add support for this.

Series-to: u-boot
Cover-letter:
patman: Minor improvements to prepare for series handling
This series includes a number of internal improvements to patman:
- Tidy-up of parsing
- Adjust how tests create the git tree
- Support for creating patches in a different git directory
- Faster determination of the upstream branch
- Ability to collect the body of a commit message as a series of lines
END

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-05-27 10:07:42 +01:00
Simon Glass
07dcd1f9b7 patman: Tidy up function comments in the series module
This module is missing some comments, so add them.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-05-27 10:07:42 +01:00
Simon Glass
2bdb98a282 patman: Move -H out of the send command
This is the help for the whole of patman, so move it to the start of the
control function, rather than being inside 'patman send'.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-05-27 10:07:42 +01:00
Simon Glass
9126e16e63 patman: Move arguments for sent into the correct parser
Most of the arguments for the main parser are actually arguments for the
'send' parser. Move them there, in a separate function.

Fix a pylint warning for -D and the imports while here.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-05-27 10:07:42 +01:00
Simon Glass
828e84ac37 patman: Split subparsers into their own functions
Simplify the main parser by moving subparser code into separate
functions. Fix a few pylint warnings while here.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-05-27 10:07:42 +01:00
Simon Glass
b77a42bc75 patman: Add tests for help
Add a few tests to make sure that help is provided when requested.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-05-27 10:07:42 +01:00
Simon Glass
79addcc904 patman: Split parser creation from parsing
Tests may want to parse their own arguments. Refactor the parser code to
support this and allow settings to receive arguments as well.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-05-27 10:07:42 +01:00
Simon Glass
a52e80aee7 patman: Correct some pylint and asyncio issues
Correct some pylint warnings in this file. Make use of the existing
asyncio event loop, instead of creating a new one, since this also
destroys it afterwards, making it unavailable for tests which want to
share an event loop. Use tools.write_file() to avoid a warning about
encoding.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-05-27 10:07:42 +01:00
Simon Glass
8465b44cb5 patman: Use git to set up the test config
At present the branch information is dropped when writing the
configuration. It is easier to get git to set up the config anyway, so
update the test to do this.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-05-27 10:07:42 +01:00
Simon Glass
3724cbade4 patman: Clean up creation of the git tree
The test starts with the HEAD pointing to the wrong place, so that the
created files appear to be deleted. Fix this by resetting the tree
before tests start. Add a check that the tree is clean.

Update pygit2 so that the enums are available.
2025-05-27 10:07:42 +01:00
Simon Glass
9302d672a5 patman: Use variables for series ID and title
Rather than hard-coding these values in the sample patches, use
variables so that we can refer to these in tests.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-05-27 10:07:42 +01:00
Simon Glass
6961eb7e5b patman: Use .git as the git directory
In tests, the 'git' directory is a subdirectory of the temporary
directory. Rename it to '.git' so that git will automatically find it
when git operations are done in the temporary directory. Set up the
config before the first git operation, so that this works correctly.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-05-27 10:07:42 +01:00
Simon Glass
8e737d41c0 patman: Use itr instead of lines for iterator
The variables 'lines' is used to hold a list of output lines within a
test, but also to hold an iterator through those lines. Use 'itr' for
the latter, to avoid confusion.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-05-27 10:07:42 +01:00
Simon Glass
8896247b58 patman: Correct use of str in code
Since str() is a reserved function we should not use it as a variable.
Fix this in the send module.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-05-27 10:07:42 +01:00
Simon Glass
a5afb75c6d patman: Allow setting a git directory when sending
Support specifying the git-directory when creating and sending patches.
This will allow better testing of this functionality, since we can use a
test directory.

For count_commits_to_branch() support an end commit while we are here.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-05-27 10:07:42 +01:00
Simon Glass
60b8709b98 patman: Allow setting the current directory when sending
Plumb a current-working-directory (cwd) through from send all the way to
the command gitutil libraries. This will allow better testing of this
functionality, since we can use a test directory.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-05-27 10:07:42 +01:00
Simon Glass
4f7bd6cae8 patman: Add all files to __init__.py
Some files are missing from the __all__ list, so add then. Reformat the
list to use more of the width of each line.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-05-27 10:07:42 +01:00
Simon Glass
afea95f653 u_boot_pylib: Speed up determining the upstream branch
Use --decorate to quickly detect the upstream branch, since this is much
faster than using 'git name-rev' on every possible commit.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-05-27 10:07:42 +01:00
Simon Glass
c885917c71 u_boot_pylib: Provide directories to gitutil functions
For testing it is useful to be able to set the current directory used
for git operations, as well as the git-repo directory. Update some of
the functions to support this.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-05-27 10:07:42 +01:00
Simon Glass
0664a956e6 u_boot_pylib: Tidy up quoting of cc and to
The current approach to calling 'git send-email' puts double quotes
around each email address to ensure that it will pass the shell
correctly. This is a bit cumbersome and requires using a shell to sort
it all out.

Drop the quotes and use command.run() instead, to simplify things. This
will also make it possible to (later) set the current directory.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-05-27 10:07:42 +01:00
Simon Glass
db5d98de55 u_boot_pylib: Support a fatal level in tout
It is convenient to be able to print a message and exit. Add a new
'fatal' level to support this.

Update some assumptions about the level, so that the tools continue to
work as now.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-05-27 10:07:42 +01:00
Simon Glass
75ae217194 patman: Update Series.ShowActions() to pass alias
Instead of using settings.alias pass this value in. This allows tests to
work without using settings.alias

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-05-27 10:07:42 +01:00
Simon Glass
abfd67cccf patman: Pass aliases to Series.MakeCcFile()
Rather than accessing settings directly, pass the aliases in, so that
we can do the same from tests. With further work this will allow the
tests to work without using settings.alias

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-05-27 10:07:41 +01:00
Simon Glass
e10201aa8c patman: Pass the alias dict into gitutil.email_patches()
Rather than accessing the settings module in this function, require the
alias dict to be passed in.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-05-27 10:07:41 +01:00
Simon Glass
7dc55435b2 patman: Pass the alias dict into gitutil.build_email_list()
Rather than accessing the settings module in this function, require the
alias dict to be passed in.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-05-27 10:07:41 +01:00
Simon Glass
e70fdbd7e4 patman: Untangle settings from gitutil
The gitutil module is supposed to be independent from patman but one
piece was missed in the series which separated them.

Move the settings setup out of gitutil

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-05-27 10:07:41 +01:00
Simon Glass
f9b84f9141 patman: Drop the old status code
Now that we have the new asyncio implementation, drop the unused
ThreadPoolExecutor code.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-05-27 10:07:41 +01:00
Simon Glass
605717405f patman: Use a pager when outputting status
The pager makes it easier to review large number of comments. Add it
around the output.

Patman normally sends multiple concurrent requests to the patchwork
server, as this is faster. Provide an option to disable this.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-05-27 10:07:41 +01:00
Simon Glass
52aef33f95 patman: Provide an option to run in single-threaded mode
Patman normally sends multiple concurrent requests to the patchwork
server, as this is faster. Provide an option to disable this.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-05-27 10:07:41 +01:00
Simon Glass
45f4f62191 patman: Switch over to asyncio
Now that all the pieces are in place, switch over from using an executor
to using asyncio.

While we are here, import defaultdict to avoid needing to specify its
module.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-05-27 10:07:41 +01:00
Simon Glass
39f258494f patman: Use defaultdict directly
Import defaultdict to avoid needing to specify its module in multiple
places. Fix an extra blank line while here.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-05-27 10:07:41 +01:00
Simon Glass
0fb0b46200 patman: Add more information to Patch
The cover letter has some information on each patch, so allow this to be
stored in the Patch object.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-05-27 10:07:41 +01:00
Simon Glass
7052173fae patman: Show patches in yellow
When comments are shown below patches it can be hard to see the patch
subject. Use yellow instead of blue, since it stands out better.

Pass the colour object into show_responses() rather than creating a new
one, since that is tidier.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-05-27 10:07:41 +01:00
Simon Glass
289f4242c5 patman: Simplify test_find_new_responses()
This test uses the find_new_responses() function which combines
accessing patchwork and processing the results.

Since the test is aimed at testing processing, adjust it to call
process_reviews() instead.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-05-27 10:07:41 +01:00
Simon Glass
3fd99e2177 patman: Adjust how the fake request() function is provided
Instead of passing the URL and function to each call, put the fake
into the Patchwork object instead.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-05-27 10:07:41 +01:00
Simon Glass
3c5f27ce12 patman: Split out the processing of find_new_responses()
This function does patchwork calls and then processes the results. Move
the processing out so we can (later) test it separately.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-05-27 10:07:41 +01:00
Simon Glass
d63b075062 patman: Split up check_and_show_status()
This function has three phases:

- collecting things from patchwork
- doing some processing
- showing the results to the user / creating a branch

Refactor into two functions so we can eventually have the patchwork part
fully separated out.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-05-27 10:07:41 +01:00
Simon Glass
e2a991398d patman: Rename check_patchwork_status()
This function actually shows the status and does some other things.

Rename it to better reflect its purpose.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-05-27 10:07:41 +01:00
Simon Glass
d65490650f patman: Add reading of series and patch status
Expand the patchwork module so that it can match the current
requirements of the 'patman status' command, i.e. reading the state of a
series and the patches associated with it.

Since the format of each patchwork response is a little tricky to
understand, add examples in comments at the top of each function.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-05-27 10:07:41 +01:00
Simon Glass
24776ab276 patman: Move Patch and Review to patchwork module
These relate to information obtained from the patchwork server, so move
their definition into the new patchwork module.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-05-27 10:07:41 +01:00
Simon Glass
aef44950db patman: Create a module for handling patchwork
At present the patchwork implementation is very simple, just consisting
of a function which calls the REST API.

We want to create a fake patchwork for use in tests. So start a new
module to encapsulate communication with the patchwork server.

Use asyncio since it is easier to handle lots of concurrent requests
from different parts of the code.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-05-27 10:07:41 +01:00