Page MenuHomeSoftware Heritage
Differential D4615

storage: Add branch names filtering support in snapshot_get_branches
ClosedPublic
Actions

Authored by anlambert on Nov 26 2020, 6:16 PM.

Details

Reviewers
vlorentz
Group Reviewers
Reviewers
Maniphest Tasks
T2782: add a "Filter Pull Requests" checkbox (or similar) in the Branches view of an origin in the web UI
Commits
rDSTO93301a1f67ac: storage: Add branch names filtering support in snapshot_get_branches
Summary

Add optional branch_name_include_substring parameter to snapshot_get_branches,
if provided only branches whose name contains the given substring will be
returned.

Add optional branch_name_exclude_prefix parameter to snapshot_get_branches,
if provided branches whose name starts with the given pprefix will not be
returned.

The purpose is to add branches filtering in swh-web. As that feature
could be useful elsewhere, I thought adding it at swh-storage level
was better than implementing it client side each time it is required.

Related to T2782

Diff Detail

Repository
rDSTO Storage manager
Branch
snapshot-branch-names-filtering
Lint
No Linters Available
Unit
No Unit Test Coverage
Build Status
Buildable 19769
Build 30692: Phabricator diff pipeline on jenkinsJenkins console · Jenkins
Build 30691: arc lint + arc unit

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes
vlorentz added a comment.Mar 4 2021, 2:45 PM

hmm, indeed, it looks like we don't have any index on (object_id, branch_name).

swh-public-ci added a comment.Mar 4 2021, 2:50 PM

Build is green

Patch application report for D4615 (id=18592)

Rebasing onto 88ff2c2fa0...

Current branch diff-target is up to date.
Changes applied before test
commit c7b6ca35f37f6634f50bd6c481ecf7d8d3ee0941
Author: Antoine Lambert <antoine.lambert@inria.fr>
Date:   Tue Mar 2 14:42:57 2021 +0100

    storage: Add branch names filtering support in snapshot_get_branches
    
    Add optional branch_name_include_pattern parameter to snapshot_get_branches,
    if provided only branches whose name contains the given pattern will be
    returned.
    
    Add optional branch_name_exclude_pattern parameter to snapshot_get_branches,
    if provided branches whose name contains the given pattern will not be
    returned.
    
    Related to T2782

See https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/1176/ for more details.

Harbormaster completed remote builds in B19684: Diff 18592.Mar 4 2021, 2:50 PM
anlambert added a comment.Mar 4 2021, 2:52 PM
In D4615#131611, @vlorentz wrote:

hmm, indeed, it looks like we don't have any index on (object_id, branch_name).

Indeed.

softwareheritage=> \d snapshot_branch
                                      Table "public.snapshot_branch"
   Column    |      Type       | Collation | Nullable |                      Default                       
-------------+-----------------+-----------+----------+----------------------------------------------------
 object_id   | bigint          |           | not null | nextval('snapshot_branch_object_id_seq'::regclass)
 name        | bytea           |           | not null | 
 target      | bytea           |           |          | 
 target_type | snapshot_target |           |          | 
Indexes:
    "snapshot_branch_pkey" PRIMARY KEY, btree (object_id)
    "snapshot_branch_name_idx" UNIQUE, btree (name) WHERE target_type IS NULL AND target IS NULL
    "snapshot_branch_name_target_target_type_idx" btree (name, target, target_type)
    "snapshot_branch_target_type_target_name_idx" UNIQUE, btree (target_type, target, name)
Check constraints:
    "snapshot_branch_target_check" CHECK ((target_type IS NULL) = (target IS NULL))
    "snapshot_target_check" CHECK ((target_type <> ALL (ARRAY['content'::snapshot_target, 'directory'::snapshot_target, 'revision'::snapshot_target, 'release'::snapshot_target, 'snapshot'::snapshot_target])) OR length(target) = 20)
Referenced by:
    TABLE "snapshot_branches" CONSTRAINT "snapshot_branches_branch_id_fkey" FOREIGN KEY (branch_id) REFERENCES snapshot_branch(object_id)
Publications:
    "softwareheritage"
anlambert updated this revision to Diff 18650.Mar 5 2021, 4:54 PM

Prefer to filter out branches by prefix only as the main use case of that
feature will be to filter out pull request branches (whose names start with
"/refs/pull/") in swh-web views.

Also it enables to count branches not starting with prefix in an efficient
way when using the cassandra backend (diff incoming).

swh-public-ci added a comment.Mar 5 2021, 4:59 PM

Build is green

Patch application report for D4615 (id=18650)

Rebasing onto 88ff2c2fa0...

Current branch diff-target is up to date.
Changes applied before test
commit 2f1fdfe322435b59d08bd56e4a521d01caddacd6
Author: Antoine Lambert <antoine.lambert@inria.fr>
Date:   Tue Mar 2 14:42:57 2021 +0100

    storage: Add branch names filtering support in snapshot_get_branches
    
    Add optional branch_name_include_pattern parameter to snapshot_get_branches,
    if provided only branches whose name contains the given pattern will be
    returned.
    
    Add optional branch_name_exclude_prefix parameter to snapshot_get_branches,
    if provided branches whose name starts with the given prefix will not be
    returned.
    
    Purpose of these new features: add a search form in the branches view
    of swh-web and filter out pull request branches (whose names start with
    "refs/pull/") by default.
    
    Related to T2782

See https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/1187/ for more details.

Harbormaster completed remote builds in B19743: Diff 18650.Mar 5 2021, 4:59 PM
anlambert added a child revision: D5208: storage: Allow to filter out branches by prefix when counting them.Mar 5 2021, 5:27 PM
anlambert updated this revision to Diff 18676.Mar 8 2021, 10:50 AM

Change branch_name_include_pattern and branch_name_exclude_prefix parameters type from str to bytes.

swh-public-ci added a comment.Mar 8 2021, 11:11 AM

Build was aborted

Patch application report for D4615 (id=18676)

Rebasing onto 88ff2c2fa0...

Current branch diff-target is up to date.
Changes applied before test
commit b1f0caaee70b5291fbe6b90f7ae1eea434ae821b
Author: Antoine Lambert <antoine.lambert@inria.fr>
Date:   Tue Mar 2 14:42:57 2021 +0100

    storage: Add branch names filtering support in snapshot_get_branches
    
    Add optional branch_name_include_pattern parameter to snapshot_get_branches,
    if provided only branches whose name contains the given pattern will be
    returned.
    
    Add optional branch_name_exclude_prefix parameter to snapshot_get_branches,
    if provided branches whose name starts with the given prefix will not be
    returned.
    
    Purpose of these new features: add a search form in the branches view
    of swh-web and filter out pull request branches (whose names start with
    "refs/pull/") by default.
    
    Related to T2782

Link to build: https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/1192/
See console output for more information: https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/1192/console

Harbormaster failed remote builds in B19769: Diff 18676!Mar 8 2021, 11:11 AM
anlambert mentioned this in D4616: common/archive: Add branch names filtering support in lookup_snapshot.Mar 8 2021, 11:19 AM
swh-public-ci added a comment.Mar 8 2021, 11:23 AM

Build is green

Patch application report for D4615 (id=18676)

Rebasing onto 88ff2c2fa0...

Current branch diff-target is up to date.
Changes applied before test
commit b1f0caaee70b5291fbe6b90f7ae1eea434ae821b
Author: Antoine Lambert <antoine.lambert@inria.fr>
Date:   Tue Mar 2 14:42:57 2021 +0100

    storage: Add branch names filtering support in snapshot_get_branches
    
    Add optional branch_name_include_pattern parameter to snapshot_get_branches,
    if provided only branches whose name contains the given pattern will be
    returned.
    
    Add optional branch_name_exclude_prefix parameter to snapshot_get_branches,
    if provided branches whose name starts with the given prefix will not be
    returned.
    
    Purpose of these new features: add a search form in the branches view
    of swh-web and filter out pull request branches (whose names start with
    "refs/pull/") by default.
    
    Related to T2782

See https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/1194/ for more details.

Harbormaster completed remote builds in B19769: Diff 18676.Mar 8 2021, 11:23 AM
anlambert edited the summary of this revision. (Show Details)Mar 9 2021, 5:26 PM
vlorentz accepted this revision.Mar 10 2021, 11:02 AM
This revision is now accepted and ready to land.Mar 10 2021, 11:02 AM
vlorentz added a comment.Mar 10 2021, 11:06 AM

do we really need branch_name_include_pattern, though?

It should at least be discouraged in the documentation

anlambert added a comment.Mar 10 2021, 11:23 AM
In D4615#132602, @vlorentz wrote:

do we really need branch_name_include_pattern, though?

It should at least be discouraged in the documentation

I intend to use it in swh-web by adding a search form in the branches view. But indeed performances are not great
with the cassandra backend as the filtering is done client side. With postgresql I did not observe any kind of performance
issues though.

vlorentz added a comment.Mar 10 2021, 1:05 PM

okay then

vlorentz added a comment.EditedMar 10 2021, 1:11 PM

EDIT: disregard this comment, I can't read

vlorentz requested changes to this revision.Mar 10 2021, 1:24 PM

You should be able to implement branch_name_exclude_prefix server-side on cassandra, by using two queries (one for the branches before, and one for the branches after the prefix)

This revision now requires changes to proceed.Mar 10 2021, 1:24 PM
anlambert added a comment.Mar 10 2021, 1:39 PM
In D4615#132626, @vlorentz wrote:

You should be able to implement branch_name_exclude_prefix server-side on cassandra, by using two queries (one for the branches before, and one for the branches after the prefix)

Ack, got it.

anlambert updated this revision to Diff 18738.EditedMar 10 2021, 6:07 PM

Filter out branches whose name starts with branch_name_exclude_prefix
server-side in the cassandra backend by adapting the query to scan branches:

  • branches in a [from_name, to_name[ range are now considered when scanning rows
  • when one wants to filter out branches whose name starts with a given prefix, the from_name and to_name values will be updated to not scan the rows whose name starts with the prefix
swh-public-ci added a comment.Mar 10 2021, 6:13 PM

Build is green

Patch application report for D4615 (id=18738)

Rebasing onto c4fdd6dbd0...

First, rewinding head to replay your work on top of it...
Applying: storage: Add branch names filtering support in snapshot_get_branches
Changes applied before test
commit 23380fe65e326ffbeaa4d9b56de88787e8ab4dd4
Author: Antoine Lambert <antoine.lambert@inria.fr>
Date:   Tue Mar 2 14:42:57 2021 +0100

    storage: Add branch names filtering support in snapshot_get_branches
    
    Add optional branch_name_include_pattern parameter to snapshot_get_branches,
    if provided only branches whose name contains the given pattern will be
    returned.
    
    Add optional branch_name_exclude_prefix parameter to snapshot_get_branches,
    if provided branches whose name starts with the given prefix will not be
    returned.
    
    Purpose of these new features: add a search form in the branches view
    of swh-web and filter out pull request branches (whose names start with
    "refs/pull/") by default.
    
    Related to T2782

See https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/1210/ for more details.

Harbormaster completed remote builds in B19833: Diff 18738.Mar 10 2021, 6:13 PM
vlorentz requested changes to this revision.Mar 11 2021, 1:27 PM
vlorentz added inline comments.
swh/storage/cassandra/storage.py
689–699

I'm confused. How does that return branches that are after the prefix?

693

b"\xff" is not the last possible branch name, there could be branches named b"\xff\xff"

730–731

would be easier by setting a variable before filtering on branch_name_include_pattern

732–740

what does this do?

This revision now requires changes to proceed.Mar 11 2021, 1:27 PM
anlambert planned changes to this revision.Mar 11 2021, 2:20 PM

Current cassandra implementation for filtering out branches whose name starts with a prefix will not work for prefix starting with \xff so two different CQL queries are needed after all, same for the branches count in D5208.

swh/storage/cassandra/storage.py
689–699

The snapshot branches CQL query have been updated to scan branches inside a range [from_branch_name, to_branch_name[.
The loop will start scanning rows inside the [branches_from, exclude_prefix[ then it will update the range to scan to `[next_exclude_prefix, b"\xff"[ when all branches before the prefix have been returned.

693

I guess I am forced to use two different CQL queries then, one with WHERE name < prefix and the other with WHERE name > prefix. I wanted to modify the less code as possible but looks like it is not possible to use a single query.

730–731

you mean for the right part of the if condition ?

732–740

It updates the names range to scan snapshot branches:

  • when all branches before the prefix have been scanned, the range is set to branches after the prefix
  • otherwise the lower bound of the range is updated
vlorentz added a comment.Mar 11 2021, 2:55 PM

It looks like you are using one loop to do one thing, then a different one; which complicates all the conditional inside because they have to check for two different things.

Maybe move the loop to a function and call it twice with different arguments?

anlambert added a comment.Mar 11 2021, 3:05 PM
In D4615#132785, @vlorentz wrote:

It looks like you are using one loop to do one thing, then a different one; which complicates all the conditional inside because they have to check for two different things.

Maybe move the loop to a function and call it twice with different arguments?

I took a wrong direction, the only solution is the one that you suggested at the beginning, split the CQL query into two when ones wants to filter out a branch name prefix.

I just finish to implement it at the cql module level which leads to few changes at the storage module level.

anlambert updated this revision to Diff 18748.Mar 11 2021, 3:09 PM

Update: Properly implement branch name prefix exclude filter for the cassandra backend.

swh-public-ci added a comment.Mar 11 2021, 3:13 PM

Build has FAILED

Patch application report for D4615 (id=18748)

Rebasing onto b8e10f00cf...

First, rewinding head to replay your work on top of it...
Applying: storage: Add branch names filtering support in snapshot_get_branches
Using index info to reconstruct a base tree...
M	swh/storage/cassandra/cql.py
M	swh/storage/cassandra/storage.py
M	swh/storage/in_memory.py
M	swh/storage/interface.py
M	swh/storage/postgresql/db.py
M	swh/storage/postgresql/storage.py
M	swh/storage/sql/30-schema.sql
M	swh/storage/sql/40-funcs.sql
M	swh/storage/tests/storage_tests.py
Falling back to patching base and 3-way merge...
Auto-merging swh/storage/tests/storage_tests.py
Auto-merging swh/storage/sql/40-funcs.sql
Auto-merging swh/storage/sql/30-schema.sql
Auto-merging swh/storage/postgresql/storage.py
Auto-merging swh/storage/postgresql/db.py
Auto-merging swh/storage/interface.py
Auto-merging swh/storage/in_memory.py
Auto-merging swh/storage/cassandra/storage.py
Auto-merging swh/storage/cassandra/cql.py
Auto-merging sql/upgrades/168.sql
CONFLICT (add/add): Merge conflict in sql/upgrades/168.sql
Patch failed at 0001 storage: Add branch names filtering support in snapshot_get_branches

Resolve all conflicts manually, mark them as resolved with
"git add/rm <conflicted_files>", then run "git rebase --continue".
You can instead skip this commit: run "git rebase --skip".
To abort and get back to the state before "git rebase", run "git rebase --abort".

Rebase failed (ret=1)!




Could not rebase; Attempt merge onto b8e10f00cf...



Already up to date.





Changes applied before test



commit df867d3c7c07f9f5944016a26da95c4b2dbb6698
Author: Antoine Lambert <antoine.lambert@inria.fr>
Date:   Tue Mar 2 14:42:57 2021 +0100

    storage: Add branch names filtering support in snapshot_get_branches
    
    Add optional branch_name_include_pattern parameter to snapshot_get_branches,
    if provided only branches whose name contains the given pattern will be
    returned.
    
    Add optional branch_name_exclude_prefix parameter to snapshot_get_branches,
    if provided branches whose name starts with the given prefix will not be
    returned.
    
    Purpose of these new features: add a search form in the branches view
    of swh-web and filter out pull request branches (whose names start with
    "refs/pull/") by default.
    
    Related to T2782



Link to build: https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/1214/

See console output for more information: https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/1214/console
Harbormaster failed remote builds in B19844: Diff 18748!Mar 11 2021, 3:13 PM
anlambert updated this revision to Diff 18749.Mar 11 2021, 3:38 PM
Rebase
swh-public-ci added a comment.Mar 11 2021, 3:52 PM
Build was aborted



Patch application report for D4615 (id=18749)



Rebasing onto b8e10f00cf...



Current branch diff-target is up to date.





Changes applied before test



commit 017055d705b3bd7b5cda2548d17e3afb2d9a75cd
Author: Antoine Lambert <antoine.lambert@inria.fr>
Date:   Tue Mar 2 14:42:57 2021 +0100

    storage: Add branch names filtering support in snapshot_get_branches
    
    Add optional branch_name_include_pattern parameter to snapshot_get_branches,
    if provided only branches whose name contains the given pattern will be
    returned.
    
    Add optional branch_name_exclude_prefix parameter to snapshot_get_branches,
    if provided branches whose name starts with the given prefix will not be
    returned.
    
    Purpose of these new features: add a search form in the branches view
    of swh-web and filter out pull request branches (whose names start with
    "refs/pull/") by default.
    
    Related to T2782



Link to build: https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/1215/

See console output for more information: https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/1215/console
swh-public-ci added a comment.Mar 11 2021, 4:03 PM
Build is green



Patch application report for D4615 (id=18749)



Rebasing onto b8e10f00cf...



Current branch diff-target is up to date.





Changes applied before test



commit 017055d705b3bd7b5cda2548d17e3afb2d9a75cd
Author: Antoine Lambert <antoine.lambert@inria.fr>
Date:   Tue Mar 2 14:42:57 2021 +0100

    storage: Add branch names filtering support in snapshot_get_branches
    
    Add optional branch_name_include_pattern parameter to snapshot_get_branches,
    if provided only branches whose name contains the given pattern will be
    returned.
    
    Add optional branch_name_exclude_prefix parameter to snapshot_get_branches,
    if provided branches whose name starts with the given prefix will not be
    returned.
    
    Purpose of these new features: add a search form in the branches view
    of swh-web and filter out pull request branches (whose names start with
    "refs/pull/") by default.
    
    Related to T2782



See https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/1217/ for more details.
Harbormaster completed remote builds in B19845: Diff 18749.Mar 11 2021, 4:03 PM
anlambert updated this revision to Diff 18754.Mar 11 2021, 7:21 PM
Update: Fix edge cases for branch name exclude filter in cassandra backend and update tests.
swh-public-ci added a comment.Mar 11 2021, 7:26 PM
Build is green



Patch application report for D4615 (id=18754)



Rebasing onto b8e10f00cf...



Current branch diff-target is up to date.





Changes applied before test



commit 09a017827eb31cbec05252a7b200fd6af245a0a8
Author: Antoine Lambert <antoine.lambert@inria.fr>
Date:   Tue Mar 2 14:42:57 2021 +0100

    storage: Add branch names filtering support in snapshot_get_branches
    
    Add optional branch_name_include_pattern parameter to snapshot_get_branches,
    if provided only branches whose name contains the given pattern will be
    returned.
    
    Add optional branch_name_exclude_prefix parameter to snapshot_get_branches,
    if provided branches whose name starts with the given prefix will not be
    returned.
    
    Purpose of these new features: add a search form in the branches view
    of swh-web and filter out pull request branches (whose names start with
    "refs/pull/") by default.
    
    Related to T2782



See https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/1218/ for more details.
Harbormaster completed remote builds in B19850: Diff 18754.Mar 11 2021, 7:27 PM
vlorentz requested changes to this revision.Mar 12 2021, 9:46 AM
vlorentz added inline comments.
swh/storage/cassandra/cql.py


644 ↗(On Diff #18754)


673–677 ↗(On Diff #18754)
can't you reuse the function from the other diff?


678–682 ↗(On Diff #18754)


swh/storage/interface.py


703
s/pattern/substring/ ?


swh/storage/tests/storage_tests.py


3044–3049
could be improved; it doesn't check that the same branch isn't returned multiple times, or bogus branches are returned.



What about something like this?



expected_branches = [
    branch_name for branch_name in snapshot.branches
    if include_pattern is None or include_pattern in branch_name
    and exclude_prefix is None or branch_name.startswith(exclude_prefix)
]
assert sorted(branches) == sorted(expected_branches)



It also makes pytest give better errors, showing what branches are missing/too much instead of just a number inequality


3074
why a regexp instead of in?


3090–3095
same comment as above on the assertions, same code should work if you add a slicing


3109–3114
ditto
This revision now requires changes to proceed.Mar 12 2021, 9:46 AM
anlambert added inline comments.Mar 12 2021, 10:55 AM
swh/storage/cassandra/cql.py


644 ↗(On Diff #18754)
yes, better name indeed


673–677 ↗(On Diff #18754)
The other diff depends on that diff and put that code in the _next_prefix function. 

As it is only use here in that diff, I did not wrap it in a function yet.


swh/storage/interface.py


703
ack, will rename


swh/storage/tests/storage_tests.py


3044–3049
ack


3074
oh right, I initially used regexps when I started that diff, this must have remain through the numerous code changes.


3090–3095
ack


3109–3114
ack
vlorentz added inline comments.Mar 12 2021, 11:28 AM
swh/storage/cassandra/cql.py


673–677 ↗(On Diff #18754)
ok
anlambert updated this revision to Diff 18780.Mar 12 2021, 3:20 PM
Update: Address latest @vlorentz comments.
anlambert edited the summary of this revision. (Show Details)Mar 12 2021, 3:22 PM
anlambert updated this revision to Diff 18782.Mar 12 2021, 3:24 PM
Update commit message
swh-public-ci added a comment.Mar 12 2021, 3:26 PM
Build is green



Patch application report for D4615 (id=18780)



Rebasing onto b8e10f00cf...



Current branch diff-target is up to date.





Changes applied before test



commit d080085e7ff75d41972e4106b76c02183fcce987
Author: Antoine Lambert <antoine.lambert@inria.fr>
Date:   Tue Mar 2 14:42:57 2021 +0100

    storage: Add branch names filtering support in snapshot_get_branches
    
    Add optional branch_name_include_pattern parameter to snapshot_get_branches,
    if provided only branches whose name contains the given pattern will be
    returned.
    
    Add optional branch_name_exclude_prefix parameter to snapshot_get_branches,
    if provided branches whose name starts with the given prefix will not be
    returned.
    
    Purpose of these new features: add a search form in the branches view
    of swh-web and filter out pull request branches (whose names start with
    "refs/pull/") by default.
    
    Related to T2782



See https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/1220/ for more details.
Harbormaster completed remote builds in B19878: Diff 18780.Mar 12 2021, 3:26 PM
swh-public-ci added a comment.Mar 12 2021, 4:03 PM
Build was aborted



Patch application report for D4615 (id=18782)



Rebasing onto b8e10f00cf...



Current branch diff-target is up to date.





Changes applied before test



commit 93301a1f67acba349319c383dac9031a132a7470
Author: Antoine Lambert <antoine.lambert@inria.fr>
Date:   Tue Mar 2 14:42:57 2021 +0100

    storage: Add branch names filtering support in snapshot_get_branches
    
    Add optional branch_name_include_substring parameter to snapshot_get_branches,
    if provided only branches whose name contains the given substring will be
    returned.
    
    Add optional branch_name_exclude_prefix parameter to snapshot_get_branches,
    if provided branches whose name starts with the given prefix will not be
    returned.
    
    Purpose of these new features: add a search form in the branches view
    of swh-web and filter out pull request branches (whose names start with
    "refs/pull/") by default.
    
    Related to T2782



Link to build: https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/1222/

See console output for more information: https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/1222/console
Harbormaster failed remote builds in B19880: Diff 18782!Mar 12 2021, 4:04 PM
swh-public-ci added a comment.Mar 12 2021, 4:15 PM
Build is green



Patch application report for D4615 (id=18782)



Rebasing onto b8e10f00cf...



Current branch diff-target is up to date.





Changes applied before test



commit 93301a1f67acba349319c383dac9031a132a7470
Author: Antoine Lambert <antoine.lambert@inria.fr>
Date:   Tue Mar 2 14:42:57 2021 +0100

    storage: Add branch names filtering support in snapshot_get_branches
    
    Add optional branch_name_include_substring parameter to snapshot_get_branches,
    if provided only branches whose name contains the given substring will be
    returned.
    
    Add optional branch_name_exclude_prefix parameter to snapshot_get_branches,
    if provided branches whose name starts with the given prefix will not be
    returned.
    
    Purpose of these new features: add a search form in the branches view
    of swh-web and filter out pull request branches (whose names start with
    "refs/pull/") by default.
    
    Related to T2782



See https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/1224/ for more details.
Harbormaster completed remote builds in B19880: Diff 18782.Mar 12 2021, 4:15 PM
vlorentz accepted this revision.Mar 15 2021, 10:19 AM
This revision is now accepted and ready to land.Mar 15 2021, 10:19 AM
Closed by commit rDSTO93301a1f67ac: storage: Add branch names filtering support in snapshot_get_branches (authored by anlambert).  ·  Explain WhyMar 15 2021, 11:25 AM
This revision was automatically updated to reflect the committed changes.
anlambert added a commit: rDSTO93301a1f67ac: storage: Add branch names filtering support in snapshot_get_branches.
Revision Contents
Changeset List
PathSizeCoverage (All)Coverage (Touched)
sql/
upgrades/
39 lines--
swh/
storage/
cassandra/
18 lines96%
Loading...
6 lines70%
Loading...
postgresql/
20 lines97%
Loading...
4 lines95%
Loading...
sql/
2 lines--
6 lines--
tests/
107 lines99%
Loading...
Diff 18676
View Options
sql/upgrades/168.sql
Loading...
View Options
swh/storage/cassandra/storage.py
Loading...
View Options
swh/storage/interface.py
Loading...
View Options
swh/storage/postgresql/db.py
Loading...
View Options
swh/storage/postgresql/storage.py
Loading...
View Options
swh/storage/sql/30-schema.sql
Loading...
View Options
swh/storage/sql/40-funcs.sql
Loading...
View Options
swh/storage/tests/storage_tests.py
Loading...
About · Developer information · Legal