Page MenuHomeSoftware Heritage
Differential D4795

FUSE: cache: add support to remove individual objects
ClosedPublic
Actions

Authored by haltode on Dec 28 2020, 3:13 PM.

Details

Reviewers
zack
Group Reviewers
Reviewers
Maniphest Tasks
T2889: FUSE: cache: support removing individual objects
Commits
rDFUSEbf87c60dd000: cache: add support to remove individual objects
Summary

Invalidate specific cached artifact using rm cache/{SWHID}

Fix T2889.

Diff Detail

Repository
rDFUSE FUSE virtual file system
Branch
feature/cache-rm-object
Lint
No Linters Available
Unit
No Unit Test Coverage
Build Status
Buildable 18110
Build 27961: Phabricator diff pipeline on jenkinsJenkins console · Jenkins
Build 27960: arc lint + arc unit

Event Timeline

haltode created this revision.Dec 28 2020, 3:13 PM
Herald added a reviewer: Reviewers. · View Herald TranscriptDec 28 2020, 3:13 PM
swh-public-ci added a comment.Dec 28 2020, 3:17 PM

Build is green

Patch application report for D4795 (id=16971)

Rebasing onto 4b304d38ee...

Current branch diff-target is up to date.
Changes applied before test
commit 393876babe7b8bd14e5538e4c852d60c4f37f5fe
Author: Thibault Allançon <haltode@gmail.com>
Date:   Thu Dec 17 16:48:40 2020 +0100

    cache: add support to remove individual objects
    
    Invalidate specific cached artifact using `rm cache/{SWHID}`
    
    Fix T2889.

See https://jenkins.softwareheritage.org/job/DFUSE/job/tests-on-diff/250/ for more details.

Harbormaster completed remote builds in B18108: Diff 16971.Dec 28 2020, 3:17 PM
zack requested changes to this revision.Dec 28 2020, 4:09 PM
zack added a subscriber: zack.
zack added inline comments.
swh/fuse/fs/entry.py
63–64

do you really need to implement this here in this base class?

if it's just for documenting in the code that some sub-classes will override this, you can just raise NotImplementedError, as you do above

if this should really raise another error, than the error message here is incorrect, as it's not the entire filesystem that is read-only, but only most dirs in it

142

I know you asked me about this, but now that i rethink of it, permissions on symlinks usually don't matter, because they're never used for anything at all.
Did you have a reason to change this? If not, please revert this change.

swh/fuse/fuse.py
327–328

don't you need to also raise a FUSEError here, in case of exception? or am I missing the reason why it's not needed?

swh/fuse/tests/test_cache.py
21

s/invalidate/purge/

(as "cache invalidation" means a different thing)

33

This is a very surface-thing test, you're only testing that the directory entry is gone, not that the corresponding entry has actually been removed from the cache (which would be the actual underlying behavior).
Can you add that part, or is it too hidden/not reachable from the outside?

Unrelated question: is the FUSE cache emptied for sure before starting this test? (e.g., by test setup that creates a new instance) Because if it isn't you assertion here might fail if, for reasons unrelated to this test, there are other entries in the cache.

This revision now requires changes to proceed.Dec 28 2020, 4:09 PM
haltode marked 3 inline comments as done.Dec 28 2020, 6:28 PM
haltode added inline comments.
swh/fuse/fs/entry.py
142

I only changed the name to be consistent with RDONLY_FILE and RDONLY_DIR.

swh/fuse/tests/test_cache.py
33

I only have access to the path of the mounting dir in the unit tests, but since the cache/ listing is pretty much 1. read cache 2. create entries with sharded prefix, it should reflect directly the cache content.

For unit tests the cache is set to be created in-memory in the fuse_mntdir fixture, so yes it is empty everytime a test start.

haltode updated this revision to Diff 16972.Dec 28 2020, 6:29 PM

Fix zack comments

swh-public-ci added a comment.Dec 28 2020, 6:32 PM

Build is green

Patch application report for D4795 (id=16972)

Rebasing onto ae3a0e9760...

First, rewinding head to replay your work on top of it...
Applying: cache: add support to remove individual objects
Changes applied before test
commit 872f11b543e767a1cfa198975755474f3168ff8e
Author: Thibault Allançon <haltode@gmail.com>
Date:   Thu Dec 17 16:48:40 2020 +0100

    cache: add support to remove individual objects
    
    Invalidate specific cached artifact using `rm cache/{SWHID}`
    
    Fix T2889.

See https://jenkins.softwareheritage.org/job/DFUSE/job/tests-on-diff/251/ for more details.

Harbormaster completed remote builds in B18110: Diff 16972.Dec 28 2020, 6:32 PM
zack accepted this revision.Dec 28 2020, 7:49 PM
zack added inline comments.
swh/fuse/fuse.py
330

Uhm, i'm not super-happy about always raising ENOENT in case of errors, as there might be other failure reasons, but looking around I see it's a common pattern also for other methods. So adding another instance won't make it worse.

This revision is now accepted and ready to land.Dec 28 2020, 7:49 PM
haltode updated this revision to Diff 16974.Dec 29 2020, 10:15 AM

Rebasing on master

swh-public-ci added a comment.Dec 29 2020, 10:19 AM

Build is green

Patch application report for D4795 (id=16974)

Rebasing onto ae3a0e9760...

Current branch diff-target is up to date.
Changes applied before test
commit bf87c60dd0003d9658ebe193a45f9bc0587e4887
Author: Thibault Allançon <haltode@gmail.com>
Date:   Thu Dec 17 16:48:40 2020 +0100

    cache: add support to remove individual objects
    
    Invalidate specific cached artifact using `rm cache/{SWHID}`
    
    Fix T2889.

See https://jenkins.softwareheritage.org/job/DFUSE/job/tests-on-diff/252/ for more details.

Harbormaster completed remote builds in B18112: Diff 16974.Dec 29 2020, 10:19 AM
Closed by commit rDFUSEbf87c60dd000: cache: add support to remove individual objects (authored by haltode). · Explain WhyDec 29 2020, 10:21 AM
This revision was automatically updated to reflect the committed changes.
haltode added a commit: rDFUSEbf87c60dd000: cache: add support to remove individual objects.

Revision Contents
Changeset List

PathSizeCoverage (All)Coverage (Touched)
swh/
fuse/
14 lines89%
Loading...
fs/
11 lines94%
Loading...
12 lines90%
Loading...
20 lines83%
Loading...
tests/
33 lines100%
Loading...

Diff 16972

View Options

swh/fuse/cache.py

  • This file was added.
/**
* Copyright (C) 2020 The Software Heritage developers
* See the AUTHORS file at the top-level directory of this distribution
* License: GNU Affero General Public License version 3, or any later version
* See top-level LICENSE file for more information
*/
const username = 'admin';
const password = 'admin';
function login(username, password) {
cy.get('input[name="username"]')
.type(username)
.get('input[name="password"]')
.type(password)
.get('form')
.submit();
}
describe('Test admin deposit page', function() {
it('Should test deposit page', function() {
cy.visit(this.Urls.admin_deposit());
// FIXME: cypress anti-pattern, do not use ui to log ¯\_(ツ)_/¯
// https://docs.cypress.io/guides/getting-started/testing-your-app.html#Logging-in
login(username, password);
cy.server();
let inputDeposits = [
{
'id': 614,
'external_id': 'c-d-1',
'reception_date': '2020-05-18T13:48:27Z',
'status': 'done',
'status_detail': 'all good',
'swh_id': 'swh:1:dir:ef04a768',
'swh_id_context': 'swh:1:dir:ef04a768;origin=https://w.s.o/c-d-1;visit=swh:1:snp:b234be1e;anchor=swh:1:rev:d24a75c9;path=/'
},
{
'id': 613,
'external_id': 'c-d-2',
'reception_date': '2020-05-18T11:20:16Z',
'status': 'done',
'status_detail': 'everything is fine',
'swh_id': 'swh:1:dir:181417fb',
'swh_id_context': 'swh:1:dir:181417fb;origin=https://w.s.o/c-d-2;visit=swh:1:snp:8c32a2ef;anchor=swh:1:rev:3d1eba04;path=/'
},
{
'id': 612,
'external_id': 'c-d-3',
'reception_date': '2020-05-18T11:20:16Z',
'status': 'rejected',
'status_detail': 'hu ho, issue!',
'swh_id': null,
'swh_id_context': null
}
];
cy.route({
method: 'GET',
url: `${this.Urls.admin_deposit_list()}**`,
response: {
'draw': 10,
'recordsTotal': 3,
'recordsFiltered': 3,
'data': inputDeposits
}
}).as('listDeposits');
cy.location('pathname')
.should('be.equal', this.Urls.admin_deposit());
cy.url().should('include', '/admin/deposit');
cy.get('#swh-admin-deposit-list')
.should('exist');
// those are computed from the
let expectedOrigins = ['https://w.s.o/c-d-1', 'https://w.s.o/c-d-2', ''];
cy.wait('@listDeposits').then((xhr) => {
cy.log('response:', xhr.response);
cy.log(xhr.response.body);
let deposits = xhr.response.body.data;
cy.log('Deposits: ', deposits);
expect(deposits.length).to.equal(inputDeposits.length);
cy.get('#swh-admin-deposit-list').find('tbody > tr').as('rows');
// only 2 entries
cy.get('@rows').each((row, idx, collection) => {
let deposit = deposits[idx];
let inputDeposit = inputDeposits[idx];
assert.isNotNull(deposit);
assert.isNotNull(inputDeposit);
expect(deposit.id).to.be.equal(inputDeposit['id']);
expect(deposit.external_id).to.be.equal(inputDeposit['external_id']);
expect(deposit.status).to.be.equal(inputDeposit['status']);
expect(deposit.status_detail).to.be.equal(inputDeposit['status_detail']);
expect(deposit.swh_id).to.be.equal(inputDeposit['swh_id']);
expect(deposit.swh_id_context).to.be.equal(inputDeposit['swh_id_context']);
let expectedOrigin = expectedOrigins[idx];
// ensure it's in the dom
expect(row).to.contain(deposit.id);
if (deposit.status === 'done') {
expect(row).to.contain(deposit.external_id);
} else {
expect(row).to.not.contain(deposit.external_id);
}
expect(row).to.contain(deposit.status);
expect(row).to.contain(expectedOrigin);
// those are hidden by default
expect(row).to.not.contain(deposit.status_detail);
expect(row).to.not.contain(deposit.swh_id);
expect(row).to.not.contain(deposit.swh_id_context);
});
// toggling all links and ensure, the previous checks are inverted
cy.get('a.toggle-col').click({'multiple': true}).then(() => {
cy.get('#swh-admin-deposit-list').find('tbody > tr').as('rows');
cy.get('@rows').each((row, idx, collection) => {
let deposit = deposits[idx];
// ensure it's in the dom
expect(row).to.not.contain(deposit.id);
if (deposit.status === 'done') {
expect(row).to.contain(deposit.external_id);
} else {
expect(row).to.not.contain(deposit.external_id);
}
expect(row).to.not.contain(deposit.status);
// those are hidden by default
expect(row).to.contain(deposit.status_detail);
if (deposit.swh_id !== null) {
expect(row).to.contain(deposit.swh_id);
expect(row).to.contain(deposit.swh_id_context);
}
});
});
cy.get('#swh-admin-deposit-list-error')
.should('not.contain',
'An error occurred while retrieving the list of deposits');
});
});
});
View Options

swh/fuse/fs/entry.py

  • This file was added.
/**
* Copyright (C) 2020 The Software Heritage developers
* See the AUTHORS file at the top-level directory of this distribution
* License: GNU Affero General Public License version 3, or any later version
* See top-level LICENSE file for more information
*/
const username = 'admin';
const password = 'admin';
function login(username, password) {
cy.get('input[name="username"]')
.type(username)
.get('input[name="password"]')
.type(password)
.get('form')
.submit();
}
describe('Test admin deposit page', function() {
it('Should test deposit page', function() {
cy.visit(this.Urls.admin_deposit());
// FIXME: cypress anti-pattern, do not use ui to log ¯\_(ツ)_/¯
// https://docs.cypress.io/guides/getting-started/testing-your-app.html#Logging-in
login(username, password);
cy.server();
let inputDeposits = [
{
'id': 614,
'external_id': 'c-d-1',
'reception_date': '2020-05-18T13:48:27Z',
'status': 'done',
'status_detail': 'all good',
'swh_id': 'swh:1:dir:ef04a768',
'swh_id_context': 'swh:1:dir:ef04a768;origin=https://w.s.o/c-d-1;visit=swh:1:snp:b234be1e;anchor=swh:1:rev:d24a75c9;path=/'
},
{
'id': 613,
'external_id': 'c-d-2',
'reception_date': '2020-05-18T11:20:16Z',
'status': 'done',
'status_detail': 'everything is fine',
'swh_id': 'swh:1:dir:181417fb',
'swh_id_context': 'swh:1:dir:181417fb;origin=https://w.s.o/c-d-2;visit=swh:1:snp:8c32a2ef;anchor=swh:1:rev:3d1eba04;path=/'
},
{
'id': 612,
'external_id': 'c-d-3',
'reception_date': '2020-05-18T11:20:16Z',
'status': 'rejected',
'status_detail': 'hu ho, issue!',
'swh_id': null,
'swh_id_context': null
}
];
cy.route({
method: 'GET',
url: `${this.Urls.admin_deposit_list()}**`,
response: {
'draw': 10,
'recordsTotal': 3,
'recordsFiltered': 3,
'data': inputDeposits
}
}).as('listDeposits');
cy.location('pathname')
.should('be.equal', this.Urls.admin_deposit());
cy.url().should('include', '/admin/deposit');
cy.get('#swh-admin-deposit-list')
.should('exist');
// those are computed from the
let expectedOrigins = ['https://w.s.o/c-d-1', 'https://w.s.o/c-d-2', ''];
cy.wait('@listDeposits').then((xhr) => {
cy.log('response:', xhr.response);
cy.log(xhr.response.body);
let deposits = xhr.response.body.data;
cy.log('Deposits: ', deposits);
expect(deposits.length).to.equal(inputDeposits.length);
cy.get('#swh-admin-deposit-list').find('tbody > tr').as('rows');
// only 2 entries
cy.get('@rows').each((row, idx, collection) => {
let deposit = deposits[idx];
let inputDeposit = inputDeposits[idx];
assert.isNotNull(deposit);
assert.isNotNull(inputDeposit);
expect(deposit.id).to.be.equal(inputDeposit['id']);
expect(deposit.external_id).to.be.equal(inputDeposit['external_id']);
expect(deposit.status).to.be.equal(inputDeposit['status']);
expect(deposit.status_detail).to.be.equal(inputDeposit['status_detail']);
expect(deposit.swh_id).to.be.equal(inputDeposit['swh_id']);
expect(deposit.swh_id_context).to.be.equal(inputDeposit['swh_id_context']);
let expectedOrigin = expectedOrigins[idx];
// ensure it's in the dom
expect(row).to.contain(deposit.id);
if (deposit.status === 'done') {
expect(row).to.contain(deposit.external_id);
} else {
expect(row).to.not.contain(deposit.external_id);
}
expect(row).to.contain(deposit.status);
expect(row).to.contain(expectedOrigin);
// those are hidden by default
expect(row).to.not.contain(deposit.status_detail);
expect(row).to.not.contain(deposit.swh_id);
expect(row).to.not.contain(deposit.swh_id_context);
});
// toggling all links and ensure, the previous checks are inverted
cy.get('a.toggle-col').click({'multiple': true}).then(() => {
cy.get('#swh-admin-deposit-list').find('tbody > tr').as('rows');
cy.get('@rows').each((row, idx, collection) => {
let deposit = deposits[idx];
// ensure it's in the dom
expect(row).to.not.contain(deposit.id);
if (deposit.status === 'done') {
expect(row).to.contain(deposit.external_id);
} else {
expect(row).to.not.contain(deposit.external_id);
}
expect(row).to.not.contain(deposit.status);
// those are hidden by default
expect(row).to.contain(deposit.status_detail);
if (deposit.swh_id !== null) {
expect(row).to.contain(deposit.swh_id);
expect(row).to.contain(deposit.swh_id_context);
}
});
});
cy.get('#swh-admin-deposit-list-error')
.should('not.contain',
'An error occurred while retrieving the list of deposits');
});
});
});
View Options

swh/fuse/fs/mountpoint.py

  • This file was added.
/**
* Copyright (C) 2020 The Software Heritage developers
* See the AUTHORS file at the top-level directory of this distribution
* License: GNU Affero General Public License version 3, or any later version
* See top-level LICENSE file for more information
*/
const username = 'admin';
const password = 'admin';
function login(username, password) {
cy.get('input[name="username"]')
.type(username)
.get('input[name="password"]')
.type(password)
.get('form')
.submit();
}
describe('Test admin deposit page', function() {
it('Should test deposit page', function() {
cy.visit(this.Urls.admin_deposit());
// FIXME: cypress anti-pattern, do not use ui to log ¯\_(ツ)_/¯
// https://docs.cypress.io/guides/getting-started/testing-your-app.html#Logging-in
login(username, password);
cy.server();
let inputDeposits = [
{
'id': 614,
'external_id': 'c-d-1',
'reception_date': '2020-05-18T13:48:27Z',
'status': 'done',
'status_detail': 'all good',
'swh_id': 'swh:1:dir:ef04a768',
'swh_id_context': 'swh:1:dir:ef04a768;origin=https://w.s.o/c-d-1;visit=swh:1:snp:b234be1e;anchor=swh:1:rev:d24a75c9;path=/'
},
{
'id': 613,
'external_id': 'c-d-2',
'reception_date': '2020-05-18T11:20:16Z',
'status': 'done',
'status_detail': 'everything is fine',
'swh_id': 'swh:1:dir:181417fb',
'swh_id_context': 'swh:1:dir:181417fb;origin=https://w.s.o/c-d-2;visit=swh:1:snp:8c32a2ef;anchor=swh:1:rev:3d1eba04;path=/'
},
{
'id': 612,
'external_id': 'c-d-3',
'reception_date': '2020-05-18T11:20:16Z',
'status': 'rejected',
'status_detail': 'hu ho, issue!',
'swh_id': null,
'swh_id_context': null
}
];
cy.route({
method: 'GET',
url: `${this.Urls.admin_deposit_list()}**`,
response: {
'draw': 10,
'recordsTotal': 3,
'recordsFiltered': 3,
'data': inputDeposits
}
}).as('listDeposits');
cy.location('pathname')
.should('be.equal', this.Urls.admin_deposit());
cy.url().should('include', '/admin/deposit');
cy.get('#swh-admin-deposit-list')
.should('exist');
// those are computed from the
let expectedOrigins = ['https://w.s.o/c-d-1', 'https://w.s.o/c-d-2', ''];
cy.wait('@listDeposits').then((xhr) => {
cy.log('response:', xhr.response);
cy.log(xhr.response.body);
let deposits = xhr.response.body.data;
cy.log('Deposits: ', deposits);
expect(deposits.length).to.equal(inputDeposits.length);
cy.get('#swh-admin-deposit-list').find('tbody > tr').as('rows');
// only 2 entries
cy.get('@rows').each((row, idx, collection) => {
let deposit = deposits[idx];
let inputDeposit = inputDeposits[idx];
assert.isNotNull(deposit);
assert.isNotNull(inputDeposit);
expect(deposit.id).to.be.equal(inputDeposit['id']);
expect(deposit.external_id).to.be.equal(inputDeposit['external_id']);
expect(deposit.status).to.be.equal(inputDeposit['status']);
expect(deposit.status_detail).to.be.equal(inputDeposit['status_detail']);
expect(deposit.swh_id).to.be.equal(inputDeposit['swh_id']);
expect(deposit.swh_id_context).to.be.equal(inputDeposit['swh_id_context']);
let expectedOrigin = expectedOrigins[idx];
// ensure it's in the dom
expect(row).to.contain(deposit.id);
if (deposit.status === 'done') {
expect(row).to.contain(deposit.external_id);
} else {
expect(row).to.not.contain(deposit.external_id);
}
expect(row).to.contain(deposit.status);
expect(row).to.contain(expectedOrigin);
// those are hidden by default
expect(row).to.not.contain(deposit.status_detail);
expect(row).to.not.contain(deposit.swh_id);
expect(row).to.not.contain(deposit.swh_id_context);
});
// toggling all links and ensure, the previous checks are inverted
cy.get('a.toggle-col').click({'multiple': true}).then(() => {
cy.get('#swh-admin-deposit-list').find('tbody > tr').as('rows');
cy.get('@rows').each((row, idx, collection) => {
let deposit = deposits[idx];
// ensure it's in the dom
expect(row).to.not.contain(deposit.id);
if (deposit.status === 'done') {
expect(row).to.contain(deposit.external_id);
} else {
expect(row).to.not.contain(deposit.external_id);
}
expect(row).to.not.contain(deposit.status);
// those are hidden by default
expect(row).to.contain(deposit.status_detail);
if (deposit.swh_id !== null) {
expect(row).to.contain(deposit.swh_id);
expect(row).to.contain(deposit.swh_id_context);
}
});
});
cy.get('#swh-admin-deposit-list-error')
.should('not.contain',
'An error occurred while retrieving the list of deposits');
});
});
});
View Options

swh/fuse/fuse.py

  • This file was added.
/**
* Copyright (C) 2020 The Software Heritage developers
* See the AUTHORS file at the top-level directory of this distribution
* License: GNU Affero General Public License version 3, or any later version
* See top-level LICENSE file for more information
*/
const username = 'admin';
const password = 'admin';
function login(username, password) {
cy.get('input[name="username"]')
.type(username)
.get('input[name="password"]')
.type(password)
.get('form')
.submit();
}
describe('Test admin deposit page', function() {
it('Should test deposit page', function() {
cy.visit(this.Urls.admin_deposit());
// FIXME: cypress anti-pattern, do not use ui to log ¯\_(ツ)_/¯
// https://docs.cypress.io/guides/getting-started/testing-your-app.html#Logging-in
login(username, password);
cy.server();
let inputDeposits = [
{
'id': 614,
'external_id': 'c-d-1',
'reception_date': '2020-05-18T13:48:27Z',
'status': 'done',
'status_detail': 'all good',
'swh_id': 'swh:1:dir:ef04a768',
'swh_id_context': 'swh:1:dir:ef04a768;origin=https://w.s.o/c-d-1;visit=swh:1:snp:b234be1e;anchor=swh:1:rev:d24a75c9;path=/'
},
{
'id': 613,
'external_id': 'c-d-2',
'reception_date': '2020-05-18T11:20:16Z',
'status': 'done',
'status_detail': 'everything is fine',
'swh_id': 'swh:1:dir:181417fb',
'swh_id_context': 'swh:1:dir:181417fb;origin=https://w.s.o/c-d-2;visit=swh:1:snp:8c32a2ef;anchor=swh:1:rev:3d1eba04;path=/'
},
{
'id': 612,
'external_id': 'c-d-3',
'reception_date': '2020-05-18T11:20:16Z',
'status': 'rejected',
'status_detail': 'hu ho, issue!',
'swh_id': null,
'swh_id_context': null
}
];
cy.route({
method: 'GET',
url: `${this.Urls.admin_deposit_list()}**`,
response: {
'draw': 10,
'recordsTotal': 3,
'recordsFiltered': 3,
'data': inputDeposits
}
}).as('listDeposits');
cy.location('pathname')
.should('be.equal', this.Urls.admin_deposit());
cy.url().should('include', '/admin/deposit');
cy.get('#swh-admin-deposit-list')
.should('exist');
// those are computed from the
let expectedOrigins = ['https://w.s.o/c-d-1', 'https://w.s.o/c-d-2', ''];
cy.wait('@listDeposits').then((xhr) => {
cy.log('response:', xhr.response);
cy.log(xhr.response.body);
let deposits = xhr.response.body.data;
cy.log('Deposits: ', deposits);
expect(deposits.length).to.equal(inputDeposits.length);
cy.get('#swh-admin-deposit-list').find('tbody > tr').as('rows');
// only 2 entries
cy.get('@rows').each((row, idx, collection) => {
let deposit = deposits[idx];
let inputDeposit = inputDeposits[idx];
assert.isNotNull(deposit);
assert.isNotNull(inputDeposit);
expect(deposit.id).to.be.equal(inputDeposit['id']);
expect(deposit.external_id).to.be.equal(inputDeposit['external_id']);
expect(deposit.status).to.be.equal(inputDeposit['status']);
expect(deposit.status_detail).to.be.equal(inputDeposit['status_detail']);
expect(deposit.swh_id).to.be.equal(inputDeposit['swh_id']);
expect(deposit.swh_id_context).to.be.equal(inputDeposit['swh_id_context']);
let expectedOrigin = expectedOrigins[idx];
// ensure it's in the dom
expect(row).to.contain(deposit.id);
if (deposit.status === 'done') {
expect(row).to.contain(deposit.external_id);
} else {
expect(row).to.not.contain(deposit.external_id);
}
expect(row).to.contain(deposit.status);
expect(row).to.contain(expectedOrigin);
// those are hidden by default
expect(row).to.not.contain(deposit.status_detail);
expect(row).to.not.contain(deposit.swh_id);
expect(row).to.not.contain(deposit.swh_id_context);
});
// toggling all links and ensure, the previous checks are inverted
cy.get('a.toggle-col').click({'multiple': true}).then(() => {
cy.get('#swh-admin-deposit-list').find('tbody > tr').as('rows');
cy.get('@rows').each((row, idx, collection) => {
let deposit = deposits[idx];
// ensure it's in the dom
expect(row).to.not.contain(deposit.id);
if (deposit.status === 'done') {
expect(row).to.contain(deposit.external_id);
} else {
expect(row).to.not.contain(deposit.external_id);
}
expect(row).to.not.contain(deposit.status);
// those are hidden by default
expect(row).to.contain(deposit.status_detail);
if (deposit.swh_id !== null) {
expect(row).to.contain(deposit.swh_id);
expect(row).to.contain(deposit.swh_id_context);
}
});
});
cy.get('#swh-admin-deposit-list-error')
.should('not.contain',
'An error occurred while retrieving the list of deposits');
});
});
});
View Options

swh/fuse/tests/test_cache.py

  • This file was added.
/**
* Copyright (C) 2020 The Software Heritage developers
* See the AUTHORS file at the top-level directory of this distribution
* License: GNU Affero General Public License version 3, or any later version
* See top-level LICENSE file for more information
*/
const username = 'admin';
const password = 'admin';
function login(username, password) {
cy.get('input[name="username"]')
.type(username)
.get('input[name="password"]')
.type(password)
.get('form')
.submit();
}
describe('Test admin deposit page', function() {
it('Should test deposit page', function() {
cy.visit(this.Urls.admin_deposit());
// FIXME: cypress anti-pattern, do not use ui to log ¯\_(ツ)_/¯
// https://docs.cypress.io/guides/getting-started/testing-your-app.html#Logging-in
login(username, password);
cy.server();
let inputDeposits = [
{
'id': 614,
'external_id': 'c-d-1',
'reception_date': '2020-05-18T13:48:27Z',
'status': 'done',
'status_detail': 'all good',
'swh_id': 'swh:1:dir:ef04a768',
'swh_id_context': 'swh:1:dir:ef04a768;origin=https://w.s.o/c-d-1;visit=swh:1:snp:b234be1e;anchor=swh:1:rev:d24a75c9;path=/'
},
{
'id': 613,
'external_id': 'c-d-2',
'reception_date': '2020-05-18T11:20:16Z',
'status': 'done',
'status_detail': 'everything is fine',
'swh_id': 'swh:1:dir:181417fb',
'swh_id_context': 'swh:1:dir:181417fb;origin=https://w.s.o/c-d-2;visit=swh:1:snp:8c32a2ef;anchor=swh:1:rev:3d1eba04;path=/'
},
{
'id': 612,
'external_id': 'c-d-3',
'reception_date': '2020-05-18T11:20:16Z',
'status': 'rejected',
'status_detail': 'hu ho, issue!',
'swh_id': null,
'swh_id_context': null
}
];
cy.route({
method: 'GET',
url: `${this.Urls.admin_deposit_list()}**`,
response: {
'draw': 10,
'recordsTotal': 3,
'recordsFiltered': 3,
'data': inputDeposits
}
}).as('listDeposits');
cy.location('pathname')
.should('be.equal', this.Urls.admin_deposit());
cy.url().should('include', '/admin/deposit');
cy.get('#swh-admin-deposit-list')
.should('exist');
// those are computed from the
let expectedOrigins = ['https://w.s.o/c-d-1', 'https://w.s.o/c-d-2', ''];
cy.wait('@listDeposits').then((xhr) => {
cy.log('response:', xhr.response);
cy.log(xhr.response.body);
let deposits = xhr.response.body.data;
cy.log('Deposits: ', deposits);
expect(deposits.length).to.equal(inputDeposits.length);
cy.get('#swh-admin-deposit-list').find('tbody > tr').as('rows');
// only 2 entries
cy.get('@rows').each((row, idx, collection) => {
let deposit = deposits[idx];
let inputDeposit = inputDeposits[idx];
assert.isNotNull(deposit);
assert.isNotNull(inputDeposit);
expect(deposit.id).to.be.equal(inputDeposit['id']);
expect(deposit.external_id).to.be.equal(inputDeposit['external_id']);
expect(deposit.status).to.be.equal(inputDeposit['status']);
expect(deposit.status_detail).to.be.equal(inputDeposit['status_detail']);
expect(deposit.swh_id).to.be.equal(inputDeposit['swh_id']);
expect(deposit.swh_id_context).to.be.equal(inputDeposit['swh_id_context']);
let expectedOrigin = expectedOrigins[idx];
// ensure it's in the dom
expect(row).to.contain(deposit.id);
if (deposit.status === 'done') {
expect(row).to.contain(deposit.external_id);
} else {
expect(row).to.not.contain(deposit.external_id);
}
expect(row).to.contain(deposit.status);
expect(row).to.contain(expectedOrigin);
// those are hidden by default
expect(row).to.not.contain(deposit.status_detail);
expect(row).to.not.contain(deposit.swh_id);
expect(row).to.not.contain(deposit.swh_id_context);
});
// toggling all links and ensure, the previous checks are inverted
cy.get('a.toggle-col').click({'multiple': true}).then(() => {
cy.get('#swh-admin-deposit-list').find('tbody > tr').as('rows');
cy.get('@rows').each((row, idx, collection) => {
let deposit = deposits[idx];
// ensure it's in the dom
expect(row).to.not.contain(deposit.id);
if (deposit.status === 'done') {
expect(row).to.contain(deposit.external_id);
} else {
expect(row).to.not.contain(deposit.external_id);
}
expect(row).to.not.contain(deposit.status);
// those are hidden by default
expect(row).to.contain(deposit.status_detail);
if (deposit.swh_id !== null) {
expect(row).to.contain(deposit.swh_id);
expect(row).to.contain(deposit.swh_id_context);
}
});
});
cy.get('#swh-admin-deposit-list-error')
.should('not.contain',
'An error occurred while retrieving the list of deposits');
});
});
});
About · Developer information · Legal