https://docs.softwareheritage.org/devel/apidoc/swh.model.html#module-swh.model.identifiers
This explains some of the fields associated with dates, timestamps and offset.

This task is completed it can be closed.

Merge with master

Ready to land made all the required changes.

Remove tox dependency

Remove + symbol from end of line

Fix typo

Add assert_called_once_with to test case

Fix test case. Silly mistakes are costly.

While I was adding comments to all the tables in the db, I experimented a bit with pgdump.

Some databases could benefit from some backups without the overhead of having point in time recovery set up for them

If I understand correctly, it means that recovery from all previous time stamps is not a concern here. In such a case, is a chron job running pgdump at regular intervals feasible?

reference to D1623

• Change key reference from origin_id to id

with patch('swh.web.common.service.idx_storage') as mock_idx_storage, \
             patch('swh.web.common.service.storage') as mock_storage:

Adding backslash solves it

with patch('swh.web.common.service.idx_storage') as mock_idx_storage,
             patch('swh.web.common.service.storage') as mock_storage:

giving sytax error on mock_idx_storage

flake8 gives syntax error on first as

I too, two noticed the two glossaries. They have varying levels of explanations and merging them is one of the tasks I have considered for my GSoD application. I will be glad to assist in the process, in any way I can.

• Remove unnecessary tox.ini dependency

• Fix intrinsic metadata enpoint and test case

I have checked autopep8 and black, none of these claim to reformat long doc strings properly. This may require custom scripting or editing it by hand. However autopep8 did suggest some other changes when I tried it out. Is there a reason a linter is used instead of a formatter?

in reference to T1613
I am returning entry for only one origin id. Is this expected functionality?

I wan to clarify the return value for this api. In swh-storage, table origin_metadata contains the metadata of an origin for a visit, listing etc. The primary key is an auto incremented value. This means that there can be multiple entries for a single origin_id. What is the expected JSON response for this API? I can include origin_id and a list of metadata values, is there anything else to return?

Ok I'll try to implement and test this by tomorrow.

This is will also require adding a function to in swh/web/common/service.py that makes the query to swh-storage. The response will then have to be converted to a json response. I would like to take this up and add this api. Please suggest if there is wrong with the changes I am suggesting.

D1582 has been pushed the task can be closed

Add comments to tables and columns

Add comments to few columns in dbversion, task and task_run

• Added comments to few columns in dbversion, task and task_run
• Made changes as per review

• Make changes as per review

If this issue is open, I can work on this. I believe I have to add the endpoint in /swh-web/swh/web/api/views/origin.py. I will probably have to populate the db with an origin and test the api by making http requests, or is there a better way?

Merge branch with master

• List all options

• Make suggested changes

Foolishly committed to master earlier, created new branch now

Add pyblake2 dependency for python<3.6

• Move platform specific dependency to install_requires

I built and tested the wheel locally, it did install pyblake2 as per the requirements.

• Make changes as per review

Add pyblake2 dependency for all python versions below 3.6

Does this diff require any other changes?

Does this diff require any other changes?

• Double quote syntax error

• Add additional column comments

The comments are not complete. I will fix the syntax issue in the next commit along with the missing comments. I am not sure what comments to add for the revision table.

Django-simple-captcha works best out of the box using Forms or ModelForms. But the origin/save page is not rendered using forms, its plain HTML. One possible solution is to use a Form for origin save submission, the other is to write custom captcha template in and include it in the page. Which one did you have in mind?

• Added comments to tables dbversion, content, skipped_content and fetch_history
• Added comments for all tables and columns
• Converted double quote comments to single quote comments

• Change double quoted comments to single quoted comments

• Change double quoted comments to single quoted comments

• Add comments to few columns in dbversion, task and task_run
• Change double quoted comments to single quoted comments

All columns commented in swh-scheduler, waiting review.
Some columns for swh-storage required a small discussion to frame appropriate comments.

in reference to T1527

• Added comments for all tables and columns

All columns are already commented in swh-indexer

Have added a few comments in D1582

This diff is in reference to T1527.

there seems to be an inconsistency between sql/upgrades and latest sql version in swh-storage. The latest upgrade is 136.sql while the version in 30-swh-schema.sql is 133. Should I name the next upgrade 137?

I am on a standard Ubuntu 16.04 Xenial distribution. Is there any specific config files you want to see.

is there anything left to be done to close the task?

I was clearing existing tox setup with rm -rf .tox, when I changed requirements. I will try tests of other modules as well and report if I find similar errors.

modules swh-scheduler, swh-indexer, swh-storage, all seem to have column comments written in 30-swh-schema.sql

https://forge.softwareheritage.org/P429
this is the error message from one of the tests. Results in all py3 tests failing.

Can you provide a few more details so I can work on this? Maybe which packages will be affected and what is expected in the comments.

• moved dependency to requirements-test.txt