Page MenuHomeSoftware Heritage
Feed Advanced Search

May 6 2019

douardda closed D1426: cli: add a generic main 'swh' command and extract the db-init command.
May 6 2019, 3:23 PM
douardda updated the diff for D1427: setup: extract db and http related parts in dedicated optional 'extras'.

rebased

May 6 2019, 3:22 PM
douardda updated the diff for D1426: cli: add a generic main 'swh' command and extract the db-init command.

include forgotten patch: add log_level to 'swh' context obj

May 6 2019, 3:22 PM
douardda added a comment to D1427: setup: extract db and http related parts in dedicated optional 'extras'.

Shouldn't we add tests to make sure that core features work without the extras?

yeah that's something I had in mind, but not sure how to test it without depending on venvs/tox ("raw" pytest)...

May 6 2019, 1:44 PM
douardda added a comment to D1427: setup: extract db and http related parts in dedicated optional 'extras'.

Shouldn't we add tests to make sure that core features work without the extras?

May 6 2019, 1:42 PM
douardda updated the diff for D1429: docs: update the getting-started document to use 'swh deposit upload' command.

rebased

May 6 2019, 1:40 PM
douardda updated the diff for D1428: cli: make the deposit cli command a subcommand of the main 'swh' one.

Use proper swh.cli.subcommands entrypoint

May 6 2019, 1:39 PM

May 2 2019

Herald added a reviewer for D1439: cli: make swh-scheduler a subcommand of swh.core's main "swh" command: Reviewers.
May 2 2019, 5:42 PM
douardda updated the diff for D1427: setup: extract db and http related parts in dedicated optional 'extras'.

rebase

May 2 2019, 3:20 PM
douardda updated the diff for D1426: cli: add a generic main 'swh' command and extract the db-init command.

Implement the entrypoint loading mechanism using functions instead of modules

May 2 2019, 3:19 PM
douardda accepted D1438: Activate save code now for hg and svn origin types.
May 2 2019, 2:13 PM
douardda requested changes to D1438: Activate save code now for hg and svn origin types.
May 2 2019, 2:07 PM
douardda added a comment to T1419: hg/svn support in save code now.

we (@anlambert and I) will try to have this task closed ASAP (like today, if no big bad stopper arise in front of us)

May 2 2019, 11:55 AM · Web app, Sprint 2018 12

Apr 19 2019

douardda updated the diff for D1426: cli: add a generic main 'swh' command and extract the db-init command.

no need to keep the cli plugins in a dict

Apr 19 2019, 4:24 PM
douardda added inline comments to D1426: cli: add a generic main 'swh' command and extract the db-init command.
Apr 19 2019, 4:19 PM
douardda updated the diff for D1429: docs: update the getting-started document to use 'swh deposit upload' command.

rebased

Apr 19 2019, 3:50 PM
douardda updated the diff for D1428: cli: make the deposit cli command a subcommand of the main 'swh' one.

Add requests to requirements.txt since it does come with swh.core any more

Apr 19 2019, 3:50 PM
douardda updated the summary of D1427: setup: extract db and http related parts in dedicated optional 'extras'.
Apr 19 2019, 3:37 PM
douardda updated the diff for D1427: setup: extract db and http related parts in dedicated optional 'extras'.

kill the PostresHandler instead of "shuffling it around" (© olasd)

Apr 19 2019, 3:36 PM
douardda added a comment to D1427: setup: extract db and http related parts in dedicated optional 'extras'.
In D1427#31359, @olasd wrote:

The PostgresHandler has been dead code for two years, I think we can just remove it rather than shuffling it around.

Apr 19 2019, 3:28 PM
Herald added a reviewer for D1429: docs: update the getting-started document to use 'swh deposit upload' command: Reviewers.
Apr 19 2019, 3:27 PM
douardda updated the summary of D1428: cli: make the deposit cli command a subcommand of the main 'swh' one.
Apr 19 2019, 2:39 PM
douardda updated the summary of D1428: cli: make the deposit cli command a subcommand of the main 'swh' one.
Apr 19 2019, 2:39 PM
Herald added a reviewer for D1428: cli: make the deposit cli command a subcommand of the main 'swh' one: Reviewers.
Apr 19 2019, 2:38 PM
douardda updated the diff for D1427: setup: extract db and http related parts in dedicated optional 'extras'.

Fix commit messages

Apr 19 2019, 2:31 PM
Herald added a reviewer for D1427: setup: extract db and http related parts in dedicated optional 'extras': Reviewers.
Apr 19 2019, 12:13 PM
douardda added a revision to T1678: Make the swh.core package define some 'extras' features to limit dependencies of the 'plain' swh.core package: D1427: setup: extract db and http related parts in dedicated optional 'extras'.
Apr 19 2019, 12:13 PM · SWH command line interface
Herald added a reviewer for D1426: cli: add a generic main 'swh' command and extract the db-init command: Reviewers.
Apr 19 2019, 12:13 PM
douardda committed rDCORE22e60ae54315: Reorder a few import statements (authored by douardda).
Reorder a few import statements
Apr 19 2019, 12:12 PM
douardda triaged T1678: Make the swh.core package define some 'extras' features to limit dependencies of the 'plain' swh.core package as Normal priority.
Apr 19 2019, 11:54 AM · SWH command line interface
douardda added a comment to T1671: Implement a uniform and consistent cli to be used by all SWH packages.

This also requires we can install only a very light base version of swh.core (typically without db or aiohttp/flask dependencies).

Apr 19 2019, 10:25 AM · SWH command line interface
douardda added a comment to T1671: Implement a uniform and consistent cli to be used by all SWH packages.

A possible way of doing such a thing is to have a master swh command declared in this package as the main entry point: the swh command group (using click) which defines a few very command command line options (like --log-level), and a mechanism to load installed SWH cli plugins via setuptools' pkg_resources

Apr 19 2019, 10:23 AM · SWH command line interface

Apr 18 2019

douardda added a comment to T1671: Implement a uniform and consistent cli to be used by all SWH packages.

This generic swh cli should be done so that is minimal dependency installed especially for "pure client" commands (e.g. swh.deposit's deposit or swh.model's identify tools).

Apr 18 2019, 3:14 PM · SWH command line interface
douardda added a parent task for T1651: Create a separate project for deposit-client: T1671: Implement a uniform and consistent cli to be used by all SWH packages.
Apr 18 2019, 3:03 PM · SWH command line interface, SWORD deposit
douardda added a subtask for T1671: Implement a uniform and consistent cli to be used by all SWH packages: T1651: Create a separate project for deposit-client.
Apr 18 2019, 3:03 PM · SWH command line interface
douardda added a parent task for T1658: Gather CLI tools spread in SWH packages: T1671: Implement a uniform and consistent cli to be used by all SWH packages.
Apr 18 2019, 3:02 PM · SWH command line interface
douardda added a subtask for T1671: Implement a uniform and consistent cli to be used by all SWH packages: T1658: Gather CLI tools spread in SWH packages.
Apr 18 2019, 3:02 PM · SWH command line interface
douardda triaged T1671: Implement a uniform and consistent cli to be used by all SWH packages as Normal priority.
Apr 18 2019, 3:02 PM · SWH command line interface
douardda updated the task description for T1651: Create a separate project for deposit-client.
Apr 18 2019, 2:59 PM · SWH command line interface, SWORD deposit
douardda updated the task description for T1651: Create a separate project for deposit-client.
Apr 18 2019, 2:57 PM · SWH command line interface, SWORD deposit
douardda added a comment to T1658: Gather CLI tools spread in SWH packages.

Currently, we have:

Apr 18 2019, 11:49 AM · SWH command line interface
douardda closed T1599: Analyze objstorage's Azure updateness as Resolved.
Apr 18 2019, 10:35 AM · Sprint 2019 03
douardda triaged T1658: Gather CLI tools spread in SWH packages as Normal priority.
Apr 18 2019, 9:36 AM · SWH command line interface
douardda added a project to T1651: Create a separate project for deposit-client: SWH command line interface.
Apr 18 2019, 9:25 AM · SWH command line interface, SWORD deposit
douardda created SWH command line interface.
Apr 18 2019, 9:25 AM
douardda added a comment to T1442: Replace Munin graphs with Grafana/Prometheus dashboards.

It is these graphs we are still missing.

Apr 18 2019, 9:19 AM · Sprint 2018 12, System administration

Apr 16 2019

douardda added a comment to T1442: Replace Munin graphs with Grafana/Prometheus dashboards.

Indeed it is the object of T1428. That's why I am a bit puzzled the work you have in progress does not simply target T1356. I was expecting some response to this very task in your grafanalib based code, which I did not find. So I was wondering if I missed something, that some data where still in munin only.

Apr 16 2019, 3:04 PM · Sprint 2018 12, System administration
douardda added inline comments to D1416: deposit: Adapt revision dates setting according to new rules.
Apr 16 2019, 12:35 PM · SWORD deposit
douardda added a comment to T1442: Replace Munin graphs with Grafana/Prometheus dashboards.

When I asked where to put such work-in-progress, you suggested the snippets repository.

Apr 16 2019, 12:04 PM · Sprint 2018 12, System administration
douardda accepted D1418: deposit.cli: Generate uuid as is.

LGTM

Apr 16 2019, 11:30 AM
douardda added a comment to T1442: Replace Munin graphs with Grafana/Prometheus dashboards.

Work-in-progress Grafanalib dashboards have been added to the https://forge.softwareheritage.org/source/snippets/ repository.

Apr 16 2019, 9:41 AM · Sprint 2018 12, System administration

Apr 15 2019

douardda added inline comments to D1418: deposit.cli: Generate uuid as is.
Apr 15 2019, 5:02 PM
douardda added a comment to T1651: Create a separate project for deposit-client.

As discussed earlier, this task should be actually the first step of a more generic software-heritage cli client module (using HTTP based AIPs/RPCs).

Apr 15 2019, 4:55 PM · SWH command line interface, SWORD deposit
douardda accepted D1335: Don't store empty metadata dicts..

LGTM but I'm not very familiar with this code.

Apr 15 2019, 4:48 PM
douardda accepted D1346: docker-compose: Add journal-backfiller.
Apr 15 2019, 4:42 PM
douardda accepted D1413: Add a from_dict() method to model classes, that does the inverse of to_dict()..
Apr 15 2019, 4:40 PM
douardda accepted D1417: Fix connection leaks in revision_add and directory_add..
Apr 15 2019, 4:39 PM
douardda requested changes to D1413: Add a from_dict() method to model classes, that does the inverse of to_dict()..
Apr 15 2019, 1:40 PM
douardda requested changes to D1417: Fix connection leaks in revision_add and directory_add..
Apr 15 2019, 1:27 PM
douardda added a comment to D1417: Fix connection leaks in revision_add and directory_add..

could you explain in the commit message why this solves a cnx leak?

Apr 15 2019, 1:27 PM
douardda requested changes to D1346: docker-compose: Add journal-backfiller.

few minor fixes to be make, then should be ok.

Apr 15 2019, 12:44 PM

Apr 12 2019

douardda accepted D1411: Refactor deposit cli and some other improvments.

lgtm (but I'm obviously biased)

Apr 12 2019, 4:40 PM
douardda accepted D1414: Improve deposit container to actually use loader/checker.

lgtm

Apr 12 2019, 4:18 PM
douardda committed rDDEP9f8709da3b60: cli: fix logging setup (authored by douardda).
cli: fix logging setup
Apr 12 2019, 9:41 AM
douardda committed rDDEPe07add73dbd7: config: do not overwrite the DJANGO_SETTINGS_MODULE if already defined (authored by douardda).
config: do not overwrite the DJANGO_SETTINGS_MODULE if already defined
Apr 12 2019, 9:41 AM
douardda committed rDDEP4fb78f6e3672: cli/admin/user: add a user existence check (authored by douardda).
cli/admin/user: add a user existence check
Apr 12 2019, 9:41 AM
douardda committed rDDEPc4de5bdb07c5: cli: kill the unneeded dry-run option (authored by douardda).
cli: kill the unneeded dry-run option
Apr 12 2019, 9:41 AM
douardda committed rDDEPa1fc0e8808b9: client: use a global logger (authored by douardda).
client: use a global logger
Apr 12 2019, 9:41 AM
douardda committed rDDEPcf58937eb04e: cli: split the cli.py module in a package with submodules (authored by douardda).
cli: split the cli.py module in a package with submodules
Apr 12 2019, 9:41 AM
douardda committed rDDEP248c43674a8e: cli/admin/user: provide more information at creation (authored by douardda).
cli/admin/user: provide more information at creation
Apr 12 2019, 9:41 AM
douardda committed rDDEP426500c3abee: cli: fix service document parsing to retrieve collection (authored by douardda).
cli: fix service document parsing to retrieve collection
Apr 12 2019, 9:41 AM
douardda committed rDDEPb07d24405772: model: kill DepositRequestType (authored by douardda).
model: kill DepositRequestType
Apr 12 2019, 9:41 AM
douardda committed rDDEP38b472f3d174: setup: extract server specific requirements in a dedicated 'server' extra (authored by douardda).
setup: extract server specific requirements in a dedicated 'server' extra
Apr 12 2019, 9:41 AM

Apr 11 2019

douardda committed R65:97981b2f2793: API: simplify a bit the RemoteStorage class (authored by douardda).
API: simplify a bit the RemoteStorage class
Apr 11 2019, 12:19 PM
douardda committed R65:4ed7d0ae0dd6: API: refactor the /directory/ls endpoint (authored by douardda).
API: refactor the /directory/ls endpoint
Apr 11 2019, 12:19 PM
douardda committed rDLSe5c355903323: tasks: fix handling of unsupported promise.save() calls (authored by douardda).
tasks: fix handling of unsupported promise.save() calls
Apr 11 2019, 11:13 AM
douardda closed D1404: tasks: fix handling of unsupported promise.save() calls.
Apr 11 2019, 11:13 AM
douardda added a comment to D1404: tasks: fix handling of unsupported promise.save() calls.

That code (and the lines before) look very similar to each other.

Apr 11 2019, 11:13 AM
Herald added a reviewer for D1404: tasks: fix handling of unsupported promise.save() calls: Reviewers.
Apr 11 2019, 11:06 AM
douardda committed rDJNL7ace7f201d31: Merge tag 'v0.0.9' (authored by douardda).
Merge tag 'v0.0.9'
Apr 11 2019, 10:59 AM

Apr 10 2019

douardda accepted D1396: config: Allow to specify development database file.

ok but see my comment plz

Apr 10 2019, 4:12 PM
douardda committed rCDFD30e9c934ef20: lister: allways run the db initialization step (authored by douardda).
lister: allways run the db initialization step
Apr 10 2019, 2:40 PM
douardda committed rCDFD6d49580cd56c: replica: ensure entrypoints are defined in replica related services (authored by douardda).
replica: ensure entrypoints are defined in replica related services
Apr 10 2019, 2:40 PM
douardda committed rCDFDb7f02d8add7f: dockerfile: install python packages in a venv under the swh user (authored by douardda).
dockerfile: install python packages in a venv under the swh user
Apr 10 2019, 2:40 PM
douardda committed rCDFD6558c6c8ad18: entrypoint: pgsql has been moved to ~swh/ (authored by douardda).
entrypoint: pgsql has been moved to ~swh/
Apr 10 2019, 2:40 PM
douardda committed rCDFD7b0ea2803606: Remove entrypoint.sh files from the Dockerfile (authored by douardda).
Remove entrypoint.sh files from the Dockerfile
Apr 10 2019, 2:40 PM
douardda committed rCDFDd1f5b87ba8c7: Dockerfile: small readability refactoring (authored by douardda).
Dockerfile: small readability refactoring
Apr 10 2019, 2:40 PM
douardda committed rCDFD53fa588c7fc2: Factorise the pip install of source repos in a shell function (authored by douardda).
Factorise the pip install of source repos in a shell function
Apr 10 2019, 2:40 PM
douardda closed D1394: Several refactorings.
Apr 10 2019, 2:40 PM
douardda closed D1394: Several refactorings.
Apr 10 2019, 2:40 PM
douardda committed rCDFD9359da6dbf95: Add support for a kafka-powered replica of the graph. (authored by douardda).
Add support for a kafka-powered replica of the graph.
Apr 10 2019, 2:40 PM
douardda closed D1393: Add support for a kafka-powered replica of the graph..
Apr 10 2019, 2:40 PM
douardda committed rCDFD587f9df57a80: lister: Use explicit postgresql URLs as a workaround for a sqlalchemy issue (authored by douardda).
lister: Use explicit postgresql URLs as a workaround for a sqlalchemy issue
Apr 10 2019, 2:40 PM
douardda closed D1392: lister: Use explicit postgresql URLs as a workaround for a sqlalchemy issue.
Apr 10 2019, 2:40 PM
douardda committed rCDFD20b0733b84f3: swh-web: enforce the database file to be created in either $HOME or /tmp (authored by douardda).
swh-web: enforce the database file to be created in either $HOME or /tmp
Apr 10 2019, 2:40 PM
douardda closed D1391: swh-web: enforce the database file to be created in either $HOME or /tmp.
Apr 10 2019, 2:40 PM
douardda committed rDJNL8d17a3c04195: bump dependencies to swh.storage>=0.0.138 and swh.model>=0.0.34 (authored by douardda).
bump dependencies to swh.storage>=0.0.138 and swh.model>=0.0.34
Apr 10 2019, 1:43 PM
Herald added a reviewer for D1394: Several refactorings: Reviewers.
Apr 10 2019, 12:29 PM
Herald added a reviewer for D1393: Add support for a kafka-powered replica of the graph.: Reviewers.
Apr 10 2019, 12:24 PM
Herald added a reviewer for D1392: lister: Use explicit postgresql URLs as a workaround for a sqlalchemy issue: Reviewers.
Apr 10 2019, 12:23 PM