Page MenuHomeSoftware Heritage
Feed All Stories

May 7 2021

ardumont added a comment to T3310: Deploy sourceforge lister on staging.

It broke with the following, sentry should have more detail [1]

May 07 15:57:03 worker0 python3[230930]: [2021-05-07 15:57:03,547: ERROR/ForkPoolWorker-4] Task swh.lister.sourceforge.tasks.FullSourceForgeLister[ec00c8cd-ff5b-47df-adfd-a8c1884b9831] raised unexpected: HTTPError('404 Client Error: Not Found for url: https://sourceforge.net/rest/p/fci-cu-library2/b396')
May 7 2021, 6:59 PM · System administration, Origin-SourceForge
Harbormaster failed to build B21399: rDPROVd0d86eadda7c: Several fixes. for rDPROVd0d86eadda7c: Several fixes.!
May 7 2021, 6:51 PM
aeviso committed rDPROVd0d86eadda7c: Several fixes. (authored by aeviso).
Several fixes.
May 7 2021, 6:50 PM
ardumont added a comment to T3310: Deploy sourceforge lister on staging.

Deployment in progress.

May 7 2021, 5:39 PM · System administration, Origin-SourceForge
swh-public-ci added a comment to D5676: Allow privileged user to trigger save code now bundle visit type.

Build is green

May 7 2021, 5:39 PM
ardumont updated the summary of D5676: Allow privileged user to trigger save code now bundle visit type.
May 7 2021, 5:30 PM
ardumont updated the diff for D5676: Allow privileged user to trigger save code now bundle visit type.

Adapt according to review

May 7 2021, 5:29 PM
zack added a parent task for T735: SourceForge lister: T3315: archive SourceForge.
May 7 2021, 5:25 PM · Origin-SourceForge
zack added a subtask for T3315: archive SourceForge: T735: SourceForge lister.
May 7 2021, 5:25 PM · Archive coverage
zack triaged T3315: archive SourceForge as Normal priority.
May 7 2021, 5:25 PM · Archive coverage
vsellier added a comment to T3222: Monitor daily indexes are present on the log cluster and logs are correctly ingested.

According to the API (TIL the catalog can be requested like that), journal0 doesn't have the new plugins declared. So the check should be disabled as the filter is using this field

May 7 2021, 5:20 PM · System administration
ardumont added a comment to T3310: Deploy sourceforge lister on staging.

I'll adapt (but afk).

May 7 2021, 5:19 PM · System administration, Origin-SourceForge
ardumont closed D5715: sourceforge/tasks: Allow incremental listing.
May 7 2021, 5:17 PM
anlambert added a comment to D5676: Allow privileged user to trigger save code now bundle visit type.

My understanding of the plan was first to deal with one artifact so the one.

May 7 2021, 5:17 PM
ardumont committed rDLS2ff549e12593: sourceforge/tasks: Allow incremental listing (authored by ardumont).
sourceforge/tasks: Allow incremental listing
May 7 2021, 5:17 PM
anlambert accepted D5715: sourceforge/tasks: Allow incremental listing.
May 7 2021, 5:11 PM
ardumont requested review of D5715: sourceforge/tasks: Allow incremental listing.
May 7 2021, 5:09 PM
ardumont added inline comments to D5676: Allow privileged user to trigger save code now bundle visit type.
May 7 2021, 5:09 PM
ardumont added a revision to T3310: Deploy sourceforge lister on staging: D5715: sourceforge/tasks: Allow incremental listing.
May 7 2021, 5:03 PM · System administration, Origin-SourceForge
ardumont added inline comments to D5676: Allow privileged user to trigger save code now bundle visit type.
May 7 2021, 5:01 PM
ardumont added a comment to D5676: Allow privileged user to trigger save code now bundle visit type.

Do you plan to handle multiple artifacts in that diff or in another one ?

May 7 2021, 5:00 PM
anlambert requested changes to D5676: Allow privileged user to trigger save code now bundle visit type.

The new introduced UI elements should be explained when switching to the bundle visit type.

May 7 2021, 4:57 PM
ardumont closed D5714: sourceforge/lister: Add credentials parameter.
May 7 2021, 4:52 PM
ardumont committed rDLS7282647bb2d3: sourceforge/lister: Add credentials parameter (authored by ardumont).
sourceforge/lister: Add credentials parameter
May 7 2021, 4:52 PM
swh-public-ci added a comment to D5714: sourceforge/lister: Add credentials parameter.

Build is green

May 7 2021, 4:50 PM
ardumont updated the diff for D5714: sourceforge/lister: Add credentials parameter.

Reword commit message

May 7 2021, 4:47 PM
anlambert accepted D5714: sourceforge/lister: Add credentials parameter.
May 7 2021, 4:24 PM
ardumont requested review of D5714: sourceforge/lister: Add credentials parameter.
May 7 2021, 4:14 PM
ardumont added a revision to T3310: Deploy sourceforge lister on staging: D5714: sourceforge/lister: Add credentials parameter.
May 7 2021, 4:11 PM · System administration, Origin-SourceForge
ardumont added a comment to T3310: Deploy sourceforge lister on staging.

Check everything is fine (it is):

psql service=admin-staging-swh-scheduler                                                      psql (12.6)
SSL connection (protocol: TLSv1.3, cipher: TLS_AES_256_GCM_SHA384, bits: 256, compression: off)
Type "help" for help.
May 7 2021, 4:04 PM · System administration, Origin-SourceForge
seirl committed rDGRPH991a96dbd08b: LabelMapBuilder: use sort by default (authored by seirl).
LabelMapBuilder: use sort by default
May 7 2021, 3:55 PM
ardumont added a comment to T3310: Deploy sourceforge lister on staging.

Update the scheduler backend with the new task type:

May 7 2021, 3:54 PM · System administration, Origin-SourceForge
ardumont closed D5713: Deploy new sourceforge lister task.
May 7 2021, 3:49 PM
ardumont committed rSPSITE06bd2cdec0d5: Deploy new sourceforge lister task (authored by ardumont).
Deploy new sourceforge lister task
May 7 2021, 3:49 PM
vsellier accepted D5713: Deploy new sourceforge lister task.
May 7 2021, 3:48 PM
ardumont added a comment to D5686: Minor typo fix in docker documentation.

@jayeshv don't forget to push it.

May 7 2021, 3:41 PM
ardumont requested review of D5713: Deploy new sourceforge lister task.
May 7 2021, 3:32 PM
ardumont added a revision to T3310: Deploy sourceforge lister on staging: D5713: Deploy new sourceforge lister task.
May 7 2021, 3:32 PM · System administration, Origin-SourceForge
ardumont added a comment to T3310: Deploy sourceforge lister on staging.

Installing the latest package on a worker:

May 7 2021, 3:25 PM · System administration, Origin-SourceForge
vsellier added a comment to T3222: Monitor daily indexes are present on the log cluster and logs are correctly ingested.

The new probe is deployed but nothing is displayed in icinga. Let's start a configuration debug session.

May 7 2021, 3:20 PM · System administration
swh-public-ci added a comment to D5696: tests: Add new ambassador user for frontend tests.

Build is green

May 7 2021, 3:16 PM
swh-public-ci added a comment to D5676: Allow privileged user to trigger save code now bundle visit type.

Build is green

May 7 2021, 3:07 PM
anlambert closed T3312: web API rate limit: 10x more quota for authenticated users as Resolved.

The new rate limit policy has been implemented and deployed to production, see X-RateLimit-Limit header value below.

May 7 2021, 3:03 PM · Web app
swh-public-ci added a comment to D5696: tests: Add new ambassador user for frontend tests.

Build is green

May 7 2021, 2:58 PM
vsellier closed D5709: Add a monitoring alert when logstash is failing to send logs to ES.
May 7 2021, 2:54 PM
vsellier committed rSPSITE6657bf88897d: Add a monitoring alert when logstash is failing to send logs to ES (authored by vsellier).
Add a monitoring alert when logstash is failing to send logs to ES
May 7 2021, 2:54 PM
vsellier closed T3203: docs: Document the firewall installation and procedures, a subtask of T3194: Upgrade opnsense firewalls from 20.7.4 to 21.1.4, as Resolved.
May 7 2021, 2:53 PM · System administration
vsellier closed T3203: docs: Document the firewall installation and procedures as Resolved.
May 7 2021, 2:52 PM · Documentation, System administration
vsellier closed D5692: document the firewalls upgrade procedure.
May 7 2021, 2:50 PM
vsellier committed rDDOC7147b9c0cac2: document the firewalls upgrade procedure (authored by vsellier).
document the firewalls upgrade procedure
May 7 2021, 2:50 PM
Harbormaster failed remote builds in B21383: Diff 20403 for D5696: tests: Add new ambassador user for frontend tests!
May 7 2021, 2:50 PM
swh-public-ci added a comment to D5696: tests: Add new ambassador user for frontend tests.

Build has FAILED

May 7 2021, 2:50 PM
ardumont accepted D5709: Add a monitoring alert when logstash is failing to send logs to ES.
May 7 2021, 2:49 PM
vsellier updated the diff for D5709: Add a monitoring alert when logstash is failing to send logs to ES.
  • fix inconsistency in check command naming
  • remove the unecessary set option on the check script
May 7 2021, 2:45 PM
anlambert closed D5712: sourceforge/tests: Ensure correct sleep function gets mocked.
May 7 2021, 2:43 PM
anlambert committed rDLS3167a6dcb736: sourceforge/tests: Ensure correct sleep function gets mocked (authored by anlambert).
sourceforge/tests: Ensure correct sleep function gets mocked
May 7 2021, 2:43 PM
swh-public-ci added a comment to D5712: sourceforge/tests: Ensure correct sleep function gets mocked.

Build is green

May 7 2021, 2:43 PM
swh-public-ci added a comment to D5685: Add new bundle type to list only when connected as privileged user.

Build is green

May 7 2021, 2:42 PM
ardumont accepted D5712: sourceforge/tests: Ensure correct sleep function gets mocked.
May 7 2021, 2:41 PM
ardumont published D5712: sourceforge/tests: Ensure correct sleep function gets mocked for review.
May 7 2021, 2:41 PM
vsellier added inline comments to D5709: Add a monitoring alert when logstash is failing to send logs to ES.
May 7 2021, 2:40 PM
anlambert added a revision to T3310: Deploy sourceforge lister on staging: D5712: sourceforge/tests: Ensure correct sleep function gets mocked.
May 7 2021, 2:40 PM · System administration, Origin-SourceForge
ardumont updated the diff for D5696: tests: Add new ambassador user for frontend tests.

Rebase

May 7 2021, 2:36 PM
ardumont updated the diff for D5676: Allow privileged user to trigger save code now bundle visit type.

Rebase

May 7 2021, 2:36 PM
Harbormaster failed remote builds in B21380: Diff 20400 for D5696: tests: Add new ambassador user for frontend tests!
May 7 2021, 2:34 PM
swh-public-ci added a comment to D5696: tests: Add new ambassador user for frontend tests.

Build has FAILED

May 7 2021, 2:34 PM
ardumont accepted D5692: document the firewalls upgrade procedure.

lgtm

May 7 2021, 2:32 PM
swh-public-ci added a comment to D5685: Add new bundle type to list only when connected as privileged user.

Build is green

May 7 2021, 2:27 PM
ardumont updated the diff for D5696: tests: Add new ambassador user for frontend tests.

Rebase and fix the html comment

May 7 2021, 2:23 PM
ardumont updated the diff for D5696: tests: Add new ambassador user for frontend tests.

Actually rebase

May 7 2021, 2:21 PM
ardumont updated the diff for D5685: Add new bundle type to list only when connected as privileged user.

Actually rebase

May 7 2021, 2:20 PM
vsellier updated the diff for D5692: document the firewalls upgrade procedure.

network schema: reactivate the description of the firewalls' group

May 7 2021, 2:19 PM
ardumont updated the diff for D5696: tests: Add new ambassador user for frontend tests.

Rebase

May 7 2021, 2:19 PM
ardumont updated the diff for D5685: Add new bundle type to list only when connected as privileged user.

Rebase

May 7 2021, 2:18 PM
anlambert closed D5710: api/throttling: Increase rate limit for authenticated users.
May 7 2021, 2:16 PM
anlambert committed rDWAPPSfcf53f1ec3ad: package.json: Upgrade dependencies (authored by anlambert).
package.json: Upgrade dependencies
May 7 2021, 2:16 PM
anlambert committed rDWAPPS2f85b65ddfbc: api/throttling: Increase rate limit for authenticated users (authored by anlambert).
api/throttling: Increase rate limit for authenticated users
May 7 2021, 2:16 PM
swh-public-ci added a comment to D5687: Make the Mercurial loader incremental.

Build is green

May 7 2021, 2:16 PM
vsellier updated the diff for D5692: document the firewalls upgrade procedure.
  • rebase
  • network schema:
    • change the vlan order to be able to use only one gateway
    • Adapt several labels
  • rst: adatp according to the feebacks
May 7 2021, 2:15 PM
Alphare updated the diff for D5687: Make the Mercurial loader incremental.

Include suggestions

May 7 2021, 2:13 PM
anlambert closed D5711: sourceforge/tests: Fix failing test with tenacity < 5.1.
May 7 2021, 2:10 PM
anlambert committed rDLS1284eb158703: sourceforge/tests: Fix failing test with tenacity < 5.1 (authored by anlambert).
sourceforge/tests: Fix failing test with tenacity < 5.1
May 7 2021, 2:10 PM
ardumont accepted D5711: sourceforge/tests: Fix failing test with tenacity < 5.1.
May 7 2021, 2:09 PM
anlambert requested review of D5711: sourceforge/tests: Fix failing test with tenacity < 5.1.
May 7 2021, 2:08 PM
ardumont added a revision to T3310: Deploy sourceforge lister on staging: D5711: sourceforge/tests: Fix failing test with tenacity < 5.1.
May 7 2021, 2:07 PM · System administration, Origin-SourceForge
vsellier added inline comments to D5692: document the firewalls upgrade procedure.
May 7 2021, 2:04 PM
ardumont updated the summary of D5676: Allow privileged user to trigger save code now bundle visit type.
May 7 2021, 2:04 PM
vsellier added inline comments to D5692: document the firewalls upgrade procedure.
May 7 2021, 2:02 PM
ardumont accepted D5709: Add a monitoring alert when logstash is failing to send logs to ES.

couple of questions inline.

May 7 2021, 1:57 PM
douardda closed D5680: Make the TenaciousProxyStorage retry when a single object add fails.
May 7 2021, 1:55 PM
douardda committed rDSTO761709957a40: Make the TenaciousProxyStorage retry when a single object add fails (authored by douardda).
Make the TenaciousProxyStorage retry when a single object add fails
May 7 2021, 1:55 PM
swh-public-ci added a comment to D5680: Make the TenaciousProxyStorage retry when a single object add fails.

Build is green

May 7 2021, 1:54 PM
douardda updated the diff for D5680: Make the TenaciousProxyStorage retry when a single object add fails.

rebase

May 7 2021, 1:47 PM
ardumont accepted D5710: api/throttling: Increase rate limit for authenticated users.
May 7 2021, 1:45 PM
Alphare closed D5691: Remove `os.environ` manipulation in tests.
May 7 2021, 1:33 PM
Alphare committed rDLDHG773d872a8164: Move `os.environ` manipulation to pre_cleanup (authored by Alphare).
Move `os.environ` manipulation to pre_cleanup
May 7 2021, 1:33 PM
Alphare planned changes to D5687: Make the Mercurial loader incremental.

I haven't gotten around to doing the changes yet, will do so shortly.

May 7 2021, 1:33 PM
swh-public-ci added a comment to D5687: Make the Mercurial loader incremental.

Build is green

May 7 2021, 1:31 PM
swh-public-ci added a comment to D5691: Remove `os.environ` manipulation in tests.

Build is green

May 7 2021, 1:29 PM
Alphare updated the diff for D5687: Make the Mercurial loader incremental.

Rebase

May 7 2021, 1:27 PM