Allow to override the beat version
- Queries
- All Stories
- Search
- Advanced Search
- Transactions
- Transaction Logs
Feed All Stories
All Stories
All Stories
Nov 10 2021
Nov 10 2021
ardumont changed the status of T3716: Migrate giverny from stretch to buster, a subtask of T3579: Meta-task: upgrade infrastructure to Debian Bullseye, from Open to Work in Progress.
ardumont changed the status of T3716: Migrate giverny from stretch to buster from Open to Work in Progress.
ardumont changed the status of T3707: Use local disks for worker scratch space from Open to Work in Progress.
ardumont moved T3722: staging: Deploy package loader v1.0, deposit server v0.16, lister v2.3 from Backlog to in-progress on the System administration board.
ardumont renamed T3722: staging: Deploy package loader v1.0, deposit server v0.16, lister v2.3 from staging: Deploy package loader v1.0 and deposit server v0.16 to staging: Deploy package loader v1.0, deposit server v0.16, lister v2.3.
ardumont changed the status of T3722: staging: Deploy package loader v1.0, deposit server v0.16, lister v2.3, a subtask of T3636: Make the opam loader write extrinsic metadata, from Open to Work in Progress.
ardumont changed the status of T3722: staging: Deploy package loader v1.0, deposit server v0.16, lister v2.3, a subtask of T3638: Make package loaders create releases objects instead of revisions, from Open to Work in Progress.
ardumont changed the status of T3722: staging: Deploy package loader v1.0, deposit server v0.16, lister v2.3 from Open to Work in Progress.
ardumont updated the task description for T3722: staging: Deploy package loader v1.0, deposit server v0.16, lister v2.3.
vlorentz updated the task description for T3722: staging: Deploy package loader v1.0, deposit server v0.16, lister v2.3.
ardumont triaged T3722: staging: Deploy package loader v1.0, deposit server v0.16, lister v2.3 as Normal priority.
ardumont committed rDLDBASE1f1bdad89936: utils: Add types and let log instruction do the formatting (authored by ardumont).
utils: Add types and let log instruction do the formatting
Build is green
ardumont committed rDLDSVN0f70f07bc35b: tests: Ensure loaders actually cleanup after themselves (authored by ardumont).
tests: Ensure loaders actually cleanup after themselves
Rebase
vlorentz committed rDDEPb14b50eaa9b2: Use release instead of revision as anchor in SWHID context instead. (authored by vlorentz).
Use release instead of revision as anchor in SWHID context instead.
c0a98a5c4cfe4beac5c0b03bf61459d244b6f132
2311ad9b365e23d1a3c532aa005e32db46bc06f3
551c55ff04774b2c4e50e54f701c514064607f6a
f6905cdf547c8a1325fe8b3a0cbf6cb2373031a6
vlorentz committed rDLDBASE059c71b673c2: Refactor package loaders to make the version part of BasePackageInfo (authored by vlorentz).
Refactor package loaders to make the version part of BasePackageInfo
vlorentz committed rDLDBASE2af673b617c5: Refactor package loaders to remove temporary revision objects (authored by vlorentz).
Refactor package loaders to remove temporary revision objects
vlorentz committed rDLDBASE89417bb072aa: Make package loaders write releases instead of revisions (authored by vlorentz).
Make package loaders write releases instead of revisions
vlorentz committed rDLDBASE054967878093: Document how each package loader populates fields. (authored by vlorentz).
Document how each package loader populates fields.
vlorentz committed rDLDBASE2311ad9b365e: tests: Hide utilities from stack traces (authored by vlorentz).
tests: Hide utilities from stack traces
tests: Remove duplicate checks
vlorentz committed rDLDBASEaeffe01a2b39: opam: Write package definitions to the extrinsic metadata storage (authored by vlorentz).
opam: Write package definitions to the extrinsic metadata storage
vlorentz committed rDLDBASE551c55ff0477: package loaders: Make test failures more helpful (authored by vlorentz).
package loaders: Make test failures more helpful
deposit: Remove 'parent' deposit
vlorentz committed rDLDBASE18bbbae719fc: Add missing documentation for `get_metadata_authority`. (authored by vlorentz).
Add missing documentation for `get_metadata_authority`.
vlorentz committed rDLS6243f800b43f: cran: Pass the package name to the loader (authored by vlorentz).
cran: Pass the package name to the loader
In T3691#73518, @stsp wrote:There is another problem related to keywords: Some CVS-based projects use custom keywords, instead of the standard $Id$ keyword. This prevents wrong expansion of $Id$ when code is imported from one project to another. Usually the project's name will be used as the custom keyword name, such as $OpenBSD$ or $NetBSD$, instead of $Id$. At present, to expand keywords correctly in this case, we need to use the pserver access method to benefit from server-side keyword expansion. But we will end up with different hashes if rsync is used to import the same origin again. We might be able to auto-detect use of custom keywords if the rsync server allows access to the CVSROOT folder, but this is not always the case. If CVSROOT is hidden from rsync, the only reliable way to detect custom keywords would be a parameter that gets passed into the loader. We could, for example, allow passing the name of a custom keyword as a parameter embedded in the origin URL.
keeping this diff as planned changes as it seems there is a conflict with the elasticsearch version for the staging and swh-search elastic nodes (redeclare the elastic::elk_version property for swh-search)
*** Running octocatalog-diff on host search-esnode4.internal.softwareheritage.org
I, [2021-11-10T12:54:23.710611 #16530] INFO -- : Catalogs compiled for search-esnode4.internal.softwareheritage.org
I, [2021-11-10T12:54:23.998043 #16530] INFO -- : Diffs computed for search-esnode4.internal.softwareheritage.org
diff origin/production/search-esnode4.internal.softwareheritage.org current/search-esnode4.internal.softwareheritage.org
*******************************************
+ Apt::Pin[journalbeat] =>
parameters =>
"codename": "",
"component": "",
"ensure": "present",
"explanation": "Use the elk stack version",
"label": "",
"order": 50,
"origin": "",
"originator": "",
"packages": [
"journalbeat"
],
"priority": 1001,
"release": "",
"release_version": "",
"version": "7.9.3" <--------------- Not good
*******************************************
Apt::Pin[swh-journalbeat] =>
parameters =>
ensure =>
- present
+ absent
explanation =>
- Use journalbeat packages from Software Heritage
originator =>
- softwareheritage
+_
packages =>
- ["journalbeat"]
+ *
priority =>
- 990
+ 0
*******************************************
+ Apt::Setting[pref-journalbeat] =>
parameters =>
"content": "# This file is managed by Puppet. DO NOT EDIT.\nExplanation: Use...
"ensure": "present",
"notify_update": false,
"priority": 50
*******************************************
Apt::Setting[pref-swh-journalbeat] =>
parameters =>
content =>
@@ -1,5 +1,5 @@
# This file is managed by Puppet. DO NOT EDIT.
-Explanation: Use journalbeat packages from Software Heritage
-Package: journalbeat
-Pin: release o=softwareheritage
-Pin-Priority: 990
+Explanation: profile: swh-journalbeat
+Package: *
+Pin: release a=swh-journalbeat
+Pin-Priority: 0
ensure =>
- present
+ absent
*******************************************
+ File[/etc/apt/preferences.d/journalbeat.pref] =>
parameters =>
"content": "# This file is managed by Puppet. DO NOT EDIT.\nExplanation: Use...
"ensure": "present",
"group": "root",
"mode": "0644",
"owner": "root"
*******************************************
File[/etc/apt/preferences.d/swh-journalbeat.pref] =>
parameters =>
ensure =>
- present
+ absent
*******************************************
File[/etc/journalbeat/journalbeat.yml] =>
parameters =>
content =>
@@ -2,4 +2,10 @@
_
journalbeat:
+ inputs:
+ # Paths that should be crawled and fetched. Possible values files and directories.
+ # When setting a directory, all journals under it are merged.
+ # When empty starts to read from local journal.
+ - paths: []
+
# What position in journald to seek to at start up
# options: cursor, tail, head (defaults to tail)
*******************************************
- File[/etc/journalbeat]
*******************************************
+ File[/etc/systemd/system/journalbeat.service.d/journalbeat.conf] =>
parameters =>
"content": "# Managed by puppet (class profile::systemd_journal::journalbeat...
"ensure": "file",
"group": "root",
"mode": "0444",
"notify": [
"Class[Systemd::Systemctl::Daemon_reload]"
],
"owner": "root",
"selinux_ignore_defaults": false,
"show_diff": true
*******************************************
+ File[/etc/systemd/system/journalbeat.service.d] =>
parameters =>
"ensure": "directory",
"group": "root",
"owner": "root",
"purge": true,
"recurse": true,
"selinux_ignore_defaults": false
*******************************************
File[/etc/systemd/system/journalbeat.service] =>
parameters =>
ensure =>
- file
+ absent
*******************************************
Package[journalbeat] =>
parameters =>
ensure =>
- present
+ 7.9.3
*******************************************
Service[journalbeat] =>
parameters =>
subscribe =>
+ ["File[/etc/journalbeat/journalbeat.yml]", "Package[journalbeat]", "Systemd::Dropin_file[journalbeat.conf]"]
*******************************************
+ Systemd::Dropin_file[journalbeat.conf] =>
parameters =>
"content": "# Managed by puppet (class profile::systemd_journal::journalbeat...
"daemon_reload": "lazy",
"ensure": "present",
"filename": "journalbeat.conf",
"group": "root",
"mode": "0444",
"notify": [
"Service[journalbeat]"
],
"owner": "root",
"path": "/etc/systemd/system",
"selinux_ignore_defaults": false,
"show_diff": true,
"unit": "journalbeat.service"
*******************************************
- Systemd::Unit_file[journalbeat.service]
*******************************************
*** End octocatalog-diff on search-esnode4.internal.softwareheritage.orgAdd opam coq instance
vsellier added a revision to T3705: Upgrade the ELK stack: D6635: elk: upgrade journalbeat and filebeat.
vsellier added a revision to T3705: Upgrade the ELK stack: D6634: elk: upgrade elasticsearch and kibana nodes.
ardumont committed rSPSITEe65b0d874f60: svn: Make the celery work fork more often (authored by ardumont).
svn: Make the celery work fork more often
Done now, closing.
add CVS commit ID support to rlog.py
ardumont committed rDLDSVNd06d3e9c5a94: test: Ensure tests use tmp_path fixture so it's cleaned up properly (authored by ardumont).
test: Ensure tests use tmp_path fixture so it's cleaned up properly
ardumont committed rDLDSVN9ddf7115b8e6: Clean up loader, add types and update docstrings (authored by ardumont).
Clean up loader, add types and update docstrings
Why is check_revision not a boolean?
Build is green
Thanks! :D
@vlorentz, thx, I'll let you land your pile of diffs first ;)
Why is check_revision not a boolean?
Add some other tests which were missing the fix
Build is green
Thanks!
rebase
Build is green
last update
Build is green
Add more types
ardumont added a comment to P1215 temporary/working directories mess with the loader svn somewhere .
so:
/tmp/tmp0tm0182g/swh.loader.svn.2zdkkl_5-1740204/tmpi0q7t29s: mounted svn repository (from a dump) /tmp/swh.loader.svn.mhx2lwbz-1740204/tmpi0q7t29s: svn loader's repository disk copy growing along the ingestion
ardumont updated the task description for T3719: loader svn fails to clean up properly disk when it crashes.
Build is green
clang-format C code
Nov 9 2021
Nov 9 2021
ardumont added a comment to T3695: Investigate revision reconstruction discrepancy with subversion export.
I've triggered runs on those origins with the patched loader btw.
Build is green
Build is green
Anyway, there's a couple of issues I can think of:...
Align docstring definition with task module
Build is green
ardumont retitled D6630: Clean up loader, add types and update docstrings from Clean up loader, add types and more docstring to Clean up loader, add types and update docstrings.
Update more docstrings