Okay so the issue was that the fulltext search wasn't actually using indexes. So lintime instead of logtime :/
- Queries
- All Stories
- Search
- Advanced Search
- Transactions
- Transaction Logs
Feed Advanced Search
Advanced Search
Advanced Search
Feb 5 2019
Feb 5 2019
In T1517#27866, @zack wrote:how is that "understandable"? :-)
how is that "understandable"? :-)
What's the python3-kafka version?
Feb 4 2019
Feb 4 2019
Feb 2 2019
Feb 2 2019
P353 is a quick and dirty wrapper to play with licensee locally after git clone
Jan 31 2019
Jan 31 2019
The only remaining crashes are caused by T1507.
Bottleneck: T1507
Jan 30 2019
Jan 30 2019
Jan 29 2019
Jan 29 2019
Resolved by D987.
vlorentz closed T1483: Store the type of intrinsic metadata that were extracted, a subtask of T1484: Provide stats on extracted metadata in the indexer storage api, as Resolved.
Resolved by D1010.
vlorentz closed T1474: Mapping warnings should log the content/revision id, not the content itself as Resolved.
Resolved by D998.
Resolved by D956.
Jan 25 2019
Jan 25 2019
Jan 24 2019
Jan 24 2019
Resolved by D994.
Resolved by D879.
Jan 21 2019
Jan 21 2019
vlorentz added a revision to T1328: Add Ruby/Gem metadata indexer: D957: Factorize list merges in indexer mappings.
In T1483#27359, @vlorentz wrote:You are correct, except I will store mapping names, not file names (eg. because gemspec files are usually named project_name.gemspec, which is harder to query).
You are correct, except I will store mapping names, not file names (eg. because gemspec files are usually named project_name.gemspec, which is harder to query).
(2) seems the best option to me.
Resolved by D747.
As some revisions/origins may have more than one metadata file (in which case we merge them), there is a m2m relation between revision/origin metadata rows and mappings. I see three ways to do it:
vlorentz renamed T1475: Test more edge cases of metadata indexer mappings from Add more test for edge cases of indexer mappings. to Add more tests for edge cases of indexer mappings..
zack added a project to T1484: Provide stats on extracted metadata in the indexer storage api: Metadata workflow.
zack added a project to T1483: Store the type of intrinsic metadata that were extracted: Metadata workflow.
vlorentz renamed T1484: Provide stats on extracted metadata in the indexer storage api from Provide stats on extracted metadata to Provide stats on extracted metadata in the indexer storage api.
vlorentz renamed T1484: Provide stats on extracted metadata in the indexer storage api from Show stats on extracted metadata to Provide stats on extracted metadata.
Useful queries:
vlorentz triaged T1484: Provide stats on extracted metadata in the indexer storage api as Normal priority.
vlorentz triaged T1483: Store the type of intrinsic metadata that were extracted as Normal priority.
Jan 16 2019
Jan 16 2019
" AttributeError on @id with a colon but less than two slashes" https://github.com/digitalbazaar/pyld/issues/91 -> T4436
vlorentz triaged T1474: Mapping warnings should log the content/revision id, not the content itself as Normal priority.
Jan 15 2019
Jan 15 2019
vlorentz added revisions to T1385: Monitor output of metadata indexers: D953: Prevent repository normalization from crashing on malformed input., D950: Prevent license normalization from crashing on malformed input., D949: Better error message on content retrieval error + don't write translated_metadata=None to the idx storage., D951: Fix normalization of npm's 'repository' and 'bugs' fields..
ardumont closed T439: Indexers: compute (and maintain up-to-date) the filetype of all blobs as Resolved.
I'd say using grafana but first we need to determine what are the sensible metrics ;)
We need to rework the current indexer implementation to use range instead (T991).
After that, we can schedule 256 ranges of contents to index using the scheduler stack instead.
And see where that goes.
ardumont closed T359: Indexers: batch content analyzer infrastructure, a subtask of T1385: Monitor output of metadata indexers, as Resolved.
ardumont closed T359: Indexers: batch content analyzer infrastructure, a subtask of T439: Indexers: compute (and maintain up-to-date) the filetype of all blobs, as Resolved.
Tests should be refactored to use celery's pytest fixtures, see D948
Jan 14 2019
Jan 14 2019
ardumont closed T1466: Deploy swh-indexer-journal-client, a subtask of T1385: Monitor output of metadata indexers, as Resolved.
ardumont closed T1468: journal: Deploy the publisher, a subtask of T1466: Deploy swh-indexer-journal-client, as Resolved.
Jan 13 2019
Jan 13 2019
heads up btw: It's deployed.
ardumont added a subtask for T1466: Deploy swh-indexer-journal-client: T1468: journal: Deploy the publisher.
ardumont closed T1277: swh-journal: Create a journal client for listing origin visits, a subtask of T1466: Deploy swh-indexer-journal-client, as Resolved.
D918 closes this.
Jan 11 2019
Jan 11 2019
vlorentz lowered the priority of T861: mimetype indexer: edge case makes the indexer fail miserably from Normal to Low.
Jan 10 2019
Jan 10 2019
vlorentz closed T1465: Journal notification of full origin visits, a subtask of T1455: Add a journal client that schedules oneshot tasks for metadata indexers, as Resolved.
vlorentz closed T1455: Add a journal client that schedules oneshot tasks for metadata indexers, a subtask of T1385: Monitor output of metadata indexers, as Resolved.
vlorentz closed T1458: docker containers for metadata indexers, a subtask of T1455: Add a journal client that schedules oneshot tasks for metadata indexers, as Resolved.
Jan 9 2019
Jan 9 2019
vlorentz lowered the priority of T1464: Auto-detect indexer tool versions instead of reading them from the config from Normal to Low.
ardumont updated the task description for T1455: Add a journal client that schedules oneshot tasks for metadata indexers.
ardumont updated the task description for T1455: Add a journal client that schedules oneshot tasks for metadata indexers.
vlorentz renamed T1455: Add a journal client that schedules oneshot tasks for metadata indexers from Add a journal client that schedules metadata indexers to Add a journal client that schedules oneshot tasks for metadata indexers.
vlorentz renamed T1455: Add a journal client that schedules oneshot tasks for metadata indexers from Schedule recurrent jobs for metadata indexers to Add a journal client that schedules metadata indexers.
vlorentz added a comment to T1464: Auto-detect indexer tool versions instead of reading them from the config.
Invalid, no need for ranges when dealing with origins.
vlorentz closed T1456: Make metadata indexers support ranges, a subtask of T1455: Add a journal client that schedules oneshot tasks for metadata indexers, as Invalid.
Jan 8 2019
Jan 8 2019
vlorentz added a comment to T1464: Auto-detect indexer tool versions instead of reading them from the config.
Proposal for Python packages:
vlorentz added a comment to T1464: Auto-detect indexer tool versions instead of reading them from the config.
any version self-declared by the tool in use