type was deleted from the revision metadata set so it shouldn't be here either.

didn't we call the table origin_intrinsic_metadata so it won't have ambiguous connotation with the main db table origin_metadata?

is it necessary to know that the metadata for an origin is missing?
as we need to compute every time there is a new most_recent_HEAD_revision the metadata associated

an origin can't be identified by sha1
is it the origin_id?

sha1_git is not the origin identifier

I don't know, but it can't hurt to have this endpoint available

reping :-)
type isn't a property we need/should list in the metadata.

this might change with the refactoring of the origin_intrinsic_metadata table

cool you changed the name.
Now the question is about the PKEY, is it only the origin_id or not?
Do we want to keep different origin_intrinsic_metadata for different revisions/tools?
Or just one most recent copy for the most recent tool used?

pinging @zack on this question.

It might be more useful if more explicit
id -> origin_id

*Note that with the origin_metadata_translation the id was the object_id of an origin_metadata entry

you should delete all translation mentions

suggestion: result-> revision_metadata ?

the long lasting question, is it a indexer_configuration_tool for origins or the one used on the revision?
if it's the one used in the revision, maybe revision_tool_id is more explicit

view comment in 130

not specific to origin_metadata
suggestion :

copy of the most recent revision intrinsic_metadata found with tool

depends on comment in 132
is it the tool used for translation or for the detection of the most recent revision

Not commenting , but this should be changed with the other changes on sql files

ping @ardumont
I remember that there was a reason why there is no return to run, but maybe I'm wrong

Can you add explicit docs string about the arguments:

• revisions_metadata
• origin_head_pairs

seems a bit weird that the ids in this current run method are revision_ids
and the ones sent to super() are the origin_ids

more human comments about this would be nice :-)

I'm not sure what tool should be used for the origin_intrinsic _metadata
the tool used for the revision is a file detector and a metadata translator to CodeMeta
in the future we might want to use a different vocabulary, a different minimal set and maybe add files to the file detection procedure
so a revision can be indexed with different tool and have different results

That's why we said it is important to have the exact revision_metadata entry identifierd in the origin_intrinsic_metadata table.
I'm not sure if this should be with the tuple <revision_id, revision_tool_id> or create a revision_metadata object_id because at the moment there is no identifier on a revision_metadata entry.

I think it's important to specify what type of object this metadata describes:
content_metadata
revision_metadata
origin_intrinsic_metadata

If this result is the expected result of an origin_intrinsic_metadata, I think there is missing information.

here both entries have the same result but shouldn't

i'd like to see a test with one of the origins here.
But you might prefer to choose for your test a real url where there is in the root directory a package.json file
for example : https://github.com/librariesio/yarn-parser

Previously next_step did not return anything (triggered a new delayed task - mimetype indexer towards orchestrator text, and that was all IIRC).

I think that changes in that diff (yes, take a look at the OriginHeadIndexer's implementation below).

To sum up, as usual, it's ok to adapt if it's needed.

Hmm? I did remove it

Now the question is about the PKEY, is it only the origin_id or not?

Yes, it is.

Do we want to keep different origin_intrinsic_metadata for different revisions/tools?

We decided we wouldn't

you should delete all translation mentions

Already done

the ids in this current run method are revision_ids

They are not revision ids, they are the metadata itself. That's because they come from a previous Celery task, and Celery forces it to be a first argument... :/

more human comments about this would be nice :-)

Sure, will do

i'd like to see a test with one of the origins here.

What do you mean?

cool, lag between your new commit and my submition

here you have a list of origins in the MockStorage for tests.
I wasn't sure you used on of them for the OriginMetadataIndexer.
But now I see that you have.

I proposed also a real life origin url that should result with metadata with the entire pipeline.