Page MenuHomeSoftware Heritage

Restrict accepted timestamps to values that can be processed all along
Closed, MigratedEdits Locked

Description

Current validation of swh.model.model.TimeStamp allows second values in the [-2**63, 2**63[ range, but these limits are way too large for several storage backends.

The postgresql backend for example can only deal with year up to 294276, which cannot even be handled by python's datetime object (which are used as intermediate structure in swh.storage.postgresql.converters.date_to_db() and db_to_date()).

Python's datetime max year is 9999 (see https://docs.python.org/3/library/datetime.html#datetime.MAXYEAR).

Related to T3200

Related Objects

Event Timeline

douardda triaged this task as High priority.Apr 7 2021, 2:30 PM
douardda created this task.

looks like there is no revision with date or committer_date > 9999-12-31 in the main storage...

I just gave this a shot, and I can't find a way to encode large timestamps in postgresql without losing microsecond precision.

Looks like we'll need to change the schema (to store it like Cassandra's) if we want to support this.