Page MenuHomeSoftware Heritage
Paste P1005

Convert webapp historical data to historical json
ActivePublic
Actions

Authored by vsellier on Apr 13 2021, 11:17 AM.
from swh.web.common.utils import parse_iso8601_date_to_utc
import json
_stat_counters_backfill = {
"2015-09-01": {"revision": 0, "origin": 0, "content": 0},
"2016-07-01": {"revision": 594305600, "origin": 22777052},
"2016-09-14": {"revision": 644628800, "origin": 25258776},
"2016-11-24": {"revision": 704845952, "origin": 53488904},
"2017-05-10": {"revision": 780882048, "origin": 58257484},
"2017-09-26": {"revision": 853277241, "origin": 65546644},
"2018-01-24": {"revision": 943061517, "origin": 71814787},
"2018-02-13": {"revision": 946216028, "origin": 81655813},
"2018-03-25": {"revision": 980390191, "origin": 83797945},
"2018-10-04": {"revision": 1126348335, "origin": 85202432},
"2019-01-27": {"revision": 1248389319, "origin": 88288721},
"2019-04-08": {"revision": 1293870115, "origin": 88297714},
"2019-06-27": {"revision": 1326776432, "origin": 89301694},
"2019-07-24": {"revision": 1358421267, "origin": 89601149},
"2019-09-22": {"revision": 1379380527, "origin": 90231104},
"2019-09-29": {"revision": 1385477933, "origin": 90487661},
"2020-01-01": {"revision": 1414420369, "origin": 91400586},
"2020-02-06": {"revision": 1428955761, "origin": 91512130},
"2020-04-07": {"revision": 1590436149, "origin": 107875943},
"2020-05-17": {"revision": 1717420203, "origin": 121172621},
"2020-05-27": {"revision": 1744034936, "origin": 123781438},
}
stat_counters_history={ "revision": [], "origin":[], "content":[]}
for d, object_counts in _stat_counters_backfill.items():
# convert date to javascript timestamp (in ms)
timestamp = int(parse_iso8601_date_to_utc(d).timestamp()) * 1000
for object_type, object_count in object_counts.items():
stat_counters_history[object_type].append([timestamp, object_count])
print(json.dumps(stat_counters_history))

Event Timeline

vsellier created this paste.Apr 13 2021, 11:17 AM

this is the result to add to the current historical data:

{"revision": [[1441065600000, 0], [1467331200000, 594305600], [1473811200000, 644628800], [1479945600000, 704845952], [1494374400000, 780882048], [1506384000000, 853277241], [1516752000000, 943061517], [1518480000000, 946216028], [1521936000000, 980390191], [1538611200000, 1126348335], [1548547200000, 1248389319], [1554681600000, 1293870115], [1561593600000, 1326776432], [1563926400000, 1358421267], [1569110400000, 1379380527], [1569715200000, 1385477933], [1577836800000, 1414420369], [1580947200000, 1428955761], [1586217600000, 1590436149], [1589673600000, 1717420203], [1590537600000, 1744034936]], "origin": [[1441065600000, 0], [1467331200000, 22777052], [1473811200000, 25258776], [1479945600000, 53488904], [1494374400000, 58257484], [1506384000000, 65546644], [1516752000000, 71814787], [1518480000000, 81655813], [1521936000000, 83797945], [1538611200000, 85202432], [1548547200000, 88288721], [1554681600000, 88297714], [1561593600000, 89301694], [1563926400000, 89601149], [1569110400000, 90231104], [1569715200000, 90487661], [1577836800000, 91400586], [1580947200000, 91512130], [1586217600000, 107875943], [1589673600000, 121172621], [1590537600000, 123781438]], "content": [[1441065600000, 0]]}
vsellier mentioned this in T3232: remove hardcoded historical values from the webapp.Apr 13 2021, 11:21 AM
About · Developer information · Legal