diff --git a/status.org b/status.org index 26f7d00..1022e8d 100644 --- a/status.org +++ b/status.org @@ -1,35 +1,72 @@ #+title: Fetch google archive source code status #+author: ardumont -|-------------------------------+-------+--------------------------------+-------------+----------------------+-----------+---------+------------------------+---------------------+-----------+----------+----------------+-------------------------+-------------+--------+---------| -| date-snapshot | vhost | name | auto_delete | consumer_utilisation | consumers | durable | exclusive_consumer_tag | idle_since | memory | messages | messages_ready | messages_unacknowledged | node | policy | state | -|-------------------------------+-------+--------------------------------+-------------+----------------------+-----------+---------+------------------------+---------------------+-----------+----------+----------------+-------------------------+-------------+--------+---------| -| Wed Apr 13 11:08:34 CEST 2016 | / | swh_fetcher_googlecode_archive | False | 0.000450898352371 | 1 | True | | | 289876472 | 1356558 | 1356548 | 10 | rabbit@moma | | running | -|-------------------------------+-------+--------------------------------+-------------+----------------------+-----------+---------+------------------------+---------------------+-----------+----------+----------------+-------------------------+-------------+--------+---------| -| Wed Apr 13 17:12:25 CEST 2016 | / | swh_fetcher_googlecode_archive | False | 0.00046890593763 | 1 | True | | | 289876472 | 1348534 | 1348524 | 10 | rabbit@moma | | running | -|-------------------------------+-------+--------------------------------+-------------+----------------------+-----------+---------+------------------------+---------------------+-----------+----------+----------------+-------------------------+-------------+--------+---------| -| Wed Apr 13 18:21:27 CEST 2016 | / | swh_fetcher_googlecode_archive | False | 0.000623232116552 | 1 | True | | | 289876472 | 1343494 | 1343484 | 10 | rabbit@moma | | running | -|-------------------------------+-------+--------------------------------+-------------+----------------------+-----------+---------+------------------------+---------------------+-----------+----------+----------------+-------------------------+-------------+--------+---------| -| Thu Apr 14 10:27:42 CEST 2016 | / | swh_fetcher_googlecode_archive | False | 0.000432774634485 | 1 | True | | | 201303592 | 1279586 | 1279576 | 10 | rabbit@moma | | running | -|-------------------------------+-------+--------------------------------+-------------+----------------------+-----------+---------+------------------------+---------------------+-----------+----------+----------------+-------------------------+-------------+--------+---------| -| Thu Apr 14 12:21:49 CEST 2016 | / | swh_fetcher_googlecode_archive | False | | 1 | True | | 2016-04-14 12:21:46 | 131995280 | 1276740 | 1276730 | 10 | rabbit@moma | | running | -|-------------------------------+-------+--------------------------------+-------------+----------------------+-----------+---------+------------------------+---------------------+-----------+----------+----------------+-------------------------+-------------+--------+---------| -| Fri Apr 15 09:53:52 CEST 2016 | / | swh_fetcher_googlecode_archive | False | 6.64040372727e-05 | 1 | True | | | 116495792 | 1212887 | 1212877 | 10 | rabbit@moma | | running | -|-------------------------------+-------+--------------------------------+-------------+----------------------+-----------+---------+------------------------+---------------------+-----------+----------+----------------+-------------------------+-------------+--------+---------| +* Fetch archives + +|-------------------------------+----------------+ +| date-snapshot | messages_ready | +|-------------------------------+----------------+ +| Wed Apr 13 11:08:34 CEST 2016 | 1356548 | +|-------------------------------+----------------+ +| Wed Apr 13 17:12:25 CEST 2016 | 1348524 | +|-------------------------------+----------------+ +| Wed Apr 13 18:21:27 CEST 2016 | 1343484 | +|-------------------------------+----------------+ +| Thu Apr 14 10:27:42 CEST 2016 | 1279576 | +|-------------------------------+----------------+ +| Thu Apr 14 12:21:49 CEST 2016 | 1276730 | +|-------------------------------+----------------+ +| Fri Apr 15 09:53:52 CEST 2016 | 1212877 | +|-------------------------------+----------------+ Reinjected 120k googlecode urls to fetch the missing project.json. -|-------------------------------+-------+--------------------------------+-------------+----------------------+-----------+---------+------------------------+------------+-----------+----------+----------------+-------------------------+-------------+--------+---------| -| date-snapshot | vhost | name | auto_delete | consumer_utilisation | consumers | durable | exclusive_consumer_tag | idle_since | memory | messages | messages_ready | messages_unacknowledged | node | policy | state | -|-------------------------------+-------+--------------------------------+-------------+----------------------+-----------+---------+------------------------+------------+-----------+----------+----------------+-------------------------+-------------+--------+---------| -| Fri Apr 15 11:07:02 CEST 2016 | / | swh_fetcher_googlecode_archive | False | 0.000441398401257 | 1 | True | | | 255490688 | 1329722 | 1329712 | 10 | rabbit@moma | | running | -|-------------------------------+-------+--------------------------------+-------------+----------------------+-----------+---------+------------------------+------------+-----------+----------+----------------+-------------------------+-------------+--------+---------| -| Fri Apr 15 16:49:51 CEST 2016 | / | swh_fetcher_googlecode_archive | False | 0.0010567392203 | 1 | True | | | 212909168 | 1311209 | 1311199 | 10 | rabbit@moma | | running | -|-------------------------------+-------+--------------------------------+-------------+----------------------+-----------+---------+------------------------+------------+-----------+----------+----------------+-------------------------+-------------+--------+---------| -| Sun Apr 17 20:12:10 CEST 2016 | / | swh_fetcher_googlecode_archive | False | 0.00184863720413 | 1 | True | | | 23880640 | 1147234 | 1147224 | 10 | rabbit@moma | | running | +|-------------------------------+----------------+ +| date-snapshot | messages_ready | +|-------------------------------+----------------+ +| Fri Apr 15 11:07:02 CEST 2016 | 1329712 | +|-------------------------------+----------------+ +| Fri Apr 15 16:49:51 CEST 2016 | 1311199 | +|-------------------------------+----------------+ +| Sun Apr 17 20:12:10 CEST 2016 | 1147224 | +|-------------------------------+----------------+ #+BEGIN_SRC sh (let ((speed (swh-worker-average-speed-per-second "Sun Apr 17 20:12:10 CEST 2016" 1147224 "Fri Apr 15 11:07:02 CEST 2016" 1329712)) ;; 0.8879848959651205 j/s (remaining-jobs 1147224)) (swh-worker-remains-in-days speed remaining-jobs));; 14.95301960189772 #+END_SRC + +* Check archives + +** round-1 + +|-------------------------------+----------------+ +| date-snapshot | messages_ready | +|-------------------------------+----------------+ +| Thu May 05 23:32:35 CEST 2016 | 1302268 | +| Fri May 06 10:47:07 CEST 2016 | 1258667 | +|-------------------------------+----------------| + +#+BEGIN_SRC sh +(let ((speed (swh-worker-average-speed-per-second "Thu May 05 23:32:35 CEST 2016" 1302268 "Fri May 06 10:47:07 CEST 2016" 1258667)) ;; 1.0773127100217434 j/s + (remaining-jobs 1258667)) + (swh-worker-remains-in-days speed remaining-jobs));; 13.522447992188424 remaining days +#+END_SRC + +** round-2 + +|-------------------------------+--------------------------------------+--------------------------------------+-------------+----------------------+-----------+---------+------------------------+------------+-----------+----------+----------------+-------------------------+-------------+---------+---------| +| date-snapshot | vhost | name | auto_delete | consumer_utilisation | consumers | durable | exclusive_consumer_tag | idle_since | memory | messages | messages_ready | messages_unacknowledged | node | policy | state | +|-------------------------------+--------------------------------------+--------------------------------------+-------------+----------------------+-----------+---------+------------------------+------------+-----------+----------+----------------+-------------------------+-------------+---------+---------| +| Fri May 06 11:46:46 CEST 2016 | / | swh_fetcher_googlecode_check_archive | False | 0.00789511690925 | 1 | True | | | 147854464 | 1374666 | 1374646 | 20 | rabbit@moma | | running | +| Sat May 07 09:46:57 CEST 2016 | / | swh_fetcher_googlecode_check_archive | False | 0.00166540633778 | 1 | True | | | 147854624 | 1294810 | 1294790 | 20 | rabbit@moma | | running | +| Sun May 08 10:40:23 CEST 2016 | / | swh_fetcher_googlecode_check_archive | False | 0.00012717060938 | 1 | True | | | 147855064 | 1198014 | 1197994 | 20 | rabbit@moma | | running | +| Mon May 09 09:27:17 CEST 2016 | / | swh_fetcher_googlecode_check_archive | False | 0.000293816037837 | 1 | True | | | 49518176 | 1130639 | 1130619 | 20 | rabbit@moma | | running | +| Thu May 12 15:00:55 CEST 2016 | / | swh_fetcher_googlecode_check_archive | False | 0.00089235353294 | 1 | True | | | 13822064 | 858386 | 858371 | 15 | rabbit@moma | | running | + +#+BEGIN_SRC sh +(let ((speed (swh-worker-average-speed-per-second "Fri May 06 11:46:46 CEST 2016" 1374646 "Sat May 07 09:46:57 CEST 2016" 1294790)) ;; 1.0081428084483215 j/s + (remaining-jobs 1294790)) + (swh-worker-remains-in-days speed remaining-jobs));; 14.864952906261363 remaining days +#+END_SRC