Page MenuHomeSoftware Heritage
Harbormaster B473 Build 696

Build 696
Public
Actions

You are viewing an older run of this build. View Current Build

Properties

Buildable
B473: rDSTOd0cbf0a54917: Archiver Director/Worker: Add copy to backend worker implementation
Build Plan
Plan 2 Software Heritage Python tests
Status
Passed

History

Run 3
Run 2
Run 1

Lint

When
Completed at Sep 14 2016, 12:12 PM · Built for 3 s
Status
Passed
1 empty logs are hidden. Show all logs.

Build Log 2395 (remote - stdout)
25 - 50 - 100 - Unlimited Lines
New View (Prototype)

1python3 -m flake8 swh bin/swh-storage-add-dir
2

Python tests

When
Completed at Sep 14 2016, 12:11 PM · Built for 2 s
Status
Failed
1 empty logs are hidden. Show all logs.

Build Log 2394 (remote - stderr)
25 - 50 - 100 - Unlimited Lines
New View (Prototype)

1W: unknown repository 'swh-environment', you might want to remove it
2Failure: ImportError (No module named 'azure') ... ERROR
3db_to_author (test_converters.TestConverters) ... ok
4db_to_git_headers (test_converters.TestConverters) ... ok
5db_to_release (test_converters.TestConverters) ... ok
6db_to_revision (test_converters.TestConverters) ... ok
7
8======================================================================
9ERROR: Failure: ImportError (No module named 'azure')
10----------------------------------------------------------------------
11Traceback (most recent call last):
12 File "/usr/lib/python3/dist-packages/nose/failure.py", line 39, in runTest
13 raise self.exc_val.with_traceback(self.tb)
14 File "/usr/lib/python3/dist-packages/nose/loader.py", line 414, in loadTestsFromName
15 addr.filename, addr.module)
16 File "/usr/lib/python3/dist-packages/nose/importer.py", line 47, in importFromPath
17 return self.importFromDir(dir_path, fqname)
18 File "/usr/lib/python3/dist-packages/nose/importer.py", line 94, in importFromDir
19 mod = load_module(part_fqname, fh, filename, desc)
20 File "/usr/lib/python3.4/imp.py", line 235, in load_module
21 return load_source(name, filename, file)
22 File "/usr/lib/python3.4/imp.py", line 171, in load_source
23 module = methods.load()
24 File "<frozen importlib._bootstrap>", line 1220, in load
25 File "<frozen importlib._bootstrap>", line 1200, in _load_unlocked
26 File "<frozen importlib._bootstrap>", line 1129, in _exec
27 File "<frozen importlib._bootstrap>", line 1471, in exec_module
28 File "<frozen importlib._bootstrap>", line 321, in _call_with_frames_removed
29 File "/home/temp-drydock/drydock/workingcopy-169/repo/swh-storage/swh/storage/tests/test_archiver.py", line 19, in <module>
30 from swh.storage.archiver import ArchiverWithRetentionPolicyDirector
31 File "/home/temp-drydock/drydock/workingcopy-169/repo/swh-storage/swh/storage/archiver/__init__.py", line 7, in <module>
32 from swh.objstorage.cloud import AzureCloudObjStorage # noqa
33 File "/home/temp-drydock/drydock/workingcopy-169/repo/swh-objstorage/swh/objstorage/cloud/__init__.py", line 2, in <module>
34 from .objstorage_azure import AzureCloudObjStorage
35 File "/home/temp-drydock/drydock/workingcopy-169/repo/swh-objstorage/swh/objstorage/cloud/objstorage_azure.py", line 12, in <module>
36 from azure.storage.blob import BlockBlobService
37ImportError: No module named 'azure'
38
39----------------------------------------------------------------------
40Ran 5 tests in 0.638s
41
42FAILED (errors=1)
43

Update environment from working copy

When
Completed at Sep 14 2016, 12:11 PM · Built for 16 s
Status
Passed
1 empty logs are hidden. Show all logs.

Build Log 2389 (remote - stdout)
25 - 50 - 100 - Unlimited Lines
New View (Prototype)

1mr update: /home/temp-drydock/drydock/workingcopy-169/repo/swh-environment/swh-cloner-git
2mr update: /home/temp-drydock/drydock/workingcopy-169/repo/swh-environment/swh-cloner-git
3Already up-to-date.
4
5mr update: /home/temp-drydock/drydock/workingcopy-169/repo/swh-environment/swh-journal
6mr update: /home/temp-drydock/drydock/workingcopy-169/repo/swh-environment/swh-journal
7Already up-to-date.
8
9mr update: /home/temp-drydock/drydock/workingcopy-169/repo/swh-environment/swh-core
10mr update: /home/temp-drydock/drydock/workingcopy-169/repo/swh-environment/swh-core
11Already up-to-date.
12
13mr update: /home/temp-drydock/drydock/workingcopy-169/repo/swh-environment/snippets
14mr update: /home/temp-drydock/drydock/workingcopy-169/repo/swh-environment/snippets
15Already up-to-date.
16
17mr update: /home/temp-drydock/drydock/workingcopy-169/repo/swh-environment/swh-fetcher-googlecode
18mr update: /home/temp-drydock/drydock/workingcopy-169/repo/swh-environment/swh-fetcher-googlecode
19Already up-to-date.
20
21mr update: /home/temp-drydock/drydock/workingcopy-169/repo/swh-environment/swh-lister-github
22mr update: /home/temp-drydock/drydock/workingcopy-169/repo/swh-environment/swh-lister-github
23From ssh://forge.softwareheritage.org/diffusion/DLSGH/swh-lister-github
24 9550ca3..e2a96f1 master -> origin/master
25 * [new branch] qless-refactoring -> origin/qless-refactoring
26Updating 9550ca3..e2a96f1
27Fast-forward
28 README | 24 +++--
29 TODO | 8 ++
30 bin/batch | 41 ++++++++
31 bin/ghlister | 150 +++++++++++++++++++++++++++
32 bin/reset.sh | 9 ++
33 bin/status | 18 ++++
34 etc/crontab | 5 +
35 requirements.txt | 7 +-
36 sql/crawler.sql | 106 +++++++++++++++++++
37 sql/pimp_db.sql | 36 +++++++
38 swh/lister/github/cache.py | 40 --------
39 swh/lister/github/constants.py | 7 --
40 swh/lister/github/converters.py | 95 -----------------
41 swh/lister/github/db_utils.py | 18 ++++
42 swh/lister/github/generate_tasks.py | 111 ++++++++++++++++++++
43 swh/lister/github/github_api.py | 137 -------------------------
44 swh/lister/github/lister.py | 197 +++++++++++++++++++++++++++++-------
45 swh/lister/github/models.py | 44 ++++++++
46 swh/lister/github/processors.py | 163 -----------------------------
47 swh/lister/github/req_queue.py | 31 ------
48 swh/lister/github/storage_utils.py | 117 ---------------------
49 21 files changed, 727 insertions(+), 637 deletions(-)
50 create mode 100755 bin/batch
51 create mode 100755 bin/ghlister
52 create mode 100644 bin/reset.sh
53 create mode 100755 bin/status
54 create mode 100644 etc/crontab
55 create mode 100644 sql/crawler.sql
56 create mode 100644 sql/pimp_db.sql
57 delete mode 100644 swh/lister/github/cache.py
58 delete mode 100644 swh/lister/github/constants.py
59 delete mode 100644 swh/lister/github/converters.py
60 create mode 100644 swh/lister/github/db_utils.py
61 create mode 100644 swh/lister/github/generate_tasks.py
62 delete mode 100644 swh/lister/github/github_api.py
63 create mode 100644 swh/lister/github/models.py
64 delete mode 100644 swh/lister/github/processors.py
65 delete mode 100644 swh/lister/github/req_queue.py
66 delete mode 100644 swh/lister/github/storage_utils.py
67
68mr update: /home/temp-drydock/drydock/workingcopy-169/repo/swh-environment/swh-loader-debian
69mr update: /home/temp-drydock/drydock/workingcopy-169/repo/swh-environment/swh-loader-debian
70Already up-to-date.
71
72mr update: /home/temp-drydock/drydock/workingcopy-169/repo/swh-environment/swh-loader-core
73mr update: /home/temp-drydock/drydock/workingcopy-169/repo/swh-environment/swh-loader-core
74Already up-to-date.
75
76mr update: /home/temp-drydock/drydock/workingcopy-169/repo/swh-environment/swh-loader-antelink
77mr update: /home/temp-drydock/drydock/workingcopy-169/repo/swh-environment/swh-loader-antelink
78Already up-to-date.
79
80mr update: /home/temp-drydock/drydock/workingcopy-169/repo/swh-environment/swh-loader-dir
81mr update: /home/temp-drydock/drydock/workingcopy-169/repo/swh-environment/swh-loader-dir
82Already up-to-date.
83
84mr update: /home/temp-drydock/drydock/workingcopy-169/repo/swh-environment/swh-loader-git
85mr update: /home/temp-drydock/drydock/workingcopy-169/repo/swh-environment/swh-loader-git
86Already up-to-date.
87
88mr update: /home/temp-drydock/drydock/workingcopy-169/repo/swh-environment/swh-loader-svn
89mr update: /home/temp-drydock/drydock/workingcopy-169/repo/swh-environment/swh-loader-svn
90Already up-to-date.
91
92mr update: /home/temp-drydock/drydock/workingcopy-169/repo/swh-environment/swh-loader-tar
93mr update: /home/temp-drydock/drydock/workingcopy-169/repo/swh-environment/swh-loader-tar
94Already up-to-date.
95
96mr update: /home/temp-drydock/drydock/workingcopy-169/repo/swh-environment/swh-model
97mr update: /home/temp-drydock/drydock/workingcopy-169/repo/swh-environment/swh-model
98Already up-to-date.
99
100mr update: /home/temp-drydock/drydock/workingcopy-169/repo/swh-environment/swh-objstorage
101mr update: /home/temp-drydock/drydock/workingcopy-169/repo/swh-environment/swh-objstorage
102From ssh://forge.softwareheritage.org/diffusion/DOBJS/swh-objstorage
103 629f836..87c85be master -> origin/master
104 * [new tag] v0.0.9 -> v0.0.9
105Updating 629f836..87c85be
106Fast-forward
107 bin/swh-objstorage-azure | 6 +++++-
108 swh/objstorage/__init__.py | 28 +++++++++++++++++++++++++++-
109 swh/objstorage/cloud/__init__.py | 8 +++++++-
110 swh/objstorage/cloud/objstorage_azure.py | 14 ++++++++++----
111 swh/objstorage/objstorage_pathslicing.py | 2 +-
112 5 files changed, 50 insertions(+), 8 deletions(-)
113
114mr update: /home/temp-drydock/drydock/workingcopy-169/repo/swh-environment/swh-py-template
115mr update: /home/temp-drydock/drydock/workingcopy-169/repo/swh-environment/swh-py-template
116Already up-to-date.
117
118mr update: /home/temp-drydock/drydock/workingcopy-169/repo/swh-environment/swh-scheduler
119mr update: /home/temp-drydock/drydock/workingcopy-169/repo/swh-environment/swh-scheduler
120Already up-to-date.
121
122mr update: /home/temp-drydock/drydock/workingcopy-169/repo/swh-environment/swh-storage-testdata
123mr update: /home/temp-drydock/drydock/workingcopy-169/repo/swh-environment/swh-storage-testdata
124From ssh://forge.softwareheritage.org/diffusion/DSTOT/swh-storage-testdata
125 a61dd8e..01a27bc master -> origin/master
126Updating a61dd8e..01a27bc
127Fast-forward
128 dumps/swh-archiver.dump | Bin 8586 -> 8097 bytes
129 dumps/swh-archiver.sql | 27 ++------
130 dumps/swh.dump | Bin 122848 -> 121778 bytes
131 dumps/swh.sql | 161 ++++++++++++++++++++----------------------------
132 4 files changed, 72 insertions(+), 116 deletions(-)
133
134mr update: /home/temp-drydock/drydock/workingcopy-169/repo/swh-environment/swh-web-ui
135mr update: /home/temp-drydock/drydock/workingcopy-169/repo/swh-environment/swh-web-ui
136Already up-to-date.
137
138mr update: /home/temp-drydock/drydock/workingcopy-169/repo/swh-environment/swh-storage
139mr update: /home/temp-drydock/drydock/workingcopy-169/repo/swh-environment/swh-storage
140From ssh://forge.softwareheritage.org/diffusion/DSTO/swh-storage
141 a71109b..791444a master -> origin/master
142Updating a71109b..791444a
143Fast-forward
144 debian/control | 3 +-
145 requirements.txt | 1 +
146 sql/archiver/swh-archiver-data.sql | 4 +-
147 sql/archiver/swh-archiver-schema.sql | 12 +-
148 sql/archiver/upgrades/004.sql | 12 +
149 sql/swh-func.sql | 17 ++
150 sql/swh-schema.sql | 2 +-
151 sql/upgrades/080.sql | 23 ++
152 swh/storage/api/client.py | 4 +
153 swh/storage/api/server.py | 6 +
154 swh/storage/archiver/__init__.py | 15 +-
155 swh/storage/archiver/db.py | 5 +-
156 swh/storage/archiver/director.py | 145 +++++++++---
157 swh/storage/archiver/tasks.py | 17 +-
158 swh/storage/archiver/worker.py | 411 ++++++++++++++++++++++++-----------
159 swh/storage/db.py | 20 +-
160 swh/storage/storage.py | 17 ++
161 swh/storage/tests/test_archiver.py | 34 +--
162 swh/storage/tests/test_storage.py | 87 ++++++++
163 19 files changed, 631 insertions(+), 204 deletions(-)
164 create mode 100644 sql/archiver/upgrades/004.sql
165 create mode 100644 sql/upgrades/080.sql
166
167mr update: finished (20 ok)
168

Clone on Software Heritage Worker

When
Completed at Sep 14 2016, 12:11 PM · Built instantly
Status
Passed
About · Developer information · Legal