Page MenuHomeSoftware Heritage
Files F9124712

No OneTemporary
Actions

View Options

diff --git a/PKG-INFO b/PKG-INFO
index 47b409e..35f7353 100644
--- a/PKG-INFO
+++ b/PKG-INFO
@@ -1,65 +1,65 @@
Metadata-Version: 2.1
Name: swh.scheduler
-Version: 0.0.51
+Version: 0.0.52
Summary: Software Heritage Scheduler
Home-page: https://forge.softwareheritage.org/diffusion/DSCH/
Author: Software Heritage developers
Author-email: swh-devel@inria.fr
License: UNKNOWN
-Project-URL: Funding, https://www.softwareheritage.org/donate
Project-URL: Source, https://forge.softwareheritage.org/source/swh-scheduler
Project-URL: Bug Reports, https://forge.softwareheritage.org/maniphest
+Project-URL: Funding, https://www.softwareheritage.org/donate
Description: swh-scheduler
=============
Job scheduler for the Software Heritage project.
Task manager for asynchronous/delayed tasks, used for both recurrent (e.g.,
listing a forge, loading new stuff from a Git repository) and one-off
activities (e.g., loading a specific version of a source package).
# Tests
## Running test manually
### Test data
To be able to run (unit) tests, you need to have the
[[https://forge.softwareheritage.org/source/swh-storage-testdata.git|swh-storage-testdata]]
in the parent directory. If you have set your environment following the
[[ https://docs.softwareheritage.org/devel/getting-started.html#getting-started|Getting started]]
document everything should be set up just fine.
Otherwise:
```
~/.../swh-scheduler$ git clone https://forge.softwareheritage.org/source/swh-storage-testdata.git ../swh-storage-testdata
```
### Required services
Unit tests that require a running celery broker uses an in memory broker/result
backend by default, but you can choose to use a true broker by setting
`CELERY_BROKER_URL` and `CELERY_RESULT_BACKEND` environment variables up.
For example:
```
$ CELERY_BROKER_URL=amqp://localhost pifpaf run postgresql nosetests
.....................................
----------------------------------------------------------------------
Ran 37 tests in 15.578s
OK
```
Platform: UNKNOWN
Classifier: Programming Language :: Python :: 3
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: GNU General Public License v3 (GPLv3)
Classifier: Operating System :: OS Independent
Classifier: Development Status :: 5 - Production/Stable
Description-Content-Type: text/markdown
Provides-Extra: testing
diff --git a/swh.scheduler.egg-info/PKG-INFO b/swh.scheduler.egg-info/PKG-INFO
index 47b409e..35f7353 100644
--- a/swh.scheduler.egg-info/PKG-INFO
+++ b/swh.scheduler.egg-info/PKG-INFO
@@ -1,65 +1,65 @@
Metadata-Version: 2.1
Name: swh.scheduler
-Version: 0.0.51
+Version: 0.0.52
Summary: Software Heritage Scheduler
Home-page: https://forge.softwareheritage.org/diffusion/DSCH/
Author: Software Heritage developers
Author-email: swh-devel@inria.fr
License: UNKNOWN
-Project-URL: Funding, https://www.softwareheritage.org/donate
Project-URL: Source, https://forge.softwareheritage.org/source/swh-scheduler
Project-URL: Bug Reports, https://forge.softwareheritage.org/maniphest
+Project-URL: Funding, https://www.softwareheritage.org/donate
Description: swh-scheduler
=============
Job scheduler for the Software Heritage project.
Task manager for asynchronous/delayed tasks, used for both recurrent (e.g.,
listing a forge, loading new stuff from a Git repository) and one-off
activities (e.g., loading a specific version of a source package).
# Tests
## Running test manually
### Test data
To be able to run (unit) tests, you need to have the
[[https://forge.softwareheritage.org/source/swh-storage-testdata.git|swh-storage-testdata]]
in the parent directory. If you have set your environment following the
[[ https://docs.softwareheritage.org/devel/getting-started.html#getting-started|Getting started]]
document everything should be set up just fine.
Otherwise:
```
~/.../swh-scheduler$ git clone https://forge.softwareheritage.org/source/swh-storage-testdata.git ../swh-storage-testdata
```
### Required services
Unit tests that require a running celery broker uses an in memory broker/result
backend by default, but you can choose to use a true broker by setting
`CELERY_BROKER_URL` and `CELERY_RESULT_BACKEND` environment variables up.
For example:
```
$ CELERY_BROKER_URL=amqp://localhost pifpaf run postgresql nosetests
.....................................
----------------------------------------------------------------------
Ran 37 tests in 15.578s
OK
```
Platform: UNKNOWN
Classifier: Programming Language :: Python :: 3
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: GNU General Public License v3 (GPLv3)
Classifier: Operating System :: OS Independent
Classifier: Development Status :: 5 - Production/Stable
Description-Content-Type: text/markdown
Provides-Extra: testing
diff --git a/swh/scheduler/tests/conftest.py b/swh/scheduler/tests/conftest.py
index cd9ff8f..db13bb9 100644
--- a/swh/scheduler/tests/conftest.py
+++ b/swh/scheduler/tests/conftest.py
@@ -1,96 +1,96 @@
import os
import pytest
import glob
from datetime import timedelta
from swh.core.utils import numfile_sortkey as sortkey
from swh.scheduler import get_scheduler
from swh.scheduler.tests import SQL_DIR
# make sure we are not fooled by CELERY_ config environment vars
for var in [x for x in os.environ.keys() if x.startswith('CELERY')]:
os.environ.pop(var)
import swh.scheduler.celery_backend.config # noqa
# this import is needed here to enforce creation of the celery current app
# BEFORE the swh_app fixture is called, otherwise the Celery app instance from
# celery_backend.config becomes the celery.current_app
# test_cli tests depends on a en/C locale, so ensure it
os.environ['LC_ALL'] = 'C.UTF-8'
DUMP_FILES = os.path.join(SQL_DIR, '*.sql')
# celery tasks for testing purpose; tasks themselves should be
# in swh/scheduler/tests/tasks.py
TASK_NAMES = ['ping', 'multiping', 'add', 'error']
@pytest.fixture(scope='session')
def celery_enable_logging():
return True
@pytest.fixture(scope='session')
def celery_includes():
return [
'swh.scheduler.tests.tasks',
]
@pytest.fixture(scope='session')
def celery_parameters():
return {
'task_cls': 'swh.scheduler.task:SWHTask',
}
@pytest.fixture(scope='session')
def celery_config():
return {
'accept_content': ['application/x-msgpack', 'application/json'],
'task_serializer': 'msgpack',
- 'result_serializer': 'msgpack',
+ 'result_serializer': 'json',
}
# override the celery_session_app fixture to monkeypatch the 'main'
# swh.scheduler.celery_backend.config.app Celery application
# with the test application.
@pytest.fixture(scope='session')
def swh_app(celery_session_app):
swh.scheduler.celery_backend.config.app = celery_session_app
yield celery_session_app
@pytest.fixture
def swh_scheduler(request, postgresql_proc, postgresql):
scheduler_config = {
'db': 'postgresql://{user}@{host}:{port}/{dbname}'.format(
host=postgresql_proc.host,
port=postgresql_proc.port,
user='postgres',
dbname='tests')
}
all_dump_files = sorted(glob.glob(DUMP_FILES), key=sortkey)
cursor = postgresql.cursor()
for fname in all_dump_files:
with open(fname) as fobj:
cursor.execute(fobj.read())
postgresql.commit()
scheduler = get_scheduler('local', scheduler_config)
for taskname in TASK_NAMES:
scheduler.create_task_type({
'type': 'swh-test-{}'.format(taskname),
'description': 'The {} testing task'.format(taskname),
'backend_name': 'swh.scheduler.tests.tasks.{}'.format(taskname),
'default_interval': timedelta(days=1),
'min_interval': timedelta(hours=6),
'max_interval': timedelta(days=12),
})
return scheduler
diff --git a/version.txt b/version.txt
index ccae2bd..72b261b 100644
--- a/version.txt
+++ b/version.txt
@@ -1 +1 @@
-v0.0.51-0-gc2b1793
\ No newline at end of file
+v0.0.52-0-g38cafb9
\ No newline at end of file

File Metadata

Mime Type
text/x-diff
Expires
Sat, Jun 21, 7:27 PM (2 w, 3 d ago)
Storage Engine
blob
Storage Format
Raw Data
Storage Handle
3320437

Event Timeline

About · Developer information · Legal