Details
- Reviewers
vlorentz - Group Reviewers
Reviewers - Maniphest Tasks
- T3533: Winery backend: basic implementation
It can be tested from sources (assuming swh-objstorage is checked out with https://forge.softwareheritage.org/D6796) with an override like this:
version: "2.1"
services:
swh-winery:
volumes:
- "../swh-objstorage:/src/swh-objstorage"and
- cd docker ; docker-compose up swh-winery-db swh-winery
Diff Detail
- Repository
- rDENV Development environment
- Branch
- wip-winery (branched from master)
- Lint
No Linters Available - Unit
No Unit Test Coverage - Build Status
Buildable 25625 Build 40069: arc lint + arc unit
Event Timeline
It depends on https://forge.softwareheritage.org/D6796 and will fail until it is merged. It can be tested from sources with an override like this:
version: "2.1"
services:
swh-winery:
volumes:
- "../swh-objstorage:/src/swh-objstorage"Instead of defining a new service, could you provide an alternative docker-compose config file? This way, it can be used as to switch all services to use it, just by adding a CLI parameter. eg. we do this to replace the postgres storage backend with cassandra: https://docs.softwareheritage.org/devel/getting-started/using-docker.html#cassandra
The winery is not intended as a replacement but as an additional service that will run along the existing swh-objstorage. The swh-objstorage may be configured to use swh-winery (or not) using the existing "remote" cls. Does your suggestion still stands knowing this? I'm not against it on principle (that's my first patch here, I have no experience or opinion really). I'm just making sure you got the context right before doing what you suggest.
I don't understand why. Currently, we have: client --> objstorage pathslicing backend (port 5003) --> disk.
What you want to do is: client --> objstorage proxy (port 5003) --> objstorage winery backend (port 5012) --> winery --> ceph, right?
(objstorage winery backend is what is launched by docker/services/swh-winery/entrypoint.sh in this diff)
objstorage winery backend must implement exactly the same interface as objstorage pathslicing backend, otherwise the proxy won't work. So why have the proxy in the first place instead of client --> objstorage winery backend --> winery --> ceph?
Anyway, assuming we use an objstorage proxy, then we will need a dockerfile that changes the objstorage config from {cls: "pathslicing", path: ...} (the pathslicing backend) to {cls: "remote", url: "http://swh-winery:5012"} (a proxy to the winery backend)
Ah wait I got it, docker/services/swh-winery/entrypoint.sh launches winery itself, not an actual objstorage backend; you're only reusing the scafholding.
Alright, let's do this and we'll remove the proxy later if it turns out to be unnecessary. Please allocate port 5012 in https://forge.softwareheritage.org/source/swh-docs/browse/master/docs/services.txt though.
| docker/services/swh-winery/entrypoint.sh | ||
|---|---|---|
| 14–15 | indentation issue | |
Exactly :-)
Alright, let's do this and we'll remove the proxy later if it turns out to be unnecessary. Please allocate port 5012 in https://forge.softwareheritage.org/source/swh-docs/browse/master/docs/services.txt though.
Oh, thanks for the hint. Done in D6837