The current implementation of the seaweedfs backend is pretty simple and naive.
It uses the [[ https://github.com/chrislusf/seaweedfs/wiki/Filer-Server-API | Filer API ]] to interact with the seaweedfs cluster.
By default, the Filer server uses an leveldb index to store the path<->object ID relations, which has good properties, but lack the distributed one.
However there is a [[ https://github.com/chrislusf/seaweedfs/wiki/Filer-Store-Replication | replication mechanism ]] available.
Nonetheless, a better production setup would probably use a redis backend, since it's one of the 2 that supports the [[https://github.com/chrislusf/seaweedfs/wiki/Super-Large-Directories | Super Large Directories]] feature.
See the list of [[https://github.com/chrislusf/seaweedfs/wiki/Filer-Stores | supported backends]] for more details.
The redis backend used to have a limitation on the size of a directory, which would be a problem for our usecase, so a possible solution was to use the pathslicer on the seaswwedfs backend.
However this limitation seems to have been released with the redis3 backend.
This task tries to keep track of needed improvements of the seaweedfs objtorage backend:
[] ~~use the pathslicer to mitigate possible directory size limitations of some Filer backends~~ (D6442 and D6492); this is probably not needed any more (at least for now)
[x] better tests (current tests bypass too much layers of the objstorage<->Filer communication) (part of D6517)
[] ~~add support for multiple Filer servers~~
Using the pathslicer is probably not needed, so let's not get it done for the moment.
Best approach for multiple Filer servers is simply use a load balancer in front of the Filers, so let's not add code for that.