It is currently fairly easy to DoS the graph service, e.g., by performing a visit on the transposed graph starting from the empty file.
We should add safeguards against this. We already have accounting for the number of edges traversed during visit, so the easiest first step would be adding a maximum number of edges traversed during a visit. It should be a configuration option for the service, not overridable a runtime.
| Status | Assigned | Task | ||
|---|---|---|---|---|
| Migrated | gitlab-migration | T2201 Indexing / mining | ||
| Migrated | gitlab-migration | T2204 Full-text search on source code (prototype) | ||
| Migrated | gitlab-migration | T2217 Plumbings | ||
| Migrated | gitlab-migration | T3096 Efficient and reliable download via the Vault | ||
| Migrated | gitlab-migration | T3550 Compute and show ETA for vault tasks | ||
| Migrated | gitlab-migration | T887 Vault: "snapshot" cooker | ||
| Migrated | gitlab-migration | T2220 swh-graph in production | ||
| Migrated | gitlab-migration | T3161 graph service: add anti-DoS limit on the number of edges traversed |
To be consistent with swh-storage, swh-graph should take a limit as query parameter, but not have a hardcoded upper bound for that limit. Instead, swh-web provides that upper bound.
I have two questions to make sure I understand well :
1/ If I wanted to reproduce a Ddos, I would have to do it on the whole dataset ? the py3k dataset is not big enough ?
2/ What we want is an argument on the visit requests of the REST API to limit the number of visited edges ? Or an argument at another level?
Thanks.
To complement what @vlorentz mentioned, we should actually stop the visit after the maximum number of edges has been reached, because it is keep doing the visit (no matter how many results are returned after it) that can DoS the swh-graph backend.
@vlorentz about the query parameters: should there be a default value for it or not? (We want this to be consistent with swh-storage, as you mentioned, but it's not clear to me if there's a default value in that case or not.)
should there be a default value for it or not? (We want this to be consistent with swh-storage
swh-storage is not consistent with itself, so either way is fine
ok, so @Hakimb: go for no default value. If the query param is not passed, the visit will not stop before the end. If it's given, it will stop once the limit is reached. Call the query param ?max_edges. You will find that the java code already keeps track of the number of edges traversed, so you should just need to compare with that.