Change Details

In order to validate the feasibility and the possible caveats to implements an elastic workers infrastructure, we will implement a poc managing the workers for the gitlab repositories - [x] First we need to refresh and land the kubernetes branch on the swh-environment to have a working example - [x] Refresh the rancher VM on uffizi to test the solution in a pseudo real environment (created from scratch, cf. terraform/staging) - [x] Create workers and register them in the rancher cluster - [ ] POC the image building / deployment process - [ ] POC the autoscaling of the workers according to the available ressources on the cluster - [ ] Take care of the monitoring and the log ingestion [1] Draft note on https://hedgedoc.softwareheritage.org/4ZHT03kRT7mHYOEm1MSueQ

In order to validate the feasibility and the possible caveats to implements an elastic workers infrastructure, we will implement a poc managing the workers for the gitlab repositories - [x] First we need to refresh and land the kubernetes branch on the swh-environment to have a working example - [x] Refresh the rancher VM on uffizi to test the solution in a pseudo real environment (created from scratch, cf. terraform/staging) - [x] Create workers and register them in the rancher cluster - [x] POC image building / deployment process (manual push on docker hub) - [x] POC worker autoscaling according to message in queues - [ ] POC worker autoscaling according to available ressources on the cluster - [ ] cluster / elastic workers monitoring (nb of running workers, statsd, ...) - [ ] Plug standard log ingestion with elastic workers [1] Draft note on https://hedgedoc.softwareheritage.org/4ZHT03kRT7mHYOEm1MSueQ

In order to validate the feasibility and the possible caveats to implements an elastic workers infrastructure, we will implement a poc managing the workers for the gitlab repositories - [x] First we need to refresh and land the kubernetes branch on the swh-environment to have a working example - [x] Refresh the rancher VM on uffizi to test the solution in a pseudo real environment (created from scratch, cf. terraform/staging) - [x] Create workers and register them in the rancher cluster - [ - [x] POC the image building / deployment process (manual push on docker hub) - [x] POC worker autoscaling according to message in queues - [ ] POC theworker autoscaling of the workers according to the available ressources on the cluster - [ ] Take care of thecluster / elastic workers monitoring and theoring (nb of running workers, statsd, ...) - [ ] Plug standard log ingestion with elastic workers [1] Draft note on https://hedgedoc.softwareheritage.org/4ZHT03kRT7mHYOEm1MSueQ