Page MenuHomeSoftware Heritage

Hedgedoc: websocket upgrade queries routed to grafana
Closed, MigratedEdits Locked

Description

Hedgedoc has some difficulties to communicate with the server with websockets.
The protocol falls back to web polling causing some weird behaviors during parallel editions (lost content, duplicate characters, ...)

websocket.js:124 WebSocket connection to 'wss://hedgedoc.softwareheritage.org/socket.io/?noteId=features&EIO=3&transport=websocket&sid=<redacted> failed: Error during WebSocket handshake: Unexpected response code: 404

The cause seems to be a wrong backend routing in varnish for the upgrade requests only:

*   << BeReq    >> 73173170  
-   Begin          bereq 73173169 pipe
-   BereqMethod    GET
-   BereqURL       /socket.io/?noteId=features&EIO=3&transport=websocket&sid=<redacted>
-   BereqProtocol  HTTP/1.1
-   BereqHeader    Host: hedgedoc.softwareheritage.org
-   BereqHeader    Connection: Upgrade
-   BereqHeader    Pragma: no-cache
-   BereqHeader    Cache-Control: no-cache
-   BereqHeader    User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/100.0.4896.75 Safari/537.36
-   BereqHeader    Origin: https://hedgedoc.softwareheritage.org
-   BereqHeader    Sec-WebSocket-Version: 13
-   BereqHeader    Accept-Encoding: gzip, deflate, br
-   BereqHeader    Accept-Language: en-US,en;q=0.9,fr;q=0.8
-   BereqHeader    Cookie: connect.sid=s%<redacted>; indent_type=space; space_units=4; keymap=sublime; loginstate=false
-   BereqHeader    Sec-WebSocket-Key: <redacted>wg==
-   BereqHeader    Sec-WebSocket-Extensions: permessage-deflate; client_max_window_bits
-   BereqHeader    X-Forwarded-For: xx.xx.xx.xx
-   BereqHeader    X-Varnish: 73173169
-   BereqUnset     Connection: Upgrade
-   BereqHeader    Connection: close
-   VCL_call       PIPE
-   BereqHeader    upgrade: websocket
-   BereqUnset     Connection: close
-   BereqHeader    connection: Upgrade
-   BereqUnset     upgrade: websocket
-   BereqHeader    upgrade: websocket
-   BereqUnset     connection: Upgrade
-   BereqHeader    connection: Upgrade
-   VCL_return     pipe
-   BackendOpen    95 grafana 192.168.xx.xx 3000 192.168.xx.xx 59022 connect <---------------- wrong backend
-   Timestamp      Bereq: 1650561229.210203 0.000000 0.000000
-   BackendClose   95 grafana close  
-   BereqAcct      0 0 0 0 0 0
-   End

Event Timeline

vsellier renamed this task from Headgedoc: websocket upgrade queries routed to grafana to Hedgedoc: websocket upgrade queries routed to grafana.Apr 21 2022, 7:22 PM
vsellier triaged this task as High priority.
vsellier created this task.
ardumont changed the task status from Open to Work in Progress.Apr 27 2022, 4:56 PM
ardumont moved this task from Backlog to in-progress on the System administration board.
olasd claimed this task.
olasd moved this task from in-progress to done on the System administration board.

Deployed the puppet changes and ran puppet on rp1.internal.admin. The websocket requests from hedgedoc get properly handled. Websocket requests for grafana also still work.