I was more thinking of lifting this up to the traversal algo, ideally not having to do a test at every edge that is followed.
Done this way, this is not gonna gain you much, only a couple of array lookup per edge.
This seems too convoluted: either you're bypassing restrictions or you need a boolean arrays for the restrictions.
So how about keeping the allowed array as a null if you're bypassing restrictions and getting rid of the boolean? (having both makes the code prone to bugs due to their mutual inconsistency).
In that case, please update the allowed docstring to mention the semantics of a null value.
with the above proposal this method can go…
… and this becomes a null pointer check