This might be correlated to the flax, ore, trees, etc when they don't respawn. After I was at Chaos Druids, I ran back to the gate and it appeared open but I was unable to close it or go through it. The only way I was able to get by relogging. When I relogged, the gate was closed but I was able to open it and go through.
Now that I think about it, the same thing happens to the gate next to the flax field in seers.