The gate is in the middle of a semicircle of flat grey rock at the base of the cliffs, very close to the door at the base of the tower. To her left, a scattering of smallish houses come up to the perimeter of the semicircle and stop; to her right, it gives out on bare dirt with a couple of buildings barely visible in the distance; the largest building visible that isn't the tower is directly in front of her, along the middle third of the semicircle's perimeter, and it says LIBRARY in large black letters on the side facing her.

That seems like it might be a good place to start.