Tower 5 is southwest of Tower4, on a penninsula to the far east.
As soon as you enter the tower, Guardian appears and throws a glowing orb at you. Luckly you have equiped the EllaCape (right?) and the orb bounces right off you. Just walk up the stairs.
The second floor had a hole in the ground, a staircase, and 2 firewheels. That's it. Simple enough. Destroy the Firewheels and go up the stairs. If you want to fall down the hole, you'll fall back down the the first floor.
The third floor looks like the second floor, except this time instead of firewheels it's wizards. Take out the wizards and go up the stairs.
Fourth floor... hmmm... looks just like the other three floors. This time it's those wall Gardners you just have to avoid. Up the stairs we go again.
Fifth floor. There is an opening right by the stairs and a pathway to the right side of the screen. You can take the opening if you want to, but you'll fall to the outside of the tower and have to climb back up again. Take the path to the right, and you'll confront Guardian. He says you have one more challenge to go, and disappears. Go through the doorway that he was in front of and walk down the corridor.
Don't return to the Elders yet. There is a small house west of the Tower 5. Enter it, and you'll come across a lot of firewheels and those stone guards. Defeat them and then go into the main room. There are a group of 3 firewheels sitting on a platform in the middle of the room and you can't jump to the platform to defeat them. Head tword the upper left of the room and the firewheels will group together on the upper left corner of their platform. Again just equip the Jewel Box and use an Ice Ring to destroy the three of them at the same time. Once you do that, you can enter the control room at the top and awake Polynese.