How to find Ultimecia Castle
Solution 1:
Ultimecia Castle is in Disk 4
Disk 3 ends after you defeat Seifer in Lunatic Pandora (the obelisk floating above Tear's Point). To get there Fly the Ragnarok into it, if you have entered before you come out to where the Ragnarok created a hole in it, if not you you'll see it happen as a semi cinematic (one of those ones that look pre-rendered yet it's still using the in-game model for the characters rather than the better looking pre-renders).
Shortly after you come off the Ragnarok you'll fight a Boss Battle against Fujin and Raijin, after that you can explore. I don't know the route but you'll come across another boss the Mobile Type 8 which blocks your way to Seifer. After you beat Mobile Type 8 you processed forward to face Seifer.
After beating Seifer he'll kidnap Rinoa you'll enter Disk 4. Be warned that once you do you'll be unable to leave Lunatic Pandora and return to the "normal" world map as such beating Seifer in Disk 3 is your Point of No Return for the game.
Following Seifer you'll come across a bitch of a boss battle against Sorceress Adel who is Junctioning Rinoa. Both are on the enemy side and become targets if you use Enemy Targeting attacks (GFs, Magic like Meteor). the bitch part of this battle is that is Rinoa dies it's game over and Adel will cast Absorb on her to recover and damage her AND Rinoa has way less HP than Adel so spamming GFs like normal won’t work (and strangely enough when I had Phoenix Auto-summon after the party is defeated it always killed Rinoa)
After getting past Adel this will trigger the story event where Ellone will cause Time Compression so that you can face Ultimecia. Before this you'll fight a bunch of Sorceresses in a continuous battle before coming to an area where you can now access Ultimecia's Castle.
NOTE: the reason why beating Seifer in Disk 3 is your Point of No Return is because while you can enter the World Map though Gates that are just before Ultimecia's Castle, most locations have a barrier around them prevent you from entering (I believe Tears Point, Fire Cavern, the diamond structure where Odin was found and the Chocobo forests are the places you can still enter. Lunatic Pandora I also believe disappears).
You can find the Ragnarok somewhere on the World Map (I don't recall but it was a hike) and inside will be a Card Club Member who will operate as an Item and Weapons Shop (and you can play cards with him). Also while you are outside of Ultimecia's Castle your abilities won’t be sealed (they are unsealed when you exit the front door but become resealed when you re-enter it)
Solution 2:
I figured this out last night... I needed to go to Esthar and talk to Pres. Laguna again before I was able to move forward.
In the meantime, all the hunting around meant I was able to beat Ultima Weapon (including drawing Eden) and finish the Shimi Village Statue quest :)
Watching the Lunatic Pandora scenes and the start of Disc 4 again, I'm sure I've played that part before, which lead to my confusion. I have vague recollections now of my old system crashing during that section (why I stopped playing), where it wasn't properly saving my progress and I lost some work.