In order to gain the maximum amount of experience points possible, you must win all monster encounters available until obtaining a certain amount of gil. You can open the menu to keep track of the amount of gil you obtained from the dungeon. Because the amount of gil you can collect is predetermined, it will indicate your progress in defeating all available monster encounters.
If you want to clear the map completely:
Defeat all encounters until you have 18,576 Gil.
Defeat Shadow Bahamut, you should have 19,576 Gil before leaving the exploration.
You should conclude the dungeon exploration with a total of 19,836 Gil and 192,796 EXP.