Your mythical units will disappear when you switch. I advise you NOT to switch until you have a second city. (especially on siege worlds where medusae come in handy to land a CS and hold the siege until the first supports arrive)
Once you get that second city, it's up to you. Zeus' mantis are useful for siege breaks or starting a revolt without clearing the navy (someone back me up or correct me here please as I'm not good with revolt) and you can use his rage to wipe out a CS escort.
Artemis has powerful griffins that are however really slow and her purification helps remove any spell on cities (except city protection) and attacks.
Athena has pegasi that are good for defense and her city protection helps in spell wars.
Hades has powerful range-attack Erinyes (or however the hell it's spelled) that need boats to travel, and helps with silver or to plague an enemy in spell wars.
Poseidon has Hydras and Cyclopses that are really useful for a CS package on a siege world as they are both powerful and mixt units that can defend and attack.
These are my opinions on Gods, in the end you make the decision. Let me know if you disagree or if I forgot anything
