To unclip, you should just need to twist the foot. Again, I can unclip
both feet in half a second.
If you need to push the shoe down to unclip, it suggests that something on
the show is catching the pedal, which also suggests the cleat is
incorrectly positioned.

What happens if you take the shoe off, and put it on your hand, and clip
it in, and unclip it. Can you see something obviously catching.

Martin.

Another possibility is that the cleats are loose. That happened to me once.
I kept twisting my foot and the cleat would turn at the same time.

Also, I found by trial and error that my left foot tends to point slightly
outward and the right foot wants to point straight ahead. I kept trying to
make my foot point straight ahead and instead wound up with knee pain. The
doc pointed out that ever since I tore my ACL I cannot expect my knee to
behave as it used to. So, now, I make sure my feet point wherever they want
to---and it helps that the cleats/pedals have enough "float" to let the foot
move around some.