Why is Set so different than Setop? It’s because the morphisms are functions: relations that can be many-to-one, but not one-to-many.
Why do many-to-one but not one-to-many relations get singled out for single treatment and dubbed ‘functions’? Because functions are supposed to be ‘deterministic’: the cause must determine the effect. We don’t care if the effect fails to determine the cause.
Why does our customary concept of determinism have this asymmetry built in? Well, we see it a lot in ordinary life. It’s often (though not always) true that the initial state of an experiment determines the final outcome. But it’s much less common for the final outcome to determine the initial state… at least, not in an easily visible way.
For example, take a heat distribution and run it forwards to equilibrium. We always get the same equilibrium, regardless of the initial conditions.
This asymmetry is built into equations like the heat equation, but it seems absent from the fundamental laws of physics — so far, anyway. This leads to a big puzzle: “why is there this ‘arrow of time’?”
Nobody knows the answer, except “that’s how this universe is: there’s a low-entropy big bang in the past, and a high-entropy expansion in the future, as far as we can see.”
So, the beautiful time-symmetric formalism of quantum theory, so nicely modelled by the dagger-categories we’ve learned to love, is far removed from common experience, where functions rule the roost.
See also the recent post “The Arrow of Time in Cat“.