I would say because the change in temperature. If you are going from a warm/hot environment, your bath, into a cold enviornment, your garden, then the change in temperature will be more drastic and you will feel the cold more.

So if you were having a cold bath and went into a cold garden then you would already be cold so the change would not be too big so you wouldn't shiver as much. You would still be shivering of course but the change would be different.