Making peace with saying no

I find it ridiculously hard to say no in work situations. I’m fine in the rest of my life which I find even stranger. The issue is clearly linked to work situations. So what’s going on?