Know how you feel, Moehat. I had "WIPE HERE" written on my back until I was well into my thirties and managed to stutter out one day that no, I really couldn't take on another volunteer job. I was amazed that the person asking didn't fall to the floor in either fury or terminal disappointment - "oh, okay, I'll go ask Sue" was all I got. It was just that easy - after decades of not wanting to incur disapproval or be disliked.
I think I can blame Mommie Dearest for all of that: having forever told me, as a single child (no siblings to bat ideas off) that one put others' feelings first, I had no idea where my own were supposed to slot in. I had no sense of self-assertion at all - in fact, the word 'assertive' was used to describe jumped-up, self-important people, not people who had decided what was acceptable to them, and what was not.
I remember thinking 'how awful' when I saw courses on 'assertiveness' training - who'd want to learn to become unpleasantly self-important? But that wasn't what they were about at all - in fact, as some aspects were included in supervisory training I undertook, I realised where I'd been going wrong for so long. Going by my mother's book, you took no prisoners in telling people where to get off - or you put their feelings first. What a contradiction in terms! It wasn't an entire epiphany as it took a while to get used to the idea that I could 'assert my rights' - from saying NO, thank you, to pushy hosts, leery bosses, salespeople, and general manipulation.
So - don't lose your temper and make yourself ill. No-one else will give a damn, anyway. I'd recommend that you get yourself a book on assertiveness or, better still, go on a little course - 'life coaches' often offer these. Try out the calm but absolutely resolute approach on some friends (not in a real situation, but in role play) and see how it feels. For me, it made a huge difference in not becoming swamped by guilt and angry resentment at the same time!