Let's do some simple math, shall we!?
10 months = 40 weeks
He dated her for 40 weeks.
He broke up with her 3 weeks ago.
You started dating 2 weeks ago.
He had one week between her and you.
Yeah, okay, so he slipped. Are you seriously asking us if you should be mad? If you are mad, be it. It's childish in my opinion and not a good star for the relationship, but if you are mad, be mad. Don't seek justification from strangers.
You should let it slide (as long as you are comfortable with it). I doubt he did in on purpose. Cut the guy some slack. If he still messes up in 40 weeks (10 months), then you might have a real problem.
I should add, it takes 30 days to form a habit. So, he still has a few weeks to make it a habit of calling you instead of her.
I'm sorry I'm coming off as rude, but truth be told... if you are this insecure int he relationship, you'll never make it.