PLEASE HELP!!! What would u do if your signif other would never spend the holidays with you?

He says he only wants to spend time with me and not my family. Says my family always looks at him wierd bec he is Mexican (I'm white), but my fam is not prejudice, I think he is making this up as an excuse or just doesn't want to hang w/everyone...we have been togaether for 5 plus years...do not live togeather though...there have been the usual ups and downs of a family, fights etc but my family is accepting and welcoming to him. He will come over for a little while but then he leaves. Sometimes he says he will come and ends up not. I know he loves me but I don't get him. He also has a big family his own self and is like the caretaker of the whole family, they are all adults but he is the responsible one.Do you think this is normal or acceptable..what would you do?