yes, they are both made by toyota, hence the annoyance. My xB was actually a Toyota bB and converted for left hand drive (Mine is an 04, the first year they made them)
But Lexus is made by Toyota too, but you never see anyone say a Toyota Lexus do you? While you're at it you could call the Pontiac Vibe a Toyota Vibe, (It's based on the matrix) along with the mid 90's Geo Prism's (based on the Corolla of the time)