Dot product on Vec2s returning incorrect results

asked 2020-04-08 18:07:31 -0600

h4k1m gravatar image

The dot product seems to be returning incorrect results for the case below (using Vec2s), and I'm not sure if it's a bug or I'm using it incorrectly:

Vec2s a(-495, 584), b(101, 8);

auto c = a.dot(b);
cout << c << "\n";
cout << a[0]*b[0] + a[1]*b[1];

In the output below, the first result is clearly incorrect:

20213
-45323
edit retag flag offensive close merge delete

Comments

The problem is probably that a short can only handle values from -32768 to 32768 so the multiplication overflows, but in the other case the compiler can magically save it... Try using ints instead of shorts in your vectors.

mvuori gravatar imagemvuori ( 2020-04-09 09:49:56 -0600 )edit