Hmm.. lt sounds quite silly, however, I hardcode World Matrix as the identity matrix and it looks much better.. I guess the issue may be closed