JEE Main · 2020 · 08 Jan 2020 · Shift 1 · Matrices

The trace of $A{A}^{\top }$ is the sum of the squares of all nine entries, and since each entry is $-1$, $0$ or $1$, each square is $0$ or $1$. For the trace to equal $3$, exactly three entries must be $\pm 1$ and the other six zero. Choosing which three of the nine positions are non-zero can be done in $\left(\genfrac{}{}{0ex}{}{9}{3}\right)=84$ ways, and the signs of those three can be all $+$, all $-$, two $+$ and one $-$, or one $+$ and two $-$, giving $1+1+3+3=8$ patterns. The number of such matrices is $84\times 8=672$.