JEE Main · 2020 · 06 Sep 2020 · Shift 1 · Matrices

For infinitely many solutions the coefficient determinant must vanish. Here $\Delta =\left|\begin{array}{ccc}1& 1& 1\\ 1& 2& 3\\ 1& 3& \lambda \end{array}\right|=\lambda -5$, so $\lambda =5$. Consistency then needs the augmented determinants to vanish too; evaluating the one in which the constants replace the third column, $\left|\begin{array}{ccc}1& 1& 2\\ 1& 2& 5\\ 1& 3& \mu \end{array}\right|=\mu -8$, which gives $\mu =8$. Hence $\lambda =5$ and $\mu =8$.