Hello, and welcome to FMH!
(a) [MATH]\left[\begin{array}{c}1 & 2 & -1 & -3 \\ 3 & 5 & k & -4 \\ 9 & k+13 & 6 & 9 \end{array}\right][/MATH]
(b) \(-3R_1+R_2\)
[MATH]\left[\begin{array}{c}1 & 2 & -1 & -3 \\ 0 & -1 & k+3 & 5 \\ 9 & k+13 & 6 & 9 \end{array}\right][/MATH]
\(-R_2\)
[MATH]\left[\begin{array}{c}1 & 2 & -1 & -3 \\ 0 & 1 & -k-3 & -5 \\ 9 & k+13 & 6 & 9 \end{array}\right][/MATH]
\(-9R_1+R_3\)
[MATH]\left[\begin{array}{c}1 & 2 & -1 & -3 \\ 0 & 1 & -k-3 & -5 \\ 0 & k-5 & 15 & 36 \end{array}\right][/MATH]
\(-(k-5)R_2+R_3\)
[MATH]\left[\begin{array}{c}1 & 2 & -1 & -3 \\ 0 & 1 & -k-3 & -5 \\ 0 & 0 & k^2-2k & 5k+11 \end{array}\right][/MATH]