I here provide a simple example of a group whose set of commutators is not a subgroup. The counterexample is due to P.J. Cassidy (1979).

### Description of the group \(G\)

Let \(k[x,y]\) denote the ring of all polynomials in two variables over a field \(k\), and let \(k[x]\) and \(k[y]\) denote the subrings of all polynomials in \(x\) and in \(y\) respectively. \(G\) is the set of all upper unitriangular matrices of the form

\[A=\left(\begin{array}{ccc}

1 & f(x) & h(x,y) \\

0 & 1 & g(y) \\

0 & 0 & 1 \end{array}\right)\] where \(f(x) \in k[x]\), \(g(y) \in k[y]\), and \(h(x,y) \in k[x,y]\). The matrix \(A\) will also be denoted \((f,g,h)\).

Let’s verify that \(G\) is a group. The products of two elements \((f,g,h)\) and \((f^\prime,g^\prime,h^\prime)\) is

\[\left(\begin{array}{ccc}

1 & f(x) & h(x,y) \\

0 & 1 & g(y) \\

0 & 0 & 1 \end{array}\right)

\left(\begin{array}{ccc}

1 & f^\prime(x) & h^\prime(x,y) \\

0 & 1 & g^\prime(y) \\

0 & 0 & 1 \end{array}\right)\]

\[=\left(\begin{array}{ccc}

1 & f(x)+f^\prime(x) & h(x,y)+h^\prime(x,y)+f(x)g^\prime(y) \\

0 & 1 & g(y)+g^\prime(y) \\

0 & 0 & 1 \end{array}\right)\] which is an element of \(G\). We also have:

\[\left(\begin{array}{ccc}

1 & f(x) & h(x,y) \\

0 & 1 & g(y) \\

0 & 0 & 1 \end{array}\right)^{-1} =

\left(\begin{array}{ccc}

1 & -f(x) & f(x)g(y) – h(x,y) \\

0 & 1 & -g(y) \\

0 & 0 & 1 \end{array}\right)\] proving that the inverse of an element of \(G\) is also an element of \(G\). Continue reading The set of all commutators in a group need not be a subgroup