So any polynomial can be prime or not.

Given any polynomial, the first process in factoring is to find a common term, also called, greatest common factor (GCD).

By definition,

GCD can be taken as the largest polynomial that divides each of the terms in the taken polynomial

In learning how to factor a polynomial, the following steps can be followed.

- Find the GCD of the polynomial
- Divide each term of the polynomial using the GCD found.
- Write the GCD outside a parenthesis and all divided terms inside it.

To factor this polynomial, lets proceed in the following manner.

First find the GCD.

For that factor each term of the polynomial first

$5x^{3}y= 1*2*x*x*x*y$

$12x^{2}y^{2} = 1*2*2*3*x*x*y*y$

It can be seen that the common factors= $1*2*x*x*y= 2x^ {2} y$

So the largest polynomial that can be factored out = GCD = $2x^ {2} y$

In the next step, divide each term of the polynomial using the GCD found.

So,

$\frac{5x^{3}y }{2x^{2}y}= \frac{5x }{2} $

$\frac{12x^{2}y^{2} }{2x^{2}y}= 6y$

Now,

in the last step, write the GCD outside a parenthesis and all divided terms inside it.

So,

we write $2x^ {2}y(\frac{5x }{2}+6y)$

Hence,

$5x^{3}y – 12x^{2}y^{2}= 2x^{2}y(\frac{5x }{2}+6y)$

Lets consider another example to learn how to factor a polynomial

$18x^{6}+ 9x^{4}-3x^{2}$

Factoring each term

$18x^{6}= 1*2*3*3*x*x*x*x*x*x$

$9x^{4}= 1*3*3*x*x*x*x$

$3x^{2}=1*3*x*x$

So the GCD = $1*3*x*x= 3x^{2}$

Divide each term of the polynomial using the GCD found.

$\frac{18x^{6}}{3x^{2}}=6x^{4} $

$\frac{9x^{4}}{3x^{2}}= 3x^{2} $

$\frac {3x^ {2}} {3x^{2}}= 1 $

Write the GCD outside a parenthesis and all divided terms inside it.

$3x^{2} (6x^ {4} +3x^ {2} - 1)$

Hence,

$18x^{6}+ 9x^{4}-3x^{2}= 3x^{2}(6x^{4}+3x^{2}- 1)$