Solving complex math problem (Complete the Square Method)