Hola Robert

Using "unas pocas horas" would change the meaning. 

What we have in the test question is:

Tardé pocas en hacerlo. 

It didn't take long to do it. [lit: I took few hours to do it.]

This means that you didn't take long, so you only took 2 or 3 hours for example, not many.

However is you say:

Tardé unas pocas en hacerlo

what you mean is "It took me a few hours" with the implication/nuance of taking a long time. 

Saludos

Inma