I've had a similar problem recently. Turns out it was the ball joint on the passenger side. To determine if this is it:
Put front end in air.
Remove wheels.
Try to wiggle suspension (up,down,left,right)
This shouldn't move. If it does, find where it is moving. The ball joint is part 20206 in the following link:
http://www.subaruparts.com/diag/?mod...category=200-1
Stevesby also suggested that it might be loose brake parts. I would check that as well.
Hope this helps.