When it's warm, the car should idle at around 600rpm. I wouldn't call 500rpm suspicious.
It sounds to me like maybe your idle air control valve might be sticking. The IAC valve in my wife's Legacy was sticky, and the symptoms were similar, but her car also loped a lot.
I'd recommend removing the IAC valve and cleaning it with some carb cleaner and whatever else you can fit in there to scrub off any deposits. Of course, don't leave the scrubbing device behind.
