My coolant reservoir is disgusting!

My car is about to get to 90K miles, so I was looking in the maintenance thing to see what all needs to be done. I was kinda curious as to how to drain the coolant reservoir, so I go out to look at it and the thing is just about empty, and has a bunch of disgusting black stuff on the bottom. I don't drive but about four miles to and from work everyday, and I'm going to change out the coolant this next week. Temp gauge is always a hair below the middle mark when it warms up. Should I maybe wait until I change it to actually pour more in there? I don't want more of that nasty stuff getting into the system.