house trained, but not...

He is peeing on the floor because he isn't totally housetrained.  Don't pee on the floor in house A doesn't always translate into don't pee on the floor in house B.  You need to give him far less freedome and either crate him or leash him to you when you can't watch him with 100% attention.  Don't give him the opportunity to make a mistake.

The most likely answer is what glenda said.

If he is peeing frequently, he might have an infection.

It may be a marking behaviour.  (Is he neutered?  Sometimes that helps) 

What are you cleaning up WITH?  If its ordinary household cleaner, it doesn't matter how well you clean, his nose can detect the timiest trace of ammonia.

Does he see you cleaning up?  Do you punish him in ANY way for mistakes indoors?  You are correct that rubbing his nose in it is bad.  Actually, any sort of punishment for house training mistakes is bad.

Be patient - he'll get there.  You will have to do the leg work though!

thanks for the suggestions. we don't punish him in any way for peeing in the house, we use positive re-enforcements. now, glenda, are you saying that we should crate him all the time? because he IS a terrier and is extremely hyper! haha. i'm not sure he'd do well being crated a lot. also, he pees right when we are with him, not when we are gone, so even if we are keeping a close eye on him, he just squats in the middle of the living room while we're sitting right there! lol.

Don't trust him an inch - pretend he is a puppy who has no idea about where he should pee and zero bladder control.  Take him outside to toilet OFTEN, especially after meals, drinking, and waking.  Also after (or even during) long periods of play or chewing. 

Exercise and excitement stimulate the bladder and bowels so try to encourage sedate activities indoors.... encourage him to chew appropriate items, train him (go to your bed/stay/watch me/trick training), massage him, encourage him to rest aand sleep or just chill out with you.  Watch out for signs he needs to go - circling, sniffing, pacing etc.

thank you for all the responses. i don't know what i would do if i hadn't found this forum! :D