We get health certificates for the dogs saying they're up to date on all their shots and in good health.
Also make sure that if you have any dog treats, food, etc. when coming back to the US you have them in the bags that says they were manufactured in the US (most contain meat by-products). We've been stopped twice now so they could check our dog food! :mumble:
I used to take my dog to Canada all the time and never once was asked for papers or anything. That being said, I have a friend who said she got stuck at the border for hours once because they didn't have a rabies certificate. Apparently the collar tag wasn't enough and they had to have the vet fax a copy of the certificate to the border. After I heard that I made sure to keep a copy in my glove box.