Not necessarily. It could go in the router, but it can also go in your properties of you wireless connection on you computer. I have both of my computers running wireless to my home router and I have set these DNS's #'s into my Wireless Network Connection Properties for both of them. Working fine.
I also set the values in my normal wired connection (Lan) on both computers for when I travel and have to plug into a network anywhere.