For the ultimate in smooth feel with an oil or oil/varnish-blend finish, sand the finish between coats while it is still wet—that is, before wiping off the excess.
You can use any grit sandpaper, but the finer grit you use, the smoother the result. I like to use 600-grit, but 400-grit also works well. Sand with the grain, of course. As long as you have sanded the wood to 180-grit or finer, you don’t have to sand much to remove the coarser scratches.
The reason this trick works so well is that the oil acts as a lubricant for the sandpaper. You could also sand the finish dry, just before applying the next coat, and still get excellent results. But you wouldn’t benefit from the oil lubricant.
When you’re finished sanding, wipe off all the excess finish. If you have ground off any wood dust, you’ll remove this with the wiping.
You could also get a similar result by sanding the wood through the grits up to 600-grit before beginning to apply the finish. But this is a lot of work and is unnecessary if you sand between coats.