There are many different ways to accomplish this. From your description it seems that the built-in FlyNavigate camera control would do what you want. The code for it is in the vizcam module. Here is a sample script that shows how to use it. Use the arrow keys and mouse to move around.
Code:
import viz
viz.go()
import vizcam
vizcam.FlyNavigate()
viz.add('gallery.ive')
Let me know if this isn't what you are looking for.