1) I'm not sure why there would be issues with the quality. You could try a 3rd party recorder to see if that works better.
2) Do you get the same error running the following code?
Code:
import viz
viz.go()
viz.setOption('viz.AVIRecorder.fps','10')
viz.add('piazza.osgb')
vizact.onkeydown('b', viz.window.startRecording, 'test.avi' )
vizact.onkeydown('e', viz.window.stopRecording )