You could use vizconnect to add your trackers and link a grabber tool to the hand. The grabber tool makes it possible to grab and move objects in the scene. It would be best to go through all the vizconnect tutorials as each one builds on the next. The last one,
avatars and tools, covers this topic.