For some reason, some of my TS and MKV files produce a "jitter" when played. They are in VC-1 format. It is rather noticeable during opening credits where there are stationary words on screen. If you look closely, you can see them moving around a bit. Why is this and how do I fix it? Converting the files to AVI reproduces them.
I am not an expert, but one thing that I've experienced which can cause jitter is when the frames per second of the source (24fps for a DVD/bluray/film) does not match or evenly divide into the frames per second that the monitor/TV can display (ie 60Hz for most TVs). This is referred to at 3:2 Pulldown, or some such. It's most noticable in scenes where there is a large, panoramic shot that is being panned slowly to one side. In that situation you will notice "tearing" of the image.
To solve this there are several options. Some video converters can convert a file to a different fps; for instance 30fps instead of 24fps, which divides cleanly into 60. Or you can change the refresh rate of your screen to a number that is divisible by 24. This means that your screen must support either 24 Hz or 120 Hz, since 24 divides evenly into 120. There are also software solutions for this, such as drivers from ATI which support "pulldown detection" and can improve the image, etc.
Again, I am not sure that this is the problem you are experiencing. But, if you can easily change the refresh rate of the screen you are using to 120Hz or to 24Hz, it'd be an easy way to test it.