|
|
|
DVD Shrink v3.2b3 comparative tests
|
|
|
ddlooping
Senior Member
|
13. July 2004 @ 19:09 |
Link to this message
|
vurbal, if it's not too much to ask, could you repeat the same test, this time switching DVD Shrink to "Custom Ratio: 100%"?
|
|
Advertisement
|
 |
|
|
Staff Member
2 product reviews
|
13. July 2004 @ 19:23 |
Link to this message
|
|
@ddlooping: No problem. If anybody wants details on my comparison, let me know and I can explain how to reproduce it.
SSIM is also a good tool for determining the worst areas of compressed movies for comparison. I was working on a comparison of some different programs using its output until I ran out of time, but the general idea was to take the list of frame comparison figures into a spreadsheet and find the biggest problem areas from each program's output. Then I was going to make short clips from each program and compare them visually.
Edit: Sure I could run that test. I'll get back to you with the results some time tomorrow.
This message has been edited since posting. Last time this message was edited on 13. July 2004 @ 19:23
|
|
ddlooping
Senior Member
|
13. July 2004 @ 19:34 |
Link to this message
|
|
This sounds very interesting indeed, and could be very useful to dvdshrink (if he's not already using it).
In any case I'd very much appreciate if you could give me more details about SSIM - where to get it, a few pointers on how to use it to make comparisons...
P.S: thanks for running the test. ;)
This message has been edited since posting. Last time this message was edited on 13. July 2004 @ 19:38
|
Staff Member
2 product reviews
|
13. July 2004 @ 20:49 |
Link to this message
|
@ddlooping: I'll explain the best I can. A lot of it is PFM (Pure #$%@!ing Magic) to me, but I understand how to use it at least. SSIM is based on a study done by the Video Quality Experts Group that was intended to find a way to objectively quantify the quality of a video clip. It basically takes the Y, U, and V planes of the 2 video clips and compares them, weighting the Y (luminance) comparison more heavily than the U and V (chroma) comparison. You can download SSIM from the main AviSynth plugins page (http://www.avisynth.org/warpenterprises/) and this is the script I used:
original = MPEG2Source("Original.d2v")
compressed = MPEG2Source("Compressed.d2v")
SSIM(original,compressed,"fram_comparisons.csv","averageSSIM.txt",lumimask=false)
Just make DVD2AVI projects from the 2 versions of the movie and plug in the names. Then you have to play the AVS file in a media player and make sure to close the file before exiting the program. The text file with the comma separated values is saved when the file is closed. I've done it with VirtualDub and Media Player Classic, and I found that MPC was slightly faster, but the difference was really negligible. In either case it played at around 90% of realtime on my computer (P4 2.4GHz, 768MB RAM).
Now the fun part begins - the spreadsheets. Unfortunately, a couple of days ago I deleted everything from my aborted test to make room on my hard drive. I learned a lot about Excel when I played with this because I'd never done anything that actually went beyond it's limits before. If you're not careful this will go far beyond what Excel is designed for. The first problem is the number of frames. Excel has a maximum of 65,536 rows, and since each frame gets it's own row, and any normal length movie is at least 150,000 frames long (usually over 200,000) you have to split the CSV file into several smaller files before you can import into Excel. I used Vim for this but any text editor should work.
Problem number 2 was keeping the spreadsheet small enough. I learned the hard way that I needed to make each group of 50,000 frames in its own spreadsheet. Otherwise the file got too big to open. Excel would tell me it ran out of memory when it tried to open it. I literally lost about 6 hours of work because of this. Now I know better than to try opening an Excel file that's over 100MB ;)
Finally I ended up with a series of spreadsheets with the results from different compression methods, each in its own column. The workbook had 1 sheet with nothing but frame numbers, incremented from the top to the bottom. The second sheet had the SSIM values imported from the CSV files with the frame numbers in one column so I could sort by the SSIM values and then copy the frame numbers for the worst frames. The third sheet had the frame numbers I copied from each program's results. I then grouped the frame numbers into small clips, added frames bfore and after, and I had the frame ranges for my clips.
I never got beyond that because I was planning to get results from more programs from other people who aren't as cheap as me and already had the programs I wanted to test, but real life got in the way so I never got any clips generated. Now I wish I had at least saved the spreadsheets so I could explain it better (and make sure I'm not forgetting anything). I had run The Matrix R1 through DVD-RB/CCE Basic, DVD Shrink 3.1.7 (or maybe 3.1.6), Rejig, and CloneDVD. Interestingly I found the numbers from Shrink and Rejig to be very similar, although Shrink's numbers looked slightly better. In most cases, each program shared the same problem areas, but there were some interesting differences. CCE (not surprisingly) spread the worst frames further apart, most likely making them less noticeable. I was surprised to find out that Shrink (and Rejig) seemed to leave every third frame untouched, which I theorized was probably a bad thing since it means giving you a frequent reminder of what the original looks like, thereby making it easier to notice low quality frames.
If you want to play with it, let me know and I'll be happy to give you whatever help I can.
|
|
ddlooping
Senior Member
|
13. July 2004 @ 20:52 |
Link to this message
|
|
Ok, I know this may not seem fair to some, but I've decided to give Vurbal access to the betas if he wants it.
He seems to be very knowledgeable, and his comparison method could be very useful for DVD Shrink development.
So Vurbal, would you be interested, and would you have the time to do some testing?
Edit: I was writing this post while you were posting your reply, Vurbal.
Thanks for the information and detailed explanation.
The offer still stands. ;)
This message has been edited since posting. Last time this message was edited on 13. July 2004 @ 20:56
|
|
ddlooping
Senior Member
|
13. July 2004 @ 21:04 |
Link to this message
|
Quote:
I was surprised to find out that Shrink (and Rejig) seemed to leave every third frame untouched, which I theorized was probably a bad thing since it means giving you a frequent reminder of what the original looks like, thereby making it easier to notice low quality frames.
I don't know what level of compression was used in your test, but v3.1.7 used to try and compress the I-frames as a last resort.
This could in turn create the effect you described (affectionaly known as pumping").
v3.2 new quality settings/compression algorithms work in a different way and should hopefully minimise this effect by spreading the compression in a different manner.
|
Staff Member
2 product reviews
|
13. July 2004 @ 21:07 |
Link to this message
|
|
Sure I'd be interested. Send me a PM and let me know how to get it.
|
Staff Member
2 product reviews
|
13. July 2004 @ 21:13 |
Link to this message
|
Quote:
I don't know what level of compression was used in your test, but v3.1.7 used to try and compress the I-frames as a last resort.
This could in turn create the effect you described (affectionaly known as pumping").
v3.2 new quality settings/compression algorithms work in a different way and should hopefully minimise this effect by spreading the compression in a different manner.
That makes sense. I wasn't using very much compression. I'm looking forward to the new version since I'd just as soon stick with Shrink for compressing menus, even if jdobbs implements menu compression in DVD-RB. For menus I value speed more than quality.
|
|
ddlooping
Senior Member
|
13. July 2004 @ 21:16 |
Link to this message
|
|
Done. ;)
Off to bed now. See you all if a few hours.
|
|
brobear
Suspended permanently
|
14. July 2004 @ 05:46 |
Link to this message
|
I have a little bit of information to add to the discussion about contrast output by DVD Shrink. My methods may not be as scientific as those of vurbal, but they were done in a manner which tried to eliminate any bias. Before I get jumped, the bias of testing, I'm not talking about personal bias.
P4, 2.8GHz, 1GB RAM, 2WD HDs-330GB, ATI 9800 Radeon card, Memorex Dual Format 4X DVD Recorder, Win XP, DVD Shrink 3.2b4 Beta, DVD Decrypter 3.2.3.0, Nero ShowTime player and WinDVD Platinum 6 player software, 20" color monitor, Samsung DVD Player with S-Video to 60" rear projection TV (with more bells and whistles). Just a little of the setup so one can get a picture of the equipment in use.
I noticed a slight variation in contrast that varied from the original and from a file ripped with DVD Decrypter. It was a slight variation but noticable to the naked eye. It is most noticable with light and colors. The shift might not be noticable against dark backgrounds or with subdued cinematography techniques, i.e. The Godfather. In movies with sunny outdoor scenes, The Pirates of the Caribbean, The Gladiator, and Lord of the Rings: Return of the King , one can see the difference. It is not a problem. I would say more of an idiosyncracy. The picture is just a little less sharp and the colors just a bit subdued. I have heard the term 'softened' used to describe the phenomenon.
I ripped 2 files with DVD Shrink, 1 at 100%, the other set at No Compression. Those 2 appeared identical. I ripped 1 file using DVD Decrypter. The original DVD was used to show the clip segment as well. I used a family member for a subject due to accessability on short notice (come here a few minutes). I used a blind setup so the viewer could not see which clips were being shown. I used the opening of the slave/gladiator sale scene in The Gladiator; sunlight, vivid color and sharp cinematography. I played 2 minutes from each file and the original disc. The subject chose the original and DVD Decrypter file as appearing the same, with the 2 DVD Shrink files appearing the same. The difference was described as the original and Decrypter files being sharper and the colors more vivid. DVD Shrink files were described as being just a little less sharp and the colors a little dull. The files were shuffled so the 2 Shrink files were mixed between the original and the Decrypter file. The test was repeated twice and the order of the clips changed. In both cases the DVD Shrink files were noted to be 'soft'. The clips were played through both the Nero Showtime and WinDVD 6 Platinum player software.
All clips were burned using CMC Magnetics +R media and the Nero 6 Burning ROM. BTW, no problems with the media. It may go south down the road, but at present is good. The soft effect was more noticable on the big screen. The order of the clips had been changed again and the two Shrink files were again described as 'soft'. The Shrink clips produced a good video, just a slight variation in contrast.
|
|
ddlooping
Senior Member
|
14. July 2004 @ 06:02 |
Link to this message
|
Thanks for this detailed report, brobear.
I do not doubt the unbiased nature of your test, neither the efficiency or thoroughness of your comparison method, but I still have to experience the "softening" effect (or any other effect) you describe when using "No Compression".
This leaves me extremely puzzled.
I hope others will be able to shade some light on this matter.
dvdshrink, for one, has always maintain "No Compression" would leave the video stream untouched.
P.S: is it something you've also experienced with other transcoders - DVD2One, IC8, Nero Recode DVDCopy...?
|
|
brobear
Suspended permanently
|
14. July 2004 @ 06:02 |
Link to this message
|
When I used the Shrink beta to transcode the movie The Gladiator, I used deep analysis and checked the boxes in Quality Settings. I removed one audio file from the Full Disc backup. The files were 6915MB and the compression level was at 57.1% (compressed 42.9%). The software worked well even under this high compression level.
I got a good picture with none of the compression woes attributed to the earlier versions of DVD Shrink. All in all, Mr Shrink has done an excellent job of improving his software to work well with higher compression loads. The deep analysis and quality settings produce a video as good as the current Shrink at low or no compression. Recording time was just over 2 hours. I give this program a big thumbs up. Well done.
|
|
ddlooping
Senior Member
|
14. July 2004 @ 06:03 |
Link to this message
|
|
Thanks for the positive feedback, it's very much appreciated. ;)
|
|
ddlooping
Senior Member
|
14. July 2004 @ 06:09 |
Link to this message
|
|
dvdshrink is lurking on the Digital Video Forums as we speak, and has been informed of your experience.
Hopefully he will respond to it, and I'll then relay his reply (as I do not think he's an Afterdawn member).
|
|
brobear
Suspended permanently
|
14. July 2004 @ 06:13 |
Link to this message
|
ddlooping
I don't have IC8 and don't use DVD2One enough to really comment. I used DVD X Copy Platinum for a while, Clone DVD2, DVDCopy2, and Recode2. I never noticed this anomaly with these. You act as though this is earth shattering. I noted this is minor and let me add, almost trivial; but it is there. If I was the only person to ever see it, then I would wonder.
An interesting side note. The program DVDCopy2 actually enhances the contrast of a copied movie. Some of the backups are actually sharper and brighter than the original. Go figure.
|
|
brobear
Suspended permanently
|
14. July 2004 @ 06:19 |
Link to this message
|
|
ddlooping
You can relay to Shrink that I am pleasantly surprised by the success of his current achievement. I was a bit skeptical that he could pull it off without a complete overhaul. He managed to do a good job with what he had.
|
Staff Member
2 product reviews
|
14. July 2004 @ 06:42 |
Link to this message
|
|
@ddlooping: This is interesting. Custom Compression - 100% does not give me the same result as None. There are some differences from the original, although given the SSIM numbers a casual viewer who hadn't trained himself to spot the difference probably wouldn't. Honestly, with a "normal" TV (like my 2 27" TVs) it still might not be visible.
|
|
ddlooping
Senior Member
|
14. July 2004 @ 06:49 |
Link to this message
|
|
This is not earth-shattering, brobear, simply contrary to what many have experienced, and what dvdshrink has stated several times.
If that was indeed proved to be true, my tests would be also skewed.
They would still have some significance though, as the source for all the backups was the same for all the softwares, if not identical to the original.
Also, v3.2b3 would be even more at a disadvantage.
For each scene, one clip was taken from each backup using DVD Shrink "Start/End" feature (first use of "No Compression", excluding the making of the source).
Each of these "clips" where then used once in each compilations (second use of "No Compression").
But for a given scene, the same DVD Shrink clip was used several times.
Example, the "flames" scene:
The number between brackets indicate the number of times each backup was run through "No Compression".
v3.2b3 (2) - DVD2One v1.4 Constant Bitrate (2)
v3.2b3 (3) - IC8 (2)
v3.2b3 (4) - CCE (2)
v3.2b3 (5) - DVDCopy2 (2)
So if "No Compression" had an adverse effect on video streams, each DVD Shrink clip would have degraded in quality each time it was inserted in a new compilation.
The v3.2b2 / DVDCopy2 comparison would have been so skewed in DVDCopy2 favor as to be totally useless.
Not earth-shattering, but a fair amount of wasted time for me and those who downloaded my tests.
_
_
This message has been edited since posting. Last time this message was edited on 14. July 2004 @ 07:15
|
Staff Member
2 product reviews
|
14. July 2004 @ 06:50 |
Link to this message
|
|
No compression is still perfect, it's the Custom Compression (100%) that appears not to be perfect.
|
|
ddlooping
Senior Member
|
14. July 2004 @ 07:06 |
Link to this message
|
Thanks for doing the test, vurbal.
Here is what dvdshrink has to say about this issue:
Quote:
...
About the brobear softening issue, he is saying the color or contrast or something seems somehow modified after passing through DVD Shrink at 100% or even 'no compression'?
Well, it is a mystery, I don't see how it is possible.
At 'no compression' the video data will be a byte-for-byte duplicate of the original. It is literally just copied. You can check with a hex editor.
At '100%' there is a chance that some frames may be slightly modified, for lengthy reasons I won't go into now, but the huge bulk of the video will be copied as above, without being touched.
DVD Navigation files (IFO and BUP) are never just copied, they are always generated from scratch, but I am not aware that there are any settings in these files which control appearance of the video. Maybe this is something to investigate...
There are some known cases when "Automatic: 100%" will still compress a title.
This would be when the quick analysis reports an overal size not requiring compression ("Automatic: 100%"), when compression is actually required.
The actual compression ratio during backup might then end-up being 98-99.x% (guessing here).
This would explain vurbal's test results, but this would be far from generating the "softening/color subdueing" effect described by brobear.
Let's hope dvdshrink's investigation will bring some answers.
This message has been edited since posting. Last time this message was edited on 14. July 2004 @ 07:08
|
|
brobear
Suspended permanently
|
14. July 2004 @ 07:28 |
Link to this message
|
|
Did any one see where I say trivial difference, good picture, you had to be looking for it and under certain conditions which I described. The 100% was custom so should have been locked. No compression was used as well. A person who could have cared less picked 2 out of 4 clips for the Shrink twice in a blind. That was 100%. With a monitor you are on top of it and a 60" exaggerates any difference. I don't have a clue as to what, but there is a minute amount of filtering happening somewhere. As I said the anomaly doesn't occur with 4 other recording programs I use. Everyone seems to act as though something extraordinary has happened. This very slight change in contrast, which most people might not notice, in no way detracts from the good aspects of the improved DVD Shrink. Everyone is making more fuss over this miniscule change than noticing my positive appraisal of the program.
This message has been edited since posting. Last time this message was edited on 14. July 2004 @ 07:39
|
Staff Member
2 product reviews
|
14. July 2004 @ 07:35 |
Link to this message
|
|
I should have mentioned that the almost perfect Custom Compression (100%) version was 9,594,880 Bytes smaller than the perfect No Compression.
|
|
ddlooping
Senior Member
|
14. July 2004 @ 07:43 |
Link to this message
|
|
brobear, your positive feedback hasn't gone unnoticed, far from it, and is very much appreciated.
But I'm pretty sure if you were in my shoes, as moderator at the offical DVD Shrink forums, you would also try and find an explanation for this problem.
That DVD Shrink "changes" color content, sharpness, etc etc, has been said many times.
The reason is always compression artefacts at moderate to high levels of compression.
For it to happen at "No Compression" or to be noticeable in a blind test at "Automatic: 100%" is another matter entirely, and the issue, whetever it is, needs to be addressed.
What do I reply to someone who reports the same problem?
"Well, it happens, but no-one knows why."?
I agree it might not be a big deal for most dvd-backupers, but it is important to me, and no doubt dvdshrink.
After all, it's his coding.
_
This message has been edited since posting. Last time this message was edited on 14. July 2004 @ 07:48
|
|
ddlooping
Senior Member
|
14. July 2004 @ 07:55 |
Link to this message
|
|
Thanks for the added information, vurbal.
9,594,880 Bytes is roughly 10MB.
This would give us a 0.25% discrepancy on an average 4GB video stream.
I do not think that would be enough to create a noticeable difference.
Let's hope dvdshrink finds an explanation to this issue.
|
|
Advertisement
|
|
|
AfterDawn Addict
1 product review
|
14. July 2004 @ 07:58 |
Link to this message
|
|
ddlooping
I backed up the 175 minute LOR II which Shrink reported at 52.8% compression, I think that qualifies as high comression. I let the movie run in its entirety and watched it as closely as I could and outside of the slight lost of sharpness I saw no identifiable artifacts.
I'm going to rip some DVD's today using no compression and 100% compression. I will compare them to rips made by DVD Decrypter and Smart ripper. Ill be looking at comparing the total file sizes.
" Please Read!!! Post your questions only in This Thread or they will go unanswered:
Help with development of BD RB: Donations at: http://www.jdobbs.com/. This message has been edited since posting. Last time this message was edited on 14. July 2004 @ 07:58
|
|
