BBC HD picture quality comparison

Sorry for the multiple posts, but I've run across a server error with larger posts.

Recently, there's been a lot of discussion about the supposedly dramatic decline in the image quality of BBC HD. Not so much around here, but elsewhere, the story has made the mainstream news, prompted the BBC's Andy Quested to explain the technical aspects of HD broadcasting in great detail over a 5-day long blog, and a petition for the channel to be restored to its (supposedly) former glory has appeared on the government's Number Ten website.

To make a long story short, in August, BBC HD upgraded their MPEG4 encoders to what they claimed was a more efficient version, and dropped the channels bitrate by nearly 40%, which caused angry viewers to complain about what they perceived as a drop in quality. Apart from identifying a small problem during certain transitions, which was quickly fixed, the BBC's own extensive testing suggested that the new encoders were so much more efficient that, even at the lower bitrate, the picture quality was as good, and often better than before. This claim that was backed up by tests performed by the consumer group Which?, but not at all by angry viewers.

Now, what a lot of people didn't know at the time, and which was confirmed by the aforementioned Andy Quested, was that Virgin's cable service is MPEG2 and is encoded by Virgin themselves from an HDSDI feed direct from BBC HQ, and thus would not have been affected by the change to the BBC's MPEG4 encoders. This didn't stop angry cable viewers from complaining that the quality, which supposedly used to be "like looking out of a window", now apparently looked worse than your average YouTube video or SD broadcast. So the question is, was this just internet and media-induced hysteria, a sort of Emperor's New Clothes syndrome in reverse, or had Virgin's service really suffered a drop in quality? Thanks to a BBC HD New Year's Eve repeat of Pirates of The Caribbean 2, we might be able to shed some light on this issue, since I also happened to have a recording of the original 2008 airing. You know, from back when BBC HD was "like looking out of a window". My comparison of the two recordings suggests that whomever made this claim should probably have their windows checked, rather than blaming the BBC for any wrong-doing. Not just because no HD broadcast anywhere in the world would ever be like looking out of a window, but also because the new recording actually proved to have marginally better picture quality than the original.

For the sake of keeping the images uncompressed, I've cropped them a little to keep them under Imageshack's 1.5mb limit. No other processing or filtering has been applied.

Anyway, enough rambling and onto...

Exhibit A:

Here we have Keira Knightley getting up close and personal with Johnny Depp. There's very little movement in this shot, so not much risk of anything going wrong. You couldn't really say one version is better than the other, there are just tiny differences in the film grain, but the detail in Knightley's hair is intact in both versions.

OLD
http://img51.imageshack.us/img51/1982/bbchd01old.png

NEW
http://img94.imageshack.us/img94/2237/bbchd01new.png


Exhibit B:

On this scene change, we can see some significant differences, though. On the old version, you can see very clear compression artifacts to the right of Depp's hand at the bottom of the image, and on the right side of the image, the link in the chain takes a pretty bad compression hit as well, which gives it a jagged, blocky looking edge that's almost perfectly smooth in the new version.

OLD
http://img64.imageshack.us/img64/8352/bbchd02old.png

NEW
http://img38.imageshack.us/img38/5418/bbchd02new.png

Exhibit C:

Depp takes on The Kraken, and again there are very few differences between the two versions. Certainly nothing you could spot from a normal viewing distance, and certainly nothing that could be described as the difference between an SD broadcast and looking out of the window. If you zoom in really close, you can find little areas, like the upper two creases on Depp's left thigh being slightly better reproduced in the new version, and there's a minor wobble on The Kraken's top tooth in the old version. You can see it as tiny difference in the two stills, and it's slightly more noticeable on the moving video. That's really quite absurd nitpicking, though, but it does give the new version a tiny edge.

OLD
http://img94.imageshack.us/img94/3971/bbchd03old.png

NEW
http://img94.imageshack.us/img94/2862/bbchd03new.png


Exhibit D

In this scene, the camera pans quickly across the floor of a ship, so everything is moving. As expected, you do see a bit more compression artifacts on such scenes, but I wouldn't exactly say there's much difference between the two, much less claim one was significantly better than the other.

OLD
http://img707.imageshack.us/img707/4498/bbchd04old.png

NEW
http://img685.imageshack.us/img685/5886/bbchd04new.png

The findings above were repeated throughout other scenes in the two recordings. The new version was as good, and in some cases slightly better than the old version, and judging by this example, there has been no dramatic change in the quality of BBC HD on cable. Keep in mind, though, it's still possible that there's something funky going on with the sat version, but considering how many cable users seem to have complained about a quality degradation that simply doesn't appear to be real, I wonder if some of the sat people might be imagining things, or if it's simply a case of certain set top boxes not being compatible with the profile level for the optimized MPEG4 encoders. I'd love to do some direct cable vs sat comparisons, if any of our resident sat users are interested? I wish I could have gotten a sat recording of the Pirates movie, but I only just happened to come across the repeat as I was heading out, and it wasn't able to track down any sat people on short notice.

What's interesting about these samples is that the old broadcast had an average bitrate of 17.60 mbps, and the new one 16.50 mbps. Checking some other older recordings against recent ones, it turns out that Virgin's version of BBC HD actually dropped its bitrate a little bit when they introduced the new HD channels, but all the testing they did to fine-tune the encoders at the time, which was quite visible on the first test versions of the new HD channels, obviously means they are now at least as efficient at the slightly lower rate, and in some cases even more so.

Now, just for the fun of it, I wanted to see how the BBC broadcast of this movie compared to the BluRay. Unfortunately, I couldn't find anyone who had a copy on BluRay, but I did find a full resolution x264 version. This obviously means it would have been recompressed, but my past experience with x264 has shown it to be an incredibly efficient codec, and you generally don't lose all that much. If anybody has an actual BluRay version of the movie and could grab a few uncompressed stills, that would be great, though.

For the samples below, I didn't bother including the old BBC recording.


Exhibit E:

I picked this scene because rain is something that really puts a strain on MPEG encoding, and because it had some nice, fairly sharp detail. I have to say, the BBC HD version holds up quite well to the X264 version. Apart from the slightly different cropping/zoom factor you always find between broadcasts and commercial releases, I doubt anyone would be able to notice much difference from a normal viewing distance.

BBC HD
http://img190.imageshack.us/img190/9429/x264vsbbchd01bbchd.png

X264
http://img94.imageshack.us/img94/7784/x264vsbbchd01x264.png

Exhibit F:

This shot was chosen because a lot of people have complained about "grain" and "noise" particularly in dark scenes, which never sounded to me like an encoding issue. HD just shows such inherit imperfections in the source material, and here the film grain is equally obvious in both versions, but other than that, there's very little difference in terms of overall sharpness and clarity, though the X264 version does have a slight edge, but it's still not something you'd be likely to spot from a normal viewing position.

BBC HD
http://img686.imageshack.us/img686/8444/x264vsbbchd02bbchd.png

X264
http://img190.imageshack.us/img190/3494/x264vsbbchd02x264.png

Side note: Despite BBC HD technically being 1080i, 24/25hz material is essentially broadcast as 1080p, so it's directly comparable to the 1080p X264/BluRay version.

very very interesting post .. rep added .

chrispeters said:

very very interesting post .. rep added .

Click to expand...

have to agree there,many thanks señor ding dong for this detailed thread

I do find however that BBCHD on freesat compared to cable is slightly better
but saying that looking at the cable /bluray comparison there isn't a lot in that at all really
so sat to cable shouldn't be that different either ?

Rat said:

have to agree there,many thanks señor ding dong for this detailed thread

I do find however that BBCHD on freesat compared to cable is slightly better
but saying that looking at the cable /bluray comparison there isn't a lot in that at all really
so sat to cable shouldn't be that different either ?

Click to expand...

You wouldn't have thought so, but there can be quite considerable differences depending on your exact hardware and settings. Some set top boxes don't have very good display hardware, there can be internal processing like sharpening and noise reduction going on, and TV sets themselves can also apply all manners of additional processing or use different deinterlacing/IVTC methods, particularly the 100hz+ sets, so it's difficult to make an accurate like-for-like comparison that way. However, when we have recordings from both sources, we can make direct comparisons of the source material using the same "test bed" for both, with the only variation being the codec (MPEG2 vs MPEG4) which is why I hope someone recorded the movie on sat yesterday.

Might be worth asking in the sat section to see if anyone recorded it via sat m8
and link them to this thread

While I'm waiting for some more test material, I thought I'd post a few more Pirates comparisons. I found these screenshots from the BluRay online, and the results are essentially the same as the x264 shots. There's very little difference between these and the BBC HD version.

BBC HD
http://img85.imageshack.us/img85/3446/pirates01bbchd.jpg

BluRay
http://img709.imageshack.us/img709/9909/pirates01bluray.jpg


BBC HD
http://img36.imageshack.us/img36/959/pirates02bbchd.jpg

BluRay
http://img709.imageshack.us/img709/3586/pirates02bluray.jpg


BBC HD
http://img197.imageshack.us/img197/9048/pirates03bbchd.jpg

BluRay
http://img138.imageshack.us/img138/3162/pirates03bluray.jpg

i think SDD has wayyyyyy to much time on his hands.... lol

thanks mate for your detailed thread... you always do amaze me with your indepth analysys on various subjects..

i am suprised you dont run your own blog / info site

thanks again mate much appreciated

neorage said:

i think SDD has wayyyyyy to much time on his hands.... lol

Click to expand...

Heh, you're not wrong.

i am suprised you dont run your own blog / info site

Click to expand...

I wanted to, but boring-ramblings-about-stuff-nobody-really-cares-about.blogspot.com was already taken.

IMO BBCHD on sat has gone down a lot in quality since they changed the encorders,worst of all my ATI cards will no longer work with hardware decording,and even the nvidia cards have pixallation at the start often.wimbledon 2009 quality was fantastic just before the change.To cable,and I have a v+ box and and dvb c tuner,while I much prefer the mpeg 2 ts(its easier to work with than the h264 ts streams) I think the pre encorder change sat BBCHD is better,as for the vbox+ I think it does such a good job of upscaling SD that the HD looks nothing special,I did capture the mtv awards on my pc and that looked fanastic though.I've been capturing BBCHD from freesat for a few years and IMHO and there was a wow factor at the quality that no longer is there for me.maybe the rumours that the BBC are degrading the quality so it works on freeview are true?

johnoo said:

IMO BBCHD on sat has gone down a lot in quality since they changed the encorders

Click to expand...

Looking at the BBC blog and the screenshots posted there, it does suggest that the encoder change caused the sat version to suffer from some problems that obviously didn't affect cable, but on the other hand, there were questions about the process used to obtain some of the screenshots, which made it difficult to say for sure. The problems you've experienced with playback and pixellation makes me wonder if there's a general compatibility problem with some hardware or codecs that's causing problems for just certain people.

To cable,and I have a v+ box and and dvb c tuner,while I much prefer the mpeg 2 ts(its easier to work with than the h264 ts streams) I think the pre encorder change sat BBCHD is better,

Click to expand...

Yes, all things being even, MPEG4 at 16mbit should certainly be better than MPEG2 at 17mbit, and I must say, the BBC's claims about being able to maintain the same quality at 9-10mbit really does sound a bit far fetched, but it's not something I've been able to look into in much depth.

maybe the rumours that the BBC are degrading the quality so it works on freeview are true?

Click to expand...

I think that's unlikely. Their frustration with this whole issue, and the amount of effort they've put into trying to explain some of the technical aspects, sounds to me like they genuinely don't believe there's a significant problem. And there really isn't all that much you can do to make video look better at lower bitrates. I mean, the bitrate is quite low now, and lots of people don't think it looks good, so....

So, someone was nice enough to supply me with some uncompressed screenshots from a recent episode of Gavin and Stacey on BBC HD on sat. It's kind of difficult to do an accurate 1:1 pixel-by-pixel comparison the way I could with the two Pirates recordings, as the channels use different codecs, and the sat version is only 1440px wide compared to the cable version's 1920, so I've instead chosen to resize the cable version down to the same size as the AR-corrected sat screenshots.

Also, since I didn't have the actual video from the sat version, it's not really possible to say how the different compression might have appeared on actual playback. Screenshots only tell part of the story, but it does seem to be a very similar story for both versions, which is that the quality of this particular broadcast was pretty crap on both sat and cable. But... that's really nothing new. I remember seeing fuzzy shows like this years ago, long before anyone started complaining about lowered bitrates and MPEG2 vs MPEG4.

I can't say I really have much of a clue as to what's causing these random, low quality broadcasts, but given the type of degradation we're seeing, I don't think it has anything to do with the encoders themselves. It would be unlikely for two such different set of specs to produce such similar degradation. There might be some additional blockiness issues on sat with fast motion, as a result of the lowered bitrate, which might be what has led people to blame the lower encoding specs for the overall poor quality of certain broadcasts. I haven't been able to test fast motion content from sat, though, but screenshots posted by users on the BBC blog suggest this might be the case. These tests were just to see how more stable and less problematic content would be reproduced on the two platforms, and it's really quite similar, as can be seen from the screenshots below.

Sat
http://img27.imageshack.us/img27/9928/virginvssat01sat.png

Cable
http://img406.imageshack.us/img406/6463/virginvssat01virginresi.png

Sat
http://img27.imageshack.us/img27/5313/virginvssat02sat.png

Cable
http://img27.imageshack.us/img27/7426/virginvssat02virginresi.png

Sat
http://img413.imageshack.us/img413/5876/virginvssat03sat.png

Cable
http://img30.imageshack.us/img30/9872/virginvssat03virginresi.png

Sat
http://img156.imageshack.us/img156/7349/virginvssat04sat.png

Cable
http://img406.imageshack.us/img406/6922/virginvssat04virginresi.png

For the most part, the sat version probably has a slight edge, but it's difficult to say for sure without being able to compare a sequence of P and I frames, as there's an element of randomness to where on the quality-slope any given screenshot might have been taken. The sat version probably has the edge on the two first sets. The 3rd set is a bit of a toss-up, as the sat version does certain things well, but loses out on others. On the 4th set, the cable version probably handles a few of the smaller details better, as there's more overall movement, and I suspect the cable version's more adequate bitrate would handle faster or more intense motion better. All in all, though, given the differences and variables, it's a far more subjective comparison than the Pirates example, but there doesn't appear to be a massive difference between the two sets, which suggests that the problem is with the source material, not the encoders. Though why certain episodes of Gavin & Stacey look pin sharp, and others, like this one, leave a lot to be desired, is impossible to say. I can't really get on board with the conspiracy theory that the BBC are deliberately softening the picture to make it look better at lower bitrates, as it obviously doesn't look good, and I doubt a sharper source image would look worse.