unfair eula

Hello!
As most of you know, there are several audio codecs available today: MP3, Ogg Vorbis, AAC, WMA, Musepack, etc. Often, there are also more encoders available that encode to a specific format. For example, you can use Nero Digital, iTunes or Coding Technologies encoders to create AAC bitstreams. MP3 can also be encoded by LAME, Shine, Blade Encoder, etc. Being an audio enthusiast and wanting to know which codec and encoder delivers the best sound quality at a given bitrate, I started to run blind listening tests. Because these tests are ran blindly, the testers cannot rate MP3 higher than WMA for example just because they are MP3 zealots or WMA haters - the tester doesn't know which sample he is listening to. Moreover, the tester cannot pretend to hear artifacts if he isn't able to pass an ABX test, For example, if he manages to tell apart sample A from sample B 8 times one after another, we can almost say for sure that there really is an audible difference between the two samples. If you want to read more about blind tests, especially about the ABC/HR or ABX method, there are several documents which you can find with Google or whatever search engine you prefer.
Probably you are asking yourself now what this has to do with Windows Vista. Windows Vista and Windows Media Player 11 introduced a new WMA encoder - WMA 9.2 and WMA 10. Even before Microsoft launched these two products to the public, they published the results of a listening test comparing HE-AAC against WMA: http://www.microsoft.com/windows/windowsmedia/forpros/codecs/Comparison.aspx Microsoft claims that the test was also done blindly and that WMA had a higher rating that HE-AAC. While in my opinion this test was unfair, since Microsoft instructed NSTL which settings to use and also provided a very recent encoder, NSTL didn't discuss the settings with the Nero Digital developers and also didn't ask for the most recent version of the HE-AAC encoder. Not wanting to believe the results, especially since listening tests in the past showed that WMA was often worse than AAC, I could hardly wait for Windows Vista or Windows Media Player 11 to become available in order to run my own test. However, the EULA of both Windows Vista and WMP 11 state that I am not allowed to publish benchmark results without having a written permission from MS to do so. Not wanting to violate the EULA, I started to get in touch with several MS developers and also with the licensing department. Half of the e-mails I sent remained unanswered, for a quarter of them I received some standard replies that MS is checking the issue internally and that someone would get in touch with me ASAP and another quarter contained some vague information that the encoder is still beta and that it can change at any time and might still contain bugs. Well, the given arguments are very weak. Nero AG, LAME developers, Vorbis developers etc. also made beta encoders available for such tests and didn't fear that the outcome of the result might harm their image. I am also very upset that while I am not allowed to disclose results of my own tests, Microsoft still has the results of their test online - most likely using the same beta encoder I want to use and which might be buggy as well. I think that with this attitude towards audio enthusiasts and possible future customers Microsoft is doing more harm than by allowing me to publish the results of the test. Such test are not made to harm a company - no way, they can also be very helpful. LAME and Nero got tuned because of such listening tests.
I really hope that someone from Microsoft who is responsible for WMA or for licenses will get in touch with me.
Regards, Sebastian

On Thu, 8 Jun 2006 12:55:01 -0700, Sebastian Mares wrote:
All EULAS are suspect and most includng MSFTs have never been tested in court to my knowlege. Thier legality is not all its cracked up to be.

Hello!
As most of you know, there are several audio codecs available today: MP3, Ogg Vorbis, AAC, WMA, Musepack, etc. Often, there are also more encoders available that encode to a specific format. For example, you can use Nero Digital, iTunes or Coding Technologies encoders to create AAC bitstreams. MP3 can also be encoded by LAME, Shine, Blade Encoder, etc. Being an audio enthusiast and wanting to know which codec and encoder delivers the best sound quality at a given bitrate, I started to run blind listening tests. Because these tests are ran blindly, the testers cannot rate MP3 higher than WMA for example just because they are MP3 zealots or WMA haters - the tester doesn't know which sample he is listening to. Moreover, the tester cannot pretend to hear artifacts if he isn't able to pass an ABX test, For example, if he manages to tell apart sample A from sample B 8 times one after another, we can almost say for sure that there really is an audible difference between the two samples. If you want to read more about blind tests, especially about the ABC/HR or ABX method, there are several documents which you can find with Google or whatever search engine you prefer.
Probably you are asking yourself now what this has to do with Windows Vista. Windows Vista and Windows Media Player 11 introduced a new WMA encoder - WMA 9.2 and WMA 10. Even before Microsoft launched these two products to the public, they published the results of a listening test comparing HE-AAC against WMA: http://www.microsoft.com/windows/windowsmedia/forpros/codecs/Comparison.aspx Microsoft claims that the test was also done blindly and that WMA had a higher rating that HE-AAC. While in my opinion this test was unfair, since Microsoft instructed NSTL which settings to use and also provided a very recent encoder, NSTL didn't discuss the settings with the Nero Digital developers and also didn't ask for the most recent version of the HE-AAC encoder. Not wanting to believe the results, especially since listening tests in the past showed that WMA was often worse than AAC, I could hardly wait for Windows Vista or Windows Media Player 11 to become available in order to run my own test. However, the EULA of both Windows Vista and WMP 11 state that I am not allowed to publish benchmark results without having a written permission from MS to do so. Not wanting to violate the EULA, I started to get in touch with several MS developers and also with the licensing department. Half of the e-mails I sent remained unanswered, for a quarter of them I received some standard replies that MS is checking the issue internally and that someone would get in touch with me ASAP and another quarter contained some vague information that the encoder is still beta and that it can change at any time and might still contain bugs. Well, the given arguments are very weak. Nero AG, LAME developers, Vorbis developers etc. also made beta encoders available for such tests and didn't fear that the outcome of the result might harm their image. I am also very upset that while I am not allowed to disclose results of my own tests, Microsoft still has the results of their test online - most likely using the same beta encoder I want to use and which might be buggy as well. I think that with this attitude towards audio enthusiasts and possible future customers Microsoft is doing more harm than by allowing me to publish the results of the test. Such test are not made to harm a company - no way, they can also be very helpful. LAME and Nero got tuned because of such listening tests.
I really hope that someone from Microsoft who is responsible for WMA or for licenses will get in touch with me.
Regards, Sebastian

How again is this unfair? You don't own them, you are given a license to use them under their rules. If you don't like it, don't use it.
"Sebastian Mares" wrote in message

Hello!
As most of you know, there are several audio codecs available today: MP3, Ogg Vorbis, AAC, WMA, Musepack, etc. Often, there are also more encoders available that encode to a specific format. For example, you can use Nero Digital, iTunes or Coding Technologies encoders to create AAC bitstreams. MP3 can also be encoded by LAME, Shine, Blade Encoder, etc. Being an audio enthusiast and wanting to know which codec and encoder delivers the best sound quality at a given bitrate, I started to run blind listening tests. Because these tests are ran blindly, the testers cannot rate MP3 higher than WMA for example just because they are MP3 zealots or WMA haters - the tester doesn't know which sample he is listening to. Moreover, the tester cannot pretend to hear artifacts if he isn't able to pass an ABX test, For example, if he manages to tell apart sample A from sample B 8 times one after another, we can almost say for sure that there really is an audible difference between the two samples. If you want to read more about blind tests, especially about the ABC/HR or ABX method, there are several documents which you can find with Google or whatever search engine you prefer.
Probably you are asking yourself now what this has to do with Windows Vista. Windows Vista and Windows Media Player 11 introduced a new WMA encoder - WMA 9.2 and WMA 10. Even before Microsoft launched these two products to the public, they published the results of a listening test comparing HE-AAC against WMA: http://www.microsoft.com/windows/windowsmedia/forpros/codecs/Comparison.aspx Microsoft claims that the test was also done blindly and that WMA had a higher rating that HE-AAC. While in my opinion this test was unfair, since Microsoft instructed NSTL which settings to use and also provided a very recent encoder, NSTL didn't discuss the settings with the Nero Digital developers and also didn't ask for the most recent version of the HE-AAC encoder. Not wanting to believe the results, especially since listening tests in the past showed that WMA was often worse than AAC, I could hardly wait for Windows Vista or Windows Media Player 11 to become available in order to run my own test. However, the EULA of both Windows Vista and WMP 11 state that I am not allowed to publish benchmark results without having a written permission from MS to do so. Not wanting to violate the EULA, I started to get in touch with several MS developers and also with the licensing department. Half of the e-mails I sent remained unanswered, for a quarter of them I received some standard replies that MS is checking the issue internally and that someone would get in touch with me ASAP and another quarter contained some vague information that the encoder is still beta and that it can change at any time and might still contain bugs. Well, the given arguments are very weak. Nero AG, LAME developers, Vorbis developers etc. also made beta encoders available for such tests and didn't fear that the outcome of the result might harm their image. I am also very upset that while I am not allowed to disclose results of my own tests, Microsoft still has the results of their test online - most likely using the same beta encoder I want to use and which might be buggy as well. I think that with this attitude towards audio enthusiasts and possible future customers Microsoft is doing more harm than by allowing me to publish the results of the test. Such test are not made to harm a company - no way, they can also be very helpful. LAME and Nero got tuned because of such listening tests.
I really hope that someone from Microsoft who is responsible for WMA or for licenses will get in touch with me.
Regards, Sebastian

You should just freaking say whatever you want, I sure as heck would. There's nothing wrong with doing a listening test like listening to 2 seperate songs and then telling people how it sounded. MS won't freaking do a dang thing.
"Sebastian Mares" wrote in message

Hello!
As most of you know, there are several audio codecs available today: MP3, Ogg Vorbis, AAC, WMA, Musepack, etc. Often, there are also more encoders available that encode to a specific format. For example, you can use Nero Digital, iTunes or Coding Technologies encoders to create AAC bitstreams. MP3 can also be encoded by LAME, Shine, Blade Encoder, etc. Being an audio enthusiast and wanting to know which codec and encoder delivers the best sound quality at a given bitrate, I started to run blind listening tests. Because these tests are ran blindly, the testers cannot rate MP3 higher than WMA for example just because they are MP3 zealots or WMA haters - the tester doesn't know which sample he is listening to. Moreover, the tester cannot pretend to hear artifacts if he isn't able to pass an ABX test, For example, if he manages to tell apart sample A from sample B 8 times one after another, we can almost say for sure that there really is an audible difference between the two samples. If you want to read more about blind tests, especially about the ABC/HR or ABX method, there are several documents which you can find with Google or whatever search engine you prefer.
Probably you are asking yourself now what this has to do with Windows Vista. Windows Vista and Windows Media Player 11 introduced a new WMA encoder - WMA 9.2 and WMA 10. Even before Microsoft launched these two products to the public, they published the results of a listening test comparing HE-AAC against WMA: http://www.microsoft.com/windows/windowsmedia/forpros/codecs/Comparison.aspx Microsoft claims that the test was also done blindly and that WMA had a higher rating that HE-AAC. While in my opinion this test was unfair, since Microsoft instructed NSTL which settings to use and also provided a very recent encoder, NSTL didn't discuss the settings with the Nero Digital developers and also didn't ask for the most recent version of the HE-AAC encoder. Not wanting to believe the results, especially since listening tests in the past showed that WMA was often worse than AAC, I could hardly wait for Windows Vista or Windows Media Player 11 to become available in order to run my own test. However, the EULA of both Windows Vista and WMP 11 state that I am not allowed to publish benchmark results without having a written permission from MS to do so. Not wanting to violate the EULA, I started to get in touch with several MS developers and also with the licensing department. Half of the e-mails I sent remained unanswered, for a quarter of them I received some standard replies that MS is checking the issue internally and that someone would get in touch with me ASAP and another quarter contained some vague information that the encoder is still beta and that it can change at any time and might still contain bugs. Well, the given arguments are very weak. Nero AG, LAME developers, Vorbis developers etc. also made beta encoders available for such tests and didn't fear that the outcome of the result might harm their image. I am also very upset that while I am not allowed to disclose results of my own tests, Microsoft still has the results of their test online - most likely using the same beta encoder I want to use and which might be buggy as well. I think that with this attitude towards audio enthusiasts and possible future customers Microsoft is doing more harm than by allowing me to publish the results of the test. Such test are not made to harm a company - no way, they can also be very helpful. LAME and Nero got tuned because of such listening tests.
I really hope that someone from Microsoft who is responsible for WMA or for licenses will get in touch with me.
Regards, Sebastian

I think it's a rouse to get interest in his test. Lack of volunteers, apparently, is his real beef. This he-aac thing is sort of like stereo AM radio -- interesting until you heard it and realized it's still not very good.
aa- [Thu, 8 Jun 2006 23:14:30 -0700]:

You should just freaking say whatever you want, I sure as heck would.

He's spam-posted this to other NG(s), so one can judge its merit on that alone.
-- 40th Floor - Software @ http://40th.com/ iPlay : the ultimate audio player for mobiles parametric eq, xfeed, reverb; all on a mobile

Agreed.
wrote in message

I think it's a rouse to get interest in his test. Lack of volunteers, apparently, is his real beef. This he-aac thing is sort of like stereo AM radio -- interesting until you heard it and realized it's still not very good.
aa- [Thu, 8 Jun 2006 23:14:30 -0700]: You should just freaking say whatever you want, I sure as heck would.
He's spam-posted this to other NG(s), so one can judge its merit on that alone.
-- 40th Floor - Software @ http://40th.com/ iPlay : the ultimate audio player for mobiles parametric eq, xfeed, reverb; all on a mobile

Hello!
As most of you know, there are several audio codecs available today: MP3, Ogg Vorbis, AAC, WMA, Musepack, etc. Often, there are also more encoders available that encode to a specific format. For example, you can use Nero Digital, iTunes or Coding Technologies encoders to create AAC bitstreams. MP3 can also be encoded by LAME, Shine, Blade Encoder, etc. Being an audio enthusiast and wanting to know which codec and encoder delivers the best sound quality at a given bitrate, I started to run blind listening tests. Because these tests are ran blindly, the testers cannot rate MP3 higher than WMA for example just because they are MP3 zealots or WMA haters - the tester doesn't know which sample he is listening to. Moreover, the tester cannot pretend to hear artifacts if he isn't able to pass an ABX test, For example, if he manages to tell apart sample A from sample B 8 times one after another, we can almost say for sure that there really is an audible difference between the two samples. If you want to read more about blind tests, especially about the ABC/HR or ABX method, there are several documents which you can find with Google or whatever search engine you prefer.
Probably you are asking yourself now what this has to do with Windows Vista. Windows Vista and Windows Media Player 11 introduced a new WMA encoder - WMA 9.2 and WMA 10. Even before Microsoft launched these two products to the public, they published the results of a listening test comparing HE-AAC against WMA: http://www.microsoft.com/windows/windowsmedia/forpros/codecs/Comparison.aspx Microsoft claims that the test was also done blindly and that WMA had a higher rating that HE-AAC. While in my opinion this test was unfair, since Microsoft instructed NSTL which settings to use and also provided a very recent encoder, NSTL didn't discuss the settings with the Nero Digital developers and also didn't ask for the most recent version of the HE-AAC encoder. Not wanting to believe the results, especially since listening tests in the past showed that WMA was often worse than AAC, I could hardly wait for Windows Vista or Windows Media Player 11 to become available in order to run my own test. However, the EULA of both Windows Vista and WMP 11 state that I am not allowed to publish benchmark results without having a written permission from MS to do so. Not wanting to violate the EULA, I started to get in touch with several MS developers and also with the licensing department. Half of the e-mails I sent remained unanswered, for a quarter of them I received some standard replies that MS is checking the issue internally and that someone would get in touch with me ASAP and another quarter contained some vague information that the encoder is still beta and that it can change at any time and might still contain bugs. Well, the given arguments are very weak. Nero AG, LAME developers, Vorbis developers etc. also made beta encoders available for such tests and didn't fear that the outcome of the result might harm their image. I am also very upset that while I am not allowed to disclose results of my own tests, Microsoft still has the results of their test online - most likely using the same beta encoder I want to use and which might be buggy as well. I think that with this attitude towards audio enthusiasts and possible future customers Microsoft is doing more harm than by allowing me to publish the results of the test. Such test are not made to harm a company - no way, they can also be very helpful. LAME and Nero got tuned because of such listening tests.
I really hope that someone from Microsoft who is responsible for WMA or for licenses will get in touch with me.
Regards, Sebastian

Hello!
As most of you know, there are several audio codecs available today: MP3, Ogg Vorbis, AAC, WMA, Musepack, etc. Often, there are also more encoders available that encode to a specific format. For example, you can use Nero Digital, iTunes or Coding Technologies encoders to create AAC bitstreams. MP3 can also be encoded by LAME, Shine, Blade Encoder, etc. Being an audio enthusiast and wanting to know which codec and encoder delivers the best sound quality at a given bitrate, I started to run blind listening tests. Because these tests are ran blindly, the testers cannot rate MP3 higher than WMA for example just because they are MP3 zealots or WMA haters - the tester doesn't know which sample he is listening to. Moreover, the tester cannot pretend to hear artifacts if he isn't able to pass an ABX test, For example, if he manages to tell apart sample A from sample B 8 times one after another, we can almost say for sure that there really is an audible difference between the two samples. If you want to read more about blind tests, especially about the ABC/HR or ABX method, there are several documents which you can find with Google or whatever search engine you prefer.
Probably you are asking yourself now what this has to do with Windows Vista. Windows Vista and Windows Media Player 11 introduced a new WMA encoder - WMA 9.2 and WMA 10. Even before Microsoft launched these two products to the public, they published the results of a listening test comparing HE-AAC against WMA: http://www.microsoft.com/windows/windowsmedia/forpros/codecs/Comparison.aspx Microsoft claims that the test was also done blindly and that WMA had a higher rating that HE-AAC. While in my opinion this test was unfair, since Microsoft instructed NSTL which settings to use and also provided a very recent encoder, NSTL didn't discuss the settings with the Nero Digital developers and also didn't ask for the most recent version of the HE-AAC encoder. Not wanting to believe the results, especially since listening tests in the past showed that WMA was often worse than AAC, I could hardly wait for Windows Vista or Windows Media Player 11 to become available in order to run my own test. However, the EULA of both Windows Vista and WMP 11 state that I am not allowed to publish benchmark results without having a written permission from MS to do so. Not wanting to violate the EULA, I started to get in touch with several MS developers and also with the licensing department. Half of the e-mails I sent remained unanswered, for a quarter of them I received some standard replies that MS is checking the issue internally and that someone would get in touch with me ASAP and another quarter contained some vague information that the encoder is still beta and that it can change at any time and might still contain bugs. Well, the given arguments are very weak. Nero AG, LAME developers, Vorbis developers etc. also made beta encoders available for such tests and didn't fear that the outcome of the result might harm their image. I am also very upset that while I am not allowed to disclose results of my own tests, Microsoft still has the results of their test online - most likely using the same beta encoder I want to use and which might be buggy as well. I think that with this attitude towards audio enthusiasts and possible future customers Microsoft is doing more harm than by allowing me to publish the results of the test. Such test are not made to harm a company - they can be very helpful. LAME and Nero got tuned because of such listening tests.
I really hope that someone from Microsoft who is responsible for WMA or for licenses will get in touch with me.
Regards, Sebastian

Hello!
As most of you know, there are several audio codecs available today: MP3, Ogg Vorbis, AAC, WMA, Musepack, etc. Often, there are also more encoders available that encode to a specific format. For example, you can use Nero Digital, iTunes or Coding Technologies encoders to create AAC bitstreams. MP3 can also be encoded by LAME, Shine, Blade Encoder, etc. Being an audio enthusiast and wanting to know which codec and encoder delivers the best sound quality at a given bitrate, I started to run blind listening tests. Because these tests are ran blindly, the testers cannot rate MP3 higher than WMA for example just because they are MP3 zealots or WMA haters - the tester doesn't know which sample he is listening to. Moreover, the tester cannot pretend to hear artifacts if he isn't able to pass an ABX test, For example, if he manages to tell apart sample A from sample B 8 times one after another, we can almost say for sure that there really is an audible difference between the two samples. If you want to read more about blind tests, especially about the ABC/HR or ABX method, there are several documents which you can find with Google or whatever search engine you prefer.
Probably you are asking yourself now what this has to do with Windows Vista. Windows Vista and Windows Media Player 11 introduced a new WMA encoder - WMA 9.2 and WMA 10. Even before Microsoft launched these two products to the public, they published the results of a listening test comparing HE-AAC against WMA: http://www.microsoft.com/windows/windowsmedia/forpros/codecs/Comparison.aspx Microsoft claims that the test was also done blindly and that WMA had a higher rating that HE-AAC. While in my opinion this test was unfair, since Microsoft instructed NSTL which settings to use and also provided a very recent encoder, NSTL didn't discuss the settings with the Nero Digital developers and also didn't ask for the most recent version of the HE-AAC encoder. Not wanting to believe the results, especially since listening tests in the past showed that WMA was often worse than AAC, I could hardly wait for Windows Vista or Windows Media Player 11 to become available in order to run my own test. However, the EULA of both Windows Vista and WMP 11 state that I am not allowed to publish benchmark results without having a written permission from MS to do so. Not wanting to violate the EULA, I started to get in touch with several MS developers and also with the licensing department. Half of the e-mails I sent remained unanswered, for a quarter of them I received some standard replies that MS is checking the issue internally and that someone would get in touch with me ASAP and another quarter contained some vague information that the encoder is still beta and that it can change at any time and might still contain bugs. Well, the given arguments are very weak. Nero AG, LAME developers, Vorbis developers etc. also made beta encoders available for such tests and didn't fear that the outcome of the result might harm their image. I am also very upset that while I am not allowed to disclose results of my own tests, Microsoft still has the results of their test online - most likely using the same beta encoder I want to use and which might be buggy as well. I think that with this attitude towards audio enthusiasts and possible future customers Microsoft is doing more harm than by allowing me to publish the results of the test. Such test are not made to harm a company - they can be very helpful. LAME and Nero got tuned because of such listening tests.
I really hope that someone from Microsoft who is responsible for WMA or for licenses will get in touch with me.
Regards, Sebastian

Hello!
As most of you know, there are several audio codecs available today: MP3, Ogg Vorbis, AAC, WMA, Musepack, etc. Often, there are also more encoders available that encode to a specific format. For example, you can use Nero Digital, iTunes or Coding Technologies encoders to create AAC bitstreams. MP3 can also be encoded by LAME, Shine, Blade Encoder, etc. Being an audio enthusiast and wanting to know which codec and encoder delivers the best sound quality at a given bitrate, I started to run blind listening tests. Because these tests are ran blindly, the testers cannot rate MP3 higher than WMA for example just because they are MP3 zealots or WMA haters - the tester doesn't know which sample he is listening to. Moreover, the tester cannot pretend to hear artifacts if he isn't able to pass an ABX test, For example, if he manages to tell apart sample A from sample B 8 times one after another, we can almost say for sure that there really is an audible difference between the two samples. If you want to read more about blind tests, especially about the ABC/HR or ABX method, there are several documents which you can find with Google or whatever search engine you prefer.
Probably you are asking yourself now what this has to do with Windows Vista. Windows Vista and Windows Media Player 11 introduced a new WMA encoder - WMA 9.2 and WMA 10. Even before Microsoft launched these two products to the public, they published the results of a listening test comparing HE-AAC against WMA: http://www.microsoft.com/windows/windowsmedia/forpros/codecs/Comparison.aspx Microsoft claims that the test was also done blindly and that WMA had a higher rating that HE-AAC. While in my opinion this test was unfair, since Microsoft instructed NSTL which settings to use and also provided a very recent encoder, NSTL didn't discuss the settings with the Nero Digital developers and also didn't ask for the most recent version of the HE-AAC encoder. Not wanting to believe the results, especially since listening tests in the past showed that WMA was often worse than AAC, I could hardly wait for Windows Vista or Windows Media Player 11 to become available in order to run my own test. However, the EULA of both Windows Vista and WMP 11 state that I am not allowed to publish benchmark results without having a written permission from MS to do so. Not wanting to violate the EULA, I started to get in touch with several MS developers and also with the licensing department. Half of the e-mails I sent remained unanswered, for a quarter of them I received some standard replies that MS is checking the issue internally and that someone would get in touch with me ASAP and another quarter contained some vague information that the encoder is still beta and that it can change at any time and might still contain bugs. Well, the given arguments are very weak. Nero AG, LAME developers, Vorbis developers etc. also made beta encoders available for such tests and didn't fear that the outcome of the result might harm their image. I am also very upset that while I am not allowed to disclose results of my own tests, Microsoft still has the results of their test online - most likely using the same beta encoder I want to use and which might be buggy as well. I think that with this attitude towards audio enthusiasts and possible future customers Microsoft is doing more harm than by allowing me to publish the results of the test. Such test are not made to harm a company - they can be very helpful. LAME and Nero got tuned because of such listening tests.
I really hope that someone from Microsoft who is responsible for WMA or for licenses will get in touch with me.
Regards, Sebastian

Try submitting this to the Sentiments site - they'd love to hear your feedback: http://go.microsoft.com/fwlink/?LinkId=55160
-- Zack Whittaker » ZackNET Enterprises: www.zacknet.co.uk » MSBlog on ResDev: www.msblog.org » Vista Knowledge Base: www.vistabase.co.uk » This mailing is provided "as is" with no warranties, and confers no rights. All opinions expressed are those of myself unless stated so, and not of my employer, best friend, Ghandi, my mother or my cat. Glad we cleared that up!
--: Original message follows :-- "Sebastian Mares" wrote in message

Hello!
As most of you know, there are several audio codecs available today: MP3, Ogg Vorbis, AAC, WMA, Musepack, etc. Often, there are also more encoders available that encode to a specific format. For example, you can use Nero Digital, iTunes or Coding Technologies encoders to create AAC bitstreams. MP3 can also be encoded by LAME, Shine, Blade Encoder, etc. Being an audio enthusiast and wanting to know which codec and encoder delivers the best sound quality at a given bitrate, I started to run blind listening tests. Because these tests are ran blindly, the testers cannot rate MP3 higher than WMA for example just because they are MP3 zealots or WMA haters - the tester doesn't know which sample he is listening to. Moreover, the tester cannot pretend to hear artifacts if he isn't able to pass an ABX test, For example, if he manages to tell apart sample A from sample B 8 times one after another, we can almost say for sure that there really is an audible difference between the two samples. If you want to read more about blind tests, especially about the ABC/HR or ABX method, there are several documents which you can find with Google or whatever search engine you prefer.
Probably you are asking yourself now what this has to do with Windows Vista. Windows Vista and Windows Media Player 11 introduced a new WMA encoder - WMA 9.2 and WMA 10. Even before Microsoft launched these two products to the public, they published the results of a listening test comparing HE-AAC against WMA: http://www.microsoft.com/windows/windowsmedia/forpros/codecs/Comparison.aspx Microsoft claims that the test was also done blindly and that WMA had a higher rating that HE-AAC. While in my opinion this test was unfair, since Microsoft instructed NSTL which settings to use and also provided a very recent encoder, NSTL didn't discuss the settings with the Nero Digital developers and also didn't ask for the most recent version of the HE-AAC encoder. Not wanting to believe the results, especially since listening tests in the past showed that WMA was often worse than AAC, I could hardly wait for Windows Vista or Windows Media Player 11 to become available in order to run my own test. However, the EULA of both Windows Vista and WMP 11 state that I am not allowed to publish benchmark results without having a written permission from MS to do so. Not wanting to violate the EULA, I started to get in touch with several MS developers and also with the licensing department. Half of the e-mails I sent remained unanswered, for a quarter of them I received some standard replies that MS is checking the issue internally and that someone would get in touch with me ASAP and another quarter contained some vague information that the encoder is still beta and that it can change at any time and might still contain bugs. Well, the given arguments are very weak. Nero AG, LAME developers, Vorbis developers etc. also made beta encoders available for such tests and didn't fear that the outcome of the result might harm their image. I am also very upset that while I am not allowed to disclose results of my own tests, Microsoft still has the results of their test online - most likely using the same beta encoder I want to use and which might be buggy as well. I think that with this attitude towards audio enthusiasts and possible future customers Microsoft is doing more harm than by allowing me to publish the results of the test. Such test are not made to harm a company - they can be very helpful. LAME and Nero got tuned because of such listening tests.
I really hope that someone from Microsoft who is responsible for WMA or for licenses will get in touch with me.
Regards, Sebastian

That's like testing which AM radio station puts out the best signal. Frankly, they are all bad. 48 kbps, no matter how much you polish it, for music is terrible (for those with normal hearing).
SM- [Thu, 8 Jun 2006 13:27:01 -0700]:

Microsoft claims that the test was also done blindly and that WMA had a higher rating that HE-AAC.

Would you expect it to have published a different outcome? AM radio is still AM radio (polished, or not). Anyway, what exactly do you think is going to happen to you if you do "benchmark" this? Just do it. Publish from the netherlands, host it on a site in china, and use godaddy's domain-by-proxy registration. Oh, you already are? hahah
-- 40th Floor - Software @ http://40th.com/ iPlay : the ultimate audio player for mobiles parametric eq, xfeed, reverb; all on a mobile

In that case, I would love to have you in my listening test. Obviously, you have no idea what you're talking about and most likely come to this conclusion based on experience with MP3 or WMA Standard.
"hel@40th.com" wrote:

That's like testing which AM radio station puts out the best signal. Frankly, they are all bad. 48 kbps, no matter how much you polish it, for music is terrible (for those with normal hearing).
SM- [Thu, 8 Jun 2006 13:27:01 -0700]: Microsoft claims that the test was also done blindly and that WMA had a higher rating that HE-AAC.
Would you expect it to have published a different outcome? AM radio is still AM radio (polished, or not). Anyway, what exactly do you think is going to happen to you if you do "benchmark" this? Just do it. Publish from the netherlands, host it on a site in china, and use godaddy's domain-by-proxy registration. Oh, you already are? hahah
-- 40th Floor - Software @ http://40th.com/ iPlay : the ultimate audio player for mobiles parametric eq, xfeed, reverb; all on a mobile

"Sebastian Mares" wrote in message

I think that with this attitude towards audio enthusiasts and possible future customers Microsoft is doing more harm than by allowing me to publish the results of the test. Such test are not made to harm a company - they can be very helpful. LAME and Nero got tuned because of such listening tests.

(said as a friend with respect...)
You actually undercut your argument there - LAME and Nero can tune their code because they got data. There is no value to LAME or Nero to have results of beta code tests published. So if your argument is that you want to improve the WMA codecs, you would simply supply that data to the Microsoft contacts you're dealing/communicating with. There's no particular **unique** value in what you're doing. It's interesting, but it's just as interesting - if not much more - when the released / final version of the codecs come out.
An MVP requested I redirect you - I don't do that kind of thing ever (I need the bridges I have and cannot overtax those), but I pointed out that Zambelli hangs out on the doom9 forums and Amir hangs out on the AVSForums .... forums.
I do not believe you will find any value or response on your quest from any non-SDK-specific newsgroups- I simply don't believe that anyone you want to talk to pays much/any attention to these newsgroups. And even then, Doom9 or AVSForums probably are light years better choices.
Fiiiiinnnnnnaalllllly: bear in mind that everyone you're dealing with is really busy. I respect this may be a top project for you, but those cats you're attempting to get in touch with are juggling a lot of things as Vista moves on closer to release. They're all good people, but I work at Microsoft and know that I've got cut them slack when I deal with because they've got so very much to do on their place. *shrug* - that's just a little insight I thought might be helpful. I've got mails going back months that I haven't had time to deal with. It's there and I'll get to 'em in time, but -- there's a lot of priorities to deal with. =)
-Zach -- Windows Media Development Team (speaking for myself only) See http://zachd.com/pss/pss.html for some helpful WMP info. This posting is provided "AS IS" with no warranties, and confers no rights.

he-aac has lots of distortion. If you need others to hear this for you, you shouldn't be conducting the test. hah You wouldn't expect a guy with glasses that magnify his eyes several times to conduct a sight test, would you? I think you should step down and let someone that has decent hearing do this (if it must be done).
Just kidding guy, but he-aac does sound very bad. I know you nero-fans over on ha think it's great to see how small you can get your lossy files, but you have to draw the line when distortion becomes so obvious. he-aac, at least from what I've heard out of nero (which frankly, never works the same way between releases, and never seems to get better, only different), is a waste. Maybe for those that are hearing impaired? Sure. Just like I don't expect a leagally-blind guy to care if a jpeg is high-quality or not, I don't expect the hearing-impaired guy to care about the quality of his music files.
SM- [Thu, 8 Jun 2006 23:53:02 -0700]:

In that case, I would love to have you in my listening test. Obviously, you have no idea what you're talking about and most likely come to this conclusion based on experience with MP3 or WMA Standard.

Obviously.
-- 40th Floor - Software @ http://40th.com/ iPlay : the ultimate audio player for mobiles parametric eq, xfeed, reverb; all on a mobile

Again, I would like to point out that it seems that you have no idea what you are talking about. As listening test conductor, I can be deaf, my hearing is not important at all. My job is to set up the testing programs, to encode the samples, to collect the results, process them and then publish the results. It's the other people who do the testing. Some people have excellent hearing (like guruboolez on Hydrogenaudio), some don't. After the test is over, there are several tools that are used in order to say with confidence that codec A is better then B.
Also, of course there will always be people who will notice artifacts more than others, but it always comes down to how many people notice them. Someone with good ears might say that MP3 at 128 kbps is crap, however, the majority of the users encode to 128 kbps because for the majority, there is no big difference between the source and the target.
"hel@40th.com" wrote:

he-aac has lots of distortion. If you need others to hear this for you, you shouldn't be conducting the test. hah You wouldn't expect a guy with glasses that magnify his eyes several times to conduct a sight test, would you? I think you should step down and let someone that has decent hearing do this (if it must be done).
Just kidding guy, but he-aac does sound very bad. I know you nero-fans over on ha think it's great to see how small you can get your lossy files, but you have to draw the line when distortion becomes so obvious. he-aac, at least from what I've heard out of nero (which frankly, never works the same way between releases, and never seems to get better, only different), is a waste. Maybe for those that are hearing impaired? Sure. Just like I don't expect a leagally-blind guy to care if a jpeg is high-quality or not, I don't expect the hearing-impaired guy to care about the quality of his music files.
SM- [Thu, 8 Jun 2006 23:53:02 -0700]: In that case, I would love to have you in my listening test. Obviously, you have no idea what you're talking about and most likely come to this conclusion based on experience with MP3 or WMA Standard.
Obviously.
-- 40th Floor - Software @ http://40th.com/ iPlay : the ultimate audio player for mobiles parametric eq, xfeed, reverb; all on a mobile

Hey Zach!
I also talked to Zambelli and he is doing his best to get in touch with someone from the licensing department or whatever. I also saw his posts on Doom9 (actually, Ivan from Nero told me about his reply on D9). I didn't visit AVSForums, though.
Anyways, your argument about testing beta software... If you say that it's so useless, how come Microsoft has no problem publishing the results of the HE-AAC vs. WMA test? Why is it suddenly unimportant that beta software was used? And about the testing methods - Microsoft got in touch with NSTL and told them exactly which settings to use and also provided them with an encoder that wasn't available to the public at that time, while NSTL didn't consider asking Nero (IIRC, they used the Nero HE-AAC encoder) which settings to use and also didn't ask Ivan / Gian-Carlo / Juha / Menno if a newer encoder is available? While I agree that testing final products is more important, testing beta encoders is also useful because I doubt that dramatic changes in quality are going to happen. Such tests at least show the direction of the development. If you have a look at Hydrogenaudio, you will see that there are a lot of people who are interested to see how the new WMA codec performs. And as I mentioned already, having the results can also help MS to improve their codec. If it wasn't the large crows to scream for tabbed browsing and RSS feeds, I doubt MS would've implemented it in IE 7. You say that if I want to help MS, I should send the results. I can do that, but what is so wrong if others also know what MS has / is been working on? Like I said before, the thing that upsets is most is not that I am no allowed to publish the results, but that you are given some arguments that suddenly don't apply when Microsoft is publishing them. "WMA is great, but dare you try to test yourself or we will sue you." What kind of attitude is that?
I understand that MS employers are busy, but several months passed already and I am not asking for a lot.
BTW, why do I always get to see "An error occurred while sending your post" after posting? Happens with both Firefox 1.5 and IE 7 Beta.
"zachd [MSFT]" wrote:

"Sebastian Mares" wrote in message I think that with this attitude towards audio enthusiasts and possible future customers Microsoft is doing more harm than by allowing me to publish the results of the test. Such test are not made to harm a company - they can be very helpful. LAME and Nero got tuned because of such listening tests.
(said as a friend with respect...)
You actually undercut your argument there - LAME and Nero can tune their code because they got data. There is no value to LAME or Nero to have results of beta code tests published. So if your argument is that you want to improve the WMA codecs, you would simply supply that data to the Microsoft contacts you're dealing/communicating with. There's no particular **unique** value in what you're doing. It's interesting, but it's just as interesting - if not much more - when the released / final version of the codecs come out.
An MVP requested I redirect you - I don't do that kind of thing ever (I need the bridges I have and cannot overtax those), but I pointed out that Zambelli hangs out on the doom9 forums and Amir hangs out on the AVSForums .... forums.
I do not believe you will find any value or response on your quest from any non-SDK-specific newsgroups- I simply don't believe that anyone you want to talk to pays much/any attention to these newsgroups. And even then, Doom9 or AVSForums probably are light years better choices.
Fiiiiinnnnnnaalllllly: bear in mind that everyone you're dealing with is really busy. I respect this may be a top project for you, but those cats you're attempting to get in touch with are juggling a lot of things as Vista moves on closer to release. They're all good people, but I work at Microsoft and know that I've got cut them slack when I deal with because they've got so very much to do on their place. *shrug* - that's just a little insight I thought might be helpful. I've got mails going back months that I haven't had time to deal with. It's there and I'll get to 'em in time, but -- there's a lot of priorities to deal with. =)
-Zach -- Windows Media Development Team (speaking for myself only) See http://zachd.com/pss/pss.html for some helpful WMP info. This posting is provided "AS IS" with no warranties, and confers no rights.

I just read my initial post again and would like to apologize for my words - it really wasn't clear who is actually testing. By reading the first post, one would think that I am doing the testing, but I am not. The test is done by a lot of people (usually 20 to 30, sometimes more, sometimes less). Then all results are collected and evaluated using various tools for analysis (result validation, ANOVA analysis...).
"hel@40th.com" wrote:

he-aac has lots of distortion. If you need others to hear this for you, you shouldn't be conducting the test. hah You wouldn't expect a guy with glasses that magnify his eyes several times to conduct a sight test, would you? I think you should step down and let someone that has decent hearing do this (if it must be done).
Just kidding guy, but he-aac does sound very bad. I know you nero-fans over on ha think it's great to see how small you can get your lossy files, but you have to draw the line when distortion becomes so obvious. he-aac, at least from what I've heard out of nero (which frankly, never works the same way between releases, and never seems to get better, only different), is a waste. Maybe for those that are hearing impaired? Sure. Just like I don't expect a leagally-blind guy to care if a jpeg is high-quality or not, I don't expect the hearing-impaired guy to care about the quality of his music files.
SM- [Thu, 8 Jun 2006 23:53:02 -0700]: In that case, I would love to have you in my listening test. Obviously, you have no idea what you're talking about and most likely come to this conclusion based on experience with MP3 or WMA Standard.
Obviously.
-- 40th Floor - Software @ http://40th.com/ iPlay : the ultimate audio player for mobiles parametric eq, xfeed, reverb; all on a mobile

"Sebastian Mares" wrote in message

Hey Zach!

Howdy Sebastian. =)

I also talked to Zambelli and he is doing his best to get in touch with someone from the licensing department or whatever. I also saw his posts on Doom9 (actually, Ivan from Nero told me about his reply on D9). I didn't visit AVSForums, though.

Yeah, Zambelli is a great guy. I figured he'd be a much better person to talk to here.

Anyways, your argument about testing beta software... If you say that it's so useless, how come Microsoft has no problem publishing the results of the HE-AAC vs. WMA test? Why is it suddenly unimportant that beta software was used?

I would never really take a position on any of the above. I'm interested in the technical angles of issues, but find debating the "why" to generally not be an interesting way to spend my time.
Abstractly, it makes sense to me that a company would say "Hey, this is where our beta is at!" and then wait to let everybody really break the codec down come RTM. If you saw some of the stupid things that linger for years after 3rd parties write random things about betas, it would drive you crazy. It makes my life more difficult, at the least.
But again: this isn't an area that I care about, know about, nor have an emotional or intellectual investment in. =)

I understand that MS employers are busy, but several months passed already and I am not asking for a lot.

You would be really surprised. You should talk to some of my friends, who are pretty much ready to never talk to me again. =P Let alone work - if you're not key to what they have to accomplish at work, you will probably be put on the back burner. Given that Vista is right around the corner, you're probably pretty low on the priority list. After all, in a short while you don't even need to worry about this. =)

BTW, why do I always get to see "An error occurred while sending your post" after posting? Happens with both Firefox 1.5 and IE 7 Beta.

No idea. I don't use the web newsreader - I think that's an abomination. =)
-Zach -- Windows Media Development Team (speaking for myself only) See http://zachd.com/pss/pss.html for some helpful WMP info. This posting is provided "AS IS" with no warranties, and confers no rights.

Here are the facts:
- He-aac is low quality, distotion is easily heard
That's all one needs to know.
But, since you main goal is to get some visibility, I'll oblige.
Three weeks and still no one to do your test? What's new there? People that care about the quality are not going to bother -- they already know what sounds good (and don't use two-dollar ear pods, either, which is about all I ever see those taking these HA tests ever seem to have).
48 kbps he-aac or whatever is frankly, pathetic. Why would you even bother? It's like seeing how small you can get your jpegs before it looks like this morning's breakfast, hurled!
Good luck with your test. You should have stopped at 128 and gone home. haha
-- 40th Floor - Software @ http://40th.com/ iPlay : the ultimate audio player for mobiles parametric eq, xfeed, reverb; all on a mobile

GIGO
SM- [Fri, 9 Jun 2006 15:56:01 -0700]:

all results are collected and evaluated using various tools for analysis (result validation, ANOVA analysis...).

-- 40th Floor - Software @ http://40th.com/ iPlay : the ultimate audio player for mobiles parametric eq, xfeed, reverb; all on a mobile

It contains arifacts, yes. MP3 at 320 kbps also contains artifacts. As I said, it all comes down to how many people hear those artifacts and how annoying they are. If you want to do audio streaming or want to listening to your tracks on a mobile phone in a noisy environment, such as train stations or whatever, I doubt you want to have FLAC, WavPack or any other lossless files with you. I think you are familiar with my other test since you mentioned it. As you can see, at 128 kbps there is no winner because all codecs sound more or less the same. At least judging by what the mass said. This guruboolez on the other hand was able to tell apart various codecs. Heck, he was also able to recognize Vorbis blindly because of the typical artifacts it produces. For me, even at 112 or 96 kbps all sound the same. The point of this test is to find out if 48 kbps really produce some crap-sounding files or if the quality is good enough for streaming and / or mobile use. Of course nobody at home would plug his 48 kbps tracks into his Onkyo or whatever stereo and listen to them in a "music room".
"hel@40th.com" wrote:

Here are the facts:
- He-aac is low quality, distotion is easily heard
That's all one needs to know.
But, since you main goal is to get some visibility, I'll oblige.
Three weeks and still no one to do your test? What's new there? People that care about the quality are not going to bother -- they already know what sounds good (and don't use two-dollar ear pods, either, which is about all I ever see those taking these HA tests ever seem to have).
48 kbps he-aac or whatever is frankly, pathetic. Why would you even bother? It's like seeing how small you can get your jpegs before it looks like this morning's breakfast, hurled!
Good luck with your test. You should have stopped at 128 and gone home. haha
-- 40th Floor - Software @ http://40th.com/ iPlay : the ultimate audio player for mobiles parametric eq, xfeed, reverb; all on a mobile

I would never really take a position on any of the above. I'm interested in the technical angles of issues, but find debating the "why" to generally not be an interesting way to spend my time.

Well, me too. However, I find it unfair that one set of rules apply to users and another set of rules apply to developers or publishers. If Microsoft thinks that its beta software is not "mature" enough for tests, I don't think they should publish results of a test ran with exactly that software either. It's natural that people want to have a proof, especially since the winner comes from the own company.
"zachd [MSFT]" wrote:

"Sebastian Mares" wrote in message Hey Zach!
Howdy Sebastian. =)
I also talked to Zambelli and he is doing his best to get in touch with someone from the licensing department or whatever. I also saw his posts on Doom9 (actually, Ivan from Nero told me about his reply on D9). I didn't visit AVSForums, though.
Yeah, Zambelli is a great guy. I figured he'd be a much better person to talk to here.
Anyways, your argument about testing beta software... If you say that it's so useless, how come Microsoft has no problem publishing the results of the HE-AAC vs. WMA test? Why is it suddenly unimportant that beta software was used?
I would never really take a position on any of the above. I'm interested in the technical angles of issues, but find debating the "why" to generally not be an interesting way to spend my time.
Abstractly, it makes sense to me that a company would say "Hey, this is where our beta is at!" and then wait to let everybody really break the codec down come RTM. If you saw some of the stupid things that linger for years after 3rd parties write random things about betas, it would drive you crazy. It makes my life more difficult, at the least.
But again: this isn't an area that I care about, know about, nor have an emotional or intellectual investment in. =)
I understand that MS employers are busy, but several months passed already and I am not asking for a lot.
You would be really surprised. You should talk to some of my friends, who are pretty much ready to never talk to me again. =P Let alone work - if you're not key to what they have to accomplish at work, you will probably be put on the back burner. Given that Vista is right around the corner, you're probably pretty low on the priority list. After all, in a short while you don't even need to worry about this. =)
BTW, why do I always get to see "An error occurred while sending your post" after posting? Happens with both Firefox 1.5 and IE 7 Beta.
No idea. I don't use the web newsreader - I think that's an abomination. =)
-Zach -- Windows Media Development Team (speaking for myself only) See http://zachd.com/pss/pss.html for some helpful WMP info. This posting is provided "AS IS" with no warranties, and confers no rights.