RMS error vs. Maximum Absolute error

[Paul H. Erlich]
>I use RMS because if two tunings have the same worst error, the
second->worst errors should allow one to judge one tuning better than the
other.
>RMS still makes the worst errors more important than the second-worst
>errors, while MAD (mean absolute deviation) puts equal weight on all
>errors.

I certainly would not consider MAD, and agree that RMS is better than MAD,
but if two tunings have the same worst error, I *do* just use the
second-worst errors to compare them. e.g. as I did in comparing certain
tunings having the half-octave as their approximation to 5:7. There is no
need to go to RMS.

The problem with RMS is that, even though it is not as bad in this regard
as MAD, it may still consider a tuning that has most ratios rendered very
accurately but one rendered badly, to be as good as another tuning where
all the errors are moderate. This is not how humans (me at least) perceive
these things.

I suspect RMS gives a very broad-bottomed parabola when applied to the
meantone spectrum, whatever the odd-limit being considered, and whatever
the weighting.

Regards,
-- Dave Keenan
http://dkeenan.com

Dave Keenan wrote,

>I certainly would not consider MAD, and agree that RMS is better than MAD,
>but if two tunings have the same worst error, I *do* just use the
>second-worst errors to compare them. e.g. as I did in comparing certain
>tunings having the half-octave as their approximation to 5:7. There is no
>need to go to RMS.

I don't like the discontinuity of only consideriing the second-worst errors
if the worst errors happen to exactly match up; the tiniest difference in
the worst errors makes the importance of the second-worst errors suddenly
drop to zero.

>The problem with RMS is that, even though it is not as bad in this regard
>as MAD, it may still consider a tuning that has most ratios rendered very
>accurately but one rendered badly, to be as good as another tuning where
>all the errors are moderate.

The former case can't happen, at least for tunings consistent in a given odd
limit. Since each interval can be constructed by stacking two others, if
those two are rendered very accurately, the original interval can't be
rendered that badly.

P.S. The minima in the harmonic entropy graph are roughly parabolic. That is
why squared error seems appropriate for each interval.

>Message: 8
> Date: Thu, 15 Apr 1999 13:20:57 -0400
> From: "Paul H. Erlich" <PErlich@Acadian-Asset.com>
>Subject: Re: RMS error vs. Maximum Absolute error
>
>Dave Keenan wrote,
>
>>I certainly would not consider MAD, and agree that RMS is better than MAD,
>>but if two tunings have the same worst error, I *do* just use the
>>second-worst errors to compare them. e.g. as I did in comparing certain
>>tunings having the half-octave as their approximation to 5:7. There is no
>>need to go to RMS.

[Paul Erlich]
>I don't like the discontinuity of only consideriing the second-worst errors
>if the worst errors happen to exactly match up; the tiniest difference in
>the worst errors makes the importance of the second-worst errors suddenly
>drop to zero.

Good point.

>>The problem with RMS is that, even though it is not as bad in this regard
>>as MAD, it may still consider a tuning that has most ratios rendered very
>>accurately but one rendered badly, to be as good as another tuning where
>>all the errors are moderate.
>
>The former case can't happen, at least for tunings consistent in a given odd
>limit. Since each interval can be constructed by stacking two others, if
>those two are rendered very accurately, the original interval can't be
>rendered that badly.

Another good point, however my objection still stands, albeit with less
extreme examples.

>P.S. The minima in the harmonic entropy graph are roughly parabolic. That
>is why squared error seems appropriate for each interval.

Now this is an excellent point. But just how parabolic are they? And if
they are, are those of Sethares' curves? They certaintly aren't V-shaped as
max absolute would have it. So I'm thinking maybe they are 4th power curves
or 6th power (or any positive power (greater than 2) of the absolute
deviation) or maybe they are the exponential of the absolute deviation
exp(|x|), or inverse Gaussian exp(x^2). These last two seem more likely in
Sethares and your (HE) cases respectively.

These all give higher weighting to the largest error than RMS does but
don't suffer from the excessive sensitivity to it that Max-Abs has.
Whadyathink?

Regards,
-- Dave Keenan
http://dkeenan.com

Dave Keenan wrote:

> >>The problem with RMS is that, even though it is not as bad in this regard
> >>as MAD, it may still consider a tuning that has most ratios rendered very
> >>accurately but one rendered badly, to be as good as another tuning where
> >>all the errors are moderate.
> >
> >The former case can't happen, at least for tunings consistent in a given odd
> >limit. Since each interval can be constructed by stacking two others, if
> >those two are rendered very accurately, the original interval can't be
> >rendered that badly.
>
> Another good point, however my objection still stands, albeit with less
> extreme examples.

Such as?

> >P.S. The minima in the harmonic entropy graph are roughly parabolic. That
> >is why squared error seems appropriate for each interval.
>
> Now this is an excellent point. But just how parabolic are they? And if
> they are, are those of Sethares' curves? They certaintly aren't V-shaped as
> max absolute would have it.

Actually, the Plomp-with-harmonic-overtones dissonance curves I've seen _are_ V-shaped around the simple
ratios. Don't you have spreadsheets on you webpage to produce these curves? (I tried downloading them but my
version of Excel didn't recognize them.)

> So I'm thinking maybe they are 4th power curves
> or 6th power (or any positive power (greater than 2) of the absolute
> deviation)

These curves are too flat -- having a zero second derivative, they look much more like a mesa than anything
in the HE curves.

> or maybe they are the exponential of the absolute deviation
> exp(|x|)

That's pointy and would suggest something like MAD.

> or inverse Gaussian exp(x^2).

In my paper (http://www-math.cudenver.edu/~jstarret/22ALL.pdf), accuracy is in fact calculated using a
Gaussian for each interval, and using the geometric mean of the accuracies so that even one really bad
interval makes the whole tuning look bad (a quality that you express a desire for above). That turns out to
be related monotonically to straight RMS, so results in the same solutions for optimization problems as RMS.
All this is mentioned in my paper, where I also gave the size of the fifth of the optimal meantone in terms
of 26ths of an octave but failed to realize that it was 7/26-comma meantone.

> These last two seem more likely in
> Sethares and your (HE) cases respectively.
>
> These all give higher weighting to the largest error than RMS does

These last two don't necessarily do so.

> > These last two seem more likely in
> > Sethares and your (HE) cases respectively.
> >
> > These all give higher weighting to the largest error than RMS does
>
> These last two don't necessarily do so.

Whoops, your "inverse Gaussian" does do so, but is way too flat to approximate the dissonance curves.