livephish.com switches to FLAC!

Livephish.com switched their lossless format from shorten
to FLAC!!

And there are a surprising number of FLAC tools now
available...

http://flac.sourceforge.net/download.html

...including Cool Edit filters, front ends, the works!
The FLACdrop link is bad, but it's available here...

http://rarewares.hydrogenaudio.org/lossless.html

Here's a great comparison of lossless codecs. It shows
FLAC clearly on top...

http://web.inter.nl.net/users/hvdh/lossless/lossless.htm

Things are happening faster than I had hoped!

7/9 at the Shoreline was my first Phish show, btw. It
was awesome, though they were stretching the tunes too
much, and I liked Mike's old, simple sound much better
than the funky thing, and Page was so low as to be almost
inaudible, and the boys looked visably exhausted. Also
the place was covered with trash afterwards. Geez. Still,
one of the most amazing concerts I've ever heard.

-Carl

--- In metatuning@yahoogroups.com, "Carl Lumma" <clumma@y...> wrote:
> Livephish.com switched their lossless format from shorten
> to FLAC!!

How much compression can one get using a lossless format?

> How much compression can one get using a lossless format?

Please follow the links I painstakingly provide before asking
stuff like this.

-Carl

>> How much compression can one get using a lossless format?
>
> Please follow the links I painstakingly provide before
> asking stuff like this.

Ok, sorry I snapped. Interestingly, only very slightly
better than general-purpose entropy coders, which is to
say quite bad. Figuring out why time-domain stuff is so
hard would be quite important. It certainly seems like
more compression should be possible. There's potentially
a huge hole in information theory here.

-Carl

> >> How much compression can one get using a lossless
> >> format?
> >
> > Please follow the links I painstakingly provide before
> > asking stuff like this.
>
> Ok, sorry I snapped. Interestingly, only very slightly
> better than general-purpose entropy coders, which is to
> say quite bad. Figuring out why time-domain stuff is so
> hard would be quite important. It certainly seems like
> more compression should be possible. There's potentially
> a huge hole in information theory here.

Just now wondered if converting PCM to delta-sigma (ie Sony
SACD) and simply doing run-length encoding wouldn't produce
better results. Quick google shows that SACD apparently
uses RLE, and gets the same as the best predictive coders
(ie FLAC) do with PCM -- about 2:1.

I wonder if there isn't a flaw in my reasoning here.
Anything humans enjoy should be compressable, but I guess
the only thing that really gives more than 2:1 is text, so
maybe audio isn't mysteriously hard...

-Carl

--- In metatuning@yahoogroups.com, "Carl Lumma" <clumma@y...> wrote:
> >> How much compression can one get using a lossless format?
>
> Ok, sorry I snapped.

We're all doing it these days.

> Interestingly, only very slightly better than general-purpose
> entropy coders, which is to say quite bad.

I'm not sure what is realistic to expect in a lossless compression scheme. But you got me excited, so I got all the flac stuff. A one-minute extracted file (opening of "Red Rain" by P. Gabriel, with lots of quick high freqs that show up munged in bad compression):

wav = 10.5 mb
flac = 6.7
mp3 = 2.4
ogg = 2.2

(last two codecs set highly, for 320 [vbr])

Pretty interesting. The deal is, with the two lossy codecs set that high it is difficult for me to hear a diff between those and the other files. Quick and dirty test, with only the computer speakers; I'll be interested to do some listening on a real system and with good cans as well.

Thanks for posting the info, Carl. I'm not sure where I'd go with it at this point, but always good to know.

Cheers,
Jon

----- Original Message -----
From: "Carl Lumma" <clumma@...>

> > How much compression can one get using a lossless format?
>
> Please follow the links I painstakingly provide before asking
> stuff like this.

Painstaking?

Are you typing with one finger or less?

* David Beardsley
* microtonal guitar
* http://biink.com/db

> wav = 10.5 mb
> flac = 6.7
> mp3 = 2.4
> ogg = 2.2
>
> (last two codecs set highly, for 320 [vbr])
>
> Pretty interesting. The deal is, with the two lossy codecs
> set that high it is difficult for me to hear a diff between
> those and the other files. Quick and dirty test, with only
> the computer speakers; I'll be interested to do some
> listening on a real system and with good cans as well.

The very best audiophiles listening on the very best
equipment available rate 256Kbps mp3 as being truly
cd quality. That means they might hear a difference, but
they can't reliably tell which is cd and which is mp3.

But for archival purposes, mastering, remastering, you don't
want loss piling up with every edit. And who knows when you
might need every bit for some kind of analysis?

-Carl

----- Original Message -----
From: "Carl Lumma" <clumma@...>

> > > Please follow the links I painstakingly provide before asking
> > > stuff like this.
> >
> > Painstaking?
> >
> > Are you typing with one finger or less?
>
> Care to guess where the other finger is, Beardsley?

So you are typing with one finger?
Where is it? Can we count thumbs as fingers?

* David Beardsley
* microtonal guitar
* http://biink.com/db

Carl,

Thanks again for the pointers on FLAC. FlacDrop works so well; I downloaded the entire FLAC as a zip, so I'm not sure what the minimum file installation it, but once I get that figured out I'm going to include all the parts in one zip file and send it to some friends/colleagues. One use I can see is that if people really do want to do collaborative recording, you could flac your wav files, send the, have them unflac, do the work, flac it back up and mail it back, etc.

I realize it isn't even 2:1 compression, but if one wants to work on wav files this at least helps in the upload/download scenario.

Cheers,
Jon

Man, FLAC is sooo cool. I just saved almost 3 gigs of hard
drive space by compressing my archived masters! And you
get checksum protection against your hard drive flipping bits
on you.

Note that FLAC supports the awesome new Replaygain standard.

>Thanks again for the pointers on FLAC. FlacDrop works so
>well; I downloaded the entire FLAC as a zip, so I'm not
>sure what the minimum file installation it, but once I get
>that figured out I'm going to include all the parts in one
>zip file and send it to some friends/colleagues.

Why not just let them download the entire distro?

I'm using the FLAC frontend, and it works great too.

>One use I can see is that if people really do want to do
>collaborative recording, you could flac your wav files,
>send the, have them unflac, do the work, flac it back up
>and mail it back, etc.

Yep, that's another great use. Try it with mp3, and you'll
end up with marshmallow goo.

>I realize it isn't even 2:1 compression,

It's slightly better than 2:1 with the default value of 5.
If you look at the graphs at the comparo site I linked to,
though, it looks like 4 is a better setting, at least for
the test corpus he used. I used 4 and still got better
than 2:1.

-Carl

> Note that FLAC supports the awesome new Replaygain standard.

http://www.replaygain.org/

I didn't use it (not sure any players support it yet), but it's
what I've been dreaming of for a long time. I suggested
something similiar be implemented in Audio Compositor, but I'm
not sure Scott was convinced.

> It's slightly better than 2:1 with the default value of 5.
> If you look at the graphs at the comparo site I linked to,
> though, it looks like 4 is a better setting, at least for
> the test corpus he used. I used 4 and still got better
> than 2:1.

At the setting I used, btw, it encodes much faster than mp3,
and (I think) takes fewer cycles to play.

-Carl

C,

--- In metatuning@yahoogroups.com, "Carl Lumma" <clumma@y...> wrote:
> Man, FLAC is sooo cool. I just saved almost 3 gigs of hard
> drive space by compressing my archived masters! And you
> get checksum protection against your hard drive flipping bits
> on you.

Um, yeah, but you *DO* have backups, right? Or would you put your critical archives in the hands of a new format/compression/tool?

> Why not just let them download the entire distro?

Like normal end users need all the source code, compilation documentation, etc.? I don't think so. All normal people need are apps and docs to run em. Everything else can be left to us geeks.

> I'm using the FLAC frontend, and it works great too.

I'll try that for more tweaking.

> Yep, that's another great use. Try it with mp3, and you'll
> end up with marshmallow goo.

Well, no, I'd never do that. Just as I wouldn't work collaboratively on a graphics project and save in .jpg.

> >I realize it isn't even 2:1 compression,
>
> It's slightly better than 2:1 with the default value of 5.

I used it straight outta the box, which must mean default, and I didn't get 2:1. Otherwise I would have said so!

Cheers,
Jon

>Um, yeah, but you *DO* have backups, right? Or would
>you put your critical archives in the hands of a new
>format/compression/tool?

I've got backups.

> > Why not just let them download the entire distro?
>
> Like normal end users need all the source code, compilation
>documentation, etc.? I don't think so. All normal people need
>are apps and docs to run em. Everything else can be left to
>us geeks.

The thing I downloaded was just binaries -- flac, the
frontend, and flacdrop, complete with nifty installer.

-Carl

--- In metatuning@yahoogroups.com, "Carl Lumma" <clumma@y...> wrote:
> The thing I downloaded was just binaries -- flac, the
> frontend, and flacdrop, complete with nifty installer.

Frequently, the choice between a zip file and exe file (installer) distribution is the convenience of an executable installer, and the actual file collection is (most often) the same. A lot of times I download the archive, which is often smaller.

In this case, however, it appears the 'installation' one is the one I should have chosen, as the 'full' archive file did, indeed, give me more than I needed.

I'll point people to the installer.

Cheers,
Jon

>Um, yeah, but you *DO* have backups, right? Or would you put
>your critical archives in the hands of a new
>format/compression/tool?

Things that help calm my fears...

() it's under xiph's loving care.
() livephish is using it.
() etree folks are using it.
() the neuros hardware mp3 player may soon support it.

() The encoder has a verify mode which runs a decode thread
in parallel with the encoder, and makes sure the thing match,
bit for bit. Show me an archiver that'll do that. Even
with this switched on it's faster than rar or mp3 encoding.

-Carl

C,

I didn't mean to imply you did something stooopid, but anyone remember the Internet bubble? Technology, especially the newest and shiniest, would always save us and be the best thing since the last best thing. So...

--- In metatuning@yahoogroups.com, "Carl Lumma" <clumma@y...> wrote:
>
> Things that help calm my fears...
>
> () it's under xiph's loving care.

How long has xiph been around? As long as analog tape? (these are all rhetorical, no answers needed...)

> () livephish is using it.

Only recently, and they don't necessarily have to worry about the un-compression.

> () etree folks are using it.

I'm stumped (bad humor indicating that means nothing to me)

> () the neuros hardware mp3 player may soon support it.

"May" means absolutely *nothing* in my book. "Does, and has for a long time" *does* mean something.

> () The encoder has a verify mode which runs a decode thread
> in parallel with the encoder, and makes sure the thing match,
> bit for bit. Show me an archiver that'll do that. Even
> with this switched on it's faster than rar or mp3 encoding.

Well, all my caveats above, so far I'm sold as well. And you're right - I can't remember a compressor that worked that fast! I haven't mucked with it yet, but I'm going to try some command line work to do batch flacing and unflacing

Two words I'm growing to love...

Cheers,
Jon