Re: Performance on 8CPU's and 32GB of RAM

On 9/5/07, Scott Marlowe <scott.marlowe@gmail.com> wrote:
> On 9/5/07, Trevor Talbot <quension@gmail.com> wrote:
> > On 9/5/07, Scott Marlowe <scott.marlowe@gmail.com> wrote:
> > > On 9/5/07, Carlo Stonebanks <stonec.register@sympatico.ca> wrote:
> > > > > Right, additionally NTFS is really nothing to use on any serious disc
> > > > > array.
> > > >
> > > > Do you mean that I will not see any big improvement if I upgrade the disk
> > > > subsystem because the client is using NTFS (i.e. Windows)
> > >
> > > No, I think he's referring more to the lack of reliability of NTFS
> > > compared to UFS / ZFS / JFS / XFS on unixen.
> >
> > Lack of reliability compared to _UFS_?  Can you elaborate on this?

> Not a lot.  Back when I was an NT 4.0 sysadmin, I had many many
> occasions where NTFS simply corrupted for no apparent reason.  No
> system crash, no obvious problems with the drive, and bang suddenly a
> file goes corrupted.  About that time I gave up on Windows and started
> supporting Linux and Solaris.  Neither is perfect, but I've never had
> either of them just corrupt a file on good hardware for no reason.

Anecdotal then.  That's fine, but needs to be qualified as such, not
presented as a general case that everyone with experience agrees is
true.

I mean, I've got a box running OpenBSD UFS that's lost files on me,
while my NTFS boxes have been fine other than catastrophic drive
failure.  But that anecdote doesn't actually mean anything, since it's
useless in the general case.  (The issues on that one UFS box have a
known cause anyway, related to power failures.)

> With the newer journalling file systems on linux, solaris and BSD, you
> get both good performance and very reliable behaviour.  Maybe NTFS has
> gotten better since then, but I don't personally know.

The thing is, most UFS implementations I'm familiar with don't
journal; that's what prompted my question in the first place, since I
figured you were thinking along those lines.  NTFS is
metadata-journaling, like most of the others, and has continued to
improve over time.

I took the original comment to be about performance, actually.  NTFS's
journaling method tends to get bashed in that department compared to
some of the more modern filesystems.  I don't have any experience with
intensive I/O on large arrays to know.

Hopefully he'll clarify what he meant.

> Oh, the other issue that NTFS still seems to suffer from that most
> unix file systems have overcome is fragmentation.  Since you can't
> defrag a live system, you have to plan time to take down the db should
> the NTFS partition for your db get overly fragmented.

Live defragmentation has been supported since NT4, although Microsoft
never included tools or publicly documented it until 2000.  The NTFS
implementation in Windows doesn't make much effort to avoid
fragmentation, but that varies among implementations of the other
filesystems too.  Modern ones tend to be better at it.

> And there's the issue that with windows / NTFS that when one process
> opens a file for read, it locks it for all other users.  This means
> that things like virus scanners can cause odd, unpredictable failures
> of your database.

It's simply a Windows platform default for file I/O; there's no hard
limitation there, and it's not about a particular filesystem.  In the
case of antivirus vs database, it's more of an administrative issue:
configure the AV to ignore the database files, harass the AV vendor to
get programmers with clue, find another AV vendor, or just don't run
AV on your dedicated database server.