Обсуждение: [HACKERS] 6.1 progress
It is amazing how much talent has been focused in just the past 24 hours on PostgreSQL. Amazing. I am excited about the memory fixes and the cleanup going on. I hope Marc is going to give us the time to complete these fixes and post-fix testing before a 6.1 final release. - -- Bruce Momjian maillist@candle.pha.pa.us ------------------------------
On Mon, 2 Jun 1997, Bruce Momjian wrote:
> It is amazing how much talent has been focused in just the past 24 hours
> on PostgreSQL. Amazing.
>
> I am excited about the memory fixes and the cleanup going on.
>
> I hope Marc is going to give us the time to complete these fixes and
> post-fix testing before a 6.1 final release.
Geez, sooooo much weight on my shoulders *grin*
Might I venture to propose an additional week? *grin*
Marc G. Fournier
Systems Administrator @ hub.org
primary: scrappy@hub.org secondary: scrappy@{freebsd|postgresql}.org
------------------------------
> > On Mon, 2 Jun 1997, Bruce Momjian wrote: > > > It is amazing how much talent has been focused in just the past 24 hours > > on PostgreSQL. Amazing. > > > > I am excited about the memory fixes and the cleanup going on. > > > > I hope Marc is going to give us the time to complete these fixes and > > post-fix testing before a 6.1 final release. > > Geez, sooooo much weight on my shoulders *grin* > > Might I venture to propose an additional week? *grin* I just lobbed it out there. I was waiting to hear your reaction to this. Sounds good to me. I don't want too much pressure on Igor. Doing this kind of cleanup is a delicate job, and needs to be done carefully. - -- Bruce Momjian maillist@candle.pha.pa.us ------------------------------
Oh the pressure :)...Anyways, PSQL has a huge leak (90% reported by
Purify, whatever that means :) ) in reading in external sql files (through
- -f option)...this might be contributing to recent reports of memory leaks
during regression tests (aside from the backend which is another issue)...
I am working on it right now, but I don't think PSQL issue is important
enough to delay the release....if everything checks out allright by Friday
then we should release it...Then starting with 6.2 make sure we eliminate
as many memory problems as possible....
Oh well, just a thought....
BTW...I will be out of town next week, then when I come back it's Jury
duty...so I'll try to finish psql/libpq by this week anyway....
=+=------------------------/\---------------------------------=+=
Igor Natanzon |**| E-mail: igor@sba.miami.edu
=+=------------------------\/---------------------------------=+=
On Mon, 2 Jun 1997, Bruce Momjian wrote:
> >
> > On Mon, 2 Jun 1997, Bruce Momjian wrote:
> >
> > > It is amazing how much talent has been focused in just the past 24 hours
> > > on PostgreSQL. Amazing.
> > >
> > > I am excited about the memory fixes and the cleanup going on.
> > >
> > > I hope Marc is going to give us the time to complete these fixes and
> > > post-fix testing before a 6.1 final release.
> >
> > Geez, sooooo much weight on my shoulders *grin*
> >
> > Might I venture to propose an additional week? *grin*
>
> I just lobbed it out there. I was waiting to hear your reaction to
> this.
>
> Sounds good to me. I don't want too much pressure on Igor. Doing this
> kind of cleanup is a delicate job, and needs to be done carefully.
>
> --
> Bruce Momjian
> maillist@candle.pha.pa.us
>
------------------------------
> > The Hermit Hacker wrote: > > > > On Mon, 2 Jun 1997, Bruce Momjian wrote: > > > > > It is amazing how much talent has been focused in just the past 24 hours > > > on PostgreSQL. Amazing. > > > > > > I am excited about the memory fixes and the cleanup going on. > > > > > > I hope Marc is going to give us the time to complete these fixes and > > > post-fix testing before a 6.1 final release. > > > > Geez, sooooo much weight on my shoulders *grin* > > > > Might I venture to propose an additional week? *grin* > > What! Once again! What are the opened issues ? Really none, but Igor is tracking down the memory leak problems people have reported, and I was not sure he would finish in time, and even if he did, we would not have enough time to test it. Do you have another recommendation? - -- Bruce Momjian maillist@candle.pha.pa.us ------------------------------
The Hermit Hacker wrote: > > On Mon, 2 Jun 1997, Bruce Momjian wrote: > > > It is amazing how much talent has been focused in just the past 24 hours > > on PostgreSQL. Amazing. > > > > I am excited about the memory fixes and the cleanup going on. > > > > I hope Marc is going to give us the time to complete these fixes and > > post-fix testing before a 6.1 final release. > > Geez, sooooo much weight on my shoulders *grin* > > Might I venture to propose an additional week? *grin* What! Once again! What are the opened issues ? Vadim ------------------------------
> > Oh the pressure :)...Anyways, PSQL has a huge leak (90% reported by > Purify, whatever that means :) ) in reading in external sql files (through > -f option)...this might be contributing to recent reports of memory leaks > during regression tests (aside from the backend which is another issue)... > > I am working on it right now, but I don't think PSQL issue is important > enough to delay the release....if everything checks out allright by Friday > then we should release it...Then starting with 6.2 make sure we eliminate > as many memory problems as possible.... > > Oh well, just a thought.... > > BTW...I will be out of town next week, then when I come back it's Jury > duty...so I'll try to finish psql/libpq by this week anyway.... OK, let's see where we are at the end of the week. Take it as far as you can. I am curious what you see in the backend. People are reporting more memory problems than in 6.1. It may be psql, or libpq, or it may be a few things that we added to the backend that are causing problems. When people are running the regression tests multiple times and seeing errors, how are you doing that? Doesn't the backend and psql restart on every iteration? Is it the postmaster that is growing too big? - -- Bruce Momjian maillist@candle.pha.pa.us ------------------------------
On Mon, 2 Jun 1997, Bruce Momjian wrote:
> >
> > On Mon, 2 Jun 1997, Bruce Momjian wrote:
> >
> > > It is amazing how much talent has been focused in just the past 24 hours
> > > on PostgreSQL. Amazing.
> > >
> > > I am excited about the memory fixes and the cleanup going on.
> > >
> > > I hope Marc is going to give us the time to complete these fixes and
> > > post-fix testing before a 6.1 final release.
> >
> > Geez, sooooo much weight on my shoulders *grin*
> >
> > Might I venture to propose an additional week? *grin*
>
> I just lobbed it out there. I was waiting to hear your reaction to
> this.
I figured it was a subtle hint *rofl*
> Sounds good to me. I don't want too much pressure on Igor. Doing this
> kind of cleanup is a delicate job, and needs to be done carefully.
Whatever happened to the guy that originally brought up the whole
Purify issue though? Why is Igor doing all the work on it? :(
Marc G. Fournier
Systems Administrator @ hub.org
primary: scrappy@hub.org secondary: scrappy@{freebsd|postgresql}.org
------------------------------
On Mon, 2 Jun 1997, Bruce Momjian wrote:
> >
> > The Hermit Hacker wrote:
> > >
> > > On Mon, 2 Jun 1997, Bruce Momjian wrote:
> > >
> > > > It is amazing how much talent has been focused in just the past 24 hours
> > > > on PostgreSQL. Amazing.
> > > >
> > > > I am excited about the memory fixes and the cleanup going on.
> > > >
> > > > I hope Marc is going to give us the time to complete these fixes and
> > > > post-fix testing before a 6.1 final release.
> > >
> > > Geez, sooooo much weight on my shoulders *grin*
> > >
> > > Might I venture to propose an additional week? *grin*
> >
> > What! Once again! What are the opened issues ?
>
> Really none, but Igor is tracking down the memory leak problems people
> have reported, and I was not sure he would finish in time, and even if
> he did, we would not have enough time to test it.
>
> Do you have another recommendation?
I was only proposing the additional week because I thought there
was something still open 'non-Purify' related that was to be addressed...
...is this not the case? If its only the Purify stuff that is
being the basis...forget it :) As I stated before, we've been unPure
for, what, 6 releases now? A start has been made, but it is *not* something
that should hold back a release...
...so, unless there are any outstanding issues that should be
resolved, let's continue with the June 6th release...
Marc G. Fournier
Systems Administrator @ hub.org
primary: scrappy@hub.org secondary: scrappy@{freebsd|postgresql}.org
------------------------------
> I was only proposing the additional week because I thought there > was something still open 'non-Purify' related that was to be addressed... > > ...is this not the case? If its only the Purify stuff that is > being the basis...forget it :) As I stated before, we've been unPure > for, what, 6 releases now? A start has been made, but it is *not* something > that should hold back a release... > > ...so, unless there are any outstanding issues that should be > resolved, let's continue with the June 6th release... OK, I agree. I have heard people complaining about excessive memory problems under 6.1. Hopefully, they were inaccurate, or the problem will be found by Friday. - -- Bruce Momjian maillist@candle.pha.pa.us ------------------------------
On Mon, 2 Jun 1997, Bruce Momjian wrote:
> > I was only proposing the additional week because I thought there
> > was something still open 'non-Purify' related that was to be addressed...
> >
> > ...is this not the case? If its only the Purify stuff that is
> > being the basis...forget it :) As I stated before, we've been unPure
> > for, what, 6 releases now? A start has been made, but it is *not* something
> > that should hold back a release...
> >
> > ...so, unless there are any outstanding issues that should be
> > resolved, let's continue with the June 6th release...
>
> OK, I agree. I have heard people complaining about excessive memory
> problems under 6.1. Hopefully, they were inaccurate, or the problem
> will be found by Friday.
Or its problems that we've had all along and the extensive testing
that has gone into this release has caused more ppl to see them? :)
I think that ppl are now actually running and taking to heart what
the regression tests are saying, with all the work that Thomas has put into
it, which has generated *alot* of the bug reports we've seen. As long as
we continue the trend we are following (stronger regression tests, Purification
of the code), v6.2 will be all that much stronger/stabler...but I think
continuing to postpone v6.1 because of Purification tests is going to start
annoying some ppl (me, for starters, but I think Vadim showed his
threshold tonight *grin*)
Marc G. Fournier
Systems Administrator @ hub.org
primary: scrappy@hub.org secondary: scrappy@{freebsd|postgresql}.org
------------------------------
> Or its problems that we've had all along and the extensive testing > that has gone into this release has caused more ppl to see them? :) > > I think that ppl are now actually running and taking to heart what > the regression tests are saying, with all the work that Thomas has put into > it, which has generated *alot* of the bug reports we've seen. As long as > we continue the trend we are following (stronger regression tests, Purification > of the code), v6.2 will be all that much stronger/stabler...but I think > continuing to postpone v6.1 because of Purification tests is going to start > annoying some ppl (me, for starters, but I think Vadim showed his > threshold tonight *grin*) No question about it. - -- Bruce Momjian maillist@candle.pha.pa.us ------------------------------
Bruce Momjian wrote: > > > > > The Hermit Hacker wrote: > > > > > > On Mon, 2 Jun 1997, Bruce Momjian wrote: > > > > > > > It is amazing how much talent has been focused in just the past 24 hours > > > > on PostgreSQL. Amazing. > > > > > > > > I am excited about the memory fixes and the cleanup going on. > > > > > > > > I hope Marc is going to give us the time to complete these fixes and > > > > post-fix testing before a 6.1 final release. > > > > > > Geez, sooooo much weight on my shoulders *grin* > > > > > > Might I venture to propose an additional week? *grin* > > > > What! Once again! What are the opened issues ? > > Really none, but Igor is tracking down the memory leak problems people > have reported, and I was not sure he would finish in time, and even if > he did, we would not have enough time to test it. > > Do you have another recommendation? Igor wrote: > > Oh the pressure :)...Anyways, PSQL has a huge leak (90% reported by > Purify, whatever that means :) ) in reading in external sql files (through > -f option)...this might be contributing to recent reports of memory leaks > during regression tests (aside from the backend which is another issue)... > > I am working on it right now, but I don't think PSQL issue is important > enough to delay the release....if everything checks out allright by Friday > then we should release it...Then starting with 6.2 make sure we eliminate > as many memory problems as possible.... Agreed. Vadim ------------------------------
> > I just lobbed it out there. I was waiting to hear your reaction to > > this. > > I figured it was a subtle hint *rofl* > > > Sounds good to me. I don't want too much pressure on Igor. Doing this > > kind of cleanup is a delicate job, and needs to be done carefully. > > Whatever happened to the guy that originally brought up the whole > Purify issue though? Why is Igor doing all the work on it? :( I thought Igor was the one to bring up the Purify issue. Was it someone else? Was it me? - -- Bruce Momjian maillist@candle.pha.pa.us ------------------------------
I am doing it cause I finally got access to a machine with licensed
Purify...:)
Hmm..I fixed two more memory leaks (bad ones) in psql, but won't post a
patch until I do some testing...Maybe Wednesday...I don't think it would
be wise to fix any more of these problems until after the release...
I will also start working on the backend as soon as I get my new
machine....which will happen in about two weeks...
=+=------------------------/\---------------------------------=+=
Igor Natanzon |**| E-mail: igor@sba.miami.edu
=+=------------------------\/---------------------------------=+=
On Tue, 3 Jun 1997, The Hermit Hacker wrote:
> On Mon, 2 Jun 1997, Bruce Momjian wrote:
>
> > >
> > > On Mon, 2 Jun 1997, Bruce Momjian wrote:
> > >
> > > > It is amazing how much talent has been focused in just the past 24 hours
> > > > on PostgreSQL. Amazing.
> > > >
> > > > I am excited about the memory fixes and the cleanup going on.
> > > >
> > > > I hope Marc is going to give us the time to complete these fixes and
> > > > post-fix testing before a 6.1 final release.
> > >
> > > Geez, sooooo much weight on my shoulders *grin*
> > >
> > > Might I venture to propose an additional week? *grin*
> >
> > I just lobbed it out there. I was waiting to hear your reaction to
> > this.
>
> I figured it was a subtle hint *rofl*
>
> > Sounds good to me. I don't want too much pressure on Igor. Doing this
> > kind of cleanup is a delicate job, and needs to be done carefully.
>
> Whatever happened to the guy that originally brought up the whole
> Purify issue though? Why is Igor doing all the work on it? :(
>
> Marc G. Fournier
> Systems Administrator @ hub.org
> primary: scrappy@hub.org secondary: scrappy@{freebsd|postgresql}.org
>
>
------------------------------
> > I am doing it cause I finally got access to a machine with licensed > Purify...:) > > Hmm..I fixed two more memory leaks (bad ones) in psql, but won't post a > patch until I do some testing...Maybe Wednesday...I don't think it would > be wise to fix any more of these problems until after the release... > > I will also start working on the backend as soon as I get my new > machine....which will happen in about two weeks... That sounds like a plan to me, or an r_plan :-) - -- Bruce Momjian maillist@candle.pha.pa.us ------------------------------
Well, one of the leaks that Purify detected was that individual lines read
from files were malloc'ed but not freed properly. So, if a line is
malloc'ed to about 8K (and we are reading thousands of lines in COPY reg
test I believe) and is not freed before next line is read, we get
significant memory usage... Plus adding all other leaks we get into major
problems...
I fixed the 'lines' leak so we shall see what happens...The backend should
be interesting...
=+=------------------------/\---------------------------------=+=
Igor Natanzon |**| E-mail: igor@sba.miami.edu
=+=------------------------\/---------------------------------=+=
On Mon, 2 Jun 1997, Bruce Momjian wrote:
> >
> > Oh the pressure :)...Anyways, PSQL has a huge leak (90% reported by
> > Purify, whatever that means :) ) in reading in external sql files (through
> > -f option)...this might be contributing to recent reports of memory leaks
> > during regression tests (aside from the backend which is another issue)...
> >
> > I am working on it right now, but I don't think PSQL issue is important
> > enough to delay the release....if everything checks out allright by Friday
> > then we should release it...Then starting with 6.2 make sure we eliminate
> > as many memory problems as possible....
> >
> > Oh well, just a thought....
> >
> > BTW...I will be out of town next week, then when I come back it's Jury
> > duty...so I'll try to finish psql/libpq by this week anyway....
>
> OK, let's see where we are at the end of the week. Take it as far as
> you can. I am curious what you see in the backend.
>
> People are reporting more memory problems than in 6.1. It may be psql,
> or libpq, or it may be a few things that we added to the backend that
> are causing problems.
>
> When people are running the regression tests multiple times and seeing
> errors, how are you doing that? Doesn't the backend and psql restart on
> every iteration? Is it the postmaster that is growing too big?
>
> --
> Bruce Momjian
> maillist@candle.pha.pa.us
>
------------------------------
No it wasn't me...Somebody mailed the questions list a while ago (half a
year maybe...my memory is not that good...) about purifying the
code...Then I think somebody else mailed hackers list a week or two ago...
=+=------------------------/\---------------------------------=+=
Igor Natanzon |**| E-mail: igor@sba.miami.edu
=+=------------------------\/---------------------------------=+=
On Mon, 2 Jun 1997, Bruce Momjian wrote:
> > > I just lobbed it out there. I was waiting to hear your reaction to
> > > this.
> >
> > I figured it was a subtle hint *rofl*
> >
> > > Sounds good to me. I don't want too much pressure on Igor. Doing this
> > > kind of cleanup is a delicate job, and needs to be done carefully.
> >
> > Whatever happened to the guy that originally brought up the whole
> > Purify issue though? Why is Igor doing all the work on it? :(
>
> I thought Igor was the one to bring up the Purify issue. Was it someone
> else? Was it me?
>
> --
> Bruce Momjian
> maillist@candle.pha.pa.us
>
------------------------------
> > Well, one of the leaks that Purify detected was that individual lines read > from files were malloc'ed but not freed properly. So, if a line is > malloc'ed to about 8K (and we are reading thousands of lines in COPY reg > test I believe) and is not freed before next line is read, we get > significant memory usage... Plus adding all other leaks we get into major > problems... > > I fixed the 'lines' leak so we shall see what happens...The backend should > be interesting... Because of the postgresql memory manager, some of the leaks are auto-cleaned after every statement/transaction, so it may not be too bad. Or it may be. - -- Bruce Momjian maillist@candle.pha.pa.us ------------------------------
> > No it wasn't me...Somebody mailed the questions list a while ago (half a > year maybe...my memory is not that good...) about purifying the > code...Then I think somebody else mailed hackers list a week or two ago... I know Kurt started on it, but I never heard if he finished. - -- Bruce Momjian maillist@candle.pha.pa.us ------------------------------
On Mon, 2 Jun 1997, Bruce Momjian wrote:
> > > I just lobbed it out there. I was waiting to hear your reaction to
> > > this.
> >
> > I figured it was a subtle hint *rofl*
> >
> > > Sounds good to me. I don't want too much pressure on Igor. Doing this
> > > kind of cleanup is a delicate job, and needs to be done carefully.
> >
> > Whatever happened to the guy that originally brought up the whole
> > Purify issue though? Why is Igor doing all the work on it? :(
>
> I thought Igor was the one to bring up the Purify issue. Was it someone
> else? Was it me?
nope, it was someone else, and I can't remember the gentleman's name...
don't you recall the thread where we were getting down to how unless we
ran Purify against the code, then PostgreSQL should only be considered
alpha code, at the very best? Damn, I hate it when I can't remember someone's
name...:( I know it wasn't Igor though...Igor is sending us patches...this
other guy was just sending out Purify output :(
Marc G. Fournier
Systems Administrator @ hub.org
primary: scrappy@hub.org secondary: scrappy@{freebsd|postgresql}.org
------------------------------
On Mon, 2 Jun 1997, Bruce Momjian wrote:
> >
> > Well, one of the leaks that Purify detected was that individual lines read
> > from files were malloc'ed but not freed properly. So, if a line is
> > malloc'ed to about 8K (and we are reading thousands of lines in COPY reg
> > test I believe) and is not freed before next line is read, we get
> > significant memory usage... Plus adding all other leaks we get into major
> > problems...
> >
> > I fixed the 'lines' leak so we shall see what happens...The backend should
> > be interesting...
>
> Because of the postgresql memory manager, some of the leaks are
> auto-cleaned after every statement/transaction, so it may not be too
> bad. Or it may be.
Actually, that would only free up memory for the backend, no? I
think the problem that Igor is reporting is that in psql itself, we are
doing (if I remember the patch correctly):
line = malloc();
fgets(line);
But never did a free(line); after we were finished with the
data in line, just assigning a new memory segment to it...
Marc G. Fournier
Systems Administrator @ hub.org
primary: scrappy@hub.org secondary: scrappy@{freebsd|postgresql}.org
------------------------------
On Mon, 2 Jun 1997, Bruce Momjian wrote:
> >
> > No it wasn't me...Somebody mailed the questions list a while ago (half a
> > year maybe...my memory is not that good...) about purifying the
> > code...Then I think somebody else mailed hackers list a week or two ago...
>
> I know Kurt started on it, but I never heard if he finished.
Damn, now you have me curious...hold, back through my logs...found it:
===========================
> Robert Withrow wrote:
> >
> > I think these need to be fixed before the release of 6.1. Otherwise
> > this release is basicly a non-starter for any serious use, IMO. I say
> > this because purify detects loads of *serious* problems in the backend
> > while running the regression tests.
> >
===========================
I knew it wasn't Igor :) That was from around the 30th of May...
Marc G. Fournier
Systems Administrator @ hub.org
primary: scrappy@hub.org secondary: scrappy@{freebsd|postgresql}.org
------------------------------
> > On Mon, 2 Jun 1997, Bruce Momjian wrote: > > > > > > > Well, one of the leaks that Purify detected was that individual lines read > > > from files were malloc'ed but not freed properly. So, if a line is > > > malloc'ed to about 8K (and we are reading thousands of lines in COPY reg > > > test I believe) and is not freed before next line is read, we get > > > significant memory usage... Plus adding all other leaks we get into major > > > problems... > > > > > > I fixed the 'lines' leak so we shall see what happens...The backend should > > > be interesting... > > > > Because of the postgresql memory manager, some of the leaks are > > auto-cleaned after every statement/transaction, so it may not be too > > bad. Or it may be. > > Actually, that would only free up memory for the backend, no? I > think the problem that Igor is reporting is that in psql itself, we are > doing (if I remember the patch correctly): > > line = malloc(); > fgets(line); > > But never did a free(line); after we were finished with the > data in line, just assigning a new memory segment to it... Yes, Igor is talking about psql and libpq. I was discussing the potential problems he may or may not find in the backend. - -- Bruce Momjian maillist@candle.pha.pa.us ------------------------------
Well..that might not be easy to find out...I suppose better be on the safe side anyway and make sure these leaks don't occur.... > > Because of the postgresql memory manager, some of the leaks are > auto-cleaned after every statement/transaction, so it may not be too > bad. Or it may be. > > -- > Bruce Momjian > maillist@candle.pha.pa.us > ------------------------------
> > On Mon, 2 Jun 1997, Bruce Momjian wrote: > > > > > > > No it wasn't me...Somebody mailed the questions list a while ago (half a > > > year maybe...my memory is not that good...) about purifying the > > > code...Then I think somebody else mailed hackers list a week or two ago... > > > > I know Kurt started on it, but I never heard if he finished. > > Damn, now you have me curious...hold, back through my logs...found it: > > =========================== > > Robert Withrow wrote: > > > > > > I think these need to be fixed before the release of 6.1. Otherwise > > > this release is basicly a non-starter for any serious use, IMO. I say > > > this because purify detects loads of *serious* problems in the backend > > > while running the regression tests. > > > > =========================== > > I knew it wasn't Igor :) That was from around the 30th of May... I guess this answers my curiosity of what Igor is going to find running Purify on the backend. When I originally thought about a postponement for the Purify run, I thought it could all be done in a week. I did not expect so many bugs being found. - -- Bruce Momjian maillist@candle.pha.pa.us ------------------------------
Actually, there was a free statement ..... in the wrong place...It was
useless there. But this problem is exactly what I am talking about ...
reading these lines....
I have tested pretty much everything in psql so far after fixing this bug,
and there hasn't been either a memory leak or some memory violation...
I'll probably run regression tests tomorrow, then post the patch...
There was also another small bug where the line was
1. Under one condition it was simply set to point to something that
was staticly allocated, so freeing is not allowed.
2. If above condition fails, it will be malloc'ed... Of course
there was no free there, and it leaked the whole 10 bytes...:(
> line = malloc();
> fgets(line);
>
> But never did a free(line); after we were finished with the
> data in line, just assigning a new memory segment to it...
>
> Marc G. Fournier
> Systems Administrator @ hub.org
> primary: scrappy@hub.org secondary: scrappy@{freebsd|postgresql}.org
>
------------------------------
> > OK, I agree. I have heard people complaining about excessive memory > > problems under 6.1. Hopefully, they were inaccurate, or the problem > > will be found by Friday. > > I can recall only 'order by' problem, but it's fixed already. Yes, I remember now. It was suggested that the additional ORDER BY's added by Tom to make the results consistent with the GEQO optimizer were causing more leaking. - -- Bruce Momjian maillist@candle.pha.pa.us ------------------------------
If I remember correctly, not all of there problems from Purifying the
backend were memory leaks....there were array bounds errors ( should be
easy to track down) and uninitialized memory being read...stuff like
that...should be interesting...
=+=------------------------/\---------------------------------=+=
Igor Natanzon |**| E-mail: igor@sba.miami.edu
=+=------------------------\/---------------------------------=+=
On Tue, 3 Jun 1997, Bruce Momjian wrote:
> >
> > On Mon, 2 Jun 1997, Bruce Momjian wrote:
> >
> > > >
> > > > No it wasn't me...Somebody mailed the questions list a while ago (half a
> > > > year maybe...my memory is not that good...) about purifying the
> > > > code...Then I think somebody else mailed hackers list a week or two ago...
> > >
> > > I know Kurt started on it, but I never heard if he finished.
> >
> > Damn, now you have me curious...hold, back through my logs...found it:
> >
> > ===========================
> > > Robert Withrow wrote:
> > > >
> > > > I think these need to be fixed before the release of 6.1. Otherwise
> > > > this release is basicly a non-starter for any serious use, IMO. I say
> > > > this because purify detects loads of *serious* problems in the backend
> > > > while running the regression tests.
> > > >
> > ===========================
> >
> > I knew it wasn't Igor :) That was from around the 30th of May...
>
> I guess this answers my curiosity of what Igor is going to find running
> Purify on the backend.
>
> When I originally thought about a postponement for the Purify run, I
> thought it could all be done in a week. I did not expect so many bugs
> being found.
>
>
> --
> Bruce Momjian
> maillist@candle.pha.pa.us
>
------------------------------
Bruce Momjian wrote: > > > I was only proposing the additional week because I thought there > > was something still open 'non-Purify' related that was to be addressed... > > > > ...is this not the case? If its only the Purify stuff that is > > being the basis...forget it :) As I stated before, we've been unPure > > for, what, 6 releases now? A start has been made, but it is *not* something > > that should hold back a release... > > > > ...so, unless there are any outstanding issues that should be > > resolved, let's continue with the June 6th release... > > OK, I agree. I have heard people complaining about excessive memory > problems under 6.1. Hopefully, they were inaccurate, or the problem > will be found by Friday. I can recall only 'order by' problem, but it's fixed already. Vadim ------------------------------
Bruce Momjian wrote: > > > > > I am doing it cause I finally got access to a machine with licensed > > Purify...:) > > > > Hmm..I fixed two more memory leaks (bad ones) in psql, but won't post a > > patch until I do some testing...Maybe Wednesday...I don't think it would > > be wise to fix any more of these problems until after the release... > > > > I will also start working on the backend as soon as I get my new > > machine....which will happen in about two weeks... > > That sounds like a plan to me, or an r_plan :-) b_plan :-) Vadim ------------------------------
The Hermit Hacker wrote:
> > I thought Igor was the one to bring up the Purify issue. Was it
> > someone else? Was it me?
> nope, it was someone else, and I can't remember the gentleman's
> name...
Robert Withrow sent mail saying that he had Purify output and wondering
if anyone would find it useful. I waited for a day or two hoping someone
else would pick up the topic and since no one did I told Robert to send
me the output and I would try to sort through it and break it into
smaller pieces.
The output was pointing at code with which I was not familiar, but as a
start I sent the output for the initdb run to Vadim. I can send any
pieces to anyone who might want to pick up some work :)
The initdb output was < 100KB and seemed to point at just a few places
with leaks if I read it correctly. The output for the regression tests
was ~ 1MB. Lots of useful info I suppose but I didn't have much of a
clue on how to start :(
- Tom
------------------------------
On Tue, 3 Jun 1997, Igor wrote: Chances are that the uninitialized memory read errors aren't things that need to be worried about; e.g. if you create a structure and only use some parts of it, then write() the whole thing to somewhere, you'll see that. I'll spend some time digging around in them and see if this is the case... The bounds errors are, IMHO, the most important to tackle... - -Tymm > If I remember correctly, not all of there problems from Purifying the > backend were memory leaks....there were array bounds errors ( should be > easy to track down) and uninitialized memory being read...stuff like > that...should be interesting... > > =+=------------------------/\---------------------------------=+= > Igor Natanzon |**| E-mail: igor@sba.miami.edu > =+=------------------------\/---------------------------------=+= > > On Tue, 3 Jun 1997, Bruce Momjian wrote: > > > > > > > On Mon, 2 Jun 1997, Bruce Momjian wrote: > > > > > > > > > > > > > No it wasn't me...Somebody mailed the questions list a while ago (half a > > > > > year maybe...my memory is not that good...) about purifying the > > > > > code...Then I think somebody else mailed hackers list a week or two ago... > > > > > > > > I know Kurt started on it, but I never heard if he finished. > > > > > > Damn, now you have me curious...hold, back through my logs...found it: > > > > > > =========================== > > > > Robert Withrow wrote: > > > > > > > > > > I think these need to be fixed before the release of 6.1. Otherwise > > > > > this release is basicly a non-starter for any serious use, IMO. I say > > > > > this because purify detects loads of *serious* problems in the backend > > > > > while running the regression tests. > > > > > > > > =========================== > > > > > > I knew it wasn't Igor :) That was from around the 30th of May... > > > > I guess this answers my curiosity of what Igor is going to find running > > Purify on the backend. > > > > When I originally thought about a postponement for the Purify run, I > > thought it could all be done in a week. I did not expect so many bugs > > being found. > > > > > > -- > > Bruce Momjian > > maillist@candle.pha.pa.us > > > > ------------------------------ End of hackers-digest V1 #375 *****************************
Thomas G. Lockhart wrote: > > The Hermit Hacker wrote: > > > I thought Igor was the one to bring up the Purify issue. Was it > > > someone else? Was it me? > > nope, it was someone else, and I can't remember the gentleman's > > name... > > Robert Withrow sent mail saying that he had Purify output and wondering > if anyone would find it useful. I waited for a day or two hoping someone > else would pick up the topic and since no one did I told Robert to send > me the output and I would try to sort through it and break it into > smaller pieces. > > The output was pointing at code with which I was not familiar, but as a > start I sent the output for the initdb run to Vadim. I can send any I've saved it... > pieces to anyone who might want to pick up some work :) > > The initdb output was < 100KB and seemed to point at just a few places > with leaks if I read it correctly. The output for the regression tests > was ~ 1MB. Lots of useful info I suppose but I didn't have much of a > clue on how to start :( What about downloading it to ftp.postgresql.org? Mark? I havn't time currently, but someday... Vadim ------------------------------
Yea..bounds errors could lead to data corruption and should be fixed even
before the leaks....
=+=------------------------/\---------------------------------=+=
Igor Natanzon |**| E-mail: igor@sba.miami.edu
=+=------------------------\/---------------------------------=+=
On Tue, 3 Jun 1997, Tymm Twillman wrote:
> On Tue, 3 Jun 1997, Igor wrote:
>
> Chances are that the uninitialized memory read errors aren't things that
> need to be worried about; e.g. if you create a structure and only
> use some parts of it, then write() the whole thing to somewhere, you'll
> see that. I'll spend some time digging around in them and see if this is
> the case... The bounds errors are, IMHO, the most important to tackle...
>
> -Tymm
>
> > If I remember correctly, not all of there problems from Purifying the
> > backend were memory leaks....there were array bounds errors ( should be
> > easy to track down) and uninitialized memory being read...stuff like
> > that...should be interesting...
> >
> > =+=------------------------/\---------------------------------=+=
> > Igor Natanzon |**| E-mail: igor@sba.miami.edu
> > =+=------------------------\/---------------------------------=+=
> >
> > On Tue, 3 Jun 1997, Bruce Momjian wrote:
> >
> > > >
> > > > On Mon, 2 Jun 1997, Bruce Momjian wrote:
> > > >
> > > > > >
> > > > > > No it wasn't me...Somebody mailed the questions list a while ago (half a
> > > > > > year maybe...my memory is not that good...) about purifying the
> > > > > > code...Then I think somebody else mailed hackers list a week or two ago...
> > > > >
> > > > > I know Kurt started on it, but I never heard if he finished.
> > > >
> > > > Damn, now you have me curious...hold, back through my logs...found it:
> > > >
> > > > ===========================
> > > > > Robert Withrow wrote:
> > > > > >
> > > > > > I think these need to be fixed before the release of 6.1. Otherwise
> > > > > > this release is basicly a non-starter for any serious use, IMO. I say
> > > > > > this because purify detects loads of *serious* problems in the backend
> > > > > > while running the regression tests.
> > > > > >
> > > > ===========================
> > > >
> > > > I knew it wasn't Igor :) That was from around the 30th of May...
> > >
> > > I guess this answers my curiosity of what Igor is going to find running
> > > Purify on the backend.
> > >
> > > When I originally thought about a postponement for the Purify run, I
> > > thought it could all be done in a week. I did not expect so many bugs
> > > being found.
> > >
> > >
> > > --
> > > Bruce Momjian
> > > maillist@candle.pha.pa.us
> > >
> >
> >
>
------------------------------
> > If I remember correctly, not all of there problems from Purifying the
> > backend were memory leaks....there were array bounds errors ( should be
> > easy to track down) and uninitialized memory being read...stuff like
> > that...should be interesting...
I checked the UMR's; all of them are generated between 2 lines of code:
fe-connect.c:
pe = getprotobyname ("TCP");
fe-auth.c:
struct passwd *pw = getpwuid(geteuid());
They are not postgeSQL code errors (yay :) )... They're all from NIS
passing things around.
BTW, Igor: wonderful job on the psql patches-- my tests confirm the no
leaked bytes on regression tests.
- -Tymm
------------------------------
End of hackers-digest V1 #376
*****************************
Well, there is one UMB in fe-connect.c that annoys the hell out of me. It
has to do with variable pacBuf .... I spend some time tracking it
down...so far I failed :(....Apparently Purify on SunOS gives slightly
more info concerning UMR's so I'll work from there...
I ran the the backend under purify today...Damn Irix...the NIS stuff
almost makes Purify cry....But I did find a few leaks in the
postgres buffer manager...
=+=------------------------/\---------------------------------=+=
Igor Natanzon |**| E-mail: igor@sba.miami.edu
=+=------------------------\/---------------------------------=+=
On Tue, 3 Jun 1997, Tymm Twillman wrote:
> > > If I remember correctly, not all of there problems from Purifying the
> > > backend were memory leaks....there were array bounds errors ( should be
> > > easy to track down) and uninitialized memory being read...stuff like
> > > that...should be interesting...
>
> I checked the UMR's; all of them are generated between 2 lines of code:
>
> fe-connect.c:
>
> pe = getprotobyname ("TCP");
>
> fe-auth.c:
>
> struct passwd *pw = getpwuid(geteuid());
>
> They are not postgeSQL code errors (yay :) )... They're all from NIS
> passing things around.
>
> BTW, Igor: wonderful job on the psql patches-- my tests confirm the no
> leaked bytes on regression tests.
>
> -Tymm
>
------------------------------
On Tue, 3 Jun 1997, Vadim B. Mikheev wrote:
> > pieces to anyone who might want to pick up some work :)
> >
> > The initdb output was < 100KB and seemed to point at just a few places
> > with leaks if I read it correctly. The output for the regression tests
> > was ~ 1MB. Lots of useful info I suppose but I didn't have much of a
> > clue on how to start :(
>
> What about downloading it to ftp.postgresql.org? Mark?
> I havn't time currently, but someday...
There is an 'incoming' directory at ftp.postgresql.org to which
stuff like this can be uploaded. Unless there is an attached .README file,
I do go through periodically and clean out old stuff...but it is there
for short term storage, as required
Marc G. Fournier
Systems Administrator @ hub.org
primary: scrappy@hub.org secondary: scrappy@{freebsd|postgresql}.org
------------------------------
The Hermit Hacker wrote:
> On Tue, 3 Jun 1997, Vadim B. Mikheev wrote:
...
> > > The initdb output was < 100KB and seemed to point at just a few places
> > > with leaks if I read it correctly. The output for the regression tests
> > > was ~ 1MB. Lots of useful info I suppose but I didn't have much of a
> > > clue on how to start :(
> >
> > What about downloading it to ftp.postgresql.org? Mark?
> > I havn't time currently, but someday...
>
> There is an 'incoming' directory at ftp.postgresql.org to which
> stuff like this can be uploaded. Unless there is an attached .README file,
> I do go through periodically and clean out old stuff...but it is there
> for short term storage, as required
This is old mail (from June 3) to which I responded on June 5.
I placed the purify results in
ftp://postgresql.org/incoming/purify/purify_*.plog.gz
Bon appetit!
- Tom
------------------------------
End of hackers-digest V1 #385
*****************************