Qt-interest Archive, December 2007
Unable to remove database ?
Pages: Prev | 1 | 2 | Next
Message 1 in thread
In a class subclassing QThread, I have a ctor that
does something like this:
Runnable::Runnable(QString name)
tname(name)
{
QString dbname = QCoreApplication::applicationDirPath()
+ "/" + "database/test";
// create a named per-thread DB connection.
db = QSqlDatabase::addDatabase("QSQLITE", tname);
db.setDatabaseName(dbname);
}
In the dtor I do this:
Runnable::~Runnable()
{
db.close();
QSqlDatabase::removeDatabase(tname);
}
When I run a test program that creates some objects
based on this class, I get messages like this:
QSqlDatabasePrivate::removeDatabase: connection 'Parent' is still in
use, all queries will cease to work.
immediately after the completion of the dtors.
Am I removing the databases incorrectly ?
--
[ signature omitted ]
Message 2 in thread
Stephen Collyer wrote:
> When I run a test program that creates some objects
> based on this class, I get messages like this:
>
> QSqlDatabasePrivate::removeDatabase: connection 'Parent' is still in
> use, all queries will cease to work.
>
> immediately after the completion of the dtors.
That should be "before" not "after"
> Am I removing the databases incorrectly ?
I've found the relevant Qt code. It's this:
void QSqlDatabasePrivate::invalidateDb(const QSqlDatabase &db, const
QString &name)
{
if (db.d->ref != 1) {
qWarning("QSqlDatabasePrivate::removeDatabase: connection '%s'
is still in use, "
"all queries will cease to work.",
name.toLocal8Bit().constData());
db.d->disable();
}
}
in src/sql/kernel/qsqldatabase.cpp
It seems that the db logic keeps a reference count for each
connection object, and that it thinks something else has a reference
to the connection I'm trying to remove. However, I can't see
how this can occur in my code - does any one know how/where
the reference count is incremented ?
--
[ signature omitted ]
Message 3 in thread
> It seems that the db logic keeps a reference count for each
> connection object, and that it thinks something else has a reference
> to the connection I'm trying to remove. However, I can't see
> how this can occur in my code - does any one know how/where
> the reference count is incremented ?
In your destructor, "db" hasn't been destroyed yet (even though you've .close()'d
it) so QSqlDatabase still sees it as a reference to the underlying database. I've
hit this situation before too. Once solution I've come up with is to not make a
QSqlDatabase object in my class and instead just do calls to
QSqlDatabase::database(name) when I need the handle.
Alternatively, you can do the remove database somewhere else in the program after
the object's destructor has run.
Also, be careful with your threading. Operations on a database are supposed to be
done in the same thread the database was created in.
Caleb
--
[ signature omitted ]
Message 4 in thread
Caleb Tennis wrote:
>> It seems that the db logic keeps a reference count for each
>> connection object, and that it thinks something else has a reference
>> to the connection I'm trying to remove. However, I can't see
>> how this can occur in my code - does any one know how/where
>> the reference count is incremented ?
>
> In your destructor, "db" hasn't been destroyed yet (even though you've .close()'d
> it) so QSqlDatabase still sees it as a reference to the underlying database. I've
> hit this situation before too. Once solution I've come up with is to not make a
> QSqlDatabase object in my class and instead just do calls to
> QSqlDatabase::database(name) when I need the handle.
Thanks for the response. However, your explanation doesn't
seem to make sense. The test in the code is:
if (db.d->ref != 1) {
qWarning("QSqlDatabasePrivate::removeDatabase: connection '%s'
is still in use, "
"all queries will cease to work.",
i.e. it looks for a reference count of 0 or 2,3,4.. when it emits
the warning. Now I have only 1 reference to the connection as far
as I can tell, so the test should fail. So yes, although db may
not have been destroyed, there shouldn't be a problem.
And while I'm thinking about it, should the test not be for
reference counts > 1 ? Presumably 0 is fine too. That looks like
a minor bug there.
> Also, be careful with your threading. Operations on a database are supposed to be
> done in the same thread the database was created in.
So presumably this implies a thread shouldn't offer a public method
to other threads that runs a query ? I'm not intending to do that,
in fact.
--
[ signature omitted ]
Message 5 in thread
> Thanks for the response. However, your explanation doesn't
> seem to make sense. The test in the code is:
>
> if (db.d->ref != 1) {
> qWarning("QSqlDatabasePrivate::removeDatabase: connection '%s'
> is still in use, "
> "all queries will cease to work.",
The single handle in this case, I believe, is the one internal to QSqlDatabase that
you are removing when you do a removeDatabase. Your "db" variable is the "2nd" open
handle.
> And while I'm thinking about it, should the test not be for
> reference counts > 1 ? Presumably 0 is fine too. That looks like a minor bug there.
The Trolls can probably better answer that one, but I assume that there's no way to
get to 0, because of the internal reference inside of QSqlDatabase.
> So presumably this implies a thread shouldn't offer a public method to other
threads that runs a query ? I'm not intending to do that, in fact.
I was more talking about doing any db calls within your run() method, which operates
in a different thread that your db was created in, in the Runnable constructor.
Caleb
--
[ signature omitted ]
Message 6 in thread
Hi,
I also had that problems, there is one more problem:
if you have a QSqlDatabase in your class, you have to name it, if not,
you always change the global object ! I had funny things with this, so I
wrote to the trolls, but it seems, they have also problems understanding
their own code...
Now, if I need more than one database connection, (and my application
needs a few of them..), I create a random name for my connection, save
the connection settings with global mutexes to prevent double accesses
to the global object, but I do not really understand, what happens here
! I get sometimes the same messages, but now together with the random
name ! but all my connections work and not only with single threaded
programs, also with parallel running threads ! So I think this warning
is a bug of QT. As I have open more than one connection to different
database servers (mysql, odbc) and no crashes and no data losses, I
think the system works, but this message is curious...
I hope, I get a answer soon (I send my question on saturday last week
but got only a message on monday, that the message reaches the trolls...)
Regards,
Peter
Caleb Tennis schrieb:
>> Thanks for the response. However, your explanation doesn't
>> seem to make sense. The test in the code is:
>>
>> if (db.d->ref != 1) {
>> qWarning("QSqlDatabasePrivate::removeDatabase: connection '%s'
>> is still in use, "
>> "all queries will cease to work.",
>
> The single handle in this case, I believe, is the one internal to QSqlDatabase that
> you are removing when you do a removeDatabase. Your "db" variable is the "2nd" open
> handle.
>
>> And while I'm thinking about it, should the test not be for
>> reference counts > 1 ? Presumably 0 is fine too. That looks like a minor bug there.
>
> The Trolls can probably better answer that one, but I assume that there's no way to
> get to 0, because of the internal reference inside of QSqlDatabase.
>
>> So presumably this implies a thread shouldn't offer a public method to other
> threads that runs a query ? I'm not intending to do that, in fact.
>
> I was more talking about doing any db calls within your run() method, which operates
> in a different thread that your db was created in, in the Runnable constructor.
>
> Caleb
>
>
>
> --
> To unsubscribe - send a mail to qt-interest-request@xxxxxxxxxxxxx with "unsubscribe" in the subject or the body.
> List archive and information: http://lists.trolltech.com/qt-interest/
--
[ signature omitted ]
Message 7 in thread
peter wrote:
> Hi,
> I also had that problems, there is one more problem:
> if you have a QSqlDatabase in your class, you have to name it, if not,
> you always change the global object ! I had funny things with this, so I
> wrote to the trolls, but it seems, they have also problems understanding
> their own code...
Well, I'm naming all of my connections anyway, so I may not see
the same problem that you've run into. What were the "funny things" ?
> Now, if I need more than one database connection, (and my application
> needs a few of them..), I create a random name for my connection, save
> the connection settings with global mutexes to prevent double accesses
> to the global object,
This seems to suggest that you are creating global connections,
accessible from any thread - is that right ?
> but I do not really understand, what happens here
> ! I get sometimes the same messages, but now together with the random
> name !
I don't understand what you are saying here.
--
[ signature omitted ]
Message 8 in thread
Stephen,
Stephen Collyer schrieb:
> peter wrote:
>> Hi,
>> I also had that problems, there is one more problem:
>> if you have a QSqlDatabase in your class, you have to name it, if not,
>> you always change the global object ! I had funny things with this, so I
>> wrote to the trolls, but it seems, they have also problems understanding
>> their own code...
>
> Well, I'm naming all of my connections anyway, so I may not see
> the same problem that you've run into. What were the "funny things" ?
The funny things was, that my queries no longer worked, because the
settings influenced each other ! This is documented (see QSqlDatabase
docs).
>
>> Now, if I need more than one database connection, (and my application
>> needs a few of them..), I create a random name for my connection, save
>> the connection settings with global mutexes to prevent double accesses
>> to the global object,
>
> This seems to suggest that you are creating global connections,
> accessible from any thread - is that right ?
No, I have the connections in the threads and these instances are named
with a random created name (because I do not know, how many threads are
started...
in my class, I always have a
private:
QSqlDatabase db;
and in init-code, I always start with db =
QSqlDatabase::addDatabase("TYPE", "MYRANDOMNAME");
in destructor I have
db.close();
>
>> but I do not really understand, what happens here
>> ! I get sometimes the same messages, but now together with the random
>> name !
>
> I don't understand what you are saying here.
>
The messages, that says, that the connection is closed and all further
calls are ceased... And the message have the RANDOMCREATEDNAME in the
message !!, but the thread is still running and working !!
Sorry for my bad english.
--
[ signature omitted ]
Message 9 in thread
peter wrote:
> The funny things was, that my queries no longer worked, because the
> settings influenced each other ! This is documented (see QSqlDatabase
> docs).
OK.
>> This seems to suggest that you are creating global connections,
>> accessible from any thread - is that right ?
>
> No, I have the connections in the threads and these instances are named
> with a random created name (because I do not know, how many threads are
> started...
> in my class, I always have a
>
> private:
> QSqlDatabase db;
>
> and in init-code, I always start with db =
> QSqlDatabase::addDatabase("TYPE", "MYRANDOMNAME");
>
> in destructor I have
>
> db.close();
Where do you run the init code ? In a ctor, or at the top
of the run() method ?
>>> but I do not really understand, what happens here
>>> ! I get sometimes the same messages, but now together with the random
>>> name !
>>
>> I don't understand what you are saying here.
>>
>
> The messages, that says, that the connection is closed and all further
> calls are ceased... And the message have the RANDOMCREATEDNAME in the
> message !!, but the thread is still running and working !!
OK, that looks very strange. I suspect that this indicates that your
using the connection in a way that the Trolls did not intend.
> Sorry for my bad english.
Your English is fine, in fact.
--
[ signature omitted ]
Message 10 in thread
Stephen,
Stephen Collyer schrieb:
> peter wrote:
>
>> The funny things was, that my queries no longer worked, because the
>> settings influenced each other ! This is documented (see QSqlDatabase
>> docs).
>
> OK.
>
>>> This seems to suggest that you are creating global connections,
>>> accessible from any thread - is that right ?
>> No, I have the connections in the threads and these instances are named
>> with a random created name (because I do not know, how many threads are
>> started...
>> in my class, I always have a
>>
>> private:
>> QSqlDatabase db;
>>
>> and in init-code, I always start with db =
>> QSqlDatabase::addDatabase("TYPE", "MYRANDOMNAME");
>>
>> in destructor I have
>>
>> db.close();
>
> Where do you run the init code ? In a ctor, or at the top
> of the run() method ?
I start a separate class function from run(), after I setup my class
with some information_transfering_function() to my class. I also have a
global mutex running that locks other threads between my
myglobalMutex.lock();
db=QSqlDatabase::addDatabase() stuff
and the
if (db.open())... part.
myglobalMutex.unlock();
so only on single access is possible to QSqlDatabase.
if an error occurs, I simply send a signal and stop the thread...
In destructor of my class, I call a close on each connection...
But I always get that message in the middle of my thread and it still
works..
>
>>>> but I do not really understand, what happens here
>>>> ! I get sometimes the same messages, but now together with the random
>>>> name !
>>> I don't understand what you are saying here.
>>>
>> The messages, that says, that the connection is closed and all further
>> calls are ceased... And the message have the RANDOMCREATEDNAME in the
>> message !!, but the thread is still running and working !!
>
> OK, that looks very strange. I suspect that this indicates that your
> using the connection in a way that the Trolls did not intend.
>
>> Sorry for my bad english.
>
> Your English is fine, in fact.
>
I hope, this clearifies it...
Regards,
Peter
--
[ signature omitted ]
Message 11 in thread
Caleb Tennis wrote:
>> Thanks for the response. However, your explanation doesn't
>> seem to make sense. The test in the code is:
>>
>> if (db.d->ref != 1) {
>> qWarning("QSqlDatabasePrivate::removeDatabase: connection '%s'
>> is still in use, "
>> "all queries will cease to work.",
>
> The single handle in this case, I believe, is the one internal to QSqlDatabase that
> you are removing when you do a removeDatabase. Your "db" variable is the "2nd" open
> handle.
I guess that's possible.
>> And while I'm thinking about it, should the test not be for
>> reference counts > 1 ? Presumably 0 is fine too. That looks like a minor bug there.
>
> The Trolls can probably better answer that one, but I assume that there's no way to
> get to 0, because of the internal reference inside of QSqlDatabase.
Any Troll reading ?
>> So presumably this implies a thread shouldn't offer a public method to other
> threads that runs a query ? I'm not intending to do that, in fact.
>
> I was more talking about doing any db calls within your run() method, which operates
> in a different thread that your db was created in, in the Runnable constructor.
Yikes. Now I'm confused. So that rule seems to imply that I can't
create a pre-thread db connection in the thread class ctor, then
call queries on that connection from run(). I guess that it further
implies that all db connections called from run() must be created
either in run(), or created in the parent thread, and passed into
the ctor ? If so, that's not what I'm doing at the moment. So, if
I've understood correctly, this is bad:
class BadThread : public QThread
{
BadThread() { QSqlDatabase mydb = QSqlDatabase::addDatabase(..); }
void run() { <use mydb object in a query> }
private:
QSqlDatabase mydb;
}
but this is OK:
int main(..)
{
QSqlDatabase db = QSqlDatabase::addDatabase(..);
GoodThread t1(db);
t1.start();
..
}
class GoodThread : public QThread
{
GoodThread(QSqlDatabase db) : mydb(db) {};
void run() { <use mydb object in a query> }
private:
QSqlDatabase mydb;
}
Is that an accurate summary of this restriction ?
--
[ signature omitted ]
Message 12 in thread
> Yikes. Now I'm confused. So that rule seems to imply that I can't
> create a pre-thread db connection in the thread class ctor, then
> call queries on that connection from run(). I guess that it further
> implies that all db connections called from run() must be created
> either in run(), or created in the parent thread, and passed into
> the ctor ? If so, that's not what I'm doing at the moment. So, if
> I've understood correctly, this is bad:
The verbage isn't very...verbose, but it's here:
http://doc.trolltech.com/4.3/threads.html#threads-and-the-sql-module
> class BadThread : public QThread
..
> class GoodThread : public QThread
I don't think your GoodThread example would work either, but the database connection
is still created in a differing thread than it is used. According to the docs, you
can't "move connections between threads".
However, I don't know what the repercussions are of doing it like this. It may end
up just working out fine.
Caleb
--
[ signature omitted ]
Message 13 in thread
Caleb Tennis wrote:
> The verbage isn't very...verbose, but it's here:
>
> http://doc.trolltech.com/4.3/threads.html#threads-and-the-sql-module
Yes, I've read that. I don't find it particularly clear though.
>> class BadThread : public QThread
> ..
>
>> class GoodThread : public QThread
>
> I don't think your GoodThread example would work either, but the database connection
> is still created in a differing thread than it is used. According to the docs, you
> can't "move connections between threads".
I've thought about this a bit more and done some experiments.
You're probably right. I've written some code that emits
the QThread::currentThreadId() from
a) main()
b) the ctor of a QThread subclass object created by main
c) the run() method in the secondary thread
e,g,
int main(void)
{
MyThread t1;
<emit QThread::currentThreadId()>
t1.start();
}
class MyThread : public QThread
{
MyThread() { <emit QThread::currentThreadId()> };
void run() { <emit QThread::currentThreadId()> };
}
The result is that the thread id is identical in main and the
ctor, and changes in the run() method. This makes sense I suppose.
Until the ctor has completed, no secondary thread exists to have
a different thread id, but when run() executes, the secondary thread
object has successfully been constructed.
I think it therefore follows from the TT docs that you *must not*
create connection objects in the ctor of a thread class, but wait
until run() executes, and do it in there. It also follows that you
must not construct a connection in main() and hand it on to a thread
due to the caveat about passing connections between threads.
--
[ signature omitted ]
Message 14 in thread
Stephen Collyer wrote:
> Caleb Tennis wrote:
>
>
>> The verbage isn't very...verbose, but it's here:
>>
>> http://doc.trolltech.com/4.3/threads.html#threads-and-the-sql-module
>>
>
> Yes, I've read that. I don't find it particularly clear though.
>
>
>>> class BadThread : public QThread
>>>
>> ..
>>
>>
>>> class GoodThread : public QThread
>>>
>> I don't think your GoodThread example would work either, but the database connection
>> is still created in a differing thread than it is used. According to the docs, you
>> can't "move connections between threads".
>>
>
> I've thought about this a bit more and done some experiments.
> You're probably right. I've written some code that emits
> the QThread::currentThreadId() from
>
> a) main()
> b) the ctor of a QThread subclass object created by main
> c) the run() method in the secondary thread
>
> e,g,
>
> int main(void)
> {
> MyThread t1;
>
> <emit QThread::currentThreadId()>
>
> t1.start();
> }
>
> class MyThread : public QThread
> {
> MyThread() { <emit QThread::currentThreadId()> };
>
> void run() { <emit QThread::currentThreadId()> };
> }
>
> The result is that the thread id is identical in main and the
> ctor, and changes in the run() method. This makes sense I suppose.
> Until the ctor has completed, no secondary thread exists to have
> a different thread id, but when run() executes, the secondary thread
> object has successfully been constructed.
>
> I think it therefore follows from the TT docs that you *must not*
> create connection objects in the ctor of a thread class, but wait
> until run() executes, and do it in there. It also follows that you
> must not construct a connection in main() and hand it on to a thread
> due to the caveat about passing connections between threads.
>
>
You're heading down the right path now :) (Sorry for so late, missed
this thread and only caught it now).
Most of these threading caveats are because sqlite is obnoxiously bad
with threading pre 3.5.0, so to work around the sqlite issues, there is
a distinct need to keep a database connection per thread.
Also, a thread doesn't actually start until run is called, so yeah,
creating it in the constructor, bad. Creating it in run, good.
Lastly, if you keep a QSqlDatabase object local to your class, then
yeah, unless you assign to it an empty QSqlDatabase() object, there'll
be a second reference still alive. (eg, db = QSqlDatabase(); should be
enough to clear out the reference to remove the warning).
Hope this helps :)
--
[ signature omitted ]
Message 15 in thread
Bill KING wrote:
> Lastly, if you keep a QSqlDatabase object local to your class, then
> yeah, unless you assign to it an empty QSqlDatabase() object, there'll
> be a second reference still alive. (eg, db = QSqlDatabase(); should be
> enough to clear out the reference to remove the warning).
Bill
Can you expand on this ? At what point are you suggesting that
this assignment be done ? In a ctor, before the run() method
does the same thing, I guess ?
--
[ signature omitted ]
Pages: Prev | 1 | 2 | Next
