Qt-interest Archive, April 2007
Advice on QPointer/QObject
Message 1 in thread
Hi
My app makes heavy use of QObject and QPointer. So much that it is
being designed with a target of easily managing at least 10000 QObject
derived classes. And to share these objects with other parts of the
app I'm using QPointer. At maximum a single QObject is used at 4
places where QPointer is used to store them.
I've successfully tested it with @ 2000 QObjects (and connected
QPointer), but am not sure of perfoormance effects regarding them. I
could not find any material documenting as such so I have no idea.
So, I need your advice on two points: -
1. How many QObject can Qt4 manage effectively.
2. Is it ok to use QPointer as cited above i.e. have 4 (at max.)
QPointer per QObject.
Hoping my problem is clear enough and you have answer for it :)
Thanks...
--
[ signature omitted ]
Message 2 in thread
On Wed, April 4, 2007 10:16, Yogesh M wrote:
> So, I need your advice on two points: -
> 1. How many QObject can Qt4 manage effectively.
As many as your memory can hold. Qt does not store references to all
QObjects, so the amount of objects in memory is no constraint on
efficiency (as long as your memory management is ok, which it should for
all major systems).
> 2. Is it ok to use QPointer as cited above i.e. have 4 (at max.)
> QPointer per QObject.
Yes.
There are two instances in which Qt stores references:
1) QObjects store references to their child objects and to their parent
object. So if an object is destroyed it can destroy its children and
notify its parent.
2) QObjects remember QPointers pointing on them, so they can notify the
pointer when they are deleted (this is hidden somewhere deep in the
destructor code of QObject).
Qt uses QList structures to remember children and QPointers. QList
operations (at least some of them) grow slower approximately linearly with
the size of the list. So for efficiency (performance) there are three
factors for you to consider:
a) how big can the lists of children grow for an individual QObject
(during high-performance computing this becomes a problem at about 1000 or
more; for GUI it becomes a nuisance at about 10000 or more)
b) how many QPointers can point to an individual QObject (your number of
about 4 is not a problem at all - you won't even notice the overhead)
c) how deep is the hierarchy of a QObjects children (as with "a)" above
this should not become a problem unless the hierarchy grows very deep and
broad)
Konrad
PS.: how did you manage to write 10000 classes? Even Qt itself contains
only about 2000 (just counted in a qt 4.3 snapshot with grep), less than
half of these are part of the interface.
--
[ signature omitted ]
Message 3 in thread
On 04/04/07, Konrad Rosenbaum <konrad@xxxxxxxxx> wrote:
> On Wed, April 4, 2007 10:16, Yogesh M wrote:
> > So, I need your advice on two points: -
> > 1. How many QObject can Qt4 manage effectively.
>
> As many as your memory can hold. Qt does not store references to all
> QObjects, so the amount of objects in memory is no constraint on
> efficiency (as long as your memory management is ok, which it should for
> all major systems).
>
> > 2. Is it ok to use QPointer as cited above i.e. have 4 (at max.)
> > QPointer per QObject.
>
> Yes.
Given that this will equal to 10000 * 4 i.e. approx. 40000 QPointer?
> There are two instances in which Qt stores references:
>
> 1) QObjects store references to their child objects and to their parent
> object. So if an object is destroyed it can destroy its children and
> notify its parent.
>
> 2) QObjects remember QPointers pointing on them, so they can notify the
> pointer when they are deleted (this is hidden somewhere deep in the
> destructor code of QObject).
>
> Qt uses QList structures to remember children and QPointers. QList
> operations (at least some of them) grow slower approximately linearly with
> the size of the list. So for efficiency (performance) there are three
> factors for you to consider:
>
> a) how big can the lists of children grow for an individual QObject
> (during high-performance computing this becomes a problem at about 1000 or
> more; for GUI it becomes a nuisance at about 10000 or more)
Those 10000 objects can be children of a single object.
> b) how many QPointers can point to an individual QObject (your number of
> about 4 is not a problem at all - you won't even notice the overhead)
>
> c) how deep is the hierarchy of a QObjects children (as with "a)" above
> this should not become a problem unless the hierarchy grows very deep and
> broad)
Hierarchy is not deep. It is only the number that matters. If you want
details, plase do notify me; I'll prepare a draft of requirements and
representation of data in my app.
> PS.: how did you manage to write 10000 classes? Even Qt itself contains
> only about 2000 (just counted in a qt 4.3 snapshot with grep), less than
> half of these are part of the interface.
"So much that it is
being designed with a target of easily managing at least 10000 QObject
derived classes. " in my earlier mail meant (Sorry for missing info)
"So much that it is
being designed with a target of easily managing at least 10000 *objects of *
QObject
derived classes.
Regards
Message 4 in thread
On Wed, April 4, 2007 16:13, Yogesh M wrote:
>> > 2. Is it ok to use QPointer as cited above i.e. have 4 (at max.)
>> > QPointer per QObject.
>>
>> Yes.
>
> Given that this will equal to 10000 * 4 i.e. approx. 40000 QPointer?
No Problem at all. It would only be a problem if they all pointed to the
same object.
>> a) how big can the lists of children grow for an individual QObject
>> (during high-performance computing this becomes a problem at about 1000
>> or
>> more; for GUI it becomes a nuisance at about 10000 or more)
>
> Those 10000 objects can be children of a single object.
I'd say: try it with a simple demo program. Keep your code flexible enough
to change it later, while you start optimizing.
BTW: why do you need those objects to be QObjects? QObject is not exactly
cheap on memory and CPU (at least if you are trying to do number crunching
or similiar tasks with them).
Konrad
--
[ signature omitted ]
Message 5 in thread
On 4/4/07, Yogesh M <yogeshm02@xxxxxxxxx> wrote:
> Given that this will equal to 10000 * 4 i.e. approx. 40000 QPointer?
Well given the number of object I would suggest consider some other
design approaches. I don't know the other constrains and your
requirement, maybe you need all the QObject stuff like signals,
metadata, children list, object names etc, for each one of those
objects, but I think it hardly the case. QObject is a really a heavy
buddy - it contains a lots of functionality and data. It is ok for GUI
and some other stuff - you need those features anyway, and the number
of objects are relatively small. In your case however, I would
consider outlining the required features, and then carefully designing
the class myself, having complexity consideration in mind. You can
save a _lot_ of CPU cycles and memory by using different techniques
and patterns - i.e. flyweight pattern, intrusive containers,
compressed data, object pools and other tools to avoid memory
fragmentation, hashes for fast lockups, etc, etc.
--
[ signature omitted ]
Message 6 in thread
First of all very warm regards for sparing your time.
On 04/04/07, Pavel Antokolsky aka Zigmar <zigmar@xxxxxxxxx> wrote:
> On 4/4/07, Yogesh M <yogeshm02@xxxxxxxxx> wrote:
> > Given that this will equal to 10000 * 4 i.e. approx. 40000 QPointer?
> Well given the number of object I would suggest consider some other
> design approaches. I don't know the other constrains and your
> requirement, maybe you need all the QObject stuff like signals,
> metadata, children list, object names etc, for each one of those
> objects, but I think it hardly the case. QObject is a really a heavy
> buddy - it contains a lots of functionality and data. It is ok for GUI
> and some other stuff - you need those features anyway, and the number
> of objects are relatively small. In your case however, I would
> consider outlining the required features, and then carefully designing
> the class myself, having complexity consideration in mind. You can
> save a _lot_ of CPU cycles and memory by using different techniques
> and patterns - i.e. flyweight pattern, intrusive containers,
> compressed data, object pools and other tools to avoid memory
> fragmentation, hashes for fast lockups, etc, etc.
I initially used QObject for signals/slots functionality (which proved
easier for initial implementation) thinking of scraping QObject later
on. But then I discovered QPointer and it proved very useful for
sharing those QObject based instances.
Now QObject can be avoided without much effort but I cant think of
replacement of QPointer.
I have done some timing tests with around 5400 objects and 2/3
functions are taking around 400 ms on "extreme conditions", otherwise
everything is fine. I am unable to reduce that 400ms time despite many
experiments and BTW the relevant (slow) code concludes to 10 lines or
so.
Thanks.
--
[ signature omitted ]
Message 7 in thread
On 4/5/07, Yogesh M <yogeshm02@xxxxxxxxx> wrote:
>
> First of all very warm regards for sparing your time.
>
> On 04/04/07, Pavel Antokolsky aka Zigmar <zigmar@xxxxxxxxx> wrote:
> > On 4/4/07, Yogesh M <yogeshm02@xxxxxxxxx> wrote:
> > > Given that this will equal to 10000 * 4 i.e. approx. 40000 QPointer?
> > Well given the number of object I would suggest consider some other
> > design approaches. I don't know the other constrains and your
> > requirement, maybe you need all the QObject stuff like signals,
> > metadata, children list, object names etc, for each one of those
> > objects, but I think it hardly the case. QObject is a really a heavy
> > buddy - it contains a lots of functionality and data. It is ok for GUI
> > and some other stuff - you need those features anyway, and the number
> > of objects are relatively small. In your case however, I would
> > consider outlining the required features, and then carefully designing
> > the class myself, having complexity consideration in mind. You can
> > save a _lot_ of CPU cycles and memory by using different techniques
> > and patterns - i.e. flyweight pattern, intrusive containers,
> > compressed data, object pools and other tools to avoid memory
> > fragmentation, hashes for fast lockups, etc, etc.
>
> I initially used QObject for signals/slots functionality (which proved
> easier for initial implementation) thinking of scraping QObject later
> on. But then I discovered QPointer and it proved very useful for
> sharing those QObject based instances.
>
> Now QObject can be avoided without much effort but I cant think of
> replacement of QPointer.
>
> I have done some timing tests with around 5400 objects and 2/3
> functions are taking around 400 ms on "extreme conditions", otherwise
> everything is fine. I am unable to reduce that 400ms time despite many
> experiments and BTW the relevant (slow) code concludes to 10 lines or
> so.
>
It depends on the constrains, but to me 400ms looks like a hell lots of a
time. :)
And since have you already narrowed the performance problem to 10 lines of
code - it should be easy to fix it.
And BTW, QPointer isn't uniq. If I understand you QPointer usage right I
guess boost::shared_prt+boost::weak_ptr would give you at least the same
features, with probably less overhead. There are some other reference
counting techniques also available, that even faster: i.e. linked_ptr -
gives you less features, but does not require any dynamic allocations like
the shared_ptr does.
--
[ signature omitted ]
Message 8 in thread
On Thursday 05 April 2007, Yogesh M wrote:
>
> I initially used QObject for signals/slots functionality (which proved
> easier for initial implementation) thinking of scraping QObject later
> on. But then I discovered QPointer and it proved very useful for
> sharing those QObject based instances.
>
> Now QObject can be avoided without much effort but I cant think of
> replacement of QPointer.
I had a similar problem in a project where the number of objects can easily
grow to ~50000 or more. I used the following approach:
During construction, each object gets a unique id which is the index into a
vector holding pointers to all objects in the project. Instead of using
QPointer or something similar I simply store the id of the referenced
object and access it via BaseObj::object(id).
Simplified example code:
#ifndef BASEOBJ_H
#define BASEOBJ_H
#include <QVector>
class BaseObj
{
public:
BaseObj() {
// assign next free id to this object
myId = ++lastId;
// increase the vector size in steps of 10000 objects
// in order to not have to resize often
if (objects.size() <= myId)
objects.resize(objects.size() + 10000);
// store pointer to this object in the vector
objects[myId] = this;
}
virtual ~BaseObj() {
// remove this object from the vector
objects[myId] = 0;
}
int id() const {
// return id of this object
return myId;
}
static BaseObj* object(int id) {
// return object with a certain id
return (id > 0 && id < objects.size()) ? objects[id] : 0;
}
private:
// id of this object
int myId;
// last occupied id
static int lastId;
// vector containing all objects derived from BaseObj
static QVector<BaseObj*> objects;
};
int BaseObj::lastId = 0;
QVector<BaseObj*> BaseObj::objects;
#endif // BASEOBJ_H
Regards,
Bernd
--
[ signature omitted ]
