Qt-interest Archive, December 2006
QT XML with DOM needs huge amounts of memory
Pages: Prev | 1 | 2 | Next
Message 16 in thread
André Somers wrote:
> Arne Schmitz wrote:
>
>> My problem: I want to parse a large XML file (50 MB) with Qt's XML
>> classes. I want to do it with DOM, because of the simplicity. I DO know
>> that this is more memory exhaustive compared to sax2. But however, after
>> parsing the memory should be freed, right? This does not happen, however.
>> Did I overlook something? I use only local variables, no new() operator
>> at all.
>
> Some time ago, there was a blog posting on Planet KDE about this issue. It
> related to reading in documents in KOffice, which also resulted in huge
> memory usage due to the fact that a DOM approch was used (ODF documents
> are XML as well). The poster in question created a lightweight version,
> that, as far as I remember, saved about an order of magnitude of memory
> and which resulted in quite significant time savings.
> It might be interesting to try and look up this posting and see if you can
> re-use it, or at least the technology. He made something that was less
> efficient when changing the data (who cares if you're only parsing a
> document), but which was way more memory efficient.
That is exactly what I am looking for! I will google for it -- you don't
remember any more infos, or do you?
Arne
--
[ signature omitted ]
Message 17 in thread
Arne Schmitz wrote:
>> Some time ago, there was a blog posting on Planet KDE about this issue.
>> It related to reading in documents in KOffice, which also resulted in
>> huge memory usage due to the fact that a DOM approch was used (ODF
>> documents are XML as well). The poster in question created a lightweight
>> version, that, as far as I remember, saved about an order of magnitude of
>> memory and which resulted in quite significant time savings.
>> It might be interesting to try and look up this posting and see if you
>> can re-use it, or at least the technology. He made something that was
>> less efficient when changing the data (who cares if you're only parsing a
>> document), but which was way more memory efficient.
>
> That is exactly what I am looking for! I will google for it -- you don't
> remember any more infos, or do you?
I found it. They developed an XML reader and writer for file format
conversion. It is tons of magnitudes more efficient than QDom as far as I
read and can be found here:
http://websvn.kde.org/trunk/koffice/libs/store/
I will try to rewrite my small test application to use it and report back
here. :-)
Cheers,
Arne
--
[ signature omitted ]
Message 18 in thread
Arne Schmitz wrote:
> I found it. They developed an XML reader and writer for file format
> conversion. It is tons of magnitudes more efficient than QDom as far as I
> read and can be found here:
>
> http://websvn.kde.org/trunk/koffice/libs/store/
>
> I will try to rewrite my small test application to use it and report back
> here. :-)
Done. It works! The parsing now takes a mere 150 MB at the maximum. Also the
memory gets freed as soon as the inner scope is left. This is totally
perfect! Also a big thanks to Ariya for writing this code.
Cheers,
Arne
--
[ signature omitted ]
