Qt-jambi-interest Archive, July 2007
Slow parsing of integers
Message 1 in thread
Hello,
First of all, thank you for releasing Qt Jambi, it is a great addition
to the Qt family!
I am reading approx. 100 000 integers mixed with 1000 String objects
both encoded using QDataStream. Reading them using
QDataStream.readInt() and readString() is very slow. It take about 6
seconds with my 1.3 GHz pentium M and the file fully loaded in memory,
which I find way too much. Do you know any workaround to deserialize
faster these data? Thanks in advance.
I am using JDK 1.6.0/Qt Jambi 4.3.0_01/OpenSuSe 10.1
Message 2 in thread
Timothee Hunter wrote:
> Hello,
> First of all, thank you for releasing Qt Jambi, it is a great addition
> to the Qt family!
Thank you ;-)
> I am reading approx. 100 000 integers mixed with 1000 String objects
> both encoded using QDataStream. Reading them using
> QDataStream.readInt() and readString() is very slow. It take about 6
> seconds with my 1.3 GHz pentium M and the file fully loaded in memory,
> which I find way too much. Do you know any workaround to deserialize
> faster these data? Thanks in advance.
>
> I am using JDK 1.6.0/Qt Jambi 4.3.0_01/OpenSuSe 10.1
Hi Timothee,
Thanks for your report!
I've done some profiling here and found the primary cause for the
slowdown. The problem is that the C++ implementation of
QDataStream::operator>>(int) returns the this pointer and this is
converted back to java using a rather unfavorable path in the JNI
binding code. On the Java side we don't need this return value so we
simply discard it, so this extra step can be completely removed. We'll
make a task for fixing this for next release. On my windows machine this
got it down from 640ms to 62ms in the case of reading and similarly for
writing.
As for workarounds currently, there are no really good ones that I can
see, but writeString() and readString() are already implemented to
ignore the return value so these are significantly faster so if you read
ints using:
intValue = Integer.parseInt(stream.readString());
and similarly for writing:
stream.writeString(Stream.valueOf(intValue));
This is also 2-3 times faster than the original on my machine.
best regards,
Gunnar
