Qt-interest Archive, March 2007
How to minimize the compiling time without care of speed or size of the exe?
Message 1 in thread
Hi,
I work on Qt in vs2003. I notice there are many compiler settings making
me confused.
Would you like to share your experience on how to save compiling time
without care of speed or size of the exe? What is the better settings
combination?
Thanks,
Lingfa
--
[ signature omitted ]
Message 2 in thread
It's not Qt- or VS-specific, but I think you'll get a relatively good
compile-time speed improvement by making sure you don't include more header
files than you need to. Don't use header files when a class stub will do.
ry
On Tuesday 27 March 2007 13:15, Lingfa Yang wrote:
> Hi,
>
> I work on Qt in vs2003. I notice there are many compiler settings making
> me confused.
>
> Would you like to share your experience on how to save compiling time
> without care of speed or size of the exe? What is the better settings
> combination?
>
> Thanks,
> Lingfa
>
>
>
>
>
> --
> To unsubscribe - send a mail to qt-interest-request@xxxxxxxxxxxxx with
> "unsubscribe" in the subject or the body. List archive and information:
> http://lists.trolltech.com/qt-interest/
--
[ signature omitted ]
Message 3 in thread
Lingfa Yang wrote:
> Hi,
>
> I work on Qt in vs2003. I notice there are many compiler settings making
> me confused.
>
> Would you like to share your experience on how to save compiling time
> without care of speed or size of the exe? What is the better settings
> combination?
I never tried to time how long does a build take with different
settings ... bug I can offer a guess:
The Microsoft compiler is fast (with optimization or without), so what
I guess matters the most is the size of your object files ... thus,
the best thing to do would be to optimize your code for size and make
sure you are not generating any debug information.
If it's the linking that takes long ... you could try incremental
linking. Incremental linking used to screw up from time to time and
then you needed to rebuild your whole project ... but I've last tried
this feature long time ago (probably not with vs2003), so may be it's
better now.
Anyway, the most important thing is memory. For bigger projects,
doubling the memory in your computer can cut the compile (especially
link) time by a factor of five.
Also restructuring your project so that less sources need to be
recompiled every time you make a change could help.
Martin.
--
[ signature omitted ]
Message 4 in thread
I'm trying to access a usb device from a QIODevice. I've been using a
QFile but I've been having problems with it. I open the
file("/dev/usbtm0") readwrite | unbuffered and the first read returns
data if there is data waiting on the device. Next, I write data to the
device using qfile.write(buffer, sizeof(buffer)). After this write, I
should be getting data back from the device and after I kill my program
and check the device(tail /dev/usbtm0) I see data. However, I tried a
qfile.read(inbuf, sizeof(buffer)). I tried qfile.bytesavailable() and
I've tried waitForReadyRead(). After the first read nothing returns that
there is data to read. Subsequent writes work fine.
I, originally, tried to use normal unix open, read, write, close calls.
But the object is inheriting from QDialog and therefore close() tries to
use the wrong function call. I was getting the correct functionality
with the unix open,read,write but the problem was it just wasn't closing
the file.
How are you supposed to handle a normal unix /dev device from inside QT.
I don't need anything fancy. I just need to be able to read and write to
the device.
--
[ signature omitted ]
Message 5 in thread
Try ::close
Allen, Matthew wrote:
> I'm trying to access a usb device from a QIODevice. I've been using a
> QFile but I've been having problems with it. I open the
> file("/dev/usbtm0") readwrite | unbuffered and the first read returns
> data if there is data waiting on the device. Next, I write data to the
> device using qfile.write(buffer, sizeof(buffer)). After this write, I
> should be getting data back from the device and after I kill my program
> and check the device(tail /dev/usbtm0) I see data. However, I tried a
> qfile.read(inbuf, sizeof(buffer)). I tried qfile.bytesavailable() and
> I've tried waitForReadyRead(). After the first read nothing returns that
> there is data to read. Subsequent writes work fine.
>
> I, originally, tried to use normal unix open, read, write, close calls.
> But the object is inheriting from QDialog and therefore close() tries to
> use the wrong function call. I was getting the correct functionality
> with the unix open,read,write but the problem was it just wasn't closing
> the file.
>
> How are you supposed to handle a normal unix /dev device from inside QT.
> I don't need anything fancy. I just need to be able to read and write to
> the device.
>
> --
> To unsubscribe - send a mail to qt-interest-request@xxxxxxxxxxxxx with "unsubscribe" in the subject or the body.
> List archive and information: http://lists.trolltech.com/qt-interest/
>
>
--
[ signature omitted ]
Message 6 in thread
AHA!!! With normal unix open/read/write/::close it works. Thank you very
much. You are an excellent person!
-----Original Message-----
From: Andrew Ward [mailto:andy.ward@xxxxxxxxxx]
Sent: Tuesday, March 27, 2007 4:27 PM
To: qt-interest@xxxxxxxxxxxxx
Subject: Re: Accessing a USB device from inside QT.
Try ::close
Allen, Matthew wrote:
> I'm trying to access a usb device from a QIODevice. I've been using a
> QFile but I've been having problems with it. I open the
> file("/dev/usbtm0") readwrite | unbuffered and the first read returns
> data if there is data waiting on the device. Next, I write data to the
> device using qfile.write(buffer, sizeof(buffer)). After this write, I
> should be getting data back from the device and after I kill my
program
> and check the device(tail /dev/usbtm0) I see data. However, I tried a
> qfile.read(inbuf, sizeof(buffer)). I tried qfile.bytesavailable() and
> I've tried waitForReadyRead(). After the first read nothing returns
that
> there is data to read. Subsequent writes work fine.
>
> I, originally, tried to use normal unix open, read, write, close
calls.
> But the object is inheriting from QDialog and therefore close() tries
to
> use the wrong function call. I was getting the correct functionality
> with the unix open,read,write but the problem was it just wasn't
closing
> the file.
>
> How are you supposed to handle a normal unix /dev device from inside
QT.
> I don't need anything fancy. I just need to be able to read and write
to
> the device.
>
> --
> To unsubscribe - send a mail to qt-interest-request@xxxxxxxxxxxxx with
"unsubscribe" in the subject or the body.
> List archive and information: http://lists.trolltech.com/qt-interest/
>
>
--
[ signature omitted ]
Message 7 in thread
Lingfa Yang wrote:
> I work on Qt in vs2003. I notice there are many compiler
> settings making me confused.
>
> Would you like to share your experience on how to save compiling time
> without care of speed or size of the exe? What is the better settings
> combination?
The canonical rules are something like the following:
Do not put more stuff than actually needed in header files.
Do not include more headers than you actually need.
Use precompiled headers.
The "Pimpl" idiom helps to keep header files lean.
Recently I stumbled over another "trick":
Using larger compilation units can help if the total
size is reasonably chosen. I.e. something like
lump.cpp:
#include "foo.cpp"
#include "bar.cpp"
#include "baz.cpp"
with lump.cpp added to the project and {foo,bar,baz}.cpp
set as "Exclude from build". The idea is that a C++
compilation unit consists usually of a few hundred lines
of "user" code, yet #include <vector> + #include <string>
pull in ~20000 lines of code. With "lumping" as above
the ratio user/system improves.
Andre'
--
[ signature omitted ]
Message 8 in thread
>
> Recently I stumbled over another "trick":
> Using larger compilation units can help if the total
> size is reasonably chosen. I.e. something like
>
> lump.cpp:
> #include "foo.cpp"
> #include "bar.cpp"
> #include "baz.cpp"
>
> with lump.cpp added to the project and {foo,bar,baz}.cpp
> set as "Exclude from build". The idea is that a C++
> compilation unit consists usually of a few hundred lines
> of "user" code, yet #include <vector> + #include <string>
> pull in ~20000 lines of code. With "lumping" as above
> the ratio user/system improves.
>
> Andre'
>
Unfortunately... Andre is right on here. The way that stl templates have been implemented is so bloated code parsing can take for ever with PCH turned off, or on template headers where PCH has less effect
Scott
--
[ signature omitted ]
Message 9 in thread
Scott Aron Bloom wrote:
> > Recently I stumbled over another "trick":
> > Using larger compilation units can help if the total
> > size is reasonably chosen. I.e. something like
> >
> > lump.cpp:
> > #include "foo.cpp"
> > #include "bar.cpp"
> > #include "baz.cpp"
> >
> > with lump.cpp added to the project and {foo,bar,baz}.cpp
> > set as "Exclude from build". The idea is that a C++
> > compilation unit consists usually of a few hundred lines
> > of "user" code, yet #include <vector> + #include <string>
> > pull in ~20000 lines of code. With "lumping" as above
> > the ratio user/system improves.
> >
> > Andre'
>
> Unfortunately... Andre is right on here. The way that stl
> templates have been implemented is so bloated code parsing
> can take for ever with PCH turned off, or on template headers
> where PCH has less effect
We are currently testing the idea with cmake based builds
for LyX (www.lyx.org), and it looks like it has the potential
of speeding up full compiles of the project by a factor of at
least 4(!). Some polishing is still needed for the "memory
hogs" (compiling with profiling on goes to swap on a 512 MB
machine) and there needs to be an easy-to-use way to get
fast roundtrips after changing a few files only (basic idea
is to not lump these files...), but it looks _really_ promising
right now.
Maybe that'd be an idea for Qt snapshots (to get even more
on-topic) as they have a tendency to compile longer than it
takes to have a cup of tea...
Andre'
--
[ signature omitted ]
Message 10 in thread
Another option might be distributed compilation. With g++ is is quite
easy (you have distcc or I used source code on networked drive +
simple wrapper around g++ which runs g++ on one of other computers via
ssh + "make -j30" (If there are 30 computers in the cluster))
In my case, it reduced time needed to recompile entire project from
about 10 minutes to about 1 minute)
Though I am not sure whether this feature is supported is MSVC, I'm
not using it. It should be probably ...
Martin Petricek
On 3/28/07, Scott Aron Bloom <scott@xxxxxxxxxxxx> wrote:
> >
> > Recently I stumbled over another "trick":
> > Using larger compilation units can help if the total
> > size is reasonably chosen. I.e. something like
> >
> > lump.cpp:
> > #include "foo.cpp"
> > #include "bar.cpp"
> > #include "baz.cpp"
> >
> > with lump.cpp added to the project and {foo,bar,baz}.cpp
> > set as "Exclude from build". The idea is that a C++
> > compilation unit consists usually of a few hundred lines
> > of "user" code, yet #include <vector> + #include <string>
> > pull in ~20000 lines of code. With "lumping" as above
> > the ratio user/system improves.
> >
> > Andre'
> >
> Unfortunately... Andre is right on here. The way that stl templates have been implemented is so bloated code parsing can take for ever with PCH turned off, or on template headers where PCH has less effect
>
--
[ signature omitted ]
Message 11 in thread
BH wrote:
> Another option might be distributed compilation. With g++ is is quite
> easy (you have distcc or I used source code on networked drive +
> simple wrapper around g++ which runs g++ on one of other computers via
> ssh + "make -j30" (If there are 30 computers in the cluster))
> In my case, it reduced time needed to recompile entire project from
> about 10 minutes to about 1 minute)
This is certainly an option (as is ccache), but it helps only a bit,
and only if you have sufficiently many nodes and decent network.
Most notably, the final linking step is not distributable, and
with tons of small compilation units you end up with quite a lot
of work for the linker (mainly reading largish .o files and
throwing away duplicated symbols)
Andre'
--
[ signature omitted ]
Message 12 in thread
> This is certainly an option (as is ccache), but it helps only a bit,
> and only if you have sufficiently many nodes and decent network.
>
> Most notably, the final linking step is not distributable, and
> with tons of small compilation units you end up with quite a lot
> of work for the linker (mainly reading largish .o files and
> throwing away duplicated symbols)
No, it is not. But even though this, speedup from using more computers
can be significant. For example our project (pdfedit) compiled 10
minutes, of which about 30 or so secs is linking (creating 60mb
executable from about 60 mb of object files .... of those 90% are
debug data :). Rest of it is compiling (one .cc to .o took about 1 to
7 secs, there is over 100 of .cc files)
But still, those 9:30 can be reduced by parallelization. Even one
extra computer will help, as you'll cut the time almost to half. Get
20 or 30 and you compile in about a minute. And 100 mb network is
usually enough, as there is not that much data that flows thru network
during compilation - for each module it is the module source and few
headers in (= cca 100kb), one object out (=cca 600kb). Final linking
step needs 120mb of data to be put thru. Sum: 200Mb/minute of
compilation = 4.5Mb/sec. 100Mbit can handle that :)
Martin Petricek
--
[ signature omitted ]
Message 13 in thread
It’s a nightmare using vmake to build windows application on network.
Here is my project building:
Build Time Environment Source Files Object Files Component MSVC Python
Qt4 Report
2:47:17 vmake Remote Remote Remote Remote Remote Remote Remote
0:07:00 Vs2003 Local Local Local Local Local Local Local
--
[ signature omitted ]
