Qt-interest Archive, July 2007
Using Qt I/O (QFile) for reading from files and stdin in various modes
Message 1 in thread
Hi!
This is going to be a kind of a long philosophical post. Not even sure
if anybody here is interested in all of these, so you may wish to just
skip the whole thing at once.
I find Unix filter programs one of the most powerful yet simple UI
designs. Programs like grep, cat, wc and such can do a lot and they are
simple to use. Every Unix user knows that. Now I have been trying to
implement these kind of programs with Qt. The following is a description
of all the problems that I have encountered. But first I have to specify
the task more precisely.
So, we have a program that reads something from a file or stdin and
writes something else to another file or stdout. No big problems on the
write part, so from now on I shall be speaking about reading only.
Reading generally can be done two ways: synchronously and
asynchronously. Or, blocking and non-blocking. Also, we can read from
file or from stdin. Stdin, in turn, can be a pipe, a console input, a
file or a socket. Also, we may stop reading either on EOF or when we
encounter a special end-of-input structure. And finally, Unix and
Windows have different ways of doing most of these things.
With ideal cross-platform toolkit I would expect to have one generic
interface for all of these cases. But world is not perfect, so there is
a lot of troubles. What I want to do is find a way to avoid most or all
of them, so I shall have an interface that will enable me to write
simple filters without thinking much about on which platform I am, where
am I reading from or whatever. For some of the troubles I have found
perfect solutions, for some others too, maybe not so perfect though, and
for the rest I am at loss.
First trouble: regular files. Regular files are not the most complicated
thing in the world, but nevertheless. If I am to stop reading at EOF,
then everything is perfect. Regular files are different from everything
else in that sense that they are usually considered always non-blocking,
at least if the file is on the local file system and everything is all
right with the system. Well, not a big problem, really. Troubles start
when I want to continue reading past of the EOF until I encounter my
special end-of-input record, or forever, like "tail -f" does on Unix.
The simplest way to do that is just wait for some time and check if
there is more data available. The problem is that this introduces
unnecessary delay. This may be fine while I am just playing around with
Open Source Edition at home, but if I manage to make those evil guys
buy me a Qt license for my job, then I am probably going to do this on
real-time ISS telemetry, where additional delays are not A Very Good
Thing(tm), definitely. Especially when there are a lot of filters
involved.
A way around this is, of course, QFileSystemWatcher. With three notes.
First one is that we need to know file name in order to use it.
Therefore it will not work if we have stdin redirected from a file.
I see no way to work around this. So we have a limitation here.
Second note: our input becomes non-blocking and event-driven even if we
do not want that. This is due to absence of
QFileSystemWatcher::waitForFileChanged() method. This leads me to
thinking that maybe Trolls have gone the wrong way with all these
waitFor... methods - maybe they should have just provided us with
QObject::waitForSignal(sender, signal) method or alike? Not sure why
they did not do that. Still, this leads us to another limitation. Maybe
there is a work around this like calling QCoreApplication::exec() in a
loop, but I am not sure if this will not break some day because it was
not designed for such kind of things. Correct me if I am wrong. Maybe I
should create a separate QThread for watching file and wake up main
thread using QWaitCondition to emulate blocking behaviour?
Third note: QFileSystemWatcher actually works properly only on a few
platforms: Q_OS_WIN, Q_OS_LINUX, Q_OS_FREEBSD and Q_OS_MAC. On other
platforms it is either not implemented at the OS level or not supported
in Qt. The only such platform I am interested in is HP-UX, and it does
not seem to have a support for it. So we have to fall back to
wait-and-see method. Qt does this with its monster
QPollingFileSystemWatcherEngine thing. But the problem is that it uses
1000 ms polling interval without any visible way to change it - it is
defined as enum and used locally in private class! Even if I was to
override the appropriate method in my derived class, I still see no way
to make QFileSystemWatcher use the modified class. It seems that I shall
have to reimplement the whole thing from scratch to make it properly
work. Why did not they just make QFileSystemWatcher::setPollingTimeout()
method? 1000 ms is too much for some tasks, like telemetry, really.
And this was all only about regular files! Now come sockets and pipes. I
am not very interested in keyboard console input, so I shall skip that.
I only know of a way of passing sockets as stdin in Unix. It seems this
can be done in Windows too, but let me skip that lltogether, and since
sockets in unix are not much different from pipes, let us think of
everything that is not a regular file as of Something Sequential(tm).
And the way of opening it is, of course, QFile::open(0, ...) or
QFile::open(stdin, ...).
This something may be blocking or non-blocking. And on Windows it seems
to be always blocking. Moreover, on Windows it is in text mode by
default, while it seems not to work in text mode - in fact, it stops
reading after the first attempt to read from stdin more that is
currently available. For example, if there are 17 bytes in stdin, and I
try to read 16 bytes and then another 16 bytes, then second attempt
returns one byte and then all subsequent reads always return 0 bytes no
matter how much data is being written to stdin. Well, this has nothing
to do with Qt anyway, so let me just say that _setmode(0, _O_BINARY)
does the job.
Still, there seems to be no way to make stdin pipe non-blocking in
Windows. And even if I manage to do that, I have no idea how to get
read notifications for it. QSocketNotifier will not work on Windows,
making Qt event loop wait on stdin handle with private QWinEventNotifier
class gives constant notifications even if there is no data available. I
had a look at Cygwin sources to see how they implemented select() call
for pipes and found out that they create a separate thread for it. So it
seems to be the only way to emulate non-blocking behaviour. Another
non-elegant solution.
In Unix, fcntl makes stdin non-blocking perfectly, whether it is a pipe
or a socket.
But in blocking mode QFile behaves strangely. If I open stdin with
QFile::open(stdin, QIODevice::ReadOnly), then it is ok on Unix. If I
open it with QFile::open(0, QIODevice::ReadOnly), then it may block
forever. The same thing happens always (I think) on Windows. The reason
behind this is that Qt tries to fill its internal buffer with 16K of
data. If there is less data available, it will block until 16K of data
arrive. If it does not happen, then it blocks forever. The reason it
works with stdin on Unix is because of this in
QFSFileEnginePrivate::nativeRead() for Unix:
if (fh && nativeIsSequential()) { // stdin opened as FILE *
// make it non-blocking if needed
// call fread()
// restore blocking state if needed
}
I am at total loss here! What the hell is this and why do they do it
only for FILE* mode, but not file descriptor mode? What is the point of
making stdin behave differently from file descriptor 0? Is this a bug or
what? And still, it does not work on Windows, so what is the point of
doing all this stuff if it is non-portable? Users may just use
QTcpSocket on Unix as well! Oh, on a second thought, they sometimes can
not, since QTcpSocket does not support QIODevice::Unbuffered mode for
some reason, while QFile does.
Now the way to work around unnecessary blocking on any platform is to
just open stdin in QIODevice::Unbuffered mode. Still, I am not sure how
it will all work with that strange block/unblock piece of code in Unix.
The last problem is when to stop reading from stdin. Or how to detect
that the pipe has been closed from the other end. Well, not a problem,
really, but requires different approaches for different platforms and
modes.
Now to the final point. It all leaves me with an impression that most of
the I/O in Qt is hopelessly broken. QFile should be used for files only,
and QTcpSocket only for buffered sockets. Any attempt to do something
unusual, even for the sake of writing very simple filter program, will
just break everything and make you do a lot of not always useful
workarounds. This is a very strange thing considered this is a version 4
based on a lot of experience of making one of the best cross-platform
toolkits.
I am thinking about filling some bug reports, but I cannot make
out separate bugs of this whole mess. Too many flaws here and there,
some of which just reflect underlying platform limitations, some would
require a huge amount of work to even design a proper solution. In fact,
only that block/unblock thing in Unix seems like a real bug or
underdeveloped feature to me.
What surprises me the most is that a filter is a one of the most simple
UI design patterns. How come this is all so complicated? Or did I go
wrong somewhere on the way?
Suggestions, thoughts, workarounds? Anything is welcome. Thank you for
reading so far.
--
[ signature omitted ]
Message 2 in thread
On 26.07.07 00:28:11, Sergey A. Tachenov wrote:
> Now to the final point. It all leaves me with an impression that most of
> the I/O in Qt is hopelessly broken. QFile should be used for files only,
> and QTcpSocket only for buffered sockets. Any attempt to do something
> unusual, even for the sake of writing very simple filter program, will
> just break everything and make you do a lot of not always useful
> workarounds. This is a very strange thing considered this is a version 4
> based on a lot of experience of making one of the best cross-platform
> toolkits.
>
> I am thinking about filling some bug reports, but I cannot make
> out separate bugs of this whole mess. Too many flaws here and there,
> some of which just reflect underlying platform limitations, some would
> require a huge amount of work to even design a proper solution. In fact,
> only that block/unblock thing in Unix seems like a real bug or
> underdeveloped feature to me.
>
> What surprises me the most is that a filter is a one of the most simple
> UI design patterns. How come this is all so complicated? Or did I go
> wrong somewhere on the way?
I didn't read all of your lengthy post, but I think you've just chosen
the wrong tool for the job. The IO support in Qt is certainly not meant
to be used for programs like the one you want to build. I mean even
though it is possible to write server apps and such, the main purpose of
the Qt toolkit is still cross-platform desktop apps - IMHO. Also the IO
support is a compromise between the various supported platforms, often
the Trolls opt for the minimal similarities in their API's among those
platforms and leave out things that are only available on some of them.
Andreas
--
[ signature omitted ]
Message 3 in thread
For your HP-UX issue, maybe fam can help.
http://oss.sgi.com/projects/fam/
The recoll project has a mode for it, if you want to see client code.
Benjamin.
On 7/25/07, Andreas Pakulat <apaku@xxxxxx> wrote:
> On 26.07.07 00:28:11, Sergey A. Tachenov wrote:
> > Now to the final point. It all leaves me with an impression that most of
> > the I/O in Qt is hopelessly broken. QFile should be used for files only,
> > and QTcpSocket only for buffered sockets. Any attempt to do something
> > unusual, even for the sake of writing very simple filter program, will
> > just break everything and make you do a lot of not always useful
> > workarounds. This is a very strange thing considered this is a version 4
> > based on a lot of experience of making one of the best cross-platform
> > toolkits.
> >
> > I am thinking about filling some bug reports, but I cannot make
> > out separate bugs of this whole mess. Too many flaws here and there,
> > some of which just reflect underlying platform limitations, some would
> > require a huge amount of work to even design a proper solution. In fact,
> > only that block/unblock thing in Unix seems like a real bug or
> > underdeveloped feature to me.
> >
> > What surprises me the most is that a filter is a one of the most simple
> > UI design patterns. How come this is all so complicated? Or did I go
> > wrong somewhere on the way?
>
> I didn't read all of your lengthy post, but I think you've just chosen
> the wrong tool for the job. The IO support in Qt is certainly not meant
> to be used for programs like the one you want to build. I mean even
> though it is possible to write server apps and such, the main purpose of
> the Qt toolkit is still cross-platform desktop apps - IMHO. Also the IO
> support is a compromise between the various supported platforms, often
> the Trolls opt for the minimal similarities in their API's among those
> platforms and leave out things that are only available on some of them.
>
> Andreas
>
> --
> Living your life is a task so difficult, it has never been attempted before.
>
> --
> To unsubscribe - send a mail to qt-interest-request@xxxxxxxxxxxxx with "unsubscribe" in the subject or the body.
> List archive and information: http://lists.trolltech.com/qt-interest/
>
>
--
[ signature omitted ]
Message 4 in thread
On onsdag den 25. Juli 2007, Sergey A. Tachenov wrote:
> Hi!
>
> This is going to be a kind of a long philosophical post. Not even sure
> if anybody here is interested in all of these, so you may wish to just
> skip the whole thing at once.
>
> I find Unix filter programs one of the most powerful yet simple UI
> designs. Programs like grep, cat, wc and such can do a lot and they are
> simple to use. Every Unix user knows that. Now I have been trying to
> implement these kind of programs with Qt. The following is a description
> of all the problems that I have encountered. But first I have to specify
> the task more precisely.
>
> So, we have a program that reads something from a file or stdin and
> writes something else to another file or stdout. No big problems on the
> write part, so from now on I shall be speaking about reading only.
>
> Reading generally can be done two ways: synchronously and
> asynchronously. Or, blocking and non-blocking. Also, we can read from
> file or from stdin. Stdin, in turn, can be a pipe, a console input, a
> file or a socket. Also, we may stop reading either on EOF or when we
> encounter a special end-of-input structure. And finally, Unix and
> Windows have different ways of doing most of these things.
>
> With ideal cross-platform toolkit I would expect to have one generic
> interface for all of these cases. But world is not perfect, so there is
> a lot of troubles. What I want to do is find a way to avoid most or all
> of them, so I shall have an interface that will enable me to write
> simple filters without thinking much about on which platform I am, where
> am I reading from or whatever. For some of the troubles I have found
> perfect solutions, for some others too, maybe not so perfect though, and
> for the rest I am at loss.
You can't generalize the different solutions into a single class, because they
all have different issues and uses. You could make a class that implements
the shared functionality, but that would most likely be too little to be of
any use.
> First trouble: regular files. Regular files are not the most complicated
> thing in the world, but nevertheless. If I am to stop reading at EOF,
> then everything is perfect. Regular files are different from everything
> else in that sense that they are usually considered always non-blocking,
> at least if the file is on the local file system and everything is all
> right with the system. Well, not a big problem, really. Troubles start
> when I want to continue reading past of the EOF until I encounter my
> special end-of-input record, or forever, like "tail -f" does on Unix.
>
> The simplest way to do that is just wait for some time and check if
> there is more data available. The problem is that this introduces
> unnecessary delay. This may be fine while I am just playing around with
> Open Source Edition at home, but if I manage to make those evil guys
> buy me a Qt license for my job, then I am probably going to do this on
> real-time ISS telemetry, where additional delays are not A Very Good
> Thing(tm), definitely. Especially when there are a lot of filters
> involved.
If you don't want delays, then saving stuff in files for other applications to
read is just the wrong approach. You can not do this without delays. Period.
> A way around this is, of course, QFileSystemWatcher. With three notes.
Yes, and if you want a crossplatform solution, this is the way to go. In the
case of tail -f, this is special because it assumes there will only be text
added to the file. I don't know if this special case is possible to do with
some platform specific shortcut.
> First one is that we need to know file name in order to use it.
> Therefore it will not work if we have stdin redirected from a file.
> I see no way to work around this. So we have a limitation here.
No we don't. Piping a file to standard in means the application doesn't get
any files. It gets the contents of the file in stdin. Just like if you do
cat myfile | myapp.
> Second note: our input becomes non-blocking and event-driven even if we
> do not want that. This is due to absence of
> QFileSystemWatcher::waitForFileChanged() method. This leads me to
> thinking that maybe Trolls have gone the wrong way with all these
> waitFor... methods - maybe they should have just provided us with
> QObject::waitForSignal(sender, signal) method or alike? Not sure why
> they did not do that.
I have also asked them for a waitForSignal method. But usually this is for
incredibly odd situations. In your case, event driven coding is definitely
the only way to go. stdin is not the only way to access a running program on
unix - you have the unix signals too.
> Still, this leads us to another limitation. Maybe
> there is a work around this like calling QCoreApplication::exec() in a
> loop, but I am not sure if this will not break some day because it was
> not designed for such kind of things. Correct me if I am wrong.
You need to wrap your head around the proper Qt way of doing things. Qt isn't
an add-on to your application. It is the centerpiece that you fill stuff in
to. Qt supplies the main loop, and you work with events. That's is The Right
Way (TM) to do Qt programming. If you don't like this and really only want to
do implement text filter apps, I suggest you take a look at the std::
namespace. Sorry if this sounds hard, but that's the way it is.
> Maybe I
> should create a separate QThread for watching file and wake up main
> thread using QWaitCondition to emulate blocking behaviour?
This is overkill.
> Third note: QFileSystemWatcher actually works properly only on a few
> platforms: Q_OS_WIN, Q_OS_LINUX, Q_OS_FREEBSD and Q_OS_MAC. On other
> platforms it is either not implemented at the OS level or not supported
> in Qt. The only such platform I am interested in is HP-UX, and it does
> not seem to have a support for it. So we have to fall back to
> wait-and-see method. Qt does this with its monster
> QPollingFileSystemWatcherEngine thing. But the problem is that it uses
> 1000 ms polling interval without any visible way to change it - it is
> defined as enum and used locally in private class! Even if I was to
> override the appropriate method in my derived class, I still see no way
> to make QFileSystemWatcher use the modified class. It seems that I shall
> have to reimplement the whole thing from scratch to make it properly
> work. Why did not they just make QFileSystemWatcher::setPollingTimeout()
> method? 1000 ms is too much for some tasks, like telemetry, really.
I suggest you file a bug report to the trolls. You should be able to control
that interval.
IIRC, HP-UX has a fam daemon. And my guess is that QFileWatcher on Linux is
implemented using fam too. So maybe it could be possible for you to implement
the HP-UX support for proper watching of the files.
> [socket stuff gone]
>
> Now to the final point. It all leaves me with an impression that most of
> the I/O in Qt is hopelessly broken. QFile should be used for files only,
> and QTcpSocket only for buffered sockets. Any attempt to do something
> unusual, even for the sake of writing very simple filter program, will
> just break everything and make you do a lot of not always useful
> workarounds. This is a very strange thing considered this is a version 4
> based on a lot of experience of making one of the best cross-platform
> toolkits.
>
> I am thinking about filling some bug reports, but I cannot make
> out separate bugs of this whole mess. Too many flaws here and there,
> some of which just reflect underlying platform limitations, some would
> require a huge amount of work to even design a proper solution. In fact,
> only that block/unblock thing in Unix seems like a real bug or
> underdeveloped feature to me.
>
> What surprises me the most is that a filter is a one of the most simple
> UI design patterns. How come this is all so complicated? Or did I go
> wrong somewhere on the way?
>
> Suggestions, thoughts, workarounds? Anything is welcome. Thank you for
> reading so far.
I think you are making the mistake that because Qt is a cross platform
framework, it should be usable for you cross platform applications. Sorry
troll-salespeople, but the case here might very well be that Qt isn't the
right tool.
You're completely wrong when you say that Qt I/O is broken. It just solves
another problem set than the one you have. Writing desktop apps with the Qt
I/O is fine.
That said, it might be that the trolls could be interested in working with you
on solving the issues you have here. The Qt core lib is a first step in the
direction of text based apps (which could be text filters, services, ...),
and as they are moving Qt in this direction, they need real problems to work
with. Maybe the Qt I/O problem set could be expanded to include your cases.
I don't know if this helps at all :-)
Bo.
--
[ signature omitted ]
Message 5 in thread
On 26.07.07 09:18:54, Bo Thorsen wrote:
> On onsdag den 25. Juli 2007, Sergey A. Tachenov wrote:
> > Third note: QFileSystemWatcher actually works properly only on a few
> > platforms: Q_OS_WIN, Q_OS_LINUX, Q_OS_FREEBSD and Q_OS_MAC. On other
> > platforms it is either not implemented at the OS level or not supported
> > in Qt. The only such platform I am interested in is HP-UX, and it does
> > not seem to have a support for it. So we have to fall back to
> > wait-and-see method. Qt does this with its monster
> > QPollingFileSystemWatcherEngine thing. But the problem is that it uses
> > 1000 ms polling interval without any visible way to change it - it is
> > defined as enum and used locally in private class! Even if I was to
> > override the appropriate method in my derived class, I still see no way
> > to make QFileSystemWatcher use the modified class. It seems that I shall
> > have to reimplement the whole thing from scratch to make it properly
> > work. Why did not they just make QFileSystemWatcher::setPollingTimeout()
> > method? 1000 ms is too much for some tasks, like telemetry, really.
>
> I suggest you file a bug report to the trolls. You should be able to control
> that interval.
>
> IIRC, HP-UX has a fam daemon. And my guess is that QFileWatcher on Linux is
> implemented using fam too. So maybe it could be possible for you to implement
> the HP-UX support for proper watching of the files.
No, QFSWatcher doesn't use fam on any platform and thats good. None of
the two fam solutions is working 90% reliable, they cause CPU-hogging,
memory-hogging or don't work at all. QFSWatcher uses inotify support on
Linux, kqueue on bsd and similar sytems and on win32 something else is
used. I'm not sure what it does on systems that don't have support for
those 3.
Andreas
--
[ signature omitted ]
Message 6 in thread
Thank you very much for such a detailed reply.
> > With ideal cross-platform toolkit I would expect to have one generic
> > interface for all of these cases. But world is not perfect, so there is
> > a lot of troubles. What I want to do is find a way to avoid most or all
> > of them, so I shall have an interface that will enable me to write
> > simple filters without thinking much about on which platform I am, where
> > am I reading from or whatever. For some of the troubles I have found
> > perfect solutions, for some others too, maybe not so perfect though, and
> > for the rest I am at loss.
>
> You can't generalize the different solutions into a single class, because they
> all have different issues and uses. You could make a class that implements
> the shared functionality, but that would most likely be too little to be of
> any use.
Well, I shall try anyway. Surely, some cases just do not have
appropriate underlying API support - well, why not emulate them?
Cygwin emulates POSIX select() call almost perfectly, for example.
> > The simplest way to do that is just wait for some time and check if
> > there is more data available. The problem is that this introduces
> > unnecessary delay. This may be fine while I am just playing around with
> > Open Source Edition at home, but if I manage to make those evil guys
> > buy me a Qt license for my job, then I am probably going to do this on
> > real-time ISS telemetry, where additional delays are not A Very Good
> > Thing(tm), definitely. Especially when there are a lot of filters
> > involved.
>
> If you don't want delays, then saving stuff in files for other applications to
> read is just the wrong approach. You can not do this without delays. Period.
Well, I agree. The funny thing is, we did that for years, and ISS is
still flying. So if those evil guys finally buy me a license I would
probably want someday to integrate some brand-new Qt application into
all this stuff.
> > A way around this is, of course, QFileSystemWatcher. With three notes.
>
> Yes, and if you want a crossplatform solution, this is the way to go. In the
> case of tail -f, this is special because it assumes there will only be text
> added to the file. I don't know if this special case is possible to do with
> some platform specific shortcut.
tail -f works perfectly with binary files too. In fact, launching tail
-f with QProcess and reading from its stdout accomplishes almost
everything I was talking about.
> > First one is that we need to know file name in order to use it.
> > Therefore it will not work if we have stdin redirected from a file.
> > I see no way to work around this. So we have a limitation here.
>
> No we don't. Piping a file to standard in means the application doesn't get
> any files. It gets the contents of the file in stdin. Just like if you do
> cat myfile | myapp.
Yes, if you are talking about piping, this is cat file | ..., but I was
talking about redirecting (myapp < myfile), not piping. This basically
means almost the same thing as opening that file as descriptor 0. We can
not do select() on it, for example, like we do with pipe. For Qt, this
also means that isSequential() returns false and size() returns
true file size, while for pipe it returns some nonsense like internal
buffer size. Consider the following Python script laj.py:
from PyQt4.QtCore import QFile, QIODevice
f = QFile()
f.open(0, QIODevice.ReadOnly)
print f.isSequential(), f.size()
And look at this:
[alqualos@brededor tmp]$ python ./laj.py < ~/video/anime/Tales.From.Earthsea.2006.TC.DiVX-iND.avi
False 888512430
[alqualos@brededor tmp]$ cat ~/video/anime/Tales.From.Earthsea.2006.TC.DiVX-iND.avi | python laj.py
True 65536
> > Second note: our input becomes non-blocking and event-driven even if we
> > do not want that. This is due to absence of
> > QFileSystemWatcher::waitForFileChanged() method. This leads me to
> > thinking that maybe Trolls have gone the wrong way with all these
> > waitFor... methods - maybe they should have just provided us with
> > QObject::waitForSignal(sender, signal) method or alike? Not sure why
> > they did not do that.
>
> I have also asked them for a waitForSignal method. But usually this is for
> incredibly odd situations. In your case, event driven coding is definitely
> the only way to go. stdin is not the only way to access a running program on
> unix - you have the unix signals too.
Well, Unix signals are definitely not for cross-platform programming.
While I sometimes use them to make my program shut down cleanly on
SIGTERM or alike, still even driven coding is not always the best thing
to do: sometimes simple linear main() is just better. For filters, this
is often the case. After all, I can check for signals on each iteration
of my main loop and yet it will be simple as this:
while (!(tm = instream.read()).isNull()) {
if (signalled)
break;
// do something with tm
}
Much better than a bunch of event-driven code. Of course, if there are a
lot of external events to process, this code becomes ugly and then
even-driven programming comes into picture. Everything should suit its
purpose.
> You need to wrap your head around the proper Qt way of doing things. Qt isn't
> an add-on to your application. It is the centerpiece that you fill stuff in
> to. Qt supplies the main loop, and you work with events. That's is The Right
> Way (TM) to do Qt programming. If you don't like this and really only want to
> do implement text filter apps, I suggest you take a look at the std::
> namespace. Sorry if this sounds hard, but that's the way it is.
Qt 3 was this way. Qt 4 seems to offer more. It does not stop me from
using QFile, QTextStream and QString as helper classes - and this is
much better than semi-standard clumsy std:: code.
> > Maybe I
> > should create a separate QThread for watching file and wake up main
> > thread using QWaitCondition to emulate blocking behaviour?
>
> This is overkill.
At first, I thought so too. But if I properly hide it in my custom
class, why not? Overhead will be negligible if I have just one QThread
for this task - after all, filter does not need to work with thousands
of streams simultaneously, it is not a web server! And Qt uses such
tricks, too. For example, they use separate QThread to write data in
QProcess stdin on Windows, and it works fine enough for anybody to
notice!
> > QPollingFileSystemWatcherEngine thing. But the problem is that it uses
> > 1000 ms polling interval without any visible way to change it - it is
> > defined as enum and used locally in private class! Even if I was to
> > override the appropriate method in my derived class, I still see no way
> > to make QFileSystemWatcher use the modified class. It seems that I shall
> > have to reimplement the whole thing from scratch to make it properly
> > work. Why did not they just make QFileSystemWatcher::setPollingTimeout()
> > method? 1000 ms is too much for some tasks, like telemetry, really.
>
> I suggest you file a bug report to the trolls. You should be able to control
> that interval.
Well, actually Qt has a lot of stuff like this. For example, QProcess
(again!) uses QTimer to check for data availability in process' stdout.
Timeout for some reason is set to 100 ms. They even call this "wonderful
windows notifier" or like. Not sure why they did not use OVERLAPPED I/O.
Maybe it has some weird side effects. But still, timeout here is fixed
too. What they really should have done is to expose
QFileSystemWatcherEngine like they did with QAbstractFileEngine. But in
any case this looks more like a feature request than a bug report.
> IIRC, HP-UX has a fam daemon. And my guess is that QFileWatcher on Linux is
> implemented using fam too. So maybe it could be possible for you to implement
> the HP-UX support for proper watching of the files.
No, they do not use FAM. I think it is good, since this is not the most
reliable thing in the world. In fact, what I always liked about Qt is
that they rarely use external libraries except for self-contained stable
ones like libpng. Lately they started to use glib and other things which
I am not sure it is a good thing. For example,
QTextCodec::codecForLocale()->name() always returned local charset name
on Unix, according to Qt docs:
If the codec is registered as a character set in the IANA character-sets
encoding file this method should return the preferred mime name for the
codec if defined, otherwise its name.
Now when they introduced iconv(3) support it just returns "System". Now
I have to disable iconv(3) support in order to get compatible behaviour.
> I think you are making the mistake that because Qt is a cross platform
> framework, it should be usable for you cross platform applications. Sorry
> troll-salespeople, but the case here might very well be that Qt isn't the
> right tool.
>
> You're completely wrong when you say that Qt I/O is broken. It just solves
> another problem set than the one you have. Writing desktop apps with the Qt
> I/O is fine.
Well, again, it was the case in Qt 3. Qt 4 has QtCore that you
mentioned, and they even mention something about QFile usability for
reading from stdin on all platforms in their changelogs - and yet I see
that Qt even treats stdin and 0 differently! Looks more like they
designed I/O API for desktop and then suddenly decided to extend it for
things like stdin. If it was explicitly stated that QFile is only for
files and everything is designed for event-driven desktop programming,
that would be a different story.
By the way, filters are very useful things in desktop programming for
performing service tasks. grep(1) for searching, for example. The Unix
way of doing things, but with Qt nothing stops you from porting this to
Windows or whatever.
> That said, it might be that the trolls could be interested in working with you
> on solving the issues you have here. The Qt core lib is a first step in the
> direction of text based apps (which could be text filters, services, ...),
> and as they are moving Qt in this direction, they need real problems to work
> with. Maybe the Qt I/O problem set could be expanded to include your cases.
Well, all of the cases sounds like impossible, but certainly they could
go at least that far as writing
if ((fd != -1 || fh) && nativeIsSequential())
instead of
if (fh && nativeIsSequential())
I think I shall file a bug report at least for this strange behaviour.
Then I shall try to implement everything I need by myself and see if
there anything else that looks like it belongs in Qt, not in my
extension.
> I don't know if this helps at all :-)
Anything helps, thank you.
--
[ signature omitted ]
