Qt-interest Archive, January 2008
synchronize two parsing processes in SAX?
Message 1 in thread
Dear SAX developers,
I have to use SAX, not DOM parser to handle two huge XML files at the
same time and synchronize the two parsing processes.
Is there a mechanism in Qt to let the two parsing processes talk each
other, suspend/resume each other?
For example, when Handler-A meets a target tag in startElement, the
processing will be suspended to wait Handler-B reaches the same tag in
the other file processing. Then, they two exchange message and resume
each other until they meet in next intersection.
Any idea?
Thanks
Lingfa
--
[ signature omitted ]
Message 2 in thread
Lingfa Yang wrote:
> Dear SAX developers,
>
> I have to use SAX, not DOM parser to handle two huge XML files at the
> same time and synchronize the two parsing processes.
>
> Is there a mechanism in Qt to let the two parsing processes talk each
> other, suspend/resume each other?
>
> For example, when Handler-A meets a target tag in startElement, the
> processing will be suspended to wait Handler-B reaches the same tag in
> the other file processing. Then, they two exchange message and resume
> each other until they meet in next intersection.
Your description looks like a decent approach to the design.
a) You have two threads, each with its own parser
b) You have a blocking message passing mechanism for threads
c) When handler A sees the target tag, it calls a blocking
receive message method and waits for handler B to send a
message.
d) When handler B sees the target tag, it calls a blocking
receive message method, etc
You will of course have to worry about the possible
deadlock problems of such an approach.
--
[ signature omitted ]
Message 3 in thread
Stephen Collyer wrote:
> Lingfa Yang wrote:
>
>> Dear SAX developers,
>>
>> I have to use SAX, not DOM parser to handle two huge XML files at the
>> same time and synchronize the two parsing processes.
>>
>> Is there a mechanism in Qt to let the two parsing processes talk each
>> other, suspend/resume each other?
>>
>> For example, when Handler-A meets a target tag in startElement, the
>> processing will be suspended to wait Handler-B reaches the same tag in
>> the other file processing. Then, they two exchange message and resume
>> each other until they meet in next intersection.
>>
>
> Your description looks like a decent approach to the design.
>
> a) You have two threads, each with its own parser
>
Yes.
> b) You have a blocking message passing mechanism for threads
>
Yes.
> c) When handler A sees the target tag, it calls a blocking
> receive message method and waits for handler B to send a
> message.
>
Yes.
> d) When handler B sees the target tag, it calls a blocking
> receive message method, etc
>
Yes.
> You will of course have to worry about the possible
> deadlock problems of such an approach.
>
SAX is a serial stream-like unidirectional event-driven parser. In
thread talk, I will choose Qt::BlockingQueuedConnection (introduced in
Qt 4.3). I do not see the "deadlock problems", but I will carefully
check the logic. Thanks.
Regards,
Lingfa
--
[ signature omitted ]
