Qt-interest Archive, March 2008
thread pool and client distribution
Message 1 in thread
Hi,
so i'd like to have this server use threads. Mainly becouse i want it to take
advantage of multicore machines. How would you distribute clients to the
threads? just open idealThreadCount() threads and round robin? how would you
handle shared resources.
I'm asking becouse there might be something ready made around for this quiet
common problem. qtconcurrent doesn't look like the right choice for a
server.
thanks.
--
[ signature omitted ]
Message 2 in thread
Hi,
On Thursday 20 March 2008 10:32, Arvid Ephraim Picciani wrote:
> so i'd like to have this server use threads. Mainly becouse i want it to
> take advantage of multicore machines. How would you distribute clients to
> the threads? just open idealThreadCount() threads and round robin?
A common approach is to create a pool of worker threads (or create them as
needed up to some maximum number) and to distribute the client connections
among them. The number of threads and how you distribute the work amongst
them is highly dependent upon the type of work that the threads will be
doing. I.e. if they are all doing CPU intensive tasks then there is perhaps
reason to limit the thread pool to the number of available cores and to queue
up the requests beyond this number.
However, if some threads will be largely sitting around waiting for some kind
of disk IO or network IO then it may be sensible to allow a greater number of
threads.
> how would you handle shared resources.
What kind of shared resources?
Cheers,
Sean
--
[ signature omitted ]
Message 3 in thread
On Thursday 20 March 2008 11:45:32 Sean Harmer wrote:
> A common approach is to create a pool of worker threads (or create them as
> needed up to some maximum number) and to distribute the client connections
> among them. The number of threads and how you distribute the work amongst
> them is highly dependent upon the type of work that the threads will be
> doing. I.e. if they are all doing CPU intensive tasks then there is perhaps
> reason to limit the thread pool to the number of available cores and to
> queue up the requests beyond this number.
> However, if some threads will be largely sitting around waiting for some
> kind of disk IO or network IO then it may be sensible to allow a greater
> number of threads.
no blocking io at all. the entire server is built on events and nonblocking
calls. which is exactly why i dont actually need threads in the first place.
but multicore.... yeah it's cpu intensive. and ram. My main question was
actually if there is something ready built for that.
> > how would you handle shared resources.
> What kind of shared resources?
well its a database server. so it needs to lock on write. i fear that a lock
might block all clients on the thread.
--
[ signature omitted ]
Message 4 in thread
On Thursday 20 March 2008 11:45:32 Sean Harmer wrote:
> What kind of shared resources?
right, i forgot to mention. the data comes from any QAbstractItemModel. (with
live updates and whatn not)
and they are known to leave an intermediate state. so isolation is in
jeapardy when using threads.
--
[ signature omitted ]
Message 5 in thread
Hi Arvid,
----- "Arvid Ephraim Picciani" <aep@xxxxxxxxxxxxxxx> skrev:
> no blocking io at all. the entire server is built on events and
> nonblocking
> calls. which is exactly why i dont actually need threads in the first
> place.
> but multicore.... yeah it's cpu intensive. and ram. My main question
> was
> actually if there is something ready built for that.
As far as I know there are two existing products for Qt that does this.
One is the FEAST client server framework:
http://trolltech.com/partners/directory/06fdc792981e9681e38804e040745a4a
http://www.clausmark.com/feast_en.phtml
The other is Qt ORB:
http://www.thorsen-consulting.dk/tcorb.html
I suggest you have a look at both and decide which one that best fits your requirements.
> > > how would you handle shared resources.
> > What kind of shared resources?
> well its a database server. so it needs to lock on write. i fear that
> a lock
> might block all clients on the thread.
>
At least with FEAST you will have one thread per client connection (automatically allocated from a thread pool). Each thread will then be allocated a different DB connection from a DB connection pool.
Now in theory everything should be fine and the calls are non-blocking as long as you don't use transactions.
However this requires the database driver to be reentrant. So pls. check this for your DB!
> --
> best regards
> Arvid Ephraim Picciani
>
> --
> To unsubscribe - send a mail to qt-interest-request@xxxxxxxxxxxxx with
> "unsubscribe" in the subject or the body.
> List archive and information: http://lists.trolltech.com/qt-interest/
--
[ signature omitted ]
Message 6 in thread
On Thursday 20 March 2008 15:05:09 you wrote:
> One is the FEAST client server framework:
> http://trolltech.com/partners/directory/06fdc792981e9681e38804e040745a4a
> http://www.clausmark.com/feast_en.phtml
we talked about it, if you remember me. i'm the QxtWeb author. (competing
open source product)
> The other is Qt ORB:
> http://www.thorsen-consulting.dk/tcorb.html
> I suggest you have a look at both and decide which one that best fits your
> requirements.
none. feast uses stateless connections and polling dbs afaik. Thats a complete
different idom. With Polling db i mean you ask for the information at the
time you display it. Like typical php applications.
Thats something QxtWeb tries to avoid from its base. Polling happens only in
the statefull->stateless adapter. like you got a QLabel and use setText()
instead of having a callback that recalculates the value each redraw. It's
more natural for applications. Besides i'm one of those wierd people who
believe http was never designed for applications.
Carrier uses a realtime mapping of QAbstractItemModel indicies. no polling,
statefull, lazy fetching, etc. and it's open source.
It's got nothing to do with QxtWeb actually. Carrier is just a database, its
not even part of Qxt. But yes i plan using it for QxtWeb applications, since
they use the same idoms (realtime, statefull, etc)
tcorb seems to be exactly the same as QxtRPC, just closed source and without
metahacks, i don't see how that would be related to my question.
> At least with FEAST you will have one thread per client connection
> (automatically allocated from a thread pool).
well, thats exactly what i try to avoid, becouse network is nonblocking.
although it would apear to solve a lot of problems. maybe i'm just to
ambitious. *shrug*
> Each thread will then be
> allocated a different DB connection from a DB connection pool. Now in
> theory everything should be fine and the calls are non-blocking as long as
you don't use transactions.
you don't lock for writing?
> However this requires the database driver to be reentrant. So pls. check
this for your DB!
QAbstractItemModel is definatly not reentrant ;D
Carrier is suposed to take any model and map it. Means i can't make any
assumptions.
--
[ signature omitted ]
Message 7 in thread
----- "Arvid Ephraim Picciani" <aep@xxxxxxxxxxxxxxx> skrev:
> On Thursday 20 March 2008 15:05:09 you wrote:
> > One is the FEAST client server framework:
> >
> http://trolltech.com/partners/directory/06fdc792981e9681e38804e040745a4a
> > http://www.clausmark.com/feast_en.phtml
>
> we talked about it, if you remember me. i'm the QxtWeb author.
> (competing
> open source product)
Ah, yes now I remember. Sorry about that.
>
> > The other is Qt ORB:
> > http://www.thorsen-consulting.dk/tcorb.html
> > I suggest you have a look at both and decide which one that best
> fits your
> > requirements.
>
> none. feast uses stateless connections and polling dbs afaik.
Correct.
> Thats a complete
> different idom. With Polling db i mean you ask for the information at
> the
> time you display it.
> Like typical php applications.
> Thats something QxtWeb tries to avoid from its base. Polling happens
> only in
> the statefull->stateless adapter. like you got a QLabel and use
> setText()
> instead of having a callback that recalculates the value each redraw.
> It's
> more natural for applications.
It is an interesting concept!
> Besides i'm one of those wierd people
> who
> believe http was never designed for applications.
True, the main reason we use HTTP it is for interoperability. However there are other protocols in the works.
> > At least with FEAST you will have one thread per client connection
> > (automatically allocated from a thread pool).
>
> well, thats exactly what i try to avoid, becouse network is
> nonblocking.
> although it would apear to solve a lot of problems. maybe i'm just to
>
> ambitious. *shrug*
>
> > Each thread will then be
> > allocated a different DB connection from a DB connection pool. Now
> in
> > theory everything should be fine and the calls are non-blocking as
> long as
> you don't use transactions.
>
> you don't lock for writing?
Not in the framework. Often you don't need locking and we have used FEAST without locking with several databases without problems. If you really require locking you are of course free to open a transaction.
>
> > However this requires the database driver to be reentrant. So pls.
> check
> this for your DB!
>
> QAbstractItemModel is definatly not reentrant ;D
> Carrier is suposed to take any model and map it. Means i can't make
> any
> assumptions.
Ok, that is a different situation indeed.
>
> --
> best regards
> Arvid Ephraim Picciani
--
[ signature omitted ]
