Task Tme Estimates
log in

Advanced search

Message boards : Number crunching : Task Tme Estimates

Author Message
Profile Charles Dennett
Avatar
Send message
Joined: 18 Dec 14
Posts: 88
Credit: 3,342,826
RAC: 0
Message 2420 - Posted: 11 Jun 2015, 12:22:32 UTC

The time to complete a task is estimated by the project. When a task is completed and returned, a correction factor is calculated for the computer that completed the task and is sent back to the computer. (Used to be called the duration correction factor in earlier versions of BOINC but is something different these days). This factor is then used to modify the estimated completion time of the tasks a user will see in BOINC Manager. It is also used by the scheduler to figure out how much work is queued up and if any additional work needs to be requested.

So, a few days ago I saw a lot of tasks on one of my crunchers that were estimated to take zero seconds. Yup, zero. Today I noticed a bunch of tasks on another of my crunchers that were estimated to take 2 seconds. In both cases these tasks actually took 30-60 seconds to complete. But, when they were reported, the project servers noticed the difference between estimated and actual time and adjusted the correction factor. As a result the estimated time of the tasks still in the queue shot way up. Some are estimated to take days. Of course, they won't and the correction factor will slowly come back to normal. A side result of this is that my system won't ask for new tasks because it thinks it has way more than enough queued up. That will change as tasks are completed, returned and the correction factor drops.

I know there was a discussion with another user who did some excellent work on coming up with an algorithm to better estimate tasks completion times and in general it is much much better than it was in the past. However, it may be that this is inaccurate at very small completion times. I just wanted to report my observations in case this user wants to refine the algorithm.

Charlie
____________

Richard Haselgrove
Send message
Joined: 30 May 15
Posts: 25
Credit: 1,979,129
RAC: 1,584
Message 2421 - Posted: 11 Jun 2015, 15:17:12 UTC - in response to Message 2420.

The replacement for the old Duration Correction Factor (maintained locally by the BOINC Client, applied to every task type across the project) is standard BOINC code, and - although I'm new here - I'd be very surprised if any project-specific improvements had been made to it by the project alone. Other projects have tried, and failed.

The adjustment work is now all done on the server, and is kept separately for each host, each application, and each application version. It's very hard for users to see the calculations and adjustments in detail, but the end results can be observed using the application details link for any computer - that happens to be one of mine, but it could be anyone's.

The figure to take note of is the 'Average processing rate', or APR. Each task is given an estimate of 'how big' (how much work needs to be done) by the project, and that used used with the APR to come up with a runtime estimate for that task running on the particular host in question. The details are in Runtime Estimation.

In particular note that for the host I linked, it appears that the 32-bit application is around six times faster than the 64-bit application. That's unlikely: more likely is that the 32-bit app happened to be allocated tasks which finished quickly (as Charles describes), and got adjusted upwards: the 64-bit app may have been processing 'chewier' tasks, and got itself adjusted downwards as a result.

Once a host gets itself into a disparity like that, it's very hard (under this new system) to escape from it. Charles, if you can remember who it was that was working in this area, there are people interested in improvements to the BOINC code who might be interested in hearing from them.

Profile Charles Dennett
Avatar
Send message
Joined: 18 Dec 14
Posts: 88
Credit: 3,342,826
RAC: 0
Message 2422 - Posted: 11 Jun 2015, 18:42:12 UTC

The discussion about calculating task times is located here:

http://findah.ucd.ie/forum_thread.php?id=251

That's in the News forum.

The user who came up with the algorithm is Ananas. This was only for estimating the time for this tasks projects. The project uses molecules of various types and sizes and a tasks run time depends on what specific molecules are used.

I thought a parameter was returned in the project's sched_reply file so that it could be used to recalulate the estimated run times of the queued tasks. It may also have been written to the client_state.xml file for that project but I have yet to discover it. I thought I had it figured out several months ago when I first learned the duration correction factor was no longer used.

Charlie
____________

Profile Charles Dennett
Avatar
Send message
Joined: 18 Dec 14
Posts: 88
Credit: 3,342,826
RAC: 0
Message 2423 - Posted: 11 Jun 2015, 18:47:22 UTC

I found it. In http://findah.ucd.ie/forum_thread.php?id=217 I am initially confused and then figure out what's happening.

Charlie
____________

Richard Haselgrove
Send message
Joined: 30 May 15
Posts: 25
Credit: 1,979,129
RAC: 1,584
Message 2424 - Posted: 11 Jun 2015, 20:20:18 UTC - in response to Message 2423.

Thyme Lawn's reply in that thread is pretty good. He's usually reliable (I know his work from CPDN).

Conan
Avatar
Send message
Joined: 20 Jul 12
Posts: 31
Credit: 451,033
RAC: 0
Message 2428 - Posted: 13 Jun 2015, 11:55:54 UTC

I wouldn't worry too much about the short time estimates as I am getting very long estimates, so far up to 588 hours.
The run time at maximum is below 2 hours.

Conan
____________

Message boards : Number crunching : Task Tme Estimates


Main page · Your account · Message boards


Copyright © 2017 Dr Anthony Chubb