Chopping a job up in knight2 and distributing it across several scalar CPUs
Over the years I have primarily used UNIX head, tail, and cat to chop jobs up. This was highly manual but workable. Under DOS this is way too much of a pain. So last weekend I modified a few lines of code in knight2.c to make a somewhat easier to distribute the chopped up jobs. I haven't actually made the modifications on the code on this machine, so I am going to make them right now and describe what I am doing. I don't want to just post the whole source code because I have also made a variety of other modifications, none of which I have documented.
Near the top of the code add a couple of variables:
At roughly line 203 in the processing of command line inputs change
to
Code: Select all
case 'i':
INTREE = 1;
--argc;
sscanf(*++argv, "%d", &BASEIN);
--argc;
sscanf(*++argv, "%d", &CHOPIN);
break;
At about line 500 change
to
Code: Select all
if (INTREE) onroad += BASEIN;
firsthit = onroad;
What this is doing is skipping over the first "BASEIN" entries of the read in search tree prior before starting the first phase of searching. Later we will turn off INTREE.
At about line 2065 change
to
Code: Select all
onroad++;
if (INTREE && (onroad >= (firsthit + CHOPIN))) onroad = offroad;
This is checking to see if at least "CHOPIN" entries have been processed on this first processed phase, and if so skip over the rest of the entries and set the termination condition for the phase processing.
At about line 2068 change
to
Code: Select all
INTREE = 0;
tempcpt = glist + 2*PERIOD;
After the first phase processed is finished, turn off INTREE.
To use this suppose you run
Code: Select all
knight2 -p 4 -d 6 -o -t -w 11 > kn2out_1o4o_11
After about 15 phase you think it is going to slow and want to parallelize it. First kill your original job. Then look at the last #D line which might look like this
Code: Select all
#D 15 0 138365390 0 114825173 0 93710064 0 313203922 0 21022011 149266 40100
The "40100" is the number of entries in the last recorded phase. What I do is open a bunch of command prompt windows in DOS and in each window I submit a cut up job as follows:
Code: Select all
type kn2out_1o4o_11 | knight2 -p 4 -d 6 -o -i 0 5000 -t -w 11 > kn2out_1o4o_11_16a
Code: Select all
type kn2out_1o4o_11 | knight2 -p 4 -d 6 -o -i 5000 5000 -t -w 11 > kn2out_1o4o_11_16b
.
.
.
Code: Select all
type kn2out_1o4o_11 | knight2 -p 4 -d 6 -o -i 40000 5000 -t -w 11 > kn2out_1o4o_11_16i
Code: Select all
type kn2out_1o4o_11 | knight2 -p 4 -d 6 -o -i 45000 5100 -t -w 11 > kn2out_1o4o_11_16j
There are probably better ways to work DOS, but I don't know them.
In UNIX I just put an "&" after each command and run them in the background.
You do not want to have more jobs running than you have logical processors on your machine. In fact you generally want to leave at least one logical processor free, because "knight2" sticks to processors much better than the many other processes (in particular internet browsing).
It does work to take advantage of hyperthreading. On a CORE I7 I run 7 jobs simultaneously and things seem to be working well.
Have a happy day,
-Tim Coe