Hacking apgsearch

[muzik]

I like asking stupid questions:

Could you mod apgsearch 3.x?

[praosylen]

Could you mod apgsearch 3.x?

In theory, I think so. In practice, I don't know enough of the low-level and hardware-specific tricks used in 3.x to do that, and it doesn't even compile using any compiler I can figure out how to get on my old Mac. It would just be far easier for me to write something completely from scratch. Anyone that knows how, though, please feel free to make a mod of 3.x (assuming it's actually possible and there's not some stupid hang-up I'm missing).

[Scorbie]

I like asking stupid questions:

Could you mod apgsearch 3.x?

I thought semitotalistoc rules are supported by 3.x, no? I **think** that it is possible by merely tweaking rule2asm.py..

[muzik]

I like asking stupid questions:

Could you mod apgsearch 3.x?

I thought semitotalistoc rules are supported by 3.x, no? I **think** that it is possible by merely tweaking rule2asm.py..

not that I know how to do that.


I haven't tried searching non totalistic rules with it, but checking hauls submitted for them shows that pretty much everyone uses the python version.

[praosylen]

I thought semitotalistoc rules are supported by 3.x, no? I **think** that it is possible by merely tweaking rule2asm.py..

They aren't; apparently the boolean circuit corresponding to some rules ended up containing too many gates or something (I don't really understand this part especially well), so it was decided against. Tweaking rule2asm.py might be possible, but I don't know asm, so I'm not the one to test it, especially on my particular old computer. There may also be some other sections of the script (I would guess at least pseudo-object separation) that require knowledge of the particular rule, so those would take more work to mod.

[drc]

Can anybody have apgmera print every new object that appears:

New object: xp2_7
New object: xs4_33
New object: xq4_153
.
.
.
10000 soups completed (99999999 per second)

[Apple Bottom]

Can anybody have apgmera print every new object that appears

Wrong thread? I think this one's really for apgsearch 1.x, not apg(mera|nano). That said: patch here.

[wildmyron]

From the apgsearch v1.0 thread,

It's having a hard time searching B2e3/S23-j, an extremely stable rule. What's going on (and I said heyyeyaaeyaaaeyaeyaa)

I think you meant to post this on the Hacking apgsearch thread, but in any case something is going wrong with the rule parsing. When I tried a search with apgsearch-2016-2-06-v0.54+0.21i-update.py it was actually trying to search B23/S23 - doomed to failure. Searching works fine when I used my version of non-totalistic apgsearch (apgsearch-isotropic-v0.2.py) The search runs as expected. (No upload capability though.) I have no idea what's going wrong there.

Btw, that's a nice rule, lots of variety in the natural spaceships.

Did either of you generate the rule table before searching the rule? This is a drawback of 0.21i: it does not autogenerate the rule table for you; you have to use isotropic-rulegen.py to make the rule and enter it using an underscore, not a slash. I'll try to improve this when I get back from vacation in three weeks, or sooner if I can.

I had generated the rule, in particular it was generated by apgsearch-isotropic-v0.2.py while testing the rule. However, I specified the rule as "B2e3/S23-j" to both programs, not "B2e3_S23-j" as you say is required. This works fine in my script and I'd forgotten that the other script requires the exact name of an existing rule file. When I use the '_' the correct rule is processed but like @drc, soup search is very slow (~3 soups/s, increasing to ~7 soups/s if I reduce the sqrtspp_optimal value to 6.

The reason for this is down to the way apgsearch does periodicity detection and the length of time the Coalesce rule is run to ensure high period objects are correctly identified. In this case, p8 and p16 objects are relatively common, but the default periods which are detected by the naive stabilisation detection only include factors of 12 and 30. So even though most soups stabilise quickly, this is not detected in soups with a p8 or p16 object.* If one soup in the page has such an object, it is implicitly assumed there may be a high period object and the coalesce rule is run for much longer than necessary. This is not a big issue in terms of evolving the soup in the auxiliary rules, but all the ships leave a long trail of "dead but has been alive" cells. I believe the slowdown occurs because the final population of the universe before object identification occurs is very large, and everyone of those "dead but has been alive" cells is grabbed by the gridobj() function as part of the ship, as well as being scanned after the object has been removed. From my previous experience this explanation doesn't quite seem to explain the drastic slowdown, but when I make the following change to stabilise3(),

Code: Select all

        # Phase I of stabilisation detection, designed to weed out patterns
        # that stabilise into a cluster of low-period oscillators within
        # about 6000 generations.

        if (self.naivestab2(12, 10)):
            return 4;

        if (self.naivestab(16, 30, 200)):
            return 4;

        if (self.naivestab(32, 30, 200)):
            return 5;

i.e. change the periods given to naivestab() from 12 to 16 and from 30 to 32, then the search speed jumps to over 150 soups/s.

I was running a test set to upload which I was going to cancel because of the plethora of un-separated small gliders, but accidentally uploaded it. At least there's a nice p16 puffer to compensate for all the noise in the p2 ships category.

[drc]

Whenever I try to search rules (i.e. B34et/S23-a4eit6) in the neighbourhood of CommonRepl, it always messes with my computer. It'll be going smoothly, then it will hit a pattern like this:

Code: Select all

x = 3, y = 7, rule = B34et_S23-a4eit6
b2o$2bo$b2o2$3o$2bo$b2o!

Then my ram explodes and chrome (playing youtube videos in the background) crashes. I don't know why this is.

[Apple Bottom]

Whenever I try to search rules (i.e. B34et/S23-a4eit6) in the neighbourhood of CommonRepl, it always messes with my computer. It'll be going smoothly, then it will hit a pattern like this:

Code: Select all

x = 3, y = 7, rule = B34et_S23-a4eit6
b2o$2bo$b2o2$3o$2bo$b2o!

Then my ram explodes and chrome (playing youtube videos in the background) crashes. I don't know why this is.

A random stab in the dark: what are your memory limits in Golly (in File->Preferences->Control)? Might be that you're running out of memory and the OOM killer is kicking in.

[Gamedziner]

Whenever I try to search rules (i.e. B34et/S23-a4eit6) in the neighbourhood of CommonRepl, it always messes with my computer. It'll be going smoothly, then it will hit a pattern like this:

Code: Select all

x = 3, y = 7, rule = B34et_S23-a4eit6
b2o$2bo$b2o2$3o$2bo$b2o!

Then my ram explodes and chrome (playing youtube videos in the background) crashes. I don't know why this is.

Is it just me, or are those tiny Knightships? Anyways, your computer may just be having trouble with infinite growth.

[Apple Bottom]

I just tried B34et/S23-a4eit6 myself, and hit one of those soups after about 2000 soups. The culprit is the error-correcting phase in apgsearch; Golly's configured memory limits don't apply, likely because the allocations are done in Python as opposed to as part of Golly's own algorithms.

Golly kept hovering somewhere between 16 and 17 GB of allocated memory here, but did not go beyond that, and the machine otherwise remained usable. I'll leave it running for a while to see whether it's going to go anywhere, but it's quite slow as well (error correction's took almost 25 minutes), so whether it'll ever finish 10k soups and submit results to Catagolue remains to be seen. (It's already hit a similar pattern again almost right after.)

The soup in question's this, BTW:

Code: Select all

x = 16, y = 16, rule = B34et/S23-a4eit6
obbbbbobboooobob$
oobbobobbbobobbo$
boooobbbobboobob$
bbooboobbbbobobo$
obbooboboboobbbb$
bbbboobbbboboboo$
obbooobbobboobbo$
boooooboobbooobo$
bbbbobbooboooobb$
ooooobbbbbbooobo$
boobooobooboobbb$
boboobbbbbobobbo$
oobbbobbbbobbbbo$
obooobbbooooobob$
boobbbbboobobobo$
bobbbbboboooooob!

Apparently it produced no less than 15 PATHOLOGICALs, in addition to a zz_LINEAR.

[drc]

16 and 17 GB

I have 4 GB... oh well, it can run most things just fine.

[Apple Bottom]

I have 4 GB... oh well, it can run most things just fine.

It gets better; apgsearch hit a soup where two of these patterns interact, and Golly's currently using about 21.5 GB of memory. (That's actually more than I have in this machine, but so far everything's still running fine. Fingers crossed.)

EDIT: here's the soup:

Code: Select all

x = 16, y = 16, rule = B34et/S23-a4eit6
booobobbbbooboob$
bbbooboobobbobob$
bboobboboobobboo$
oobooboobbooboob$
oooooobboobbbbbo$
bbbobooooboobobb$
bobobobobboooooo$
ooboobbooboooobb$
bbbobobbbbobbbob$
ooobobobbobboooo$
oboooboobobooooo$
ooboobobboobobbo$
bobbooobbobbobbb$
obobobbbboboobob$
obbboboobooobbob$
oooobbobbbbbbbbo!

Let this run for a while (in HashLife), it's really quite neat.

[muzik]

..how do you run non totalistic rules in HashLife?

[Apple Bottom]

..how do you run non totalistic rules in HashLife?

Good question! I'm honestly not sure. Let me clarify the above saying "run it using hyperspeed", which seems to be using some sort of hash-based algorithm -- the pattern ran to a billion generations (with a population to match) in just a few minutes.

Unrelated side note: Golly's now up to using 25 GiB of memory, and browsing the web on the same machine has become noticeably more laggy. Perhaps I should invest in some more RAM.

[muzik]

Does this still work with 2.8?

[praosylen]

Does this still work with 2.8?

I don't know, but I doubt it's terribly difficult to fix. Try it, and if it doesn't work, just go through the source code -- you can skip the long dictionary of object names and scores -- and add the line

Code: Select all

g.setalgo("RuleLoader")

before every call to g.setrule as a dirty fix. It may take a while to find them all, but it should work (hopefully) after that.

P.S. You can actually use find and replace to do the same thing, at least on good text editors. Just replace all occurrences of

Code: Select all

g.setrule(

with

Code: Select all

g.setalgo("RuleLoader")
g.setrule(

.

[Saka]

I'm trying to add size symmetries for APG 1. They are L (32x32), S (8x8), and 1D (1 x 20). How do I do this?

[wildmyron]

I'm trying to add size symmetries for APG 1. They are L (32x32), S (8x8), and 1D (1 x 20). How do I do this?

Start by modifying the hashsoup function to generate soups with the desired "symmetry" For 32x32 you will need more random bits than SHA256 gives you. SHA512 gives you double, but you still need more. You can do this by calculating multiple hash values from slightly modified versions of the input string, e.g:

Code: Select all

def hashsoup(instring, sym):

    s = hashlib.sha512(instring+'a').digest() + hashlib.sha512(instring+'b').digest()
    
    ...

For 8x8 you need less. You can just use as much as you need, though obviously it's a bit wasteful. For 1x20 there are only just over a million configurations. You can iterate through them all without using any randomness.

Then you need to generate the soup from the random string. You can do this by adjusting the modular arithmetic for the 16x16 case to suit the new soup size.

The real question is: Why? What are you hoping to find that you otherwise won't?

[Saka]

@wildmyron This is my first attemp at "real" apg hacking. How do I use rhe script you gave? I dont get it as of now

[wildmyron]

@Saka: Well, I only quoted a small script fragment to show how you can generate the required random string. Have a good look at how the original apgsearch hashsoup function worked. I suggest that you special case the generation of the soup for your custom "symmetries" rather than trying to adapt the hashsoup function in apgsearch v1. There are also quite a few other places where small changes need to be made to accommodate new symmetries. I think it's unlikely anyone other than you will make these adaptations because you still haven't answered the question: Why?

[Saka]

you still haven't answered the question: Why?

I answered it already: I just want to try this as my first "real" hacking attempt

[wildmyron]

you still haven't answered the question: Why?

I answered it already: I just want to try this as my first "real" hacking attempt

Well, in that case: Carry on!

I suggest you start with a standalone version of hashsoup() for each of your desired symmetries. Then you can work on integrating it into the rest of apgsearch.

[Apple Bottom]

Start by modifying the hashsoup function to generate soups with the desired "symmetry" For 32x32 you will need more random bits than SHA256 gives you. SHA512 gives you double, but you still need more.

On that note, SHA512 would yield enough for 20x20 soups -- might be interesting to investigate those since that's what Nathaniel's census used.

Alternatively, it would also be precisely enough for 16x32, which would be interesting to investigate for much the same reason that 8x32 is.