apgsearch v1.0

[Scorbie]

@dvgrn
I should have been clear I was aware of that fast implementation(thx calcyman), but I thought that the format is quite strict and typing that every time would be quite error-prone, so it would be nice if there was a button that automatically does that. (like the [quote]

Code: Select all

[list] tags in this forum.)

[dvgrn]

I should have been clear I was aware of that fast implementation(thx calcyman), but I thought that the format is quite strict and typing that every time would be quite error-prone, so it would be nice if there was a button that automatically does that. (like the

Code: Select all

[list] tags in this forum.)[/quote][/quote]
Yeah, sorry, I kind of figured you had seen those notes, since you mentioned several of the items in the same order.  And I certainly agree about it being potentially error-prone -- half the exploratory Catagolue comments I've posted so far have come out wrong, and there's no mechanism to go back and fix them.  (Thanks, Nathaniel, for allowing editing on previous posts on the forums -- I've been able to fix a very large number of my own silly mistakes...!)

On the other hand, I think most of the pitfalls are documented now, and after a dozen posted comments, I hope that at least my own error rate will be much lower from now on.  Arguably Catagolue syntax is easier to type by hand than forum tags, especially the bold and italics... and if you forget and type something that comes out looking like /non-italics/ or *not quite bold*, well, people will still know what you mean.

An editing function for one's own comments, or at least a delete option, would certainly be nice, though.

[calcyman]

As a test, I sent about 100 soups but it wasn't shown even as a uncommitted haul.

The server rejects hauls of fewer than 80000 soups as a sort of spam-prevention system (it would be a waste of resources to spend a /apgcron operation committing so few soups). Now apgsearch has an internal hardcoded minimum of 500000 soups between successive uploads (but of course pressing 'q' could attempt to send a smaller haul).

[calcyman]

http://catagolue.appspot.com/census/b3s23/D4%20+4

Erm, codeholic, symmetry specifications are supposed to use an underscore, not a space. Consequently, you've just been running 8.5 million soups of C1 symmetry.

The latest version (apgsearch v1.02, available from the usual place) now throws an error if you try an invalid symmetry string.

[Extrementhusiast]

Just a minor quip about the Catagolue website:

This pattern is a oscillator.

Wrong article. Use either a flag to check if an N is needed, or just put a(n) for everything.

[velcrorex]

Does anyone else think that the example soups on object pages should be sorted by symmetry? I ask because, for example, on the object page for the tumbler can you spot the one asymmetric soup out of 41 sample soups?

[dvgrn]

Does anyone else think that the example soups on object pages should be sorted by symmetry? I ask because, for example, on the object page for the tumbler can you spot the one asymmetric soup out of 41 sample soups?

I just noticed that today, about the xs31 that has finally showed up. It's listed as having one occurrence in C1 soups, but when you click on the apgcode link you get to a page with three soups on it. I suspect a lot of high-bit-count symmetric still lifes in asymmetric searches will get drowned out by their more common clones in symmetric searches.

You can point to the various links and quickly find the "C1" one -- or in extreme cases, view page source and search for "/C1" -- but that's all a little awkward. The soups do seem to be grouped by symmetry, so it's probably just a matter of adding a <br> and the new symmetry string every now and then.

[Freywa]

Now I have a performance issue here. On my laptop the (new) program accelerates for the first 20000 soups or so up to around 185 a second and then the speed takes a slow yet excruciating nosedive – certainly undesirable when I'm trying to search 5 million soups. What is causing the slowdown? (In other words: Whose Line Is It Anyway?)

[gameoflifeboy]

Is there a way to make a list of objects combining all the apgsearch hauls people have done for a given rule and symmetry, in order of frequency?

[flipper77]

Is there a way to make a list of objects combining all the apgsearch hauls people have done for a given rule and symmetry, in order of frequency?

As far as displaying this list on catagolue, Adam mentions there's a 1 MB limit for individual pages here.
If possible, it would be nice to have a python script that retrieves all pages for a certain rule and symmetry, combines all the results, and outputs a .txt file listing all objects that have appeared, starting with the most common objects, and ending with the really rare objects including infinitely growing patterns. Unfortunately, I don't have too much knowledge of Python to know if something like this is even possible, but I don't see too much need to see a cumulative census, at least not for now, since catagolue has been online for not even 3 days, even though it would be interesting to see what common, and what's not, considering over 200 million soups have already been searched in such a short period of time.

[calcyman]

Is there a way to make a list of objects combining all the apgsearch hauls people have done for a given rule and symmetry, in order of frequency?

(By the way, I renamed pages to tabulations when I implemented them, to avoid confusion with HTML pages.)

I was thinking of having a tabulation called 'common' in addition to the ordinary tabulations (xp2, xs18, yl144 etc.) which contains only the 10000 most common objects in a particular census. Then this can be updated concomitantly with the other tabulations when hauls are committed (which is now an entirely automatic process performed every two hours, on the hour).

[calcyman]

Now I have a performance issue here. On my laptop the (new) program accelerates for the first 20000 soups or so up to around 185 a second and then the speed takes a slow yet excruciating nosedive – certainly undesirable when I'm trying to search 5 million soups. What is causing the slowdown?

How much free RAM do you have? My laptop had 2 GB total RAM, which was definitely not enough, so I dismantled it to insert an extra 4 GB and now it runs rather smoothly (at the moment, at 128 soups per second per core).

And then there's the famous `Golly scripts run roughly twice as fast when the window is minimised' phenomenon.

[velcrorex]

From the asymmetric life search I see that we're missing only one 10 cell still life (10.15) and two 11 cells (11.31 and 11.41) as numbered here. Even just a count of how many types of n-cell still lifes on each page would help to keep track of what we're still missing.

[gameoflifeboy]

Is there a way to access all the hauls made by a specific user, or at least the hauls that aren't listed in http://catagolue.appspot.com/haul/b3s23/C1?

For example, I was able to access one of my hauls here from the list of most recent hauls, but now it's gone from the list and the only way I can reach it is through my history.

EDIT: Found it here, but that list doesn't list all 119 hauls either so eventually it will be gone from there too.

[flipper77]

So far, the "b3s23/C1" census has accumulated almost 300 million soup results in just a little over 3 days, I'm amazed.

I have a minor suggestion for apgsearch that's not terribly important: have it find what version is available online, and if it doesn't match the version you have, it prompts you to go to the appropriate location to download the latest version if you want to get it. This would be a good thing to have because if the next version fixes a bug of some sort or something else, others will be aware of the newer version.

[gameoflifeboy]

Also, when are the "D4 +4" (a mistyping of D4_+4) censuses getting moved to C1, which is their actual symmetry?

[flipper77]

Also, when are the "D4 +4" (a mistyping of D4_+4) censuses getting moved to C1, which is their actual symmetry?

Yes, C1 is the true symmetry, the post is on this very page, I hope those soups get moved over to the b3s23/C1 census eventually, but I wouldn't know if that's possible. It would be a real shame if those 14 million soups don't get included in that census, even though 14 million pales in comparison to 300 million.

Also, if you don't already have the latest version of apgsearch(v1.02), which accounts for the problem of incorrect symmetry options, go to catagolue's apgsearch section.

[dvgrn]

Is it possible to sort lists of still lifes numerically instead of alphabetically? I got a little worried temporarily about the prevalence of xs9_178426 and xs9_31248go at the very end of haul reports, near all kinds of much rarer-looking objects with 20+ bits.

Really those two nine-bitters are just unusually rare -- all the others are five or ten times more common -- so they're the only ones that there's likely to be only one of, in a multi-million-soup haul. Still, changing the sort order would be nice, both on the haul reports and on the main census pages.

[Nathaniel]

Fantastic work, calcyman I don't have much else to say, but I'm finally running some soups now.

One question: are the URLs (like http://catagolue.appspot.com/object/xq3 ... dhb5/b3s23) fairly stable, or might they change in the coming weeks? I'd like to add some of them as external links in the wiki.

[codeholic]

Is it possible to sort lists of still lifes numerically instead of alphabetically?

I think this might help: http://stackoverflow.com/questions/4836 ... r-18415320

[calcyman]

So far, the "b3s23/C1" census has accumulated almost 300 million soup results in just a little over 3 days, I'm amazed.

And there's just been an overnight explosion to 429 million soups. It's very fortunate that I decided to use 64-bit integers for everything, now that we have too many blocks and blinkers for (signed) 32-bit integers:

http://catagolue.appspot.com/census/b3s23/C1/xp2

I have a minor suggestion for apgsearch that's not terribly important: have it find what version is available online, and if it doesn't match the version you have, it prompts you to go to the appropriate location to download the latest version if you want to get it.

Agreed. It would be nice to auto-update, but Golly imposes an annoying security restriction to prevent one script from executing another.

I hope those soups get moved over to the b3s23/C1 census eventually, but I wouldn't know if that's possible.

It's possible, but I would have to write another servlet to accomplish that.

Meanwhile, the actual b3s23/D4_+4 census has found a genuinely new (i.e. not featured in jslife) p30 shuttle hassler:

http://catagolue.appspot.com/object/xp3 ... 0103/b3s23

I don't have much else to say, but I'm finally running some soups now.

Thanks! I've seen you appear to be using a quad-core processor, so you're generating plenty of results!

One question: are the URLs (like http://catagolue.appspot.com/object/xq3 ... dhb5/b3s23) fairly stable, or might they change in the coming weeks? I'd like to add some of them as external links in the wiki.

Yes, the /object and /census links are stable (the pages themselves may be updated to include more information, but the URLs will remain the same). The /haul endpoints, on the other hand, are likely to be cleaned periodically when I need more room in the Datastore. In particular, there's a 1GB free limit, of which I'm about 1% of the way to reaching, beyond which I would need to pay for extra storage (still very cheap, but nonzero).

Although I may need to enable billing long before then, not due to the storage but rather the amount of load on the server (Catagolue is already really close to the daily limits for read/write operations and CPU time).

[David]

The B36/S125 C1 census listed the double version of the only known 2x2 wickstretcher as period 4, but it is still period 8, doesn't it??

[flipper77]

The B36/S125 C1 census listed the double version of the only known 2x2 wickstretcher as period 4, but it is still period 8, doesn't it??

Yes, but the way linearlyse() works is by checking the population, and finding the period by which the population oscillates instead of the pattern itself, which is why the block-laying switch engine is labeled as period 144 by apgsearch instead of the actual period 288.

[David]

There seems to be a bug. Catalogue doesn't report any uncommitted hauls if there are existing committed hauls in the rule:symmetry. Please verify.

I'm not sure I understand. This screenshot shows a census with five committed hauls and one uncommitted haul:

The attachment committed.png is no longer available

I've also got this error.
[img]error.png[/img]

[dvgrn]

I have a minor suggestion for apgsearch that's not terribly important: have it find what version is available online, and if it doesn't match the version you have, it prompts you to go to the appropriate location to download the latest version if you want to get it.

Agreed. It would be nice to auto-update, but Golly imposes an annoying security restriction to prevent one script from executing another.

Well, but you don't need to have one script executing another to auto-update. There are no Golly security restrictions on writing text files to disk -- even when the file in question happens to have the same path and filename as the Python script that's currently executing...!

This script seems to be able to find and update itself wherever I put it on my Windows 7 laptop. Can somebody check and see if this is equally reliable on other OSes?

script-selfupdate-test.py:

Code: Select all

# simple self-modifying Golly script
import golly as g 
import os
import inspect

filename = inspect.getframeinfo(inspect.currentframe()).filename
scriptpath = os.path.dirname(os.path.abspath(filename))
scriptfname = os.path.abspath(filename)

with open(scriptfname,'w') as f:
  f.write("""import golly as g

g.note("The updated script isn't the same as the old script,\\n   (though it certainly could be, given enough\\n   Hofstadterian self-referential cleverness.)")
""")

g.note("This script, " + scriptfname + ", has updated itself.  Run it again.")

Probably save to your hard drive first, and run from there -- it works just as well when run from clipboard, but golly_clip.py doesn't get added to Recent Files, so it's kind of annoying to go and find it.

-- Of course I could have had the script update the clipboard instead if it found out that its name was "golly_clip.py", but that's not really the point here.

I suppose it would make sense to only check the version when the script is first started, not every time a haul is uploaded. If there's a new version, just overwrite the script file, print the "Run me again!" message, and stop. If the test is run after every haul, then an update might unnecessarily stop somebody's multi-day run on a little-visited computer.

The only downside that I can see is that now I won't be able to sleep at night for fear that someone will hack into Catagolue and replace apgsearch with some malware program that steals all my CPU cycles... hmm, wait, apgsearch is doing that already.