Tutorials/Contributing to Catagolue

Flowchart explaining how to choose the right apgsearch version (click to enlarge)

So you've decided to contribute to Adam P. Goucher's Catagolue? Excellent! Are you familiar with cellular automata yet? If no, you may want to read the rules tutorial first, but this is optional.

Generating a Payosha256 key

If you want to contribute anonymously, you can skip this section.

If you want to contribute under your name or a pseudonym, you will have to create a Payosha256 key. (Payosha256 is a protocol based on Adam Back's HashCash proof-of-work system).

In order to generate a Payosha key, you need to be logged into a Google account; create a new one if you don't have one or don't want to use your existing one. If you can't or don't want to do this, you'll not be able to contribute to Catagolue non-anonymously.

Go to Catagolue's Payosha256 page, log in using the login link, and authorize Catagolue. Once you're logged in and back on Catagolue, create a new key/displayed name pair. The key itself can be chosen by you and should be kept safe, the same way a password would. Contributions made using the key will carry the displayed name you provided.

You can create as many keys as you'd like. (But you won't need more than one.)

The unofficial Twitter bot

Significant discoveries in Conway's Game of Life (B3/S23) are automatically reported on the independently-operated conwaylife Twitter bot account, and credited to you if they're from asymmetric soups. If your displayed name starts with an @ sign, it will be considered a Twitter username.

If your displayed name does not start with an @ sign, codeholic (the maintainer/operator of the Twitter bot) can map your displayed name to your Twitter user name. Send him a PM on the Conway Life forums (or otherwise get in touch with him) to have this done.

Your Catagolue user page

Your Catagolue user page tracks your contributions for B3/S23/C1, awards badges for certain achievements, and displays your findings.

Unlike on the unofficial Twitter bot account, your findings include any of the first 20 occurrences of any oscillator with period greater than 2, and any spaceship or yl pattern. For oscillators with period 2 and "interesting" still lives (above a certain population or precursors of important patterns), only the first occurrence is credited. Other still lives, zz patterns and PATHOLOGICALs are not reported.

The badges are:

• Conchita: find a soup containing a phoenix.
• Gemini: discover a new twin bees shuttle variant.
• Gigamyriad: contribute 10¹³ objects.
• Hitchhiker: find a soup containing a Kok's galaxy.
• Limitless: observe a new natural infinite-growth pattern.
• Monarchist: discover a new queen bee shuttle variant.
• Sprotsmanship: contribute one third of a trillion objects to a different rule or symmetry.
• Trillionaire: contribute one trillion objects.
• Voyager: find one of the first twenty occurrences of a spaceship.

Initiating a soup search

Start by choosing the right version of apgsearch as indicated in the flowchart at the top of this page.

• If you want to look at non-totalistic rules, you'll need either apgluxe 5.x (command line) or the hacked version of apgsearch 1.x.
• If your CPU is not x86_64 (all modern Intel/AMD CPUs are), or does not support at least SSE2 (most modern Intel/AMD CPUs do), you'll need apgsearch 1.x (regular or hacked).
  • You're not sure what your CPU can do? Take a look at /proc/cpuinfo on Linux, or use a tool like CPU-Z on Windows.
• If you do not have access to Unix-ish command line (Cygwin is fine), or you don't feel comfortable compiling C/C++ code, you'll need either apgsearch 1.x (regular or hacked) or apgluxe 5.x (precompiled).
• If you want to search other rules or symmetries besides B3/S23/C1 using apgluxe 5.x, you'll need the command line version rather than the precompiled version.
• If you want to search Generations or Larger than Life rules, you'll need the command line version of apgluxe 5.x. (Note that since this requires a reasonably modern x86_64 CPU, you cannot currently search such rules if you don't have a sufficiently modern computer.)

The older Python version 0.x is deprecated, as are apgnano (2.x) and apgmera (3.x) since they are superseded by apgluxe 5.x.

Soup-searching with apgsearch 1.x

You will need:

1. Golly
2. Python 2.7 (currently 2.7.13)
3. apgsearch 1.1.

If you want your contributions attributed to you, be sure to generate a Paoysha256 key (see above).

Install Golly and Python 2.7 if you haven't yet. Newer Python versions (3.x) will NOT work! Be sure to use a 32-bit Python if you're using a 32-bit Golly, and a 64-bit Python if you're using a 64-bit Golly. Also note that Python 2.7.11 is incompatible with Golly.

To install apgsearch, copy it into Golly's scripts/python directory. You may also want to edit the apgsearch script to default to using your Payosha256 key.

To search, simply start Golly and click on the script to run it.

Local logging is enabled by default; on Windows, you'll find logs in %APPDATA%\Golly\apgsearch\progress. On Linux/Unix-like systems, it's likely ~/.golly/apgsearch/progress (or something similar). Logs are named e.g. search_b1838fd357f0b003f985c20d19fd664a.txt.

If you want to search a different rule, it's recommended that you quit and restart Golly; this is due to Python scripts not releasing all their memory when they finish.

Soup-searching non-totalistic rules

You will need:

1. Golly and Python (as above)
2. A for awesome's hacked version of apgsearch (currently at version 0.54+0.32i)
3. If you're using a Golly version prior to 2.8, wildmyron's isotropic-rule.py.

Install everything the same way as above.

To search:

1. Choose a rule.
2. Run the hacked apgsearch by clicking on it; when prompted, enter the rule like this: "B3/S234c".

As with semi-totalistic rules, it is recommended that you quit and restart Golly if you want to search a different rule.

Soup-searching using apgluxe 5.x

Unix command line

You will need:

1. A computer with an x86_64 CPU and a 64-bit OS.
2. A Unix-ish command line (Cygwin works if you're on Windows, and WSL for Windows 10).
3. A C/C++ compiler and make tool.
4. Python (apgluxe is compatible with both Python 2 or Python 3).
5. apgluxe 5.x (current version: 5.09-ll2.2.12).

If you're on Windows, install the 64-bit version (!) of Cygwin (any mirror works fine). Be sure to select the "gcc-g++", "git", "make" and "python" packages. Include their dependencies when asked to.

To download the repository and compile apgluxe, run the following commands:

$ git clone https://gitlab.com/apgoucher/apgmera.git
$ cd apgmera
$ ./recompile.sh

The recompilation script will automatically pull in the latest lifelib version from Gitlab if necessary. The resulting binary will support SSE2, AVX1 and AVX2, and use the best instruction set your machine supports. You must recompile the program each time you want to change the rule or symmetry; it may be advisable to keep different binaries on hand.

To execute apgluxe, execute it from the command line like so:

$ ./apgluxe -n 10000000 -k my_payosha256_key

When running apgluxe, supply the options you want and note that local logging is DISABLED BY DEFAULT and must be enabled by passing "-L 1" on the command line; again, the "1" is mandatory. Logs are named according to the current Unix timestamp and seed root, such as log.1457882181.m_cDbsaQswZHeL.txt. To perform an offline test without uploading to Catagolue, include "-t 1".

apgluxe also allows you to parallelise the search over several cores, essentially creating multiple instances of apgsearch which all contribute to the same haul, significantly speeding up the search. You can enable this by including, for instance, "-p 4", to parallelise over 4 cores. However, this has been found to not work with Cygwin, so you'll have to open multiple separate command prompts and run a non-parallelised search with the same parameters on each of them in that case.

You may want to create a wrapper script to call apgluxe with your desired options. You can also pass rulestring and symmetry options; if these do not match those currently compiled into the current executable, the recompilation script will be invoked automatically.

$ ./apgluxe -k my_payosha256_key --rule b38s23 --symmetry C1

You can fetch upstream updates to apgluxe by running:

$ ./recompile.sh --update

This will restore the lifelib working directory to b3s23, and pull any updates from the remote repository.

Precompiled version

The "apgsearch" page on Catagolue contains a link to download a precompiled .exe of the latest version of apgluxe, compiled to search B3/S23/C1. If you want to search a different rule or symmetry, you'll need the Unix command line version.

You will need:

1. A computer with an x86_64 CPU and a 64-bit OS.

When you run the executable, you'll be asked to provide search parameters in the following order:

$ Please enter number of soups per haul (minimum 10000000):
$ Please enter payosha256 key (e.g. '#anon'):
$ Please enter number of CPU threads to use (e.g. 4):

For each of these, type in a value and press Enter and the search will begin afterwards. You may notice that various pieces of text contain odd non-ASCII characters; these are intended to be color codes but they aren't compatible with the default Windows command prompt, so they just appear as plain text:

$ Linear-growth pattern detected: �[1;32myl144_1_16_afb5f3db909e60548f086e22ee3353ac�[0m

When the search is running

You'll receive updates about the progress of the search every 10 seconds (when non-parallelised) or every 100,000 soups (when parallelised). Occasionally, you'll also see the apgcodes of rare oscillators and spaceships (B3/S23 only) as well as infinite growth patterns (in all rules) that appear in your search.

There are two ways to stop a soup search. In the Unix version, you can press Q to immediately submit the current haul to Catagolue and end the search. For both the Unix version and the precompiled version, you can also press Ctrl+C to stop it, but this does not upload the results to Catagolue and instead throws away the haul.

Caveats

The executive summary of the executive summary

Slow down cowboy!

The executive summary

1. Run test hauls.
2. Mind Catagolue's haul requirements (size and number of soups/objects).
3. Be careful with rules combining high search speed and large hauls.
4. When in doubt, don't submit hauls too often to help keep hosting costs down.
5. Don't test client modifications on regular symmetries.
6. Don't break Catagolue.

The nitty-gritty

Catagolue imposes certain requirements on submitted hauls: they must contain at least 10,000 soups or 250,000 objects (or both); and they must not exceed a total size of 1 MiB.

In order to avoid running afoul of these limits, as well as to gauge the speed with which your rule of choice can be searched, try submitting a small test haul first: 10,000 soups works well for apgsearch 1.x, 1 million soups is a good choice for apgluxe 5.x. Take note of how long it takes to submit your test haul, and how big the resulting log is.

If your rule takes too long to search, try fewer soups. (If you're already at the server-imposed minimum of 10,000 and your rule doesn't have enough objects per soup... tough luck.) If your test haul finished too quickly, try a bigger test. This'll also help you get a feel for how quickly hauls grow as you add soups.

You may find that with some rules (such as B/S012345678), even the smallest possible haul will exceed the 1 MiB limit. These rules are not soup-searchable, at least not on Catagolue; again, this is tough luck.

There are also some rules that combine fast searching and big hauls. Day & Night (B3678/S34678) is a notable example; a C1 haul of 8 million soups may take roughly 5 minutes (give or take a few, depending on how your computer's speed and CPU load) to finish using apgluxe 5.x, and will weigh in at just under one megabyte. Keep in mind that hauls must be processed server-side, so when searching rules like these, don't run too many searcher instances at once. If Catagolue is struggling, e.g. by producing 500 Internal Server Errors or by failing to process all hauls submitted, cut down on the number of searchers. If you're already running just one, write a wrapper script to search something else (like Conway Life) in between hauls to give Catagolue time to cool down. This also applies if you're contributing to such a rule if another user is already submitting hauls for it.

The above also applies to other rules. Catagolue needs to carry out a datastore operation for each type of object in a haul, and datastore operations cost money. Thus, if you have the choice between submitting a haul of 200 distinct objects every 30 seconds, and submitting a haul of 1,000 distinct objects every 30 minutes, then absent a compelling reason to choose the former... choose the latter. You should not generally upload more than one haul every 20 minutes from a single machine.

A word on server-side haul processing: new hauls are commissioned to the census every 12 minutes. This process usually takes only a few minutes to finish; if it does not finish before the next commissioning process is started, 12 minutes later, it will be one of those days and Adam P. Goucher will not be happy.

All this said, most rules are fine to search and do not need any special considerations or attention.

Testing client modifications

Modifications to the client program should be tested with a special "test" symmetry, rather than one of the "production" symmetries such as C1; by convention, all such symmetries should be suffixed "_Test" (case-insensitive).

If a client program is modified to allow additional rules, symmetries, or topologies, then it should be done in a way that adheres to the Catagolue naming conventions.

Take care to disable haul verification if you're modifying the client, too, or your modified client may wrongly reject good hauls.

Support threads

• apgsearch v1.0 (discussion thread) at the ConwayLife.com forums

• Hacking apgsearch (discussion thread) at the ConwayLife.com forums

• apgsearch 2.2 (discussion thread) at the ConwayLife.com forums

• apgsearch 3.1 (discussion thread) at the ConwayLife.com forums

• apgsearch 4.0 (discussion thread) at the ConwayLife.com forums

External links

• Catagolue quota (discussion thread) at the ConwayLife.com forums