ConwayLife.com - A community for Conway's Game of Life and related cellular automata
Home  •  LifeWiki  •  Forums  •  Download Golly

Scatter plot for still lifes (R help needed)

For discussion directly related to ConwayLife.com, such as requesting changes to how the forums or wiki function.

Scatter plot for still lifes (R help needed)

Postby Apple Bottom » July 27th, 2016, 7:43 pm

I was thinking it would be nice to have a scatter plot of (total) still life counts vs. populations in the Catagolue article on the LifeWiki (which is why I'm sticking this in "Website Discussion", BTW), akin to the upper plot on page 2 of the full results for Andrzej Okrasinski's census.

That sort of thing's best done programmatically, so I turned to R, which I vaguely recalled being told was useful; but I've never used that before myself, and found it rather unwieldy, so I was wondering if anyone here either a) was an R wizard who could help improve the following plot, or b) had suggestions for a better tool.

For now here's the current best attempt at a plot I have:

Image

Here's the R script used:

library(bit64)
options(scipen=100)
stilllifes <- read.table("xs_scatterplot.dat", header=TRUE);
attach(stilllifes)
## svg("xs_scatterplot.svg", width=10, height=10, antialias="subpixel")
png("xs_scatterplot.png", width=640, height=640, antialias="cleartype")
plot(Population, Count, log="y", lab=c(length(Population), ceiling(log10(max(Count)))+1, 7), xaxs="i")
## text(Population, Count, Count, cex=0.6, pos=4, col="red")
legend("topright", inset=0.05, title="Total still lifes", c("Count", "Linear fit", expression(paste("LOWESS fit, ", alpha, "=0.3"))), pch=c(21, -1, -1), lty=c(0, 1, 1), col=c("black", "black", "blue"))
title(main="Total still lifes by population")
abline(lm(log10(Count)~Population))
lines(lowess(stilllifes, f=0.3), col="blue")
dev.off()


And finally, the data file (note that this was generated from somewhat older data; I'd use newer data when generating an image to upload to the LifeWiki):

Population   Count
4   37230663244700
5   5217667971195
6   22811439059639
7   6028058512999
8   1151083489759
9   6130605168
10   4179800708
11   1236556937
12   186490260778
13   157670526
14   71439981537
15   423074176
16   2672307142
17   140665171
18   502771638
19   17640474
20   62870208
21   1460833
22   9283930
23   454126
24   409529
25   209479
26   142231
27   30761
28   2932260
29   8988
30   43911
31   17135
32   7052
33   177
34   300
35   23
36   51
37   9
38   402
39   4
40   5092
41   2
42   64
45   1
46   1


So, yeah. Any tips on how to improve this plot? R seems to offer precious little control over formatting, axes, labels, ticks etc., but maybe I'm just looking in the wrong places.

Alternatively, any other (better) tools to generate such plots? Programs that can be used non-interactively (i.e. called with a script as part of a shell script) would be very strongly preferred.
Living proof that a little knowledge is a dangerous thing.

Catagolue: Apple Bottom • Life Wiki: Apple Bottom • Twitter: @_AppleBottom_

Proud member of the Pattern Raiders!
User avatar
Apple Bottom
 
Posts: 743
Joined: July 27th, 2015, 2:06 pm

Re: Scatter plot for still lifes (R help needed)

Postby Scorbie » July 28th, 2016, 5:34 am

Python + matplotlib?
Disclaimer: I have only heard of it.
Best wishes to you, Scorbie
User avatar
Scorbie
 
Posts: 1328
Joined: December 7th, 2013, 1:05 am

Re: Scatter plot for still lifes (R help needed)

Postby Apple Bottom » July 28th, 2016, 5:55 am

Scorbie wrote:Python + matplotlib?
Disclaimer: I have only heard of it.


"matplotlib tries to make easy things easy and hard things possible" -- that certainly sounds promising. (Also, they're taking more than a subtle cue from Perl there, aren't they.)

I'll look into it, thanks!

EDIT: not much luck so far. Cygwin doesn't provide a package, and the pip package can't even be installed:

$ pip install matplotlib
[...]
    running build_ext
    building 'matplotlib.ft2font' extension
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-build-gwez0e7l/matplotlib/setup.py", line 277, in <module>
        **extra_args
      File "/usr/lib/python3.4/distutils/core.py", line 148, in setup
        dist.run_commands()
      File "/usr/lib/python3.4/distutils/dist.py", line 955, in run_commands
        self.run_command(cmd)
      File "/usr/lib/python3.4/distutils/dist.py", line 974, in run_command
        cmd_obj.run()
      File "/usr/lib/python3.4/site-packages/setuptools/command/install.py", line 61, in run
        return orig.install.run(self)
      File "/usr/lib/python3.4/distutils/command/install.py", line 539, in run
        self.run_command('build')
      File "/usr/lib/python3.4/distutils/cmd.py", line 313, in run_command
        self.distribution.run_command(command)
      File "/usr/lib/python3.4/distutils/dist.py", line 974, in run_command
        cmd_obj.run()
      File "/usr/lib/python3.4/distutils/command/build.py", line 126, in run
        self.run_command(cmd_name)
      File "/usr/lib/python3.4/distutils/cmd.py", line 313, in run_command
        self.distribution.run_command(command)
      File "/usr/lib/python3.4/distutils/dist.py", line 974, in run_command
        cmd_obj.run()
      File "/usr/lib/python3.4/site-packages/setuptools/command/build_ext.py", line 50, in run
        _build_ext.run(self)
      File "/usr/lib/python3.4/distutils/command/build_ext.py", line 339, in run
        self.build_extensions()
      File "/usr/lib/python3.4/distutils/command/build_ext.py", line 448, in build_extensions
        self.build_extension(ext)
      File "/usr/lib/python3.4/site-packages/setuptools/command/build_ext.py", line 183, in build_extension
        _build_ext.build_extension(self, ext)
      File "/usr/lib/python3.4/distutils/command/build_ext.py", line 500, in build_extension
        include_dirs=ext.include_dirs,
      File "/tmp/pip-build-gwez0e7l/matplotlib/setupext.py", line 785, in __get__
        result = obj._hooks[self._name]() + result
      File "/tmp/pip-build-gwez0e7l/matplotlib/setupext.py", line 805, in include_dirs_hook
        import numpy
      File "/usr/lib/python3.4/site-packages/numpy/__init__.py", line 170, in <module>
        from . import add_newdocs
      File "/usr/lib/python3.4/site-packages/numpy/add_newdocs.py", line 13, in <module>
        from numpy.lib import add_newdoc
      File "/usr/lib/python3.4/site-packages/numpy/lib/__init__.py", line 17, in <module>
        from . import scimath as emath
    ImportError: cannot import name 'scimath'

    ----------------------------------------
Command "/usr/bin/python3 -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-gwez0e7l/matplotlib/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-bb_8_3xg-record/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /tmp/pip-build-gwez0e7l/matplotlib/
$


Google's not helpful, either. So, no dice with Cygwin.

There's apparently precompiled binaries for Python on Windows, but Python itself (3.5.2) can't be installed, failing with "0x80080005 server execution failed" and advising me to check its install log (which of course contains nothing whatsoever related). But it matters little, because there are not actually any precompiled windows packages anyway, either for 3.5 or the older 2.7 I still have installed for use with Golly.

So currently I'm downloading an 800 MiB bundle called Python(x, y) which bundles matplotlib and which I hope will finally actually work.

EDIT 2: matplotlib as bundled with Python(x, y) works, but Python(x, y) itself does not contain the python27.dll file needed for Python scripts in Golly. And unfortunately it also cannot coexist with a vanilla Python 2.7, so this isn't a solution either.

OTOH I was able to pip-install matplotlib on Cygwin's Python after installing the (unrelated, IMO, but what do I know) gtk 2.0 library/headers packages. After also installing statsmodels (which necessitated, among other things, LAPACK and a Fortran compiler) I whipped up this:

Image

Which is a good start. Script:

#!/usr/bin/python3

import numpy as np
import matplotlib.pyplot as plt
import statsmodels.api as sm

# read data file
datafile=open("xs_scatterplot.dat","rb")
x, y=np.loadtxt(datafile,delimiter="\t",skiprows=1,unpack=True)

# generate a scatter plot
fig, ax=plt.subplots()
ax.scatter(x, y, label='Still lifes')
ax.set_yscale('symlog', linthreshy=1)
ax.set_xlabel('Population', fontsize=15)
ax.set_ylabel("Count", fontsize=15)
ax.set_title('Total still lifes', fontsize=20)
plt.ylim(ymin=0)
ax.grid(True)
fig.tight_layout()

# plot linear fit
logy=np.log(y)
coefficients=np.polyfit(x, logy, 1)
polynomial=np.poly1d(coefficients)
ys=polynomial(x)
plt.plot(x, np.exp(ys), label='Linear fit')

# plot LOWESS fit
lowess=sm.nonparametric.lowess
z=lowess(logy, x, frac=0.3, it=3, return_sorted=False)
plt.plot(x, np.exp(z), label=r'LOWESS fit, $\alpha=0.3$')

# add legend
plt.legend(loc='upper right')

# save plot to file
fig.set_size_inches(8, 8)
plt.savefig("xs_scatterplot.png", dpi=80)


Data:

Population      Count
4       38443380059190
5       5387615762362
6       23554442686741
7       6224388388190
8       1188573288568
9       6330295738
10      4315921916
11      1276821800
12      192563548091
13      162807618
14      73766540016
15      436861751
16      2759365374
17      145246934
18      519150780
19      18213412
20      64918210
21      1507933
22      9586023
23      468929
24      422767
25      216249
26      146749
27      31728
28      3027923
29      9266
30      45332
31      17680
32      7284
33      181
34      304
35      24
36      55
37      9
38      410
39      4
40      5265
41      2
42      65
45      1
46      1


I'm fairly happy with how sane matplotlib's defaults are and how good its plots look without much tweaking -- most of the time required for this was installing Python modules, figuring out issues with the same, and trying to understand matplotlib's documentation. ;)
Living proof that a little knowledge is a dangerous thing.

Catagolue: Apple Bottom • Life Wiki: Apple Bottom • Twitter: @_AppleBottom_

Proud member of the Pattern Raiders!
User avatar
Apple Bottom
 
Posts: 743
Joined: July 27th, 2015, 2:06 pm

Re: Scatter plot for still lifes (R help needed)

Postby Scorbie » July 31st, 2016, 1:25 am

Hmm... So you use python from cygwin with golly? That's very unique. Vanilla python for windows installs matplotlib without a problem:
PS C:\Users\Scorbie> python -m pip install matplotlib
Collecting matplotlib
  Downloading matplotlib-1.5.1-cp35-none-win32.whl (6.2MB)
    100% |################################| 6.2MB 126kB/s
Collecting numpy>=1.6 (from matplotlib)
  Downloading numpy-1.11.1-cp35-none-win32.whl (6.6MB)
    100% |################################| 6.6MB 125kB/s
Collecting pytz (from matplotlib)
  Downloading pytz-2016.6.1-py2.py3-none-any.whl (481kB)
    100% |################################| 481kB 1.4MB/s
Collecting pyparsing!=2.0.4,>=1.5.6 (from matplotlib)
  Downloading pyparsing-2.1.5-py2.py3-none-any.whl (42kB)
    100% |################################| 51kB 3.7MB/s
Collecting python-dateutil (from matplotlib)
  Downloading python_dateutil-2.5.3-py2.py3-none-any.whl (201kB)
    100% |################################| 204kB 2.4MB/s
Collecting cycler (from matplotlib)
  Downloading cycler-0.10.0-py2.py3-none-any.whl
Collecting six>=1.5 (from python-dateutil->matplotlib)
  Downloading six-1.10.0-py2.py3-none-any.whl
Installing collected packages: numpy, pytz, pyparsing, six, python-dateutil, cycler, matplotlib
Successfully installed cycler-0.10.0 matplotlib-1.5.1 numpy-1.11.1 pyparsing-2.1.5 python-dateutil-2.5.3 pytz-2016.6.1 six-1.10.0



Edit That's a very decent short script indeed! couldn't get statsmodels library to work in windows, but except that it's kinda nice :)
Note for readers: the xs_scatterplot.dat file has its tabs converted to spaces by phpBB.
Best wishes to you, Scorbie
User avatar
Scorbie
 
Posts: 1328
Joined: December 7th, 2013, 1:05 am

Re: Scatter plot for still lifes (R help needed)

Postby Apple Bottom » July 31st, 2016, 5:49 am

Scorbie wrote:Hmm... So you use python from cygwin with golly? That's very unique. Vanilla python for windows installs matplotlib without a problem:[/code]

No, Golly's using the regular Windows version of Python. But when working on the command line I prefer Cygwin, and use Cygwin's python there.

Scorbie wrote:Edit That's a very decent short script indeed! couldn't get statsmodels library to work in windows, but except that it's kinda nice :)
Note for readers: the xs_scatterplot.dat file has its tabs converted to spaces by phpBB.


Thanks! And good to know about phpBB playing tricks there. (I suppose I could use commas instead of tabs when generating that file anyway; the only reason I used tabs is because that's what R liked best.)
Living proof that a little knowledge is a dangerous thing.

Catagolue: Apple Bottom • Life Wiki: Apple Bottom • Twitter: @_AppleBottom_

Proud member of the Pattern Raiders!
User avatar
Apple Bottom
 
Posts: 743
Joined: July 27th, 2015, 2:06 pm

Re: Scatter plot for still lifes (R help needed)

Postby Apple Bottom » October 4th, 2016, 6:44 pm

BTW, just for fun I now generated images for not just B3/S23/C1 but B3/S23/!C1 (i.e. all other symmetries combined) as well. These are all using data from October 1st (right now; MediaWiki does not support linking to stable versions of the current revision of an uploaded file, so these will always point to the latest image):

Image
Image
Image
Image

I'm honestly not sure the LOWESS fit in particular is actually meaningful, but the plots sure are interesting to look at.

For the data and scripts used, see the file description pages on the wiki, linked e.g. from the Catagolue article.
Living proof that a little knowledge is a dangerous thing.

Catagolue: Apple Bottom • Life Wiki: Apple Bottom • Twitter: @_AppleBottom_

Proud member of the Pattern Raiders!
User avatar
Apple Bottom
 
Posts: 743
Joined: July 27th, 2015, 2:06 pm

Re: Scatter plot for still lifes (R help needed)

Postby Scorbie » October 8th, 2016, 10:37 pm

Wow, seriously nice work with the plots and the wiki article!
Best wishes to you, Scorbie
User avatar
Scorbie
 
Posts: 1328
Joined: December 7th, 2013, 1:05 am

Re: Scatter plot for still lifes (R help needed)

Postby Apple Bottom » October 9th, 2016, 5:15 am

Scorbie wrote:Wow, seriously nice work with the plots and the wiki article!


Aww shucks, sugarcube. Thank you! :D
Living proof that a little knowledge is a dangerous thing.

Catagolue: Apple Bottom • Life Wiki: Apple Bottom • Twitter: @_AppleBottom_

Proud member of the Pattern Raiders!
User avatar
Apple Bottom
 
Posts: 743
Joined: July 27th, 2015, 2:06 pm


Return to Website Discussion

Who is online

Users browsing this forum: No registered users and 1 guest