Page 1 of 2

Catagolue quota

Posted: August 3rd, 2017, 3:58 pm
by calcyman
It's been noted that there have been several instances of Catagolue downtime due to straying over the daily quota. I'd like to hear what your preferences are out of the following possibilities:
  • Leave things as they are;
  • Raise the quota substantially, but with a slight charge for hauls uploaded for non-B3/S23 rules (of course, B3/S23 will remain free forever). The fairest way to do this is per megabyte of hauls -- say, $0.10 per megabyte?
Thoughts?

Re: Catagolue quota

Posted: August 3rd, 2017, 4:16 pm
by muzik
Definitely leave it the way it is. Pretty much any amount of money would probably deter some newcomers to the scene.

Re: Catagolue quota

Posted: August 3rd, 2017, 4:18 pm
by drc
I feel like maybe we should work on ways to reduce the size of hauls? I wouldn't know how to do that, so this might sound a bit out there.

Definitely not the second option, though.

Re: Catagolue quota

Posted: August 3rd, 2017, 4:25 pm
by muzik
If the second option is to be implemented, the charge for other sufficiently researched rules or rules considered important as well as Life, such as Flock, Day and Night, HighLife, Pedestrian Life, etc. should be removed/decreased as well.

I'm not too sure about this one, but maybe users with the Trillionaire badge could search other rules for free as well?


Either way, a voluntary donation button would also be fine with me, especially if it too promises a (temporary or permanent) quota increase.

Re: Catagolue quota

Posted: August 3rd, 2017, 7:11 pm
by Saka
Please not the second one. I would hate to pay even a bit every time I search a new rule.

What about delte rules that have been ignored for say, 5 months?

Re: Catagolue quota

Posted: August 3rd, 2017, 7:18 pm
by gmc_nxtman
I support muzik's idea of a donation button - it would help reduce the chance of a quota overload but not force the user to pay for non-B3S23 hauls, which might be difficult for some of the younger members of this forum to arrange. Fixing occasional inconvenient outages is not quite worth money in my opinion.

Re: Catagolue quota

Posted: August 3rd, 2017, 7:22 pm
by toroidalet
Saka wrote:What about delte (∆?) rules that have been ignored for say, 5 months?
That doesn't seem very good, considering some people (Rhombic and Apple Bottom) apgsearch rules with novel things and then don't report them. If someone later decides to apgsearch the same rule, then they won't have any idea that many of the things they find are already known.
I second the donation button.

Re: Catagolue quota

Posted: August 3rd, 2017, 7:48 pm
by muzik
Saka wrote:What about delte rules that have been ignored for say, 5 months?
...which would completely invalidate the stuff I did for the last two days.

Which probably was the cause behind the outages, but oh well.

Re: Catagolue quota

Posted: August 4th, 2017, 1:06 am
by calcyman
The quota is mainly caused by disk I/O operations, rather than disk space (so deleting rules won't help that much).

I'll leave things the way they are for now.

Re: Catagolue quota

Posted: August 4th, 2017, 4:01 pm
by Rhombic
A donation button seems a decent idea in any case.

Re: Catagolue quota

Posted: August 4th, 2017, 11:52 pm
by codeholic
  • donation button (subscription-based donations too)
  • throttling non-CGoL hauls after daily traffic reaches a certain threshold
  • banning stupid rules full of unremarkable patterns, such as B/S2345678 (if it's not done already)

Re: Catagolue quota

Posted: August 5th, 2017, 12:04 am
by Saka
codeholic wrote:
  • donation button (subscription-based donations too)
  • throttling non-CWOL hauls after daily traffic reaches a certain threshold
  • banning stupid rules full of unremarkable patterns, such as B/S2345678 (if it's not done already)
1. Agree
2. This might be a good idea (assuming you mean CGoL instead of CWOL) but I think similar popular rules should be non-throttled.
3. Depending on how the rule filter will work, it might dissallow rules like B/S45, which I am planning to use to search for SLs with only 4 or 5 neighbours.

Re: Catagolue quota

Posted: August 5th, 2017, 12:19 am
by codeholic
Saka wrote:assuming you mean CGoL instead of CWOL
My brain broke. Yes, I do.
Saka wrote:but I think similar popular rules should be non-throttled.
3. Depending on how the rule filter will work, it might dissallow rules like B/S45, which I am planning to use to search for SLs with only 4 or 5 neighbours.
There are no still lifes in B/S45. All patterns die out in this rule.

Re: Catagolue quota

Posted: August 5th, 2017, 12:23 am
by Saka
codeholic wrote: There are no still lifes in B/S45. All patterns die out in this rule.
Oh. But still, you get my point. Rules like B456/S35678 or whatever with oscs might be blocked.

Re: Catagolue quota

Posted: August 5th, 2017, 12:24 am
by gmc_nxtman
codeholic wrote:There are no still lifes in B/S45. All patterns die out in this rule.

Code: Select all

x = 6, y = 6, rule = B/S45
2b2o$b4o$2o2b2o$2o2b2o$b4o$2b2o!
Although I do agree there isn't really any priority to apgsearch this specific rule.

EDIT: Regarding codeholic's original points, I agree with them but 2/3 should have less rigid regulations.

Re: Catagolue quota

Posted: August 5th, 2017, 12:39 am
by codeholic
Oops, true. Infinitely many of them

Code: Select all

x = 10, y = 10, rule = B/S45
4b2o$3b4o$2b2o2b2o$b2o4b2o$2o6b2o$2o6b2o$b2o4b2o$2b2o2b2o$3b4o$4b2o!

Re: Catagolue quota

Posted: August 5th, 2017, 2:38 pm
by muzik
codeholic wrote:
  • banning stupid rules full of unremarkable patterns, such as B/S2345678 (if it's not done already)
Rules like that would have the minimum haul size probably exceed the 1MB limit. So already done in a sense, unless someone gets extremely lucky.

Re: Catagolue quota

Posted: August 5th, 2017, 4:05 pm
by drc
maybe rules would have to be able to grow in size in order to be posted on the site. (Have one of B2c or B3i and one of B2e and B3a

Re: Catagolue quota

Posted: August 5th, 2017, 7:16 pm
by Saka
drc wrote:maybe rules would have to be able to grow in size in order to be posted on the site. (Have one of B2c or B3i and one of B2e and B3a
But some people like to search for oscs and SLs in non-growing rules.

Re: Catagolue quota

Posted: August 5th, 2017, 7:21 pm
by toroidalet
drc wrote:maybe rules would have to be able to grow in size in order to be posted on the site.
Don't forget about B2a and B1e. A for Awesome's working on one that supports photons.
I'm pretty sure a filter would require more code and more memory devoted to the server for filters. (I'm not an expert so don't quota me on this)

Re: Catagolue quota

Posted: August 5th, 2017, 8:05 pm
by drc
toroidalet wrote: Don't forget about B2a and B1e. A for Awesome's working on one that supports photons.
I'm pretty sure a filter would require more code and more memory devoted to the server for filters. (I'm not an expert so don't quota me on this)
I got a couple close calls to rules with B2a, but all of them had siderakes that occurred pretty plentifully naturally

Re: Catagolue quota

Posted: August 6th, 2017, 7:36 am
by Apple Bottom
toroidalet wrote:That doesn't seem very good, considering some people (Rhombic and Apple Bottom) apgsearch rules with novel things and then don't report them.
Well, it's often difficult to decide what to report.

If (to give just one example) a high-period oscillator appeared in Life, say something with period 168, that would obviously be a very important discovery (even if it was already known). But if a period-168 oscillator shows up in ANY rule, especially a non-totalistic one, is it still important, or does it just have novelty value? There are so many different non-totalistic rules that such oscillators are bound to exist in large numbers. So hitting one of them doesn't strike me as necessarily being an important discovery, and I'm hesitant to keep calling everyone's attention to things that are perhaps not all that consequential after all.[1] (This is not to say Saka was wrong in posting it. I'm quite happy to defer to others' judgement about what does and doesn't warrant reporting.)

The above is doubly true when you put the cart before the horse and engineer rules in such a manner as to afford high-period oscillators, spaceships, and other such things. ;) (Which I myself am not doing; I'm not smart enough for that.)

(And on the subject of not being smart enough, I actually mostly just search rules that others request hauls for these days.)

Now, to get back to the original question--
calcyman wrote:It's been noted that there have been several instances of Catagolue downtime due to straying over the daily quota. I'd like to hear what your preferences are out of the following possibilities:
  • Leave things as they are;
  • Raise the quota substantially, but with a slight charge for hauls uploaded for non-B3/S23 rules (of course, B3/S23 will remain free forever). The fairest way to do this is per megabyte of hauls -- say, $0.10 per megabyte?
Thoughts?
I'll number these, for the sake of convenience...
  1. I think leaving things as they are is a workable option, for now. I've noticed two instances of over-the-quota downtime recently, and one back in March (if memory serves). It's not a particularly pressing problem yet, and now that more people are aware of it perhaps we can come to a gentlemen's agreement to try our best to not break the site. (For my part, that means no more searching Day & Night for the time being, much as I like the rule.)
  2. I don't particularly like the idea of charging for haul submissions; that should be a last resort. People are already donating CPU time (and thus money, in the form of electricity costs).

    I'm also not sure how pay-to-submit-hauls would work. Taking micro-payments would require either a payment processor, which'd likely mean fees and other hassles, or using a cryptocurrency such as BTC, which not everyone's using. Actually tracking and managing payments is another complication: how would that work in practice?

    Finally, how would you make sure people do pay? What would happen if someone submitted hauls without paying? What if that person was submitting anonymously? (And, as someone mentioned, how do you communicate this to new contributors?)

    On to the next idea--
  3. Since donation buttons have been mentioned -- that strikes me as a better option. Setting up a Patreon page or a Flattr account might also be worth considering. (That said, I wouldn't count on voluntary donations to make a real dent in the costs of running the site.)
  4. Here's another idea. Getting back to the aforementioned "gentlemen's agreement to not break Catagolue", it's actually very difficult right now to gauge the load you, as a contributor, are putting on Catagolue. Perhaps there could be an accounting system of sorts -- a daily breakdown of costs you incurred, grouped by rule (or rule and symmetry). If I saw that, say, I've caused costs of 0.37 USD, of which 0.36 USD went to ${MOSTLY_IRRELEVANT_RULE}, it would be easy for me to decide to stop searching that rule. So I'd say this would be worthwhile.
  5. Another thing that would be worthwhile: an overall "Catagolue Quota-O-Meter™", displaying how far into its daily quota Catagolue currently is, perhaps color-coded as well: green (<50%), yellow (50-70%), orange (70-90%), red (90-100%). (One might argue that this would make it easier for malicious actors to bring Catagolue down, but I don't think anyone's actively trying that, and if they did they could just swamp it with requests anyway; they wouldn't need the Quota-O-Meter™.)
  6. Vaguely related -- is the Catagolue server code ever going to be published? I don't want to encourage fragmentation by having people set up Catagolue clones (Cataglones?), but if someone's hell-bent on searching a silly rule, this would allow them to set something up for themselves and avoid overwhelming the main site. (How intimately is Catagolue tied to Google's infrastructure, BTW? Could you set it up on your own servers running Hadoop etc., or does it really have to be Google's App Engine?)
  7. Finally -- although this is a very remote option, perhaps in the distant future there could be a charitable (or at least non-profit) foundation set up to support research into cellular automata (especially, but not limited to, Conway Life)? Such a foundation could either run sites directly, or it could dispense funds to allow others to do so, in the way that e.g. the Perl Foundation dispenses grant money for working on Perl-related projects. People could then donate to the foundation, and the foundation could also solicit donations from the industry, and from other benefactors.

    Granted, this is a rather outlandish idea, but like a one-in-a-million chance, it might just work...! ;) (It did for the OEIS, to name one example.)
Footnotes:

1. Goodness knows I get miffed when people keep on posting useless replies to threads I'm subscribed to, so I can't well do the same to others. "Let thy speech be better than silence, or be silent." - Dionysius of Halicarnassus

----

EDIT: if you do to implement a pay-per-haul system, BTW, I was just thinking it might be worthwhile to give each user a certain number of free hauls (or megabytes) per day -- say, 10 or 20 MiB --, and only charge for submissions that exceed that limit. This would ensure that users aren't kept from searching non-CGOL rules entirely, while at the same time keeping costs low.

In fact, thinking about it, this could be implemented even without pay-per-haul and the associated complicated infrastructure it'd require; just track how much each user is uploading, and alert them if they go over their daily limit. Ideally this would have client-side support as well, with the server sending appropriate status code on haul submission ("429 Slow Down Cowboy!"), and the client handling these appropriately, e.g. by alerting the user and quitting when the daily quota's reached. (Or automatically switching to searching Conway Life instead. ;))

This would also be good to have in general, since the problem with overly large hauls that previously caused Catagolue to break when a blip in my Internet connection caused a downward spiral of ever-larger Day & Night hauls to be uploaded to the site and rejected is still present in principle -- submitting an overly large haul won't work, and the client will save it, keep on searching, and try to submit an even larger haul next time. Having the server send an appropriate status (e.g. "413 Payload Too Large") and the client react accordingly would be much better.

Finally, getting back to the "the-first-X-megabytes-are-free" idea -- this in turn would also go well with that swish dashboard for logged-in users that I keep imagining. ;)

EDIT 2: (spelling/grammar)

Re: Catagolue quota

Posted: August 6th, 2017, 8:54 am
by Saka
Apple Bottom wrote:
Another thing that would be worthwhile: an overall "Catagolue Quota-O-Meter™", displaying how far into its daily quota currently is, perhaps color-coded as well: green (<50%), yellow (50-70%), orange (70-90%), red (90-100%).
Yes, I like this. I think it should display on the home and census pages. Maybe we can have life patterns instead of colors. Smiley for <50%, Weekender for 50-70%, Jolson for 70-90%, and I dunno for 90-100%.
Apple Bottom wrote:
Vaguely related -- is the Catagolue server code ever going to be published? I don't want to encourage fragmentation by having people set up Catagolue clones (Cataglones?), but if someone's hell-bent on searching a silly rule, this would allow them to set something up for themselves and avoid overwhelming the main site.
Maybe if you have a Payosha account on the Catagolue and you have say, searched 500 different rules, there could be a "Secret Developer Things" button placed on the user page that would link to various code options and the link to the main source code.
Apple Bottom wrote: Finally -- although this is a very remote option, perhaps in the distant future there could be a charitable (or at least non-profit) foundation set up to support research into cellular automata (especially, but not limited to, Conway Life)?
This is amazing. We could set up a lemonade stand thingy. Or sell pattern plushies (Bugs spaceships make very good plushies)! If this goes to far we will have some kind of matrix controlled by a massive conscious patern in a 163910038462015183 state ruleTable....

Re: Catagolue quota

Posted: August 6th, 2017, 9:04 am
by Rhombic
Another thing that would be worthwhile: an overall "Catagolue Quota-O-Meter™", displaying how far into its daily quota currently is, perhaps color-coded as well: green (<50%), yellow (50-70%), orange (70-90%), red (90-100%).
This is a good idea.
Maybe for >95% only allow CGoL to be submitted? That can bring problems I guess.
People would then hopefully be aware of it and try to be consequent with their submissions, leaving B6ae/S1457c for some other day.

Re: Catagolue quota

Posted: August 6th, 2017, 1:24 pm
by Apple Bottom
Saka wrote:Maybe if you have a Payosha account on the Catagolue and you have say, searched 500 different rules, there could be a "Secret Developer Things" button placed on the user page that would link to various code options and the link to the main source code.
I'd give it about five minutes before the whole thing would get leaked. ;) Better not count on it being limited to certain users.

Also, fun fact -- if you made it 500, I'd be the only one to qualify right now:

Code: Select all

Apple Bottom                 553
Rhombic                      384
Daniel R. Collazo            378
muzikbike                    192
Saka                         175
Josh Ball                    153
dmqwerty425@gmail.com        142
Anonymous                    122
Saka wrote:This is amazing. We could set up a lemonade stand thingy. Or sell pattern plushies (Bugs spaceships make very good plushies)! If this goes to far we will have some kind of matrix controlled by a massive conscious patern in a 163910038462015183 state ruleTable....
I can't quite tell if you're being sarcastic there, but indeed, why not plushies -- why not other merchandising? Conway Life, the t-shirt! Conway Life, the coloring book! Conway Life, the lunch box! Conway Life, the breakfast cereal! Conway Life, the flamethrower! (OK, maybe not that one.) And last but not least, Conway Life, the doll... that being good ol' John Horton himself, of course.