safeopenclip.lua usability issues

[Andrew]

Just to make sure that I understand the request properly: if I set URL redirects on the server so that, for example, https://conwaylife.com/rules/DoubleB3S23.rule displayed the exact same content as https://conwaylife.com/w/index.php?titl ... action=raw would that work?

Yep, I think that should work. (Is that how links to patterns/*.rle files are also handled?)

[Nathaniel]

Yep, I think that should work. (Is that how links to patterns/*.rle files are also handled?)

This is now set up (e.g., https://conwaylife.com/rules/DoubleB3S23.rule points to the rule that's on the wiki). This is done via a server-side URL rewrite, so it is instantaneous (i.e., it's not a script that runs once per day or anything like that). Let me know if it does/doesn't work for your purposes.

This actually isn't how RLEs are handled. RLEs are handled by a script... and I'm not sure if there's a good reason for that. I suppose that one reason is that the "all.zip" file that contains the bundle of all RLEs really requires actual RLE files to be created, not just URL rewrites.

[Andrew]

This is now set up (e.g., https://conwaylife.com/rules/DoubleB3S23.rule points to the rule that's on the wiki).

Thanks! Works fine.

[rowett]

I've just commited an update to safeopenclip.lua for community testing.

Changes in this version:

• Better supports bounded grids. Will detect and remove invalid bounded grid definitions if requested.
• Will open a Golly Help window if an unsuported rule is found with a link to download and install the rule from the LifeWiki repo.
• Detects invalid pattern data.
• Puts a message in the status line saying what is has done.

Note: This version benefits from a new build of Golly to suppress warnings while testing different rule variants.

[dvgrn]

I've just commited an update to safeopenclip.lua for community testing.
...
Note: This version benefits from a new build of Golly to suppress warnings while testing different rule variants.

Looks good! The script is eminently testable using the current beta -- there's just one extra pop-up error window at the beginning, that will be suppressed in the next beta build.

Unsupported Rule
The clipboard contains a pattern with an unsupported rule LiterallyAnEntireRuleForOneStupidSpaceship so it has been opened in a safe rule.
If you want Golly to support this rule then it may already exist in the LifeWiki rule repository.
Clicking the link below will download and install the rule if it exists or display "Web request failed" if it doesn't.
https://www.conwaylife.com/rules/LiterallyAnEntireRuleForOneStupidSpaceship.rule

(except Golly's link is a get:url link, so it actually installs the rule when you click on the link, instead of just displaying the rule text)

[Andrew]

There are quite a few posts like this that have a pattern with @RULE info after the pattern data. I think it would be nice if safeopenclip.lua looked for that info and if the @RULE name matches the earlier rule string then it should create a suitable .rule file in the user's rule directory (but only if there is no existing file) and open the pattern using that rule. Probably no need for any warning dialog at all? Or maybe just a message in the status bar saying that a .rule file was created?

[confocaloid]

There are quite a few posts like this that have a pattern with @RULE info after the pattern data. I think it would be nice if safeopenclip.lua looked for that info and if the @RULE name matches the earlier rule string then it should create a suitable .rule file in the user's rule directory (but only if there is no existing file) and open the pattern using that rule. Probably no need for any warning dialog at all? Or maybe just a message in the status bar saying that a .rule file was created?

̶ ̶I̶ ̶t̶h̶i̶n̶k̶ ̶i̶t̶ ̶w̶o̶u̶l̶d̶ ̶h̶e̶l̶p̶ ̶t̶o̶ ̶h̶a̶v̶e̶ ̶a̶n̶ ̶"̶o̶n̶e̶ ̶c̶l̶i̶c̶k̶ ̶a̶w̶a̶y̶"̶ ̶p̶o̶s̶s̶i̶b̶i̶l̶i̶t̶y̶ ̶t̶o̶ ̶u̶s̶e̶ ̶t̶h̶e̶ ̶r̶u̶l̶e̶ ̶d̶a̶t̶a̶,̶ ̶i̶f̶ ̶i̶t̶ ̶i̶s̶ ̶a̶l̶r̶e̶a̶d̶y̶ ̶a̶v̶a̶i̶l̶a̶b̶l̶e̶ ̶a̶p̶p̶e̶n̶d̶e̶d̶ ̶t̶o̶ ̶t̶h̶e̶ ̶p̶a̶t̶t̶e̶r̶n̶ ̶d̶a̶t̶a̶.̶
EDIT (2024-04-08): earlier I thought that it might be a good idea to have safeopenclip.lua somehow process the appended @RULE data.
However, after further discussion in the thread, I now believe that it would be much better to avoid any processing of the appended @RULE data in the script safeopenclip.lua. There are several independent reasons for that. The main reason is that any attempt to cover "appended @RULE data" in a script will be inconsistent with what LifeViewer already does. The idea of an appended @RULE definition is to serve as a specific version of a CA rule, or as a temporary/one-time CA rule, that is placed in a single file with the pattern (as opposed to being stored in a separate rulefile). There should be no additional filesystem lookups/checks, when there's an appended @RULE definition; an appended @RULE definition is meant to be used in preference to any existing CA rules.

Considering that it does not seem feasible to implement this functionality using just a script, and implementing it into Golly will likely need more effort and testing than what could be done before Golly 4.3 release, I think it is best to admit that at this time Golly doesn't support the feature "transient rules" (a pattern with appended @RULE data which is used for this specific pattern).

Also, in many cases it will be counterproductive to copy the appended @RULE to an external rulefile, because it will often be a temporary/experimental CA rule, or one of multiple versions of a CA rule. Locally installed permanent rulefiles should not be mixed with "transient rules" which are used for a specific pattern.

Below is the rest of old text of this post.
--------
I think a warning/confirmation dialog would still be helpful. When copying a sufficiently large pattern snippet from a forum post, it may be hard to notice that there is a ruletable or a ruletree appended after the pattern data. A Golly user may just want to see the pattern data, without necessarily installing any rulefile.
Additionally, a Golly user may already have a "cosmetic" modification of the same rulefile (e.g. different palette; added comments) in their user rule directory. In that case, the existing rulefile should be used and should not be replaced.

[dvgrn]

Additionally, a Golly user may already have a "cosmetic" modification of the same rulefile (e.g. different palette; added comments) in their user rule directory. In that case, the existing rulefile should be used and should not be replaced.

Andrew's post mentioned "(but only if there is no existing file)". If there is an existing .rule file, then the "safe" opening functionality in safeopenclip.lua will never get triggered in the first place, right? The pattern will be opened using the already-installed rule.

I'm thinking that there are very few use cases where it will do any harm to create an appropriate .rule file, if there isn't one there already and if there's appended @RULE data in the pattern on the clipboard. Basically it would be just good user-friendly behavior to create the rule automatically, and mention that in a status-bar message but probably without an actual pop-up.

If someone is knowledgeable enough to want to see the pattern displayed in Display256, they can just copy to the clipboard only the pattern and not the appended rule. And/or if they make a rare mistake and get a rule table they didn't want, they can just go and delete the rule table. The fact that they're using safeopenclip.lua in the first place should mean that they want to see the pattern open in Golly and as functional as possible, as quickly and easily as possible... if they don't want the "safe open" functionality, they're most likely going to be using plain File > Open Clipboard or Ctrl+Shift+O anyway.

(I'm not sure I'm ever going to use plain File > Open Clipboard again, though, once Alt+O is available! There just doesn't seem to be any downside to getting an actual pattern reliably displayed, instead of a loud DING and a big error message.)

[confocaloid]

[...]
I'm thinking that there are very few use cases where it will do any harm to create an appropriate .rule file, if there isn't one there already and if there's appended @RULE data in the pattern on the clipboard. Basically it would be just good user-friendly behavior to create the rule automatically, and mention that in a status-bar message but probably without an actual pop-up.
[...]

I still think there should be a pop-up warning/confirmation message, before any attempts to use/save the rule data (if any). This would be more user-friendly, because you are not doing something that is likely to be unexpected.

When copying a sufficiently large pattern snippet from a forum post, it may be hard to notice that there is a ruletable or a ruletree appended after the pattern data. Multiple lines of multistate RLE data push whatever might be appended down beyond immediate visibility, and you end up not knowing what you have copied in the clipboard. It is easy to do "copy all" from a posted pattern snippet, but that does not mean it is a good idea to automatically assume the user wants to save a new ruletable. The rule may have a temporary name, it may be a testing version, it might be a non-renamed temporary modification of another existing rule that is not yet installed, etc.

I think a confirmation for using ruledata is necessary, so that the Golly user actually knows what is about to happen.

[dvgrn]

I still think there should be a pop-up warning/confirmation message, before any attempts to use/save the rule data (if any). This would be more user-friendly, because you are not doing something that is likely to be unexpected.

I'm not sure I understand the use case that you're worried about here. Why would saving a rule file to Golly's Rules folder be likely to be unexpected?

In the use case where a pattern comes with its own appended rule file, and the pattern can't be opened with full functionality without installing the rule, I would say that having safeopenclip.lua install the rule and inform me that it has done so ... is always going to be precisely the functionality that I expect, even if I didn't notice that the @RULE section was appended.

Hypothetical cases like "non-renamed temporary modification of another existing rule that is not yet installed" seem so rare in practice that it's not worth bothering people with a pop-up every time, just to avoid them. Golly is very good about letting people install updated versions of rule tables, just by copying the rule data and using File > Open Clipboard -- that's how you'd fix a "temporarily modified rule" problem if it actually happened.

Installing a rule immediately can really only do damage when there is already an existing rule in place that gets overwritten -- and that's not going to happen here.

[confocaloid]

I still think there should be a pop-up warning/confirmation message, before any attempts to use/save the rule data (if any). This would be more user-friendly, because you are not doing something that is likely to be unexpected.

I'm not sure I understand the use case that you're worried about here. Why would saving a rule file to Golly's Rules folder be likely to be unexpected?

In the use case where a pattern comes with its own appended rule file, and the pattern can't be opened with full functionality without installing the rule, I would say that having safeopenclip.lua install the rule and inform me that it has done so ... is always going to be precisely the functionality that I expect, even if I didn't notice that the @RULE section was appended.

In the use case where there is a ruledata appended, and the pattern would otherwise be opened in Display256, I believe opening the pattern in Display256 unless the user explicitly confirms that they want to use the appended ruledata is the expected functionality.

From the viewpoint of a Golly user that isn't an expert, it is very likely to be unexpected and confusing to silently use appended ruledata and save a rulefile without asking for confirmation.

These cases aren't "hypothetical". I already ran into multiple cases where there is an appended ruledata, but I don't really want to use that ruledata; I just want to copy the pattern into Golly.

Installing a rule immediately can really only do damage when there is already an existing rule in place that gets overwritten -- and that's not going to happen here.

I disagree. That is not the only possibility for damage.

By installing rules automatically without asking the user for confirmation, you are opening yourself for "jokes" (which may be either accidental or intentional) of the following form:

Code: Select all

x = 41, y = 19, rule = Serizawa
(pattern data goes here)
(pattern data goes here)
(pattern data goes here)
(pattern data goes here)
(pattern data goes here)
(pattern data goes here)
(pattern data goes here)
(pattern data goes here)
(pattern data goes here)
(pattern data goes here)
(pattern data goes here)
(pattern data goes here)
(pattern data goes here)
(pattern data goes here)
(pattern data goes here)
(pattern data goes here)
(pattern data goes here)
(pattern data goes here)
(pattern data goes here)
(pattern data goes here)

@RULE Serizawa

@TABLE
n_states: 137
(either completely unrelated rule data, or related-but-subtly-different rule data)

[dvgrn]

These cases aren't "hypothetical". I already ran into multiple cases where there is an appended ruledata, but I don't really want to use that ruledata; I just want to copy the pattern into Golly.

It's perfectly possible that you've got a use case there that I simply haven't been able to imagine.

Can you be a little more specific? A brief walkthrough of what you were working on would be really helpful, with a link to a pattern that you wanted to see displayed in Display256 -- but that would have caused some kind of problem if the attached rule table had been installed.

It's important to note that use cases along the lines of "I know I already have a good copy of that rule installed, so I absolutely don't want this bad/old/temporary/joke one" usually shouldn't be an issue. safeopenclip.lua will never overwrite a good rule table with a bad one -- the situation just won't ever come up, because the initial attempt to open the pattern will work, using the already-installed rule -- no error to catch.

The only possible problem I can see is a case where a bad/old/temporary/joke pattern-with-attached-rule is loaded first -- and then it gets stuck in the Rules folder because safeopenclip.lua will refuse to replace it. Replacement would have to be done via copying a corrected rule and using File > Open Clipboard.

For sure it can happen that a bad copy of a rule gets loaded, but this new proposed safeopenclip.lua functionality isn't the only way it can happen -- and a pop-up will only prevent it from happening in this use case, if the user knows enough to answer "no" to the question about loading the rule. Would a hypothetical Golly user who isn't an expert know enough to answer that question correctly?

[confocaloid]

Installing rules silently and automatically is unsafe. If the script named "safeopenclip.lua" is going to include functionality for creating rulefiles, then there should be always a confirmation/warning pop-up before the rulefile is created. If the user does not want to install a new rulefile, then no rulefile should be installed, and the pattern should be opened in Display256.

[...]
It's important to note that use cases along the lines of "I know I already have a good copy of that rule installed, so I absolutely don't want this bad/old/temporary/joke one" usually shouldn't be an issue. safeopenclip.lua will never overwrite a good rule table with a bad one -- the situation just won't ever come up, because the initial attempt to open the pattern will work, using the already-installed rule -- no error to catch. [...]

I'm not entirely sure "never overwrite an existing rule table" is the most helpful behaviour in this case. Maybe there should be instead a confirmation/warning message "Do you want to use the appended ruledata and overwrite the old file?", letting the Golly user to decide what they want to do in this case.

I'm sure there should be a confirmation popup, either way.

[...]
For sure it can happen that a bad copy of a rule gets loaded, but this new proposed safeopenclip.lua functionality isn't the only way it can happen -- and a pop-up will only prevent it from happening in this use case, if the user knows enough to answer "no" to the question about loading the rule. Would a hypothetical Golly user who isn't an expert know enough to answer that question correctly?

The confirmation/warning pop-up is helpful not only because it will help to prevent unintentionally saving a temporary/broken/incompletely copied/joke ruledata.
The confirmation/warning pop-up is also helpful, because the Golly user gets a chance to understand what is going on. If the script creates a rulefile, let the Golly user know that. Status bar is clearly not enough for this, because the message is easy to miss and disappears after a keyboard/mouse action.

Even if the ruledata is correct, asking the user directly whether or not they want to install and use it is better than deciding for them.

[dvgrn]

I'm sure there should be a confirmation popup, either way.

You've implied that you're sure about that because "I already ran into multiple cases where there is an appended ruledata, but I don't really want to use that ruledata; I just want to copy the pattern into Golly."

Can you please give some more details about these specific multiple cases? A description of an actual situation that you have found yourself in -- where it would have caused some difficulty for safeopenclip.lua to automatically install a rule table, and then notify you that it had done so -- might well be very convincing, as a reason for needing a pop-up instead of just a standard notification message in the status bar.

[confocaloid]

I'm sure there should be a confirmation popup, either way.

You've implied that you're sure about that because "I already ran into multiple cases where there is an appended ruledata, but I don't really want to use that ruledata; I just want to copy the pattern into Golly."

Can you please give some more details about these specific multiple cases? A description of an actual situation that you have found yourself in -- where it would have caused some difficulty for safeopenclip.lua to automatically install a rule table, and then notify you that it had done so -- might well be very convincing, as a reason for needing a pop-up instead of just a standard notification message in the status bar.

The point is not in specific cases. In the vast majority of cases, when I'm trying to load a pattern (with or without appended ruledata), I would want to see a confirmation pop-up before any rulefile-installing operations happen. As a Golly user, I want to be able to know that there is some appended ruledata, and I want to be able to decide whether or not that ruledata should be installed as a file.

Obviously "automatically install a rule table, and then notify" will not work due to what I already wrote. The warning should be before installing, and it should be possible to avoid installing the rulefile.

[Andrew]

Andrew's post mentioned "(but only if there is no existing file)". If there is an existing .rule file, then the "safe" opening functionality in safeopenclip.lua will never get triggered in the first place, right? The pattern will be opened using the already-installed rule.

Oops -- of course!

@confocoloid: If you're not happy with how the final version of safeopenclip.lua works you can always modify it and use your own version.

[confocaloid]

Andrew's post mentioned "(but only if there is no existing file)". If there is an existing .rule file, then the "safe" opening functionality in safeopenclip.lua will never get triggered in the first place, right? The pattern will be opened using the already-installed rule.

Oops -- of course!

@confocoloid: If you're not happy with how the final version of safeopenclip.lua works you can always modify it and use your own version.

Well there was a question about need for a warning dialog, and so I tried to provide an answer. I believe having a confirmation pop-up before installing a rulefile would be better, compared to installing rules silently. Especially considering that the script is called 'safeopenclip' and is supposed to be safe to use.

Of course a sufficiently experienced Golly user can modify scripts, but I thought the question was about what is best for the majority of Golly users who aren't sufficiently experienced.

If anything, disabling a pop-up someone doesn't like is easier than enabling a pop-up someone needs.

[Andrew]

I believe having a confirmation pop-up before installing a rulefile would be better, compared to installing rules silently. Especially considering that the script is called 'safeopenclip' and is supposed to be safe to use.

I'd have to disagree. There are already two cases where Golly installs a .rule file without requiring confirmation:

1. When using Open Clipboard and the clipboard contains @RULE data.
2. When opening a zip file that contains one or more .rule files.

I don't remember a single user complaining about this behavior.

[confocaloid]

Well, in my opinion those two cases are different enough from the case "pasting pattern data with an appended @RULE data".

When using Open Clipboard and the clipboard contains @RULE data, the user likely indeed wants to paste @RULE data (as explained in the section "The easy way to install a new .rule file" in "Golly Help: File Formats". So in this case guessing "install the rule" is more likely to be correct.

(On the other hand, I think a pop-up warning would be helpful in that case too, to clarify what happens. Does the user actually want to install such-and-such rulefile? Or they wanted to paste a pattern, and instead accidentally copied some ruletable?)

When opening a zip file that contains one or more .rule files (as explained in the section "Zip files" in "Golly Help: File Formats"), clipboard is not involved. A zip file is likely to be prepared by someone who knows how it is going to be processed and can decide on the correct file/folder structure.

In comparison, when the user is pasting a pattern with an optional appended @RULE data part, they are still pasting a pattern. If a rulefile must be installed to be able to run that pattern and the data appears to be available in the appended part, then I think it's a good idea to ask for confirmation in this case (because the primary intent is to open a specific pattern, rather than to install some rulefile)

[Andrew]

Well, in my opinion those two cases are different enough from the case "pasting pattern data with an appended @RULE data".

Not in my opinion. I'm not going to waste any more time arguing over this.

[confocaloid]

It was not my intention to argue over opinions. I just saw a question, and I wanted to provide an answer.
(After that, I got replies to my reply, which led me first to attempt to explain my opinion, and then to decide whether I want to defend my opinion.)

FWIW I'm not sure if safeopenclip.lua really needs to be concerned at all with whether or not there's some appended text (after the pattern data) that looks like an associated @RULE definition.

• Should there be (attempts of) validation of what appears to be @RULE data? What if the rulename is different? What if the ruledata is broken / truncated?
• What if the appended @RULE data happens to be obsolete, and a newer better compatible version of the same rule can be found in the "Rule:" namespace on the wiki? Silently installing an obsolete version fails to be an improvement over either asking the Golly user what they want to do, or else ignoring the appended @RULE data entirely.
• And vice versa, what if the appended @RULE data is actually a newer version of an already installed rule, and it should actually be used (overwriting the existing rulefile)? Again, either asking the user what to do, or ignoring appended @RULE data, seems to be better, compared to trying to guess.
• It may be hard to notice that there is a ruletable or a ruletree appended after the pattern data.
  For example, the user may end up accidentally installing a testing rule named "test_rule", and several weeks later failing to load a different pattern in a different testing rule also named "test_rule", because the newer "test_rule" is 20-state while the older (installed) "test_rule" is 3-state.

It would be nice if all of that could be accounted for in some useful way. But the simplest solution might be just to let Golly users install their rulefiles separately (manually), so that the Golly user will know that they have installed such-and-such rulefile.

When trying to open a pattern using a ruletable/ruletree, the rulefile is a dependency.
From my limited experience, trying to silently install dependencies is likely to become a good way to shoot oneself in the foot.

[dvgrn]

Anybody planning to add the @RULE-loading functionality?

I should have a first attempt at fixing up the "toSuper", "toHistory", and "toStandard" Lua scripts checked in a little later today -- the simple starter option, along these lines. Then I'm planning to go and wrestle with pattern files again for a while -- swapping in muzik's updated versions of MCell files, plus the other things in confocaloid's summary.

I won't get to the safeopenclip.lua @RULE-pattern-suffix support until after that's all done, so it's certainly safe for someone else to claim the task in the next few days.

[rowett]

When I first added the capability to LifeViewer (build 470, December 2019) to have an @RULE definition appended to a pattern it was to create a self-contained single file with everything needed to view the pattern.

Initially it was used to allow people to post patterns to the forum before the used rule was added to the LifeWiki repository. Later I used it in the showinviewer.lua script in Golly 3.4 (August 2020) to export a pattern in any rule that could then be viewed on the local machine in a web browser without needing access to the file system or a remote repository.

So the intention is that if there is an @RULE definition in the pattern file it should be used in preference to any local rule or repository rule. That way you can safely share a pattern knowing that it will work as you intended since it comes with the specific version of the rule you chose to append.

Ideally safeopenclip.lua should detect if there is an appended @RULE definition before attempting to open the pattern and if so use this as the rule for the pattern.
If no rule exists with the same name then a rule file could be created in the user's rule directory and the pattern then opened.
If a rule with the same name already exists things get awkward. It needs to be ignored but there's no tidy way I can see to do this in Golly from scripts.

[confocaloid]

I like the idea of a single self-contained file that contains both the pattern and a complete definition of the CA rule. I think it would be really nice to have identical functionality in Golly. Including ability to load both a pattern and an appended @RULE definition from a single text file. Maybe even directly reading the rule definition into memory, without needing any intermediate tempfiles. I understand that's probably unrealistic to expect at least in Golly 4.3.

[...]
So the intention is that if there is an @RULE definition in the pattern file it should be used in preference to any local rule or repository rule. That way you can safely share a pattern knowing that it will work as you intended since it comes with the specific version of the rule you chose to append.
[...]

[dvgrn]

If a rule with the same name already exists things get awkward. It needs to be ignored but there's no tidy way I can see to do this in Golly from scripts.

It would be a little bit fiddly, but it doesn't seem impossible to check that the @RULE appended to the pattern is character-for-character identical to the one that Golly is using. In most cases it will be identical, and then there's nothing awkward -- no need to tell the user anything, since that's the normal business-as-usual situation. Very often the reason it will be identical is that that exact rule was loaded on some previous occasion when that same pattern was opened.

To do the check properly, I suppose it would be necessary to look in both the user-specific Rules folder and the "supplied Rules folder" in Golly's root directory. Both of those locations can be retrieved from g.getdir(). Golly's Help has a section on how it finds .rule files:

If the RuleLoader algorithm is asked to switch to a rule called "Foo" then it first looks for Foo.rule in your rules folder. If not found, or if Foo.rule has no @TABLE or @TREE section, it will then look for Foo.rule in the supplied Rules folder. If Foo.rule doesn't exist then it will use the same search procedure to look for Foo.table, then Foo.tree.

When a rule can be found in Golly's Rule folder(s) with a name matching what's in the pattern comments, but the @RULE text in the pattern is not character-for-character identical with it, then that's definitely a situation where it would work to ask the user what to do. Possible actions that a user could choose would be

1) replace Golly's stored rule with the rule from the pattern comments, and open the pattern;
2) ignore the rule in the comments and use Golly's stored rule, and open the pattern;
3a) cancel opening the pattern;
3b) pop up an HTML window showing the two conflicting rule texts in two columns next to each other, and cancel opening the pattern.