Windows Python 3.x / Golly 4.x Troubleshooting Guide

Has something gone haywire? Let us know about it!
Post Reply
User avatar
dvgrn
Moderator
Posts: 11047
Joined: May 17th, 2009, 11:00 pm
Location: Madison, WI
Contact:

Windows Python 3.x / Golly 4.x Troubleshooting Guide

Post by dvgrn » April 30th, 2023, 5:32 pm

There's a Mac version of this thread from a couple of years ago, and there's been a little troubleshooting done on Discord recently for vaguely equivalent problems with getting Golly to find Python in Windows. Seems reasonable to document the current state of things in a separate thread.

The short summary is: things are still pretty much as bad as Randall Munroe said (as well as still being pretty much just as good as Randall Munroe said).

A short checklist for the steps to do a good install of Python can be found at the bottom of this post.

If You Don't Read Anything Else, Read This
You need Python 3.{something} if you want it to work with Golly 4.{something}. But the version of Python available from the Microsoft Store is, as far as I can tell so far, just plain not compatible with Golly.

It installs Python executables, but doesn't seem to install the Python DLLs that Golly expects to find, anywhere on the system.

Furthermore, when you install it and then uninstall it, it seems to leave annoying litter around -- zero-length files in a hidden directory, which keep getting pointed to by the "where python" command even when they should definitely have been cleaned up by the uninstaller.

The Rest of the News is Better
I have been able to get Python 3.10, Python 3.11, and Python 3.12 to work with Golly, with either of two different types of installs: the default install, which goes to the usual absolutely painfully ridiculous path -- e.g.,

C:\Users\{yourusername}\AppData\Local\Programs\Python\Python{version}\ (see screenshot below)
Default-Python-install.png
Default-Python-install.png (161.44 KiB) Viewed 9785 times
The custom install, where you choose to "install for all users" and then pick a specific path, can also be made to work:
Install-for-all-users.png
Install-for-all-users.png (161.94 KiB) Viewed 9785 times
As long as you haven't touched the Microsoft Store installation of Python, even with a ten-foot pole, you should be able to find the location of your Python install at any time by typing in "cmd" in the Start bar, opening a standard Windows command-line prompt, and typing where python.

Beware of Stupid Hidden Folders in Windows
If you choose the default options and end up with a per-user install of Python with the ridiculously long filename, you should be aware that Microsoft made a wonderfully silly design decision a long time ago, where they chose to put various important files in a folder that users can't see by default. The folder in question is

C:\Users\{your_user_name}\AppData
The folder is absolutely present and perfectly accessible by anyone... but you have to type the file path in to the path bar in File Explorer -- you can't click to navigate to it. This has always seemed utterly weird to me, and more than a little bit evil. Totally unnecessary confusion for users?
Way to go, Microsoft.

User avatar
dvgrn
Moderator
Posts: 11047
Joined: May 17th, 2009, 11:00 pm
Location: Madison, WI
Contact:

Re: Windows Python 3.x / Golly 4.x Troubleshooting Guide

Post by dvgrn » April 30th, 2023, 6:03 pm

dvgrn wrote:
April 30th, 2023, 5:32 pm
As long as you haven't touched the Microsoft Store installation of Python, even with a ten-foot pole, you should be able to find the location of your Python install at any time by typing in "cmd" in the Start bar, opening a standard Windows command-line prompt, and typing where python.
Here's what I had to do to get where python working again:

Microsoft says here that
You can disable the shortcuts without installing Python by opening "Manage app execution aliases" from Start, finding the "App Installer" Python entries and switching them to "Off".
Doing this will immediately remove the mysterious length-0 "python.exe" and "python3.exe" from

C:\Users\{your_user_name}\AppData\Local\Microsoft\WindowsApps

Moral: if you do a search and find Python executable files with a file length of zero, those are a bright new confusing idea by Microsoft called "app execution aliases", and Golly won't be happy with them because it wasn't designed with them in mind.

Getting Back to Working Order
Removing the evil aliases doesn't make "where python" work again immediately, however. Instead, even though I ran a complete installation of Python 3.11, and can see the relevant file in the right location ( C:\Python311\python311.dll ) ... in answer to "where python" I get
C:\>where python
INFO: Could not find files for the given pattern(s).

C:\>
I uninstalled Python 3.11, and installed it again with this checkbox checked:
Default-Python-install-add-to-PATH.png
Default-Python-install-add-to-PATH.png (186.31 KiB) Viewed 9769 times
After that, where python reported the correct location. And everything worked in Golly. -- Well, not quite instantly: the error dialog popped up, because GollyPrefs remembered the previous specific path that I had tried before. That was no longer the correct path, so that had to be fixed. But I was able to replace the incorrect path with just with the plain filename python311.dll, no need for a specific path.

I tried installing without that checkbox checked, and I was unable to get Golly to find the dll, even when I typed in the exact path. I uninstalled again, and installed to the same folder but with that checkbox checked, and everything Just Worked again.

Moral: install Python from python.org, not from Microsoft Store -- and check that Add to Path checkbox when installing.

EDIT: As long as you haven't previously messed up your system with a Microsoft-Store-type install, it's not a requirement to check the Add to Path checkbox. Golly will work fine without it -- I just tried this with a fresh install of Python 3.12.

Another Detail That's Probably Important
Earlier in this round of experimentation, I found that I could NOT leave the Golly error window open (the "type the exact path to python3*.dll" one), go install a good version of Python, come back and type in the DLL path, and have Golly work. Golly didn't seem to notice the new installation until I closed the application and re-opened. Then it worked fine.

It also might not be safe to try re-typing where python in an open command-line window after doing any install/uninstall changes, by the way. I believe command-line windows are likely to retain cached environment variable information. Safer to close and re-open.

Environment Variables
To look at current environment-variable settings in Windows, type "advanced" in the Start bar. Windows should offer a "View advanced system settings" choice. In the dialog that pops up, click "Environment Variables".

The installer adds two rows to the System PATH variable -- you can click on "Path" in the lower list, then click "Edit" to see the full list of folders that Windows will search by default. The Python installer adds two paths. I installed to C:\Python311, so I got C:\Python311 and C:\Python311\Scripts\ added to my PATH variable.

User avatar
dvgrn
Moderator
Posts: 11047
Joined: May 17th, 2009, 11:00 pm
Location: Madison, WI
Contact:

Re: Windows Python 3.x / Golly 4.x Troubleshooting Guide

Post by dvgrn » May 17th, 2023, 9:44 pm

I just spent half an hour on Discord trying to help someone get an installation of 64-bit Python 3.11 onto their computer, that Golly could use to run Python scripts.

Unfortunately, it seems that it's getting more and more difficult to convince the installer to lay down the DLL files that Golly needs.

In some but not all cases, what shows up by default are the 0-byte "App execution reparse points" or "Execution aliases", in %localappdata%\Microsoft\WindowsApps\. In that folder, you get python.exe, python3.11.exe, python3.exe, pythonw.exe, etc., etc. -- all listed as 0-byte "applications", but with no DLLs anywhere to be found.

I haven't figured out yet why the installer does different things for different people. It doesn't seem to be anything as simple as having/not having admin access on the device.

It turns out you can run fsutil on the execution aliases to find out where they're really stored...

Code: Select all

> fsutil reparsepoint query %localappdata%\Microsoft\WindowsApps\python.exe

Reparse Tag Value : 0x8000001b
Tag value: Microsoft

Reparse Data Length: 0x1c2
Reparse Data:
0000:  03 00 00 00 50 00 79 00  74 00 68 00 6f 00 6e 00  ....P.y.t.h.o.n.
0010:  53 00 6f 00 66 00 74 00  77 00 61 00 72 00 65 00  S.o.f.t.w.a.r.e.
0020:  46 00 6f 00 75 00 6e 00  64 00 61 00 74 00 69 00  F.o.u.n.d.a.t.i.
0030:  6f 00 6e 00 2e 00 50 00  79 00 74 00 68 00 6f 00  o.n...P.y.t.h.o.
0040:  6e 00 2e 00 33 00 2e 00  31 00 30 00 5f 00 71 00  n...3...1.0._.q.
0050:  62 00 7a 00 35 00 6e 00  32 00 6b 00 66 00 72 00  b.z.5.n.2.k.f.r.
0060:  61 00 38 00 70 00 30 00  00 00 50 00 79 00 74 00  a.8.p.0...P.y.t.
0070:  68 00 6f 00 6e 00 53 00  6f 00 66 00 74 00 77 00  h.o.n.S.o.f.t.w.
0080:  61 00 72 00 65 00 46 00  6f 00 75 00 6e 00 64 00  a.r.e.F.o.u.n.d.
0090:  61 00 74 00 69 00 6f 00  6e 00 2e 00 50 00 79 00  a.t.i.o.n...P.y.
00a0:  74 00 68 00 6f 00 6e 00  2e 00 33 00 2e 00 31 00  t.h.o.n...3...1.
00b0:  30 00 5f 00 71 00 62 00  7a 00 35 00 6e 00 32 00  0._.q.b.z.5.n.2.
00c0:  6b 00 66 00 72 00 61 00  38 00 70 00 30 00 21 00  k.f.r.a.8.p.0.!.
00d0:  50 00 79 00 74 00 68 00  6f 00 6e 00 00 00 43 00  P.y.t.h.o.n...C.
00e0:  3a 00 5c 00 50 00 72 00  6f 00 67 00 72 00 61 00  :.\.P.r.o.g.r.a.
00f0:  6d 00 20 00 46 00 69 00  6c 00 65 00 73 00 5c 00  m. .F.i.l.e.s.\.
0100:  57 00 69 00 6e 00 64 00  6f 00 77 00 73 00 41 00  W.i.n.d.o.w.s.A.
0110:  70 00 70 00 73 00 5c 00  50 00 79 00 74 00 68 00  p.p.s.\.P.y.t.h.
0120:  6f 00 6e 00 53 00 6f 00  66 00 74 00 77 00 61 00  o.n.S.o.f.t.w.a.
0130:  72 00 65 00 46 00 6f 00  75 00 6e 00 64 00 61 00  r.e.F.o.u.n.d.a.
0140:  74 00 69 00 6f 00 6e 00  2e 00 50 00 79 00 74 00  t.i.o.n...P.y.t.
0150:  68 00 6f 00 6e 00 2e 00  33 00 2e 00 31 00 30 00  h.o.n...3...1.0.
0160:  5f 00 33 00 2e 00 31 00  30 00 2e 00 33 00 30 00  _.3...1.0...3.0.
0170:  35 00 36 00 2e 00 30 00  5f 00 78 00 36 00 34 00  5.6...0._.x.6.4.
0180:  5f 00 5f 00 71 00 62 00  7a 00 35 00 6e 00 32 00  _._.q.b.z.5.n.2.
0190:  6b 00 66 00 72 00 61 00  38 00 70 00 30 00 5c 00  k.f.r.a.8.p.0.\.
01a0:  70 00 79 00 74 00 68 00  6f 00 6e 00 33 00 2e 00  p.y.t.h.o.n.3...
01b0:  31 00 30 00 2e 00 65 00  78 00 65 00 00 00 30 00  1.0...e.x.e...0.
01c0:  00 00                                             ..
If I could read that a little better, maybe I could extract a path to a location that actually contains the DLLs -- C:\Program Files\WindowsApps\PythonSoftwareFoundation\Python... something.

Unfortunately, even with admin permissions I can't easily get so much as view access to the Program Files\WindowsApps folder. And searches don't seem to find the Python DLLs in that location. So it's not clear if Golly could access that location, even if I could find the path and put it in the dialog box.

... What goofball designed this method of installing Python, anyway? Seems even sillier than the horrible hidden AppData location where previous versions were ending up.

Here's the advice that I tried with Python 3.11.0, after the default local install didn't work:

1) Uninstall Python, if you've installed it unsuccessfully before.
2) Run the Python installer again -- making sure it's the 64-bit one downloaded direct from python.org, not some Microsoft-contaminated installer.
3) Click the checkbox for "Add python.exe to PATH".
4) Click "Customize installation"
5) Check the "install Python for all users" checkbox -- probably not the one on the current page, that's just for the Python launcher, but rather at the top of the next page.
6) Take the rest of the defaults for the Python installer.

That was basically what had worked for me, for installing visible DLLs for Python 3.10 and (some subversion of) Python 3.11 -- but when we tried that today, we still got the useless execution aliases' location as a response to "where python".

User avatar
dvgrn
Moderator
Posts: 11047
Joined: May 17th, 2009, 11:00 pm
Location: Madison, WI
Contact:

Re: Windows Python 3.x / Golly 4.x Troubleshooting Guide

Post by dvgrn » September 3rd, 2023, 6:01 pm

Just for the record, I tested the latest 3.11.5 Python on Windows 11, with the following settings:
Python-3.11.5-install-settings.png
Python-3.11.5-install-settings.png (167.46 KiB) Viewed 9205 times
The DLL showed up at C:\Python311\python311.dll -- and once I told Golly about that location (by editing C:\Users\{my_username}\AppData\Roaming\Golly\GollyPrefs ) it seems to be working fine.

EDIT: Same for Python 3.12, experimentally installed this time to C:\Program Files\Python312.

I went and found the relevant GollyPrefs and, while Golly was closed, updated the key line as follows:
python3_lib=C:\Program Files\Python312\python312.dll
NOTE: The key location for me was not the AppData\Roaming location. Golly looks first for a GollyPrefs in the same folder as the application executable -- and I had a GollyPrefs in there. So the first time around it didn't work, because I went and edited the old default one in AppData\Roaming. But as soon as I found and fixed the right GollyPrefs, it worked fine.

Post Reply