Using PyAutoGUI on the APCD
Posted: July 20th, 2025, 9:50 pm
With all the talk of AI and automation and bots scrolling social media, I have always wondered if there was anything that could be done to automate work in the APCD. After a few YouTube videos and getting into coding with Python, I have found a way to automate the clicks needed to add vertices around a shape. The results so far are very encouraging. PyAutoGUI searches the current screen for a small image and then moves and clicks the mouse on the location of the small image within the big image (screen). No automation is perfect and there is still manual work needed to clean up the course mesh after the verts are added, but this is a considerable time saver in the texturing process.
Here is a link to the script I created. Hopefully there are better coders out there than me who can improve on it. You will need to copy all the image files and the beep sound to go with the .py script.
https://drive.google.com/drive/folders/ ... sp=sharing
To use the Python .py script in the share folder, you will need to install Python. Once installed, you will need to install the pyautogui, opencv and pillow modules for python. Adding Python modules is done via the command line prompt in windows which I don't like to use, but it is a straight forward process. There is lots of information on installing modules, running Python scripts, and code examples that comes up with a google search.
Before you try to run the python script, you will need to edit it for the dimensions of your computer screen. My desktop screen is 1920x1200 and the active area of the APCD is smaller. Take a screenshot of your screen and determine the dimensions of the top - left corner (x1,y1) and the bottom - right corner (x2,y2). Fill those numbers into the top part of the script. For my screen it is roughly (200,120) and (1850,1050).
The work flow in the APCD goes as follows:
1) Draw a shape of the "None" category around the terrain to be inserted.
2) Turn off the display of textures. Shapes and hull need to be visible against the grey background. Turning vert display on helps to reduce verts being added close to existing verts.
3) Set up the terrain painter to add vertices.
4) When ready to start clicking to add vertices around the shape, go to the Python script and run it.
5) There is a time delay of 5 seconds for you to get back to the APCD screen.
6) The script will move the mouse and click to add vertices. When done, there is a beep.
Important considerations:
1) The fail safe for PyAutoGUI is to move the mouse to one of the extreme corners of the screen. Leave it there until the script is done. Remember this if the script starts clicking on places you don't want and you want to abort.
2) I set the move time and click time delays to be relatively slow. They can be set lower to speed up the script. The APCD screen needs time to redraw after a vert is added so don't set the time delays too fast.
3) The script looks for the "None" shape color (RGB = 255,0,255) with a square mesh (RGB = 0,0,0) against the standard APCD background color (RGB = 176,176,176). The hole boundary shapes will get clicked on (RGB = 255,0,0). Be mindful of any shapes on the screen that are not "None". The confidence parameter can be adjusted but I found it tough to get the right balance.
4) The script can also be run on the add vertices function instead of the terrain painter. Be very careful of verts that "slide" toward another vert when added. The script will keep clicking to add more verts and make quite a mess. Adding verts will leave more edges to turn during clean up than the terrain painter but no vert clusters the terrain painter is known for.
I am likely leaving out some details that I can't think of at the moment. Hopefully someone out there can use Python and see how this script works for them. There are likely other things in the APCD that could be click automated. I am sure this will start some interesting discussions!
Here is a link to the script I created. Hopefully there are better coders out there than me who can improve on it. You will need to copy all the image files and the beep sound to go with the .py script.
https://drive.google.com/drive/folders/ ... sp=sharing
To use the Python .py script in the share folder, you will need to install Python. Once installed, you will need to install the pyautogui, opencv and pillow modules for python. Adding Python modules is done via the command line prompt in windows which I don't like to use, but it is a straight forward process. There is lots of information on installing modules, running Python scripts, and code examples that comes up with a google search.
Before you try to run the python script, you will need to edit it for the dimensions of your computer screen. My desktop screen is 1920x1200 and the active area of the APCD is smaller. Take a screenshot of your screen and determine the dimensions of the top - left corner (x1,y1) and the bottom - right corner (x2,y2). Fill those numbers into the top part of the script. For my screen it is roughly (200,120) and (1850,1050).
The work flow in the APCD goes as follows:
1) Draw a shape of the "None" category around the terrain to be inserted.
2) Turn off the display of textures. Shapes and hull need to be visible against the grey background. Turning vert display on helps to reduce verts being added close to existing verts.
3) Set up the terrain painter to add vertices.
4) When ready to start clicking to add vertices around the shape, go to the Python script and run it.
5) There is a time delay of 5 seconds for you to get back to the APCD screen.
6) The script will move the mouse and click to add vertices. When done, there is a beep.
- 1_screen_before.png (93.2 KiB) Viewed 41636 times
- 2_screen_after.png (117 KiB) Viewed 41636 times
- 3_textured.png (1.83 MiB) Viewed 41636 times
Important considerations:
1) The fail safe for PyAutoGUI is to move the mouse to one of the extreme corners of the screen. Leave it there until the script is done. Remember this if the script starts clicking on places you don't want and you want to abort.
2) I set the move time and click time delays to be relatively slow. They can be set lower to speed up the script. The APCD screen needs time to redraw after a vert is added so don't set the time delays too fast.
3) The script looks for the "None" shape color (RGB = 255,0,255) with a square mesh (RGB = 0,0,0) against the standard APCD background color (RGB = 176,176,176). The hole boundary shapes will get clicked on (RGB = 255,0,0). Be mindful of any shapes on the screen that are not "None". The confidence parameter can be adjusted but I found it tough to get the right balance.
4) The script can also be run on the add vertices function instead of the terrain painter. Be very careful of verts that "slide" toward another vert when added. The script will keep clicking to add more verts and make quite a mess. Adding verts will leave more edges to turn during clean up than the terrain painter but no vert clusters the terrain painter is known for.
I am likely leaving out some details that I can't think of at the moment. Hopefully someone out there can use Python and see how this script works for them. There are likely other things in the APCD that could be click automated. I am sure this will start some interesting discussions!
