AppleScript click Ui Element with modifier keys
I want to click an Ui Element button with modifier keys pressed.
Here is my code:
tell application "System Events"
tell process "Pro Tools"
tell (1st window whose title contains "Edit: ")
tell table "Group List"
tell row 2
tell UI element 2
click
end tell
end tell
end tell
end tell
end tell
end tell
I tried adding key down (option) and key up (option) between the click lines, but it doesn't work.
I do not have Pro Tools, so I can only offer some information that may help you.
First I have to say that UI Scripting can be kludgy and may not always work consistently depending on how its coded and the UI element
being targeted. It's always better if one can find a way to do something without resorting to UI Scripting first.
As I originally said to you in a comment... Basic vanilla AppleScript does not have a provision to preform a key down
and mouse click
simultaneously. You could try using a third-party utility like cliclick wrapped in a do shell script
command. You'd have to programmatically determine the position of UI element 2
to then click on it with cliclick
.
Without Pro Tools, I'm going to use Safari as an example, and the Show sidebar UI element
on its Toolbar in conjunction with cliclick
in a do shell script
command.
The purpose of following example AppleScript code is to demonstrate how to programmatically get the position
and size
of the target UI element
, in order to calculate where to click. It also assumes that Safari is already running with a standard window opened to, e.g., this question you posted.
activate application "Safari"
delay 1
tell application "System Events" to ¬
set ps to {position, size} ¬
of UI element 2 ¬
of toolbar 1 ¬
of window 1 ¬
of application process "Safari"
set p to item 1 of ps
set s to item 2 of ps
set x to (item 1 of p) + (item 1 of s) / 2 as integer
set y to (item 2 of p) + (item 2 of s) / 2 as integer
do shell script "$HOME/bin/cliclick c:" & x & "," & y
• Change $HOME/bin/cliclick
as appropriate to: /path/to/cliclick
When run in Script Editor, this clicked the Show sidebar UI element
on the Toolbar in Safari.
Now I know you said you want to click with the option key down, however for a practical example I'm going to modify the example AppleScript code, shown above, to control-click to the right of the Show sidebar UI element
on the Toolbar in Safari, which will pop-up the Customize Toolbar… menu.
Change:
set x to (item 1 of p) + (item 1 of s) / 2 as integer
To:
set x to (item 1 of p) + (item 1 of s) * 2 as integer
- Instead of dividing the
x
axis size by two to get the center of the targetUI element
on itsx
axis , we'll multiply itsx
axis size by two so it will click on the Toolbar itself.
Also Change:
do shell script "$HOME/bin/cliclick c:" & x & "," & y
To:
do shell script "$HOME/bin/cliclick kd:ctrl c:" & x & "," & y & " ku:ctrl"
Now when run in Script Editor, this control-clicked to the right of the Show sidebar UI element
on the Toolbar in Safari and popped-up the Customize Toolbar… menu.
With all of this in mind, maybe the following example AppleScript code will work for you, however remember it's untested as I do not have Pro Tools and may require you to rework the tell application "System Events" ...
statement:
activate application "Pro Tools"
delay 1
tell application "System Events" to ¬
set ps to {position, size} ¬
of UI element 2 ¬
of row 2 ¬
of table "Group List" ¬
of (first window whose title contains "Edit: ") ¬
of application process "Pro Tools"
set p to item 1 of ps
set s to item 2 of ps
set x to (item 1 of p) + (item 1 of s) / 2 as integer
set y to (item 2 of p) + (item 2 of s) / 2 as integer
do shell script "$HOME/bin/cliclick -r kd:alt c:" & x & "," & y & " ku:alt"
• Change $HOME/bin/cliclick
as appropriate to: /path/to/cliclick
Note: The example AppleScript code is just that and does not contain any error handling as may be appropriate. The onus is upon the user to add any error handling as may be appropriate, needed or wanted. Have a look at the try statement and error statement in the AppleScript Language Guide. See also, Working with Errors.
Note: The example AppleScript code for use with Safari and cliclick
was tested and worked as presented in macOS Catalina.
Note: I am not affiliated with the developer of cliclick
, just a satisfied user of the product.