I just had my first proper go at adapting some stock worklet code to install some different software and noticed that testing evaluation code in Automox isnt terribly practical. According to the docs “Manual Execution… triggers the remediation script regardless of the compliance status of the device.” So when using manual execution (which is the easiest way to test new worklets) the evaluation code is completely ignored.
I think the solution I would like to see would be a manual execution “Debug” mode that you could run to test both evaluation and remediation code, and would preferably also return the full output from both scripts (feedback from the scripts themselves seems a bit thin atm, so it would be nice if the activity log or something was more verbose). Am I just doing things wrong atm or is this actually a worthwhile suggestion? How does everyone else go about testing their scripts before running them in production?
Page 1 / 1
I’ve run into the same issue. The fact that Execute Now doesn’t run the evaluation code first is counter-intuitive. The PMs here agreed so hopefully we can either change that or give you a setting to choose. Agreed also on the full output of both code sections. Great feedback! You’re not doing anything wrong, we just have more features and UX design to work on to make it even easier to use.
Thanks @Nic, glad Im not the only one who is having trouble with this! Do you have an internal ticket or something for this issue that you could link to this thread to provide updates or is that not really something you guys do?
Anyway while Im thinking about it, here are a bunch more quality of life changes regarding the creation of worklets that I think would make the whole process much nicer:
Some basic syntax highlighting for shell script would be cool (like it is for powershell)
Being able to expand the size of the script editor would also be useful
When creating a new policy, the policy should automatically prompt for saving when the the “add group button” is pressed (currently pressing this button destroys unsaved policies)
Under “schedule” the months, weeks and days should all have “select all” buttons, or at least there should be some presets at the top to automatically schedule running for once a day, once a week or once a month. (Or maybe I am misunderstanding how this works currently?)
Now that I think about it the policy creation screen would be the prefect place to have some sort of test run option like the ‘debug’ mode I mentioned earlier. If you could press the a button to run a full test of the script on a specified test device with realtime feedback on the output of the script on the device, you wouldnt even need to leave the worklet creation UI to build a worklet ready for production.
In the system management screen show the OS icon next to policies based on the os the policy is aimed at
Just a few thoughts, hope there is some useful feedback here!
Those are all good idea and I’ll pass them along to the UX and product folks here. The policy not saving when you press Add Group sounds like a bug and we should fix that one for sure.
Super clever idea Ryan, I’m taking note. I’ll hit you up when we are doing this design for input/feedback!
I just noticed the section where you asked about updates on internal tickets. To my knowledge we don’t have anything like that, but I’ll ask around. We have the product board but I’m not sure if it sends out email notifications when we move something into the competed section.
Cool, thanks @Nic! If that policy saving issue is confirmed to be a bug then ill take one of them bug finder badges ?
I just noticed the section where you asked about updates on internal tickets. To my knowledge we don’t have anything like that, but I’ll ask around. We have the product board but I’m not sure if it sends out email notifications when we move something into the competed section.
Regarding the above it would be nice to have a bit more feedback on what is being worked on. The product board doesnt seem to be updated terribly often, and Im not sure if email notifications are sent either as nothing that I have voted on or suggested has ever moved categories or been added to the under consideration board.
We are working on it - we’re in the midst of some growing pains at the moment. We’re hiring another product manager to help with the workload and that will let us keep the product board more current. The other thing we’re working on is a new system to collect, number crunch and then respond to all customer feedback, so that you know what features you care about are being worked on and when they are released. Stay tuned!
One of the PMs did opine that the evaluation code issue was a bug, so the badge is yours!
The original topic of this thread seems to still be an issue though other features have been added. I only learned this today using Start-Transcript in my evaluation code to dump info to a text file, finding that it was working as planned, but that my remediation code was running.. specifically in cases I did not want it to run. Luckily this was just on a test machine, but its not expected, intuitive, or described behavior on the policy page and could lead to accidentally running remediation code against machines the policy is designed not to make changes to for good reason.
Not only is it potentially dangerous, it reduces the ability to manually run policies to immediately take action, rather than changing a policy schedule you may want to remain unchanged.
