Bit of a random one, but while troubleshooting an issue with one of my multi-tenant platforms, I logged into the Batch server (as FNMS sys admin account) to verify that the system was processing the backlog of jobs that built up, but the "BatchProcessTaskConsole.exe list-tasks" command is not displaying anything, not even an error message. It only just thinks about something for a couple moments and then spits out a new command line:
I've checked that the FNMS Batch Processor and FNMS Batch Process Scheduler services are running along with the Messaging Queue service. I've stop and restarted all the services, and still nothing comes up. I can see that heaps of ShadowHost.exe processes are popping in and out which lead me to think that the backlog of jobs are being processed and completed, but it would be ideal to be able to actually verify that they're completing.
For reference, I logged into one of my other multi-tenant platforms' Batch server (as FNMS sys admin) and ran the command and it worked as expected, so I'm a bit stumped as to what might be the issue.
Thanks in advance 🙂
Aug 17, 2023 12:13 AM
Have you checked the BatchProcessScheduler and BatchProcessTask logs (C:\ProgramData\Flexera Software\Compliance\Logging)? Maybe increase the log level?
etap.trace does not seem to have anything related to the batch scheduler/processor.
Aug 17, 2023 06:23 AM
Thanks for suggestion. With help of a colleague I was able to use a SQL query to put a list of the queued jobs up to get my confirmation. Didn't see anything that stuck out in the logs, but I'll try increasing the level to see what it spits out.
Aug 22, 2023 07:29 PM
Yes sir, it was run in an elevated command prompt. The other commands of the .EXE seemed to execute as expected, which just added to my confusion. I'm going to try @mfranz's suggestion and increase the logging level to see if there's anything that comes through in there.
Aug 22, 2023 07:32 PM
Can you try running "BatchProcessTaskConsole.exe list-tasks" as the Windows service user account, please?
This command will query the Microsoft Message Queue (MS MQ) information. The account running the command must at least have read access to the MS MQ. Running this command using the "run as Administrator" option will use the built in local "Administrator" account, which does not necessarily has read access to MS MQ.
As an alternative, you can also query the [BatchProcessExecution] table in the [FNMSCompliance] database. Any process with NULL in the [FinishTime] column is considered to be running still (or it has been terminated unexpectedly ...)
Aug 23, 2023 06:52 AM
I wasn't able to run it as the Windows service user account, however I was able to verify that the fnms-sys-admin account that I logged onto the batch server as has read access to the messaging queue. I verified this by going into Computer Management > Services and Applications > Message Queuing > Private Queues and am able to see the FNMS queues listed, although all of them show 0 messages or journal messages in their queues.
@mfranz I increased the logging level of both logs as you have suggested, but unfortunately it does not look like I'm getting any information out of them, or that the log is even capturing anything.
Aug 29, 2023 01:42 AM