After Upgrade to FNMS 2022 R1, Beacon List Says All Beacons Have Connectivity Issues. But Open a Specific Beacon's Details and the Details Page Says The Beacon is Operating Normally.

[mfeinman]

Hi all ...

I successfully upgraded my production environment from FNMS 2019 R1 to FNMS 2022 R1 last week.  Everything appears to be working properly, yet there is one issue that I can't seem to resolve.

When I go to my Beacon list, the page says that every beacon has a connectivity issue and the beacon status is "not reporting normally" (real names redacted):

However, the policy status and total to upload columns update.  When I log onto an individual beacon, I see inventory being placed in the Incoming\Inventories folder and then leaving the folder a short time thereafter.  I see the inventory uploaded to the FNMS server,  According to the online help, the connectivity status "Check connectivity" means "the central application server has never received a status message from this inventory beacon".  When I look at an individual beacon's BeaconEngine.log, I see it generating a Beacon status every 10 minutes (and have also seen the .bstat file come and go in the Incoming folder):

2023-01-30 11:29:32,002 [Scheduler.ScheduledJob|status] [INFO ] Executing job: status
2023-01-30 11:29:32,002 [llers.BeaconController|status] [INFO ] Scheduled job triggered: report-status.
2023-01-30 11:29:32,002 [llers.ReportController|status] [INFO ] Running job to report current beacon status.

I can then see the corresponding PUT statements for the .bstat files from the IIS log of the FNMS server with a valid 200 http code; for example:

2023-01-30 00:21:02 10.193.215.246 PUT /ManageSoftRL/BeaconStatus/20230129191935-{DC882199-F895-4DF8-82B7-C4F921417627}.bstat - 443 Domain1\Beacon1$ 10.11.12.13 FlexNet+Manager+Platform/13.2+(Windows+x86) - 200 0 0 5

FNMS's own beaconStatus.log indicates the beacon status files are being imported; I saw no errors at all in this log.  For example:

[INFO 2023-01-30 11:59:04,037 89162059ms StatusImportPlugin ] Completed Import of 20230130115834-{E16723EA-8EDE-4786-B2CE-FF6F39A7AB03}.bstat : 47ms

Doesn't that tell me that the Beacon status was received?

I have Beacons running 2021 R1 and 2019 R1 Beacon server code (I didn't upgrade them yet); the same status is being reported for both on the Beacon List page.

However, if I drill down to the Beacon details, the status there is "Operating Normally":

Have any of you ever seen this? How do I debug this inconsistency?

--Mark

[jasonlu]

Hi, in the past I've seen strangeness like this, which from memory went away when a restarted the beacon service and (I think) save the Inventory Settings page again, which then updates all the beacons which a revised policy.

I've also stood up new beacons that had this issue until a restart of the beacon service.

Yeah I'm very interested in understanding why the difference in status between the beacon list and the individual beacon pages.

j

[mfeinman]

On the Inventory Settings page, I changed the interval for beacon updates from 30 to 20 minutes to effect a change.  Then I watched the List page.  All beacons' policy status went from "up to date" to "out of date" while awaiting the new policy.  Slowly all beacons eventually reported that they were up to date again, so that piece seems to be working, but the Connectivity & Beacon Status columns remain as "Check Connectivity" and "Not Reporting Normally".

I then went to one of my Beacons and restarted the Beacon Service.  I waited a while but did not see any change to the List.

Since I restarted the Beacon service, I went back to the beacon settings page and changed the interval for beacon updates from 20 to 15 minutes.  While I saw the policy being sent over, the status columns still did not change.

There's some process doing this monitoring, right? What log is associated with this process so I can turn on extended debugging and see if that yields any clues?

--Mark

[durgeshsingh]

hi

can you share parent connection status. 

[mfeinman]

No issue with the parent connection from the beacon:

[durgeshsingh]

Hi

Please check Beacon Upload status in task scheduler & stop & start task.

[mfeinman]

Upload Beacon Status task in Task Scheduler appears to run normally every minute.

[mfeinman]

I've also attached a recent Beacon status file. It looks OK to me. (I had to upload it with a .XML extension; couldn't upload it as a .TXT file.)

--Mark

[mfeinman]

@jasonlu @durgeshsingh 

Evidently this is a known issue when the database server and the application server are set to have different time zones, which is true in my case.

Known Issue: All Beacons grid may show the "Not reporting normally" or "Check connectivity" status for fully functioning beacons if the inventory and database servers have different time zones (IOJ-2244613)

I may now have a case for an exception to my agency standard that the SQL Server time for the FNMS database server has to be in UTC.

--Mark

[mfeinman · Accepted solution]

To close the loop on this one, the time zone on the database server was changed to match all the others (US Eastern) and now the Beacon List page reports all Beacons are communicating properly.

--Mark