Hello all,
I wanted to see if what i'm experience is expected. We have an EC2 instance running the beacon and is assigned an IAM role to grant the policies it needs to read all the instance data from the various accounts. Our Cloud OPS team has granted this across the board so it appears to be working as designed. There are currently about 70+ accounts that our beacon has access to, providing info on almost 4000 instances.
Best practice says that this job should be running every 30 minutes so that you capture "terminated" instances before they disappear and that association is lost. The problem is that this job takes almost 1213 hours to complete.
Here's the log snippet that shows how long each piece is taking on the reader.
---------------------
2022-01-05 10:00:54,054 [INFO ] Reading 'Computer' data from 'AWS Inventory' (ver 1.0) data source
2022-01-05 10:00:54,054 [INFO ] Reading data for 'Computer' successfully processed in 0 seconds.
2022-01-05 10:00:54,054 [INFO ] Reading 'VirtualMachine' data from 'AWS Inventory' (ver 1.0) data source
2022-01-05 10:00:54,069 [INFO ] CreateTempTableForEc2Instances (Execute on FNMP)
2022-01-05 10:00:54,069 [INFO ] ExecuteOnTarget steps only execute when connected to the target.
2022-01-05 10:00:54,069 [INFO ] Completed in 0 seconds.
2022-01-05 10:00:54,069 [INFO ] Get AWS EC2 dedicated hosts information (Transfer data from source 'AWS Inventory' to FNMP)
2022-01-05 15:34:52,353 [INFO ] Wrote 0 records to C:\ProgramData\Flexera Software\Beacon\IntermediateData\AWS Inventory\VirtualMachine--110--Get AWS EC2 dedicated hosts information.int.xml
2022-01-05 15:34:52,353 [INFO ] Successfully processed in 5 hours, 33 minutes, 57 seconds.
2022-01-05 15:34:52,353 [INFO ] Get AWS EC2 instance information (Transfer data from source 'AWS Inventory' to FNMP)
2022-01-05 18:11:40,311 [INFO ] Wrote 3540 records to C:\ProgramData\Flexera Software\Beacon\IntermediateData\AWS Inventory\VirtualMachine--120--Get AWS EC2 instance information.int.xml
2022-01-05 18:11:40,311 [INFO ] Successfully processed in 2 hours, 36 minutes, 47 seconds.
2022-01-05 18:11:40,311 [INFO ] Get AWS EC2 reserved instance information (Transfer data from source 'AWS Inventory' to FNMP)
2022-01-05 20:51:43,311 [INFO ] Wrote 757 records to C:\ProgramData\Flexera Software\Beacon\IntermediateData\AWS Inventory\VirtualMachine--130--Get AWS EC2 reserved instance information.int.xml
2022-01-05 20:51:43,311 [INFO ] Successfully processed in 2 hours, 40 minutes, 2 seconds.
2022-01-05 20:51:43,311 [INFO ] Get AWS EC2 account information (Transfer data from source 'AWS Inventory' to FNMP)
2022-01-05 20:52:30,109 [INFO ] Wrote 2 records to C:\ProgramData\Flexera Software\Beacon\IntermediateData\AWS Inventory\VirtualMachine--140--Get AWS EC2 account information.int.xml
2022-01-05 20:52:30,109 [INFO ] Successfully processed in 46 seconds.
2022-01-05 20:52:30,109 [INFO ] Populate the AWS EC2 dedicated hosts data on the target database (Execute on FNMP)
2022-01-05 20:52:30,109 [INFO ] ExecuteOnTarget steps only execute when connected to the target.
2022-01-05 20:52:30,109 [INFO ] Completed in 0 seconds.
2022-01-05 20:52:30,109 [INFO ] Populate the AWS EC2 instance data on the target database (Execute on FNMP)
2022-01-05 20:52:30,109 [INFO ] ExecuteOnTarget steps only execute when connected to the target.
2022-01-05 20:52:30,109 [INFO ] Completed in 0 seconds.
2022-01-05 20:52:30,109 [INFO ] Populate the AWS EC2 reserved instance data on the target database (Execute on FNMP)
2022-01-05 20:52:30,109 [INFO ] ExecuteOnTarget steps only execute when connected to the target.
2022-01-05 20:52:30,109 [INFO ] Completed in 0 seconds.
2022-01-05 20:52:30,109 [INFO ] Set terminated instances on the staging tables (Execute on FNMP)
2022-01-05 20:52:30,109 [INFO ] ExecuteOnTarget steps only execute when connected to the target.
2022-01-05 20:52:30,109 [INFO ] Completed in 0 seconds.
2022-01-05 20:52:30,109 [INFO ] CleanUpTargetTempTable (Execute on FNMP)
2022-01-05 20:52:30,109 [INFO ] ExecuteOnTarget steps only execute when connected to the target.
2022-01-05 20:52:30,109 [INFO ] Completed in 0 seconds.
2022-01-05 20:52:30,109 [INFO ] Reading data for 'VirtualMachine' successfully processed in 10 hours, 51 minutes, 33 seconds.
----------------------------
If it only runs once a day, i'm fine with that....i just want to make sure there isn't another issue, and that this is expected with these numbers.
We have another environment equal to this, and another that is double to triple this size....so i'm concerned about the larger environment when that time comes.
Thanks.
