Installing Data Platform Extractor to upload Data to IT Visibility

[#1]

Starting from beacon version 14.2.0, Data Platform standalone extractors can be executed on the beacon to collect data using specific loader configurations (refer to the table below). The data will be imported into IT Visibility. 

Existing custom Data Platform extractors can be used with IT Visibility as long as they use loader configurations listed below. If you need to create a new custom extractor for IT Visibility, contact Flexera Services.

Prerequisites

• You are using IT Visibility, as the data is meant to be imported into IT Visibility.
• Beacon version 14.0.2 or later.
• If you are also using IT Asset Management, you do not need to install a separate new beacon; just make sure one of your IT Asset Management beacons is upgraded to version 14.0.2 or later. Note that the extracted data will be sent to IT Visibility from IT Asset Management but will not be processed in IT Asset Management.

To start using standalone extractors on beacon

1. Copy the attached BDNA_Normalize_Extractor_5.5.35_3003.zip file (at the bottom of this article) to a directory on Beacon server, e.g. C:\DPV5. Then, extract the zip file.
2. After extracting the zip file, you will see ExtractorLoader.cmd, FLEXERA_Normalize_Extractor_XXXX.exe, and a few config files for Solarwinds Orion and JAMF Casper in the extracted files. Double-clicking FLEXERA_Normalize_Extractor_XXXX.exe will create a new folder called EXTRACTION in the same directory. In the EXTRACTION folder, you should be able to see Extractor.exe, which can be run later with a configuration file supplied.
   
   
3. Create a folder with the name DataPlatformExtractor if it does not exist under the directory C:\ProgramData\Flexera Software\Beacon\IntermediateData on the Beacon server.
4. Open a command line terminal and run the following command as an example. You'll need to replace the corresponding options according to your environment.

   C:\DPV5\EXTRACTION\Extractor.exe -X "C:\DPV5\orion(11.5.1).extractor(MSS).config" -O "C:\ProgramData\Flexera Software\Beacon\IntermediateData\DataPlatformExtractor" -H localhost -PT 1433 -C SolarWindsOrion -U sa -P Admin123456789 -I false -F SolarWindsOrion_US.zip

   Extractor Command Options and Example Values

   [-X]  <Xml Config file full path> ; C:\DPV5\orion(11.5.1).extractor(MSS).config
   [-O]  <Output file path> ; C:\ProgramData\Flexera Software\Beacon\IntermediateData\DataPlatformExtractor
   [-F]  <Output file name> ; SolarWindsOrion_US.zip
   [-H]  <Database host name> ; localhost
   [-PT] <Database port> ; 1433
   [-C]  <Database Catalog name> ; SolarWindsOrion
   [-U]  <Database user name> ; sa
   [-P]  <Database user password> ; Admin123456789
   [-I]  <Database Windows Authentication> ; true or false

   For the other Extractor options, run the command Extractor.exe -?.

   1. Choose the correct config file to read data from your source. This is a mapping between their versions and the corresponding config file you should use in the command line.
      
      

      Extractor	Extractor Config File
      Solarwinds Orion (11.5.1,12.0,12.1) (MSS)	orion(11.5.1).extractor(MSS).config
      Solarwinds Orion (12.1) (MSS)	orion(12.1.0).extractor(MSS).config
      JAMF Casper (earlier than version 9)	casper.extractor(mysql).config
      JAMF Casper 9	casper9.extractor(mysql).config
      JAMF Casper 10	casper10.extractor(mysql).config
      ASG Trackbird	trackbird.extractor(MSS).config, trackbird.extractor(ORA).config
      BDNA Discover	discover.extractor(ORA).config
      BMC BladeLogic	bladelogic(8.8).extractor(ORA).config, bladelogic.extractor(ORA).config, bladelogic.schema_name.extractor(ORA).config
      CA IT Client Manager (ITCM)	itcm.extractor(MSS).config
      CiscoWorks LAN Management Solution (LMS)	ciscoworks.extractor(SQA).config
      HP Network Automation	hpna.extractor(ORA).config
      LANDesk	landesk.extractor(MSS).config, landesk.extractor(ORA).config, landesk.schema_name.extractor(ORA).config
      Lansweeper	lansweeper.extractor(MSS).config, lansweeper5.extractor(MSS).config, lansweeper6.extractor(MSS).config
      Novell ZENworks	zenworks.extractor(MSS).config
      OCS	ocs.full.extractor(mysql).config
      Scalable	scalable.extractor.config
      HP Discovery and Dependency Mapping Advanced (DDMa)	hpddma.extractor(MSSQL).config, hpddma.extractor(ORA).config, hpddma.schema_name.extractor(ORA).config
      IBM TCM	tivoli.tcm.extractor(MSS).config, tivoli.tcm.extractor(ORA).config
      IBM Tivoli Application Dependency Discovery Manager	taddm.extractor(DB2)(9.5)(L2INTRFC).config, taddm.extractor(DB2).config
      IBM Tivoli Asset Discovery for Distributed	tad4d.extractor(DB2)(7.5).config, tad4d.extractor(DB2).config
      IBM Tivoli Asset Discovery for z/OS	tad4z.extractor(DB2).config
      Microsoft Assessment and Planning Toolkit (MAP) 7	map.full.extractor.config
      Microsoft Assessment and Planning Toolkit (MAP) 8	map8.extractor(MSS).config
      Radia Client Automation Enterprise	radia.extractor(MSSQL).config
      Lakeside SysTrack	systrack.extractor(MSS).config

   2. Use a unique name for the output file name for each of the individual connections. The output file name will be used as a unique identifier in IT visibility and as the connection name, which will be visible on FlexeraOne IT Visibility UI.
   3. The output file path must be C:\ProgramData\Flexera Software\Beacon\IntermediateData\DataPlatformExtractor. Unless your beacon is installed on a different drive then you can replace C drive with the correct one.

      If the same file already exists in folder C:\ProgramData\Flexera Software\Beacon\IntermediateData\DataPlatformExtractor, it will overwrite it next time when generating the new one.

      

      If you cannot see the generated zip file, please review the command options you used or see more details in the log folder. e.g. C:\DPV5\log\normalize.txt.

      
5. Once the above command executes successfully and you can see the generated zip file in C:\ProgramDataFlexera SoftwareBeaconIntermediateDataDataPlatformExtractor, use Windows Task Scheduler to schedule each of your individual extractor calls. In Windows Task Scheduler, you can create a task that runs regularly, for example, daily or weekly.

   On the Action tab, specify the full path of Extractor.exe for Program/script. For Add arguments (optional),  enter the command options you used in step five.

   
6. Once the scheduled task is created, it will generate the zip file in C:\ProgramData\Flexera Software\Beacon\IntermediateData\DataPlatformExtractor regularly based on the trigger you set in the scheduled task.

   NOTE: If you have multiple connections to several data sources, create a scheduled task for each connection. Leave enough time between them for one to finish before the next one starts to run.

7. Each time the scheduled task Upload third-party inventory data runs, it will upload any zip files within  C:\ProgramData\Flexera Software\Beacon\IntermediateData\DataPlatformExtractor folder to C:\ProgramData\Flexera Software\Beacon\IntermediateData\Upload.

   Any error during the upload of the file will be moved to the Error folder C:\ProgramData\Flexera Software\Beacon\IntermediateData\DataPlatformExtractor\Error.

   It will only keep the latest error zip file per connection. Upload third-party inventory data is defaulted to run every ten minutes. If the Data Platform extractor is scheduled to run more often than ten minutes, only the latest zip file within the ten minutes will be uploaded.

[aburns]

I found that running FLEXERA_Normalize_Extractor_5.5.35_3003.exe as described in step 2 did not create the EXTRACTION folder.  It only displayed the BDNA Normalize Extractor dialog.  The FLEXERA_Normalize_XXX.zip file is a 7-Zip self-extracting file, so you can use the 7-Zip File Manager to extract the files and manually put them in the EXTRACTION folder.

1. Open 7-Zip File Manager.
2. Browse to the file FLEXERA_Normalize_Extractor_5.5.35_3003.exe and select it.  (See Figure 1.)
3. Click the Extract button on the toolbar.
4. On the "Extract" dialog, specify the location of the EXTRACTION folder, which should be an empty directory. (See Figure 2.)

Figure 1: Use the 7-Zip File Manager to extract the files from the self-extracting executable.

Figure 2: Specify the "EXTRACTION" directory where you want to save the files.

[MurrayPeters]

Hi - SCCM is missing from the list of extractor configs listed in this article - is that deliberate, i.e. does that mean that the SCCM extractors that always worked with Data Platform will *not* work in with this process?

[JohnSorensenDK]

@MurrayPeters 

I think the list above describes this method as a workaround for not having these data sources directly supported by IT Visibility at the time. Please refer to Inventory Data Sources Supported for IT Visibility for the current list.

That doesn't mean that it wouldn't work, but Flexera would obviously point at the supported approach instead as the preferred and supported method.

[MurrayPeters]

Thanks John,

I get that, but the data imported by sharing of SCCM data via a "supported" ITAM connected SCCM source is not equivalent to what is/was brought in by an SCCM extractor (Data Platform style) connection.

For example:
* Usage data is not imported from the ITAM (supported) method, so if that's required in ITV then you would need the extractor
* If you have pass-thru query requirements, that is only possible when extractor based connections

Whilst I used SCCM as an example the question I'm really asking is whether the extractor method is really limited to what's listed in the documentation, or whether it would work for any properly developed/tested extractor that would have worked with Data Platform on prem.

Of course, there are design considerations such as any extractor based import will only be processed in ITV and not flow into ITAM.    On the other hand - an ITAM connected inventory source such as SCCM will automatically flow into ITV (as of Feb 2023).  However if I build an ITAM inventory import for Solarwinds, the Solarwinds data won't flow into ITV and if I need it in ITV as well I need to build/setup an ITAM based connector as well as an extractor based connector. 

Overall I've found it challenging to properly understand what does/doesn't flow automatically and what needs additional customisation, so we are totally dependent on the list of supported apps in the documentation being kept up to date.

Thanks for the feedback.

regards,
Murray

[JohnSorensenDK]

@MurrayPeters 

I understand. The good news is that it will work, and that Flexera will continue to add more data sources to the supported  list.

Please be aware that passthru isn't supported with ITV though. (It's in the roadmap backlog...)

Thanks,