Feb 02, 2022
10:32 AM
1 Kudo
I use this API endpoint (https://docs.revenera.com/ui560/report/Content/helplibrary/Request_Response_Parameters_Summary_17.htm#raw-data-export_745315351_1115800) to download raw data files. It takes a long time to get the full list of URLs for all of our products, because it's going back and retrieving 90 days' worth. I'd like it if you added another parameter(s) for start/end date for files, so I could just retrieve from a smaller date range. We are collecting the files every day and generally I just want to get the URLs for the previous day files. The process to get these files and download them takes about 26 minutes, most of which is the URL's. I believe 75% + of that time could be eliminated, assuming that if I asked for 1/90th of the file URL's, it would eliminate 90% of the time for those api calls.
... View more
Feb 02, 2022
10:26 AM
2 Kudos
Hannes: I have an R package I created that can automate the download of the daily files. We were storing those as zip and using them ad hoc, until very recently we implemented a process in Azure Data Factory that unzips them and stores it all as parquet files that we can use for analysis and reporting. https://cran.r-project.org/web/packages/reveneraR/index.html
... View more
Latest posts by umphlettTSC
Subject | Views | Posted |
---|---|---|
360 | Feb 02, 2022 10:32 AM | |
505 | Feb 02, 2022 10:26 AM |
Activity Feed
- Got a Kudo for reporting API change request. Aug 10, 2022 01:41 AM
- Got a Kudo for Re: Downloading data - Custom events and Raw data. Feb 02, 2022 12:00 PM
- Got a Kudo for Re: Downloading data - Custom events and Raw data. Feb 02, 2022 10:49 AM
- Posted reporting API change request on Usage Intelligence Forum. Feb 02, 2022 10:32 AM
- Posted Re: Downloading data - Custom events and Raw data on Usage Intelligence Forum. Feb 02, 2022 10:26 AM