In Order feed from API

[hamish]

I want to create an in-order feed where I can be sure I'm not going to

1. Miss items
2. Have to process the same item multiple times

I notice there's an ID in the advisories returned from /api/advisories on the flexera APi at https://api.app.secunia.com/api/advisories/ but there's very little said about the id apart from it being an internal id.

I'd like to think it's exactly what I need (i.e. Is in sequence/time order). but

1. That's not documented, so maybe it isn't (In sequence)
2. There's no way to filter on the id being > some_value (i.e. Any id > the last one I processed)

While I can filter and order by unix timestamp, leaves me with having to select all items >= the last timestamp processed and would lead to processing the same advisories multiple times (Which I could do, but it's messy and wasteful of CPU cycles) 

TIA.

[bkelly]

While it may be common for the I’d value to only increase, that is not something documented or guaranteed. It is recommended you leverage the modified date as a reliable method.

For example, the API will return the below and always has a value (equal to created until modified).
"modified_date": "2019-11-05T17:58:09Z",

[hamish]

I did look at that. And had a couple of issues with using modified date.

1. It doesn't seem to be documented as an ordering field
2. It doesn't work from the API explorer as an ordering field (Even though it is in the list of selectable sorting methods)
3. There are times where there are a large number of advisories with the exact same modified time.
   This means when restarting a terminated feed, you can have a large number of advisories to process. As this usually includes looking up the actual advisory it can take a while. And with the API throttled, not always a pleasant thing to happen.
   This may be an historical thing. But there's no guarantee it won't become the norm.
   
   

Those issues don't stop me from using the mod date. It's just not as good or efficient as I'd like it to be.

What I'd really like is a proper in-order unambiguous changelog. i.e. to be able to restart without extra processing, and to determine what the state of each advisory was at a given time. 

I've found a couple of other bugs too in the API.  e.g. where if you present wrong or unknown parameters, the API ignores you and just returns by whatever filter or ordering it wants to. Which is annoying.

[RDanailov]

Hi @hamish, 

Can you check again if you see the Modified Date entry in the list of filters in the API Explorer? We can see it.

I would like to suggest to you to use the Released field instead of the modified_date and see if that's not better.  There shouldn't be duplication with this field and the feed should be in-order as you requested to have it. 
Please try this alternative and let me know if you see any problems similar to using the modified_date. 

With regards to the API Explorer, there seems to be a problem with browser caching of cookies that seems to lead to an inability to sort with the available filters after switching the filters a few times. We will report back to what we've found out after we perform a thorough investigation of this particular issue with Engineering. 

In the meanwhile, you can make use of the API using Powershell scripts, as this will not yield any errors. 
https://helpnet.flexerasoftware.com/svm/api/Default.htm#helplibrary/PowerShell_Script_to_Save_All_Advisories_within_a_Date_Range_to_CSV.htm#s

In response to what you referred to as bugs in the API e.g. where if you present wrong or unknown parameters, the API ignores you and just returns by whatever filter or ordering it wants to, please note that this is not necessarily a bug and it boils down to how the API was designed to be used. We will reply back to you about that through the separate support case we have open with you as soon as we have more input on that. 

Do let us know if you any additional questions, we'd love to help. 

[hamish]

Release date isn't appropriate as it doesn't change when an advisory is updated.

[hamish]

A journal or actual changelog access would be nice...

[RDanailov]

Hi @hamish,

It doesn't work from the API explorer as an ordering field (Even though it is in the list of selectable sorting methods)

---

We have opened an internal case IOJ-2081329 to look into it further in more depth. 
Please keep this number in your records in case you need to follow-up on it.

---

I've found a couple of other bugs too in the API.  e.g. where if you present wrong or unknown parameters, the API ignores you and just returns by whatever filter or ordering it wants to. Which is annoying.

---

We have sent our comments via support case 01941987  back to you, but for transparency reasons, I'd mention that the original API design has not intended to be handling wrong or unknown parameters. Ignoring those is a matter of design practice and not a technical malfunction. At this point, we highly recommend double-checking any parameters before running them to avoid any confusion with the data return of the API.

What I'd really like is a proper in-order unambiguous changelog. i.e. to be able to restart without extra processing, and to determine what the state of each advisory was at a given time. 

---

Our Product Manager is monitoring this space and I'm sure that your kind feedback would be considered. But to be on the safe side, our process of logging such requests or feedback is through the below forum thread:
https://community.flexera.com/t5/Software-Vulnerability/We-Still-Want-Your-Ideas-about-Software-Vulnerability-Management/m-p/123145#M195

We'll get back to you about pending items through support case 01941987 when there's more about that.