piideleteJune asked a question.
DUJ and INV Snowpack Processing
Until about Inventory 6.4 the Inventory server would still process files while a DUJ was active potentially causing transaction locks and the DUJ to error.
A change was made to INV to put back the checks that were in INV3.2 / IDP to see if the DUJ was running and suspend processing of files until it completed.
We've upgrade a platform to INV 6.6.3 and this suspension takes place but now for only part of the DUJ and once it gets to the apply software rules steps it kicks back in processing of Snowpack's rather than waits till the job fully completes.
Is this now expected behaviour?
or is this a bug in 6.6.3
Hi Joe
I think that is expected behaviour... I think that once the Inv DB is moved into a staging area and completes the import it does continue processing
Incremental data transfer (snowsoftware.com)
Adam
Hi Joe,
I agree with Adam - this is expected behaviour.
Tom
Hi @Tom Lineen / @Adam Davies
Thanks for the link the the incremental DUJ step details, good to see it's workings but this doesn't mention any changes to Inventory Server and neither does the release notes.
When i worked with the Dev's on a problem ticket we logged with Snow to get it re-introduced the Inventory Server processing checks in 6.2/3 (still can't remember) it suspended the processing until it sees the Job Successful message appear in the tbl.JogLog table.
Hence that step requirement being needed when you reset the DUJ via SQL if it hangs or INV never starts again.
So if this is now not the case and it's checking for the staging tables to be loaded, it must be looking for another status or maybe the JobParallelStep table to see when the staging table steps have moved to completed.
Either way i've tested this on an old 6.x system and can see it totally stops processing until complete but in the later version doesn't - welcome change just one that's not documented... unless it is a bug and ur DUJ's aren't bringing over everything ...
Hi Joe,
Just wanted to clarify - I think its been working like this for a while now perhaps since 6.6.0.
I've got systems running SI 6.6.2 which do not seem to stop processing while the DUJ is running so its not new to 6.6.3
I can't point you to a specific entry in the release notes apart from the obscure mention in the release notes for SI 6.6.0. "Database stored procedures used in the Data Update Job now take into consideration the start time of the job itself."
T
Interesting comment from those release notes i'll try find out more on that.
For the original question and yours's and mine observed behaviour i can only think of these theory's.
a) They have removed the need for the INV checks on DUJ running as they have solved the issues of record locking in SQL when the SP's accessed tables being used.
b) The incremental staging has lessened the need and the processing starts quicker and doesn't rely on the finish message but another to start up again.
c) They broke it and that'll explain it starting midway through a DUJ and also the increase in SQL lock errors i've seen on a few customer platforms crashing DUJ's when they have been stable since 6.2/3
I've got it logged with Snow anyways so i'll see what the official word is if/when support pickup the ticket.
Hi Joe - that last point is really interesting... I'm seeing lots of deadlocks since 9.13 for some reason. They seemed to think it was SIM 5.30 related, but that isn't the case..... will let you know what I find too
Easy test - before your DUJ kicks off stop the Inventory Server service. Let the DUJ run and see if you get the deadlocks. I'm going to say you wont see any.
If that's the case then they have broke the processing checks in INV 6.6 or so.
Hi Joe
Not sure if you've seen this but in the Best Practices for Large Environments documentation - section 7.9 it says
For very large environments, it is recommended to stop the Master Inventory
Server before running the DUJ to stop it from processing files and running the
Garbage Collector during it.
Adam
Hi Adam,
I've seen that before and that section has been in that document since INV5/SLM8 came out.
When we started seeing deadlocks across most SPE platforms of notable size Snow's Dev's basically gave us powershell/batch files to suspend the processing but still recieve files and then start it back up.
The downside was this was on a timed basis and didn't take in to account the actual DUJ possibly stopping hours before the scheduled times.
The Dev's then implemented the same logic the IDP had to check for the DUJ running - guess that's now gone (again)
This is taken from INV 6.4 Release Notes:
When used together with Snow License Manager version 9.6.1 or later, Snow Inventory Server will check if the Data Update Job is active and, in that case, will stop tasks like processing files and performing maintenance on the Snow Inventory database to avoid concurrency problems in SQL Server (deadlocks); normal operations will resume after the Data Update Job completes; note: this correction does not apply when Snow Inventory Serveris used together with Snow License Manager 8 (PRB0042992).