DavidH asked a question.
How is everybody getting on with the "Complete Ruleset" DUJ?
I posted about this last year ... I'm wondering how everybody else's complete ruleset DUJ is doing? For us, it now takes nearly 5 days to run 😲
So it's really impossible for us to run this every week. The best we can do it run it occasionally and even then it's very painful to have the DUJ running for a full week like this. Is anybody else having similar problems?
I do have a support ticket open about the complete ruleset, but Snow Support are reluctant to investigate while we're still on SIS 6.6.3 (we don't want to upgrade to SIS 6.7 because it looks like it will make our DUJ even slower).
I'm guessing other people are having similar issues with the complete ruleset?
Hi David,
Sadly not here.
Looking over several systems one on 9.14.1 and 6.6.3 average DUJ time is 1hr 30mins, with a full ruleset moving this just over 2hrs with 25k seats on it
Another is the same version and comes in around 5hrs for normal and 7 for full but that has just shy of 60k seats.
Those table / garbage optimisations in 6.7.1/2 may help but as you say it looks like it then impacts other steps in bad ways.
Ouch. 5 days!! For reference we have around 70K seats and the last time we ran a complete ruleset run (before we disabled it) was 40 hours. It definitely sounds like Snow need to review and tune the DUJ queries that are causing this. Given the response from Joe who can run it in 7 hours for 60k seats it does sounds like an environment issue (more memory/faster disks?). Given that we are running on recommended spec hardware and have followed all the best practice guidance and still have such huge DUJ times then, for now, we just aren't running the complete ruleset.
Thanks! We're also on recommended spec hardware. We think it's quite possible that lots of redundant data is building up in our database and slowing things down. But we just moved to SLM 9.16 last week so I'm wondering if that release is also slower. It sounds like there are DUJ performance improvements coming in 9.17, although we're running that in our test environment and have not seen a significant difference.
In the meantime we're going to take a look at this article which might help clean up some old data. Hopefully that will make a difference...
For information for us with 65k devices and same for users it takes 2:50 to 4:00 (version 9.13)
We have multiple platforms and I don't think a single one of them takes less than 1 day to run a complete ruleset. We had a platform up for two weeks and around 6K, done a complete ruleset and it still took longer than an entire day to run, all platforms are actually over specced.
We just don't turn it on at all, it causes too many issues in my experience.
Hi David,
111 hours really impressive 😣
I can say, that I didn't dare to run Complete_Ruleset in production for a long time now.
In our test environment, this DUJ including Complete_Ruleset takes around 22 hours for nearly 50K devices. The daily DUJ is at around 2:15 meanwhile.
I used the Olivers bigwipe cleanup once in our test environment. The following DUJ with Complete_Ruleset then reduced to 11 hours, but 2 weeks later was back on 22 hours.
At the moment I only do play around with that in the test environment, but did not run it in Prod.
Has anybody already compared the installed computer application results after a normal DUJ with the results after a Complete_Ruleset DUJ?
So to say is it worth to run that long DUJ regularly?
As you also mentioned with SLM 9.17.0 some jobs in DUJ should have better performance, but also in my tests I did not really notice shorter DUJ times.
In SLM 9.17.0 a new service is introduced Cleanup Metering. Perhaps that may help?
Thanks. We used Olivers big wipe last year and I think we will try it again soon. Our regular DUJ is currently sitting at around 5 hours for ~35k seats. We have lots of software on a continuous release cycle and that doesn't seem to help because there are constant new versions on an almost daily basis. Snow struggles with that.
We also noticed the new Metering Cleanup service in SLM 9.17.0, and we hope that will help. In our test environment we adjusted the settings to make it delete the metering records faster, but I haven't noticed much improvement in the DUJ times yet...
So having had to perform a complete ruleset run (due to SRS hiccup last week), its currently 40 hours in, and sitting on the step JobInventorySoftwareApplyRules. Reviewing current SQL activity (just via Activity Monitor in SSMS) I can see the following being run....
DELETE FROM tblApplicationRuleWork
WHERE SoftwareCheckSum IN (
SELECT s.SoftwareCheckSum
FROM dbo.tblSoftware s
INNER JOIN dbo.tblApplicationRuleWork rw ON rw.SoftwareChecksum = s.SoftwareChecksum
WHERE ((s.OSType = @ExOSType AND (s.OSType > 0 OR s.IsRegistry = @ExIsRegistry)) AND s.[Name] LIKE @ExName AND (s.[Version] LIKE @ExVersion OR s.[Version] LIKE @ExSMSVersion OR s.[Version] LIKE @ExSMSVersion2 OR ( @ExIncludeVirtualVersions = 1 AND s.[Version] LIKE @ExVTVersion ))
AND ( s.ExecutablePath LIKE @ExExecutablePath )))
...now I don't understand the context of the query in relation to the process as a whole and indeed the query shown may just be a by-product of it waiting for another process to complete, but any queries that include so many LIKEs, ORs and @parameters are typically non-SARGable and will most likely be poor performance wise.
Would be good to get Snow's input into whether they are looking to address the woeful performance of the complete ruleset DUJ runs?
Hi Colin,
After the SRS team changed the rule encryption last week, my DUJ in production worked good and I did not see SRS issues again.
But I did start the Complete ruleset in my Test environment. This finally took 52 hours !!
After that I noticed again a lot of assigned applications, and less unassigned applications. I thought this was perhaps a problem in my test environment.
When your DUJ is ready, then you perhaps should check this again.
I finally cleaned my tblsoftware, did another Complete Ruleset DUJ yesterday, that finished after 12 hours. Now that looks good again.
Yes, it would be really good to have a comment from Snow to understand if the complete ruleset is on their roadmap.
@Colin Hardie​ thanks for tracing that SQL, all those 'likes' look like a performance bottleneck don't they?
We have recently done Olivers big wipe (warning: only attempt that if you understand what you're doing) and DUJ times have come down by at least 50%, so we will try a complete ruleset and hope it doesn't take another 5 days!
David