Advantages with WMI event over powershell script to modify files?
-
Thursday, February 14, 2013 4:11 PM
Hi!
I am in need to create a powershell script that will go through a large amount of data, at least 100GB and look for a certain type of file extension and modify the file attributes. While surfing on google how to proceed i stumbled on the WMIevents.
Let's say i want to monitor all the .txt files that i created on the disk and change the attribute on it.
My question is really if WMIevents are able to handle so much data? Does WMIevent need less performance then a script that uses the "Get-ChildItem -Recurse"-command.
I know this depends on a lot of thing like IO-performance on the server and so on but i want to if it is realistic to create such a wmi-event or should i just create a script that i schedule to run with an interval like for example every 4 hours or so?
Thanks for your help!
All Replies
-
Thursday, February 14, 2013 4:24 PMModerator
What's the purpose? Do you need a file-auditing solution? If so, a script is probably not the best choice.
Bill
-
Thursday, February 14, 2013 4:35 PM
The purpose is that a lot of our PDF files are gets the attribute "Temorary" which is causing a lot of trouble because then DFS won't sync these files as they are treated as Temp-files. We have tried to get DFS to sync the files without success and we have tried to find out why the attribute gets added in the first place and we think it is one of our business that causes this and it is no way we can change it so...
We need to remove the attribute after the files are copied to the file server and i know how to do it with powershell that i was planning to schedule to run every 4 hours or so. Otherwise the files won't get distributed to other file servers.
So when i found out about WMI-events i thought that it would be great if WMI could find out when a PDF-file was created on the server and then if it had the attribute, remove it. But then i realized that it would probably take to much IO for WMI to monitor this. Right? :)
-
Thursday, February 14, 2013 4:45 PMModerator
Not a complete answer, but more explanation:
Bill
-
Thursday, February 14, 2013 4:49 PMModerator
Even better. This post gives us a PowerShell script for removing the FILE_ATTRIBUTE_TEMPORARY attribute:
As the Directory Services Team: DFSR Does not Replicate Temporary Files
(Searching is helpful for finding information)
Bill
-
Thursday, February 14, 2013 5:32 PM
Maybe i was not clear..
To quote the article:
"So you figured out that DFSR is not replicating some files because they have the temporary attribute set. There is no way to change this behavior in DFSR, so the only option is to live with it, or remove the temporary attribute from the files you want to replicate"
I know this, that's why i need to create a powershell script to remove attribute.
A WMI-event could take care of this directly when the file is created on the file server. With a powershell script i need to schedule it to run every hour or so hence i thought "a WMI-Event will take care of the problem directly when the file gets created instead of waiting for the script to run"
So back to my original question: Can a WMI-event handle to monitor so many folders and files or will it take up all the CPU and IO capacity to do so?
Thanks!
-
Thursday, February 14, 2013 6:05 PMModerator
That probably depends on the data set. You could give it a try and see.
Bill
-
Thursday, February 14, 2013 8:47 PM
What do you mean by data set? Not how much data it is on the server but what kind of data? How many files and so on?That probably depends on the data set. You could give it a try and see.
Bill
-
Thursday, February 14, 2013 10:30 PM
What do you mean by data set? Not how much data it is on the server but what kind of data? How many files and so on?That probably depends on the data set. You could give it a try and see.
Bill
I think you want to know what WMI eventing brings to t5eh party. Well it can trigger on file creationg and alter the file. It is light weight.
For a permanent eventing setup I would use a M<OF imlementation that calls a script with teh filename to be altered.
Look in the repository for numerous WMI eventing examples. The Scripting Guy blog has a numnber of entries on creating permanent events. As soon as the bit is reset the file should be replicated. DFSR also uses eventing to replicate files on change/create/delete. I believe it currently monitors at the flag level so a file changong from 'temporary' would cause it to be copied.
Check the repository and check back with question.
¯\_(ツ)_/¯
-
Friday, February 15, 2013 4:41 PM
What do you mean by data set? Not how much data it is on the server but what kind of data? How many files and so on?That probably depends on the data set. You could give it a try and see.
Bill
I think you want to know what WMI eventing brings to t5eh party. Well it can trigger on file creationg and alter the file. It is light weight.
For a permanent eventing setup I would use a M<OF imlementation that calls a script with teh filename to be altered.
Look in the repository for numerous WMI eventing examples. The Scripting Guy blog has a numnber of entries on creating permanent events. As soon as the bit is reset the file should be replicated. DFSR also uses eventing to replicate files on change/create/delete. I believe it currently monitors at the flag level so a file changong from 'temporary' would cause it to be copied.
Check the repository and check back with question.
¯\_(ツ)_/¯
Hi jrv!
So is more light weight than script? It should be right? Instead of going through all data on the drive it only monitor files being written to the disk.
What do you mean by a "a M<OF imlementation" ?
-
Friday, February 15, 2013 6:07 PM
Sorry for the typo. It should be MOF.
A MOF is a text file that is compiled into WMI as a permanent event subscription. It wil always run but can be disabled via WBEM.
See: http://msdn.microsoft.com/en-us/library/aa713453(v=vs.71).aspx
You can also use PowerShell to create a permanent event subscription.
¯\_(ツ)_/¯
-
Friday, February 15, 2013 6:10 PM
PowerShell method:
You want Cim_Datafile 'InstanceCreated' event
¯\_(ツ)_/¯
-
Friday, February 15, 2013 7:58 PM
PowerShell method:
You want Cim_Datafile 'InstanceCreated' event
¯\_(ツ)_/¯
Thanks for the help jrv!
Though it does not look like i can look through multiple folders right? So i would have to create an event for each folder and that i a lot so thats not a good solution :)
Or am i wrong?
-
Friday, February 15, 2013 8:07 PM
YOu didn't ask about that initially. You just said you wanted a more efficient method. YOu can event on as many as you like.
You can also set auditing on a floder hierarchy and set an event on the event log to trap the audit events.
You can also use FileSystemWatcher which is an advanced eventing class.
http://msdn.microsoft.com/en-us/library/system.io.filesystemwatcher.aspx
Of course you can manually scan thousands of files which would not be too bad if you have an index on teh exte4nsion and the creatinon date. Just scan for new files every hour. It should run in a few minutes.
Eventing is much more efficient.
¯\_(ツ)_/¯
- Marked As Answer by Samus-Aran Saturday, February 16, 2013 10:26 AM
-
Saturday, February 16, 2013 8:21 AM
What do you mean by data set? Not how much data it is on the server but what kind of data? How many files and so on?That probably depends on the data set. You could give it a try and see.
Bill
I think you want to know what WMI eventing brings to t5eh party. Well it can trigger on file creationg and alter the file. It is light weight.
For a permanent eventing setup I would use a M<OF imlementation that calls a script with teh filename to be altered.
Look in the repository for numerous WMI eventing examples. The Scripting Guy blog has a numnber of entries on creating permanent events. As soon as the bit is reset the file should be replicated. DFSR also uses eventing to replicate files on change/create/delete. I believe it currently monitors at the flag level so a file changong from 'temporary' would cause it to be copied.
Check the repository and check back with question.
¯\_(ツ)_/¯
Lightweight compared to what?
-
Saturday, February 16, 2013 9:16 AM
Lightweight compared to what?
Compared to scanning repeatedly over 1000s of files.
Windows has an event system that knows about nearly every change to any part of the Windows system. This can be asked to notify us when Windows does almost anything. We do not have to look for anything or run any scripts. Windows will tell us when a file has been created that meets our criteria.
Without events we would have to run a script that scans through 100+ Gb of files repeatedly looking for new files that meet our criteria.
Lightweight in this sense indicates that we do not have to consume CPU looking for new files.
To understand why this may be required look at the criteria for DFSR and read the thread very carefully.
¯\_(ツ)_/¯
- Marked As Answer by Samus-Aran Saturday, February 16, 2013 10:26 AM
- Unmarked As Answer by Samus-Aran Saturday, February 16, 2013 10:26 AM
-
Saturday, February 16, 2013 10:25 AM
Hi jrv! I have been playing around with the FileSystemWatcher and it is so cool and powerfull!!! I have read the get-help register-objectevent -full and done some gooogleling and what i can see i can't register a permanent objectevent that will survice a reboot. So my thought is that i register this ObjectEvent in a permanent WMI-event. What do you say?
If you think it could work then i will study up on WMI-events, i usually understand code syntax pretty fast but when it comes to WMI-events i just see words in a row! :)
I will mark you post as an answer but i will problably come back and ask more :)
- Edited by Samus-Aran Saturday, February 16, 2013 10:30 AM
.png)
