How to Write a Filewatcher in Three Possible Ways

During my excessive web-sessions regarding .NET I often stumble over questions and articles regarding file monitoring applications. Either as stand-alone applications or in conjunction with Windows Services.

For years know this discussions goes on. With this post I want to show the three methods I have used over the last few years and also give a good usage-scenario for “File Watchers” on Windows.

Clarifications

In this post I will use the term “service” always as a synonym for “Windows Service” and not Web-Services or something similar.

Ok. Fasten your seatbelts and enjoy the journey!

System.IO.FileSystemWatcher – The “Standard” Way

This class exists in the holy System.IO-Namespace since .NET 1.1. And was used since then in endless projects to enalbe file watching in applications and services – and what’s not soo good – it was not changed too much, since then by MS.

There is always a customer somwhere who wants to monitor something somebody or some kind of system created. And so the lazy developler searches for the nearest and most convinient solution: Run Google and search for “Watch directory c#” or something similar. This will lead you in 99% of all cases to this magic FileSystemWatcher-Class.

Sample of a System.IO.FileSystemWatcher Implementation

This sample was directly taken from the MSDN-Documentation:

using System;
using System.IO;
using System.Security.Permissions;

public class Watcher
{

    public static void Main()
    {
    Run();

    }

    [PermissionSet(SecurityAction.Demand, Name="FullTrust")]
    public static void Run()
    {
        string[] args = System.Environment.GetCommandLineArgs();

        // If a directory is not specified, exit program.
        if(args.Length != 2)
        {
            // Display the proper way to call the program.
            Console.WriteLine("Usage: Watcher.exe (directory)");
            return;
        }

        // Create a new FileSystemWatcher and set its properties.
        FileSystemWatcher watcher = new FileSystemWatcher();
        watcher.Path = args[1];
        /* Watch for changes in LastAccess and LastWrite times, and
           the renaming of files or directories. */
        watcher.NotifyFilter = NotifyFilters.LastAccess | NotifyFilters.LastWrite
           | NotifyFilters.FileName | NotifyFilters.DirectoryName;
        // Only watch text files.
        watcher.Filter = "*.txt";

        // Add event handlers.
        watcher.Changed += new FileSystemEventHandler(OnChanged);
        watcher.Created += new FileSystemEventHandler(OnChanged);
        watcher.Deleted += new FileSystemEventHandler(OnChanged);
        watcher.Renamed += new RenamedEventHandler(OnRenamed);

        // Begin watching.
        watcher.EnableRaisingEvents = true;

        // Wait for the user to quit the program.
        Console.WriteLine("Press 'q' to quit the sample.");
        while(Console.Read()!='q');
    }

    // Define the event handlers.
    private static void OnChanged(object source, FileSystemEventArgs e)
    {
        // Specify what is done when a file is changed, created, or deleted.
       Console.WriteLine("File: " +  e.FullPath + " " + e.ChangeType);
    }

    private static void OnRenamed(object source, RenamedEventArgs e)
    {
        // Specify what is done when a file is renamed.
        Console.WriteLine("File: {0} renamed to {1}", e.OldFullPath, e.FullPath);
    }
}

Well, this is fascinating in the beginning, but if you need something like an “High-End-Filewatcher” you will be disappointed by this class, which is not able to handle high-traffic file changes. In other words (Taken from the original MSDN-Documentation):

..When you first call ReadDirectoryChangesW, the system allocates a buffer to store change information. This buffer is associated with the directory handle until it is closed and its size does not change during its lifetime. Directory changes that occur between calls to this function are added to the buffer and then returned with the next call. If the buffer overflows, the entire contents of the buffer are discarded and the lpBytesReturned parameter contains zero…

Unfortunately FileSystemWatcher uses the API function ReadDirectoryChangesW, with the known limitations above.

It is quite good for usage with a small amount of files to check. The buffer is configured to take a maximum of 64KB and a minimum of 4KB of filenames to be watched.

On the other side, the usage is really fairly simple. You create a FileSystemWatcher object and set a few properties:

The path you want to watch
The notify filter (determines which “file-changes” to watch, like ChangedDirectories, LastFileAccess and more)
The Filter to determine which files need to be watched (e.g. *.*,*.txt,*.doc,*.pdf)
Wire-Up the events like Changed, Created, Deleted or renamed

The next problem are big files, e.g. files copied over the network. Files are locked during the copy-process, but FileSystemWatcher does not recognize the copy process and informs you through eventing, that a new file was written to the watched directory, on which you like to perform a custom action, like unzipping the file or to copy it to a special location.

This will cause problems, and you have to wait either until the file is copied to put it into a queue for later processing, or setting a relative timeout to access the file later-which again means workarounds and performance losses. This makes the FileSystemWatcher not really a big deal to run inside a service. But like i mentioned before: For a smaller amount of files, and small files it is perfect.

With a small piece of software and a simple formula, I have calculated only the average length of a filename on a windows system (not including the full path, only the filename) and on my development machine the magic number 17 appeared. One of my partitions contains a file count of 68446 files, which should be a quite representative number of files. Imagine now watching all files in all subdirectories, because you need to monitor your drives for changes with FileSystemWatcher – this task is simply not possible.

Fortunately, there are other possibilities and solutions.

Using WMI – Windows Messaging Instrumentarium to Watch Directories

The main description on the MSDN-Site, is that WMI is something only for C++ developers and administrators. You can read it on the Windows Management Instrumentarium page.

But ever since the tool WMI Code Creator (a code generation utility for different development languages to acces WMI resources), this has rapidly changed. More and more C# and VB developers use WMI to implement system based functionalities in their applications. There is also an additional link for .NET devs “WMI in .NET Framework” at the end of the post (Referencing .NET 1.1).

To explain WMI in short: You can access your device informations, your boot configuration, file system informations (is what we do here) and much, much more using WMI Providers (COM Objects) and WMI Classes. Important to know is, that WMI is a COM based technology.

A really good overview regarding WMI can be foundon the WMI Architecture site. This should give you a good overview, regarding WMI. And – the most important thing I can give you as advice is, that you should learn WQL – SQL For WMI.

In practice implementing a file watching component can be better explained in code:

/// &lt;summary&gt;
        /// Creates the new WMI event watcher.
        /// &lt;/summary&gt;
        /// &lt;param name="directoryToWatch"&gt;The directory to watch.&lt;/param&gt;

        private void CreateNewWMIEventWatcher(string directoryToWatch)
        {
            try
            {
                StringBuilder wqlText = new StringBuilder();

                wqlText.Append("SELECT * FROM __InstanceOperationEvent WITHIN 1 WHERE TargetInstance ISA ");
                wqlText.AppendFormat("{0} ", "'CIM_DirectoryContainsFile'");
                wqlText.AppendFormat(" AND TargetInstance.GroupComponent ={0}", "'Win32_Directory.Name="" + directoryToWatch + ""'");

                this._Query = new WqlEventQuery(wqlText.ToString());

                this._Watcher = new ManagementEventWatcher(_Query);

                this._Watcher.EventArrived += new EventArrivedEventHandler(this.watcher_EventArrived);
                this._Watcher.Disposed += new EventHandler(this._Watcher_Disposed);

            }
            catch (ManagementException mgmtException)
            {

                //Log the exception here or do something
                //else to handle the error
                throw;
            }
            catch (Exception ex)
            {
                 //Log the exception here or do something
                //else to handle the error
                throw;
            }
        }

          /// &lt;summary&gt;
        /// Handles the EventArrived event.
        /// &lt;/summary&gt;
        /// &lt;param name="sender"&gt;The source of the event.&lt;/param&gt;
        /// &lt;param name="e"&gt;The &lt;see cref="System.Management.EventArrivedEventArgs"/&gt; instance containing the event data.&lt;/param&gt;
        private void watcher_EventArrived(object sender, EventArrivedEventArgs e)
        {
            ManagementBaseObject obj = e.NewEvent["TargetInstance"] as ManagementBaseObject;

            string fileName = obj["PartComponent"] as string;

            fileName = fileName.Substring(fileName.IndexOf("=") + 1);

            fileName = this.ReverseString(fileName);
            fileName = fileName.Substring(1);
            fileName = fileName.Substring(0, fileName.IndexOf(''));
            fileName = this.ReverseString(fileName);

            string eventOccured = e.NewEvent.ClassPath.ClassName;

            switch (eventOccured)
            {
                case "__InstanceDeletionEvent":
                    //Some file was deleted
                    break;
                case "__InstanceCreationEvent":
                   //New file was added/created
                    break;
            }

        }

        /// &lt;summary&gt;
        /// Handles the Disposed event of the _Watcher control.
        /// &lt;/summary&gt;
        /// &lt;param name="sender"&gt;The source of the event.&lt;/param&gt;
        /// &lt;param name="e"&gt;The &lt;see cref="System.EventArgs"/&gt; instance containing the event data.&lt;/param&gt;
        void _Watcher_Disposed(object sender, EventArgs e)
        {
            throw new NotImplementedException();
        }

The above example is already running in a real world application, and was tested by creating 100.000 files with a size of 20-400 MB, unzipping this files and moving them after a check to a network share.

StringBuilder wqlText = new StringBuilder();

                wqlText.Append("SELECT * FROM __InstanceOperationEvent WITHIN 1 WHERE TargetInstance ISA ");
                wqlText.AppendFormat("{0} ", "'CIM_DirectoryContainsFile'");
                wqlText.AppendFormat(" AND TargetInstance.GroupComponent ={0}", "'Win32_Directory.Name="" + directoryToWatch + ""'");

The WQL-Query used in this example selects all available data through an WMI-InstanceOperationEvent of the TargetInstance, which is a CIM_DirectoryContainsFile-WMI-Class. The WITHIN 1 clause represents the polling interval for the event, which here is every second. The TargetInstance.GroupComponent property receives all instances of the CIM_DirectoryConainsFileClass, which have the Win32Directory-Class reference property Win32Directory.Name set to the current value.

TIP: Escape the path you like to watch, always by using this pattern @”c:\test\folder” (four backslashes work for me)

this._Query = new WqlEventQuery(wqlText.ToString());

                this._Watcher = new ManagementEventWatcher(_Query);

                this._Watcher.EventArrived += new EventArrivedEventHandler(this.watcher_EventArrived);
                this._Watcher.Disposed += new EventHandler(this._Watcher_Disposed);

The next few lines show how to wire-up the events EventArrived (something happend to the folder) and Disposed, when the WMI event watcher was shutdown.

The Third and Last Method to Write a FileWatcher – The Windows 7 API-Codepack

The Windows 7 API-Codepack is a bunch of classes, that use the P/Invoke mechanism to access the native Windows API. One integral part of the codepack is the windows shell functionality.

The “shell” assembly contains a class called ShellObjectWatcher, which can be used to monitor directories for changes. Internally the ShellObjectWatcher uses the SHChangeNotifyRegister function located in Shell32.dll. The SHChangeNotifyRegister function uses a window-handle to register a window to recieve messages from the file system or Shell.

Here is the original code from a WPF-Example from the codepack, that demonstrates the usage quite well:

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Windows;
using System.Windows.Controls;
using System.Windows.Data;
using System.Windows.Documents;
using System.Windows.Input;
using System.Windows.Media;
using System.Windows.Media.Imaging;
using System.Windows.Shapes;
using Microsoft.WindowsAPICodePack.Dialogs;
using Microsoft.WindowsAPICodePack.Shell;

namespace ShellObjectWatcherSampleWPF
{
    /// &lt;summary&gt;
    /// Interaction logic for Window1.xaml
    /// &lt;/summary&gt;
    public partial class Window1 : Window
    {
        public Window1()
        {
            InitializeComponent();
        }
         private ShellObjectWatcher _watcher = null;

        private void btnBrowse_Click(object sender, RoutedEventArgs e)
        {
            CommonOpenFileDialog cfd = new CommonOpenFileDialog();
            cfd.AllowNonFileSystemItems = true;
            cfd.EnsureReadOnly = true;
            cfd.IsFolderPicker = true;

            if (cfd.ShowDialog() == CommonFileDialogResult.Ok)
            {
                StartWatcher(cfd.FileAsShellObject);
            }
        }

        private void btnBrowseFile_Click(object sender, RoutedEventArgs e)
        {
            CommonOpenFileDialog cfd = new CommonOpenFileDialog();
            cfd.AllowNonFileSystemItems = true;
            cfd.EnsureReadOnly = true;

            if (cfd.ShowDialog() == CommonFileDialogResult.Ok)
            {
                StartWatcher(cfd.FileAsShellObject);
            }
        }

        private void StartWatcher(ShellObject shellObject)
        {
            if (_watcher != null) { _watcher.Dispose(); }
            eventStack.Children.Clear();

            txtPath.Text = shellObject.ParsingName;

            _watcher = new ShellObjectWatcher(shellObject, chkRecursive.IsChecked ?? true);
            _watcher.AllEvents += AllEventsHandler;

            _watcher.Start();
        }

        void AllEventsHandler(object sender, ShellObjectNotificationEventArgs e)
        {
            eventStack.Children.Add(
                new Label()
                {
                    Content = FormatEvent(e.ChangeType, e)
                });
        }

        private string FormatEvent(ShellObjectChangeTypes changeType, ShellObjectNotificationEventArgs args)
        {
            ShellObjectChangedEventArgs changeArgs;
            ShellObjectRenamedEventArgs renameArgs;
            SystemImageUpdatedEventArgs imageArgs;

            string msg;
            if ((renameArgs = args as ShellObjectRenamedEventArgs) != null)
            {
                msg = string.Format("{0}: {1} ==&gt; {2}", changeType,
                    renameArgs.Path,
                    System.IO.Path.GetFileName(renameArgs.NewPath));

            }
            else if ((changeArgs = args as ShellObjectChangedEventArgs) != null)
            {
                msg = string.Format("{0}: {1}", changeType, changeArgs.Path);
            }
            else if ((imageArgs = args as SystemImageUpdatedEventArgs) != null)
            {
                msg = string.Format("{0}: ImageUpdated ==&gt; {1}", changeType, imageArgs.ImageIndex);
            }
            else
            {
                msg = args.ChangeType.ToString();
            }

            return msg;
        }

        private void chkRecursive_Checked(object sender, RoutedEventArgs e)
        {
            if (_watcher != null &amp;&amp; _watcher.Running)
            {
                StartWatcher(ShellObject.FromParsingName(txtPath.Text));
            }
        }

        #region IDisposable Members

        public void Dispose()
        {
            if (_watcher != null)
            {
                _watcher.Dispose();
            }
        }

        #endregion

        private void Window_Closing(object sender, System.ComponentModel.CancelEventArgs e)
        {
            if (_watcher != null) { _watcher.Dispose(); }
        }
    }
}

I hope you have now a good overview regarding FileWatching/Monitoring on Windows.

Thank you for reading.