VitoPlantamura's Website

Click here to go to the archives page...

"extremely impressive work"
- Mark Russinovich, www.sysinternals.com
(referring to my BugChecker)

Homepage of Vito Plantamura (@, LinkedIn), Windows Researcher and Developer.

[user=Guest] - updated: August 08, 2007

IMAGE DOWNLOADER 1.0

Image Downloader was shut down on December 2008, after 5 years of operation and hundreds of copies sold.

I wrote Image Downloader 1.0 in 2002 for personal purposes: I needed a program for helping me download a large number of JPEG images from several internet sites without too much hassle. The features of Image Downloader and the technology behind it are definitely unique and are the result of several weeks of dedicated research. Image Downloader has won several awards from specialized download sites around the web.

Image Downloader extends Internet Explorer for allowing to select images and thumbnails from an HTML page as one is used doing with files in a Windows Explorer window, for later or immediate download. In the screenshot above, you can see a selection operation as it happens: the user is navigating the web, finds one or more interesting images and then clicks the middle-mouse button selecting the images or thumbnails of interest: a yellow marker appears on the upper-left corner of each selected image (if the selection extends over the borders of the current Internet Explorer window, the HTML frame scrolls automatically in the direction of the cursor). The unique Image Downloader selector is able to deal with html anchors, maps and frames, JavaScript links and "Standards-Compliant" html pages. After the selection operation in Internet Explorer, the images can be actually downloaded pressing a button in the Image Downloader toolbar or in the Internet Explorer context menu:

FEATURES

	Image Downloader is the only program that allows you to select one or more thumbnails, links or pictures from an Internet Explorer page and then download the selected images later. Pressing tipically the middle-mouse-button, when normally navigating in Internet Explorer, you can perform a selection of thumbnails or images and, according to the action chosen, the program will store the links to the selected items on disk or will save and/or download the images immediately.

	Image Downloader allows to filter the types of links that can be selected from the currently viewed page.

	Image Downloader is able to deal with Html Anchors, Maps and Frames, JavaScript links and "Standards-Compliant" Html Pages.

	The "Hot Rectangle" feature of the program allows you to automate the download of images from a predefined portion of a set of Html Pages, in an automatic manner.

	Image Downloader creates and operates only on standard Internet Explorer Shortcut files (*.URL), for maximum integration and compatibility.

INTRODUCTION

Image Downloader is implemented as an Internet Explorer Extension (i.e. "Browser Helper Object") written in C++ with ATL/WTL and MFC. I have implemented several unique solutions and features in this program that you will not find in any other similar product: I will discuss some of those features later. The Image Downloader dedicated site is at this address: from it, you can take the Guided Tour or buy the full-featured program in a completely automated purchase procedure written with ASP.NET in C#.

SCREENSHOTS AND CONSIDERATIONS

For Image Downloader, I have implemented a completely dedicated server-side system for issuing evaluation and purchase keys. Immediately after download and installation, the program shows the dialog above in order to register the evaluation version of the program. After having typed in the personal informations of the user (name, company and email) the program requests to the Image Downloader server to issue an evaluation key for that user. At the client side, the query takes the form of a "URLDownloadToCacheFile" (proxy/firewall friendly) request to an ASP.NET page. The server-side implementation stores the user details in a SQL Server table and then calculates an highly encrypted 26-digits alphanumeric key that is emailed to the user. Then the user, upon receiving the mail, can enter in the same window above (previously used for the registration) the activation code so generated. Then the program connects again to the server and asks whether the activation code typed in by the user was actually generated by it (all the issued keys are stored in a SQL Server table) and so whether the key has to be considered valid. If everything goes ok, the program is unblocked and the evaluation period begins.

When the program is bought, the purchase system of the used e-commerce server talks through a private and authenticated interface to my ASP.NET server implementation communicating the details of the purchase, including the email address of the final user. At this point, the server-side implementation computes a new key (that has the same format of the evaluation key previously described) that is emailed to the final user in order to remove the evaluation period constraint and to enable the crippled features.

The "Hot Rectangle" feature allows to define portions of one or more HTML pages that contain images of interest. When downloading, the program will consider and download only those images that are comprised in the specified rectangle.

As you can see in the screenshot, the Hot Rectangle area can be defined in the context of a resolution-reduced version on an Internet Explorer page. This nice effect can be achieved through a method I have researched and implemented just for Image Downloader. Specifically, what you need to do is to make a WebBrowser control draw into a dedicated device context and then to "StretchBlt" from that device context to the target DC. This can be achieved simply calling the IViewObject::Draw method on the control itself (incidentally, you may want to create the WebBrowser control itself omitting the WS_VISIBLE style, thus causing the control to be hidden). This is the code snippet from the Image Downloader project that actually handles this task:

BOOLEAN DrawWbControlStretched( BOOLEAN bQualityBlt )
{
      // get the pointers to the interfaces.
 
      CComPtr<IWebBrowser2>         piWebBrowser = m_wbBrowser.GetWebBrowser2InterfacePtr();
      if ( piWebBrowser == NULL )
            return FALSE;
 
      CComQIPtr< IViewObject, &IID_IViewObject >            piViewObject = piWebBrowser;
      if ( piViewObject == NULL )
            return FALSE;
 
      CComQIPtr< IOleObject, &IID_IOleObject >       piOleObject = piWebBrowser;
      if ( piOleObject == NULL )
            return FALSE;
 
      // get the control dimensions.
 
      HRESULT           hr;
 
      SIZEL       slSizeHM, slSizePX;
      hr = piOleObject->GetExtent( DVASPECT_CONTENT, & slSizeHM );
      if ( FAILED( hr ) )
            return FALSE;
 
      ::ConvertHiMetricsToPixels( & slSizeHM, & slSizePX, m_dcWebBrowserDrawDc.GetSafeHdc() );
 
      // draw in the device context.
 
      RECTL       rctWebBrowserClientArea;
      rctWebBrowserClientArea.left = 0;
      rctWebBrowserClientArea.top = 0;
      rctWebBrowserClientArea.right = slSizePX.cx;
      rctWebBrowserClientArea.bottom = slSizePX.cy;
 
      hr = piViewObject->Draw(
                  DVASPECT_CONTENT,
                  -1,
                  NULL,
                  NULL,
                  NULL,
                  m_dcWebBrowserDrawDc.GetSafeHdc(),
                  & rctWebBrowserClientArea,
                  NULL,
                  NULL,
                  0
            );
 
      if ( FAILED( hr ) )
            return FALSE;
 
      // draw the stretched version.
 
      if ( bQualityBlt ) // ...slower...
      {
            m_dcCurrentSnapshot.SetStretchBltMode( HALFTONE );
            ::SetBrushOrgEx( m_dcCurrentSnapshot.GetSafeHdc(), 0, 0, NULL );
      }
      else
      {
            m_dcCurrentSnapshot.SetStretchBltMode( COLORONCOLOR );
      }
 
      m_dcCurrentSnapshot.StretchBlt(
            0, 0,
            m_sCurrentSnapshotExtent.cx, m_sCurrentSnapshotExtent.cy,
            & m_dcWebBrowserDrawDc,
            0, 0,
            slSizePX.cx, slSizePX.cy,
            SRCCOPY );
 
      // return.
 
      return TRUE;
}

The Settings Window allows to control various aspects of the functioning of the program. I have implemented a set of proprietary sub-controls that are embedded inside the main tree view control and that act just as children items, with a perfect support for scrolling and repainting. They are displayed when a particular tree view item is selected, thus allowing a convenient in-place modification of the parameter.

This is the enumeration (from the application sources) that describes the various types of sub-controls that are available:

enum EItemRenderingType
{
      EIRT_DropDownList,
      EIRT_DoubleDropDownList,
      EIRT_DropDownPlusDoubleEditCtrlPlusOkAndDeleteBtns,
      EIRT_EditCtrl,
      EIRT_FileEditCtrl,
      EIRT_DirectoryEditCtrl,
      EIRT_CheckListBox
};

A minimal set of base controls is created at the dialog initialization. Then, when a compound control (as in the enumeration above) is required for visualization (tipically when an item is selected), the necessary base controls are retrieved from that controls pool, are packaged in an STL vector and are passed to the various routines that initialize, manage and place them.

The source codes of the Setting Windows controls are available upon request.

ADVANCED FEATURES

During my initial research for Image Downloader, I studied various advanced features (later implemented in the program) for circumventing one or more limitations of Internet Explorer and Windows Explorer. This is a partial list of them.

N.B. Most of these features are very much Explorer and Windows version specific. They may not work (or more likely have to be updated/modified) for future versions of the browser and of the platform.

COMPLETE LINES IN WINDOWS EXPLORER

Image Downloader is implemented as a Browser Helper Object. For maximum integration with Windows Explorer, the settings that can be modified on link files are saved inside the *.URL files themselves and are presented to the user directly in the Windows Explorer panes (when the details view is selected) using a particular Shell Extension class named "Column Handler Shell Extension". Well, the problem is pretty formal in this case: enabling the "full row select" feature on the details view pane of Windows Explorer may increase a lot the general readability when browsing the folders owned by the program.

This can be implemented in the connection-point-attached IDispatch::Invoke code of the IE DeskBand that is usually used for handling navigation and WebBrowser-alike events in the DeskBand implementation.

This is the actual code that implements this behaviour:

if ( m_bFullRowSelectApplied == FALSE )
{
      HWND        hwMainWnd = NULL;
      HRESULT           hr = m_pWebBrowser->get_HWND( (long*) & hwMainWnd );
 
      if ( FAILED( hr ) == FALSE && hwMainWnd != NULL )
      {
            HWND        hwndListView = GetSysListView32Hwnd( hwMainWnd );
            if ( hwndListView )
            {
                  ::SendMessage( hwndListView, LVM_SETEXTENDEDLISTVIEWSTYLE, LVS_EX_FULLROWSELECT, LVS_EX_FULLROWSELECT );
                  m_bFullRowSelectApplied = TRUE;
            }
      }
}

The m_bFullRowSelectApplied flag can be reset in the same IDispatch::Invoke implementation when handling, for example, the DISPID_NAVIGATECOMPLETE2 event (when browsing to a new folder in Windows Explorer, that event is triggered specifying a "file://" URL to the local or network folder in question). This flag is reset (thus enabling this feature) only for the folders owned by the program.

The GetSysListView32Hwnd function that I have implemented is useful for tracking down the handle to the underlying list view control used for showing the list of files in a folder in Windows Explorer and its behaviour is dependent upon the current version of Microsoft Windows and Internet Explorer:

HWND GetSysListView32Hwnd( HWND hwMainWnd )
{
      // enumerate through the various children of the frame
 
      HWND        hwndDefView = ::WndEnumFnShell( hwMainWnd, "SHELLDLL_DefView" );
      if ( hwndDefView == NULL )
            return NULL;
 
      HWND        hwndListView = NULL;
      HWND        hwndDUIView = NULL;
      HWND        hwndIEServer = ::WndEnumFnShell( hwndDefView, "Internet Explorer_Server" );
      if ( hwndIEServer == NULL )
      {
            hwndDUIView = ::WndEnumFnShell( hwndDefView, "DUIViewWndClassName" );
            if ( hwndDUIView == NULL )
            {
                  hwndListView = ::WndEnumFnShell( hwndDefView, "SysListView32" ); // Windows 98 / Windows NT 4
                  if ( hwndListView == NULL )
                        return NULL;
            }
      }
 
      if ( hwndListView == NULL )
      {
            if ( hwndIEServer ) // Windows 2000
            {
                  HWND        hwndAtlSE = ::WndEnumFnShell( hwndIEServer, "ATL Shell Embedding" );
                  if ( hwndAtlSE == NULL )
                        return NULL;
 
                  hwndListView = ::WndEnumFnShell( hwndAtlSE, "SysListView32" );
                  if ( hwndListView == NULL )
                        return NULL;
            }
            else // Windows XP
            {
                  HWND        hwndDirectUI = ::WndEnumFnShell( hwndDUIView, "DirectUIHWND" );
                  if ( hwndDirectUI == NULL )
                        return NULL;
 
                  HWND        hwndNotSink = ::WndEnumFnShell( hwndDirectUI, "CtrlNotifySink" );
                  if ( hwndNotSink == NULL )
                        return NULL;
 
                  hwndListView = ::WndEnumFnShell( hwndNotSink, "SysListView32" );
                  if ( hwndListView == NULL )
                        return NULL;
            }
      }
 
      // return to the caller
 
      return hwndListView;
}

The WndEnumFnShell function is a simple routine that enumerates through the children of a window doing a class name comparison:

typedef struct SWndEnumDetails
{
      charstring        csClassName;
      HWND              hwndRes;
 
} SWndEnumDetails;
 
static BOOL CALLBACK WndEnumFn( HWND hwnd, LPARAM lParam )
{
      // get the class name of the child and eventually stop enumerating
 
      SWndEnumDetails*        pswedEnumDetails = (SWndEnumDetails*) lParam;
 
      char                    szClassName[ MAX_PATH ];
      if ( ::GetClassName( hwnd, szClassName, sizeof( szClassName ) ) != 0 )
      {
            if ( ::stricmp( szClassName, pswedEnumDetails->csClassName.c_str() ) == 0 )
            {
                  // window found... return to the caller
 
                  pswedEnumDetails->hwndRes = hwnd;
                  return FALSE;
            }
      }
 
      // return to the caller
 
      return TRUE;
}
 
static HWND WndEnumFnShell( HWND hwndParent, char* pszClassName )
{
      // call the api for enumerating the window children
 
      SWndEnumDetails         swedEnumDetails;
      swedEnumDetails.csClassName = pszClassName;
      swedEnumDetails.hwndRes = NULL;
 
      ::EnumChildWindows( hwndParent, & WndEnumFn, (LPARAM) & swedEnumDetails );
      if ( swedEnumDetails.hwndRes == NULL )
            return NULL;
 
      return swedEnumDetails.hwndRes;
}

AUTOMATIC DETAILS VIEW

N.B. this is a pretty extreme hack and should be used only with versions of Windows prior to Windows XP. With newer versions of the OS, you should call the IFolderView::SetCurrentViewMode method to achieve the same results...

Because of the reasons I pointed out in the previous section, it is desirable to have Windows Explorer in the "Details View" mode when browsing the folders owned by the program.

This can be achieved sending the appropriate WM_COMMAND message to the Windows Explorer window, fooling the program into thinking that the command originated from a user interaction. Despite the fact that resource IDs are not much likely to change in an application from version to version, it is recommended to check out at runtime the version of the program before sending a command in this way and to test extensively the resulting implementation (also with differently localized versions of the software...).

It is advisable to send the command in the same IDispatch::Invoke handler of the DISPID_NAVIGATECOMPLETE2 event described earlier. This is the source code:

HWND        hwMainWnd = NULL;
HRESULT           hr = m_pWebBrowser->get_HWND( (long*) & hwMainWnd );
if ( FAILED( hr ) == FALSE && hwMainWnd != NULL )
{
      // send the command
      ::SendMessage( hwMainWnd, WM_COMMAND, MAKEWPARAM( 28716, 0 ), NULL );
}

OPENING MAXIMIZED WINDOWS

This can be achieved calling the IHTMLWindow2::open method and then querying for the HWND of the new window as shown in this snippet:

BOOL OpenNewWindow( const char* pszUrlLink, BOOLEAN bMaximizeNew )
{
      HRESULT                 hr;
 
      // open the new window
 
      if ( m_pWebBrowser )
      {
            // get a pointer to the html document...
 
            CComPtr<IDispatch>            pDisp;
            hr = m_pWebBrowser->get_Document( &pDisp );
            if ( FAILED( hr ) )
                  return FALSE;
 
            CComQIPtr<IHTMLDocument2, &IID_IHTMLDocument2>          spHTML = pDisp;
 
            if ( spHTML )
            {
                  // get the parent window of the document
 
                  CComPtr<IHTMLWindow2>         pHTMLWindow2;
                  hr = spHTML->get_parentWindow( & pHTMLWindow2 );
                  if ( FAILED( hr ) )
                        return FALSE;
 
                  // open the new window
 
                  CComPtr<IHTMLWindow2>         pNewHTMLWindow2;
                  hr = pHTMLWindow2->open( _bstr_t( pszUrlLink ), _bstr_t( "_blank" ), _bstr_t( "" ), FALSE, & pNewHTMLWindow2 );
                  if ( FAILED( hr ) || pNewHTMLWindow2 == NULL )
                        return FALSE;
 
                  // check if we have to maximize this window...
 
                  if ( bMaximizeNew )
                  {
                        CComPtr<IHTMLWindow2>         pTopWnd;
                        hr = pNewHTMLWindow2->get_top( & pTopWnd );
                        if ( FAILED( hr ) || pTopWnd == NULL )
                              return FALSE;
 
                        CComQIPtr<IServiceProvider, &IID_IServiceProvider>        pServiceProvider = pTopWnd;
                        if ( pServiceProvider == NULL )
                              return FALSE;
 
                        CComPtr<IWebBrowser2>         pNewWebBrowser2;
                        hr = pServiceProvider->QueryService( IID_IWebBrowserApp, IID_IWebBrowser2, (void**) & pNewWebBrowser2 );
                        if ( FAILED( hr ) || pNewWebBrowser2 == NULL )
                              return FALSE;
 
                        HWND        hwNewMainWnd = NULL;
                        hr = pNewWebBrowser2->get_HWND( (long*) & hwNewMainWnd );
                        if ( FAILED( hr ) || hwNewMainWnd == NULL )
                              return FALSE;
 
                        ::ShowWindow( hwNewMainWnd, SW_MAXIMIZE );
                  }
 
                  // return to the caller
 
                  return TRUE;
            }
      }
 
      // return to the caller
 
      return FALSE;
}

GETTING THE SELECTION IN WINDOWS EXPLORER

In Image Downloader I need to get the selection in Windows Explorer in order to update the DeskBand buttons when the selection changes (for example when the user has just selected an .URL file, the related buttons have to be enabled in the DeskBand) and obviously when processing a file command that targets the Windows Explorer selection (such as the command for setting the download target path on .URL files). This somewhat simple task can be very tricky, specially if you need to be notified when the selection changes in the case you are implementing a simple user interface update mechanism like the one I described earlier.

Well, for resolving the problem of getting the current selection, you could exploit a somewhat documented Explorer private message (WM_GETISHELLBROWSER) that needs to be sent to the Explorer main window in order to get a pointer to the IShellBrowser interface associated to the current instance of the browser. Then, from that interface pointer, it is possible to get a pointer to the IShellView interface and finally, from this pointer, to get a IDataObject interface pointer representing the current selection. For speeding up the entire procedure, before querying for the IDataObject pointer and before iterating through the selected files, my implementation checks first whether the number of currently selected files in the underlying List View control has changed. Only if the number is actually different, the whole WM_GETISHELLBROWSER procedure is initiated:

BOOL GetSelectedFilesInListView( charstring_vector& vcsSelection )
{
      BOOL        bRetVal = FALSE;
 
      MemoryPointersManager      mpmMPManager;
 
      // get the handle to the explorer main window
 
      if ( m_pWebBrowser == NULL )
            return bRetVal;
 
      HRESULT           hr;
 
      HWND        hwMainWnd = NULL;
 
      hr = m_pWebBrowser->get_HWND( (long*) & hwMainWnd );
      if ( FAILED( hr ) )
            return bRetVal;
 
      if ( hwMainWnd == NULL )
            return bRetVal;
 
      // get an handle to the list view
 
      HWND        hwndListView = GetSysListView32Hwnd( hwMainWnd );
      if ( hwndListView == NULL )
            return bRetVal;
 
      // check if we have to update the collection of files
 
      int               nSelectedItems = ::SendMessage( hwndListView, LVM_GETSELECTEDCOUNT, 0, 0 );
      if ( nSelectedItems != 1 && vcsSelection.size() == nSelectedItems )
            return bRetVal;
 
      // clear the collection of files
 
      vcsSelection.clear();
 
      bRetVal = TRUE;
 
      // get the selection of the list view as a dataobject interface
 
      IShellBrowser*          pShBrowser =
            (IShellBrowser*) ::SendMessage( hwMainWnd, WM_GETISHELLBROWSER, 0, 0 ); // no reference counting on this...
 
      if ( pShBrowser == NULL )
            return bRetVal;
 
      CComPtr<IShellView>                 pShView;
 
      hr = pShBrowser->QueryActiveShellView( & pShView );
      if ( FAILED( hr ) )
            return bRetVal;
 
      if ( pShView == NULL )
            return bRetVal;
 
      CComPtr<IDataObject>          pDtObject;
 
      hr = pShView->GetItemObject( SVGIO_SELECTION, IID_IDataObject, (void**) & pDtObject );
      if ( FAILED( hr ) )
            return bRetVal;
 
      if ( pDtObject == NULL )
            return bRetVal;
 
      // get the data in the data object...
 
      FORMATETC fmt = { CF_HDROP, NULL, DVASPECT_CONTENT, -1, TYMED_HGLOBAL };
      STGMEDIUM stgm;
 
      hr = pDtObject->GetData( & fmt, & stgm );
      if ( FAILED( hr ) )
            return bRetVal;
 
      // iterate through the selection and update the string vector
 
      mpmMPManager.Add( & Automatic_ReleaseStgMedium, & stgm );
 
      UINT      nFiles = ::DragQueryFile( (HDROP)stgm.hGlobal, -1, NULL, 0 );
 
      TCHAR      szPath[ MAX_PATH ];
      for ( int i = 0; i < nFiles; i++ )
      {
            ::DragQueryFile( (HDROP)stgm.hGlobal, i, szPath, MAX_PATH );
            vcsSelection.push_back( szPath );
      }
 
      // return to the caller
 
      return bRetVal;
}

The notification problem of being informed when the selection has changed in order to update (for example) the user interface is even trickier: it required an hack to be invented in order to be properly handled. Specifically, when researching the solution of this problem, I noticed that, after the selection was changed, the browser was sending one or more non-system (WM_USER) custom messages to the status bar (supposedly for triggering the update of its text). Intercepting through a WH_CALLWNDPROC hook this kind of messages (aimed to the status bar window) is the key for resolving the selection change event problem.

For handling the message hook filter in the most optimized way possible, I wrote the CGenericThunk class in order to pass in the EDX register an additional parameter to the CallWndProc static function:

#pragma pack(push,1)
      typedef struct SGenericThunk
      {
            BYTE      m_mov;          // mov edx, pThis
            DWORD   m_this;         //
            BYTE    m_jmp;          // jmp static func
            DWORD   m_relproc;      // relative jmp
      } SGenericThunk;
#pragma pack(pop)
 
class CGenericThunk
{
public:
      SGenericThunk           thunk;
 
      void Init( void* pvProc, void* pThis )
      {
            thunk.m_mov = 0xBA;
            thunk.m_this = (DWORD)pThis;
            thunk.m_jmp = 0xe9;
            thunk.m_relproc = (int)pvProc - ((int)this+sizeof(SGenericThunk));
 
            ::FlushInstructionCache( ::GetCurrentProcess(), &thunk, sizeof(thunk) );
      }
};

The hook is installed calling the well-known SetWindowsHookEx API (notice that the hook procedure pointer is specified as being the address of the thunk, which in turn points to the real implementation):

// hook the messages
 
cgtMessagesHookThunk.Init( & MessagesProc, this );
 
hhMessagesHook = ::SetWindowsHookEx( WH_CALLWNDPROC,
      (HOOKPROC) & cgtMessagesHookThunk.thunk,
      NULL,
      ::GetCurrentThreadId() );

The actual implementation of the CallWndProc function is as follows (consider that the UpdateDeskBandUi function below calls in turn the GetSelectedFilesInListView function described earlier in order to get the changed file selection):

LRESULT CALLBACK MessagesProc( int nCode, WPARAM wParam, LPARAM lParam )
{
      CAtlDeskBand*           pThis;
      __asm mov pThis, edx
 
      CWPSTRUCT*                          pcwpsThis = (CWPSTRUCT*) lParam;
 
      // Typical start-off for any hook
 
      if( nCode < 0 )
            return ::CallNextHookEx( pThis->hhMessagesHook, nCode, wParam, lParam );
 
      // Intercept messages sent to the status bar
 
      if ( pThis->m_hwndStatusBar && pcwpsThis->hwnd == pThis->m_hwndStatusBar )
      {
            if ( pcwpsThis->message >= WM_USER && pcwpsThis->message <= 0x7FFF )
            {
                  pThis->UpdateDeskBandUi( DISPID_STATUSTEXTCHANGE );
            }
      }
 
      // Typical end for any hook procedure
 
      return ::CallNextHookEx( pThis->hhMessagesHook, nCode, wParam, lParam );
}

WINDOWS EXPLORER FOLDER VIEW STATE MANIPULATION: INTRODUCTION

N.B. The hook approaches presented here are further examples of "extreme hacks" and should be considered only for learning and research purposes (unless you know exactly what you are doing)...

Circumventing the limitations of an hosting application may take extreme forms when you are absolutely determined to achieve a specific desired effect. In this case, as said above, for showing several properties and settings of .URL files in Windows Explorer, I am using a "column handler" shell extension in order to display that info in dedicated "details view" columns. The problem is that, in the ideal case, these column handlers have to be enabled automatically (without the need to manually add them in Explorer) and only for the folders owned by the program.

As stated in the MSDN, there is no way to achieve this effect programmatically. So I researched and invented two separate mechanisms for achieving this effect -one for Windows XP and above platforms and one for non-Windows XP systems- because the way Explorer saves the state of folder views differs between Windows XP and non-Windows XP operating systems.

The folder view state comprises informations for restoring the state of a folder view (i.e. the type of columns to display in "details view" mode, the selected sorting order -by name, by type etc.-) when a particular folder is reopened after having been browsed, customized and then closed at an earlier time. Specifically in the case of non-Windows XP Explorers, that view state information is saved in the registry in the following location:

HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\Explorer\Streams

and in the following location Explorer saves the association between a particular folder and its Stream subkey:

HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\Explorer\StreamMRU

as can be noticed in this screenshot:

The stream information that is saved in the "Streams" subkeys is written by the folder view using an IStream interface pointer tipically when Explorer calls the IShellView::SaveViewState method on the view object. When called, the shell view implementation of that method tipically calls the IShellBrowser::GetViewStateStream function in order to obtain an IStream interface pointer for writing its own view state data, that Explorer will then later store in the "ViewView2" binary values in the "Streams" subkeys.

In the Windows XP case, the whole system has changed, supposedly for the following reasons:

	The new system dismisses the old linear approach for storing the mappings between a particular folder and its stream slot in favor of a quicker hierarchical one.

	Supposedly, for allowing to store heterogeneous types of informations from different software components in a more extensible and open way, the plain "stream" approach has been modified for allowing to store in the registry also other types of settings in the form of traditional typed registry values, as shown in this screenshot:

The Windows XP folder view information is stored in a series of so called "property bags". The actual bag data is saved in the following registry key:

HKEY_CURRENT_USER\Software\Microsoft\Windows\ShellNoRoam\Bags

while in the following registry key, Explorer stores the mapping between a particular folder path and its corresponding bag slot:

HKEY_CURRENT_USER\Software\Microsoft\Windows\ShellNoRoam\BagMRU

The folder path information is stored in this registry key in a "hierarchical" way: i.e. Explorer creates a series of sub-keys for representing each part of a folder path clearly for speeding up look-up searches.

FOLDER VIEW STATE MANIPULATION: WINDOWS XP

I am including here these three functions for interacting with the Windows XP bag nodes for reference purposes:

winxp_bagnode_fns.cpp

This is a brief explanation of each function:

	GetBagNodeSlot is a function that, given a full path, returns the corresponding bag node slot index.

	StoreBagNodeSettings is a function that, given a bag node slot index, saves the information of several relevant bag properties in the registry (the data is taken from the following member variables, whose definition is omitted: m_pbColInfo, m_ulColInfoSize, m_dwSort, m_dwSortDir and m_dwCol.).

	RetrieveBagNodeSettings is the opposite function of StoreBagNodeSettings: it takes the specified property bag data from the registry and stores it in the aforementioned variables.

As Windows reads the bag node information from the registry just before a particular folder is browsed, the basic idea of this workaround is to change the registry data just before Explorer reads it. The new "ColInfo" data (that is the equivalent of the "Streams" data in non-Windows XP systems) is composed using an hard-coded byte template in which the application replaces only the values that are installation-specific (specifically these values are only the ordinals of the column handlers as registered in the registry). These column handlers ordinals are retrieved calling the "GetColExtFirstColOrdinal" function, whose implementation is as follows:

GetColExtFirstColOrdinal.cpp

N.B. With Windows XP Service Pack 2 the criteria by which the column handler ordinals are generated seems to have been modified (or the GetColExtFirstColOrdinal implementation simply needs to be refined). I will update the GetColExtFirstColOrdinal function as soon as possible. The rest of the hook implementation and logic, however, is still valid and working.

As an example of the final core implementation, this is the portion of the code (stripped from the Image Downloader source files) that actually handles the whole bag nodes manipulation job:

winxp_bagnode_manipcode.cpp

FOLDER VIEW STATE MANIPULATION: NON-WINDOWS XP

The system used for manipulating the view state data on non-Windows XP platforms is based on a dedicated COM-hook approach researched and developed just for Image Downloader.

The basic idea is to patch the virtual table of the IShellBrowser interface in order to hook all the invocations to the GetViewStateStream method. As explained earlier, this method is called each time a shell view object needs to read or save its folder specific view state data on persistent storage and specifically returns an IStream interface pointer that the view implementation can call for that purpose. Providing the calling implementation with a pointer to a fake IStream-implementing object, may allow, on a folder path basis, to control the view state data that is read and written by the folder view in the registry. This effectively resolves our problem.

As in the case of the Windows XP solution, we will force the calling view code to fetch a specially composed view state stream data made up from an hard-coded byte template in which the application replaces only the values that are installation-specific (specifically these values are only the ordinals of the column handlers as registered in the registry). As in the case of the Windows XP hook, these column handlers ordinals are retrieved calling the "GetColExtFirstColOrdinal" function, whose implementation is provided in the Win XP section above.

This is the code that hooks the interface virtual table:

HRESULT           hr;
 
HWND        hwMainWnd = NULL;
 
hr = pWebBrowser->get_HWND( (long*) & hwMainWnd );
if ( FAILED( hr ) )
      return FALSE;
 
if ( hwMainWnd == NULL )
      return FALSE;
 
IShellBrowser*          pShBrowser =
      (IShellBrowser*) ::SendMessage( hwMainWnd, WM_GETISHELLBROWSER, 0, 0 ); // no reference counting on this...
 
if ( pShBrowser == NULL )
      return FALSE;
 
// hook GetViewStateStream, if not already done
 
if ( m_pfnOrigMethod == NULL )
{
      // get a pointer to the virtual table
 
      void*       pvVirtTbl = (void*) *( (DWORD*) pShBrowser );
 
      // hook the original method
 
      DWORD       dwOldProtect;
      ::VirtualProtect( (DWORD*) pvVirtTbl + 12, 4, PAGE_READWRITE, & dwOldProtect );
 
      m_pdwVirtTblSlot = (DWORD*) pvVirtTbl + 12;
      m_pfnOrigMethod = (METHODTYPE) * m_pdwVirtTblSlot;
      * m_pdwVirtTblSlot = (DWORD) & GetViewStateStreamHook;
}

The GetViewStateStreamHook function is called in place of the original GetViewStateStream implementation provided by Explorer. Selectively, based on the currently viewed folder, the program decides whether to delegate the execution to the original function (and thus allowing the reading of the data from the registry) or to return to the calling implementation an IStream fake object pointer that will fool the shell view into thinking that the actual data read and written comes from the "Streams" subkeys in the registry, thus allowing to manipulate and control the view state stream. The actual implementation of the code that creates the fake stream data from a byte template is very similar to the corresponding code used in the Windows XP case.

This is the implementation of the CStreamImpl class used for returning the fake IStream interface pointer to the shell view:

class CStreamImpl : public IStream
{
      DWORD             m_dwRef;
      CAtlDeskBand*      m_pDeskBandObj;
 
      IStream*          m_pContainedStream;
      HGLOBAL                 m_hGlobal;
 
      friend class CAtlDeskBand;
 
public:
      CStreamImpl( CAtlDeskBand* pDeskBandObj = NULL, BYTE* pbBuffer = NULL, size_t sBufferLen = 0 ) :
            m_dwRef( 1 ),
            m_pDeskBandObj( pDeskBandObj ),
            m_pContainedStream( NULL ), m_hGlobal( NULL )
            {
                  m_hGlobal = ::GlobalAlloc ( GMEM_MOVEABLE, (DWORD) sBufferLen );
                  if ( m_hGlobal == NULL )
                        return;
 
                  if ( pbBuffer )
                  {
                        PBYTE pData = (PBYTE) ::GlobalLock ( m_hGlobal );
                        ::CopyMemory ( pData, pbBuffer, sBufferLen );
                        ::GlobalUnlock ( m_hGlobal );
                  }
 
                  if ( FAILED( ::CreateStreamOnHGlobal( m_hGlobal, TRUE, & m_pContainedStream ) ) )
                  {
                        m_pContainedStream = NULL;
                        if ( m_hGlobal )
                        {
                              ::GlobalFree ( m_hGlobal );
                              m_hGlobal = NULL;
                        }
                  }
 
                  return;
            }
 
// IUnknown
 
      STDMETHOD(QueryInterface)(REFIID iid, void **ppvObject)
      {
            if( ppvObject == NULL )
                  return E_POINTER;
 
            *ppvObject = NULL;
 
            if ( iid == IID_IUnknown )
                  *ppvObject = (IUnknown*) this;
            else if ( iid == IID_ISequentialStream )
                  *ppvObject = (ISequentialStream*) this;
            else if ( iid == IID_IStream )
                  *ppvObject = (IStream*) this;
            else
                  return E_NOINTERFACE;
 
            ((IUnknown*)(*ppvObject))->AddRef();
 
            return S_OK;
      }
      STDMETHOD_(ULONG, AddRef)()
      {
            return ++m_dwRef;
      }
      STDMETHOD_(ULONG, Release)()
      {
            if ( m_dwRef == 1 )
            {
                  if ( m_pDeskBandObj )
                        m_pDeskBandObj->StreamImplCallback( this );
                  if ( m_pContainedStream )
                        m_pContainedStream->Release();
 
                  delete this;
                  return 0;
            }
            else
                  return --m_dwRef;
      }
 
// ISequentialStream
 
      STDMETHOD(Read)(void *pv, ULONG cb, ULONG *pcbRead)
      {
            return ( m_pContainedStream == NULL ) ? E_FAIL : m_pContainedStream->Read(pv, cb, pcbRead);
      }
 
      STDMETHOD(Write)(void const *pv, ULONG cb, ULONG *pcbWritten)
      {
            return ( m_pContainedStream == NULL ) ? E_FAIL : m_pContainedStream->Write(pv, cb, pcbWritten);
      }
 
// IStream
 
      STDMETHOD(Seek)(LARGE_INTEGER dlibMove, DWORD dwOrigin, ULARGE_INTEGER *plibNewPosition)
      {
            return ( m_pContainedStream == NULL ) ? E_FAIL : m_pContainedStream->Seek(dlibMove, dwOrigin, plibNewPosition);
      }
 
      STDMETHOD(SetSize)(ULARGE_INTEGER libNewSize)
      {
            return ( m_pContainedStream == NULL ) ? E_FAIL : m_pContainedStream->SetSize(libNewSize);
      }
 
      STDMETHOD(CopyTo)(IStream *pstm, ULARGE_INTEGER cb, ULARGE_INTEGER *pcbRead, ULARGE_INTEGER *pcbWritten)
      {
            return ( m_pContainedStream == NULL ) ? E_FAIL : m_pContainedStream->CopyTo(pstm, cb, pcbRead, pcbWritten);
      }
 
      STDMETHOD(Commit)(DWORD grfCommitFlags) {return E_NOTIMPL;}
 
      STDMETHOD(Revert)() {return E_NOTIMPL;}
 
      STDMETHOD(LockRegion)(ULARGE_INTEGER libOffset, ULARGE_INTEGER cb, DWORD dwLockType) {return E_NOTIMPL;}
 
      STDMETHOD(UnlockRegion)(ULARGE_INTEGER libOffset, ULARGE_INTEGER cb, DWORD dwLockType) {return E_NOTIMPL;}
 
      STDMETHOD(Stat)(STATSTG *pstatstg, DWORD grfStatFlag) {return E_NOTIMPL;}
 
      STDMETHOD(Clone)(IStream **ppstm) {return E_NOTIMPL;}
 
};

DOWNLOAD

A trial version of Image Downloader can be downloaded from here.

Quotes

"Among the Windows experts I know personally, no one can beat Vito Plantamura."

- Francesco Balena, Code Architects SRL

"Your NDIS Monitor application, is amongst the most impressive networking code I have seen on the .Net framework."

- Ben Hakim.

Photos

Various images from italian conferences and events (keep the mouse on a thumbnail for a short description):

Me at the Microsoft/HP/Intel organized Route64 event in Milan in May 2005, explaining how COM+ behaves on 64-bit Microsoft operating systems. I was there with the friends of Code Architects.

Me at the Microsoft Security Roadshow event in Bari in April 2006, explaining how the logon process works in Windows NT. There were 250 attendees.

Microsoft Security Roadshow 2006 in Treviso. This is an image of the huge 700-seats conference room.

Me at the Microsoft Security Roadshow 2006 in Treviso. This is a moment of the 3-hours session.

Everything here (code, binaries, text, graphics, design, html) is © 2010 Vito Plantamura and VPC Technologies SRL (VATID: IT06203700965).
If you download something (compilable or not) from the site, you should read the license policy file.
If you want to contact me via email, write at this address.