Feb
24
2010

How to Write Native C++ Debugger Visualizers in Visual Studio for Complicated Types

Introduction

This explains how to change how the visual studio native debugger displays data for very complicated types. It explains how to change this:

debugger_before

to this:

after

In other words, it greatly simplifies displaying native code in the debugger window.

This is not a tutorial on how to modify or customize autoexp.dat in general. This is a very specific tutorial on how to custom the debugger variable display for types that are very complicated and involve deep aggregation or inheritance.

Problem

Last year, I was trying to debug a particular problem and found it difficult to view a particular data type in the visual studio debugger. This particular data type was way over engineered and  complicated just for the sake of being complicated. It was a class that simply held a string and did path operations on the string. But it held the string in a complicated morass of class hierarchies that made displaying the actual string very difficult in the visual studio native debugger. So I embarked on a task to display the data I wanted in the debugger windows. This is possible by modifying the file:

“C:\Program Files (x86)\Microsoft Visual Studio 9.0\Common7\Packages\Debugger\autoexp.dat”

I have created a demo class hierarchy that is needlessly complex and demonstrates the problem that you have to drill down deep in the VS debugger to see the data. I also included code to run it:

  1. // NativeVisualizerTest.cpp : Defines the entry point for the console application.
  2. //
  3. #include “stdafx.h”
  4. #include <string>
  5. #include <memory> // for autoptr
  6. // The inner string, taken from std::string
  7. typedef std::basic_string<TCHAR> StupidString;
  8. // Another string that makes debugging that much harder
  9. class SpecialString    : public StupidString
  10. {
  11. public:
  12. SpecialString()     {}
  13. SpecialString(TCHAR* s)    : StupidString(s)    {}
  14. ~SpecialString()    {}
  15. };
  16. // Another class that wraps the SpecialString
  17. class PathPrivateInternal
  18. {
  19. public:
  20. PathPrivateInternal() {}
  21. PathPrivateInternal(TCHAR* s)
  22. : mString(s) {}
  23. ~PathPrivateInternal(){}
  24. private:
  25. SpecialString mString;
  26. };
  27. // The final class that holds a PathPrivateInternal
  28. // Visualizing this crazy class in the debugger will NOT be fun!
  29. class Path
  30. {
  31. public:
  32. Path()
  33. : mPtr(new PathPrivateInternal())
  34. { }
  35. Path(TCHAR* s)
  36. : mPtr(new PathPrivateInternal(s))
  37. { }
  38. ~Path()    { }
  39. private:
  40. std::auto_ptr<PathPrivateInternal> mPtr;
  41. };
  42. int _tmain(int argc, _TCHAR* argv[])
  43. {
  44. StupidString        str(_T(“I am a foo bar”));
  45. SpecialString      strB(_T(“I am a foo bar”));
  46. PathPrivateInternal ppi(_T(“I am a foo bar”));
  47. Path                  p(_T(“I am a foo bar”));
  48. return 0;
  49. }

So in this example, I have three concrete classes, a smart pointer (i.e. std::autoptr), and a typedef that have to be deciphered in order to display the string I really want to see. This is really a yucky mess: all for a class to simply hold a string and perform path like operations on it.

Demonstration

If I put a breakpoint in line 54, and add p to the watch window, I will see this in the debugger:

debugger_before

Notice how many sub tree’s I had to open in the debugger in order to see the string? !! Yuck !!

Unraveling the Gordian knot

The trick to displaying this in the debugger is to write a visualizer for each different class in this hierarchy. First you start from the inner class, and work towards the outer class. In this example the inner class is std::basic_string<TCHAR> and the outer string is Path.

The following list shows the order we need to write visualizers:

Type

StupidString
SpecialString
PathPrivateInternal
std::autoptr
Path

StupidString

In the case of StupidString we are in luck. autoexp.dat already contains a visualizer for std::basic_string that displays the data nicely.

image

But the other three types do not display helpful information in the preview window.

SpecialString

Most of the work to visualize all the data types takes place here in SpecialString. The class SpecialString isn’t a mere typedef, so there is a little more work to get it to display in the debugger. The trick is to treat it like a std::basic_string. So find the section in autoexp.dat that displays std::basic_string and plagiarize that code like crazy. Except don’t copy the children section.

Here is the ANSI string version of std::basic_string

  1. ;——————————————————————————
  2. ;  std::string/basic_string
  3. ;——————————————————————————
  4. std::basic_string<char,*>{
  5. preview        ( #if(($e._Myres) < ($e._BUF_SIZE)) ( [$e._Bx._Buf,s]) #else ( [$e._Bx._Ptr,s]))
  6. stringview    ( #if(($e._Myres) < ($e._BUF_SIZE)) ( [$e._Bx._Buf,sb]) #else ( [$e._Bx._Ptr,sb]))
  7. children
  8. (
  9. #if(($e._Myres) < ($e._BUF_SIZE))
  10. (
  11. #([actual members]: [$e,!] , #array( expr: $e._Bx._Buf[$i], size: $e._Mysize))
  12. )
  13. #else
  14. (
  15. #([actual members]: [$e,!],  #array( expr: $e._Bx._Ptr[$i], size: $e._Mysize))
  16. )
  17. )
  18. }

In this code, all that is needed is the “preview” code in line 5. Now there is a nice gotcha here. In order to properly display the type for SpecialString this program has to be compiled for non Unicode. That is because the visualizer for std::basic_string plagiarized from above is for <char> types. If I want to compile for Unicode, I need to plagiarize the following code (some stuff omitted for clarities sake):

  1. std::basic_string<unsigned short,*>|std::basic_string<wchar_t,*>{
  2. preview
  3. (
  4. #if(($e._Myres) < ($e._BUF_SIZE)) ( [$e._Bx._Buf,su] )
  5. #else ( [$e._Bx._Ptr,su] )
  6. )
  7. stringview
  8. (
  9. #if(($e._Myres) < ($e._BUF_SIZE)) ( [$e._Bx._Buf,sub] )
  10. #else ( [$e._Bx._Ptr,sub] )
  11. )
  12. }

The stringview part allows you to display your data Text Visualizer dialog. Copy that to the SpecialString visualizer too:

  1. SpecialString{
  2. preview        ( #if(($e._Myres) < ($e._BUF_SIZE)) ( [$e._Bx._Buf,s]) #else ( [$e._Bx._Ptr,s]))
  3. stringview    ( #if(($e._Myres) < ($e._BUF_SIZE)) ( [$e._Bx._Buf,sb]) #else ( [$e._Bx._Ptr,sb]))
  4. }

image

Now the debugger properly displays the string in the watch windows…

image

Notice now that the last two classes PathPrivateInternal and Path both now show some useful information in their preview windows. This is good, but not the final solution.

PathPrivateInternal

Now to visualize class PathPrivateInternal, will take just a little work.

  1. PathPrivateInternal{
  2. preview (
  3. [$e.mString]
  4. )
  5. }

Notice I only had to specify the member variable mString. There was no need to specify a type to render the data as, since the debugger already knows that mString is of type SpecialString. And the debugger already knows how render that. Here are the results:

debugger_middle

Path

The last part is to specify the visualizer for class Path.

  1. Path{
  2. preview (
  3. [$e.mPtr]
  4. )
  5. }

Which displays this:

image

However notice that it rendered the text as auto_ptr “I am a foo bar”. This is because of the visualizer for auto_ptr, which is also included in autoexp.dat:

  1. ;——————————————————————————
  2. ;  std::auto_ptr
  3. ;——————————————————————————
  4. std::auto_ptr<*>{
  5. preview
  6. (
  7. #(
  8. “auto_ptr “,
  9. (*(($T1 *)$e._Myptr))
  10. )
  11. )
  12. children
  13. (
  14. #(
  15. ptr: (*(($T1 *)$e._Myptr))
  16. )
  17. )
  18. }

Notice the hard coded string in the preview section. Also notice that the template type in the auto_ptr declarations is a star (*) meaning use this visualizer for all type’s that auto_ptr is using. There are a few things that need to be done to fix this. First, simply specialize the visualizer for when auto_ptr is holding a type of PathPrivateInternal.

  1. std::auto_ptr<PathPrivateInternal>{
  2. preview
  3. (
  4. [$e._Myptr]
  5. )
  6. }

Now, in the autoexp.dat file there are three sections, listed in order:

  1. [AutoExpand]
  2. [Visualizer]
  3. [hresult]

If the custom visualizer for std::auto_ptr<PathPrivateInternal> is placed after std::auto_ptr<*> then this custom visualizer will get ignored. This is because the debugger uses the first visualizer that it finds in the autoexp.dat file that satisfies the type criteria. And the star (*) template type unfortunately accepts all types including the std::auto_ptr<PathPrivateInternal>. Therefore put all custom visualizers at the beginning of the Visualizer section, NOT the end. After doing that, you will get the results shown below:

image

Here is another view with the Path type expanded:

image

Summary

Here is the final listing of all the visualizers:

  1. ;——————————————————————————
  2. ;  My Custom Types
  3. ;——————————————————————————
  4. SpecialString{
  5. preview        ( #if(($e._Myres) < ($e._BUF_SIZE)) ( [$e._Bx._Buf,s]) #else ( [$e._Bx._Ptr,s]))
  6. stringview    ( #if(($e._Myres) < ($e._BUF_SIZE)) ( [$e._Bx._Buf,sb]) #else ( [$e._Bx._Ptr,sb]))
  7. }
  8. PathPrivateInternal{
  9. preview (
  10. [$e.mString]
  11. )
  12. }
  13. std::auto_ptr<PathPrivateInternal>{
  14. preview
  15. (
  16. [$e._Myptr]
  17. )
  18. }
  19. Path{
  20. preview (
  21. [$e.mPtr]
  22. )
  23. }

And remember these main points:

  • Start with inner nested types and work your way outwards
  • Copy from STL types when needed.
  • Put all custom visualizers at the beginning of the [Visualizer] section in autoexp.dat
  • Use stringview to display text in the Text Visualizer if needed.
  • Use specialized template types to display special cases if needed.

posted in visual studio, windows by Chris Johnson

Follow comments via the RSS Feed | Leave a comment | Trackback URL

7 Comments to "How to Write Native C++ Debugger Visualizers in Visual Studio for Complicated Types"

  1. ron wrote:

    great post! great article!

  2. Alex wrote:

    Nice. This makes C# and Java ToString() look like heaven :) But at least this is do-able in C++ too, if somewhat painful.

  3. PBY wrote:

    I need to debug the graphics program, and I want to get the position info more easier.
    Thanks to the article, saving me a lot of debugging time!

  4. PBY wrote:

    And there is another yuck when text parser.
    e.g. “vector” is not valid because there is a blank between “int,” and “*”!!!

  5. Shyamal wrote:

    Nice article. A good read. Will definitely help me debug better.

  6. Debugger visualizer of GDI objects for unmanaged C++ | trouble86.com wrote:

    [...] from the MSDN documentation and examples, I found this: http://www.idigitalhouse.com/Blog/?p=83 … however, it “only” covers textual data. Virtually all other information was [...]

  7. John wrote:

    What do I do if there were no visualizer for StupidString?

    Suppose I want to display the first byte as binary data: 00101110.

    Can that be done with this trick?

Leave Your Comment

 
Powered by Wordpress and MySQL. Theme by openark.org