door:
FransBouma
This is the first post of a series about the new features in LLBLGen Pro v4.0, which was released on April 6th, 2013. Starting with v4.0, the LLBLGen Pro designer is usable both as standalone (as it was before) and as an integrated designer in Visual Studio. I'll explain in this article what this integration is all about and what we had to do to make it work. If you want to know more about how to use this feature, please check out the manual page on this feature.
So how does this integration look like? Below is a screenshot from the manual:
It almost looks like the stand-alone designer, eh? 
Integrating in Visual Studio can be done in several ways: through e.g. a plug-in/add-in, an extension to the editor or a Visual Studio extension. Our integration is done through a Visual Studio extension (VSIX).
How the integration works, user perspective
When you start Visual Studio, the extension isn't visible: no menus, no tool windows, no tool bars. When you load an LLBLGen Pro project (a file with the extension '.llblgenproj') into Visual Studio, or create a new one inside Visual Studio using the normal item creation system, the designer is loaded inside Visual Studio: all panes are opened as tool windows and the project file itself is opened as a document editor inside the document well (central tab area) of Visual Studio. When this document editor, which is a single tab inside the document well, has the focus, the toolbar and menu are visible, and the project's contents is visible in the various tool windows (see screen shot above).
Opening another LLBLGen Pro project inside Visual Studio opens another document editor, but no extra tool windows. Instead, the contents of the tool windows is switched when you switch between document editors so they always show the project contents of the document editor which has focus. For a background on this, please see the section below called 'How we wrote the VSIX'.
Generating code will add the generated VS.NET projects to the loaded solution, if they're not already present there. This way of working as the advantage that you don't have to leave VS.NET to work with the designer, you can add the LLBLGen Pro project to your solution as a bonus don't have to manually add the generated code to your solution.
All functionality, except imports (see below) is the same as with the stand-alone designer, as the stand-alone designer is actually loaded inside Visual Studio and its parts are displayed within the IDE.
How we wrote the VSIX (or: how to lose your hair in 2 months)
Visual Studio is mainly an IDE around editing files: a file is opened with the associated editor of the file's extension in the document well and the editor associated with the file's extension can open 0 or more tool windows and do other things. You can of course open tool windows inside the document well which edit 'objects' and not 'files', however this isn't as simple as it sounds. And precisely that was a problem we had to solve: our designer is also an IDE with a central tab area but instead of opening files in that tab area, we open editors of objects.
To make Visual Studio work like our designer, we were forced to go what I'd call, 'the All-In' route: create custom project types for Visual Studio, rewrite our project explorer to build objects in the Solution Explorer in Visual Studio, open custom tool windows inside the document well of Visual Studio for each object edited. We quickly learned that creating an extension to Visual Studio is not a walk in the park: the code isn't that complex, it's the lack of serious documentation, examples and above all, the quirks of the IDE which made it a big a project to complete. Looking at the work for the All-in variant, we decided to go for the alternative: create a file-extension bound editor. Looking back this was a wise decision, as this extension too took us a long time to create, almost two months.
With v3.0, released in early 2010, we designed the designer to be modular so it wouldn't be to much work to make it usable from within another application, e.g. Visual Studio. The UI is broken up in several parts and all designer logic is structured in an event-oriented system with several controllers on top. The advantage of this is that whenever something happens inside the UI, one or more events are raised and observers to these events (or the events as result of these events) are picking them up and act accordingly. These observers are all located outside the UI. The main advantage of this is that even though the designer in stand-alone form looks like an MDI window with tool windows docked inside it, the control of these docked windows and what happens inside them is not within the MDI window.
The v4.0 designer is a successor of v3's so with the modular design at hand we decided to create a Visual Studio extension tied to the LLBLGen Pro project file file-extension '.llblgenproj'. In Visual Studio terms: a custom document editor. A Visual Studio extension is actually a Visual Studio package, packed in the form of an 'extension'. The extension contains of the following parts:
- Each docked 'pane' in the designer is opened as a Visual Studio Tool window and docked inside the IDE
- The document well of the designer is opened as a Visual Studio Document Editor in the document well.
- The toolbar is created as a button bar inside Visual Studio
- The menus are pivoted into a single menu in the main menu of Visual Studio
- A factory which produces new document editors
To achieve this, we wrote an extension which loads the standalone designer into Visual Studio, and instead of telling it to open an MDI window and dock some panes into it, it simply tells the loaded designer to hand the pane windows and its own window to the extension object. The extension object then opens the Visual Studio tool windows and its main editor and there you are: integrated inside Visual Studio.
Simple? Well… not quite.
The fun starts with where Visual Studio extensions are located. There are two locations: inside the user's My Documents folder and inside the Visual Studio installation folder. Both locations are not ideal if you want your code to also be usable as stand-alone. The trick was to create a registry key at installation which is read by the Visual Studio extension which points to the folder where LLBLGen Pro was installed. I tried numerous things, including junctions but these all failed. The registry key is easy enough though.
The next problem was something we never had to run into before: what if multiple LLBLGen Pro projects are opened inside Visual Studio? Although opening multiple LLBLGen Pro projects will open multiple instances of the designer, without separating the AppDomains, we'd run into problems where statics were suddenly shared among instances. However, creating a new AppDomain per instance wouldn't work as that would require the editor panes we would like to show inside the designer to serializable, which isn't possible. Luckily we had grouped all statics inside objects which lived in a few singletons which act as Services inside our designer.
Whenever a designer instance was loaded into the extension, we assigned to it an ID and we'd request an object we dubbed integration controller from the designer, which would act like the controller of the designer and data exchange. Through this controller we also could tell the designer instance which document editor instance was active. The services/singletons then switched their state to the instance ID passed to the integration controller and all code of the designer and its sub-systems would think they were running inside the stand-alone version.
This state-switching also helped with the tool windows of Visual Studio: by default each Visual Studio tool window is created just once. You can create multiple instances if you must, but you'll soon learn that multiple instances of a tool window makes things less usable. So each time a document editor became active inside the IDE, the document editor would get called by the IDE and we simply made the extension to update the associated designer's state to the one associated with the document editor's ID and as well update the tool windows to the ones of the designer instance associated with the ID.
On to undo-redo. Our own designer has a full undo-redo system, based on our OSS library Algorithmia (available on github), and so does VS.NET. Of course these aren't compatible with each other however the designer's undo-redo should work in sync with the VS.NET undo-redo system. After looking into making things compatible for a few days, it occurred to me it doesn't matter: whenever a user does something through the document editor, the IDE will query your editor if the undo/redo state of the buttons should change. Our designer already kept track of that, so simply returning this state was enough. Whenever the user pressed the undo/redo buttons (or the keystrokes) inside the IDE, the document editor with focus got a call which was passed through to the integration controller to do the actual undo/redo on the model, similar to if the user would undo/redo in the stand-alone designer.
Undo-redo is a global system, however with multiple instances, there are multiple undo/redo stacks active at once, one per instance. Algorithmia's undo/redo system is able to work with that: it works with 'scopes'. This means you can have local undo/redo separated from the global undo/redo. We utilize this in the integration so each designer instance is a separate scope inside the undo/redo system, so undo/redo information isn't shared.
The last problem I'd like to address is installation. Visual Studio extensions, VSIXs, can be installed in two ways: you either dump all files in a folder in one of the two extension locations and let VS.NET know some extension installation took place or you wrap your code inside a .vsix file. The latter has as advantage that registration of your extension is done for you without a slow re-scan of the complete extension folders. It also takes care of locating the extensions folders. I first looked at the former alternative, as we use our own installer but the logic to locate the folders and install everything in both 2010 and 2012 was a pain, and everything was taken care of with a .vsix file: we simply run it silently during installation and it installs the extension in both 2010 and 2012 if present, in the right folder and registers the extension as well.
Was it all worth it?
Yes. Developing this extension was a true pain, I'd be lying if I called it something else, but as a tool / framework developer it's part of our job to write the painful code so our customers don't have to; that's why they pay for a license. In the end, I can only say I'm very pleased with the result and have heard only positive feedback from users, so in short: yes it was worth it.
As I said above, creating a Visual Studio extension isn't a walk in the park: using .NET for an extension is possible, the SDK looks OK, the starter project gives you a lot of code to start with, but after a short while you'll learn that below this thin layer of chrome, the actual metal is pretty rusty and shows its true age. The technical debt of the C/C++/COM based system is so big, I doubt it will ever get better, especially as there's little incentive to do so: the biggest 'customer' of VS.NET's integration system is Microsoft itself, and as we all know internal code is more rough, less documented than software that's delivered to customers outside the company.
If you're planning to develop your own Visual Studio extension, be aware of the fact you'll spend a lot of time cursing the IDE, it's long list of quirks and inconsistencies, missing docs and serious lack of examples how to do things: you'll run into many posts with similar questions as yours but with no or unusable answers. You're entitled to throw your hands into the air and scream the filthiest curse words in the direction of Redmond, but that won't get you near your goal. As an engineer, you have to think and act like one: write some prototype code and attach a debugger in another VS.NET instance to your test VS.NET instance to see what's happening, what's called when you do X.
It will be a struggle at first, but you'll get there: and you'll have to do it yourself, there's no-one out there to help you. But hey, that's the life of an engineer.
