Rafael Rivera

Generating metadata for use with the windows crate for Rust

(revised )

The windows crate for Rust has reached a sweet spot. It ships with pre-generated bindings for the entire Windows API surface and a code generator that can be used to generate your own bindings and create dependent crates. And if you pair the crate with the Windows Metadata Generator, you can transform all of your favorite C and C++ headers into Rust bindings. In fact, I already generated a few crates, such as the Microsoft Debug Interface Access (DIA) SDK, Microsoft DirectStorage SDK and the Microsoft Windows App SDK.

Let's walk through using the metadata generator to recreate most of the DIA SDK crate from scratch.

Scaffolding

First, create a new crate using a library template (cargo init --lib).

Now, create the .windows\winmd directory.

This directory will store metadata—Common Language Runtime (CLR) assemblies (as defined in ECMA-335) that provide a bunch of information about the Windows APIs. This information is useful to developers looking to generate bindings for different programming languages. (This path is historically where Rust for Windows stored its libraries, metadata, etc. This can be changed.)

Create a .metadata folder as well to store the Microsoft Build (MSBuild) project and supporting files needed to generate metadata. (This can also be changed.)

dia-rs
|-- .gitignore
|-- .metadata/
|-- .windows/
|   `-- winmd/
|-- Cargo.toml
`-- src
    `-- lib.rs

Metadata project

Create .metadata\generate.proj and populate it with some minimal XML:

<?xml version="1.0" encoding="utf-8"?>
<Project>
</Project>

This is the start of a MSBuild project.

Add the Sdk attribute to the root node. Set its value to Microsoft.Windows.WinmdGenerator/0.40.14-preview. (The version suffix is required as the package is not marked as stable.)

 <?xml version="1.0" encoding="utf-8"?>
-<Project>
+<Project Sdk="Microsoft.Windows.WinmdGenerator/0.40.14-preview">
 </Project>

This is the start of a Software Development Kit (SDK)-style MSBuild project.

Microsoft.Windows.WinmdGenerator refers to a NuGet package identifier tied to the Microsoft Windows Metadata Generator SDK. This project SDK contains a bunch of actions that will execute on what we put into this project file moving forward.

Next, add a few global properties that the winmd generator needs:

 <?xml version="1.0" encoding="utf-8"?>
 <Project Sdk="Microsoft.Windows.WinmdGenerator/0.40.14-preview">
+   <PropertyGroup Label="Globals">
+       <OutputWinmd>../.windows/winmd/Microsoft.Dia.winmd</OutputWinmd>
+       <WinmdVersion>255.255.255.255</WinmdVersion>
+   </PropertyGroup>
 </Project>

  • OutputWinmd specifies where the generator will write its output to.

  • WinmdVersion indicates the version number that will be attached to the output.

    Its value is not important for this crate, so it is set to its maximum value to make that clear. (If you want to share metadata externally, you may want to swap this out with a value that is consistent with Semantic Versioning.)

Now, define a partition to group together all the DIA APIs and their respective sources:

 <?xml version="1.0" encoding="utf-8"?>
 <Project Sdk="Microsoft.Windows.WinmdGenerator/0.40.14-preview">
     <PropertyGroup Label="Globals">
         <OutputWinmd>../.windows/winmd/Microsoft.Dia.winmd</OutputWinmd>
         <WinmdVersion>255.255.255.255</WinmdVersion>
     </PropertyGroup>
+    <ItemGroup>
+        <Headers Include="..." />
+        <Partition Include="main.cpp">
+            <TraverseFiles>@(Headers)</TraverseFiles>
+            <Namespace>Microsoft.Dia</Namespace>
+        </Partition>
+    </ItemGroup>
 </Project>

The partition Include attribute points to the C/C++ file used as the generator tooling starting point. This file must be syntactically correct. That is, it must effectively compile without errors.

The partition contains several elements:

  • TraverseFiles points to one or more headers that will be scraped for content.

    These headers should be reachable (directly or otherwise) from the Include starting point.

    The Headers property used here is not specific to the winmd generator. It is simply a MSBuild property our future include string and keep the project file tidy.

  • Namespace assigns a name to the logical grouping of functions and constants pulled out of the inputs provided.

DIA APIs are primarily described in an Interface Definition Language (IDL) file dia2.idl and an ancillary header cvconst.h. (The DIA SDK ships with a C/C++ header dia2.h but it's outdated.) These files are located at %VSINSTALLDIR%\DIA SDK\idl and %VSINSTALLDIR%\DIA SDK\include.

Create and add additional MSBuild properties to point to those locations:

 <?xml version="1.0" encoding="utf-8"?>
 <Project Sdk="Microsoft.Windows.WinmdGenerator/0.40.14-preview">
     <PropertyGroup Label="Globals">
         <OutputWinmd>../.windows/winmd/Microsoft.Dia.winmd</OutputWinmd>
         <WinmdVersion>255.255.255.255</WinmdVersion>
+        <DiaSdkRoot>$(VsInstallDir)\DIA SDK</DiaSdkRoot>
+        <DiaIdlRoot>$(DiaSdkRoot)\idl</DiaIdlRoot>
+        <DiaIncRoot>$(DiaSdkRoot)\include</DiaIncRoot>
     </PropertyGroup>
     <ItemGroup>
        <Headers Include="..." />
        <Partition Include="main.cpp">
            <TraverseFiles>@(Headers)</TraverseFiles>
            <Namespace>Microsoft.Dia</Namespace>
        </Partition>
    </ItemGroup>
 </Project>

Then specify the input files using these new properties.

 <?xml version="1.0" encoding="utf-8"?>
 <Project Sdk="Microsoft.Windows.WinmdGenerator/0.40.14-preview">
     <PropertyGroup Label="Globals">
         <OutputWinmd>../.windows/winmd/Microsoft.Dia.winmd</OutputWinmd>
         <WinmdVersion>255.255.255.255</WinmdVersion>
         <DiaSdkRoot>$(VsInstallDir)\DIA SDK</DiaSdkRoot>
         <DiaIdlRoot>$(DiaSdkRoot)\idl</DiaIdlRoot>
         <DiaIncRoot>$(DiaSdkRoot)\include</DiaIncRoot>
     </PropertyGroup>
     <ItemGroup>
-        <Headers Include="..." />
+        <Headers Include="$(DiaIncRoot)\cvconst.h;..." />
         <Partition Include="main.cpp">
             <TraverseFiles>@(Headers)</TraverseFiles>
             <Namespace>Microsoft.Dia</Namespace>
         </Partition>
     </ItemGroup>
 </Project>

But wait, you need a way to handle dia2.idl.

To compile the IDL, add a reference to one or more IDL files with the Idl element. The winmd generator will send these files to the IDL compiler and write out headers to the location pointed to by the built-in property CompiledHeadersDir.

Using that property, specify the auto-generated dia2.h input:

 <?xml version="1.0" encoding="utf-8"?>
 <Project Sdk="Microsoft.Windows.WinmdGenerator/0.40.14-preview">
     <PropertyGroup Label="Globals">
         <OutputWinmd>../.windows/winmd/Microsoft.Dia.winmd</OutputWinmd>
         <WinmdVersion>255.255.255.255</WinmdVersion>
         <DiaSdkRoot>$(VsInstallDir)\DIA SDK</DiaSdkRoot>
         <DiaIdlRoot>$(DiaSdkRoot)\idl</DiaIdlRoot>
         <DiaIncRoot>$(DiaSdkRoot)\include</DiaIncRoot>
     </PropertyGroup>
     <ItemGroup>
+        <Idls Include="$(DiaIdlRoot)\dia2.idl" />
-        <Headers Include="$(DiaIncRoot)\cvconst.h;..." />
+        <Headers Include="$(CompiledHeadersDir)\dia2.h;$(DiaIncRoot)\cvconst.h" />
         <Partition Include="main.cpp">
             <TraverseFiles>@(Headers)</TraverseFiles>
             <Namespace>Microsoft.Dia</Namespace>
         </Partition>
     </ItemGroup>
 </Project>

Then, create the partition's main.cpp and include the header files to be associated with the partition. Also add all dependent headers to ensure compilation succeeds:

#include <windows.h>
#include <dia2.h>
#include <cvconst.h>

Finally, in a Windows Terminal, issue a dotnet build command. You should see something similar to the below. (You may need to launch a Visual Studio Command Prompt due to a bug at the moment.)

Determining projects to restore...
...
Compiling idl files...
Scraping headers for crossarch...
Microsoft.Dia:common - Scanning to C:\Sources\dia-rs\.metadata\obj\generated\common\Microsoft.Dia.cs...
Scraping constants and enums...
Writing winmd...
Winmd emitted at: C:\Sources\dia-rs\.windows\winmd\Microsoft.Dia.winmd

Build succeeded.
  0 Warning(s)
  0 Error(s)

Rust Crate: Tooling

With freshly generated metadata now available, let's work on the Rust crate.

First, edit Cargo.toml, change the crate name to microsoft-dia, and configure a workspace. This will allow us to store related crates (e.g. libraries, tools, samples) inside our main crate.

[package]
name = "dia-rs"
version = "0.1.0"
edition = "2021"

-# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
+[workspace]
+members = [
+    "crates/tools/*",
+]

[dependencies]

Then, create another crate using the binary (application) template (cargo init --bin crates\tools\api).

dia-rs
|-- .gitignore
|-- .metadata
|-- .windows
|   `-- winmd
|-- Cargo.toml
|-- crates/
|   `-- tools/
|       `-- api/
|         `-- src/
|             `-- main.rs
|         `-- .gitignore
|         `-- Cargo.toml
`-- src
    `-- lib.rs

Afterwards, add two dependencies (cd crates\tools\api; cargo add windows-bindgen; cargo add windows-metadata).

Time to write some code!

Open main.rs and add a use declaration to bring in the windows_metadata reader File type.

use windows_metadata::reader::File;

Then delete all the sample code in fn main() and create a vector of metadata files.

Use the with_default helper function whenever possible. This function prepends to your vector the paths to standard Windows metadata files that ship with the windows_metadata crate, like Windows.Win32.winmd. This allievates the need for you to tow those files around yourself.

use windows_metadata::reader::File;

+fn main() {
+    let files = File::with_default(&[".windows/winmd/Microsoft.Dia.winmd"]).unwrap();
+}

Now, call the windows_bindgen component function to generate the bindings we need. Pass in the metadata root namespace and a reference to a vector of metadata files to read and generate code for.

+use std::{fs, path::PathBuf};
use windows_metadata::reader::File;

fn main() {
     let files = File::with_default(&[".windows/winmd/Microsoft.Dia.winmd"]).unwrap();

+    let output_path = PathBuf::from("src/bindings.rs");
+    if output_path.exists() {
+        fs::remove_file(&output_path).unwrap();
+    }
+    fs::write(
+        &output_path,
+        windows_bindgen::component("Microsoft.Dia", &files),
+    )
+    .unwrap();

Finally, execute rustfmt against the newly generated code to make it more readable. (This is also very helpful for debugging.)

-use std::{fs, path::PathBuf};
+use std::{fs, path::PathBuf, process::Command};
use windows_metadata::reader::File;

fn main() {
    let files = File::with_default(&[".windows/winmd/Microsoft.Dia.winmd"]).unwrap();

    let output_path = PathBuf::from("src/bindings.rs");
    if output_path.exists() {
        fs::remove_file(&output_path).unwrap();
    }
    fs::write(
        &output_path,
        windows_bindgen::component("Microsoft.Dia", &files),
    )
    .unwrap();

+   let mut child = Command::new("rustfmt")
+       .args([&output_path])
+       .spawn()
+       .expect("Failed to start rustfmt");
+
+   child.wait().expect("rustfmt failed");

Here's the complete program, for reference.

use std::{fs, path::PathBuf, process::Command};
use windows_metadata::reader::File;

fn main() {
    let files = File::with_default(&[".windows/winmd/Microsoft.Dia.winmd"]).unwrap();

    let output_path = PathBuf::from("src/bindings.rs");
    if output_path.exists() {
        fs::remove_file(&output_path).unwrap();
    }
    fs::write(
        &output_path,
        windows_bindgen::component("Microsoft.Dia", &files),
    )
    .unwrap();

    let mut child = Command::new("rustfmt")
        .args([&output_path])
        .spawn()
        .expect("Failed to start rustfmt");

    child.wait().expect("rustfmt failed");
}

Let's try it out.

Navigate to the workspace root (cd ../../../) and run the binary output of the crate we just created (cargo run -p api). After a few moments, you should now have a bunch of automatically generated code in src\bindings.rs!

Rust Crate: DIA SDK

Let's now complete some chores to finish off the DIA crate.

First, open Cargo.toml and add the windows crate as a dependency, with the implement and Win32_Foundation features enabled.

[dependencies.windows]
version = "0.44.0"
features = [
    "implement",
    "Win32_Foundation",
]

Then remove everything in lib.rs and add a windows external crate declaration. This will bring the windows crate into scope so that various types in our bindings.rs resolve correctly (e.g. ::windows::Win32::Foundation::BOOL).

extern crate windows;

Now use the newly generated bindings and re-export all the types within. This is not strictly required but is recommended to remove an awkward level of indirection. That is, instead of typing use dia-rs::bindings::IDiaAddressMap you can now type use dia-rs::IDiaAddressMap.

extern crate windows;

+pub mod bindings;
+pub use bindings::*;

Attempt to build the crate (cargo build).

You will receive quite a few build errors. That's because we haven't brought in enough of the windows crate to satisfy the needs of our crate. The Windows API surface is huge and to keep build times under control, the types and APIs are grouped together and tied to features in the windows crate. To simplify development, the feature names match the type's namespace. So, for example, the ::windows::Win32::System::Com::VARIANT type lives in the Win32::System::Com namespace and is tied to a feature named Win32_System_Com. Alternatively, you can also look up a type or API in the documentation.

Using the above knowledge, resolve the build errors by addding the missing windows features.

[dependencies.windows]
version = "0.44.0"
features = [
    "implement",
    "Win32_Foundation",
+   "Win32_System_Com_StructuredStorage",
+   "Win32_System_LibraryLoader",
+   "Win32_System_Ole",
]

Attempt to build the crate again (cargo build). Success!

Reader Homework

At this point, you have built nearly everything in the official Microsoft DIA SDK Rust crate!

Some remaining tasks include:

  • creating a helper to instantiate a DIA COM class that implements IDiaDataSource (hint: diaguids.lib in the SDK has a NoRegCoCreate function))
  • adding some missing constants (e.g. E_PDB_INVALID_SIG) and types (e.g. SymTag)
  • writing sample crates

See also