There's not much to say about this generic Bluetooth-enabled keyboard key blaster.
I did enjoy discovering it's just a re-skinned and reconfigured Satechi Siri Button. (But that's totally reasonable.)
I haven't explored the UART interface yet; it would be nice to reconfigure the device to send keys of my choosing.
Since the reintroduction of Aero Glass in Windows 10, I've been receiving questions on how to incorporate that functionality into 3rd party applications. A few nights ago I looked into it and here's my guidance:
Abandon code that uses DwmEnableBlurBehindWindow. This function hasn't been deprecated, oddly enough, but is effectively dead on Windows 10. Stop using it. (Also consider abandoning DwmExtendFrameIntoClientArea.)
Start using SetWindowCompositionAttribute directly. It's not officially documented but here's the plumbing you need, if you're writing C# utilizing Interop Services tooling:
On the view side, you don't need to worry about chroma keys anymore! Simply ensure your window uses a (background) brush with an alpha channel and the compositor will handle the rest.
The sample project I used to create the screenshot above can be found on GitHub.
Have fun!
Thursday, I picked up a Surface 3 — my first Surface with pen input — and was surprised at the lack of customization options for the pen's top button. Searching around, I found some clever hacks using AutoHotkey and EventGhost but I wasn't really interested in installing middleware. So I took a peek under the hood and found an inbox solution instead.
Thankfully, Microsoft was nice enough to bake in some overrides for which app gets launched. These overrides, unsurprisingly, live in the Lockscreen ClickNote component (lockscreencn.dll).
Upon every click of the pen button, this component reaches out to a registry key, specifically:
\Software\Microsoft\Windows\CurrentVersion\Authentication\LogonUI\CN
It looks for an AppID or DesktopAppPath value and if one is found, retrieves its data and executes an immersive or desktop app accordingly. Otherwise, OneNote is launched via a built-in AppID:
Microsoft.Office.OneNote_8wekyb3d8bbwe!Microsoft.OneNoteIm
Before you go stuffing Notepad or your app into the DesktopAppPath value, be aware that the path is passed through GetFileAttributes to test for existence. And before passing through ShellExecuteEx, a command line argument of /screenclip, /fromlockscreen, or /hardwareinvoke is tacked on, depending on how the button is clicked or the state of the Surface at time of click. So, for example, if you want to avoid Notepad complaining about the lack of a /hardwareinvoke.txt, you will want to wrap it in a script.
I haven't spent much time on the immersive app side, so am eager to see what folks do in that space. I plugged in FreshPaint's AppID, for example:
Microsoft.FreshPaint_8wekyb3d8bbwe!Microsoft.FreshPaint
... and had mixed (but mostly positive) results.
Free idea corner: PowerPoint could benefit from button-based slide navigation. An app-aware button remapping hub would be cool too.
As part of a recent open specifications update, Microsoft has revealed it's extending the Wi-Fi Display specification to improve a number of wireless display scenarios. These extensions are described in the MS-WFDPE-Preview and MS-WDHCE-Preview specifications and include additions like a low-latency stream to carry mouse cursor data, a method of managing desired latency, and better error reporting.
Let's take a look at each of the additions.
In scenarios where the source device changes video stream resolution or refresh rate -- think gaming -- devices normally require Real-Time Streaming Protocol (RTSP) renegotiation or, more often, freak the hell out and require you to restart your streaming experience. To smooth that over, Microsoft is introducing a method of detecting these changes (and a way for devices to report they support such). More specifically, devices that report support for this feature will monitor the H.264 stream's sequence parameter set/picture parameter set (SPS/PPS) for changes in resolution and frame rate and will adapt seamlessly.
When it comes to latency, we typically think lower is better. But that's not always the case. For example, gamers require low latency to minimize input lag. Because video frames are pumped through as fast as possible, it's common for some tearing or artifacts to appear. But movie viewers don't care about latency. They want a pixel perfect jitter-free viewing experience. To achieve that, devices may extend their input buffer and hold onto video frames longer, a method that introduces a measurable but completely acceptable amount of latency.
The device manufacturer's dilemma surfaces here: Do they optimize for gaming? Or for casual movie viewers? Or do they release two SKUs of the same hardware with slightly tweaked software?
To overcome this huge pain point, Microsoft is introducing a capability for devices to receive a "latency mode" from source devices. The idea is that the source will have the context and responsibility of communicating the user's intended use of the wireless display. For example, the source could detect which app is in use (e.g. Windows Media Player or Microsoft PowerPoint) and send the appropriate latency mode (e.g. high or low, respectively).
Wi-Fi Display is pretty simple in terms of its inputs. It supports one stream that is chock full of audio and video data. That works great for movies but not so much for scenarios involving input. And that's especially true for a mouse.
Microsoft is introducing a capability that will eliminate the move-the-mouse-and-wait game by decoupling the mouse from the video completely. This works by enabling a source device to send a separate mouse stream to a target device. The receiving side would then be responsible for combining the mouse cursor data with whatever is being displayed on the screen at the time.
If this sounds familiar, that's because Microsoft already does something very similar as part of its Remote Desktop Protocol (RDP).
You're streaming a game then poof, the stream is dead. What happened? From the perspective of the source device, you lost connection to the target. It knows something happened. But that's all the information you're going to get. Good luck troubleshooting that.
Microsoft is introducing a more formal method of reporting error details back to the source device. Supported devices will be on guard for "teardown" requests and provide reasons to enhance diagnostics and improve overall usability.
Microsoft is also opening up some metadata enhancements made by Intel for its Intel Wireless Display (WiDi) solution. (These enhancements are listed in the tightly controlled Intel WiDi Specification.) Devices can use Intel-defined fields to report back rich metadata such as a friendly name, support URL, version, and logo.
From an operating system perspective, most of these features are available for use in Windows 10 Technical Preview. But my testing indicates no devices currently implement the new capabilities. This is likely a sign of a Microsoft Wireless Display Adapter update on the horizon.
I've written about the Wi-Fi Certified Miracast program (and related Wi-Fi Display specification) before, so I'll spare you the intro. But what you may not know is that for about two years now, I've been testing and collecting wireless display dongles, an esoteric hobby for sure. Per request, I put together a chart of my devices and associated latency observations and am sharing that today.
When I refer to latency, I'm talking about the time it takes for an image on a source device (e.g. a Surface Pro) to appear on the target device (e.g. TV).
My test parameters are as follows:
Testing involves running a simple program that displays a counter and loops through a movie trailer that covers over 90% of the screen to exercise motion compensation algorithms and observe quality and latency hits, if present. I then take a picture with a DSLR of both the source and target (in the same frame) and subtract the counts to determine latency.
| Cost | Firmware | Avg. Latency (ms) | Works with Windows | Miracast Certified | |
|---|---|---|---|---|---|
| Actiontec ScreenBeam Pro | $68.99 | 1.1.1.10 | 83 | Yes | Yes |
| MOCREO iPush | $41.80 | 3.0.0-rc1 | No | No | |
| Netgear Push2TV (PTV3000) | $48.99 | 2.4.53 | 116.3 | Yes | Yes |
| Samsung Wi-Fi AllShare Cast Hub | $68.98 | LJ02 | No | No | |
| Lenovo Wireless Display Adapter (WD100-SL) | $29.51 | Unknown | 143.5 | Yes | No |
| Belkin Miracast Video Adapter (F7D7501) | $59.99 | 2.51 | 376 | Yes, but unstable | Yes |
| Microsoft Screen Sharing for Lumia Phones (HD-10) | $69.99 | 2.0 | 149.8 | Yes | Yes |
| Amazon Fire TV Stick | $39.99 | 54.1.1.0 | 2971.13 | Yes, but unstable | Yes |
| Microsoft Wireless Display Adapter | $57.98 | 1.0 | 83.16 | Yes | Yes |
| Tronsmart T1000 Mirror2TV | $29.99 | 13414 | 176.66 | Yes | No |
| Xbox One 🔧 | $345.99 | 6.2.12815.0 | 106.8 | Yes, but unstable | No |
| HDMI cable reference | $5.09 | 32 | Yes | No |
🔧 Xbox One latency was measured with different hardware (Surface 3), so its number isn't directly comparable to the other devices. This will be fixed when all devices are re-tested with new hardware.
03/31/2015 - Amazon Fire TV Stick now works with Windows, measured latency added
05/01/2015 - Added preliminary latency for Xbox One
05/15/2015 - Modified latency for Xbox One using new Surface 3 hardware
After I picked up (and disassembled) a Microsoft Wireless Display Adapter, I started bugging Microsoft for the source code to the Linux-powered base image -- the underlying host operating system and support infrastructure. After three months of back and forth, the Source Code Compliance Team finally uploaded a buildroot that can be used to generate a base image. Unfortunately, it came with zero documentation.
Here's my general step-by-step on building the base image with Hyper-V and CentOS 7. Feedback is welcome.
If you choose to go down the Generation 2 VM route (recommended), don't forget to shut off Secure Boot.
Ensure you create a non-root administrative user during install. Mine will be rafael and will be used for all commands henceforth.
You only need the buildroot package from the Third Party Source Code Disclosures website.
A little background: Initially, Microsoft uploaded a bunch of smaller buildroot output artifacts. But after some back and forth, I was able to convince them to upload the buildroot itself, making the other files less useful.
You'll also want to get the zip copied over and unzipped in the virtual machine. The following instructions will assume you unzipped the buildroot folder in the user's home directory (~/buildroot).
Buildroot requires a few packages to operate. They can be installed easily via one command:
sudo yum install net-tools flex which sed make bison binutils gcc gcc-c++ bash patch gzip bzip2 texinfo perl tar cpio unzip rsync wget ncurses-devel cvs git mercurial subversion python bzr
If you decide to use 64-bit version of CentOS, you'll also need some 32-bit support packages. They, too, can be installed easily via one command:
sudo yum install compat-libstdc++-33.i686 libstdc++.i686 libstdc++-devel.i686
Let's first set up the required root folder then download and extract the software:
mkdir glibc-build
cd glibc-build
wget http://ftp.gnu.org/gnu/glibc/glibc-2.15.tar.gz
tar -zxvf glibc-2.15.tar.gz
Now, still in the root folder we created, issue the commands:
./glibc-2.15/configure --prefix=/usr
make
Compilation will take a few minutes. When it's done, copy the replacement ldconfig into the buildroot toolchain:
cp ./elf/ldconfig ~/buildroot/output/host/usr/bin/arm-none-linux-gnueabi-ldconfig
Before we can build the toolchain and packages, we need to write out a .config file. First, enter the buildroot folder and issue the command:
make menuconfig
In the configurator that appears, select Load an Alternate Configuration File and provide the following configuration file:
~/buildroot/microsoft.buildroot_config
(Microsoft has provided a slew of configuration files but none of them perfectly match the retail device configuration. For our purposes, however, it's good enough. You're free to select more packages for compilation.)
Exit the configurator and save your changes.
Now let's quickly fix up some missing execute permissions and we'll be ready to go:
chmod +x ~/buildroot/support/scripts/*
chmod +x ~/buildroot/support/gnuconfig/config.guess
chmod +x ~/buildroot/support/dependencies/*.sh
make
And wait.
My virtual machine was configured with a mere 1GB of RAM and 4 virtual processors clocking in at 3.4ghz. The build completed in about 10 minutes.
After the build completes, you'll have a nearly complete root filesystem for the adapter stored in the target folder.
If you have root on the device -- coming in a later post -- you can use the toolchain to compile packages and transfer them over to the device.
Know how to take this further? Or how to connect to/reflash NAND? Or have experience building jigs? Email me. I'd love to pick your brain.