How to use Intel Media SDK for screen capturing?
Submitted by Harsh Jain and Anna B.
Nowadays, cloud gaming is becoming more and more popular. The benefits are obvious: Instead of downloading and installing large files, you can just connect to the server from a diversity of devices.
How Video-Streaming Cloud Games Play
Let’s take a look how it works in case of video-streaming cloud games. First, a user runs a game on their device. Then an application sends a signal to the servers, it chooses the best available server in cloud and connects the user to it. Actually, the game is stored and played on the server. To allow a user to see video streams, the server application captures the image, encodes it and sends it via the Internet to the client app, which decodes the received pictures. Next, the user makes some actions like pressing buttons on a keyboard or game controllers. The final step is sending this information back to the server. This endless cycle continues while the user plays the game.
As we can see, screen capturing is one of essential parts of cloud gaming. In this case, screen capture speed is extremely important, because we want to play in real time without any freezes, interruptions and latency delays. Along with cloud gaming, the screen capture feature is also useful in remote desktop and other content capturing scenarios etc.
Optimise Cloud Game-Play with Intel Media SDK Screen Capture Plug-in
Intel has a solution – especially for cloud gaming developers to help manage quick these challenges – with an optimised Screen Capture plug-in as part of the Intel Media SDK and Intel Media Server Studio. This plug-in is a development library that enables the media acceleration capabilities of Intel platforms for Windows* desktop front-frame buffer capturing. The Intel Media SDK Screen Capture package includes a hardware-accelerated plug-in library that exposes graphics acceleration capabilities and is implemented as an Intel Media SDK Decode plug-in. The plug-in can only be loaded into/used with the Intel Media SDK Hardware Library and combines with the Intel-powered encoder, decoder and screen capturing to make your game server work faster.
The Screen Capture procedure can use the decode plug-in and other SDK components. Capturing display are available in NV12 or RGB4 formats. Moreover, the following Screen Capturing features are supported:
- Display Selection – This an opportunity to choose a display to capture for systems with enabled virtual display. The display selection feature is available on systems with virtual displays only (without physical display connected) and RGB4 output fourcc format.
- Dirty Rectangles Detection – This function creates the possibility to detect only changed regions on the captured display image.
Launch Screen Capture Plug-in
Following is an example command line to run screen capture plugin with sample_decode:
sample_decode.exe capture -scr:w [Width] -scr:h [Height] -d3d11 -p 22d62c07e672408fbb4cc20ed7a053e4 -o [output] -n [number of frames] – hw -r
(If plug-in is installed at a different directory, provide the complete path to the plug-in or copy the plug-in to the same folder directory before running sample_decode.)
On hybrid graphics (Intel Graphics + Discrete Graphics), screen capturing also supports a software implementation (replace parameter -hw with -sw in above command line). Also, please note that a software implementation is not optimised and when using that approach, expect a performance drop vs what you can achieve via hardware implementation.
To learn more about Intel Media SDK and all its features, click here.