Virtual Keyboard

WIP

The new ScummVM virtual keyboard hopes to provide a generalized implementation of a virtual keyboard that will be usable on all of the small backends that require it.

Introduction

The virtual keyboard requires a valid 'keyboard pack' in order to operate. The pack should consist of an XML configuration file and a collection of 24-bit bitmaps. It can come in the form of a compressed zip file or a directory.

In order to make pack creation quick and easy, the virtual keyboard display uses complete bitmap images. To go with each bitmap a HTML-style image map must be created that defines the clickable regions of the image. The primary purpose of the XML configuration file is to contain the image map definitions along with associated event definitions and additional data.

As it is impractical to display all possible keys at the same time, the virtual keyboard has the ability to switch to different modes when special keys are pressed. Also, the virtual keyboard has to work at a range of different resolutions. Therefore, the pack must provide a bitmap and configuration data for each mode at each resolution.

Features

• automatic resolution selection - with re-loading on change of resolution

• preview display - to show the keys that have been pressed while keyboard is active

• modifier keys - allow shift, control, and alt to be added to keystroke without using a separate mode

• special key events - to submit or cancel the keyboard, switch mode, etc

• transparency support - using keycolor bitmaps

• fully draggable

Information for keyboard pack creators

TODO: start by explaining all the different types of buttons needed for the bitmaps

In order to create a keyboard pack there are two main stages: creating the bitmaps, and, writing the XML configuration file.

Keyboard bitmaps

ScummVM's image manager can only decode 24-bit BMP images, so make sure you save your images in this format. You need to reserve a particular color value to be used for transparency. By default the color used for transparency is set to be #FF00FF, but if you want to use a different color this can be specified in the configuration file.

XML configuration file

The first step in creating the XML configuration file is to create the HTML imagemap data. The easiest way to do this is by using an standard image map creation tool such as GIMP's Image Map plugin (find it under Filters -> Web -> Image Map...). You will need to do this for each bitmap, ensuring that you define areas for all the clickable areas (and the display box). Also ensure that you only use rectangle or polygon map areas, currently circle areas are not supported.

After creating all the image maps you should put the exported HTML for all the bitmaps into a single XML file. Then you can start adding the rest of the configuration XML around it. The rest of this section provides descriptions of the tags and example complete file.

Tag descriptions

TODO: format table properly

Tag	Description	Required attributes	Optional attributes	Children
<keyboard>	This is the required, root element of the file format.	initial_mode: name of the mode the keyboard will show initially.	v_align/h_align: where on the screen should the keyboard appear initially (defaults to bottom/center).	mode
<mode>	This tag encapsulates a single mode of the keyboard. Within are a number of layouts, which provide the specific implementation at different resolutions.	name: the name of the mode resolutions: list of the different layout resolutions	layout event
<event>	These tags describe a particular event that will be triggered by a mouse click on a particular area. The target attribute of each image map area should be the same as an event's name.	name: name of the event type: key | modifier | switch_mode | submit | cancel | clear | delete | move_left | move_right - see VirtualKeyboard::EventType for explanation for key events code / ascii: describe a key press in ScummVM KeyState format for key and modifier events modifiers: modifier keystate as comma-separated list of shift, ctrl and/or alt. for switch_mode events mode: name of the mode that should be switched to
<layout>	These tags encapsulate an implementation of a mode at a particular resolution.	resolution: the screen resolution that this layout is designed for bitmap: filename of the 24-bit bitmap that will be used for this layout	transparent_color: color in r,g,b format that will be used for keycolor transparency (defaults to (255,0,255). display_font_color: color in r,g,b format that will be used for the text of the keyboard display (defaults to (0,0,0).	map: this describes the image map using the same format as html image maps
<map>	These tags describe the image map for a particular layout. It uses the same format as HTML image maps. The only area shapes that are supported are rectangles and polygons. The target attribute of each area should be the name of an event for this mode (see <event> tag). They will usually be generated by an external tool such as GIMP's Image Map plugin, and so will not be written by hand, but for more information on HTML image map format see W3Schools pages on the <map> and <area> tags.

Example configuration file

<syntax type="xml"> <keyboard modes="normal,caps" initial_mode="normal" v_align="bottom" h_align="center">

 <mode name="normal" resolutions="640x400,320x200">
   <layout resolution="640x400" bitmap="normal_640x400.bmp" transparent_color="255,0,255">
     <map>
       <area shape="poly" coords="65,50,67,48,94,48,96,50,96,77,94,79,67,79,65,77" target="q" />
       <area shape="poly" coords="105,50,107,48,134,48,136,50,136,77,134,79,107,79,105,77" target="w" />
       <area shape="poly" coords="146,50,148,48,174,48,176,50,176,77,174,79,148,79,146,77" target="e" />
       ...
       <area shape="poly" coords="11,89,12,88,69,88,70,89,70,116,69,117,12,117,11,116" target="caps" />
     </map>
   </layout>
   <layout resolution="320x200" bitmap="normal_320x200.bmp" transparent_color="255,0,255">
     ...
   </layout>
   <event name="a" type="key" code="97" ascii="97" modifiers="" />
   <event name="b" type="key" code="98" ascii="98" modifiers="" />
   <event name="c" type="key" code="99" ascii="99" modifiers="" />
   ...
   <event name="caps" type="switch_mode" mode="caps" />
 </mode>

 <mode name="caps" resolutions="640x400">
   <layout resolution="640x400" bitmap="caps_640x480.bmp" transparent_color="255,0,255">
     <map>
       <area shape="poly" coords="65,50,67,48,94,48,96,50,96,77,94,79,67,79,65,77" target="Q" />
       ...
     </map>
   </layout>
   <event name="A" type="key" code="97" ascii="65" modifiers="shift" />
   <event name="B" type="key" code="98" ascii="66" modifiers="shift" />
   <event name="C" type="key" code="99" ascii="67" modifiers="shift" />
   ...
 </mode>

</keyboard> </syntax>

Information for porters

By default the virtual keyboard code is disabled from a ScummVM build. In order to enable it use the --enable-vkeybd flag when running the configure script.

After you have enabled the virtual keyboard in a build the loading code must be provided with a path to find the keyboard pack(s), and the name of the keyboard pack to use.

For the path the loading code uses the value of the 'vkeybdpath' config file variable if it is defined. If it is not defined it will revert to the 'extrapath' variable. Likewise, if that is not defined it default to the current directory.

For the name of the keyboard pack, the loading code uses the 'vkeybd_pack_name' variable if defined, if not it defaults to using 'vkeybd_default' for the pack name.

Once the keyboard pack path and name are defined the loading code looks for a valid keyboard pack. First it looks for an uncompressed pack by looking for a file with the name [packname].xml in the keyboard pack directory. If it does not find this it will look for a compressed pack by looking for a file with the name [packname].zip.

To use the keyboard within your port ensure that your choice of keyboard pack has a suitable resolution (<= your overlay resolution) and that you have wired up something in your event handling code to generate Common::KEYCODE_F7.

Information for users

TODO pauses system preview area