Difference between revisions of "Virtual Keyboard"

Latest revision as of 21:11, 10 July 2011

Introduction

Note: The virtual keyboard is being actively developed.

The ScummVM virtual keyboard provides a generalized implementation of a virtual keyboard that will be usable on any backends that wish to make use of it.

Design

The 1st key design consideration people need to be aware of is that the keyboard is rendered onto our 16bit overlay. This means that the keyboard exists outside of the scope of the running engine (the engine is paused when the overlay/keyboard is shown).

By virtue of this design consideration the keyboard uses a 'queued events' model (so anything you select is queued and then injected back to the main event routine when the keyboard is closed) so it is best used for entering strings (save names etc.) rather than as part of the game input system (look towards the Keymapping_Improvements for systems for real time input).

The virtual keyboard requires a valid 'keyboard pack' in order to operate. The pack should consist of an XML configuration file and a collection of 24-bit bitmaps. It can come in the form of a compressed zip file or a directory.

In order to make pack creation quick and easy, the virtual keyboard display uses complete bitmap images. To go with each bitmap a HTML-style image map must be created that defines the clickable regions of the image. The primary purpose of the XML configuration file is to contain the image map definitions along with associated event definitions and additional data.

As it is impractical to display all possible keys at the same time, the virtual keyboard has the ability to switch to different modes when special keys are pressed. Also, the virtual keyboard has to work at a range of different resolutions. Therefore, the pack must provide a bitmap and configuration data for each mode at each resolution.

Key Features

Automatic resolution selection - with re-loading on change of resolution - Not sure this works.

Preview display - to show the keys that have been pressed while keyboard is active

Modifier keys - allow Shift, Control, and Alt to be added to keystroke without using a separate mode

Special events - to submit or cancel the keyboard, switch mode, etc

Transparency support - using keycolor bitmaps

Can be dragged around the screen

Special Events

In addition to the mapping of keys and the keyboard modes you can also wire up a number of special events to image map areas.

The current 'special events' are as follows.

submit - Submit all the events queues (as shown in the display area) and close the keyboard.

cancel - Close the keyboard and cancel all queued events.

clear - Clear all queued events.

delete - Delete the last event from the queue (think backspace).

move_left - Move left one place in the stacked event list (moves the position in the display_area).

move_right - Move right one place in the stacked event list if you’re not at the last event (moves the position in the display_area).

Information for keyboard pack creators

NOTE: It should be noted that the wise keyboard pack author uses SVG for there keyboard image(s) and just saves bitmaps at whatever resolutions are needed. It also makes it easy to add extra resolution images later if needed.

In order to create a keyboard pack these are the main stages:

Create the keyboard bitmaps.
Construct the HTML image maps (use something like GIMP).
Writing the XML configuration file and incorporate the image maps.

You can view the files that make up our default keyboard pack in our source code repository here. This should serve as a useful reference in conjunction with the information below.

Keyboard bitmaps

ScummVM's image manager can only decode 24-bit BMP images, so make sure you save your images in this format. You need to reserve a particular color value to be used for transparency. By default the color used for transparency is set to be #FF00FF, but if you want to use a different color this can be specified in the configuration file.

XML configuration file

The first step in creating the XML configuration file is to create the HTML imagemap data. The easiest way to do this is by using an standard image map creation tool such as GIMP's Image Map plugin (find it under Filters -> Web -> Image Map...). You will need to do this for each bitmap, ensuring that you define areas for all the clickable areas (and the display box). Also ensure that you only use rectangle or polygon map areas, currently circle areas are not supported.

After creating all the image maps you should put the exported HTML for all the bitmaps into a single XML file. Then you can start adding the rest of the configuration XML around it. The rest of this section provides descriptions of the tags and example complete file.

Tag Descriptions

Tag	Description	Required Attributes	Optional Attributes	Children
<keyboard>	This is the required, root element of the file format.	initial_mode: name of the mode the keyboard will show initially.	v_align/h_align: where on the screen should the keyboard appear initially (defaults to bottom/centre).	<mode>
<mode>	This tag encapsulates a single mode of the keyboard. Within are a number of layouts, which provide the specific implementation at different resolutions.	name: the name of the mode resolutions: list of the different layout resolutions		<layout> and <event>
<layout>	These tags encapsulate an implementation of a mode at a particular resolution.	resolution: the screen resolution that this layout is designed for bitmap: filename of the 24-bit bitmap that will be used for this layout (with no path)	transparent_color: color in r,g,b format that will be used for keycolor transparency (defaults to (255,0,255). display_font_color: color in r,g,b format that will be used for the text of the keyboard display (defaults to (0,0,0).	<map>: This describes the image map using the same format as HTML image maps.
<map>	These tags describe the image map for a particular layout. It uses the same format as HTML image maps. The only area shapes that are supported are rectangles and polygons. The target attribute of each area should be the name of an event for this mode (see <event> tag). They will usually be generated by an external tool such as GIMP's Image Map plugin, and so will not be written by hand, but for more information on HTML image map format see W3Schools pages on the <map> and <area> tags.			<area>: This is part of the image map using the same format as HTML image maps.
<event>	These tags describe a particular event that will be triggered by a mouse click on a particular area. The target attribute of each image map area should be the same as an event's name.	name: name of the event type: key \| modifier \| switch_mode \| submit \| cancel \| clear \| delete \| move_left \| move_right for key events code / ASCII: describe a key press in ScummVM KeyState format for key and modifier events modifiers: modifier keystate as comma-separated list of shift, ctrl and/or alt. for switch_mode events mode: name of the mode that should be switched to

Example configuration file

 <mode name="normal" resolutions="640x400,320x200">
   <layout resolution="640x400" bitmap="normal_640x400.bmp" transparent_color="255,0,255">
     <map>
       <area shape="poly" coords="65,50,67,48,94,48,96,50,96,77,94,79,67,79,65,77" target="q" />
       <area shape="poly" coords="105,50,107,48,134,48,136,50,136,77,134,79,107,79,105,77" target="w" />
       <area shape="poly" coords="146,50,148,48,174,48,176,50,176,77,174,79,148,79,146,77" target="e" />
       ...
       <area shape="poly" coords="11,89,12,88,69,88,70,89,70,116,69,117,12,117,11,116" target="caps" />
     </map>
   </layout>
   <layout resolution="320x200" bitmap="normal_320x200.bmp" transparent_color="255,0,255">
     ...
   </layout>
   <event name="a" type="key" code="97" ascii="97" modifiers="" />
   <event name="b" type="key" code="98" ascii="98" modifiers="" />
   <event name="c" type="key" code="99" ascii="99" modifiers="" />
   ...
   <event name="caps" type="switch_mode" mode="caps" />
 </mode>

 <mode name="caps" resolutions="640x400">
   <layout resolution="640x400" bitmap="caps_640x480.bmp" transparent_color="255,0,255">
     <map>
       <area shape="poly" coords="65,50,67,48,94,48,96,50,96,77,94,79,67,79,65,77" target="Q" />
       ...
     </map>
   </layout>
   <event name="A" type="key" code="97" ascii="65" modifiers="shift" />
   <event name="B" type="key" code="98" ascii="66" modifiers="shift" />
   <event name="C" type="key" code="99" ascii="67" modifiers="shift" />
   ...
 </mode>

</keyboard> </syntax>

Information for porters

By default the virtual keyboard code is disabled from a ScummVM build. In order to enable it use the --enable-vkeybd flag when running the configure script.

After you have enabled the virtual keyboard in a build the loading code must be provided with a path to find the keyboard pack(s), and the name of the keyboard pack to use.

For the path the loading code uses the value of the 'vkeybdpath' config file variable if it is defined. If it is not defined it will revert to the 'extrapath' variable. Likewise, if that is not defined it default to the current directory.

For the name of the keyboard pack, the loading code uses the 'vkeybd_pack_name' variable if defined, if not it defaults to using 'vkeybd_default' for the pack name.

Once the keyboard pack path and name are defined the loading code looks for a valid keyboard pack. First it looks for an uncompressed pack by looking for a file with the name [packname].xml in the keyboard pack directory. If it does not find this it will look for a compressed pack by looking for a file with the name [packname].zip.

To use the keyboard within your port ensure that your choice of keyboard pack has a suitable resolution (<= your overlay resolution) and that you have wired up something in your event handling code to generate Common::KEYCODE_F7.

Information for users

The keyboard will pause the underlying game engine when in use. All text/events that are going to be passed back to the game engine are shown in the preview area (display_area in the keyboard pack).

@@ Line 1: / Line 1: @@
-'''''WIP'''''
+== Introduction ==
+'''Note:''' The virtual keyboard is being actively developed.
+The ScummVM virtual keyboard provides a generalized implementation
+of a virtual keyboard that will be usable on any backends that wish to make use of it.
+== Design ==
+The 1st key design consideration people need to be aware of is that the keyboard is rendered onto our 16bit overlay. This means that the keyboard exists outside of the scope of the running engine (the engine is paused when the overlay/keyboard is shown).
+By virtue of this design consideration the keyboard uses a 'queued events' model (so anything you select is queued and then injected back to the main event routine when the keyboard is closed) so it is best used for entering strings (save names etc.) rather than as part of the game input system (look towards the [[Keymapping_Improvements]] for systems for real time input).
+The virtual keyboard requires a valid 'keyboard pack' in order to operate. The pack should consist of an XML configuration file and a collection of 24-bit bitmaps. It can come in the form of a compressed zip file or a directory.
-The new ScummVM virtual keyboard hopes to provide a generalized implementation
+In order to make pack creation quick and easy, the virtual keyboard display uses complete bitmap images. To go with each bitmap a HTML-style image map must be created that defines the clickable regions of the image. The primary purpose of the XML configuration file is to contain the image map definitions along with associated event definitions and additional data.
-of a virtual keyboard that will be usable on all of the small backends that
-require it.
-== Introduction ==
+As it is impractical to display all possible keys at the same time, the virtual keyboard has the ability to switch to different modes when special keys are pressed. Also, the virtual keyboard has to work at a range of different resolutions. Therefore, the pack must provide a bitmap and configuration data for each mode at each resolution.
+== Key Features ==
-The virtual keyboard requires a valid 'keyboard pack' in order to operate. The
+* ''Automatic resolution selection - with re-loading on change of resolution'' - Not sure this works.
-pack should consist of an XML configuration file and a collection of 24-bit
-bitmaps. It can come in the form of a compressed zip file or a directory.
-In order to make pack creation quick and easy, the virtual keyboard display
+* Preview display - to show the keys that have been pressed while keyboard is active
-uses complete bitmap images. To go with each bitmap a HTML-style image map must
-be created that defines the clickable regions of the image. The primary purpose
-of the XML configuration file is to contain the image map definitions along
-with associated event definitions and additional data.
-As it is impractical to display all possible keys at the same time, the virtual
+* Modifier keys - allow Shift, Control, and Alt to be added to keystroke without using a separate mode
-keyboard has the ability to switch to different modes when special keys are
-pressed. Also, the virtual keyboard has to work at a range of different
-resolutions. Therefore, the pack must provide a bitmap and configuration data
-for each mode at each resolution.
-== Features ==
+* Special events - to submit or cancel the keyboard, switch mode, etc
-* automatic resolution selection - with re-loading on change of resolution
+* Transparency support - using keycolor bitmaps
-* preview display - to show the keys that have been pressed while keyboard is active
+* Can be dragged around the screen
-* modifier keys - allow shift, control, and alt to be added to keystroke without using a separate mode
+== Special Events ==
-* special key events - to submit or cancel the keyboard, switch mode, etc
+In addition to the mapping of keys and the keyboard modes you can also wire up a number of special events to image map areas.
-* transparency support - using keycolor bitmaps
+The current 'special events' are as follows.
-* fully draggable
+* submit - Submit all the events queues (as shown in the display area) and close the keyboard.
+<syntax type="xml">
+<event name="name_of_image_map_rect" type="submit" />
+</syntax>
+* cancel - Close the keyboard and cancel all queued events.
+<syntax type="xml">
+<event name="name_of_image_map_rect" type="cancel" />
+</syntax>
+* clear - Clear all queued events.
+<syntax type="xml">
+<event name="name_of_image_map_rect" type="clear" />
+</syntax>
+* delete - Delete the last event from the queue (think backspace).
+<syntax type="xml">
+<event name="name_of_image_map_rect" type="delete" />
+</syntax>
+* move_left - Move left one place in the stacked event list (moves the position in the display_area).
+<syntax type="xml">
+<event name="name_of_image_map_rect" type="move_left" />
+</syntax>
+* move_right - Move right one place in the stacked event list if you’re not at the last event (moves the position in the display_area).
+<syntax type="xml">
+<event name="name_of_image_map_rect" type="move_right" />
+</syntax>
 == Information for keyboard pack creators ==
-'''''TODO: start by explaining all the different types of buttons needed for the bitmaps'''''
+'''NOTE:''' It should be noted that the ''wise'' keyboard pack author uses SVG for there keyboard image(s) and just saves bitmaps at whatever resolutions are needed. It also makes it easy to add extra resolution images later if needed.
+In order to create a keyboard pack these are the main stages:
-In order to create a keyboard pack there are two main stages: creating the bitmaps, and, writing the XML configuration file.
+* Create the keyboard bitmaps.
+* Construct the HTML image maps (use something like GIMP).
+* Writing the XML configuration file and incorporate the image maps.
+You can view the files that make up our default keyboard pack in our source code repository [https://github.com/scummvm/scummvm/tree/master/backends/vkeybd/packs/vkeybd_default here]. This should serve as a useful reference in conjunction with the information below.
 === Keyboard bitmaps ===
@@ Line 53: / Line 85: @@
 After creating all the image maps you should put the exported HTML for all the bitmaps into a single XML file. Then you can start adding the rest of the configuration XML around it. The rest of this section provides descriptions of the tags and example complete file.
-==== Tag descriptions ====
+==== Tag Descriptions ====
-'''''TODO: format table properly'''''
+{| border="1" cellspacing="0"
-{|
+! Tag
-| Tag
+! Description
-| Description
+! Required Attributes
-| Required attributes
+! Optional Attributes
-| Optional attributes
+! Children
-| Children
+|- style="background:AliceBlue; color:black"
-|-
+! <keyboard>
-| <keyboard>
 | This is the required, root element of the file format.
-| initial_mode: name of the mode the keyboard will show initially.
+| '''initial_mode''': name of the mode the keyboard will show initially.
-| v_align/h_align: where on the screen should the keyboard appear initially (defaults to bottom/center).
+| '''v_align'''/'''h_align''': where on the screen should the keyboard appear initially (defaults to bottom/centre).
-| mode
+! <mode>
-|-
+|- style="background:Azure; color:black"
-| <mode>
+! <mode>
 | This tag encapsulates a single mode of the keyboard. Within are a number of
 layouts, which provide the specific implementation at different resolutions.
-| name: the name of the mode
+| '''name''': the name of the mode
-resolutions: list of the different layout resolutions
+'''resolutions''': list of the different layout resolutions
-| layout
+|
-event
+| '''<layout>''' and '''<event>'''
+|- style="background:HoneyDew; color:black"
+! <layout>
+| These tags encapsulate an implementation of a mode at a particular resolution.
+| '''resolution''': the screen resolution that this layout is designed for
+'''bitmap''': filename of the 24-bit bitmap that will be used for this layout (with no path)
+| '''transparent_color''': color in r,g,b format that will be used for keycolor transparency (defaults to (255,0,255).
+'''display_font_color''': color in r,g,b format that will be used for the text of the keyboard display (defaults to (0,0,0).
+| '''<map>''': This describes the image map using the same format as HTML image maps.
 |-
-| <event>
+! <map>
+| These tags describe the image map for a particular layout. It uses the same
+format as HTML image maps. The only area shapes that are supported are
+rectangles and polygons. The target attribute of each area should be the name
+of an event for this mode (see <event> tag). They will usually be generated by
+an external tool such as GIMP's Image Map plugin, and so will not be written
+by hand, but for more information on HTML image map format see W3Schools pages on the [http://www.w3schools.com/TAGS/tag_map.asp <map>] and [http://www.w3schools.com/TAGS/tag_area.asp <area>] tags.
+|
+|
+| '''<area>''': This is part of the image map using the same format as HTML image maps.
+|- style="background:HoneyDew; color:black"
+! <event>
 | These tags describe a particular event that will be triggered by a mouse click
 on a particular area. The target attribute of each image map area should be the
 same as an event's name.
 | name: name of the event
-type: key | modifier | switch_mode | submit | cancel | clear | delete | move_left | move_right - see VirtualKeyboard::EventType for explanation
+type: '''key''' | '''modifier''' | '''switch_mode''' | '''submit''' | '''cancel''' | '''clear''' | '''delete''' | '''move_left''' | '''move_right'''
 for key events
-code / ascii: describe a key press in ScummVM KeyState format
+code / ASCII: describe a key press in ScummVM KeyState format
 for key and modifier events
@@ Line 92: / Line 142: @@
 for switch_mode events
 mode: name of the mode that should be switched to
-|
-|
-|-
-| <layout>
-| These tags encapsulate an implementation of a mode at a particular resolution.
-| resolution: the screen resolution that this layout is designed for
-bitmap: filename of the 24-bit bitmap that will be used for this layout
-| transparent_color: color in r,g,b format that will be used for keycolor transparency (defaults to (255,0,255).
-display_font_color: color in r,g,b format that will be used for the text of the keyboard display (defaults to (0,0,0).
-| map: this describes the image map using the same format as html image maps
-|-
-| <map>
-| These tags describe the image map for a particular layout. It uses the same
-format as HTML image maps. The only area shapes that are supported are
-rectangles and polygons. The target attribute of each area should be the name
-of an event for this mode (see <event> tag). They will usually be generated by
-an external tool such as GIMP's Image Map plugin, and so will not be written
-by hand, but for more information on HTML image map format see W3Schools pages on the [http://www.w3schools.com/TAGS/tag_map.asp <map>] and [http://www.w3schools.com/TAGS/tag_area.asp <area>] tags.
-|
 |
 |
@@ Line 158: / Line 189: @@
 By default the virtual keyboard code is disabled from a ScummVM build. In order to enable it use the '''--enable-vkeybd''' flag when running the configure script.
-After you have enabled the virtual keyboard in a build the loading code must be provided with a path to find the keyboard pack(s), and the name of the keyboard pack to use. For the path the loading code uses the value of the''' 'vkeybdpath' '''config file variable if it is defined. If it is not defined it will revert to the 'extrapath' variable. Likewise, if that is not defined it defaults to the current directory.
+After you have enabled the virtual keyboard in a build the loading code must be provided with a path to find the keyboard pack(s), and the name of the keyboard pack to use.
-For the name of the keyboard pack, the loading code uses the''' 'vkeybd_pack_name' '''variable if defined, if not it defaults to using 'vkeybd' for the pack name.
+For the path the loading code uses the value of the''' 'vkeybdpath' '''config file variable if it is defined. If it is not defined it will revert to the 'extrapath' variable. Likewise, if that is not defined it default to the current directory.
+For the name of the keyboard pack, the loading code uses the''' 'vkeybd_pack_name' '''variable if defined, if not it defaults to using''' 'vkeybd_default' '''for the pack name.
 Once the keyboard pack path and name are defined the loading code looks for a valid keyboard pack. First it looks for an uncompressed pack by looking for a file with the name ''[packname]''.xml in the keyboard pack directory. If it does not find this it will look for a compressed pack by looking for a file with the name ''[packname]''.zip.
-''Maybe the loading process should be simplified to use a single path variable that could point to a single xml or zip file (the facility for naming a pack was meant to allow switching between packs at runtime, but this is probably not required)''
+To use the keyboard within your port ensure that your choice of keyboard pack has a suitable resolution (<= your overlay resolution) and that you have wired up something in your event handling code to generate Common::KEYCODE_F7.
 == Information for users ==
+The keyboard will pause the underlying game engine when in use.
-'''''TODO'''''
+All text/events that are going to be passed back to the game engine are shown in the preview area (display_area in the keyboard pack).
-pauses system
-preview area